Help us test Nubits v2.1.0

Cybnate · January 12, 2016, 11:17pm

Can confirm the reduced memory requirements on Linux, but on m y PC it is about 1Gb for Nu 2.03 and around 650Mb for Nu 2.1.

Something else which doesn’t appear to be right is the synchronization. Nu2.1 is 2-3 times slower than 2.03. Earlier I noticed that syncing Nu 2.1 is very slow, but this emphasizes it further.

masterOfDisaster · January 15, 2016, 9:02am

nud got stuck at block 690,960:

nud getinfo
{
    "version" : "v2.1.0-RC8-dirty-beta",
    "protocolversion" : 2000000,
    "walletversion" : 1,
    "walletunit" : "S",
    "balance" : 0.0,
    "newmint" : 0.0,
    "stake" : 0.0,
    "parked" : 0.0,
    "blocks" : 690960,
    "moneysupply" : 833757804.61170006,
    "timeoffset" : -4,
    "connections" : 8,
    "proxy" : "",
    "ip" : "redacted",
    "difficulty" : 0.000211,
    "testnet" : false,
    "keypoololdest" : 1452165519,
    "keypoolsize" : 101,
    "paytxfee" : 1.0,
    "errors" : ""
}
tail -n 20 ~/.nu/debug.log
2016-01-15 08:26:00 accepted liquidity info from BEU9G1YCwCtaVa3mMPvX4NcqKngbA2jJ7C
2016-01-15 08:26:00 accepted liquidity info from B8DZADD7UrMkzEmTWyvn2cEoxcnog1D8MK
2016-01-15 08:26:00 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3
2016-01-15 08:26:00 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3
2016-01-15 08:26:01 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3
2016-01-15 08:26:01 received block d072c16c6205814dc239211af9352395236bbadf09862e36e981181e5cdac3c4
2016-01-15 08:26:01 ERROR: CheckProofOfStake() : INFO: check kernel failed on coinstake 55ce9710986bbe72ee4062454be5d7f6cd75fa5e5560f6b1d43fe0213655a1c1, hashProof=0000000000000000000000000000000000000000000000000000000000000000
2016-01-15 08:26:01 WARNING: ProcessBlock(): check proof-of-stake failed for block d072c16c6205814dc239211af9352395236bbadf09862e36e981181e5cdac3c4
2016-01-15 08:26:01 Flushed 9055 addresses to peers.dat  184ms
2016-01-15 08:26:08 ThreadRPCServer method=getinfo
2016-01-15 08:26:08 Misbehaving: 192.230.167.139:7890 (6 -> 7)
2016-01-15 08:26:09 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3
2016-01-15 08:26:09 accepted liquidity info from BEU9G1YCwCtaVa3mMPvX4NcqKngbA2jJ7C
2016-01-15 08:26:09 accepted liquidity info from B8DZADD7UrMkzEmTWyvn2cEoxcnog1D8MK
2016-01-15 08:26:09 accepted liquidity info from BEbkDnBh71ZaXHXkr63HXP4bMda9m9BN7B
2016-01-15 08:26:09 accepted liquidity info from BEbkDnBh71ZaXHXkr63HXP4bMda9m9BN7B
2016-01-15 08:26:10 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3
2016-01-15 08:26:10 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3
2016-01-15 08:26:10 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3
2016-01-15 08:26:10 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3

I decided to restart nud.
Is that new - “nu-shutoff” after issuing “nud stop”? Haven’t recognized that before. Anyway: nice!

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
28317 pi        20   0  829m 652m 6880 R 100.0 67.0   1898:23 nu-shutoff

After restarting nud, nud stopped. The debug.log shows:

tail -n 20 ~/.nu/debug.log
/bin/nud[0x5e234]
/bin/nud[0x5e234]
/bin/nud[0x5e234]
/bin/nud[0x5e234]
/bin/nud[0x5e234]
/bin/nud[0x5e234]
/bin/nud[0x5e234]
2016-01-15 08:42:01 ERROR: bool CBlock::ReadFromDisk(const CDiskBlockPos&)() : deserialize or I/O error
2016-01-15 08:42:01 ERROR: VerifyDB() : *** block.ReadFromDisk failed at 690915, hash=21c3ce1f5d0cb4828dce3753b2ce8fa5dfb1646a90c608582cb7310e1cd742b6
2016-01-15 08:42:01 : Corrupted block database detected.
Do you want to rebuild the block database now?
2016-01-15 08:42:01 Shutdown : In progress...
2016-01-15 08:42:01 Flush(false)
2016-01-15 08:42:01 DBFlush(false) ended               1ms
2016-01-15 08:42:01 StopNode()
2016-01-15 08:42:01 Flushed 0 addresses to peers.dat  11ms
2016-01-15 08:42:01 Committing 135 changed transactions to coin database...
2016-01-15 08:42:01 Flush(true)
2016-01-15 08:42:01 DBFlush(true) ended               0ms
2016-01-15 08:42:01 Shutdown : done

Looks like converting from BerkeleyDB to LevelDB can’t be done (due to SD card issues?) and I need to sync from scratch, right?

Are there already bootstrap files available for LevelDB?

Update:
syncing from scratch failed at block 8,776 from debug.log:

2016-01-15 10:27:41 WARNING: ProcessBlock(): check proof-of-stake failed for block 40b8290a5d783d1fcae0089e10c34378d87f35ab4604a758ef0ef1cf76b89d6c

And there’s a lot messages about misbehaving nodes in debug.log - whatever that means.

Another finding: it’s impressive how responsive nud is; right after starting it, issuing commands like nud getinfo is possible and lead to output.

I’ve started with syncing the blockchain from scratch once more.

Update2:
Syncing blockchain from scratch got stuck again at block 8,776.
I added

addnode=104.156.247.229
addnode=104.238.165.61
addnode=104.239.228.107
addnode=192.237.200.146
addnode=212.129.19.120
addnode=128.199.96.127
addnode=188.226.223.94

(from Super nodes - for nu.conf ) to nu.conf, restarted nud and downloading the blockchain could be continued.

Update3:
nud on my RaPi2 (1 GB RAM) is syncing the blockchain awfully slow (only 46,198 blocks so far with 8 connections in the last 8 hours), but the memory comsumption is (in difference to the database conversion I tried before) constantly on very low level (approx. 125 MB!):

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
29590 pi        20   0  332m 121m 8684 S   2.3 12.5 270:40.64 nud

Cybnate · January 16, 2016, 2:48am

I’m seeing the same on my 1Gb and 2Gb Linux Debian VPS’s syncing very slow no matter what, but low memory usage of Nud, although 125Mb is a record. I haven’t seen it under 350Mb, 400-450Mb on average, peaks to 600Mb Haven’t encountered being stuck though yet, running for 3 days on a row now. RaPi Nud apparently behaves differently for some reason.

sigmike · January 16, 2016, 12:02pm

Berkeley DB is used for the conversion but it’s also still used for the wallet. So it’s still required.

CoinGame · January 16, 2016, 8:55pm

Sorry I haven’t posted an update here. Just got back from traveling and may have bricked my laptop trying to move everything over to a bigger hard drive. Fun times! Hopefully will get back into the grind by tomorrow

sigmike · January 17, 2016, 3:23pm

I did some tests on the initial download. It’s expected to be slower on 2.1 because everything is not kept in memory anymore so the blocks need to be loaded from disk on demand. So I/O speed is an important factor here. But even with optimal I/O and network conditions it’s still 2 times slower. We will try to improve that, but it may be better to start with a bootstrap file.

Nagalim · January 17, 2016, 3:32pm

Does this mean downloading the blockchain from a trusted source instead of on the network initially?

masterOfDisaster · January 17, 2016, 3:37pm

Exactly that!
The blocks still need to be verified by the wallet application, but not downloaded from other Nu network peers.

Nagalim · January 17, 2016, 3:40pm

Weak subjectivity at its finest. We can compile the source code and download directly from peers, but it takes a lot of time and effort. Or we can download from trusted Nu sources both the client and the blockchain. Shouldn’t we offer this bootstrap service? In some ways it’s also like a check pointing process.

mhps · January 18, 2016, 4:30am

They are not the same. Modified source code will compile and run. Modified blockchain will cause a re-download from scratch as I understand, as long as the network you see is good.

So you don’t need to trust the source of bootstrap file if you have trusted peers.

Nagalim · January 18, 2016, 4:31am

true, so the burden of trust is even less for the boot strap than for the client software. All the more reason why we should offer a direct download, or something.

mhps · January 18, 2016, 4:39am

right. for those paranoids an official sha256 can be provided

Cybnate · January 18, 2016, 4:42am

Would be good, a full blockchain download took me almost 48h with V2.1.x. Maybe we can setup a torrent as the traffic to host these files might be high and expensive. We could even use something like Joystream later (when out of alpha), to pay users for seeding the files.

masterOfDisaster · January 18, 2016, 10:41am

+1

Until we have bootstrap files:
would you consider uploading the bare blockchain files somewhere?
Compressing might save some space.
And of course the Nu wallet needs to be exited - but why do I tell you that?

It would help big time being able to run nud 2.1.0 on my RaPi2, because of the low RAM requirements.
Downloading it on my RaPi2 will take some more days:

Sun Jan 17 17:59:29 UTC 2016
nud getinfo | grep blocks
    "blocks" : 205501,
Mon Jan 18 10:24:05 UTC 2016
nud getinfo | grep blocks
    "blocks" : 264482,

60,000 blocks in 16 hours or roughly 90,000 per day if the download speed stays the same.

willy · January 18, 2016, 10:45am

@cybnate were you able to run Nu (QT) on mint in the meantime?

Cybnate · January 18, 2016, 10:39pm

Will send you a PM

Haven’t tried yet, just have it running for 5 consecutive days. When no issues, I might give it a go over the weekend now I’ve just managed to copy the files to another instance of Nu without downloading them from scratch.

masterOfDisaster · January 19, 2016, 4:52pm

I just had a crash of the Nu wallet application (v2.1.0; Windows x64 version).

The last lines of the debug.log are:

2016-01-19 16:24:04 ProcessBlock: ORPHAN BLOCK, prev=64d1ad696a6d92868b22
2016-01-19 16:27:52 BlockMap cleanup: 13447 removed
2016-01-19 16:28:29 received block f29e447af3ce559758e52a3989ad3ab1e070099c5fdc3d851d03b060b00c65d6
2016-01-19 16:28:30 Committing 4 changed transactions to coin database...
2016-01-19 16:28:30 SetBestChain: new best=f29e447af3ce559758e52a3989ad3ab1e070099c5fdc3d851d03b060b00c65d6  height=708240  log2_trust=39.804072  moneysupply(S)=826284469.3315  moneysupply(B)=3218302.5095  tx=1444556  date=2016-01-19 15:44:48 progress=0.004960
2016-01-19 16:28:30 ProcessBlock: ACCEPTED
2016-01-19 16:28:30 BlockMap cleanup: 25592 removed
2016-01-19 16:28:30 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3
2016-01-19 16:28:30 NotifyLiquidityChanged
2016-01-19 16:28:30 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3
2016-01-19 16:28:30 NotifyLiquidityChanged
2016-01-19 16:28:30 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3
2016-01-19 16:28:30 NotifyLiquidityChanged
2016-01-19 16:28:30 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3
2016-01-19 16:28:30 NotifyLiquidityChanged
2016-01-19 16:28:30 accepted liquidity info from BEU9G1YCwCtaVa3mMPvX4NcqKngbA2jJ7C
2016-01-19 16:28:30 NotifyLiquidityChanged
2016-01-19 16:28:30 accepted liquidity info from B8DZADD7UrMkzEmTWyvn2cEoxcnog1D8MK
2016-01-19 16:28:30 NotifyLiquidityChanged
2016-01-19 16:28:30 received block 853983cdd706b77cbfdc38a1ec8b5e1d0252e7c6a05706ee095f0849a86963a0
2016-01-19 16:28:30 ProcessBlock: ORPHAN BLOCK, prev=3576dad533799658f21f
2016-01-19 16:28:36 BlockMap cleanup: 16764 removed
2016-01-19 16:28:36 accepted liquidity info from BEbkDnBh71ZaXHXkr63HXP4bMda9m9BN7B
2016-01-19 16:28:36 NotifyLiquidityChanged
2016-01-19 16:28:36 accepted liquidity info from BEbkDnBh71ZaXHXkr63HXP4bMda9m9BN7B
2016-01-19 16:28:36 NotifyLiquidityChanged
2016-01-19 16:28:36 received block 951d8555bd7fc86b8ede537c7cd59705b14dd41a3ddba14c822456b70eda439b
2016-01-19 16:28:36 ProcessBlock: ORPHAN BLOCK, prev=2fc8460256e553791689
2016-01-19 16:28:44 BlockMap cleanup: 10442 removed
2016-01-19 16:28:44 received block 2fc8460256e55379168917bce72ec6f8c89d01150643bb6226a7021d97be2f11
2016-01-19 16:28:44 ProcessBlock: ORPHAN BLOCK, prev=1fcb0eb47b89df02bb4b
2016-01-19 16:28:52 BlockMap cleanup: 25517 removed
2016-01-19 16:28:52 accepted liquidity info from BEU9G1YCwCtaVa3mMPvX4NcqKngbA2jJ7C
2016-01-19 16:28:52 NotifyLiquidityChanged
2016-01-19 16:28:52 received block 3b4a221d58cba3797e934b35c9bd4533aaed670392e00ce18c1a73164a6a3644
2016-01-19 16:28:53 ProcessBlock: ORPHAN BLOCK, prev=853983cdd706b77cbfdc
2016-01-19 16:28:59 BlockMap cleanup: 16969 removed
2016-01-19 16:28:59 accepted liquidity info from BA9tumP4zyM1g2M8ZNieD2iAnMgdVPuQe3
2016-01-19 16:28:59 NotifyLiquidityChanged
2016-01-19 16:28:59 received block 64d1ad696a6d92868b22bead81d0c9bcd0ff99f1442dc5cb4710bf81fdf72ff8
2016-01-19 16:29:00 Committing 7 changed transactions to coin database...
2016-01-19 16:29:00 SetBestChain: new best=64d1ad696a6d92868b22bead81d0c9bcd0ff99f1442dc5cb4710bf81fdf72ff8  height=708241  log2_trust=39.804073  moneysupply(S)=826284509.3315  moneysupply(B)=3218302.5095  tx=1444558  date=2016-01-19 15:47:26 progress=0.004960
2016-01-19 16:29:00 Committing 3 changed transactions to coin database...
2016-01-19 16:29:00 SetBestChain: new best=3576dad533799658f21fe2f62c31187be27fa8d39c3f7a83c38da314a7c7fde9  height=708242  log2_trust=39.804074  moneysupply(S)=826284549.3315  moneysupply(B)=3218302.5095  tx=1444560  date=2016-01-19 15:47:49 progress=0.004960
2016-01-19 16:29:01 Committing 3 changed transactions to coin database...
2016-01-19 16:29:01 SetBestChain: new best=853983cdd706b77cbfdc38a1ec8b5e1d0252e7c6a05706ee095f0849a86963a0  height=708243  log2_trust=39.804076  moneysupply(S)=826284589.3315  moneysupply(B)=3218302.5095  tx=1444562  date=2016-01-19 15:47:07 progress=0.004960
2016-01-19 16:29:03 Committing 3 changed transactions to coin database...
2016-01-19 16:29:03 SetBestChain: new best=3b4a221d58cba3797e934b35c9bd4533aaed670392e00ce18c1a73164a6a3644  height=708244  log2_trust=39.804077  moneysupply(S)=826284629.3315  moneysupply(B)=3218302.5095  tx=1444564  date=2016-01-19 15:49:22 progress=0.004960
2016-01-19 16:33:57 Flushed 8355 addresses to peers.dat  114ms
2016-01-19 16:40:40 ProcessBlock: ACCEPTED
2016-01-19 16:40:41 BlockMap cleanup: 529034 removed
2016-01-19 16:40:41 ResendWalletTransactions()
2016-01-19 16:40:54

zoro · January 21, 2016, 10:05am

I don’t see any differences in memory usage than the previous version
(win7 64)

masterOfDisaster · January 21, 2016, 12:10pm

I just fired it up on my win7 64-Bit machine (the x64 version):

332.228 k RAM equals approximately 324 MB RAM. I don’t know how much it consumed before.
The RaPi version consumes even less - unless you sync a blockchain with that one, which I currently do and the RAM consumption is increasing each minute.

Remark - the RaPi2 has 1 GB RAM, so 84.3% mean over 800 MB RAM consumption.

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
 4939 pi        20   0 1045m 821m 3056 S 100.4 84.3   1690:39 nud

That’s why I restart it from time to time (which I did just now). After a fresh start it’s like this:

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
 7843 pi        20   0  375m 163m 8052 S 195.3 16.8   3:02.40 nud

And yes, it’s really running and syncing the blockchain:

tail ~/.nu/debug.log
2016-01-21 12:08:46 received block 2b2f09b4c05b16dc8b21a899ae70662864050e06962134d92326fe328dab78e8
2016-01-21 12:08:47 SetBestChain: new best=2b2f09b4c05b16dc8b21a899ae70662864050e06962134d92326fe328dab78e8  height=394648  log2_trust=39.094131  moneysupply(S)=1004071280.849  moneysupply(B)=4672942.3474  tx=803237  date=2015-06-13 14:59:57 progress=0.002753
2016-01-21 12:08:47 ProcessBlock: ACCEPTED
2016-01-21 12:08:47 received block c1658e523fa3b89941d553a07d3e8812bc47d8789b5762fc4103b6b95b6ceb05
2016-01-21 12:08:47 SetBestChain: new best=c1658e523fa3b89941d553a07d3e8812bc47d8789b5762fc4103b6b95b6ceb05  height=394649  log2_trust=39.094135  moneysupply(S)=1004071320.849  moneysupply(B)=4672942.3474  tx=803239  date=2015-06-13 15:02:24 progress=0.002753
2016-01-21 12:08:47 ProcessBlock: ACCEPTED
2016-01-21 12:08:47 received block 047b660da7bd23faadea3781b17e200a89fed2a0d7af8ed36d9aecf598c97607
2016-01-21 12:08:47 SetBestChain: new best=047b660da7bd23faadea3781b17e200a89fed2a0d7af8ed36d9aecf598c97607  height=394650  log2_trust=39.094138  moneysupply(S)=1004071360.849  moneysupply(B)=4672942.3474  tx=803241  date=2015-06-13 15:04:47 progress=0.002753
2016-01-21 12:08:47 ProcessBlock: ACCEPTED
2016-01-21 12:08:47 received block 3a784db424045d9dffbf48617f7ba173244abfc8cacf56646fcda318fdc9eaeb

zoro · January 21, 2016, 12:47pm

just go to sleep
and let it run some days. At first ram is much less, then it rises!