Help us test Nubits v2.1.0

[CoinGame]

[sportscliche]

I’ve had Nu 2.1 running continuously for 5 days. During that time, I’ve observed memory usage slowly but continuously increase. It’s now consuming between 700–800 Mb. Still seeing the CPU spikes.

[CoinGame]

Did you use a bootstrap file? You’ll need to do a restart after syncing.

[sportscliche]

No, I let the client synch on its own. I was interested to see how long it would take. I did indeed restart after synching. Am I minting on the wrong chain because I did not use the bootstrap file?

[CoinGame]

No you should be fine. If you restart now that it’s fully synced does the memory still rise?

[sportscliche]

I restarted several times after getting synched last week. It was using no more than 300–400 Mb. I have let it run/mint continuously since then and memory usage has roughly doubled in that time.

[sportscliche]

Memory usage began nudging the 1 Gb level, so I re-started the client. That fixed the problem for now…back to ~300 Mb. I will continue to monitor. The annoying banner that tells me “Minting is suspended due to locked Nushares wallet” has returned, even though I have unlocked it.

[CoinGame]

Does the client mint while it’s in this state, or does it show this while it’s still minting blocks? Curious if it’s a cosmetic issue or if the wallet is actually locked and prevented from minting.

[sportscliche]

It’s cosmetic. I am minting blocks while Nu is loudly warning me that I can’t.

RAM usage has steadily climbed from 300 Mb to ~600 Mb in about 12 hours.

[masterOfDisaster]

On my RaPi2 nud (RC8) has completed the blockchain download. I had to restart nud several times, because the RAM consumption was increasing during download.
Btw it looks like this:

nud getinfo
{
    "version" : "v2.1.0-RC8-dirty-beta",
    "protocolversion" : 2000000,
    "walletversion" : 1,
    "walletunit" : "S",
    "balance" : 0.0,
    "newmint" : 0.0,
    "stake" : 0.0,
    "parked" : 0.0,
    "blocks" : 731373,
    "moneysupply" : 822299048.60339999,
    "timeoffset" : -1,
    "connections" : 8,
    "proxy" : "",
    "ip" : "obfuscated",
    "difficulty" : 0.00022463,
    "testnet" : false,
    "keypoololdest" : 1452165519,
    "keypoolsize" : 101,
    "paytxfee" : 1.0,
    "errors" : ""
}

nud is running for some time as you can see at the total CPU time, but RAM usage is still very low:

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
 3565 pi        20   0  543m 330m 7504 S   0.0 33.9   1017:56 nud

[woodstockmerkle]

So the good news is I was able to upgrade to 2.1.0. But it was very complicated.

• Reverted to 2.0.3
• deleted the block database
• downloaded new blockchain from scratch, with wallets in place on 2.0.3
• performed ‘repairwallet’, with an OK result
• closed Nu 2.0.3
• ran Nu 2.1.0; same UI hanging problem
• killed Nu
• deleted wallets
• ran Nu 2.1.0 – it generated a new walletS, then crashed
• ran Nu 2.1.0 – it generated a new waleltB, then crashed
• ran Nu 2.1.0 – it loaded, and converted the block database. Woo.
• closed Nu gracefully. Put old 2.0.3 wallets back in place
• Ran Nu 2.1.0 – it came up OK, and re-scanned for missing Tx and everything was OK.

However, there is very frequent freezing / pausing of the UI for about 30 seconds at a time, with a single processor pegged 100%. This seems to be correlated with a BlockMap cleanup process and subsequent log entries
(example): BlockMap Cleanup: 149141 // BlockMap Cleanup: 148836 … sometimes over 250k.

Is there some sort of thrashing going on?

Also is the pruning based on the hardcoded value BLOCK_MAP_KEPT_HEIGHT? Can this be made tunable for a machine with a lot of memory? Or perhaps prune no more than 64 blocks per pass or something so it doesn’t block everything?

Tempted to get a dev environment up so I can see if tweaks to the pruning would have an effect, and also to see why the startup is hanging.

What would be the effect of having peers downloading. Could this instigate some form of thrashing?

[woodstockmerkle]

I’ve gotten a development environment online and was able to reproduce the “Splash Screen hanging” in the debugger.

The workaround to move aside the wallets during the conversion is logical.

The hanging startup screen appears to be a mutex deadlock condition between what seems to be the startup code that “re-sends” the transactions, and the code that starts the conversion. (I think.)

i.e.:

• Thread #1 (main) grabs cs_wallet, then cs_setpwalletRegistered
• Thread #8 (conversion thread) grabs cs_setpwalletRegistered, then grabs cs_wallet.

@sigmike @CoinGame @pennybreaker does this help?

A final note that I suppressed Nu network interaction with a nu.conf entry of “connect=” to an unavailable host.

Stack traces below:

Thread #1

0    ntdll!ZwWaitForSingleObject            0x7ff96a3e0e7a    
1    WaitForSingleObjectEx            0x7ff967751118    
2    boost::detail::basic_timed_mutex::lock    basic_timed_mutex.hpp    84    0x70381e    
3    boost::detail::basic_recursive_mutex_impl::lock    basic_recursive_mutex.hpp    56    0x70381e    
4    AnnotatedMixin::lock    sync.h    22    0x70381e    
5    boost::unique_lock<AnnotatedMixin>::lock    lock_types.hpp    346    0x70381e    
6    CMutexLock<AnnotatedMixin>::Enter    sync.h    70    0x70381e    
7    CMutexLock<AnnotatedMixin>::CMutexLock    sync.h    91    0x70381e    
8    SyncWithWallets    main.cpp    165    0x4496c5    
-->  LOCK(cs_setpwalletRegistered);
9    CTxMemPool::accept    main.cpp    1026    0x4562ce    
10    CTransaction::AcceptToMemoryPool    main.cpp    1034    0x45bb04    
11    CMerkleTx::AcceptToMemoryPool    main.cpp    1155    0x45bced    
12    CWalletTx::AcceptWalletTransaction    main.cpp    1174    0x45bced    
 -->   LOCK(mempool.cs);
13    CWallet::ReacceptWalletTransactions    wallet.cpp    961    0x4b97f2    
 -->   LOCK(cs_wallet);
14    AppInit2    init.cpp    1138    0x47bab0    
15    qMain    bitcoin.cpp    251    0x40353f    
16    WinMain            0x58dca8    
17    __tmainCRTStartup    C:/devel/build-nu/release/nu.exe    332    0x4013ed    
18    WinMainCRTStartup    C:/devel/build-nu/release/nu.exe    184    0x4014fb    


Thread #8 

0    ntdll!ZwWaitForSingleObject            0x7ff96a3e0e7a    
1    WaitForSingleObjectEx            0x7ff967751118    
2    boost::detail::basic_timed_mutex::lock    basic_timed_mutex.hpp    84    0x70381e    
3    boost::detail::basic_recursive_mutex_impl::lock    basic_recursive_mutex.hpp    56    0x70381e    
4    AnnotatedMixin::lock    sync.h    22    0x70381e    
5    boost::unique_lock<AnnotatedMixin>::lock    lock_types.hpp    346    0x70381e    
6    CMutexLock<AnnotatedMixin>::Enter    sync.h    70    0x70381e    
7    CMutexLock<AnnotatedMixin>::CMutexLock    sync.h    91    0x70381e    
8    CWallet::UpdatedTransaction    wallet.cpp    2811    0x4b658b
    -->    LOCK(cs_wallet);    
9    UpdatedTransaction    main.cpp    195    0x464391    
        -->    LOCK(cs_setpwalletRegistered);
10    CBlock::AddToBlockIndex    main.cpp    2564    0x464391    
11    CBlock::AcceptBlock    main.cpp    2885    0x464627    
12    ProcessBlock    main.cpp    3123    0x466280    
13    LoadExternalBlockFile    main.cpp    3898    0x46716b    
        -->  LOCK(cs_main);

14    ThreadImport    init.cpp    417    0x46fc29    
15    boost::_bi::list1<boost::_bi::value<std::vector>>::operator()<void ( *)(std::vector), boost::_bi::list0>    bind.hpp    253    0x768456    
16    boost::_bi::bind_t<void, void ( *)(std::vector), boost::_bi::list1<boost::_bi::value<std::vector>>>::operator()    bind_template.hpp    20    0x768456    
17    boost::detail::thread_data<boost::_bi::bind_t<void, void ( *)(std::vector), boost::_bi::list1<boost::_bi::value<std::vector>>>>::run    thread.hpp    116    0x768456    
18    boost::(anonymous namespace)::thread_start_function(void *)            0x6efa6d    
19    msvcrt!_strupr            0x7ff968780b13    
...                    

[sigmike]

Yes I found this deadlock too and I’m working on a fix. Thank you for the investigation.