NuBit Blockchain Bloat Problem - Voting denied

[crypto_coiner]

We have an issue.
Nu blockchain is preventing me from voting (minting) because I have apparently reached a max_number of concurrent votes left in my client (0.5.4)

I know it because I carefully have added one after the other my vote (motion, custodial, parking rates) until the minting stops.

How do we solve that?

[crypto_coiner]

For example, If you configure your votes as per below, you will be denied minting.
Please confirm as we need to solve that issue.

{
“custodians” : [
{
“address” : “B5viA2YWcoLBvT8TWJywasQyAT9xkwhmWc”,
“amount” : 1.0
},
{
“address” : “BNiUcbLZ8UwD1xpqcS1EyKBnE5q4724oGA”,
“amount” : 1.0
},
{
“address” : “BSf6xxQ7BjXLUhhvvFheeFd1NiiV11ce1n”,
“amount” : 1.0
},
{
“address” : “BGUChgeJX3BUc1Sh5LfXS96XgUbUXZsND8”,
“amount” : 1.0
},
{
“address” : “BTgryZQ1dQNJYMjm74K3ajdRnDfsCjh3c3”,
“amount” : 499.0
},
{
“address” : “BB1eCrqde2g5vWTH7RsaBJ8NWTUF2D8ogq”,
“amount” : 1.0
},
{
“address” : “BPuTsT9FiDLSDihCHxzaieVXoksXqoffHk”,
“amount” : 1.0
},
{
“address” : “BCFQHtHjcisGVHY7sY2jERBHTYqZLhSxmM”,
“amount” : 1.0
},
{
“address” : “B6tBUhTuetuZezgoJCwwqkJNXNQXPCiand”,
“amount” : 27840.0
},
{
“address” : “BRTkmQ7Fz6vkg48iadLD1WfzmF7vech22v”,
“amount” : 320.0
},
{
“address” : “BMKCHYVwRpoAS1iB5iiqqudYy7USctAbkd”,
“amount” : 1.0
}
],
“parkrates” : [
{
“unit” : “B”,
“rates” : [
{
“blocks” : 8192,
“rate” : 0.00077877
},
{
“blocks” : 16384,
“rate” : 0.00155753
},
{
“blocks” : 32768,
“rate” : 0.00311507
},
{
“blocks” : 65536,
“rate” : 0.00623013
},
{
“blocks” : 131072,
“rate” : 0.01246026
},
{
“blocks” : 262144,
“rate” : 0.02492053
},
{
“blocks” : 524288,
“rate” : 0.04984105
},
{
“blocks” : 1048576,
“rate” : 0.29904632
}
]
}
],
“motions” : [
“65cb60d096508a7fa9ecc2017b38bc3afeb5663d”,
“0a4c19b33eb40845268a942067416af2ccdc4c93”,
“fe41d64247aea55fd480a3f1bd471bd6cc6a2efe”,
“eaa95d4d23ecf46e42aed90ba95448d50cb9c5e8”,
“4c1fa7b6b7b435bb215b93a771681232f1fd6237”,
“6f361693a7b248730b41d4292f89dc6f6f166bc8”,
“387e7f6ffc7d69311ac2e1d0ca7cd98d4ecbdea9”,
“ed7b9fc65dc4e9d9acad61e528420c2690f599d2”,
“876e10611809f3a90f2216fb61f107d3b7c9c469”
]
}

[creon]

That’s very interesting, I thought there would be enough space for many more motions in the coinbase string. In your case you of course can easily fix it by just removing the votes that already have passed.

How do we solve that?

In general there are two (obvious) approaches to support more hashes:
Increase the allowed block size or use smaller hashes

It would be nice to hear from someone with access to the code what the current limitations in terms of hashes roughly is, instead of relying on your exhaustive attempt to reverse engineer it.

[crypto_coiner]

In general there are two (obvious) approaches to support more hashes:Increase the allowed block size or use smaller hashes

Would there be any way for the blockchain to ignore votes that already passed?

It would be nice to hear from someone with access to the code what the current limitations in terms of hashes roughly is, instead of relying on your exhaustive attempt to reverse engineer it.

@sigmike Your comment is appreciated

[creon]

Would there be any way for the blockchain to ignore votes that already passed?

Its not about the blockchain, too large blocks get rejected very early before clients even look into them. A synchronized client perfectly knows which motion has passed and could internally avoid writing passed motions into the voting even though the user might have specified them in the GUI.

Furthermore, when the client finds a block, and adds the coinbase string and the staking transaction, it should immediately recognize that this block is too large (the block actually should never get broadcasted because the client should know that nobody will accept it). This at least should trigger a visible error to the user.

[crypto_coiner]

Furthermore, when the client finds a block, and adds the coinbase string and the staking transaction

What’s the difference between the block, the coinbase string and the staking transaction?

[creon]

OK I wanted to point you to code, but github doesn’t let us search in forked repositories right now

Anyway, the staking transaction is a transaction to yourself with a certain UTXO containing at least 10k NSR and a negative transaction fee of -40 NSR (since you will get the block, this will give you a block reward of 40 NSR). The more coin are contained in the UTXO, the more likely they meet the target in the kernel function [EDIT: I think coin age is not in the equation here in Nu, but I’m not 100% sure].

A transaction itself is implemented using a (not very powerful) scripting language. Internally this is just a stack of bytes (8 bit units) and describes a set of commands that have to be executed by the client.

However, in the coinbase transaction, the script does not need to obey any rules and you can dump anything into it you want. In Bitcoin large mining pools usually add a identifier here. Nu uses it to store a byte string that represents a voting.

In the end each element in the script buffer increases the block size by 1 byte ( + overhead).

EDIT: If you open the file “blk0001.dat”, which you will find in the folder of every bitcoin clone, with a hex editor you can see all the coinbase strings that were used in the corresponding blockchain. On linux you can just run the tool strings on this file.

[cryptog]

Very nice work, @crypto_coiner . It must have been hard work.

I am very surprised.

First of all, you got duplicate minting error (while this has nothing to do with duplicating).
Second, you put only 8 motions, 8 rates and 11 custodian votes, it seems, to get the error, which is in fact an overflow of memory space taken in the blockchain.

If Nu is successful, one can argue that we could have say simultaneously 20 motions, 20 custodian votes and 10 rates going on at the same time. Or is it exaggerated?

At least the max information inputtable at the same time should be in the documentation, imho.

[sigmike]

Nu blockchain is preventing me from voting (minting) because I have apparently reached a max_number of concurrent votes left in my client (0.5.4)

There’s a limit of 1 kB on the CoinStake transaction (the transaction that generates the PoS reward and contains the vote and some other things).

But according to the logs you showed me that’s not what’s happening. If you produced too large CoinStakes you should have lines like this one in the log:

CreateCoinStake : exceeded coinstake size limit

But maybe that’s what is happening and there’s something that prevent this message from appearing.

Thank you for identifying something reproducible. I’ll be able to investigate that now.

[creon]

There’s a limit of 1 kB on the CoinStake transaction

I just checked the whitepaper and it actually contains a lot of information about that I didn’t recall first. So a motion is just the RIPEMD-160 hash (20 bytes), a custodial grant is the address + a uint64 (28 bytes) and park rates are an uint8 for the block height and a uint32 for the interest rate (5 byte). For @crypto_coiner’s voting posted in the OP we get

9 x 20 + 11 x 28 + 8 x 5 = 528 byte

Which is significantly smaller than 1024 bytes (but its larger than the allowed size if it for some reason gets written twice)

[sigmike]

The CoinStake transaction also contains at least 1 input, 1 output, and the result of the park rate votes.

[creon]

But thats a constant, right? I mean at least if you don’t add other transactions.

The staking transaction has only one input and one output. Should be below 100 byte, right? And with currently three park rates set, I assume the result of the park rate votes is around 15 bytes?

[crypto_coiner]

txs for your elaboration

[crypto_coiner]

Thank you for identifying something reproducible. I’ll be able to investigate that now

Looking forward to the result/report of your investigation.

For now, I think I will subscribe to a feed to have a clean vote list that “works”

[crypto_coiner]

@sigmike Hello! Is this issue (blockchain bloat) solved? I would like to feel comfortable voting with/for as many motions/custodials/rates as needed.

[sigmike]

For now I’m focused on open sourcing and its implications. I’ll investigate this issue just after.

[crypto_coiner]

Looking forward

[sigmike]

I did some investigations and found a limitation I was not aware of: push commands in the script system have a size limit of 520 bytes. So votes are actually limited to 520 bytes, and the vote you described above has 591 bytes.

Expanding this limit is a protocol change so it can’t be fixed until the next protocol update.

That doesn’t explain why you don’t have any error in your log though. In my experiments I got some errors. And I don’t see how this could be related to the duplicate stake errors you get.

[crypto_coiner]

That doesn’t explain why you don’t have any error in your log

Which log?

I don’t see how this could be related to the duplicate stake errors

In my case, these errors correlated and appeared at the same time as the bloat issue.
However, in other cases, I believe ppl got some duplicate stake errors without the bloat ie being prevented from minting

So votes are actually limited to 520 bytes

That is good to know, eventually.
Hope the open sourcing of the code will enable ppl to scrutinize this kind of limitations and bugs much faster and more toroughly.

[sigmike]

Which log?

The one you sent me in a PM.