[URGENT] [IMPORTANT] ERROR: ProcessBlock() : duplicate proof-of-stake

crypto_coiner 2015-02-27 05:11:35 UTC #1

Hello, this is crypto_coiner.
As you can see from my post here,
It’s been 6 days now since I have not been minting a single block with the latest client 0.5.4
@CoinGame and/or @creon have suggested to re-download the blockchain to solve 2 issues that happend to be co-incidental:

the loss of minting ability (my blocks are not accepted, my stake is currently null)
ERROR: ProcessBlock() : duplicate proof-of-stake in the log file

I re-downloaded the whole blockchain 2 days ago.
Until yesterday, there were no duplicate proof-of-stake errors showing up in my log file, but checking today, these errors are now appearing in large quantity.

I believe this is a serious issue -
Am I using the right build?

JordanLee 2015-02-27 05:34:44 UTC #2

This is our fix for the nothing at stake issue in action. It suggests you are minting on two machines (or VMs) with the same stake. Is that possible?

crypto_coiner 2015-02-27 05:41:01 UTC #3

I have been minting on 1 solve machine up to this day.
Never ever have (or attempted to) minted on 2 or more machines (or VMs) with the same stake.

sigmike 2015-02-27 07:17:15 UTC #5

This error means someone is minting multiple times with the same wallet, not necessarily you. All the wallets should have these errors in their logs because such duplicate blocks are propagated.

We can get the NSR addresses involved to try to identify this person.

crypto_coiner 2015-02-27 07:25:34 UTC #6

This prevents me already from staking, which means minting hence voting, which is critical.

sigmike 2015-02-27 07:29:07 UTC #7

It doesn’t, unless the duplicate block is yours. Do you have any “new block found” in your log?

crypto_coiner 2015-02-27 07:30:13 UTC #8

As an experiment, I tried my wallet on a friend PC client (updated to the latest version, but without the blockchain DL)
That gives that screenshot:

As a comparion, my wallet on my PC client gives that screenshot:

There’s a dicrepancy in the transaction history and the number of transactions.

cryptog 2015-02-27 07:32:06 UTC #9

I have the same type errors showing up recently in log file but with i can still mint right at this moment.

crypto_coiner 2015-02-27 07:33:39 UTC #10

Absolutely not - I dont have any new block found in my log

sigmike 2015-02-27 07:42:05 UTC #11

Then the duplicate problem is certainly not related.

Your stake seems to be unable to find a block. Is it conceivable you provide (privately if you want) an NSR address that should be able to mint so that I can try to figure out why it’s not minting?

crypto_coiner 2015-02-27 07:53:15 UTC #12

Certainly - I ve just sent you a private message

JordanLee 2015-02-27 07:54:02 UTC #13

That is correct, but the symptom the double minter would experience is every block they mint would become stale. Attributing it to a specific minting address is a bit complicated. The log records the block hash of duplicate blocks, but because the blocks are rejected they do not appear on the blockchain. There is a getblock RPC that will return the address if the block exists on the blockchain. A duplicate block will only exist on the blockchain for the few seconds or so between when the first duplicate (not yet known to be a duplicate) and the second duplicate is received by the node. You would have to automate a call to getblock using the result of getblockhash, which in turn uses the blocks result from getinfo RPC every couple seconds to capture the minting address. Can anyone think of an easier way?

Anyway, @crypto_coiner you have mentioned using three machines already (serially, not simultaneously), I believe. Are you sure you aren’t forgetting a VPS or RaspPi that you are using simultaneously? It’s OK if you are. If so, you successfully tested our remedy for the nothing at stake issue so many bitcoiners think is such a problem. Thanks

One way to verify this is to shut your Nu client down long enough that you would normally mint blocks. When you open the client later, if you find valid minted blocks in your wallet that appear as you sync up, then you know you were attempting double minting.

crypto_coiner 2015-02-27 08:04:49 UTC #14

I have been always minting on 1 sole machine from day 1

creon 2015-02-27 10:22:30 UTC #15

Seriously? Its some kind of holy grail of PoS coins and the only approaches I know of to tackle it are tendermint, DPOS or slasher (Vitalik Buterin’s consensus mechanism). All of these solutions also have their drawbacks. What is Nu’s solution?

I also see the problem of people staking on multiple forks as a minor problem in Nu, because the block reward is not the only incentive to find blocks. However, reusing old stake from empty private keys bought from previous shareholders can lead to an attack on the voting.

@crypto_coiner I assume the block explorer agrees with your friend’s PC? Its really strange and looks like someone if feeding you with an invalid chain. Maybe explicitely addnode some of the v0.5.4 IP’s here and see if something changes.

crypto_coiner 2015-02-27 10:40:50 UTC #16

Half of these nodes still have not upgraded to the latest version… which node would I have to select?

creon 2015-02-27 10:44:47 UTC #17

The fifth column shows the client version. Here, just add this to your nu.conf:

addnode=198.52.199.75
addnode=218.1.18.78
addnode=212.114.48.31
addnode=212.129.19.120
addnode=176.9.113.75
addnode=84.76.78.204
addnode=167.114.116.72
addnode=79.165.43.191
addnode=52.10.23.117
addnode=97.123.141.92
addnode=108.26.50.128

If this works, then this is still not a very nice solution, but a hint.

JordanLee 2015-02-27 16:05:47 UTC #18

If a client receives more than one block from the same stake on the highest block, all these are rejected as invalid blocks. The mechanism moves the top block off the blockchain when another block is received that is minted by the same stake within a small time interval. To prevent network forking, it only applies the top block.

The result is that if you attempt to exploit the nothing at stake issue by minting with the same stake on multiple machines, nearly all or all blocks you find will become stale, having been rejected by the network. The only way you would get a block accepted is if your duplicate is detected as such after it is no longer the top block in the chain. This would be rare and could only happen when network latency exceeds the time needed to find the next block.

Yes, we implemented this very effective mechanism to penalize staking on multiple forks because there is more incentive to do it on our network than most, due to voting.

We prevent this by adding a checkpoint to the source code with every release, similar to what is done in Bitcoin. This is different than the centrally issued checkpoint that can be issued in real time in Peercoin because it doesn’t allow one person to control which fork is preferred in real time. I plan to make it possible for data feeds to broadcast checkpoints and clients that choose to trust a particular data feed will apply them in the future. So eventually we will have a decentralized and optional capability to accept checkpoints.

creon 2015-02-27 17:24:36 UTC #19

I don’t understand. So you say if someone has a single UTXO and for some reason this UTXO is a valid stake to reach the target on two competing forks, then the blocks will get rejected if this person broadcasts both of them?

How likely is that? I mean, the kernel method will create two completely different values that can be assumed to be stochastically independent, and therefore the probabilities of finding a block on both forks is the product of the probabilities of finding a block on each fork. So its pretty safe to stake on multiple forks even with the same UTXO and if there should be the case where the target is reached on both forks, then I can decide based on the tx fees and only broadcast this block.

Furthermore if I have C coins and have separated them in T=C/10,000 UTXO’s, then I can still start staking on T different forks in each block by using different permutations of the UTXO’s, right? So after N blocks there can be T^N different forks where I can safely stake on without even considering your rule.

I don’t see how NaS is prevented by doing this.

Decentralized checkpointing of course would solve everything, including all aspects of the NaS problem. But we have to be really careful here. Consensus of the blockchain is established by minting. Period. If a data feed is able to cut a fork by broadcasting a specific checkpoint, then data feeds have a greater power regarding consensus than the minting clients.

Large data feed providers are points of failures, and this is not different with voting, but harder to manipulate than a 1 minute block time blockchain with a recommended number of 10 confirmations. Just imagine what happens if a large data feed provider gets hacked and broadcasts a malicious checkpoint. This very well may lead to a temporary network split and if the hacker is able to hold this up for 10 minutes then a lot of damage can be done.

sigmike 2015-02-28 11:27:23 UTC #20

I ran some tests with the address you sent me and during the last 7 days it was able to find many blocks with 0.5.4. The last one being at 2015-02-28 10:59:59 UTC, about half an hour ago. Were you running the client at that time and had connections to other nodes (minting is disabled when there’s no node to propagate blocks to)?

crypto_coiner 2015-02-28 12:06:32 UTC #21

Thanks @sigmike for taking the time to test and investigate this issue -
Yes, I have left the client running (and unlocked for minting) since yesterday.