Trust-less liquidity pool

[benjyz]

I believe this is a known issue, i.e. #253, see https://bitbucket.org/JordanLeePeershares/nubottrading/issue/253/poloniex-nonce-bug

I think it will be best if we treat all bugs in the Bitbucket issue tracker, so that for example longer log messages can be archived, issues can be referenced etc. I haven’t been able to re-produce this yet. The more people systematically test the code the better. There are some improvements in how things are tested, which are not released/summarized yet.

[Ben]

Awesome job!

I’m planning on diving into a bit of testing tomorrow after I fix a few issues with the new build of our internal testing exchange (testnet only). I can give you access to the tool if you’d like, @creon (and anyone else who is testing the liquidity pool concept).

[creon]

I can give you access to the tool if you’d like

Oh sure that would be great. I just implemented a wrapper for CCEDK and probably write one for Allcoin and Bitcoin.co.id soon.

Having a testnet exchange would be very helpful when testing the payout system.

[willy]

I would be happy to assist here as well.

[cryptog]

You don’t have to. You get payed for valid signed orders you send to the pool operator (similar to shares in mining). You can run the bot for 5 minutes, 3 hours or 24 hours if you like.

So you are being paid by orders sent by your NuBot implementation every second or so?
The counterpart of that in Bitcoin’s PoW would be hash per second.

But in Nu’s PoS world, to simplify things, it is orders per second and the pool is decentralized.

I feel it is extremely promising.

[creon]

A little update:

• The pool now supports the following exchanges: poloniex, ccedk, bitcoincoid
  (note that the nubot is unable to submit sell orders on bitcoin.co.id as described here)
• The GET client API of the server shows missing requests, rejected requests and the last error for each unit
• The server has a general GET method status that shows the number of validations performed, the liquidity and the current sampling. Click here for an example.

Thanks to @willy’s help we could figure out that the timing is more of a problem than expected. Fortunately the nonce turned out to be a nice parameter to overcome this issue.

The new API methods allow the client to tweak the nonce for each exchange if too many rejects or missing requests were seen. The client outputs the current balance, efficiency and orders every thirty seconds and tries to adjust the shift if necessary.

It would be awesome if we could get some more people to test the client. For Linux users its really simple:

• Create a new exchange account on Poloniex, CCEDK or Bitcoin.co.id
• Create API keys on their website (ask if you need help, this can be confusing)
• Clone the repository:
  git clone https://github.com/creon-nu/nu-pool
• Switch to the folder
  cd nu-pool/python
• create a file called users.dat in the format
  payout-address units exchange api-key api-secret
  e.g.
  echo "BQRKF4X8gFLyHGVeym8tuff57PcJczYPVg BTC poloniex 85BQTQG1-N3D3PFKX-YUNIHIO1-WQCWTRER 1234abc" > users.dat
• run the client:
  ./client.py 104.245.36.10:2019

Then report if there is a problem and if not, and it outputs some balance and efficiency, then send a small amount of NBT or BTC to the exchange account (or PM me your address and I will send you some).

[woolly_sammoth]

Running now on CCEDK.
All seems good so far. I had one instance of an invalid nonce but nothing apart from 100% efficiency for ten minutes or so now.
Good work @creon

[creon]

Thanks so much. I see your missings are increasing. This could be due to a bug I fixed around 20 minutes ago, so you may want to update the repo once again and restart the client.

[woolly_sammoth]

I’ll pull now and restart.

I have accounts on all supported exchanges. Would it help if I run multiple instances, one for each?

[creon]

Would it help if I run multiple instances, one for each?

This would be awesome, but please not with more funds than you are using here, I really don’t want anyone to risk something. However, you don’t need to run multiple clients, just add one line for each exchange in the users.dat

[woolly_sammoth]

excellent. I was just setting up new dat files for each exchange.
I’ll add as seperate lines instead.
The small amounts on the exchanges are there for testing NuBot anyway so I’ve effectively written them off. Happy to help with this project, it really is very impressive.

[creon]

@woolly_sammoth are you T1WBB90W on bitcoin.co.id? If so, are your orders placed via nubot or manually?

[woolly_sammoth]

Yup, that’s me. The bitcoin.co.id comes were failing every time when I first ran the client and the nonce was being tweaked a lot.
I’ve left the client running overnight though and all sends are getting through on all exchanges. Seems the api issue has been fixed?

[cryptog]

Will try to give it a try over the coming days.

[creon]

Will try to give it a try over the coming days.

Thank you! I will soon start a run with (small but real) payouts, so stay tuned.

Furthermore a little update which might encourage more people to join the fun:

The client does not need to run a NuBot anymore, it places orders itself which allows to better handle the nonce problem and to distribute liquidity over different coins if required. This is currently only available for Poloniex and CCEDK, I will write the Bitcoin.co.id wrapper extension tomorrow.

With @willy’s indispensable help we could also greatly improve the search for a good nonce and clients should approach around 100% performance in less than a minute.

The robustness of the whole system exceeds my expectations and if the concept continues to evolve I am planning to implement it after the term of my current liquidity operation passed. This program is as easy to use as a miner and requires no special hardware and I believe that we could attract a very large number of crypto enthusiasts to try it.

[woolly_sammoth]

I agree that it is easy to use. The hardest part is generating the API keys from the exchange.

I’m interested in the fact that the system no longer needs NuBot to be run by the client. Does the server run a NuBot instance? If not, does tthe client mirror the NuBot functionality.

I’m looking forward to continuing to use this system.

[Joe]

I will test this out tomorrow. Not sure what to call it… “The Invisible Hand”?

[woolly_sammoth]

Just had a look at the code. ignore my previous question about the server running a Nubot instance. Your last post clearly says that the client places the orders itself.
Are there plans to add other price feeds aside from bitfinex? I know @benjyz has been implementing a price feed server for NuBot. It might be worth seeing if the price calculation can be mirrored for TIH or, eventually, if the price can be fetched from that server. Then there would be a ‘Nu Price’ - a sort of network sanctioned view of the value of ‘$1’, calculated from various different sourcesb - that can give some consitency across these services.

Other than that, it all looks really good. My only criticism would be that you said it would be 50 lines of code!

[benjyz]

To make pool implementation easier and more robust it might make sense that NuBot would offer an API of some sorts, so that third party applications can access functionality. So NuBot would a port open and accept JSON requests, such as submitOrder(exchangeName), getOpenOrders(exchangeName), etc.

[woolly_sammoth]

I just hit a crash. seems to have been caused by a 403 http error being retured from CCEDK

urllib2.HTTPError: HTTP Error 403: Forbidden

A restart fixed it so I assume it was just a temporary blip in the APi. I’m sure it’s already in your mind but it would be better to handle http errors more gracefully. Then 24 hour, unmanned opperation is easier.

Edit:
Another issue. It seems that the CCEDK nonce calculation got stuck. I’m not sure how it had been going but when I returned to the client it was attempting to adjust the nonce by +/- 10000, so I assume it had been going a while.
The way we handle this nonce offset in Nubot is to take the upper bound in the returned error
``must be from 1426070646 till `1426070666```. If you subtract the system time from that figure you get an offset figure compared to the local time. That offset should be good for all nonces going forward.