NuBot releases

NuBits Automated Trading
#1

This topic contains periodic updates on the status of development of NuBot.

[Passed] A grant to make NuBot work with Liquidity Pools
#2
#3
#4

##19-09-2014 NuBot development, update #1

This update was initially published on a private git the day before the release.

###Updates

  • Moved everything over a new repository , clean, fresh, reviewed in depth.

  • added GNU licence 2 (license.md) , and also a small boilerplate on each source file

  • added readme.md that will be the official documentation from now on

  • added changelog.md for keeping track of versions

  • prepared the first automated build task to build from sources, do some cleanup, include the sources in the packet, exclude sensitive informatio from the distributed packet

  • Release 0.1.0 : zipped and closed what I would like to consider the first public release . Its available on atlassian and the new repo under downloads.

  • Now that I am confident in all of the above, I will start working on closing and testing 0.1.1 which I would like to release as soon as possible. Not sure if I can make to release date. 0.1.1 will include ( is written in the changelog ) :

    1. New strategy for trading NBT versus an asset different than USD (e.g. BTC, PPC, CNY …) keeping the peg at ~ 1 USD : This has been hard to test since we don’t have test exchange with BTC/NBT . I ll do some other tests on btc-e user another key pair.
    2. New strategy for keeping a fraction of proceedings from sales apart. (KTm proposal. )
    3. Bter API ( This should be straightforward enough hopefully)
    4. CCEDK Api (their API is very confusing , I mostly worked it out but I am pending from answers from their bielorussians developers. )

  • the 0.1.0 will be released on github @ https://github.com/nubits/nubot together with the website.

  • I worked out a proper .gitignore file and build ANT scripts to keep sensitive data out of public domain

  • I’ve built an interactive simple HTML app to look at the logs generate by the bot. (also included in the sources) From now on will be easy to look at logs searching for errors and spotting bugs.

#5

##28-09-2014 NuBot development, update #2

This has been one of the most intense weeks so far in the development of nubot. With the public release of the 23d os september, the bot underwernt several challanges. I worked closely with the team and the custodian to make sure that everything worked as expected. The bot operated on NBT/BTC(bter) and NBT/PPC (ccedk) .

###Updates

  • the 0.1.0 has been released on github @ https://github.com/nubits/nubot together with nubits.com website
  • The 0.1.0 contains the API wrappers for CCEDK and BTER, we didn’t had to wait for 0.1.1.
  • 0.1.1 has been the main focus on this week development. In particular :
    1. New strategy for trading NBT versus an asset different than USD (e.g. BTC, PPC, CNY …) keeping the peg at ~ 1 USD. This strategy demonstrated pretty solid and can be controlled by a set of parameters in the configuration file. An email can be send everytime there is the need of a wall-shift. There are several parameters that can be configured to control its functioning. An initial set of 7 price feeds are included to track price of btc and ppc. The bot saves on the filesystem in the logs/ directory, the history of wallchanges.
    2. New strategy for keeping a fraction of proceedings from sales apart. (KTm proposal. ) This is currently under test. Everytime a sell order is (partially) executed, the bot detects a new balance. Instead of putting all the balance back on a buy wall, it freezes a % of the total sum, keeping an hystorical filesystem and sending notifications to hipchat. The % of funds to keep apart, can be controled via a parameter from the config file.
  • With multiple instances of the bot used on different pairs, we encountered an issue with the liquidity info. Each instance of the bot called RPC liquidityinfo assuming there were no other instances, therefore overwriting others liquidity info. We are currently testing a possible fix for that which accesss the current liquidity (getliquidityinfo) before submitting new info. This approach is subject to race conditions from time to time. This is way we will request to new custodian to get grants on multiple public addresses to separate the calls to liquidityinfo.
  • When properly tested, 0.1.1 will be released on github.
  • 0.1.2 will include several improvement and bug fixes : graceful wall shifts instead of resets on price change, dynamic tx fees where available, thourough balance checks, hipchat notifications on wall shifts, unlock wallet warnings, and more.
#6

##05-10-2014 NuBot development, update #3

Last week report I started with

This has been one of the most intense weeks so far in the development of nubot.

The same can be said for the current week. While the release 0.1.2 is not ready yet, mcuh work has been done

###Updates

  • all the development, versioning, and releasing activities has been moved from github to bitbucket https://bitbucket.org/JordanLeePeershares/nubottrading
  • Fixed the problem where the same custodian running multiple instance of the bot on different pair was overwriting the data. Now when sending RPC liquidity info, the bot sends an additional parameter with an unique identifier generated by combining custodian address with trading market and currency pair. Mike changed the client to aggregate info.
  • Fixed the bug that submitted liquidity info in the wrong unit for ccedk buy orders
  • Got rid of two important exit point that was causing the bot to quit in case of errors.
  • Fixed : one price feed failing was displaying an error message, while now its just a regular warning.
  • EUR and CNY price feed added ( see updated readme with the new price feeds)
  • Renamed cryptopeg to secondary peg
  • The bot can now work on a NBT/EUR or CNY pair.
  • All the percentages parameters are expressed in 0…100 scale
  • Fix: it was required to run the secondary peg bot with at least 3 backup feeds, now its just two.
  • Default price increment changed from 0.001 to 0.0003 USD
  • price-offset option disabled - to avoid custodians having the freedom of arbitrarily setting their own price, this parameter will be disabled until further decision is taken.
  • The bot now shows a warning if the nubits wallet is locked
  • Fix : all the decimal numbers in the options are now unquoted
  • Refactored the secondary peg strategy for reduce code duplicates and improve readability
  • Graceful wall shift instead of resetting all orders, it now updates walls one at the time. Need more testing
  • Fix : When checking existence of an order, do not return an error if its not found.
  • Fix : before timing out, try to clear orders again.
#7

#12-10-2014 NuBot development, update #4

Last week NuBot’s development has been slowed down to provide full-assistance to custodians and help spotting and solving existing bugs. First of all, we have all seen a high-volume and the bot freezing a few times which caused troubles in the market, especially for custodians. Everything looks ok now, since custodians upgraded the bot to a new more stable version. Here a short analysis of what happened, divided in three key points.

  • A new custodian’s bot came online on the very same exchange and pair where KTm’s bot was already operating. The lack of sync between the timing of the two bot instances led to one custodian’s sell orders being bought by the other custodians. This exposed an issue in the functioning of NuBot on non-USD pair that - at the current state - will not allow more than one custodian at the time on the same market/pair. While the solution to this issue is not straightforward, we started working on designing a possible decentralized solution today. For the moment, multiple custodians’ bots on the same non-USD pair are not allowed.
  • Even after the second custodian’s bot went offline, the bot executed several trades against it’s own orders, driving the trading volume to the moon. Despite we have seen an increase of trades during BTC price crash, we immediately started investigating the issue. It took us more than 36 hours to identify what was causing it, but we finally did find and fixed it. To make it short, the fault was in the sequence of execution and deletion of existing orders during a price shift.
  • The bot seemed to freeze a few times, not updating the price of existing orders for hours. Promptly we identified and fixed the problem, that was introduced when adjusting the bot for the previous bug and a condition was preventing the bot from adjusting the price.

###Updates

  • NuBot v0.1.2 released
  • In HTML logging, “Trades” became “Warnings/Trades” to accommodate another category of messages.
  • Temporarily disabled the executed order aggregation function, made possible by ad additional parameter : aggegate (boolean) in the options. Now the proceeds from sales and buys are updated on price shift. For NBT/USD pair, the aggregation is still untouched.
  • restored price-offset option for testing reasons.
  • Added method getLastTrades in trade interface to return trades happened in the last 24 hours. Implemented for bter and ccedk.
  • New csv output with list of active orders at each time (refreshed when submitting liquidity info)
  • The bot creates a different folder for logs of each session
  • Hipchat notifications for wall shifts

Bug fixes :

  • A control prevented wallshifts to be saved on filesystem
  • Bter API errors sometime comes with boolean, sometime String . Patched the function handling errors .
  • Graceful order shifts when price changes is ready but not yet deployed. It is scheduled for the next releases after more proper testing.
  • CCEDK bad nonce problem is now more controlled but still not perfect.

Next steps :

  • Now that the bot is stable enough, we will move to a pull-request release process, with versioning supervised by @pennybreaker.
  • Move documentation to official documentation page readme.io
  • Poloniex wrappers
  • Allow multiple custodians to act on the same non-USD pair.
  • KTm’s bot modification
  • Restore graceful wall shift
  • Prevent the bot to send empty liquidityinfo while shifting orders.
  • Tackle CCEDK nonce problem
  • Orders amount auto-rebalance from time to time.
  • First UI prototype

Thank you all for the patience demonstrated during NuBot hardest week to date.

3 Likes
Is the current volume 'real'?
#8

#21-10-2014 NuBot development, update #5

After last week fixes, we watched closely the market to monitor the bot’s behaviour. Nine days later, after 0.1.2 has been released, we didn’t experienced any issues with the bot executing self orders, or freezing because of collisions. This means that the volume you saw during last nine days is indeed real.

This week we have an important announcements :smile: : @woolly_sammoth - author of NuBippy - joined the Nu team and will support development, testing and documentation of NuBot (among other projects).

###Updates

  • @sigmike proposed a solution for allowing multiple custodians on the same (non-USD) pair at the same time without having them consuming each others’ walls. We are currently testing the proposed solution.
  • Implemented the getLastTrades() method wrapper around exchange’s API entry point to retrieve the list of executed trades.
  • Created a stand-alone program to execute the above mentioned method and save the output on a json file. This will facilitate a custodian’s reporting capabilities.
  • Released the source of three additional stand-alone programs that respectively execute a list of orders from a csv input list, clear all the active orders on one’s account and monitor the price of a secondary asset sending notifications when a threshold is reached.
  • @Ben worked on a dockerized version of NuBot and made it available on https://github.com/CryptoAssure/Custodian . It will finally act as a tool-kit for custodians running NuBot(s) : admin panel, dashboard, reporting. Feedback is more than welcome.
2 Likes
#9

#24-10-2014 NuBot development, update #6

This week we have been working on some major improvement for new bot as well as fixing bugs.

###Updates

  • Implemented and tested a solution that allows multiple custodians on the same (non-USD) pair at the same time without having them consuming each others’ walls. All decentralized with no need of bot communicating with each other.
  • Removed options that allowed custodian to control timing of executions to avoid one custodian’s instance executing faster/slower than others
  • Added support for poloniex
  • Added support for ccex
  • Added a process that compiles a README.html from the markdown and includes it into the distributed binary packet.
  • Added a control which will prevent the bot to submit liquidityinfo during a wall shift. Thanks @Chronos for spotting this.
  • Fixed a bug that was preventing the last of the 4 orders to be submitted.
  • NuLastTrade bot : implemented json outputs
  • Add log message on startup with initial PEG price used.
  • Renamed sendrpc option parameter to submitliquidity
  • Refactored the Order model
3 Likes
#10

#31-10-2014 NuBot development, update #7

This week we have been working on finalising and testing the 0.1.3 release. The Release Candidate build is currently just four issues away to be approved. We aim to make the release it before or during the weekend.

###Updates

  • Fixed a critical bug that caused the block to get stuck when the connection with one or more pricefeeds failed.
  • Fixed a critical bug that caused CCEDK nonce parameter failing every now and then, causing some ugly malfunctioning.
  • Moved logs assets outside the logging directory to improve separation of concerns.
  • Fixed some bugs that was causing troubles with locale settings different from english.
  • Fixed several small faults that emerged during QA.
  • With more users, more lines of code, more features and more developers coming, we needed some change. For this reason, at project development level, we are working hard to introduce some rigorous processes. We started by switching to git flow, a structured model to develop software which fits our needs: traceability, distributed team, QA PR, mission critical software and issue-tracker driven.
  • We are making the first steps towards fully-automated tests : systematic tests with the support of checklists and tables.

The bot have seen an overall increased stability and reliability.

2 Likes
#11

#09-11-2014 NuBot development, update #8

During the past week we released NuBot v0.1.3, and shortly after we performed a hotfix of a rounding issue that led to a minor v0.1.3a release. All custodians should have the update.

###Updates

  • Despite previous updates, the current version of NuBot is not (yet) ready for supporting multiple custodians on the same market. Some scenarios emerged during testing that will require a few updates to the bot to declare it risk-free for multiple custodians. Stay tuned.

  • Started working on release 0.1.4 which contains some major improvements.

  • Happily moved to a full gitflow as development workflow.

  • Added support for AllCoin (now testing)

  • Added support for BitSpark (now testing)

  • Set up a dedicated SMTP server for stable and reliable email notifications.

  • Implemented a mechanism that prevents orders to be placed at low price. It does that by using a moving average of 30 datapoints, after verifying reliability of a single price feed.

  • Several minor and major bug fixing and features are in progress. Please check out the public repository and its commits if you are interested in details.

  • Picked up tradingbot.nu for future use as official website. For the time being it simply redirect to the bitbucket repository.

#12

#18-11-2014 NuBot development, update #9

This week we worked on v0.1.4 which we hope we can release at the end of this week after thorough testing.

###Updates

  • Ben and Sam are completing a Docker container ready to ship for NuBot development and execution.
  • Sam implemented and tested a system that will help preventing the bot to place orders at low price. The solution is almost ready to be merged into develop branch.
  • BitSpark wrappers tested [ok]
  • AllCoin wrappers tested [ok]
  • Improved HipChat notifications
  • We now use our internal SMTP server for mail notification to custodians.
  • Cancel all orders after price failing to update

###Bug fixes

  • Solved problems while parsing (some) Peatio dates.
  • TX fee taken into account when computing prices at the first execution
  • Fixed an issue that made the bot look frozen on start under particular circumstances .
1 Like
#13

#01-12-2014 NuBot development, update #10

NuBot 0.1.4 release branch is closed and undergoing testing. See the full CHANGELOG .

We expect to release it in the next days or so, asking custodians to update shortly after the announcement.

We started working on the next version, 0.1.5, which will feature most major bug fixing, new exchanges, wrappers, and added functionalities that starts to move into JL proposal for multiple tiers of liquidity.

1 Like
#14

#13-12-2014 NuBot development, update #11

NuBot 0.1.4 has been released after one week of testing and 7 release candidates. Custodians are invited to update and review their configuration file.

New Features:

  • Implemented an additional system to prevent the risk of NuBot placing orders at low price.
  • Added a configuration parameter (“keep-proceeds” ) that lets the custodian specify an amount of proceeds from sale to be put apart. This feature is designed specifically for custodians acting in a way similar to KTm.
  • Added support for multiple custodians on the same non-USD pair.
  • Bitspark API wrappers
  • AllCoin API wrappers

Improvements:

  • On exchanges with a 0 transaction fee, force a price spread of 0.1%
  • Removed the aggregate parameter and made it automatic.
  • NTP Client makes the bots sync.
  • Reset or cancel all orders after price failing to update
  • When computing the initial USD price, take the TX fee into account
  • Computed wait_time on wall shifts
  • Changed the API error-handling system
  • Refactored PriceFeedManager
  • Reduce damount of compulsory option parameters by introducing conditions
  • The bot now uses a dedicated SMTP server to send email notifications
  • Removed debug messages from bterwrapper
  • Removed printing warnings for non-global TX fee
  • Forced order reset on firstTime execution
  • Assigned a name to threads for better runtime debugging
  • Avoid printing buy prices if SELL side only

Bug fixes:

  • Bot doesn’t stop when connection lost with Peatio.
  • Solved a problm that led to wrong pricing with a sell-side and a dual-side custodian working on the same market/pair.
  • Solved Peatio wrapper problems while parsing some date
  • Part of the balance was not placed on (buy) order

Next version 0.1.5, is already being developed and probably one week away from QA. Stay tuned.

3 Likes
#15

#09-01-2015 NuBot development, update #12

NuBot 0.1.4a has been released after 3 release candidates. It contains three hotfixes. Custodians are invited to update (no need to review the configuration file.).

Next version (0.1.5) contains some major improvements and upgrades that will be detailed here before release, a couple of weeks from now.

1 Like
#16

#13-01-2015 NuBot development, update #13

NuBot 0.1.4b has been released after 4 release candidates. It contains two hotfixes to adapt to CCEDK renewed effort to provide a more efficient API and a BTER glitch fix. Custodians are invited to update (no need to review the configuration file.).

https://bitbucket.org/JordanLeePeershares/nubottrading/downloads/nubot-v0.1.4b.zip

#17

#16-01-2015 NuBot development, update #14

NuBot 0.1.4c has been released. It contains one hotfix for the AllCoin wrapper.

https://bitbucket.org/JordanLeePeershares/nubottrading/downloads/nubot-v0.1.4c.zip

#18

#16-02-2015 NuBot development, update #15

After 9 release candidates today we released NuBot v0.1.5 :tada:

Download at https://bitbucket.org/JordanLeePeershares/nubottrading/downloads/nubot-v0.1.5.zip

:warning: All custodians are invited to upgrade to new version and follow the new setup tutorial. The old JSON configuration files (0.1.4) will not work. :warning:

Beside containing multiple updates, features, refactoring and bug fixes, the most important feature is a temporary way for more than one custodian to co-exist on the same non-USD pair.

Multi-custodian mode will sync with remote NTP and reset orders every three minutes, regardless of the peg price. This mode can be activated by changing this setting :

"multiple-custodians":true

Please set it to false if you are sure that no other custodian’s bot are active on the same pair, as it will be more efficient.

It has been tested thourougly with good results an no wall collisions. However a wall collision can happen in case one bot stop functioning and leaves the orders up (connection interrupted, machine crash … ) . This is inevitable, but for all other cases I still have 600NBT that are allocated to reimburse TX fees paid on collision.

Full changelog : please read carefully :space_invader:

v0.1.5 Changelog

New Features:

  • Multi-custodian mode will sync with remote NTP and reset orders every three minutes
  • Support for NBT-denominated pairs (swapped logic and pricing)
  • Bitcoin.co.id API wrappers
  • exco.in API wrappers
  • Added config parameter(s) to manually adjust the maximum wall sizes.
  • The bot accept a list of input json files as input and combines them internally
  • Bitfinex and Bitstamp price feeds
  • Submit liquidity info tier 1 and tier 2 using a different identifier
  • Logging the balance history

Improvements:

  • Reduced time between price-check and wall-shift to close-to-zero
  • Detecting a large price different will pause the bot for one cycle instead of shutting it down
  • BitSpark reporting tool
  • Debug message at the end of order initialization
  • On shift, cancel orders in a random order to avoid revealing the following movement direction
  • Enhance wall_shifts reporting with .json
  • Reset liquidityinfo on quit
  • Implemented getLastTrades for all supported wrappers
  • Reduced wait time during wall shifts
  • Keep 1NBT-equivalent aside to avoid “not enough balance” error
  • Polished build.xml ant script
  • In HipChat and Emails round numbers to two decimal digits
  • Send critical notifications to a different hipchat room
  • Load fiat currencies from database in CurrencyPair.java
  • GetCurrencyPairFromString, add a lookup table for names
  • Refactored the Strategy extracting re-usable methods to a different class
  • Try to cancel all orders before shutting down the bot
  • Cancel existing orders on startup
  • Sleep time parametrized
  • Display bot version number in logs
  • Renamed price-Offset parameter into price-spread (and compute it as 50% per side)
  • Wait random time before placing an order to ensure competitivity with multiple bots
  • LOG timestamp in UTC
  • Logging format improvements
  • In RPC identifier replaced custodianAddress with sessionId

Bug fixes:

  • Looping on moving average detection
  • Bitspark wrong liquidity reported
  • Reporting open orders in wrong unit
  • BTER handle small amounts of orders
  • BitSpark reporting tool ignoring timestamp
  • Other minor issues
2 Likes
NuBot development roadmap - From 0.1 to 1.0
#19

#18-02-2015 NuBot development, update #16

A new release (0.1.5b) with an hotfix is out. It contains a new keystore with updated SSL certificates, a fix in the wall-shift logic, and a fix for bitcoin.co.id renewed trade API.

Download @ https://bitbucket.org/JordanLeePeershares/nubottrading/downloads/nubot-v0.1.5b.zip

#20

#24-02-2015 NuBot development, update #17

A new release (0.1.5c) with an hotfix is out. It contains a new keystore with updated SSL certificates and a FIX that was preventing liquidityinfo to be sent on USD pairs.

Download @ https://bitbucket.org/JordanLeePeershares/nubottrading/downloads/nubot-v0.1.5c.zip