We are pleased to announce that btcd, our full-node bitcoind alternative written in Go, is finally ready for public testing!
The installation instructions and source code can be found on github at:
A Brief History
Back in May, we first announced our plans to release btcd. A week later we released our first core package from btcd, btcwire, and announced our plans to continue releasing the component packages of btcd in a staggered fashion.
Over the next month, we released btcjson, btcdb, and btcscript. Then in mid-July we released btcchain at which time we announced btcd was next. At that point, btcd had most of the core bits and we figured we’d be releasing it within a few weeks. Well, as you have no doubt noticed, it is now 10 weeks later…
What Took so Long?
As typically happens with development, several things came up, all of which took a few days (or a week) here and there.
The following list is a brief overview of the various things that were tackled:
- Decided to separate chain and wallet architecture
- Added built-in support for the “official” bitcoin block acceptance regression test
- Added support for testnet
- 3 separate issues with the Go run-time garbage collector leaking memory
- Extremely poor SQLite performance
- Provided LevelDB as another backend database written in pure Go instead of requiring cgo like SQLite
- Worked with the goleveldb author to identify and squash several bugs
- Added support for proxies and tor (including proper DNS resolution via tor to avoid leaking your IP)
- Implemented complete address manager including bucketizing to increase geographic separation and reduce attack vectors
- Finished the connection manager which deals with talking to multiple peers simultaneously
- Implemented a transaction pool and relay complete with proper chain rule adherence and additional filtering for miners
So, as you can see from the list above, we were able to cram quite a bit in 10 weeks!
Separate Chain and Wallet
While implementing the wallet and starting to build the GUI, we quickly realized that there is, what we consider, a fundamental design flaw in the other implementations we have seen to date. In particular they have the wallet and chain functionality integrated in the same process.
It certainly makes the code significantly easier to work with when chain and wallet code can be intermingled without having to deal with IPC, as we were originally going to do, but that design has several issues, some of which will be covered in the next section. Rather than simply taking the easy way out to get something out quickly, we decided to go back and spend the time necessary to split the chain from the wallet.
Why Separate Chain and Wallet?
One of the major problems with wallet and chain functionality integrated in the same process is multi-user support. For example, when using bitcoind or bitcoin-qt, two users sharing the same computer must each maintain their own block chain. This results in duplicated effort and wasted disk space, as the block chain is public data and should be sharable. Due to this, btcd is designed to provide chain services for separate wallet processes. These processes then are able to request updates to the chain and submit transactions to the network without having to deal with all the complexities of chain management.
Multiple devices and thin wallets can also be supported based on this design. In future versions of btcd, wallet communication will occur over HTTPS connections using websockets. Separate wallets sharing a single chain are not limited to multiple users on just a single computer. Multiple users, each with their own devices and wallets, can authenticate and connect to a shared btcd server. Because the hardware requirements for every additional wallet are so small, old hardware which is not capable of handling the computing needs of full-node validation can be re-purposed.
There are some other benefits which will be covered in an upcoming blog post on btcwallet, the wallet implementation which works with btcd to provide wallet services.
What about the cons?
- Slightly more difficult to configure for a single user that will only ever be using a single machine (increasingly less common these days)
- Increased complexity during the testing process, as separate binaries must be tested while executing at the same time
- SCM tools like ‘git bisect’ may not be to able locate when and how a bug was introduced as easily
Architecture Overview for btcd
The following diagram depicts the internal architecture of btcd as well as how all of the bitcoin related packages we have released fit together:
btcd has built in support for the “official” bitcoin block acceptance regression test. The following screenshot shows the results of running the regression test against btcd. As you can see, the regression test tool thinks it’s talking to bitcoind, but it’s really talking to btcd. The highlighted lines show that the block acceptance behavior of all tested cases is the same between btcd, bitcoind, and bitcoinj.
We are looking for early adopters to help us test btcd. So, if you’re interested, check out the installation instructions here.
One of the outstanding issues is the ECDSA cryptography provided by the Go standard library is extremely slow. There are some significant improvements in the upcoming Go 1.2 release, but even with those improvements, it is still orders of magnitude slower than OpenSSL.
On the btcd front in the near term, we are going to continue development to complete some of the remaining unfinished items in the code, improve the documentation, and address issues as they arise. A longer term goal is to work on optimizing the cryptography.
Our next planned software release is btcwallet. It is the separate wallet process we have been developing alongside btcd for wallet functionality. More details regarding this software will be announced soon in an upcoming blog post.