In a recent post on bitcointalk, I stated that focusing solely on the forking risk created by alternative implementations is a red herring because, while there is certainly risk involved, it is merely a symptom of a more fundamental issue that all implementations, including Bitcoin Core itself, suffer. This statement seems to have confused a few people, so, in this blog post, I’d like to delve a little deeper into why this is the case, identify what the real underlying issue is, and offer a potential solution.
First, let’s establish a few baseline facts that, to my knowledge, everyone agrees with:
- Every fully-validating node on the network must follow the exact same consensus rules or a fork will occur
- The consensus rules are complex and contain various non-intuitive corner cases, some of which probably still have not been identified
- Chain forks can be abused to create double spends and generally wreak havoc
- Every new version of Bitcoin Core carries some level of forking risk (if there is any doubt about this, see the March 2013 fork that already happened)
- Alternative implementations carry some level of forking risk
The Red Herring and the Real Underlying Issue
There is something suspiciously similar about those last two baseline facts. Whenever this occurs, there is a very high probability that there is an underlying cause and the observed behavior is actually a red herring. In this case, the red herring is only focusing on forking risk introduced by alternative implementations. It is very much true that alternative implementations carry some forking risk, but it is equally true that every new version of Bitcoin Core carries some forking risk too. You can certainly debate which one has more or less forking risk, but the underlying fact that both carry forking risk is indisputable. That fact leads to the real underlying issue:
There is currently no way to guarantee that any two versions of Bitcoin software, whether they are two different versions of Bitcoin Core, two different versions of alternative implementations, a version of Bitcoin Core versus a version of an alternative implementation, or even two copies of the same version of Bitcoin Core built with different compiler versions are in exact consensus agreement. Doing so is incredibly difficult and borders on impossible. The issue is implementation independent.
The Bitcoin Core Solution?
First, I want to make it clear that my intention here is not an attack on Bitcoin Core or its developers, who I think are talented individuals doing a fine job overall. My goal here is to point out an issue that is largely being ignored with the desire of having discussions/working towards providing real solutions for it.
The current solution as proposed by Bitcoin Core is, when you get right down to it, “let’s just be really careful with code changes and hope for the best”. There is no real disaster recovery or prevention plan. Yes, there is ongoing work to split the consensus critical bits out into a library and I think this is a good idea and am glad to see it happening. However, it still does nothing to address the real fundamental issue stated above nor does it provide disaster recovery or prevention.
Anyone who has been involved with software development for even a modicum of time knows that mistakes do and will happen regardless of how talented the people working on it are. Believing there will never again be an inadvertent mistake in Bitcoin Core, or any of the dependent technology, which causes a chain fork is not a realistic expectation and certainly not a solution.
In my opinion, the current state just isn’t good enough for a financial system that we all hope will one day be a major force in the global economy. It is akin to running a data center with critical customer data without disaster recovery mechanisms, such as backups, and hoping for the best.
Now that we’ve covered why the currently proposed methodology is problematic, what can we do? There are various approaches that can be taken, but, in order to illustrate that it is possible to solve the fundamental issue, I’d like to propose just one solution which possesses this property along with a few other benefits.
Consider for a moment if miners consulted multiple community-blessed implementations about whether they consider the block valid (with the exception of the proof of work check). This would allow a miner to construct a block and obtain consensus from multiple implementations before they ever even start solving the block. Keep in mind here when I say multiple implementations, I’m speaking in very generic terms. For example, that includes multiple versions of Bitcoin Core as well as alternative implementations like btcd, Obelisk, etc.
With an approach such as this, a miner could detect that, say Bitcoin Core 0.9.5 and btcd 0.9.0 consider the block valid, but Bitcoin Core 0.10.0 does not, or Bitcoin Core 0.9.5 and Bitcoin Core 0.10.0 consider the block valid, but btcd 0.9.0 does not. In any case, there is an extremely high probability that the 2 of the 3 which agree are right while the other one has a bug. As the number of community-blessed implementations you check against increases, the probability that a debilitating chain fork can occur decreases. Unlike the current scheme, this would allow the Bitcoin network as a whole to continue functioning properly even if a mistake were made in a new version of Bitcoin Core, and as a natural extension, alternative implementations.
The only people that would be affected then are the ones running the specific version/implementation which does not agree. Compare this to the current approach where it could potentially bring down the entire network. Upon updating/reverting their version, they would sync back up to the main chain, which the rest of the implementations agreed upon, and continue business as normal. This approach provides real disaster prevention for the network and disaster recovery for the affected nodes.
The good news is this capability already exists in the form of BIP0023 block proposals. The icing on the cake is that both btcd and Bitcoin Core already support them!
We, as a community, would have to do some work to make it easier for miners to plug in to the capability and educate them why it is in their best interest to do so, but these do not seem to be insurmountable obstacles.
Wait, Doesn’t That Take More Resources?
Yes. Disaster recovery and, more importantly, prevention, invariably require more resources. Keeping with the data backup analogy, more resources are required in the form of disk space, processor cycles, etc to backup data. However, anyone mining should understand why they’d not want to be the odd man out.
Additionally, there are a myriad of techniques to reduce the burden such as the use of distributed super nodes of various blessed implementations which allow miners to check against without having to incur a lot of extra resource hits themselves.
We have been accused on various occasions of not understanding the issues in play, but I hope this blog post shows that is not the case. We just simply disagree that the current Bitcoin Core approach is healthy for the longevity of Bitcoin. I would hope those who think the current approach Bitcoin Core is adopting is a real solution would seriously consider the content of this post with an open mind. The first step to recovery is admitting there is a problem.
What should be clear is that if you gracefully handle Bitcoin Core forking against itself, you also gracefully handle forking of alternative implementations. That very fact exposes why solely focusing on forking risk created by alternative implementations is a red herring which is a mere distraction from the real issue of the ecosystem not having any infrastructure to gracefully prevent and/or recover from mistakes in Bitcoin Core, regardless of the existence of alternative implementations.
We’re aware there are other potential issues with the alternative solution proposed which haven’t been covered here, but this post is already too long and the goal of it is not to 100% nail down the perfect solution, rather it is to elucidate why the current approach is ill-fated, demonstrate it’s not an unsolvable problem, and hopefully start serious discussion on approaches that address the real fundamental issue that exists.