Position Paper : BFT : the Time is Now

Data centers strive to provide reliable access to the data and services that they host. This reliable access requires the hosted data and services hosted by the data center to be both consistent and available. Byzantine fault tolerance (BFT) replication offers the promise of services that are consistent and available despite arbitrary failures by a bounded number of servers and an unbounded number of clients. The thesis of this position paper is simple: BFT is on the verge of becoming a practical reality—but clearing the last hurdles will require to rethink, once again, how BFT systems must be designed and implemented. Three fundamental trends support our thesis that widespread adoption of Byzantine fault tolerance is at hand. First, falling hardware costs and the increased value and importance of services are making significant non-BFT replication a standard commercial practice [5, 12, 13]. Although fault tolerance has long been an after-thought for non-critical applications, it is becoming increasingly worthwhile to use hardware generously to defend against component failures and geographic catastrophes [20]. For example, the Google file system (GFS) relies on three-way replication as a way to protect data from crash failures [13]. Second, in systems where both reliability and availability are important properties, crash tolerance is not enough. Byzantine faults are a frequent occurrence in the wild, manifesting themselves as disks that do not operate in a fail-stop manner [4], file systems that implement inadequate actions to recover from disk faults [22], file systems with bugs in their crash recovery code [25, 26], human errors [14, 21], or processors that exhibit transient Byzantine behavior because of soft errors [19, 23] and so on. Simple hardware failures can have dramatic consequences. For example, a single malfunctioning NIC at Los Angeles International Airport stalled immigration for more than 12 hours [8, 10]. Third, ten years of research in practical Byzantine fault tolerance [7, 6, 1, 11, 18, 17, 15, 16, 24, 27] have reduced significantly the costs traditionally associated with BFT. Throughput and latency of BFT replicated services are now competitive with those offered by their unreplicated counterparts [17] and, by separating agreement from execution [27], the effective overhead for BFT replication has been brought in line with the 3-way replication used by existing commercial systems such as GFS. In summary, we believe that we are on the verge of reaching the inflection point where it becomes advantageous to trade increasingly inexpensive hardware for the piece of mind provided by BFT replication. However, we also believe that, unless the community changes fundamentally its approach towards building high performance BFT systems, this opportunity may be lost. From PBFT [6, 7] to Zyzzyva [17], the design and implementation of high performance BFT systems has adopted the common-sense engineering principle of optimize for the common case, carefully tuning their systems to be extremely efficient during fault-free execution. We contend that this otherwise sound approach should be considered harmful when designing BFT system, as it encourages the proliferation of corner cases that result in dramatic performance degradations in the presence of faults. In other words, the design choices that have informed today’s BFT systems may have made them too fragile for actual deployment: current BFT systems can safely survive Byzantine faults, but can hardly be said to tolerate them, as Byzantine faults can render these systems virtually unavailable. We conclude by briefly reporting on Aardvark, our recent effort that demonstrates that true Byzantine fault tolerance can be achieved without sacrificing high performance. For complete details of the protocol and system design see [9]. Section 2 reviews the advances that have reduced the overheads of BFT replication. Section 3 exposes the dangers of over aggressive optimizations in existing BFT systems. Section 4 makes the case for a new approach in designing BFT systems which we demonstrate