A Detailed Look at Erasure Codes in OpenStack Swift
Channel: Linux.conf.au 2016 -- Geelong, Australia
Date: 06/02/2016

John Dickinson
https://linux.conf.au/schedule/30125/view_talk
OpenStack Swift has recently implemented erasure codes as a way to durably stored data. In this talk we'll start with a very brief overview of Swift and erasure codes and quickly move on to a detailed look at the implementation the erasure code feature. First we will cover the simple read/write data path. Then we'll move into some interesting challenges the community faced when implementing the feature, and how they were solved. We'll go over how Swift, an eventually-consistent (AP) system, does a multi-phase commit. We'll show where Swift uses two-dimensional time to handle failures and reach a steady-state. Swift's supports multiple external erasure code libraries, and we'll review the different ones available. There will be code. There will be performance graphs. There will be math. Come learn how we have implemented this feature in Swift and why it's such a big deal.