Vaultaire: a data vault for system metrics, backed onto Ceph
Channel: Linux.conf.au 2015 -- Auckland, New Zealand
Date: 03/05/2015

Andrew Cowie

Before you can do serious analytics you need to have serious data. You can't go averaging it away just to save space; you need the raw data.

Vaultaire is a distributed metrics store. It is fault tolerant, performs well, is space efficient, and — since it stores data directly in Ceph — scales horizontally as your cluster grows.

Most systems crumble when faced with the thousands of metrics per second typically being gathered in a production environment. Ceph is good at absorbing lots of writes, and is especially good at handling randomly distributed reads.

You may have experienced Ceph via it's S3-compatible HTTP gateway or via the RBD block device service. You may not know these are built atop a simple library that handles talking to the storage cluster and locating your data, librados, and that's how Vaultaire stores its time-series data.

Using librados directly opens some pretty cool possibilities. It's powerful, but low-level, and as you'd expect there are gotchas. We'll talk about building a librados-based application, lessons learned relying on Ceph in production, and the challenges of building predictive modelling tools on top of a metrics store that doesn't throw data away.