Merge pull request #516 from dachary/master

erasure code : plugin, interface and glossary documentation updates

Reviewed-by: Samuel Just <sam.just@inktank.com>
erasure code abstract interface

The erasure coded pool relies on this abstract interface to encode and
decode the chunks stored in the OSD. It has been designed to be
generic enough to accomodate the libraries and algorithms that are
most likely to be used. It does not claim to be universal.

http://tracker.ceph.com/issues/5878 refs #5878

Signed-off-by: Loic Dachary <loic@dachary.org>
default erasure code implementation : M=2 K=1

This is the default erasure code implementation. Although it is fully
functional and could be used on actual data, it is mainly provided for
testing purposes. It splits data in two, computes an XOR parity and
can sustain the loss of one chunk.

http://tracker.ceph.com/issues/5878 refs #5878

Signed-off-by: Loic Dachary <loic@dachary.org>
erasure code plugin system

An erasure coded pool relies on encoding/decoding functions
implemented by a plugin and loaded with something like:

    parameters["directory"] = "/usr/lib/ceph/erasure-code"
    ErasureCodeInterfaceRef erasure_code;
    ErasureCodePlugin::factory(&erasure_code, "example", parameters)

It will then use it thru the ErasureCodeInterface methods:

    virtual int encode(const set<int> &want_to_encode,
		       const bufferlist &in,
		       map<int, bufferlist> *encoded) = 0;

    virtual int decode(const set<int> &want_to_read,
		       const map<int, bufferlist> &chunks,
		       map<int, bufferlist> *decoded) = 0;

For a given plugin, there will be as many ErasureCodeInterface
instances as there are erasure pools, each of them presumably using
a different set of parameters (i.e. M=10, K=5 or M=20, K=5 ).

A ErasureCodePluginRegistry singleton holds all erasure plugin objects
and dlopen(2) handles for the lifetime of the OSD and is cleaned up by
the destructor.

An erasure code plugin must provide the following entry point:

  void __erasure_code_init(char *plugin_name)

which is responsible for registering an object derived from
ErasureCodePlugin in the registry singleton :

  registry.plugins[plugin_name] = new ErasureCodePluginExample();

The ErasureCodePlugin derived object must provide a factory method
from which the concrete implementation of the ErasureCodeInterface
object can be generated:

  virtual int factory(ErasureCodeInterfaceRef *erasure_code,
                      const map<std::string,std::string> &parameters) {
    *erasure_code = ErasureCodeInterfaceRef(new ErasureCodeExample(parameters));
    return 0;
  }

The plugin loader uses the same assumptions as ClassHandler.{cc,h} and
will eventually have to be modified in the same way for better
portability.

A plugin is loaded the first time it is requested so that an OSD which
is not used for erasure coded pool does not load any.

An example plugin based on ErasureCodeExample is provided and used in
the associated unit test.

http://tracker.ceph.com/issues/5878 refs #5878

Signed-off-by: Loic Dachary <loic@dachary.org>