erasure-code: add crush ruleset creation API

Because only the erasure code plugin knows enough to create a ruleset
that is best suited for a given set of parameters.

Signed-off-by: Loic Dachary <loic@dachary.org>
erasure-code: implement example create_ruleset

And the associated unit tests.

Signed-off-by: Loic Dachary <loic@dachary.org>
erasure-code: implement ErasureCodeJerasure::create_ruleset

It is based on CrushWrapper::add_simple_ruleset, using a "default" root
and "host" failure domain by default. They can be overridden with
erasure-code parameters ( erasure-code-ruleset-root and
erasure-code-ruleset-failure-domain respectively ).

Signed-off-by: Loic Dachary <loic@dachary.org>
erasure-code: test ErasureCodeJerasure::create_ruleset

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: helper for pool properties parsing

Add the prepare_pool_properties to convert the properties vector into a
properties map suitable for either initializing the pg_pool_t member or
an erasure code plugin.

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: no default ruleset except for replicated pools

Remove the hardcoded default ruleset for erasure coded pools and only
keep it for replicated pools. Move the logic up in the  prepare_new_pool
method so that an error code can be returned before allocating the new
pending pool in case the ruleset is not initialized.

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: pool creation helper for size

The size of the replicated pools are by default set to
osd_pool_default_size but it may vary depending on the pool type. Create
a helper to compute the pool size.

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: pool create helper for crush ruleset

The crush ruleset of the replicated pools are by default set to
osd_pool_default_crush_replicated_ruleset but it may vary depending on
the pool type. Create a helper to compute the crush ruleset.

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: erasure code plugin loader helper

The get_erasure_code helper loads the erasure code plugin found in the
erasure-code-plugin string of the properties argument. It is meant to be
used to query the plugin to determine the desired size of a pool, the
more suitable ruleset to use etc.

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: compute the size of erasure-coded pools

It is K+M ( data chunks + coding chunks ) as returned by the
get_chunk_count() method of the erasure code plugin.

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: compute the ruleset of erasure-coded pools

The default ruleset of an erasure coded pool may depend on the
parameters used to configure it. In the case of a pyramidal /
hierarchical plugin, the desired ruleset will, for instance, chose from
datacenters and then from racks and disperse local coding chunks among
them.

For this reason the default ruleset cannot be hardcoded in config_opts
as it is for replicated pools. Instead, the "crush_ruleset" property is
interpreted to be the name of an existing crush ruleset to be used.

If the corresponding ruleset is found in a pending crushmap, the
prepare_pool_crush_ruleset will return EAGAIN. The "osd pool create"
caller is modified to handle the EAGAIN error and reschedules the message.

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: osd crush rule create-erasure

Delegates the creation of the rule to the erasure code plugin associated
with the specified pool.

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: do not force proposal when no osds

If there are no OSDs, there is no need to propose to paxos. It does not
hurt on a production cluster but it matters when running tests because
it effectively alway ignores --paxos-propose-interval.

Signed-off-by: Loic Dachary <loic@dachary.org>
pybind: fix tests that do not fail as expected

A missing argument make the test fail indeed, but the intended test is
to demonstrate something else ( either character validation or excess of
arguments etc. ). The result is {} instead of None which is what should
have been expected in the first place.

Ideally there would be a more verbose way to check for syntactic errors
to make such mistakes less probable.

Signed-off-by: Loic Dachary <loic@dachary.org>
pybind: cosmetic changes to tests

* untabify
* re-indent
* 2014 to copyright notice

Signed-off-by: Loic Dachary <loic@dachary.org>
pybind: osd crush rule create-erasure tests

Signed-off-by: Loic Dachary <loic@dachary.org>
unittests: reduce paxos propose interval to increase speed

The MONs are stressed more often and there is less aggregation of the
pending requests. But the unit tests are only meant to verify that a
known code path exists and performs as expected, therefore it will not
make a difference. And if it does, it is a bug that needs fixing.

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: osd-pool-create shows logs on error

extracting the logs would otherwise require a modification of the test
file to not clobber the directory containing the logs when it exits

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: osd-pool-create test initialization

* reduce the paxos propose interval speed up tests
* load erasure code plugins from sources

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: osd-pool-create test must kill -9

If a MON is stuck, kill -15 won't be enough and the test will hang
forever.

Signed-off-by: Loic Dachary <loic@dachary.org>