Revert "osd/OSDMap: do not create erasure rule by default"

This reverts commit 08fa34d94e40f2f7230dec4dabad72107cdee27b.
vstart.sh: add erasure plugin config

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG,PGBackend: expose acting and backfill_targets

Signed-off-by: Samuel Just <sam.just@inktank.com>
PGBackend: add some additional helpers.

ECBackend's primary specific logic mostly won't treat the
primary shard specially, so it'll be handy to have primary
agnostic helpers for get_shard_info and get_shard_missing.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: allow PGBackend to set criteria for PG up-ness

ECBackend needs to be able to require that a readable
set of the most recent interval to write be available
in order to ensure that it rolls back the log far
enough.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG,ReplicatedPG: Generalize missing_loc for ECBackend

Prior to EC pools, unfound => missing.  Now, unfound (unreadable,
really) is dependent on the PGBackend requirements for reconstituting
an object.  This also means recovering an object missing on a replica
but not the primary requires tracking the missing_loc set.

Thus, rather than maintaining missing_loc only for objects missing
on the primary, the MissingLoc structure will track all missing
objects actingbackfill-wide until each object is recovered.

For simplicity, since we don't really know what objects need recovery
until activation (and since we can't do anything with that information
prior to activation anyway), we defer populating the missing_loc
information until activation.

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: differentiate missing from unreadable

Due to EC pools, missing is necessary, but not sufficient
for readability.  Thus, we instead check is_unreadable
for cases where we need to read the object and reserve
is_missing for cases where we need the object context.

wait_for_missing_object becomes waiting_for_unreadable_object
in order to avoid having another layer of waiting_for_*
maps.

Signed-off-by: Samuel Just <sam.just@inktank.com>
src/osd/: add ECBackend stubs

Signed-off-by: Samuel Just <sam.just@inktank.com>
messages,osd: add EC messages and associated types

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/ECUtil: add ec encode/decode util helpers

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/ECTransaction: add type encapsulating EC transaction

Signed-off-by: Samuel Just <sam.just@inktank.com>
ECBackend: flesh out ECBackend implementation

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: instantiate the right backend based on pool

Signed-off-by: Samuel Just <sam.just@inktank.com>