documentation: comment the CompatSet data members

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: isa/lrc plugin feature

There are two new plugins (isa and lrc). When upgrading a cluster, there
must be a protection against the following scenario:

  * the mon are upgraded but not the osd
  * a new pool is created using plugin isa
  * the osd fail to load the isa plugin because they have not been
    upgraded

A feature bit is added : PLUGINS_V2. The monitor will only agree to
create an erasure code profile for the isa or lrc plugin if all OSDs
supports PLUGINS_V2. Once such an erasure code profile is stored in the
OSDMap, an OSD can only boot if it supports the PLUGINS_V2 feature,
which means it is able to load the isa and lrc plugins.

The monitors will only activate the PLUGINS_V2 feature if all monitors
in the quorum support it. It protects against the following scenario:

  * the leader is upgraded the peons are not upgraded
  * the leader creates a pool with plugin=lrc because all OSD have
    the PLUGINS_V2 feature
  * the leader goes down and a non upgraded peon becomes the leader
  * an old OSD tries to join the cluster
  * the new leader will let the OSD boot because it does not contain
    the logic that would excluded it
  * the old OSD will fail when required to load the plugin lrc

This is going to be needed each time new plugins are added, which is
impractical. A more generic plugin upgrade support should be added
instead, as described in http://tracker.ceph.com/issues/7291.

http://tracker.ceph.com/issues/9343 Refs: #9343

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Merge pull request #2551 from dachary/wip-9343-erasure-code-feature

erasure code feature

Reviewed-by: João Eduardo Luís <joao@redhat.com>