osd: do not make pg_pool_t incompat when hit_sets are enabled

If we enable HitSet tracking, the OSD needs to know this, but clients do
not care.  Setting the compat version is too heavyweight as it locks out
older kernels (*any* currents, currently) who are unaffected by the new
fields.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: prevent setting hit_set unless all OSDs support it

We are using PRIMARY_AFFINITY as a proxy for the support for the tiering
OSDMap infrastructure.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: require TIER feature before using tiering features

The OSDs need to support this feature before we allow users to turn it
on.  This is similar to what the erasure pool support does.

Signed-off-by: Sage Weil <sage@inktank.com>
Merge pull request #1543 from ceph/wip-7875

osd: do not make pg_pool_t incompat when hit_sets are enabled

Reviewed-by: Samuel Just <sam.just@inktank.com>