osd/OSDMap: add osd_primary_affinity fields, accessors, encoding

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: add 'osd primary-affinity ...' command

Signed-off-by: Sage Weil <sage@inktank.com>
osd/OSDMap: fix _raw_to_up_osds for EC pools

When filtering down OSDs out of the result set (i.e., raw -> up), we can
shift positions for replicated pools but need to preserve position for
EC pools.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/OSDMap: pass pps value out from _pg_to_osds

The primary affinity code needs a seed to determine it's pseudo-random
behavior.  Pass pps (optionally) out of _pg_to_osds() for this purpose.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/OSDMap: apply primary_affinity to mapping

The behavior is a bit different for replicated and indep/erasure mode.
In the first case, we are rearranging the result.  In the second case,
we can just set the primary argument to the right value.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/OSDMap: add primary_affinity feature bit

Indicate that we support it.  Indicate when an OSDMap requires it.

Signed-off-by: Sage Weil <sage@inktank.com>
ceph_psim: some futzing to test primary_affinity

- map to acting
- count first position, primary

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: add 'mon osd allow primary affinity' bool option

By default, disallow adjustment of primary affinity unless the user has
opted in by adjusting their monitor config.  This will avoid some user
pain because inadvertantly setting the affinity will prevent older clients
from connecting to and using the cluster.

Signed-off-by: Sage Weil <sage@inktank.com>
osd/OSDMap: remove bad assert

You can have an erasure poool with all CRUSH_ITEM_NONE and primary == -1.
acting is not empty.

Signed-off-by: Sage Weil <sage@inktank.com>