osd/OSDMap: add osd_is_valid_op_target()

Helper to check whether an osd is a given op target for a pg.  This
assumes that for EC we always send ops to the primary, while for
replicated we may target any replica.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 89c02637914ac7332e9dbdbfefc2049b2b6c127d)
osd: discard rank > 0 ops on erasure pools

Erasure pools do not support read from replica, so we should drop
any rank > 0 requests.

This fixes a bug where an erasure pool maps to [1,2,3], temporarily maps
to [-1,2,3], sends a request to osd.2, and then remaps back to [1,2,3].
Because the 0 shard never appears on osd.2, the request sits in the
waiting_for_pg map indefinitely and cases slow request warnings.
This problem does not come up on replicated pools because all instances of
the PG are created equal.

Fix by only considering role == 0 for erasure pools as a correct mapping.

Fixes: #9835
Signed-off-by: Sage Weil <sage@redhat.com>
osd: use OSDMap helper to tell if ops are misdirected

calc_pg_role doesn't actually take into account primary affinity.

Fixes: #9835
Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge pull request #2760 from ceph/wip-9835-firefly

osd: fix erasure hung op bug (9835)

Reviewed-by: Samuel Just <sjust@redhat.com>