osd_types: introduce spg_t, pg_shard_t

Signed-off-by: Samuel Just <sam.just@inktank.com>
PGBackend/ReplicatedBackend: move rollback methods into PGBackend

These will end up essentially unchanged in ECBackend as it turns out.

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedBackend/PGBackend: move objects_* methods into PGBackend

These also are unchanged in ECBackend.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PGBackend: use whoami_shard().shard in rollback*, objects*

Signed-off-by: Samuel Just <sam.just@inktank.com>
PGBackend/ReplicatedBackend: move temp handling into PGBackend

Temp handling is also the same in ReplicatedBackend as in ECBackend.

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedBackend: excise OSDService*

This should eventually make it easier to mock out a PGBackend::Listener.

Signed-off-by: Samuel Just <sam.just@inktank.com>
messages/: include shard information in various pg messages

We can no longer use the messenger source information to determine
the origin of the message since an osd might have more than one
shard of a particular pg.  Thus, we need to include a pg_shard_t
from field to indicate origin.  Similarly, pg_t is no longer
sufficient to specify the destination pg, we instead use spg_t.
In the event that we get a message from an old peer, we default
from to pg_shard_t(get_source().num(), ghobject_t::no_shard())
and spg_t to spg_t(pgid, ghobject_t::no_shard()).  This suffices
because non-NO_SHARD shards can only appear once ec pools have
been enabled -- and doing that bans unenlightened osds.

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: refer to pg by spg_t and replica by pg_shard_t

We may have multiple pg shards on the same osd for an
ec pool.  Thus, replicas must be referred to by <osd, shard>
and pgs by <pgid, shard>.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: refactor calc_acting to handle pg_shard_t changes

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: introduce calc_ec_acting

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG,osd_types: seperate require_rollback from ec_pool

It's handy to allow a pool to answer false to ec_pool() and
true to require_rollback() in order to allow a replicated
pool to test the rollback mechanisms without allowing
non-NO_SHARD shards.

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSDMap: fill in get_primary_shard

Signed-off-by: Samuel Just <sam.just@inktank.com>
FileStore: permit NULL onreadable callback

Signed-off-by: Samuel Just <sam.just@inktank.com>
os/: allow objects with NO_GEN but non-NO_SHARD shard

Signed-off-by: Samuel Just <sam.just@inktank.com>
SimpleMessenger: init_local_connection whenever my_inst changes

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG,PGBackend: expose acting and backfill_targets

Signed-off-by: Samuel Just <sam.just@inktank.com>
PGBackend: add some additional helpers.

ECBackend's primary specific logic mostly won't treat the
primary shard specially, so it'll be handy to have primary
agnostic helpers for get_shard_info and get_shard_missing.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: allow PGBackend to set criteria for PG up-ness

ECBackend needs to be able to require that a readable
set of the most recent interval to write be available
in order to ensure that it rolls back the log far
enough.

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG,ReplicatedPG: Generalize missing_loc for ECBackend

Prior to EC pools, unfound => missing.  Now, unfound (unreadable,
really) is dependent on the PGBackend requirements for reconstituting
an object.  This also means recovering an object missing on a replica
but not the primary requires tracking the missing_loc set.

Thus, rather than maintaining missing_loc only for objects missing
on the primary, the MissingLoc structure will track all missing
objects actingbackfill-wide until each object is recovered.

For simplicity, since we don't really know what objects need recovery
until activation (and since we can't do anything with that information
prior to activation anyway), we defer populating the missing_loc
information until activation.

We need peers to rollback divergent log entries before we attempt to
read the relevant objects.  The simplest way to accomplish this seems to
be the simply choose to always activate peers if search_for_missing
turns up missing objects.

Finally, due to EC pools, missing is necessary, but not sufficient for
readability.  Thus, we instead check is_unreadable for cases where we
need to read the object and reserve is_missing for cases where we need
the object context.

wait_for_missing_object becomes waiting_for_unreadable_object in order
to avoid having another layer of waiting_for_* maps.

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: replace bufferptr with bufferlist for attrs in recovery

Now we won't have to keep converting back and forth!

Signed-off-by: Samuel Just <sam.just@inktank.com>