encoding: allow users to specify a different compatv after encoding

This way we can set the compatv preferentially depending on whether
we've actually encoded new information or not.

Signed-off-by: Greg Farnum <greg@inktank.com>
osd/osd_types: add explicit hash to object_locator_t

Instead of hashing the object name or key, we allow the hash position to be
provided explicitly.

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
osd/OSDMap: use new object_locator_t::hash to place object in a pg

The hash value, if provided, becomes the ps (placement seed) portion of the
pg_t, skipping any hashing of the object name (or locator key).

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
osd: capture hashing of objects to hash positions/pgs in pg_pool_t

The hashing is dependent on pool properties; capture (more of) it in a
method instead of having it in OSDMap.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
osdc/Objecter: expose methods for getting object hash position and pg

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
librados: expose methods for calculating object hash position

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
librados: add wait_for_latest_osdmap()

There are times when users may need to make sure the client has the
latest osdmap, for example after sending a mon command modifying
pool properties.

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>

squash "librados: add wait_for_latest_osdmap()"
osd/PG: factor out get_next_version()

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
osd/ReplicatedPG: factor out simple_repop_{create,submit} helpers

This makes it easier to create repops correctly, and should help
prevent bugs like the one we remove here in process_copy_op (we were
serializing on the wrong object!)

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
osd_types: add generic HitSet type with bloom and explicit implementations

Track a set of hash values, either explicitly or using a bloom_filter. Hide
the implementation and allow us to transparently encode and decode.

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
osd/osd_types: add pg_hit_set_{info,history}_t

Signed-off-by: Sage Weil <sage@inktank.com>
osd/osd_types: include pg_hit_set_history_t in pg_info_t

Track metadata about the currently accumulating HitSet as well as
previously archived ones in the pg_info_t.  This will not scale well for
extremely long histories, but does let us avoid explicitly sharing this
metadata during recovery or other normal update activity.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: add hit_set_* parameters to pg_pool_t

Add pool properties to control what type of HitSet we want to use, along with
some (mostly generic) parameters.

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
mon/OSDMonitor: set hit_set fields

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
osd/ReplicatedPG: add basic HitSet tracking

Signed-off-by: Sage Weil <sage@inktank.com>
osd/ReplicatedPG: use vectorized osd_op outdata for pg ops

This lets us put PGLS in a compound operation.  Nothing does that yet, but
this would allow it.
Despite appearances, this is not a protocol change and does not require
a feature bit for clients: using the osd_ops vector mechanisms store all
the data in the same places as before, it just fills in some of the
already-decoded-but-empty data structures in the MOSDOpReply header.
<Greg note:> We may need a feature bit to let clients know they can send
compound PG ops to OSDs, though? Or maybe we can let it be covered
by supporting hitset ops.

Signed-off-by: Sage Weil <sage@inktank.com>
librados, osd: list and get HitSets via librados

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
librados: create new ceph_test_rados_api_tier target

Move the dirty/undirty test to it, and add one for HitSets.

Signed-off-by: Sage Weil <sage@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>