RWLock: assert pthread function return values

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
OSD::_share_map_incoming: line wrap debug output

Formatting only.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
OSD: add handle_osd_map debug output

Signed-off-by: Greg Farnum <greg@inktank.com>
OSD: replace handle_pg_scan, handle_pg_backfill with handle_replica_op

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
OSD: remove unused push_wq

Signed-off-by: Greg Farnum <greg@inktank.com>
OSD: rename gen_wq, schedule_work, and PG_QueueAsync to include "recovery"

These all hook into the recovery thread pool and need to make that obvious.

Signed-off-by: Greg Farnum <greg@inktank.com>
OSD: remove never-activated while loop from send_incremental_map

Signed-off-by: Greg Farnum <greg@inktank.com>
DispatchQueue: factor out pre_dispatch and post_dispatch

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
Messenger,DispatchQueue: add ms_fast_dispatch mechanism

This adds a Dispatcher interface allowing the implementation
to accept ms_fast_dispatch calls for some messages without
going through the DispatchQueue. To support that, we also add new
synchronous notifications on connect and accept events.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
OSDService: add osdmap reservation mechanism

The goal here is to be able to get "reserved" refs
to next_map, and ensure that pgs won't see a newer
map until the ref is "released".  I haven't done
a cute RAII trick here yet...probably not worth
the effort.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>

Conflicts:
	src/osd/OSD.h
Signed-off-by: Greg Farnum <greg@inktank.com>
OSD: add a RWLock pg_map_lock

If we're going to dispatch ops without grabbing the osd lock, we need
something else to protect the pg map (and it'll be a little
contended, so use a read-write lock).

Signed-off-by: Samuel Just <sam.just@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
OSD: pass osdmap to handle_op and handle_replica_op

We need a map to process them, and we don't want to
take the OSD lock to access one. (And we can't just
use the service because we need all processing of
a message to be done with the same map.)

Signed-off-by: Samuel Just <sam.just@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
OSD: add session waiting_for_map mechanisms

This will replace the existing waiting_for_osdmap mechanism
with a per-session wait list.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
OSD: remove wake_all_pg_waiters

We shouldn't need this -- we check the pg waiters list on each
map.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
OSD: wake_pg_waiters atomically with pg_map update

Also, call enqueue_op directly rather than going back
through the entire dispatch machinery.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
OSD: protect access to boot_epoch, up_epoch, bind_epoch

We need to access these members in some call chains via fast_dispatch,
where they're otherwise unprotected.

Signed-off-by: Greg Farnum <greg@inktank.com>
OSD::_share_map_incoming: pass osdmap in explicitly

We'll want to be able to use this method without the osd_lock. Note
that we can't do so yet -- we call send_incremental_map, which is not
safe to call unlocked.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
OSD::send_incremental_map: use service superblock so we can avoid locking osd_lock

TODO: make it actually safe by dealing with build_incremental_map_msg()

Signed-off-by: Samuel Just <sam.just@inktank.com>
OSD: use safe params in map-sharing functions

We were previously using unprotected access to OSD members.

Unfortunately, this does not make them completely safe: we are looking up
maps asynchronously from when we got access to the cached map bounds, and
so the OSD could delete a map out from underneath us. Fixing that will
require some kind of map bounds lock. :/

Signed-off-by: Greg Farnum <greg@inktank.com>
OSD: use an OSDMapRef& and require the Session* in _share_map_incoming

You can pass in a NULL Session*, but both callers do that; and using
an OSDMapRef& reduces shared_ptr copies.

Signed-off-by: Greg Farnum <greg@inktank.com>