librados: avoid unnecessary locks

Revise wait_for_osdmap to be called outside of RadosClient::lock
and only take the lock if it has to wait for a map.

Also, now that objecter handles its own locking nicely,
there are various places where it is no longer necessary
for RadosClient to take its own lock -- all the calls that
go directly into objecter (RadosClient::pool_*) don't need
to hold RadosClient::lock.

Signed-off-by: John Spray <john.spray@redhat.com>
messages/MWatchNotify: include an error code in the message

Document the fields, while we are here.

Signed-off-by: Sage Weil <sage@redhat.com>
osd: include ETIMEDOUT in notify reply on timeout

If a notify operation times out (all watchers to not ACK in time), include
an ETIMEDOUT in the final error message back to the client, so that they
know about it.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: rename watch/notify callback register functions

Make it clear these are for watch OR notify, not just watch.  I was
confused.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: prefix msg handler with handle_

Signed-off-by: Sage Weil <sage@redhat.com>
librados: refactor watch/notify; return notify error code

Get rid of a level of intermediate classes with confusing names and put
the notify and notify finish logic in a single place so that it is easier
to follow and understand.

Pass the return value from the notify completion message to the caller.

Signed-off-by: Sage Weil <sage@redhat.com>