osd: rejigger watch connect/disconnect callers

Signed-off-by: Sage Weil <sage@redhat.com>
encoding: multiset encode/decode

Signed-off-by: Sage Weil <sage@redhat.com>
librados: include missed watchers in notify reply data

Include a list of who missed their notify in the notify reply.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: use (gid,cookie) to identify notify participants

Signed-off-by: Sage Weil <sage@redhat.com>
osdc/Objecter: use recast LingerOp* as OTW cookie, too

This makes the OSD reply's cookie values match up with what the librados
users see.  And it makes life less confusing when debugging.

Signed-off-by: Sage Weil <sage@redhat.com>
osdc/Objecter: make watch_check time reflect async delivery

When a librados user calls watch_check(), the age they get back should
establish a timestamp for which:

 1) we know that the watch was registered at least through this time,
 2) we have received and processed notifies sent as of this time,
 3) we have processed any async error events as of this time.

We already accomplish 1 by updating watch_valid_thru based on the time
the ping is sent (not received).

This patch gets us 2 and 3 by using the MIN of watch_valid_thru and the
oldest queued async event for the watch_check result.

Signed-off-by: Sage Weil <sage@redhat.com>
osdc/Objecter: use RWLock for watch_lock

This makes the linger_check fast path an RWLock read lock.  Drop the
unnecessary write lock on rwlock and drop the unused Cond.

Signed-off-by: Sage Weil <sage@redhat.com>
osdc/Objecter: set last_error before async error event delivery

No reason to delay this.

Signed-off-by: Sage Weil <sage@redhat.com>
test_async_driver: fix warning

Signed-off-by: Sage Weil <sage@redhat.com>
PendingReleaseNotes: a few notes about watch/notify changes

Signed-off-by: Sage Weil <sage@redhat.com>
ceph_test_rados_api_watch_notify: test ENOENT case

Signed-off-by: Sage Weil <sage@redhat.com>
ceph_test_rados_watch_notify: do not unwatch on invalid handle

It is not safe to call rados_unwatch on an undefined handle.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: s/handle/cookie/ in headers

Be consistent.

Signed-off-by: Sage Weil <sage@redhat.com>