librados: define CEPH_OSD_WATCH_OP_*

Signed-off-by: Sage Weil <sage@redhat.com>
librados: use new watch op codes; simplify Objecter helpers

- drop the useless add_watch() helper; do it explicitly
- drop the unused var arg everywhere
- make a separate notify member of the union that excludes
  the other unused fields

Signed-off-by: Sage Weil <sage@redhat.com>
osdc/Objecter: fix assert_src_version

Don't use the add_watch() helper; do it like assert_version
does it.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: define RECONNECT and PING watch ops

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: handle PING and RECONNECT watch ops

The ping will essentially assert that a watch is still valid.

A reconnect will reestablish session state *only* if the watch is
still persistent.  If not, it will fail (and the client will know it
may have missed something).

Note that the only difference here is that a PING is a bit lighter
weight; it will not reestablish the session state (which should already
be established).  We could use a single op here but the unique op
code makes the messages easier to understand and simplifies the code
path a bit for PING.

Signed-off-by: Sage Weil <sage@redhat.com>
librados: remove crufty declaration

Signed-off-by: Sage Weil <sage@redhat.com>
librados: define error callback

This is the async notification of a watch error (failure to reconnect to
the object's OSD after a peering event or network disconnect).

Signed-off-by: Sage Weil <sage@redhat.com>
librados: add infrastructure to deliver an error notification

Use a reusable context in the WatchNotifyInfo to trigger an error
event, delivered via the existing Finisher thread.  Re-lookup the cookie
in the thread to cope with races with unregister (just as we do with
notify events).

Signed-off-by: Sage Weil <sage@redhat.com>
rados: improve watch command output

- hexdump the notify payload, and include all of the new metadata
- print error events too

Signed-off-by: Sage Weil <sage@redhat.com>
osdc/Objecter: separate WATCH from RECONNECT

Use WATCH op for the initial registration.  This is idempotent in that
it will succeed whether the watch information has been persisted or not.
It is used by the client if it does not know that it is registered.

The RECONNECT op is used for any subsequent session reconnect.  It will
fail if the watch state isn't already persisted on the OSD.

Signed-off-by: Sage Weil <sage@redhat.com>
osdc/Objecter: queue an error if reconnect fails

If we fail to reconnect, trigger the on_error (reusable) context to
let the user know about it.

Signed-off-by: Sage Weil <sage@redhat.com>
osdc/Objecter: send regular PING ops

Send a full PING op to the object to ensure we are still connected. For
now just use the existing ping interval; we may want to change this in
the future.

Signed-off-by: Sage Weil <sage@redhat.com>