ceph/ceph — pull request #2924 by liewegas (2014-11-13 23:54:40+00:00)
Title: DNM: new watch/notify framework

This is mostly complete. The main remaining items are to convert rgw and
librbd over to use the new version of the API.

Questions:

1) Not sure if the handle_failed_notify() notifications are actually useful.
The original notifier doesn't know whether or not they were delivered, and
if they really want to tell people they missed one notify they could send
a second notify with that information. The only information it conveys is
that we were too slow acking a notify.

2) Does the watch_check() API make sense?

3) I wasn't quite motivated enough to pull all of the RadosClient watch/notify
machinery into Objecter where it probably belongs. As such, there is still
an odd separation between the two that is awkward in a couple of places
(getting error code back to librados, and the watch_check check). Good
enough?

4) Do we need to optimize the OSD watch ping path at all? Currently it cancels
and reschedules a timeout event.

5) Is anything else needed for librbd to make good use of this?

The other thing I ran out of steam before doing is writing a reference user for
the API that uses all elements to full effect. I think the use case I have in
mind is a shared cache with invalidates, like what RGW is doing. But instead of
writing a toy or test implementation, it would be best to write a generic
shared-cache module that has all the features RGW needs and then migrate to using
it. Maybe that will be simpler than fixing up RGW to make use of watch_check()
properly...?