DNM: librados: fix watch/notify event flushing

We have some occasional crashes in QA that are due to notify callbacks that are
still running when teh IoCtx gets shut down.  This series adds a flush that lets
them make sure they are completed before shutting down.

I originally took a different appraoch and tried to do this in the unwatch() request
itself.  I'm not sure that's the best path though because it may be that we want
to call unwatch in a context where it is not a good idea to block (on a user
callback needing to complete).

I like this better, but now it occurs to me that RBD as a user may not find a global
"flush all callbacks for this librados clietn" call to be as useful.. perhaps we want
to flush just one image's callbacks when we close the image.  @jdurgin, what do you
think?
we are holding watch_lock

On Fri, 19 Dec 2014, Jason Dillaman wrote:

> 
> In src/osdc/Objecter.cc:
> 
> > @@ -522,8 +525,10 @@ void Objecter::_linger_reconnect(LingerOp *info, int r)
> >      if (!info->last_error) {
> >        r = _normalize_watch_error(r);
> >        info->last_error = r;
> > -      if (info->watch_context)
> > -	finisher->queue(new C_DoWatchError(info, r));
> > +      if (info->watch_context) {
> > +	finisher->queue(new C_DoWatchError(this, info, r));
> > +	_linger_callback_queue();
> 
> Potential race condition if finisher invoked C_DoWatchError before you called _linger_callback_queue?
> 
> ?
> Reply to this email directly or view it on GitHub.[AAabh3jDVaWdhgzn2JqDVe2QzhX_YwK3ks5nZIe4gaJpZM4DKs-L.gif]
> 
> 
>