msg/Pipe: do not wait for self in Pipe::stop_and_wait()

The fast dispatch code necessitated adding a wait for the fast dispatch
to complete when taking over sockets back in commit
2d5d3097c3998add1061ce253104154d72879237.  This included mark_down()
(although I am not certain mark_down was required to fix the previous set
of races).

In any case, if the fast dispatch thread itself tries to mark down its
own connection, it will deadlock in this method waiting for itself to
return and clear reader_dispatching.  Skip this wait if we are in fact
the reader thread.  This avoids the deadlock.

Alternatively, we could change mark_down() to not use stop_and_wait(), but
I am less clear about the potential races there, so I'm opting for the
minimal (though ugly) fix.

Fixes: #9057
Signed-off-by: Sage Weil <sage@redhat.com>
Merge pull request #2238 from ceph/wip-9057

msg/Pipe: do not wait for self in Pipe::stop_and_wait() 

Reviewed-by: Samuel Just <sam.just@inktank.com>