ceph/ceph — pull request #1227 by gregsfortytwo (2014-02-12 22:19:06+00:00)
Title: Improve the OSD and generic monitor timeout mechanisms

We have previously timed out monitors in the OSD logic by looking at when they ack PGStats messages, and have done very little within the MonClient. This series improves the generic and specific issues:
1) The MonClient exposes more information to users about the state of its connection, by optionally triggering callbacks on session completion.
2) The MonClient times out when reopening sessions, and tries a different monitor after a long enough time (which it previously did only on the initial connection) — with backoffs.
3) The OSD suspends its PGStats-based timeouts while the MonClient is still trying to establish a connection.

The net effect of this is that when the monitors are misbehaving, they might get more MAuth messages attempting to open new sessions (because the timeout backoff maxes out at 30s by default; the same as it was getting via the PGStats mechanism), but it will get much fewer messages requiring any other real work because the OSD won't be resending its messages and then timing out.