mon: set forwarded message recv stamp

Set it to the stamp of the MForward that carried us.  One could argue
we really want the original receive stamp on the origin, but that is
not available to us, and this is better than nothing.

In particular, this gives 'ceph log ...' commands a timestamp when they
are forwarded via a peon.  The stamp is still between when the request
is sent and when it is committed/acked, so all is well from the
client's perspective.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/PaxosService: remove redundant check

_active checks is_active() and rescheduled itself already.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
mon/PaxosService: do not prepare new pending if still proposing

The _active callback can get called while are already proposing.  If
that happens, we should not prepare a fresh new pending but should
wait for the previous proposal to finish.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/Paxos: bootstrap peon too if monmap updates

If we get a monmap update, the leader bootstraps.  Peons should do the
same.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
mon: fix scrub vs paxos race: refresh on commit, not round completion

Consider:

 - paxos starts a commit N+1
 - a majority of the peers ack it
  - paxos::commit() writes N+1 it to disk
  - tells peers to commit
 - peers commit N+1, *and* refresh_from_paxos(), and generate N+1 full map
 - leader does _scrub on N+1, without latest full osdmap
 - peers do _scrub on N+1, with latest full osdmap
 - leader finishes paxos gather, does refresh_from_paxos()
 -> scrub fails.

Fix this by doing the refresh_from_paxos() at commit time and not when
the paxos round finishes.  We move the refresh out of finish_proposal
and into its own helper, and update all callers accordingly.  This
keeps on-disk state more tightly in sync with in-memory state and
avoids the need for a e.g., kludgey workaround in the scrub code.

We also simplify the bootstrap checks a bit by doing so immediately
and relying on the normal bootstrap paxos reset paths to clean up
any waiters.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/PaxosService: assert not proposing in propose_pending

Signed-off-by: Sage Weil <sage@inktank.com>
mon/PaxosService: consolidate resetting in restart()

We had duplicated code in election_finished() and restart(), and it was
incomplete.  Put it all in reset() only (the mon should have called
restart() long before the election finishes).  Note that we cannot
assert as much in election_finished() because another service may have
just cross-proposed.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/Paxos: separate proposal commit from the end of the round

Each commit should match with exactly one proposal; finish it when we
actually commit it and make sensible asserts.

The old finish_proposal() turns into finish_round(), and performs
generic checks and cleanup associated with the transition from
updating -> active.

Signed-off-by: Sage Weil <sage@inktank.com>