mon: explicitly refresh_from_paxos() when leveldb state changes

Instead of opportunistically calling each service's update_from_paxos(),
instead explicitly refresh all in-memory state whenever we know the
paxos state may have changed.  This is simpler and less fragile.

Signed-off-by: Sage Weil <sage@inktank.com>
mon: remove unnecessary update_from_paxos calls

The refresh() will do this when the state changes; no need to
opportunistically call this method all of the time.

Signed-off-by: Sage Weil <sage@inktank.com>
mon: no need to refresh from _active

The refresh is done explicitly by the monitor, independent of the more
fragile PaxosService callbacks.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/PaxosService: cache {first,last}_committed

Refresh the in-memory values when we are told the on-disk paxos state
may have changed.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon/Paxos: do paxos refresh in finish_proposal; and refactor

Do the paxos refresh inside finish_proposal, ordered *after* the leader
assertion so that MonmapMonitor::update_from_paxos() calling bootstrap()
does not kill us.

Also, remove unnecessary finish_queued_proposal() and move the logic inline
where the bad leader assertion is obvious.

Signed-off-by: Sage Weil <sage@inktank.com>
mon: safely signal bootstrap from MonmapMonitor::update_from_paxos()

Signed-off-by: Sage Weil <sage@inktank.com>
mon/Paxos: go active *after* refreshing

The update_from_paxos() methods occasionally like to trigger new activity.
As long as they check is_readable() and is_writeable(), they will defer
until we go active and that activity will happen in the normal callbacks.

This fixes the problem where we active but is_writeable() is still false,
triggered by PGMonitor::check_osd_map().

Signed-off-by: Sage Weil <sage@inktank.com>
mon: simplify Monitor::init_paxos()

Signed-off-by: Sage Weil <sage@inktank.com>
mon: debug proposal timers

Signed-off-by: Sage Weil <sage@inktank.com>
mon: fix whitespace

Signed-off-by: Sage Weil <sage@inktank.com>
mon/MonmapMonitor: fix typo in check to remove mkfs info

Signed-off-by: Sage Weil <sage@inktank.com>
mon/MonmapMonitor: remove some dead code

Signed-off-by: Sage Weil <sage@inktank.com>
mon/PaxosService: simplify readable check

Recast this in terms of the paxos check and our additional conditions,
which match wait_for_readable().

Signed-off-by: Sage Weil <sage@inktank.com>
mon/PaxosService: simplify is_writeable

Recast this in terms of paxos check + our conditions, and make it
match wait_for_writeable().

Signed-off-by: Sage Weil <sage@inktank.com>
mon/Paxos: cleanup: drop unused PREPARING state bit

This is never set when we block, and nobody looks at it.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/Paxos: not readable when LOCKED

If we are re-proposing a previously accepted value from a previous quorum,
we should not consider it readable, because it is possible it was exposed
to clients as committed (2/3 accepted) but not recored to be committed, and
we do not want to expose old state as readable when new state was
previously readable.

Signed-off-by: Sage Weil <sage@inktank.com>
mon: simplify states

- make states mutually exclusive (an enum)
- rename locked -> updating_previous
- set state prior to begin() to simplify things a bit

Signed-off-by: Sage Weil <sage@inktank.com>
mon/PaxosService: not active during paxos UPDATING_PREVIOUS

Treat this as an extension of the recovery process, e.g.

 RECOVERING -> ACTIVE
or
 RECOVERING -> UPDATING_PREVIOUS -> ACTIVE

and we are not active until we get to "the end" in both cases.

Signed-off-by: Sage Weil <sage@inktank.com>