mon: explicitly refresh_from_paxos() when leveldb state changes

Instead of opportunistically calling each service's update_from_paxos(),
instead explicitly refresh all in-memory state whenever we know the
paxos state may have changed.  This is simpler and less fragile.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit cc339c07312006e65854207523f50542d00ecf87)
mon: remove unnecessary update_from_paxos calls

The refresh() will do this when the state changes; no need to
opportunistically call this method all of the time.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 03014a4ecc06cde420fad0c6c2a0177ebd7b839d)
mon: no need to refresh from _active

The refresh is done explicitly by the monitor, independent of the more
fragile PaxosService callbacks.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit d941363d6e4249e97b64faff0e573f75e918ac0c)
mon/PaxosService: cache {first,last}_committed

Refresh the in-memory values when we are told the on-disk paxos state
may have changed.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
(cherry picked from commit 2fccb300bdf6ffd44db3462eb05115da11322ed4)
mon/Paxos: do paxos refresh in finish_proposal; and refactor

Do the paxos refresh inside finish_proposal, ordered *after* the leader
assertion so that MonmapMonitor::update_from_paxos() calling bootstrap()
does not kill us.

Also, remove unnecessary finish_queued_proposal() and move the logic inline
where the bad leader assertion is obvious.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit a42d7582f816b45f5d19c393fd45447555e78fdd)
mon: safely signal bootstrap from MonmapMonitor::update_from_paxos()

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit dc83430124a5fd37573202a4cc0986c3c03739ef)
mon/Paxos: go active *after* refreshing

The update_from_paxos() methods occasionally like to trigger new activity.
As long as they check is_readable() and is_writeable(), they will defer
until we go active and that activity will happen in the normal callbacks.

This fixes the problem where we active but is_writeable() is still false,
triggered by PGMonitor::check_osd_map().

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e68b1bd36ed285e38a558899f83cf224d3aa60ed)
mon: simplify Monitor::init_paxos()

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e832e76a4af04b091c806ad412bcfd0326d75a2d)
mon/PaxosService: simplify readable check

Recast this in terms of the paxos check and our additional conditions,
which match wait_for_readable().

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 3aa61a0beb540e48bf61ceded766d6ff52c95eb2)
mon/PaxosService: simplify is_writeable

Recast this in terms of paxos check + our conditions, and make it
match wait_for_writeable().

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit f985de28f86675e974ac7842a49922a35fe24c6c)
mon/Paxos: cleanup: drop unused PREPARING state bit

This is never set when we block, and nobody looks at it.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 7b7ea8e30e20704caad9a841332ecb2e39819a41)
mon/Paxos: not readable when LOCKED

If we are re-proposing a previously accepted value from a previous quorum,
we should not consider it readable, because it is possible it was exposed
to clients as committed (2/3 accepted) but not recored to be committed, and
we do not want to expose old state as readable when new state was
previously readable.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit ec2ea86ed55e00265c2cc5ad0c94460b4c92865c)
mon: simplify states

- make states mutually exclusive (an enum)
- rename locked -> updating_previous
- set state prior to begin() to simplify things a bit

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit ee34a219605d1943740fdae0d84cfb9020302dd6)
mon/PaxosService: not active during paxos UPDATING_PREVIOUS

Treat this as an extension of the recovery process, e.g.

 RECOVERING -> ACTIVE
or
 RECOVERING -> UPDATING_PREVIOUS -> ACTIVE

and we are not active until we get to "the end" in both cases.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 392a8e21f8571b410c85be2129ef62dd6fc52b54)
mon/PaxosService: allow paxos service writes while paxos is updating

In commit f985de28f86675e974ac7842a49922a35fe24c6c I mistakenly made
is_writeable() false while paxos was updating due to a misread of
Paxos::propose_new_value() (I didn't see that it would queue).
This is problematic because it narrows the window during which each service
is writeable for no reason.

Allow service to be writeable both when paxos is active and updating.

Signed-off-by: Sage Weil <sage@inktank.com>
(cherry picked from commit 11169693d086e67dcf168ce65ef6e13eebd1a1ab)
common/pick_addresses: behave even after internal_safe_to_start_threads

ceph-mon recently started using Preforker to working around forking issues.
As a result, internal_safe_to_start_threads got set sooner and calls to
pick_addresses() which try to set string config values now fail because
there are no config observers for them.

Work around this by observing the change while we adjust the value.  We
assume pick_addresses() callers are smart enough to realize that their
result will be reflected by cct->_conf and not magically handled elsewhere.

Fixes: #5195, #5205
Backport: cuttlefish
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
(cherry picked from commit eb86eebe1ba42f04b46f7c3e3419b83eb6fe7f9a)