mon/PGMonitor: store PGMap directly in store, bypassing PaxosService stash_full

Instead of encoding incrementals and periodically dumping the whole encoded
PGMap, instead store everything in a range of keys, and update them
between versions using transactions.  The per-version values are now
breadcrumbs indicating which keys were dirtied so they can be refreshed
via update_from_paxos().

This has several benefits:
 - we avoid every encoding the entire PGMap
 - we avoid dumping that blob into leveldb keys
 - we limit the amount of data living in forward-moving keys, which leveldb
   has a hard time compacting away
 - pgmap data instead lives over a fixed range of keys, which leveldb
   excels at
 - we only keep the latest copy of the PGMap (which is all we care about)

Bump the internal monitor protocol version.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/PGMonitor: use accessors for main pg_map fields

Signed-off-by: Sage Weil <sage@inktank.com>
mon/PaxosService: allow paxos service writes while paxos is updating

In commit f985de28f86675e974ac7842a49922a35fe24c6c I mistakenly made
is_writeable() false while paxos was updating due to a misread of
Paxos::propose_new_value() (I didn't see that it would queue).
This is problematic because it narrows the window during which each service
is writeable for no reason.

Allow service to be writeable both when paxos is active and updating.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/Paxos: drop unnecessary last_committed loads

Drop (apparently) ad-hoc refreshes of last_committed from the store.
These are unnecessary and confusing.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/Paxos: assert that the store gives us back what we just wrote

In bug #5424 I observed leveldb failing internally and then returning
bad info.  We then hit a random/confusing assert.  Try to detect this
earlier by verifying that a get of a just-written last_committed gives
us back the right thing.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: enable leveldb cache by default

512 MB sounds reasonable to me.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/Paxos: never write first_committed except during trim

The trimming is handled by proposing transactions.  Do not confuse matters
by writing (incorrect) first_committed values at any other point.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/Paxos: set first_committed on first commit

Signed-off-by: Sage Weil <sage@inktank.com>
mon/Paxos: update first_committed only from paxos

Do not touch the in-memory first_committed until the trim commits.  This
avoids any possible confusion due to races and keeps commit() as similar
to store_state() as possible.

Similarly, do not touch first_committed from store_state.  We should
*only* pull it out of the kv store.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/Paxos: clean up removal of pre-conversion paxos states

Use a helper, independent of trim machinery, and call on leader, too.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/PaxosService: rename scrub and call from refresh()

Call at the earliest point where we know the paxos state beneath us is
consistent.  Make the name patch the one in Paxos.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/Paxos: simplify trim()

Collapse all the trim methods into a single simple method.

Signed-off-by: Sage Weil <sage@inktank.com>
mon: do not reopen MonitorDBStore during startup

level doesn't seem to like this when it races with an internal compaction
attempt (see below).  Instead, let the store get opened by the ceph_mon
caller, and pull a bit of the logic into the caller to make the flow a
little easier to follow.

    -2> 2013-06-25 17:49:25.184490 7f4d439f8780 10 needs_conversion
    -1> 2013-06-25 17:49:25.184495 7f4d4065c700  5 asok(0x13b1460) entry start
     0> 2013-06-25 17:49:25.316908 7f4d3fe5b700 -1 *** Caught signal (Segmentation fault) **
 in thread 7f4d3fe5b700

 ceph version 0.64-667-g089cba8 (089cba8fc0e8ae8aef9a3111cba7342ecd0f8314)
 1: ceph-mon() [0x649f0a]
 2: (()+0xfcb0) [0x7f4d435dccb0]
 3: (leveldb::Table::BlockReader(void*, leveldb::ReadOptions const&, leveldb::Slice const&)+0x154) [0x806e54]
 4: ceph-mon() [0x808840]
 5: ceph-mon() [0x808b39]
 6: ceph-mon() [0x806540]
 7: (leveldb::DBImpl::DoCompactionWork(leveldb::DBImpl::CompactionState*)+0xdd) [0x7f363d]
 8: (leveldb::DBImpl::BackgroundCompaction()+0x2c0) [0x7f4210]
 9: (leveldb::DBImpl::BackgroundCall()+0x68) [0x7f4cc8]
 10: ceph-mon() [0x80b3af]
 11: (()+0x7e9a) [0x7f4d435d4e9a]
 12: (clone()+0x6d) [0x7f4d4196bccd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

Signed-off-by: Sage Weil <sage@inktank.com>