mon: Paxos: add support to enable/disable trimming

This support is essential for the Monitor's store synchronization
mechanisms.

We must revisit this patch, since we have yet to make 'trim_enable()'
trim the Paxos state if it is needed.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: MonitorDBStore: add store iterators to obtain chunks for sync

We created an interface specific to the MonitorDBStore, which can be used
to create iterators to obtain chunks for sync.

Two different iterators were defined: one that will iterate over the whole
store, focusing on the specified set of prefixes; another that will
iterate over only one specific prefix.

These two different iterators allow us build the sync process in two
distinct phases: 1) obtain all key/value pairs for paxos and all paxos
services, bundle them in chunks and send them over the wire; and 2) obtain
all the paxos versions, bundle them in chunks and send them over the wire.

Also, we are currently considering a chunk to be (at most) 1 MB worth of
data, although it can be tuned using 'mon_sync_max_payload_size' option.

mon: MonitorDBStore: add crc support when --mon-sync-debug is set

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
message: MMonSync: Monitor Synchronization message (WIP)

This commit message will be updated with the full description once this is
no longer WIP. It is mostly done, but there are certainly things that will
end up going away and other things that will be introduced.

(update as of 08/10/12: This design is mostly complete; will change the
commit message as soon as we are sure nothing is missing)

msg: Message: Add MMonSync msg type id and create message payload when
decoding.
message: MMonSync: add crc field
message: MMonSync: add crc to encode/decode functions
message: MMonSync: Add 'reply_to' field
message: MMonSync: add an 'abort' operation type.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Monitor: Add initial support for synchronization (WARNING: WIP)

This is very on-going work, and I've actually stopped working on it for the
time being while addressing other pieces of this puzzle. And I feel I can't
emphasize this enough: this is on-going work.

There is a lot of stuff missing, some other stuff was completely left in
a limbo of half-done/half-undone, and it will certainly NOT WORK.

Also, there are a couple of things that are to be replaced, some other
stuff will be stripped away.

(update as of 08/10/12: the Monitor class compiles; will not work as the
'handle_sync()' function is commented out)

mon: Monitor: messaging for synchronizing two monitor stores
mon: Monitor: ignore heartbeat replies if we are no longer synchronizing
mon: Monitor: backoff trim enable once the last sync requester finishes
mon: Monitor: wait for the leader to reply to 'finish' before bootstrapping
mon: Monitor: clear store before starting fresh sync
paxos: change paxos_max_join_drift comment to sync instead of slurp
mon: Monitor: add chunk crc support when --mon-sync-debug is set
mon: Monitor: add '_sync_status()' command to the monitor (via asok)
mon: Monitor: Add kill at intruction points for debugging purposes
mon: Monitor: remove some old dependencies/checks for slurp
mon: Monitor: abort in-progress sync if leader changes
mon: Monitor: sync: send heartbeats at configurable intervals
mon: Monitor: send aborts to every on-going sync when leader is demoted
mon: Monitor: allow sync even if we should trim iff there is no quorum
mon: Monitor: Clean up the store after a failed sync
  Do not leave a potentially inconsistent lingering state in the store,
  even if it could potentially lead to a successful quorum membership,
  given that it  could still pose a problem if one or more store keys
  didn't get to be sync'ed.
mon: Monitor: add state barriers to synchronization functions
	Don't let stray messages mess with the monitor.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Monitor: rework when and how we sync when there's no quorum

mon: Monitor: rework when and how we sync when there's no quorum
message: MMonProbe: slightly adapt for a single-paxos approach
mon: Monitor: rework sync and election condition in quorum absence

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Monitor: pretty up the sync code (remove/add comments)

mon: Monitor: clean up old commented out code

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Monitor: get rid of old slurp code.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: PaxosService: clean up and add docs

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Paxos: get rid of slurp-related code

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Paxos: cleanup old commented out code

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Monitor: sync docs

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>