mon: Paxos: output proposal queue for debug purposes

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: PaxosService: Crank up the transaction dump log level

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Paxos: Crank up the transaction dump log level

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Paxos: Get rid of the is_leader() function and add an is_lease_valid() function

The Paxos::is_leader() wasn't being used anywhere. Get rid of it for the
sake of simplicity and because it if far more clear if we call the
Monitor::is_leader() function instead.

The Paxos::is_lease_valid() will return true if the lease is valid. It is
useful to consider that on the PaxosService::is_{readable,writeable}
functions.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: PaxosService: redefine the Paxos::is_{active,readable,writeable}() conditions

We should not rely on the Paxos::is_*() functions, since they do not apply
directly to the PaxosService.

All the PaxosService classes share the same Paxos class, but they do not
rely on its values. Each service only relies, uses and updates its own
values on the k/v store. Thus, we may have a given service (e.g., the
OSDMonitor) proposing a new value, hence updating or waiting to update its
store, and we may still consider the LogMonitor as being able to read and
write its own values on the k/v store. In a nutshell, different services
do not overlap on their access to their own store when it comes to reading,
and since the Paxos will queue their updates and deal with them in a FIFO
order, their updates won't overlap either.

Therefore, the conditions for the PaxosService::is_{active,readable,writeable}
differ from those on the Paxos::is_{active,readable,writeable}.

  * PaxosService::is_active() - the PaxosService will be considered as
  active iff it is not proposing and the Paxos is not recovering. This
  means that a given PaxosService (e.g., the OSDMonitor) may be considered
  as being active even though some other service (e.g., the LogMonitor) is
  proposing a new value and the Paxos is on the UPDATING state. This means
  that the OSDMonitor will be able to read its own versions and queue any
  changes on to the Paxos. However, if the Paxos is on state RECOVERING,
  we cannot be considered as active.

  * PaxosService::is_writeable() - We will be able to propose new values
  iff we are the Leader, we have a valid lease, and we are not already
  proposing. If we are proposing, we must wait for our proposal to finish
  in order to proceed with writing to our k/v store; otherwise we could
  incur in assuming that our last committed version was, say, 10; then
  assign map epochs/versions taking that into consideration, make changes
  to the store based on those values, just to come to smash previously
  proposed values on the store. We really don't want that. To be fair,
  there was a chance we could assume we were always writable, but there
  may be unforeseen consequences to this; so we take the conservative
  approach here for now, and we will relax it in the future if we believe
  it to be fruitful.

  * PaxosService::is_readable() - We will be readable iff we are not
  proposing and the Paxos is not recovering; if our last committed version
  exists; and if we are either a cluster of one or we have a valid lease.
  Given everything that was described on this commit message, does it make
  sense? :)

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: PaxosService: Use our own waiting list and waiting functions

We were relying on Paxos to wait for state changes, but since our state
became somewhat independent from the Paxos state, we have to deal with
callbacks waiting for 'readable', 'writable' or 'active' on different
terms than those that Paxos provide.

So, basically, we will take one of two approaches when it comes to waiting:

  * If we are proposing, queue ourselves on our own list, waiting for the
  proposal to finish;
  * Otherwise, the cause for the need to wait comes from Paxos, so queue
  the callback directly on Paxos.

This approach means that we must make sure to check our desired state
whenever the callback is fired up, and re-queue ourselves if the state
didn't quite change (or if it changed but our waiting condition result
didn't). For instance, if we were waiting for a proposal to finish due to
a failed 'is_active()', we will need to recheck if we are active before
continuing once the callback is fired. This is mainly because we may have
finished our proposal, but a new Election may have been called and the
Paxos may not be active.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: MDSMonitor: Use the PaxosService is_{active,readable,writeable} functions

I.e., instead of using the Paxos ones.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: OSDMonitor: Use the PaxosService is_{readable,writeable,active} functions

I.e., instead of using the Paxos ones.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Paxos: cleaning up the proposal queue mechanism.

This commit removes the proposal lock, as we do not believe it is even
necessary (since we lock the Monitor, we are mostly protected by the
state change and the whole mechanism is based on a queue).

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: PaxosService: fix compilation error

note: to rebase.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Paxos: fix compilation error

note: to rebase.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
auth: cephx: KeyServer: add 'has_secrets()' function

We need this in the AuthMonitor to assess if we should encode a full version
of the KeyServer and submit it to the Paxos along with the incrementals.

Not performing such a check will lead us to an erroneous version, from which
we won't be able to recover.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: AuthMonitor: Don't always purpose full versions

We cannot purpose a full version if the KeyServer has no keys. This should
only happen when we start with a brand new store, and at the time we are
creating the first pending proposal.

However, committing a full version full of nothingness creates a big
problem, since we won't care for the incrementals if we have a full version
on store -- which we would then have, however corrupted it was.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Paxos: modify the criteria to propose queued proposals in finish_proposal()

This is nothing much. We just propose everytime we finish a proposal, as
long as we are in state ACTIVE and the head of the proposal queue has not
been proposed yet.

Also, we cannot assume the head of the queue has been proposed, since this
function is called from message handler functions, and we incur in the
possibility of being called more than once during the same proposal (fyi,
this usually happens from handle_accept(), which leads us to believe
that there is the possibility of a same message to be delivered twice).

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>