mon: Monitor: force timecheck cleanup on finish_election()

Fixes: #3854

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Elector: reset the acked leader when the election finishes and we lost

Failure to do so will mean that we will always ack the same leader during
an election started by another monitor.  This had been working so far
because we were still acking the existing leader if he was supposed to
still be the leader; or we were acking a new potentially leader; or we
would eventually fall behind on an election and start a new election
ourselves, thus resetting the previously acked leader.  While this wasn't
something that mattered much until now, the timechecks code stumbled into
this tiny issue and was failing hard at completing a round because there
wouldn't be a reset before the election started -- timechecks are bound
to election epochs.

Fixes: #3854

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
mon: Monitor: rework timecheck code to clarify logic boundaries

The initial timecheck implementation relied on a cleanup function to
clean the state each time we changed epochs (or we got out of quorum),
and we would have to clean up the state in-between rounds in a potentially
confusing way some time down the line.

This patch creates logic boundaries in the code flow, making it clear
where we set up or clear the state when we start or finish an epoch, and
where we set up or clear the round state in-between rounds.  It also
allowed for some other changes in behavior, such as when we set-up the
timecheck event, or when we cancel it.  Despite the slight increase in
size, the mechanism just got more easily understandable than it was before.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
Merge remote-tracking branch 'gh/wip-timecheck

Reviewed-by: Sage Weil <sage@inktank.com>