Kfsio library: define net manager dispatcher interface in order to allow to invoke dispatcher methods at the pre-defined place in the net manager's event dispatch loop. At the moment the interface has a single method DispatchStart() invoked immediately after the event dispatch mutex (if one is used) is re-acquired. The interface is intended to be used in the cases where the invocation order matters and therefore ITimeout interface cannot be used.
Meta server: fix checkpoint write race that can result in processing an extra rpc(s) by "client" threads, from under DoFork() / PrepareCurrentThreadToFork() invocation. Use net manager's dispatcher interface, instead of ITimeout, to resolve (theoretical) ambiguity with prepare to fork in the main thread. Invoke prepare to fork method immediately after acquiring dispatch mutex in the main and the client threads, as the prepare to fork method can release and re-acquire the mutex while waiting on conditional variable. Handle theoretically possible sporadic wakeups on fork done conditional variable by maintaining fork counter and keep invoking wait until the counter changes. Implement checkpoint write correctness check by ensuring that the meta data mutation count remains 0 before starting checkpoint write, do not write checkpoint otherwise. Panic the parent if the mutation count isn't 0 immediately after forking the child process with DoFork().
The race appears to be rare. The workaround is to use prior checkpoint by hard linking it to the "latest", in the case if log replay fails. If the prior checkpoint isn't available, then it should be possible to recover by removing extraneous offending entry in the transaction log pointed by the checkpoint, along the transaction log segment checksum lines, or updating the log segment checksum after editing.