Mutex: Instrument Mutex with perfcouter for Lock() wait

Signed-off-by: Samuel Just <sam.just@inktank.com>
test/bench: small io benchmarker

Precreates objects and does writes to random offsets within
random objects.

Includes rados, filestore, and vanilla fs variants

Signed-off-by: Samuel Just <sam.just@inktank.com>
test/bench: add tp bench

Signed-off-by: Samuel Just <sam.just@inktank.com>
FileStore: next_finish is not used

Signed-off-by: Samuel Just <sam.just@inktank.com>
JournalingFileStore: pass -1 as the alignment if unimportant

Previously, data_align began at 0 and remained that way if no
transaction contained a large data segment.  This 0 was propagated
to prepare_single_write, which padded out most of a page to ensure
that the bl started with 0 alignment.  Passing -1 will ensure that
we don't prepad these small segments.

Signed-off-by: Samuel Just <sam.just@inktank.com>
FileStore: remove trigger_commit

This is no longer used.

Signed-off-by: Samuel Just <sam.just@inktank.com>
JournalingObjectStore: whitespace fix

Signed-off-by: Samuel Just <sam.just@inktank.com>
JournalingObjectStore: remove force_commit, no longer needed

Signed-off-by: Samuel Just <sam.just@inktank.com>
JournalingFileStore: create submit_manager to order op submission

Previously, we ensured op ordering by queueing for journal and
the op queue under the journal lock.  All that is required is
that obtaining an op sequence, queueing for journal, and
(for parallel) queueing for application to the fs are done
atomically.  To that end, submit_manager now handles op submission.

Signed-off-by: Samuel Just <sam.just@inktank.com>
JournalingFileStore: move apply/commit sequencing to apply_manager

syncing the filestore requires a stable commit point (i.e., all ops
up to applied_seq must have been applied).  Previously, we used
journal_lock to atomically block new applies while waiting for
the remaining ones to finish.  This creates unnecessary contention.
We now use apply_manager to manage that state atomically with its
own lock.

Signed-off-by: Samuel Just <sam.just@inktank.com>
perf_counters: add dec()

Signed-off-by: Samuel Just <sam.just@inktank.com>
FileStore: don't lock op_tp in queue_op

Neither caller of queue_op can race.
1) in queue_transactions, already under submit lock
2) in _journaled_ahead, journal finisher is single threaded

Signed-off-by: Samuel Just <sam.just@inktank.com>
FileStore: add op_throttle_lock

Avoid using op_tp lock for the op throttle.

Signed-off-by: Samuel Just <sam.just@inktank.com>
os: instrument submit lock, apply lock, queue_lock, write_lock

Adds Mutex perfcounter tracking to mutexes of interest.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Throttle: reduce lock hold periods

Previously, we tended to dump a lot of log output under
the Throttle lock.  The log level for most log statements
has been reduced to 10.

Additionally, count and max are now atomic_t and can be
read without the Throttle lock.

Finally, most of the perf counter manipulations have been
moved outside of the lock.

Signed-off-by: Samuel Just <sam.just@inktank.com>
FileJournal: break writeq locking from queue_lock

This prevents the relatively long process of queueing
finishers from preventing op submission.

In submit_entry, we no longer check for full before placing
the write in the writeq, committed_thru should work anyway,
and we don't want to grab the required lock.

Signed-off-by: Samuel Just <sam.just@inktank.com>
FileJournal: write_cond is not used

Signed-off-by: Samuel Just <sam.just@inktank.com>
FileJournal: rename queue_lock to finisher_lock

Signed-off-by: Samuel Just <sam.just@inktank.com>
Finisher: add perf counter for queue len

Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: actually delay op for backfill_pos

3f952afe5da644b30015fead8e3d42a129b59989 neglected to
actually delay the op in ReplicatedPG::do_op.

Signed-off-by: Samuel Just <sam.just@inktank.com>