ObjectCacher: optionally make writex always non-blocking

Add a callback argument to writex, and a finisher to run the
callbacks. Move the check for dirty+tx > max_dirty into a helper that
can be called from a wrapper around the callbacks from writex, or from
the current place in _wait_for_write().

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
librbd: make aio_writes to the cache always non-blocking by default

When the ObjectCacher's writex blocks, it affects the thread requesting
the aio, which can cause starvation for other I/O when used by QEMU.

Preserve the old behavior via a config option in case this has any
bad side-effects, like too much memory usage under heavy write loads.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
librbd: add an async flush

not finished, just the C interface for now

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>