New Completion Path.

This change pulls up Marcus' new XioCompletion logic, with
further cleanups in the same direction.  With this change, the
message completion path no longer requires special claim behavior,
and we may wish to remove the claim method.

A cosmetic change is XioCompletionHook::cleanup_xio() is renamed
to release_msgs().  While doing this, go further, and revert the
change to hide XioConnection and XioCompletionHook from the
release thread, and instead destroy the entire msg sequence there,
rather than one xio_msg at a time.

Signed-off-by: Matt Benjamin <matt@linuxbox.com>

Conflicts:
	src/msg/XioPortal.h
Fix up xiopool to not use malloc at all, also fix up noreg pool
initialization to just use "generic" power of 2 sizes and
not look like fortran.

Signed-off-by: Marcus Watts <mdw@cohortfs.com>
Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Fix mempool create parameters.

Signed-off-by: Matt Benjamin <matt@linuxbox.com>

Conflicts:
	src/msg/XioMessenger.cc
Remove unused registered mpool.

Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Fix virutal buffer::xio_msg_buffer destruction.

It was not sufficient to override operator delete in the
xio_msg_buffer, this did not handle deletion of raw buffers
via (e.g.) buffer::ptr.

In addition, since XioMessenger does not require explicit CRC
and because std::map is expensive, move the CRC machinery from
buffer::raw to a new descendent buffer:raw_crc.

Signed-off-by: Matt Benjamin <matt@linuxbox.com>

Conflicts:
	src/common/buffer.cc
For the osd case, m isn't valid after the first reference countdown
on xiocompletion (because the buffer pointers were claimed).  So, must
capture and keep a connectionref sooner, so that the xio completion hook
can do a proper xiosubmit.

Signed-off-by: Marcus Watts <mdw@cohortfs.com>
Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Restore conn XioCompletionHook invariant.

This was missed in the previous XioSubmit refactoring
pass.

Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Restore MDataPing factory.

Signed-off-by: Matt Benjamin <matt@linuxbox.com>