Add safe-sharing to buffer::list and buffer::ptr.

Formerly known as "strong claim," this change introduces a new
shareable() property to ceph::buffer::raw objects, which can be
used to prevent unplanned sharing of buffers.

The default value of shareable() is true, but derived classes may
override this.  For example, Accelio message buffers may point to
registered or pool-allocated memory which should not be held for
long periods.

Cloning of non-shareable buffers is now the unmarked case, so is
performed automatically in the buffer::list copy constructor.
Low-level operations should allow explicit volatile sharing,
so copying a buffer::list:iterator or a buffer::ptr works as normal.

For explicit sharing of "non-shareable" buffers, a new boolean is added to
the public claim_* methods, and a buffer::list::share method is also
added, to share an entire sequence.

Signed-off-by: Vu Pham <vu@mellanox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Restore zero-copy buffers in OSD fast path.

This change restores volatile sharing semantics in the Message
decode path, and also in the OSD write path for FileStore/FileJournal.

This can be verified with a breakpoint set at the clone/COW case in
buffer::ptr::clone_nonshareable(currently buffer.cc:690).

Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Include common/likely.h in buffer.h

Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Add diagnostic unsharable buffer.

Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Add unittests for clone_unshareable.

Checks three assertions:

1. a bufferlist shared with buffer::list::share yields shared
raw memory

2. a bufferlist copied with buffer::list::list(const list&) yields
shared raw memory

3. an unsharable bufferlist (bufferlist with unshareable segments)
copied with buffer::list::list(const list&) yields unshared raw
memory

Signed-off-by: Matt Benjamin <matt@cohortfs.com>
xio: fix for size in pool stats

Signed-off-by: Casey Bodley <casey@linuxbox.com>
Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Add in/outbound msg counters to XioPoolStats.

Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Pool stats are easier to read.

Pool stats are easier to read, and if enabled, print on session
teardown. This is a convenient time to view stats, and with a small

Signed-off-by: Vu Pham <vu@mellanox.com>
Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Make pool stats counters atomic.

This appears necessary to avoid false positives.

Signed-off-by: Vu Pham <vu@mellanox.com>
Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Track requests using hook ctor/dtor.

Lockless, portal thread provides atomicity.

Signed-off-by: Vu Pham <vu@mellanox.com>
Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Adapt to recent changes on Accelio for_next

* Accelio options now of opaque type
* on_msg_err with extra direction param
* RDMA behavior now governed by 2 new options
    XIO_OPTNAME_MAX_INLINE_DATA
    XIO_OPTNAME_MAX_INLINE_HEADER
* Separated send and recv queue depth

Signed-off-by: Vu Pham <vu@mellanox.com>
Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>
xio_messenger: Change xio optname queue depth msgs

* Set 16k threshold to rdma buffers instead of send
* Change xio optname for queue depth msgs
  XIO_OPTNAME_SND/RCV_QUEUE_DEPTH_MSGS

Signed-off-by: Vu Pham <vu@mellanox.com>
Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>
xio_messenger: Protect Accelio queue depth.

(Minimal send flow control.)

The guard is per xio_connection, and considering batches.
Increment happens only if xio_send_msg succeeded, decrement in
on_ms_ow_send_complete and on_msg_error.  Note that we don't need
atomics because counters are touched only in the correct portal
thread.

Signed-off-by: Vu Pham <vu@mellanox.com>
Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Fix CMake build for Hammer.

Build mds/Beacon.cc, mds/MDSAuthCaps.cc, common/Readahead.cc,
EventEpoll.cc.

Also moves msg/XioPool.cc to msg/xio (missed).

Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Find XioMsg.h in msg/xio

Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Fixup int_types.h.

Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>

Conflicts:
	src/include/rados/librados.h

Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Don't use __cplusplus to mean !__KERNEL__

Of course, this is Linux-centric.

Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Null tracepoint macro when !WITH_LTTNG.

Signed-off-by: Matt Benjamin <matt@linuxbox.com>
Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Find XioMessenger.h and QueueStrategy.h in msg/xio (tests).

Signed-off-by: Matt Benjamin <matt@cohortfs.com>
Fix name and linkage of libec_lrc.

This library was renamed, and has new dependencies.

Signed-off-by: Matt Benjamin <matt@cohortfs.com>