Revert "librbd: remove limit on number of objects in the cache"

Disabling this limit causes too much memory usage in some
workloads.

This reverts commit 0559d31db29ea83bdb6cec72b830d16b44e3cd35.
Revert "ObjectCacher: remove unused target/max setters"

This reverts commit e1a49e5386c3ed4a6bc4870f01630349cb04d749.
common: -- support for env_to_vec

When CEPH_ARGS is parsed each side of the -- must be appended to the
corresponding side of the existing argument list. For instance when

   -a -b -- foo bar

is merged with a CEPH_ARGS containing

   -c -d -- frob nitz

it must become

   -a -b -c -d -- foo bar frob nitz

http://tracker.ceph.com/issues/7578 fixes #7578

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: add SETALLOCHINT operation

This is primarily for librbd/krbd's benefit and is supposed to combat
fragmentation:

"... knowing that rbd images have a 4m size, librbd can pass a hint
that will let the osd do the xfs allocation size ioctl on new files so
that they are allocated in 1m or 4m chunks.  We've seen cases where
users with rbd workloads have very high levels of fragmentation in xfs
and this would mitigate that and probably have a pretty nice
performance benefit."

SETALLOCHINT is considered advisory, so our backwards compatibility
mechanism here is to set FAILOK flag for all SETALLOCHINT ops.

xfs is hooked up in the subsequent commits.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
FileStore: refactor FS detection checks a bit

Refactor FS detection checks in FileStore::_detect_fs() so that they
look the same as the ones in FileStore::mkfs().  This is in preparation
for adding XfsFileStoreBackend class.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
FileStore: introduce XfsFileStoreBackend class

Introduce XfsFileStoreBackend class, currently the only filestore
backend implementing SETALLOCHINT op.  This commit adds a build-time
dependency on libxfs as xfs-specific ioctl (XFS_IOC_FSSETXATTR /
XFS_XFLAG_EXTSIZE) is used to implement the new set_alloc_hint()
method.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
FileStore: add option to cap alloc hint size

Add a new config option, filestore_max_alloc_hint_size, to cap
SETALLOCHINT hint size.  The unit is a byte, the default value is
1 megabyte.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
librbd: prefix rbd writes with CEPH_OSD_OP_SETALLOCHINT osd op

In an effort to reduce fragmentation, prefix every rbd write with
a CEPH_OSD_OP_SETALLOCHINT osd op with an expected_write_size value set
to the object size (1 << order).  Backwards compatibility is taken care
of on the osd side.

"The CEPH_OSD_OP_SETALLOCHINT hint is durable, in that it's enough to
do it once.  The reason every rbd write is prefixed is that rbd doesn't
explicitly create objects and relies on writes creating them
implicitly, so there is no place to stick a single hint op into.  To
get around that we decided to prefix every rbd write with a hint (just
like write and setattr ops, hint op will create an object implicitly if
it doesn't exist)."

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
Merge branch 'wip-hint' into firefly

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Merge pull request #1329 from ceph/wip-doc-cache-pool

fix up docs for cache pools

These are definitely not complete, but they bring a few of the commands more up to date with current syntax.

Reviewed-by: Greg Farnum <greg@inktank.com>
msg: Make SimpleMessenger shutdown safer

...in the case where no dispatchers were ever added, previously
we would throw an assertion in wait().

Signed-off-by: John Spray <john.spray@inktank.com>
osd: be a bit more verbose on startup

load_pgs can take a while and it is nice to know what ceph-osd is doing
without cranking up logging.

Did a quick audit of dout(1)'s and making this the default.  This lets us
see basic OSD state changes (load_pgs, boot, active) at the default level.

At this point all osd state changes should be logged at level 1.

Signed-off-by: Sage Weil <sage@inktank.com>
Merge pull request #1341 from ceph/wip-osd-status

osd: 'status' admin socket command

Reviewed-by: Loic Dachary <loic@dachary.org>
OSD,config_opts: log osd state changes at level 0 instead

Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge pull request #1356 from ceph/wip-7458

osd: stray pg ref on shutdown

Reviewed-by: Samuel Just <sam.just@inktank.com>
TestPGLog: not worth maintaining tests of assert behavior

Signed-off-by: Samuel Just <sam.just@inktank.com>
TestPGLog: ignore merge_old_entry return value

No callers use the merge_old_entry return value.  _merge_divergent_entries
won't have one.

Signed-off-by: Samuel Just <sam.just@inktank.com>
TestPGLog:rewind_divergent_log: set prior_version for delete

Signed-off-by: Samuel Just <sam.just@inktank.com>
TestPGLog::merge_old_entry: we no longer use merge_old_entry this way

This needs to be replaced with an equivalent test of
_merge_object_divergent_entries.

Signed-off-by: Samuel Just <sam.just@inktank.com>
TestPGLog::merge_old_entry: ne.version cannot be oe.version

Otherwise, it would not be divergent!

Signed-off-by: Samuel Just <sam.just@inktank.com>