buffer: uninline, constify crc32c()

Signed-off-by: Sage Weil <sage@inktank.com>
common: fix Mutex, Cond no-copy declarations

Signed-off-by: Sage Weil <sage@inktank.com>
mds: fix MDCache::truncate_inode_finish() journal

we should add projected parent directory's context to the journal

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: re-integrate stray when link count >= 1

no reason not to rename inode out of the stray directory if the
inode's link count > 1

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: touch dentry bottom recursively

Deleted directory inode's dirfrags may contain some null dentries.
When touch_dentry_bottom() is called, also move these null dentries
to the tail of LRU.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: allow delay in evaluating stray

Add a new parameter 'delay' to MDCache::eval_stray(). If 'delay'
is true, MDCache::eval_stray() adds the stray dentry to a delayed
list. Delayed stray dentries are processed in MDCache::trim().

This change is required by later commit that evaluates stray when
reference to cache object is released.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: evaluate stray when releasing inode/dentry's reference

Current method to purge stray inode is call MDCache::maybe_eval_stray()
after releasing a reference to the stray inode/dentry. It's difficult
to make this method work correct, because there are so many places that
can release reference.

This patch solves the issue by calling MDCache::maybe_eval_stray()
in MDSCacheObject::put(). This avoids adding code that calls
MDCache::maybe_eval_stray() to each place that releases reference.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: remove unnecessary MDCache::maybe_eval_stray() calls

Now we call MDCache::maybe_eval_stray() in MDSCacheObject::put().
So there is no need to call MDCache::maybe_eval_stray() after
releasing inode/dentry's refernece.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: don't trim stray inode from the cache.

don't trim stray inode from the cache, purge it instead. This ensures
the stary directories at minimum size.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: avoid fetching backtrace when purging stray

we save old data pools in both inode_backtrace_t::old_pools and
inode_t::old_pools. We have the inode in the cache when purging
stray, so no need to fetch backtrace to find old data pools.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: remove dirfrags when purging stray directory

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
autotools: group test scripts in check_SCRIPTS

The check_SCRIPTS is added in Makefile-env.am to list the tests that do
not require compilation. The scripts listed in check-local and in the
TESTS variable use check_SCRIPTS instead.

The PYTHONPATH environment variable is added to Makefile-env.am and
includes the pybind directory so that python unit tests can load the
libraries from sources.

http://tracker.ceph.com/issues/6274 refs #6274

Reviewed-by: Roald J. van Loon <roaldvanloon@gmail.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
autotools: set noinst_PROGRAMS

to be used by unit test support programs that do not need to be installed

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: convenience function to convert commands to json

The get_command_descriptions is added to Monitor.h and contains
the code previously inlined in Monitor::handle_command to implement
the get_command_descriptions command. It is intended for tests to
convert command descriptions into json, including error cases.

http://tracker.ceph.com/issues/6274 refs #6274

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
mon: get_command_descriptions support program

The get_command_descriptions function is not designed to be tested in
C++ because all the validation happens in pybind/ceph_argparse.py. The
get_command_descriptions program is designed to be used by python unit
tests as a mean to get a JSON dump of the content of mon/MonCommands.h

      get_command_descriptions --all

      {"cmd000":{"sig":["pg","stat"],"help": ... "avail":"cli,rest"}}

It also provides a way to reproduce and keep track of past errors
( typos etc. ) to ensure the python validation keeps catching it.

      get_command_descriptions --pull585

Add /get_command_descriptions to .gitignore so that

      git ls-files --exclude-standard --others

does not see it which is required for
https://github.com/ceph/autobuild-ceph/blob/f018d220f2622a9fc8c86a31e1fa13263790c399/build-ceph.sh#L73

http://tracker.ceph.com/issues/6274 refs #6274

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
.gitignore gtags(1) generated files

Signed-off-by: Loic Dachary <loic@dachary.org>
pybind: unit tests for ceph_argparse::parse_json_funcsigs

Run parse_json_funcsigs against the builtin commands found
in mon/MonCommands.h. Although it does not provide for a full
validation, it will detect some degenerate cases.

It is expected to raise if a space is missing at the end of an option
specification ( see https://github.com/ceph/ceph/pull/585 ) and this
case is tested. New tests of the same kind can be added whenever an
undetected error is found. Ideally a validation function is implemented
and it would be updated ( with an associated test ) when a new
pathological case is found.

The json string given to parse_json_funcsigs is obtained from
the support program get_command_descriptions.

The python-nose dependencies are added to the build requirements in
debian/control and ceph.spec.in because make check should always be run
at built time.

http://tracker.ceph.com/issues/6274 refs #6274

Reviewed-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
mon: validate mon dump epoch as a positive integer

All other epochs are validated in the same way

http://tracker.ceph.com/issues/6274 refs #6274

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
pybind: catch EntityAddress missing /

If the / is missing in an EntityAddress, an ArgumentValid exception must
be raised so that it can be caught in the same way other argument
validation exceptions are.

http://tracker.ceph.com/issues/6274 refs #6274

Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
pybind: ceph_argparse unit tests foundations

The general idea is to have a series of commands, in the same order as
they show in mon/MonCommands.h, as if they were input to the ceph
client. For each command a valid combination is verified. And at least
one validation error is checked to produce a validation error. For
instance:

    ['pg', 'stat']

is a valid command and the validate_command function is expected to
return a value that is not None or {}. The command

    ['pg', 'stat', 'toomany' ]

is also given to validate_command to check that an error occurs when
an extra argument is given.

The TestArparse class implements a few methods to reduce the verbosity
of the tests. It does not provide many methods : only those that
significantly reduce the verbosity have been implemented. The drawback
of writing too many convenience methods is that they are more difficult
to read and maintain.

The signature dictionary is made a global variable so that
it is only extracted once for all classes. It is immutable.

http://tracker.ceph.com/issues/6274 refs #6274

Reviewed-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>