mon/MDSMonitor: don't reset incarnation when creating newfs

Fixes: #6279

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Merge pull request #589 from ceph/wip-6279

mon/MDSMonitor: don't reset incarnation when creating newfs
Reviewed-by: Greg Farnum <greg@inktank.com>
osd/OSD.cc: Use MIN() so that we don't exceed osd_recovery_max_active

Caused by 944f3b73531af791c90f0f061280160003545c63

Fixes: #6291

Backport: dumpling

Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
osd/ReplicatedPG.cc: Verify that recovery is truly complete

Backportable change to insure that even if no new ops started or
are running that indeed recovery is complete.  Prevents some
error condition or unforseen code path from crashing an osd.

Backport: dumpling, cuttlefish

Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktank.com>
Merge branch 'wip-6226'
make: add tmalloc lib dep in tcmalloc guard

Fixes --without-tcmalloc on boxes without libtcmalloc.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Merge pull request #594 from ceph/fix-no-tcmalloc-build

make: add tmalloc lib dep in tcmalloc guard

Reviewed-by: Sage Weil <sage@inktank.com>
Merge pull request #538 from dachary/wip-5879

jerasure-1.2A plugin

Reviewed-by: Samuel Just <sam.just@inktank.com>
autotools: group test scripts in check_SCRIPTS

The check_SCRIPTS is added in Makefile-env.am to list the tests that do
not require compilation. The scripts listed in check-local and in the
TESTS variable use check_SCRIPTS instead.

The PYTHONPATH environment variable is added to Makefile-env.am and
includes the pybind directory so that python unit tests can load the
libraries from sources.

http://tracker.ceph.com/issues/6274 refs #6274

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: convenience function to convert commands to json

The get_command_descriptions is added to Monitor.h and contains
the code previously inlined in Monitor::handle_command to implement
the get_command_descriptions command. It is intended for tests to
convert command descriptions into json, including error cases.

http://tracker.ceph.com/issues/6274 refs #6274

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: minimal test for get_command_descriptions

The get_command_descriptions is not designed to be tested in C++ because
all the validation happens in ceph_argparse.py. Each test function calls
get_command_descriptions exactly once and does a trivial check on the
generated output. It does not really do more that code coverage.

With --debug-mon=20 each test function is expected to write the json
string to the standard output. The unit test can then be used as a
helper by other tests to generate various degenerated cases for test
purposes.

For instance:

  unittest_get_command_descriptions --gtest_filter=*.pull585 \
      --debug-mon=20

will generate a bogus json replicating the problem which was fixed
by https://github.com/ceph/ceph/pull/585

http://tracker.ceph.com/issues/6274 refs #6274

Signed-off-by: Loic Dachary <loic@dachary.org>
pybind: unit tests for ceph_argparse::parse_json_funcsigs

Run parse_json_funcsigs against the builtin commands found
in mon/MonCommands.h. Although it does not provide for a full
validation, it will detect some degenerate cases.

It is expected to raise if a space is missing at the end of an option
specification ( see https://github.com/ceph/ceph/pull/585 ) and this
case is tested. New tests of the same kind can be added whenever an
undetected error is found. Ideally a validation function is implemented
and it would be updated ( with an associated test ) when a new
pathological cases is found.

The json string given to parse_json_funcsigs is obtained from
designated functions in unittest_get_command_descriptions.

The python-nose dependencies are added to the build requirements in
debian/control and ceph.spec.in because make check should always be run
at built time.

http://tracker.ceph.com/issues/6274 refs #6274

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: osd pool create samples for Q&A

Including key=value with bad chars expecting to fail.

http://tracker.ceph.com/issues/6274 refs #6274

Signed-off-by: Loic Dachary <loic@dachary.org>
mon: validate mon dump epoch as a positive integer

All other epochs are validated in the same way

http://tracker.ceph.com/issues/6274 refs #6274

Signed-off-by: Loic Dachary <loic@dachary.org>
pybind: catch EntityAddress missing /

If the / is missing in an EntityAddress, an ArgumentValid exception must
be raised so that it can be caught in the same way other argument
validation exceptions are.

http://tracker.ceph.com/issues/6274 refs #6274

Signed-off-by: Loic Dachary <loic@dachary.org>
pybind: ceph_argparse unit tests foundations

The general idea is to have a series of commands, in the same order as
they show in mon/MonCommands.h, as if they were input to the ceph
client. For each command a valid combination is verified. And at least
one validation error is checked to produce a validation error. For
instance:

    ['pg', 'stat']

is a valid command and the validate_command function is expected to
return a value that is not None or {}. The command

    ['pg', 'stat', 'toomany' ]

is also given to validate_command to check that an error occurs when
an extra argument is given.

The TestArparse class implements a few methods to reduce the verbosity
of the tests. It does not provide many methods : only those that
significantly reduce the verbosity have been implemented. The drawback
of writing too many convenience methods is that they are more difficult
to read and maintain.

The signature dictionary is made a global variable so that
it is only extracted once for all classes. It is immutable.

http://tracker.ceph.com/issues/6274 refs #6274

Signed-off-by: Loic Dachary <loic@dachary.org>
ceph_argparse: unit tests for validate_command pg

http://tracker.ceph.com/issues/6274 refs #6274

Signed-off-by: Loic Dachary <loic@dachary.org>
ceph_argparse: unit tests for validate_command auth

http://tracker.ceph.com/issues/6274 refs #6274

Signed-off-by: Loic Dachary <loic@dachary.org>
ceph_argparse: unit tests for misc validate_command

Contrary to all other classes, this series of command ( Monitor ) does
not have a common prefix.

http://tracker.ceph.com/issues/6274 refs #6274

Signed-off-by: Loic Dachary <loic@dachary.org>
ceph_argparse: unit tests for validate_command mds

http://tracker.ceph.com/issues/6274 refs #6274

Signed-off-by: Loic Dachary <loic@dachary.org>