osd: log when scrub,deep scrub, repair starts

So it can be matched, in the logs, with the log messages announcing it
is complete.

http://tracker.ceph.com/issues/9817 Fixes: #9817

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
osd/ReplicatedPG: flush snaps immediately

Snaps are immutable, so there is no point in delaying flush and hoping
to capture additional updates first.

Signed-off-by: Sage Weil <sage@redhat.com>
OSD: batch trim for pg_log

Signed-off-by: Xinze Chi <xmdxcxz@gmail.com>
osd: Check filter ops for pgls and pgnls

Fixes: #9439

Signed-off-by: David Zafman <dzafman@redhat.com>
doc: update debian compilation dependencies

Using the content of debian/control.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
ceph-disk: encapsulate partprobe / partx calls

Add the update_partition function to reduce code duplication.
The action is made an argument although it always is -a because it will
be -d when deleting a partition.

Use the update_partition function in prepare_journal_dev

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
ceph-disk: use update_partition in prepare_dev and main_prepare

In the case of prepare_dev the partx alternative was missing and is not
added because update_partition does it.

http://tracker.ceph.com/issues/9721 Fixes: #9721

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
ceph-disk: run partprobe after zap

Not running partprobe after zapping a device can lead to the following:

* ceph-disk prepare /dev/loop2
* links are created in /dev/disk/by-partuuid
* ceph-disk zap /dev/loop2
* links are not removed from /dev/disk/by-partuuid
* ceph-disk prepare /dev/loop2
* some links are not created in /dev/disk/by-partuuid

This is assuming there is a bug in the way udev events are handled by
the operating system.

http://tracker.ceph.com/issues/9665 Fixes: #9665

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
ceph-disk: implement init=none for block devices

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
tests: helper to run unit / function tests in docker

For instance

   test/docker-test.sh --os-type ubuntu --os-version 14.04 \
        test/ceph-disk.sh

runs test/ceph-disk.sh in a ubuntu 14.04 docker container. Once the
container is populated and ceph compiled, running a test script roughly
requires entering the container and running make TESTS=tests/foo.sh check

* docker build ceph-ubuntu-14.04 using ubuntu.dockerfile as a Dockerfile
* it will run apt-get install ceph compilation / run dependencies
* git clone the-local-clone ceph-ubuntu-14.04
* docker run ceph-ubuntu-14.04 make -j4 in the ceph-ubuntu-14.04 clone
* docker run test/ceph-disk.sh

test/docker-test.sh is the command line interface for
test/docker-test-helper.sh which can be invoked from shell scripts.
test/ubuntu.dockerfile and test/ubuntu.dockerfile are regular
Dockerfiles which allow substitution of environment variables.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
ceph-disk: test prepare / activate on a device

This indirectly tests that partprobe is called after zap because it
would fail to map the partitions to /dev/disk/by-partuuid otherwise.

It also indirectly test the implementation of init=none when using a
block device because the test would fail to put an object into the rbd
pool using the device otherwise.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
autotools: add --enable-docker

Docker based tests should be explicit instead of auto-detected. It is
good that they do not run if docker is not available. It would be bad if
they run when the developer does not expect them to create docker
containers.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Merge pull request #2648 from dachary/wip-9665-ceph-disk-partprobe

ceph-disk: run partprobe/partx after zap and data partition creation

Reviewed-by: Sage Weil <sage@redhat.com>
ceph_objectstore_tool: Clear ...INCOMPAT_SHARDS from feature if exporting replicated pg

Don't require importing OSD to have shards feature for replicated pg

http://tracker.ceph.com/issues/10077 Fixes: #10077

Signed-off-by: David Zafman <dzafman@redhat.com>
ceph_objectstore_tool: Add utility routine get_osdmap()

Signed-off-by: David Zafman <dzafman@redhat.com>
ceph_objectstore_tool: Add feature called set-allow-sharded-objects

Uses --op set-allow-sharded-objects option
This operation will be rejected if on the target OSD's osdmap there is
    at least one OSD which does not support ERASURE CODES.
Prompt the user that they could import if sharded state allowed
Prompt the user to use new feature if sharded state found inconsistent

Fixes: #10077

Signed-off-by: David Zafman <dzafman@redhat.com>
KeyValueStore: Remove assert for collection_getattr method

Merge 22b6c1a5317504e87042ffeff2b2d0d0b71cac5e introduce
5dc990c1ca821620993417ef4acc51a2b0a34606 commit which will lookup "remove"
attr. It shouldn't exist for most of cases but KeyValueStore will crash
if attr isn't exist. It seemed that this assert isn't correct.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
StoreTest: Add collection_getattr(s) tests

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
KeyValueStore: Return -ENODATA if collection_getattr is nonexist

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
KeyValueStore: Fix collection_getattrs problem when zero-length attr

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>