add calamari to the api/index section

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>
PG: wait until we've build the missing set to discover_all_missing

Fixes: #9179
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
PG::init: clear rollback info for backfill as well

Otherwise, we won't remove the old rollback objects from a resurrected pg.  In
rare cases, this can cause us to get an EEXIST if we happen to reuse the same
rename id on the same object in a subsequent interval.

Fixes: #9293
Related to: 8346e10755027e982f26bab4642334fd91cc31aa
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
tests: flush logs before grepping them

Otherwise the test races with the daemon writing the logs and can
sometime fail.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
rbd should use write-back when caching is enabled

librbd now defaults to write-through until the first flush
is received.  For rbd, force the use of write-through.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
KeyValueStore: Fix upgrade from non-exist superblock OSD version error

If user upgrade OSD from Firefly or lower than 0.84, it won't get valid
keyvaluestore backend.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
ObjectStore Test: Add test for simulating scrub behavior

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
KeyValueStore: Fix scrub causing inconsistence state

Fix #8589
Backport #Firefly
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
packaging: add all erasure code plugins to RPM packages

It means distributing a few plugins that are only used for unit testing
but it does not use much disk space and this is otherwise harmless.

Explicitly listing which plugins are to be installed is problematic
because some of them (isa for now and maybe more later) are not
available for all architectures. Properly maintaining the list of
plugins to install would therefore mean exactly matching which
architecture has which plugins.

http://tracker.ceph.com/issues/9381 Fixes: #9381

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Merge pull request #2429 from yuyuyu101/kvstore-fix-superblock

KeyValueStore: Fix upgrade from non-exist superblock OSD version error

Reviewed-by: Sage Weil <sage@redhat.com>
Merge pull request #2430 from yuyuyu101/kvstore-scrub-inconsistence

Kvstore scrub inconsistence

Reviewed-by: Sage Weil <sage@redhat.com>
Merge pull request #2435 from athanatos/wip-9179

PG: wait until we've build the missing set to discover_all_missing

Reviewed-by: Sage Weil <sage@redhat.com>
Merge pull request #2436 from athanatos/wip-9293

PG::init: clear rollback info for backfill as well
ReplicatedPG: create max hitset size

Otherwise, hit_set_create could create an unbounded size hitset
object.

Fixes: #9339
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
Revert "osd/ReplicatedPG: avoid dereferencing iterator at end()"

This reverts commit ded1cf4a93d15ca3be312d296c3ab956109587e3.
Revert "ReplicatedPG:start_flush send a second delete"

This reverts commit a525bf3190a712fdceb018267a0aa08e90c44e62.

Conflicts:
	src/osd/ReplicatedPG.cc
Merge pull request #2256 from ceph/wip-9005

doc: add calamari to the api/index section.

Reviewed-by: John Wilkins <jowilkin@redhat.com>
ReplicatedPG:start_flush send a second delete

Suppose we start with the following in the cache pool:

30:[29,21,20,15,10,4]:[22(21), 15(15,10), 4(4)]+head

The object doesn't exist at 29 or 20.

First, we flush 4 leaving the backing pool with:

3:[]+head

Then, we begin to flush 15 with a delete with snapc 4:[4] leaving the
backing pool with:

4:[4]:[4(4)]

Then, we finish flushing 15 with snapc 9:[4] with leaving the backing
pool with:

9:[4]:[4(4)]+head

Next, snaps 10 and 15 are removed causing clone 10 to be removed leaving
the cache with:

30:[29,21,20,4]:[22(21),4(4)]+head

We next begin to flush 22 by sending a delete with snapc 4(4) since
prev_snapc is 4 <---------- here is the bug

The backing pool ignores this request since 4 < 9 (ORDERSNAP) leaving it
with:

9:[4]:[4(4)]

Then, we complete flushing 22 with snapc 19:[4] leaving the backing pool
with:

19:[4]:[4(4)]+head

Then, we begin to flush head by deleting with snapc 22:[21,20,4] leaving
the backing pool with:

22[21,20,4]:[22(21,20), 4(4)]

Finally, we flush head leaving the backing pool with:

30:[29,21,20,4]:[22(21*,20*),4(4)]+head

When we go to flush clone 22, all we know is that 22 is dirty, has snaps
[21], and 4 is clean. As part of flushing 22, we need to do two things:
1) Ensure that the current head is cloned as cloneid 4 with snaps [4] by
sending a delete at snapc 4:[4].
2) Flush the data at snap sequence < 21 by sending a copyfrom with snapc
20:[20,4].

Unfortunately, it is possible that 1, 1&2, or 1 and part of the flush
process for some other now non-existent clone have already been
performed.  Because of that, between 1) and 2), we need to send
a second delete ensuring that the object does not exist at 20.

Fixes: #9054
Backport: firefly
Related: 66c7439ea0888777b5cfc08bcb0fbd7bfd8653c3
Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge pull request #2439 from athanatos/wip-9054

Wip 9054

Reviewed-by: Sage Weil <sage@redhat.com>
ceph_test_rados_api_*: expose nspace

Signed-off-by: Sage Weil <sage@redhat.com>