rgw: subuser creation fixes

Fixes: #8587
There were a couple of issues, one when trying to identify whether swift
user exists, we weren't using the correct swift id. The second problem
is that we relied on the gen_access flag in the swift case, where it
doesn't really need to apply.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
FileStore: Race condition during object delete is fixed

There was a race condition (hence OSD crash) between lfn_unlink
and lfn_open. The reason was FDCache lookup was called without
taking index lock from lfn_open. Lookup will increase reference
count and thus Clear will not be able to delete those FDs. FDs
will be leaked. The assert within FDCache clear was hitting
because of this.

Fixes: #9480

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
os/FileJournal: do not request sync while shutting down

FileStore calls should_commit_now() to determine whether it should
loop and do a second sync (among other things).  During shutdown, this
can force us into a livelock: the journal is shutting down, but the
sync_entry loop never completes and repeatedly syncs because the
journal is full.  Since the journal is otherwise stopped, no expire
happens and we never become unfull, and we're stuck.

This seems to be triggered semi-reliably by the ceph_objectstore_tool
import function.

Fix by not requesting a sync while shutting down.

Fixes: #9545
Signed-off-by: Sage Weil <sage@redhat.com>
ceph_test_rados_api_tier: add EvictSnap2 test case

Verify an evict doesn't create a snapdir object.  Reproduces #8629

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: do not clone or preserve snapdir on cache_evict

If we cache_evict a head in a cache pool, we need to prevent
make_writeable() from cloning the head and finish_ctx() from
preserving the snapdir object.

Fixes: #8629
Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
erasure-code: restore jerasure BlaumRoth default w

Changing from W=7 to W=6 by default for the BlaumRoth technique is
correct but introduces a regression. The content that was encoded with
the previous version cannot be read again. Although the prime(w+1)
constraint was not obeyed by W=7, the encoded content was useable and
should keep being readable.

The W=7 remains the default for backward compatibility and an exception
to the prime(w+1) check.

http://tracker.ceph.com/issues/9572 Fixes: #9572

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
PG: check full ratio again post-reservation

Otherwise, we might queue 30 pgs for backfill at 0.80 fullness
and then never check again filling the osd after pg 11.

Fixes: #9574
Backport: dumpling, firefly, giant
Signed-off-by: Samuel Just <sam.just@inktank.com>
osd: initialize purged_snap on backfill start; restart backfill if change

If we backfill a PG to a new OSD, we currently neglect to initialize
purged_snaps.  As a result, the first time the snaptrimmer runs it has to
churn through every deleted snap for all time, and to make matters worse
does so in one go with the PG lock held.  This leads to badness on any
cluster with a significant number of removed snaps that experiences
backfill.

Resolve this by initializing purged_snaps when we finish backfill.  The
backfill itself will clear out any stray snaps and ensure the object set
is in sync with purged_snaps.  Note that purged_snaps on the primary
that is driving backfill will not change during this period as the
snaptrimmer is not scheduled unless the PG is clean (which it won't be
during backfill).

If we by chance to interrupt backfill, go clean with other OSDs,
purge snaps, and then let this OSD rejoin, we will either restart
backfill (non-contiguous log) or the log will include the result of
the snap trim (the events that remove the trimmed snap).

Fixes: #9487
Backfill: firefly, dumpling
Signed-off-by: Sage Weil <sage@redhat.com>
ReplicatedPG: don't move on to the next snap immediately

If we have a bunch of trimmed snaps for which we have no
objects, we'll spin for a long time.  Instead, requeue.

Fixes: #9487
Backport: dumpling, firefly, giant
Reviewed-by: Sage Weil <sage@redhat.com>
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: clean out completed trimmed objects as we go

Also, explicitely maintain a max number of concurrently trimming
objects.

Fixes: 9113
Backport: dumpling, firefly, giant
Signed-off-by: Samuel Just <sam.just@inktank.com>
osdc/Objecter: only post_rx_buffer if no op timeout

If we post an rx buffer and there is a timeout, the revocation can happen
while the reader has consumed the buffers but before it has decoded and
constructed the message.  In particular, we calculate a crc32c over the
data portion of the message after we've taken the buffers and dropped the
lock.

Instead of fixing this race (for example, by reverifying rx_buffers under
the lock while calculating the crc.. bleh), just skip the rx buffer
optimization entirely when a timeout is present.

Note that this doesn't cover the op_cancel() paths, but none of those users
provide static buffers to read into.

Fixes: #9582
Backport: firefly, dumpling
Signed-off-by: Sage Weil <sage@redhat.com>
ceph-disk: add Scientific Linux as a Redhat clone

Scientific Linux is a RHEL clone and needs to use partx.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
(cherry picked from commit 5ca7ea5b531f28b3c87161d53aee907d6957cfc2)
qa/workunits/cephtool/test.sh: fix thrash (ultimate)

Keep the osd trash test to ensure it is a valid command but make it a
noop by giving it a zero argument (meaning thrash 0 OSD maps).

Remove the loops that were added after the command in an attempt to wait
for the cluster to recover and not pollute the rest of the tests. Actual
testing of osd thrash would require a dedicated cluster because it the
side effects are random and it is unnecessarily difficult to ensure they
are finished.

http://tracker.ceph.com/issues/9620 Fixes: #9620

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Merge pull request #2594 from dachary/wip-9620-test-mon-thrash

qa/workunits/cephtool/test.sh: fix thrash (ultimate)

Reviewed-by: Sage Weil <sage@redhat.com>
Merge pull request #2598 from ceph/wip-9582

librados: fix other timeout segfault

Reviewed-by: Greg Farnum <greg@inktank.com>
debian: python-flask is needed by ceph, not python-ceph

It's used by ceph-rest-api which is in the 'ceph' (server) package.

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
ceph.spec: fix python-flask dependency

This is needed by ceph-rest-api, which is in ceph.rpm; it's not related to
python-ceph (except that ceph-rest-api happens to require that too).

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
Merge pull request #2510 from somnathr/wip-obj-delete-fix

FileStore: Race condition during object delete is fixed

Reviewed-by: Samuel Just <sam.just@inktank.com>
Merge pull request #2550 from ceph/wip-8629

osd: fix cache_evict vs make_writeable/finish_ctx snapdir bug #8629

Reviewed-by: Samuel Just <sam.just@inktank.com>
Merge pull request #2549 from ceph/wip-9545

os/FileJournal: do not request sync while shutting down

Reviewed-by: Samuel Just <sam.just@inktank.com>