rgw: subuser creation fixes

Fixes: #8587
There were a couple of issues, one when trying to identify whether swift
user exists, we weren't using the correct swift id. The second problem
is that we relied on the gen_access flag in the swift case, where it
doesn't really need to apply.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Cache tiering: do not allow flush/evict when there are requests pending

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
OpTracker: Sharding logic is implemented to improve performance

Enabling OpTracking is degrading performance significantly. The
main reason for this is the contention in the ops_in_flight_lock
during registration/unregistration. Sharding this lock and hence
the xlist (storing the ops) helped to boost the performance.
Sharding is been done based on the op seq number % num_shards.
atomic64_t is used as data type to protect the global sequence
number of OpTracker. Introduced a lock in the OpHistory to
protect its data structure as ealier it was protected by global
ops_in_flight_lock. This is only accessed during unregistration
and thus not contended.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: Removed _dump_op_descriptor_unlocked call for log level < 5

_dump_op_descriptor_unlocked call within the function _mark_event()
was invoked unnecessarily every time for all log levels. This was
degrading performance significantly! The stream is only used during
log level 5 and thus the _dump_op_descriptor_unlocked() is only called
now if user sets the log level to 5 and beyond.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: The TrackedOp::events list was not protected while dumping ops

OpTracker::Mutex::lock is there to protect the event list and it is
only used in the mark_event while pushing the events in the list.
It is not used while dumping the contents of the events list.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: Race condition removed while dumping ops through admin socket

OSD was crashing due to a race condition while IO was going on and
user wants to dump in flight ops. This was happening because Message
data and payloads are removed before the op itself is removed from
the in flight list. Calling op->_unregistered() after removing
the op from the in flight list fixes the issue.

Fixes: #9384

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
ReplicatedPG: cancel cb on blacklisted watcher

Fixes: #8315
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
osd_types: fix pg_log_t tail dump

Fixes: #9453
Signed-off-by: Samuel Just <sam.just@inktank.com>
DBObjectMap: restructure map_header_in_use locking

Tieing map_header_in_use to a _Header is a mistake since ownership of
the map_header_in_use can move (clone).  Instead, grab the
map_header_in_use entry at the top of each call chain and release at the
end.  To facilitate that, we introduce a MapHeaderLock object and
require that it be passed to the various map header manipulation
methods.

Fixes: #9326
Signed-off-by: Samuel Just <sam.just@inktank.com>
osdc/Objecter: cancel timeout before clearing op->session

The C_CancelOp path assumes op->session != NULL.  Cancel that op before
we clear it.  This fixes a crash like

#0  pthread_rwlock_wrlock () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_rwlock_wrlock.S:39
#1  0x00007fc82690a4b1 in RWLock::get_write (this=0x18, lockdep=<optimized out>) at ./common/RWLock.h:88
#2  0x00007fc8268f4d79 in Objecter::op_cancel (this=0x1f61830, s=0x0, tid=0, r=-110) at osdc/Objecter.cc:1850
#3  0x00007fc8268ba449 in Context::complete (this=0x1f68c20, r=<optimized out>) at ./include/Context.h:64
#4  0x00007fc8269769aa in RWTimer::timer_thread (this=0x1f61950) at common/Timer.cc:268
#5  0x00007fc82697a85d in RWTimerThread::entry (this=<optimized out>) at common/Timer.cc:200
#6  0x00007fc82651ce9a in start_thread (arg=0x7fc7e3fff700) at pthread_create.c:308

Signed-off-by: Sage Weil <sage@redhat.com>
osdc/Objecter: fix command op cancellation race

Cancel the command op timeout event before we clear out the op from the
session struct.  This isn't strictly necessary because command_op_cancel
will "gracefully" handle the case where the tid is no longer present, but
this avoids that noise and is cleaner.

Signed-off-by: Sage Weil <sage@redhat.com>
tests: add get_osds() and get_pg() helpers

To get the ordered list of OSD to which an object is mapped and the name
of the corresponding PG.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
tests: set the failure domain to OSD by default

So that tests do not need to do it to be able to use the default rbd
pool to store objects.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
os: FileStore::lfn_unlink always clears FDCache

Otherwise the FDCache will keep a file descriptor to a file that was
removed from the file system. This may create various type of errors
because the OSD checking the FDCache will assume the file that contains
information for an object exists although it does not. For instance in
the following:

      * rados put object file
      * rm file from the primary
      * repair the pg to which the object is mapped

if the FDCache is not cleared, repair will incorrectly pull a copy from
a replica and write it to the now unlinked file. Later on, it will
assume the file exists on the primary and only be partially correct :
the data can still be accessed via the file descriptor but any operation
using the path name will fail.

http://tracker.ceph.com/issues/8914 Fixes: #8914

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
tests: flush logs before grepping them

Otherwise the test races with the daemon writing the logs and can
sometime fail.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
mon: add the get_crush_ruleset helper

By factoring a code snippet from prepare_pool_crush_ruleset with no
modification.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
mon: pool create must not always create a ruleset

The implicit creation of a ruleset when creating a pool is convenient
when nothing is specified. However, if the caller sets a ruleset name,
it should not implicitly create it but return ENOENT instead. Silently
creating a ruleset when there is a typo in the ruleset name is
confusing.

http://tracker.ceph.com/issues/9304 Fixes: #9304

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
crushtool: safeguard for missing --num-rep when --test

http://tracker.ceph.com/issues/9490 Fixes: #9490

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
FileStore: Race condition during object delete is fixed

There was a race condition (hence OSD crash) between lfn_unlink
and lfn_open. The reason was FDCache lookup was called without
taking index lock from lfn_open. Lookup will increase reference
count and thus Clear will not be able to delete those FDs. FDs
will be leaked. The assert within FDCache clear was hitting
because of this.

Fixes: #9480

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
mon: prevent cache pools being used CephFS

Fixes two things:
 * EC pools are now permissible if they have a cache overlay
 * Pools are not permissible if they are a cache tier.

Fixes: #9435

Signed-off-by: John Spray <john.spray@redhat.com>