Cache tiering: do not allow flush/evict when there are requests pending

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
OpTracker: Sharding logic is implemented to improve performance

Enabling OpTracking is degrading performance significantly. The
main reason for this is the contention in the ops_in_flight_lock
during registration/unregistration. Sharding this lock and hence
the xlist (storing the ops) helped to boost the performance.
Sharding is been done based on the op seq number % num_shards.
atomic64_t is used as data type to protect the global sequence
number of OpTracker. Introduced a lock in the OpHistory to
protect its data structure as ealier it was protected by global
ops_in_flight_lock. This is only accessed during unregistration
and thus not contended.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: Removed _dump_op_descriptor_unlocked call for log level < 5

_dump_op_descriptor_unlocked call within the function _mark_event()
was invoked unnecessarily every time for all log levels. This was
degrading performance significantly! The stream is only used during
log level 5 and thus the _dump_op_descriptor_unlocked() is only called
now if user sets the log level to 5 and beyond.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: The TrackedOp::events list was not protected while dumping ops

OpTracker::Mutex::lock is there to protect the event list and it is
only used in the mark_event while pushing the events in the list.
It is not used while dumping the contents of the events list.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: Race condition removed while dumping ops through admin socket

OSD was crashing due to a race condition while IO was going on and
user wants to dump in flight ops. This was happening because Message
data and payloads are removed before the op itself is removed from
the in flight list. Calling op->_unregistered() after removing
the op from the in flight list fixes the issue.

Fixes: #9384

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
tests: flush logs before grepping them

Otherwise the test races with the daemon writing the logs and can
sometime fail.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
mon: add the get_crush_ruleset helper

By factoring a code snippet from prepare_pool_crush_ruleset with no
modification.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
mon: pool create must not always create a ruleset

The implicit creation of a ruleset when creating a pool is convenient
when nothing is specified. However, if the caller sets a ruleset name,
it should not implicitly create it but return ENOENT instead. Silently
creating a ruleset when there is a typo in the ruleset name is
confusing.

http://tracker.ceph.com/issues/9304 Fixes: #9304

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: fix assert overflow

If the file size does not fit in 32 bits the (unsigned) cast will
overflow. Cast to uint64_t which is the type of the value returned by
get_total_chunk_size.

http://tracker.ceph.com/issues/9537 Fixes: #9537

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
include/util.h: prevent multiple inclusion of header

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
crypto: don't hash zero sized buffer

libnss returns an error and we assert in that case.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
rgw: calculate hash after writing data

Since data is written asynchronously, we should do the hash calculation
while it's pending.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Merge pull request #2539 from ceph/wip-9529

Wip 9529

Reviewed-by: Sage Weil <sage@redhat.com>
LogEntry.h: init LogEntry::seq with 0

CID 1238904 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use_in_call: Using uninitialized value e.seq when calling
 log_to_syslog.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 2c2b413ee0af2a151999fcf72f44745b0a9d4ab2)
ErasureCodeLrc.h: fix data_chunk_count UNINIT_CTOR

CID 1238903 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member data_chunk_count is not
 initialized in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 2bb35bba995af1983a81669711fda7855e1ce532)
ErasureCodeLrc.h: fix UNINIT_CTOR

Fix Coverity issue, preinit with 0:

uninit_member: Non-static class member chunk_count is not initialized
 in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit ea02dc37dc92613d2d35b9572e72c3b9ab6bbb69)
mds/Beacon.*: fix UNINIT_CTOR cases

CID 1238905 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member want_state is not initialized
  in this constructor nor in any functions that it calls.
 uninit_member: Non-static class member last_send is not initialized
  in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit ff6148324ad0957f74e8b4b49c1a1e8df9fb94e4)
os/GenericObjectMap.cc: pass big parameter by reference

CID 1188142 (#1 of 1): Big parameter passed by value (PASS_BY_VALUE)
 pass_by_value: Passing parameter header of type
 GenericObjectMap::_Header (size 176 bytes) by value.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 13b8c92a290ea078af26638fbd23bdb2b8c27bdf)
rgw_main.cc: add missing virtual destructor for RGWRequest

CID 1160858 (#1 of 1): Non-virtual destructor (VIRTUAL_DTOR)
 nonvirtual_dtor: Class RGWLoadGenRequest has a destructor
 and a pointer to it is upcast to class RGWRequest which doesn't
 have a virtual destructor.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit b82ceda7775ff85943d9143b73789eb37b09bfa9)
Merge remote-tracking branch 'somnathr/wip-optracker-optimization' into giant