Cache tiering: do not allow flush/evict when there are requests pending

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
OpTracker: Sharding logic is implemented to improve performance

Enabling OpTracking is degrading performance significantly. The
main reason for this is the contention in the ops_in_flight_lock
during registration/unregistration. Sharding this lock and hence
the xlist (storing the ops) helped to boost the performance.
Sharding is been done based on the op seq number % num_shards.
atomic64_t is used as data type to protect the global sequence
number of OpTracker. Introduced a lock in the OpHistory to
protect its data structure as ealier it was protected by global
ops_in_flight_lock. This is only accessed during unregistration
and thus not contended.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: Removed _dump_op_descriptor_unlocked call for log level < 5

_dump_op_descriptor_unlocked call within the function _mark_event()
was invoked unnecessarily every time for all log levels. This was
degrading performance significantly! The stream is only used during
log level 5 and thus the _dump_op_descriptor_unlocked() is only called
now if user sets the log level to 5 and beyond.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: The TrackedOp::events list was not protected while dumping ops

OpTracker::Mutex::lock is there to protect the event list and it is
only used in the mark_event while pushing the events in the list.
It is not used while dumping the contents of the events list.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: Race condition removed while dumping ops through admin socket

OSD was crashing due to a race condition while IO was going on and
user wants to dump in flight ops. This was happening because Message
data and payloads are removed before the op itself is removed from
the in flight list. Calling op->_unregistered() after removing
the op from the in flight list fixes the issue.

Fixes: #9384

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
DBObjectMap: restructure map_header_in_use locking

Tieing map_header_in_use to a _Header is a mistake since ownership of
the map_header_in_use can move (clone).  Instead, grab the
map_header_in_use entry at the top of each call chain and release at the
end.  To facilitate that, we introduce a MapHeaderLock object and
require that it be passed to the various map header manipulation
methods.

Fixes: #9326
Signed-off-by: Samuel Just <sam.just@inktank.com>
tests: add get_osds() and get_pg() helpers

To get the ordered list of OSD to which an object is mapped and the name
of the corresponding PG.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
tests: set the failure domain to OSD by default

So that tests do not need to do it to be able to use the default rbd
pool to store objects.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
os: FileStore::lfn_unlink always clears FDCache

Otherwise the FDCache will keep a file descriptor to a file that was
removed from the file system. This may create various type of errors
because the OSD checking the FDCache will assume the file that contains
information for an object exists although it does not. For instance in
the following:

      * rados put object file
      * rm file from the primary
      * repair the pg to which the object is mapped

if the FDCache is not cleared, repair will incorrectly pull a copy from
a replica and write it to the now unlinked file. Later on, it will
assume the file exists on the primary and only be partially correct :
the data can still be accessed via the file descriptor but any operation
using the path name will fail.

http://tracker.ceph.com/issues/8914 Fixes: #8914

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Merge pull request #2505 from dachary/wip-8914-unlink-fdcache

os: FileStore::lfn_unlink always clears FDCache

Reviewed-by: Sage Weil <sage@redhat.com>
mon: osd find / metadata --format plain fallback

ceph --format plain osd find 1 (and metadata) are not implemented and
must fallback to the default (json-pretty).

http://tracker.ceph.com/issues/9538 Fixes: #9538

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: fix assert overflow

If the file size does not fit in 32 bits the (unsigned) cast will
overflow. Cast to uint64_t which is the type of the value returned by
get_total_chunk_size.

http://tracker.ceph.com/issues/9537 Fixes: #9537

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Merge pull request #2533 from dachary/wip-9538-format-plain

mon: osd find / metadata --format plain fallback

Reviewed-by: John Spray <john.spray@redhat.com>
Merge pull request #2517 from athanatos/wip-9326

DBObjectMap: restructure map_header_in_use locking

Reviewed-by: Sage Weil <sage@redhat.com>
crypto: don't hash zero sized buffer

libnss returns an error and we assert in that case.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
rgw: calculate hash after writing data

Since data is written asynchronously, we should do the hash calculation
while it's pending.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
Merge pull request #2539 from ceph/wip-9529

Wip 9529

Reviewed-by: Sage Weil <sage@redhat.com>
LogEntry.h: init LogEntry::seq with 0

CID 1238904 (#1 of 1): Uninitialized scalar variable (UNINIT)
 uninit_use_in_call: Using uninitialized value e.seq when calling
 log_to_syslog.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 2c2b413ee0af2a151999fcf72f44745b0a9d4ab2)
ErasureCodeLrc.h: fix data_chunk_count UNINIT_CTOR

CID 1238903 (#1 of 1): Uninitialized scalar field (UNINIT_CTOR)
 uninit_member: Non-static class member data_chunk_count is not
 initialized in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit 2bb35bba995af1983a81669711fda7855e1ce532)
ErasureCodeLrc.h: fix UNINIT_CTOR

Fix Coverity issue, preinit with 0:

uninit_member: Non-static class member chunk_count is not initialized
 in this constructor nor in any functions that it calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
(cherry picked from commit ea02dc37dc92613d2d35b9572e72c3b9ab6bbb69)