Cache tiering: do not allow flush/evict when there are requests pending

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
OpTracker: Sharding logic is implemented to improve performance

Enabling OpTracking is degrading performance significantly. The
main reason for this is the contention in the ops_in_flight_lock
during registration/unregistration. Sharding this lock and hence
the xlist (storing the ops) helped to boost the performance.
Sharding is been done based on the op seq number % num_shards.
atomic64_t is used as data type to protect the global sequence
number of OpTracker. Introduced a lock in the OpHistory to
protect its data structure as ealier it was protected by global
ops_in_flight_lock. This is only accessed during unregistration
and thus not contended.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: Removed _dump_op_descriptor_unlocked call for log level < 5

_dump_op_descriptor_unlocked call within the function _mark_event()
was invoked unnecessarily every time for all log levels. This was
degrading performance significantly! The stream is only used during
log level 5 and thus the _dump_op_descriptor_unlocked() is only called
now if user sets the log level to 5 and beyond.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: The TrackedOp::events list was not protected while dumping ops

OpTracker::Mutex::lock is there to protect the event list and it is
only used in the mark_event while pushing the events in the list.
It is not used while dumping the contents of the events list.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: Race condition removed while dumping ops through admin socket

OSD was crashing due to a race condition while IO was going on and
user wants to dump in flight ops. This was happening because Message
data and payloads are removed before the op itself is removed from
the in flight list. Calling op->_unregistered() after removing
the op from the in flight list fixes the issue.

Fixes: #9384

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
msg: prefetch data when doing recv()

Since that when we're reading the header we do lot's of small IOs, we
can really benefit from doing bigger reads.
Set a limit on the buffer size, so that it should mainly affect header
reads and not the actual data transfer of large IOs.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
qa/workunits/fs/misc: Add a workunit for ACL

Signed-off-by: Yan, Zheng <zyan@redhat.com>
client: introduce a new flag indicating if dentries in directory are sorted

When creating a file, Client::insert_dentry_inode() set the dentry's offset
based on directory's max offset. The offset does not reflect the real
postion of the dentry in directory. Later readdir reply from real postion
of the dentry in directory. Later readdir reply from MDS may change the
dentry's position/offset. This inconsistency can cause missing/duplicate
entries in readdir result if readdir is partly satisfied by dcache_readdir().

The fix is introduce a new flag indicating if dentries in directory are
sorted. We use _readdir_cache_cb() to handle readdir only when the flag is
set, clear the flag after creating/deleting/renaming file.

Fixes: #9178
Signed-off-by: Yan, Zheng <zyan@redhat.com>
client: preserve ordering of readdir result in cache

Preserve ordering of readdir result in a list, so that the result of cached
readdir is consistant with uncached readdir.

As a side effect, this commit also removes the code that removes stale dentries.
This is OK because stale dentries does not have valid lease, they will be
filter out by the shared gen check in Client::_readdir_cache_cb()

Signed-off-by: Yan, Zheng <zyan@redhat.com>
mds: fix not journaling client metadata

Previously the code was there for storing in
the SessionMap table, but not for the ESession
logevent.

Fixes: #9518

Signed-off-by: John Spray <john.spray@redhat.com>
Merge pull request #2515 from ceph/wip-9518

mds: fix not journaling client metadata

Reviewed-by: Yan, Zheng <ukernel@gmail.com>
Merge pull request #2445 from wonzhq/req-pending

Cache tiering: do not allow flush/evict when there are requests pending

Reviewed-by: Samuel Just <sam.just@inktank.com>
Merge pull request #2440 from somnathr/wip-optracker-optimization

Wip optracker optimization

Reviewed-by: Samuel Just <sam.just@inktank.com>
Merge pull request #2457 from ceph/wip-7767-b

msg: prefetch data when doing recv()

Reviewed-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Samuel Just <sam.just@inktnak.com>
mon/Paxos: fix off-by-one in last_ vs first_committed check

peon last_committed + 1 == leader first_committed is okay.  Note that the
other check (where I clean up whitespace) gets this correct.

Fixes: #9301 (partly)
Signed-off-by: Sage Weil <sage@redhat.com>
mon: re-bootstrap if we get probed by a mon that is way ahead

During bootstrap we verify that our paxos commits overlap with the other
mons we will form a quorum with.  If they do not, we do a sync.

However, it is possible we pass those checks, then fail to join a quorum
before the quorum moves ahead in time such that we no longer overlap.
Currently nothing kicks up back into a probing state to discover we need
to sync... we will just keep trying to call or join an election instead.

Fix this by jumping back to bootstrap if we get a probe that is ahead of
us.  Only do this from non probe or sync states as these will be common;
it is only the active and electing states that matter (and probably just
electing!).

Fixes: #9301
Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
Merge pull request #2514 from ceph/wip-acl-test

qa/workunits/fs/misc: Add a workunit for ACL

Reviewed-by: Greg Farnum <greg@inktank.com>
common: Add cctid meta variable

Fixes: #6228
Signed-off-by: Adam Crume <adamcrume@gmail.com>
documentation: fix bugous osd stop/start example

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
documentation: tiering typo

Signed-off-by: Loic Dachary <loic-201408@dachary.org>