ReplicatedPG: cancel cb on blacklisted watcher

Fixes: #8315
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
osd_types: fix pg_log_t tail dump

Fixes: #9453
Signed-off-by: Samuel Just <sam.just@inktank.com>
DBObjectMap: restructure map_header_in_use locking

Tieing map_header_in_use to a _Header is a mistake since ownership of
the map_header_in_use can move (clone).  Instead, grab the
map_header_in_use entry at the top of each call chain and release at the
end.  To facilitate that, we introduce a MapHeaderLock object and
require that it be passed to the various map header manipulation
methods.

Fixes: #9326
Signed-off-by: Samuel Just <sam.just@inktank.com>
tests: add get_osds() and get_pg() helpers

To get the ordered list of OSD to which an object is mapped and the name
of the corresponding PG.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
tests: set the failure domain to OSD by default

So that tests do not need to do it to be able to use the default rbd
pool to store objects.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
os: FileStore::lfn_unlink always clears FDCache

Otherwise the FDCache will keep a file descriptor to a file that was
removed from the file system. This may create various type of errors
because the OSD checking the FDCache will assume the file that contains
information for an object exists although it does not. For instance in
the following:

      * rados put object file
      * rm file from the primary
      * repair the pg to which the object is mapped

if the FDCache is not cleared, repair will incorrectly pull a copy from
a replica and write it to the now unlinked file. Later on, it will
assume the file exists on the primary and only be partially correct :
the data can still be accessed via the file descriptor but any operation
using the path name will fail.

http://tracker.ceph.com/issues/8914 Fixes: #8914

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
tests: flush logs before grepping them

Otherwise the test races with the daemon writing the logs and can
sometime fail.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
mon: add the get_crush_ruleset helper

By factoring a code snippet from prepare_pool_crush_ruleset with no
modification.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
mon: pool create must not always create a ruleset

The implicit creation of a ruleset when creating a pool is convenient
when nothing is specified. However, if the caller sets a ruleset name,
it should not implicitly create it but return ENOENT instead. Silently
creating a ruleset when there is a typo in the ruleset name is
confusing.

http://tracker.ceph.com/issues/9304 Fixes: #9304

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
PG::choose_acting: let the pg go down if acting is smaller than min_size

Even if the backfill peer would bring us up to min_size, we can't go
active since build_prior will not consider the interval maybe_went_rw.

Fixes: #9497
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
PG::find_best_info: let history.last_epoch_started provide a lower bound

If we find a info.history.last_epoch_started above any
info.last_epoch_started, we must be missing updates and
min_last_update_acceptable should provisionally be max().

Fixes: #9482
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
PGLog::claim_log_and_clear_rollback_info: fix rollback_info_trimmed_to

We have been setting it to the old head value.  This is usually
harmless since the new head will virtually always be ahead of the
old head for claim_log_and_clear_rollback_info, but can cause trouble
in some edge cases.

Fixes: #9481
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
client: introduce a new flag indicating if dentries in directory are sorted

When creating a file, Client::insert_dentry_inode() set the dentry's offset
based on directory's max offset. The offset does not reflect the real
postion of the dentry in directory. Later readdir reply from real postion
of the dentry in directory. Later readdir reply from MDS may change the
dentry's position/offset. This inconsistency can cause missing/duplicate
entries in readdir result if readdir is partly satisfied by dcache_readdir().

The fix is introduce a new flag indicating if dentries in directory are
sorted. We use _readdir_cache_cb() to handle readdir only when the flag is
set, clear the flag after creating/deleting/renaming file.

Fixes: #9178
Signed-off-by: Yan, Zheng <zyan@redhat.com>
client: preserve ordering of readdir result in cache

Preserve ordering of readdir result in a list, so that the result of cached
readdir is consistant with uncached readdir.

As a side effect, this commit also removes the code that removes stale dentries.
This is OK because stale dentries does not have valid lease, they will be
filter out by the shared gen check in Client::_readdir_cache_cb()

Signed-off-by: Yan, Zheng <zyan@redhat.com>
mds: fix not journaling client metadata

Previously the code was there for storing in
the SessionMap table, but not for the ESession
logevent.

Fixes: #9518

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 1395275e0f4d12631b1e4074598a7682810f00f9)
Merge pull request #2522 from athanatos/wip-8315

ReplicatedPG: cancel cb on blacklisted watcher

Reviewed-by: Sage Weil <sage@redhat.com>
Merge pull request #2518 from athanatos/wip-9481

PGLog::claim_log_and_clear_rollback_info: fix rollback_info_trimmed_to

Reviewed-by: Sage Weil <sage@redhat.com>
Merge pull request #2520 from athanatos/wip-9497

PG::choose_acting: let the pg go down if acting is smaller than min_size

Reviewed-by: Sage Weil <sage@redhat.com>
Merge pull request #2519 from athanatos/wip-9482

PG::find_best_info: let history.last_epoch_started provide a lower bound

Reviewed-by: Sage Weil <sage@redhat.com>
Merge pull request #2521 from athanatos/wip-9453

osd_types: fix pg_log_t tail dump

Reviewed-by: Sage Weil <sage@redhat.com>