client: Add routine to get caps of file/fd

In order to properly validate the client capabilities,
we need to be able to access them from libcephfs.

Signed-off-by: Sam Lang <sam.lang@inktank.com>
test/libcephfs: Add a test for validating caps

Signed-off-by: Sam Lang <sam.lang@inktank.com>
mds/locker:  Add debugging for excl->mix trans

Signed-off-by: Sam Lang <sam.lang@inktank.com>
mds: move from EXCL to SYNC if nobody wants to write

We were moving to the MIX even if nobody wanted to write; that is not
useful, since if we only want to read SYNC will let us cache those reads.
SYNC is also a more friendly place (all things equal) to be.

Signed-off-by: Sage Weil <sage@inktank.com>
filestore: simplify op quescing

The delicate balancing with op_apply_start() and that fact that it can
block was making it very hard to determine how long commit_start() should
wait, since requests in the workqueue threads could op_apply_start() in
any order.  For example,

 threadA: gets osr1 from wq
 threadA: gets osr2 from wq
 threadA: dequeue seq 11 from osr1, op_apply_start
 threadC: commit_start on 11
 threadA: op_apply_finish on seq 11
 threadC: commit_started, commit_finish
 threadB: dequeue seq 10 from osr2
   <failed assert, badness>

Instead, rip out all this code, and use the ThreadPool pause() method to
quiesce operations.  Keep some of the (now unnecessary) fields around
for sanity checks (blocked, open_ops, max_applying_seq, etc.).

Signed-off-by: Sage Weil <sage@inktank.com>
os/JourningObjectStore: drop now-useless max_applying_seq

Signed-off-by: Sage Weil <sage@inktank.com>
os/JournalingObjectStore: remove unused ops_submitting

Signed-off-by: Sage Weil <sage@inktank.com>
os/JournalingObjectStore: simplify op_submitting sanity check

A list is overkill; just use a seq and make sure it increments to ensure
the op_submit_finish calls are in order.

Signed-off-by: Sage Weil <sage@inktank.com>
Merge remote-tracking branch 'gh/wip-3559' into next

Reviewed-by: Sage Weil <sage@inktank.com>
config_opts.h: adjust recovery defaults

osd max backfills: 5 was too low for a default, 10
 seems to work better in testing.  The message
 priority system should minimize disruption of
 push and pull operations anyway.

osd recovery max chunk: 1MB was too small for a
 default.  8MB is reasonable for a single push
 and will allow us to recover an rbd block in
 one push rather then 4 reducing client io
 latency during log-based recovery.

osd recovery op priority: 10 rather than 30 will
 further reduce the client io latency impact of
 push and pull operations.

Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge remote-tracking branch 'gh/wip-filestore2' into next

Reviewed-by: Sam Just <sam.just@inktank.com>
mds: fix journaling issue regarding rstat accounting

Rename operation can call predirty_journal_parents() several times.
So a directory fragment's rstat can also be modified several times.
But only the first modification is journaled because EMetaBlob::add_dir()
does not update existing dirlump.

For example: when hanlding 'mv a/b/c a/c', Server::_rename_prepare may
first decrease directory a and b's nested files count by one, then
increases directory a's nested files count by one.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: alllow handle_client_readdir() fetching freezing dir.

At that point, the request already auth pins and locks some objects.
So CDir::fetch() should ignore the can_auth_pin check and continue
to fetch freezing dir.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: properly mark dirfrag dirty

If predirty_journal_parents() does not propagate changes in dir's
fragstat into corresponding inode's dirstat, it should mark the
inode as dirfrag dirty. This happens when we modify dir fragments
that are auth subtree roots.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: no bloom filter for replica dir

We should delete dir fragment's bloom filter after exporting the dir
fragment to other MDS. Otherwise the residual bloom filter may cause
problem if the MDS imports dir fragment later.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: set want_base_dir to false for MDCache::discover_ino()

When frozen inode is encountered, MDCache::handle_discover() sends
reply immediately if the reply message is not empty. When handling
"discover ino" requests, the reply message always contains the base
directory fragment. But requestor already has the base directory
fragment, the only effect of the reply message is wake the requestor
and make it send same "discover ino" request again. So the requestor
keeps sending "discover ino" requests but can't make any progress.

The fix is set want_base_dir to false for MDCache::discover_ino().
After set want_base_dir to false, also need update the code that
handles "discover ino" error.

This patch also remove unused error handling code for flag_error_dn

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: fix error hanlding in MDCache::handle_discover_reply()

The error hanlding code in MDCache::handle_discover_reply() has two
main issues. MDCache::handle_discover_reply() does not wake waiters
if dir_auth_hint in reply message is equal to itself's nodeid. This
can happen if discover race with subtree importing. Another issue is
that it checks the existence of cached directory fragment to decide
if it should take waiter from inode or from directory fragment. The
check is unreliable because subtree importing can add directory
fragments to the cache.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: always send discover if want_xlocked is true

If want_xlocked is true, we can not rely on previously sent discover
because it's likely the previous discover is blocking on the xlocked
dentry.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: re-issue caps after importing caps

The imported caps may prevent unstable locks from entering stable
states. So we should call Locker::eval_gather() with parameter
"first" set to true after caps are imported.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: take export lock set before sending MExportDirDiscover

Migrator::export_dir() only check if it can lock the export lock set
but not take the lock set. So someone else can change the path to
the exporting dir and confuse Migrator::handle_export_discover().

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>