mds: delay processing cache expire when state >= EXPORT_EXPORTING

It's possible that MDS receives cache expire in EXPORT_LOGGINGFINISH
and EXPORT_NOTIFYING states.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: fix file existing check in Server::handle_client_openc()

Creating new file needs to be handled by directory fragment's auth
MDS, opening existing file in write mode needs to be handled by
corresponding inode's auth MDS. If a file is remote link, its parent
directory fragment's auth MDS can be different from corresponding
inode's auth MDS. So which MDS to handle create file request can be
affected by if the corresponding file already exists.

handle_client_openc() calls rdlock_path_xlock_dentry() at the very
beginning. It always assumes the request needs to be handled by
directory fragment's auth MDS. When handling a create file request,
if the file already exists and remotely linked to a non-auth inode,
handle_client_openc() falls back to handle_client_open(),
handle_client_open() forwards the request because the MDS is not
inode's auth MDS. Then when the request arrives at inode's auth MDS,
rdlock_path_xlock_dentry() is called, it will forward the request
back.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: fix race between send_dentry_link() and cache expire

MDentryLink message can race with cache expire, When it arrives at
the target MDS, it's possible there is no corresponding dentry in
the cache. If this race happens, we should expire the replica inode
encoded in the MDentryLink message. But to expire an inode, the MDS
need to know which subtree does the inode belong to, so modify the
MDentryLink message to include this information.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: compare sessionmap version before replaying imported sessions

Otherwise we may wrongly increase mds->sessionmap.version, which
will confuse future journal replays that involving sessionmap.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: don't force in->first == dn->first

The fullbit sets it now.  For multiversion inodes, it's "first" can be in
the future, since this dentry may not have changed when the inode was
cowed in place.  (OTOH, the dentry cannot have changed without the inode
also have changing.)

Signed-off-by: Sage Weil <sage.weil@dreamhost.com>
mds: replace closed sessions on connect

If a connection comes and there is a closed session attached, remove it.
This is probably a failure of an old session to get cleaned up properly,
and in certain cases it may even be from a different client (if the addr
nonce is reused).  In that case this prevents further damage, although
a complete solution would also clean up the closed connection state if
there is a fault.  See #3630.

This fixes a hang that is reproduced by running the libcephfs
Caps.ReadZero test in a loop; eventually the client addr is reused and
we are linked to an ancient Session with a different client id.

Backport: bobtail
Signed-off-by: Sage Weil <sage@inktank.com>
doc/man/8/mkcephfs: update --mkfs a bit

Document that 'devs' and 'osd mkfs type' must be defined.

Signed-off-by: Sage Weil <sage@inktank.com>
qa: remove xfstests 172 and 173 from qemu testing

These seem to require newer xfs.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
mds: do not check for pool existence in osdmap

We don't have a wait mechanism to ensure the MSDMap has the latest osdmap
here.  Just trust the MDSMap.

Signed-off-by: Sage Weil <sage@inktank.com>
mds: add waiting_for_mdsmap queue

Defer events until we get a specific MDSMap epoch.

Signed-off-by: Sage Weil <sage@inktank.com>
docs:  Merge changes from release-process2 document.
doc: journaler config options

Signed-off-by: Sage Weil <sage@inktank.com>
doc: document mds config options

Signed-off-by: Sage Weil <sage@inktank.com>
mds: wait for client's mdsmap when specifying data pool

The client may have a newer map than we do; make sure we wait for it lest
we inadvertantly reply because we think the pool doesn't exist.

Signed-off-by: Sage Weil <sage@inktank.com>
mds: use set to store MDSMap data pools

Signed-off-by: Sage Weil <sage@inktank.com>
journal: reduce journal max queue size

Keep the journal queue size smaller than the filestore queue size.

Keeping this small also means that we can lower the latency for new
high priority ops that come into the op queue.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: drop 'osd recovery max active' back to previous default (5)

Having this too large means that queues get too deep on the OSDs during
backfill and latency is very high.  In my tests, it also meant we generated
a lot of slow recovery messages just from the recovery ops themselves (no
client io).

Keeping this at the old default means we are no worse in this respect than
argonaut, which is a safe position to start from.

Signed-off-by: Sage Weil <sage@inktank.com>
docs:  remove extra release-process2 file.

This file mostly duplicated the existing release documentation.  Differences
have been merged into the primary file.

Signed-off-by: Gary Lowell <gary.lowell@inktank.com>
dropping xfs test 186 due to bug: 3685

Signed-off-by: tamil <tamil.muthamizhan@inktank.com>
osd: move rmw_flags to OpRequest, out of MOSDOp

It was very sloppy to put a server-side processing state inside the
messsage.  Move it to the OpRequestRef instead.

Note that the client was filling in bogus data that was then lost during
encoding/decoding; clean that up.

Signed-off-by: Sage Weil <sage@inktank.com>