Merge pull request #1601 from ceph/wip-7576

osd: prevent pg map epochs from lagging too far behind

Reviewed-by: Samuel Just <sam.just@inktank.com>
osd/ReplicatedPG: fix whiteouts for other cache mode

We were special casing WRITEBACK mode for handling whiteouts; this needs to
also include the FORWARD and READONLY modes.  To avoid having to list
specific cache modes, though, just check != NONE.

Fixes: #8296
Backport: firefly
Signed-off-by: Sage Weil <sage@inktank.com>
ObjectCacher: remove useless assignment

left is not read after the break. Caught by clang-analyzer.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
librbd: check return value during snap_unprotect

This would only fail if the header object was corrupted, so it's
unlikely to occur in practice.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Pipe: wait for Pipes to finish running, instead of just stop()ing them

Add a stop_and_wait() function that, in addition to closing the Pipe and killing
its socket, waits for any fast_dispatch call which is in-progress. Use this in
several parts of the Pipe and SimpleMessenger code where appropriate.

This fixes several races with fast_dispatch and other avenues; here are two:
1) It could be that we grab the lock while the existing pipe is fast_dispatching
and then proceed to dispatch messages ourself, beating it. Instead, wait for
the other pipe. Add a "reader_dispatching" member which tells bus this is
happening, and when re-locking, signal the cond if we're shutting down.

2) It could be that a normally-dispatched Message in the OSD triggers a
mark_down() on the Connection and then clears out the Session
(Connection::priv) pointer, causing a racing fast_dispatch()'ed function to
assert out in the OSD because it requires a valid Session.

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
rgw: cut short object read if a chunk returns error

Fixes: #8289
Backport: firefly, dumpling
When reading an object, if we hit an error when trying to read one of
the rados objects then we should just stop. Otherwise we're just going
to continue reading the rest of the object, and since it can't be sent
back to the client (as we have a hole in the middle), we end up
accumulating everything in memory.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Merge pull request #1774 from ceph/wip-8296

osd/ReplicatedPG: fix whiteouts for other cache mode

Reviewed-by: Samuel Just <sam.just@inktank.com>
rgw: don't allow multiple writers to same multiobject part

Fixes: #8269
Backport: firefly, dumpling

A client might need to retry a multipart part write. The original thread
might race with the new one, trying to clean up after it, clobbering the
part's data.
The fix is to detect whether an original part already existed, and if so
use a different part name for it.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
ceph-disk: fix list for encrypted or corrupt volume

Continue gracefully if an fs type is not detected, either because it is
encrypted or because it is corrupted.

Signed-off-by: Sage Weil <sage@inktank.com>
mds: maintain auth bits during replay

Objects' STATE_AUTH bits are set when replaying EImportStart event.
MDCache::trim_non_auth_subtree() clear objects' STATE_AUTH bits when
replaying EExport event.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: send dentry unlink message to replicas of stray dentry

stray dentry may have more replicas than the unlinked dentry has.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: include authpinned objects in remote authpin request

Server::handle_slave_auth_pin() may drop old authpins if it encounters
object that is not authpinable. So it is better to include objects that
have already been authpinned in the remote authpin reuqest.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: fix frozen inode check in MDCache::handle_discover()

When MDCache::handle_discover() encounters a frozen dirfrag, it should
proceed if the dirfrag is being merged, but the MDS hasn't frozen all
dirfrags yet. When MDCache::handle_discover() checks if a inode is
frozen, it should use CInode::is_frozen_inode() (which doesn't check if
inode's parent dirfrag is frozen).

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: pre-allocate inode numbers less frequently

no need to refill the pre-allocated inode numbers each time an inode
number is used.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: tolerate bad sessionmap during journal replay

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: remove mdsdir in the final step of shutdown MDS

Otherwise we may get bad subtree map if we restart the MDS before
the shutdown process finishes.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: clear aborted flag before rollback slave requests

There is a special case that the MDRequest needs to be preserved after
rolling back slave rename. The preserved MDRequest will be used later.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: fix-up inode's fragstat/rstat according its dirfrags

Extend the code that fixup inode's fragstat/rstat to handle multiple
dirfrags

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: encode dirfrag base in cache rejoin ack

Makes sure recovering MDS get uptodate fragstat/rstat for subtree root
dirfrags. it's required the codes that fix-up inode's fragstat/rstat.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: ignore stale rstat/fragstat when splitting dirfrag

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>