rgw: add compatibility for MultipartUpload

Fixes: #6677
It seems that there's some external library that uses MultipartUpload
instead of CompleteMultipartUpload, even though it's not part of the
documented api. Support it.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: set bucket permissions, cors on swift creation

Fixes: #7271
Swift bucket creation was ignoring the x-container-read,
x-container-write and the various cors related headers. Now going
through the same logic that bucket modify goes.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
monc: let users specify a callback when they reopen their monitor session

Then the callback is triggered when a new session is established, and the
daemon can do whatever it likes. There are no guarantees about how long it
might take to trigger, though. In particular we call the provided callback
while not holding our own lock in order to avoid deadlock. This could lead
to some funny ordering from the user's perspective if they call
reopen_session() again before getting the callback, but there's no way around
that, so they just have to use it appropriately.

Signed-off-by: Greg Farnum <greg@inktank.com>
monc: set "hunting" to true when we reopen the mon session

If we don't have a connecton to a monitor, we want to retry to another
monitor regardless of whether it's the first time or not.

Signed-off-by: Greg Farnum <greg@inktank.com>
monc: backoff the timeout period when reconnecting

If the monitors are systematically slowing down, we don't want to spam
them with reconnect attempts every three seconds. Instead, every time
we issue a reconnect, multiply our timeout period by a configurable; when
we complete the connection, reduce that multipler by 50%. This should let
us respond to monitor load.
Of course, we don't want to do that for initial startup in the case of a
couple down monitors, so don't apply the backoff until we've successfully
connected to a monitor at least once.

Signed-off-by: Greg Farnum <greg@inktank.com>
OSD: disable the PGStatsAck timeout when we are reconnecting to a monitor

Previously, the timeout counter started as soon as we issued the reopen,
but if the reconnect process itself took a while, we might time out and
issue another reopen just as we get to the point where it's possible to
get work done. Since the mon client has its own reconnect timeouts (that is,
the OSD doesn't need to trigger those), we instead disable our timeouts
while the reconnect is happening, and then turn them back on again starting
from when we get the reconnect callback.

Signed-off-by: Greg Farnum <greg@inktank.com>
common: ping existing admin socket before unlink

When a daemon initializes it tries to create an admin socket and unlinks
any pre-existing file, regardless. If such a file is in use, it causes
the existing daemon to loose its admin socket.

The AdminSocketClient::ping is implemented to probe an existing socket,
using the "0" message. The AdminSocket::bind_and_listen function is
modified to call ping() on when it finds existing file. It unlinks the
file only if the ping fails.

http://tracker.ceph.com/issues/7188 fixes: #7188

Backport: emperor, dumpling
Reviewed-by: Sage Weil <sage@inktank.com>
Signed-off-by: Loic Dachary <loic@dachary.org>
Merge pull request #1246 from dachary/wip-7188

common: ping existing admin socket before unlink
mds: allow acquiring wrlock and remote wrlock at the same time

Rename may move file from one dirfrag to another dirfrag of the same
directory inode. If two dirfrags belong to different auth MDS, both
MDS should hold wrlocks on filelock/nestlock of the directory inode.
If a lock is in both wrlocks list and remote_wrlocks list, current
Locker::acquire_locks() only acquires the local wrlock. The auth MDS
of the source dirfrag doesn't have the wrlock, so slave request of
the operation may modify the dirfrag after fragstat/neststat of the
dirfrag have already been gathered. It corrupts the dirstat/neststat
accounting.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: introduce nonlocking auth pin

Add a parameter to Locker::acquire_locks() to enabled nonblocking
auth pin. If nonblocking mode is enabled and an object that can't
be auth pinned is encountered, Locker::acquire_locks() aborts the
MDRequest instead of waiting.

The nonlocking mode is acquired by the cases that we want to acquire
locks after auth pinning a directory or freezing a dirfrag/subtree.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: acquire locks required by exporting dir

Start internal MDS request to acquire locks required by exporting dir.
It's more reliable than using Locker::rdlock_take_set(), It also allows
acquiring locks besides rdlock.

Only use Locker::acquire_locks() to acquire locks in the first stage of
exporting dir (before freeze the subtree). After the subtree is frozen,
to minimize the time of frozen tree, still use 'try lock' to re-acquire
the locks.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: acquire scatter locks when exporting dir

If auth MDS of the subtree root inode is neither the exporter MDS
nor the importer MDS and it gathers subtree root's fragstat/neststat
while the subtree is exporting. It's possible that the exporter MDS
and the importer MDS both are auth MDS of the subtree root or both
are not auth MDS of the subtree root at the time they receive the
lock messages. So the auth MDS of the subtree root inode may get no
or duplicated fragstat/neststat for the subtree root dirfrag.

The fix is, during exporting a subtree, both the exporter MDS and
the importer MDS hold locks on scatter locks of the subtree root
inode. For the importer MDS, it tries acquiring locks on the scatter
locks when handling the MExportDirPrep message. If fails to acquire
all locks, it sends a NACK to the exporter MDS. The exporter MDS
cancels the exporting when receiving the NACK.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: revert commit 15a5d37a

commit 15a5d37a (mds: fix race between scatter gather and dirfrag export)
is incomplete, it doesn't handles the race that no fragstat/neststat is
gathered. Previous commit prevents scatter gather during exporting dir,
which eliminates races of this type.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: introduce fine-grained discover dirfrag wait queue

Current discover dirfrag code only allows discover one dirfrag at
a time. This can cause deadlock if there are directories that are
fragmented to several dirfrags. For example:

mds.0                        mds.1
-----------------------------------------------------------------
                             freeze subtree (1.*) with bound (2.1*)
discover (2.0*) ->
                             handle discover (2.0*), frozen tree, wait
                          <- export subtree (1.*) to with bound (2.1*)
discover (2.1*), wait

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: use discover_path to open remote inode

MDCache::discover_ino() doesn't work well for directories that are
fragmented to several dirfrags. Because MDCache::handle_discover()
doesn't know which dirfrags the inode lives in when the sender has
outdatad frag information.

This patch replaces all use of MDCache::discover_ino() with
MDCache::discover_path().

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: handle frag mismatch for discover

When handle discover dirfrag message, choose an approximate frag if
the requested dirfrag doesn't exist. When handling discover dirfrag
replay, wake up appropriate waiters if the reply is different from
the the requested dirfrag.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: fix MDCache::get_force_dirfrag_bound_set()

don't force dir fragments according to the subtree bounds in resolve
message. The resolve message was not sent by the auth MDS of these
subtree bounds dirfrags.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: fix MDCache::adjust_subtree_after_rename()

process subtree dirfrags first, then process nested dirfrags. because
the code that processes nested dirfrags treats unprocessed subtree
dirfrags as child directories' dirfrags.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: fix CInode::get_dirfrags_under()

make the function work when opened dirfrags don't match the
dirfragtree.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
mds: properly set COMPLETE flag when merging dirfrags

don't keep the COMPLETE flag when merging dirfrags during journal
replay, because it's inconvenience to check if the all dirfrags
under the 'basefrag' are in the cache and complete. One special case
is that newly created dirfrag get fragmented, then the fragment
operation get rolled back. The COMPLETE flag should be preserved in
this case because the dirfrag still doesn't exist on object store.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>