rgw: Fix S3 auth when using response-* query string params

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>
Signed-off-by: Sylvain Munaut <s.munaut@whatever-company.com>
(cherry picked from commit 7a7361d7e798d94796d4985e2e4b35af22463ae2)
rgw: lower some debug message

Fixes: #6084
Backport: dumpling, emperor

Reported-by: Ron Allred <rallred@itrefined.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
(cherry picked from commit b35fc1bc2ec8c8376ec173eb1c3e538e02c1694e)
rgw: bucket meta remove don't overwrite entry point first

Fixes: #6056

When removing a bucket metadata entry we first unlink the bucket
and then we remove the bucket entrypoint object. Originally
when unlinking the bucket we first overwrote the bucket entrypoint
entry marking it as 'unlinked'. However, this is not really needed
as we're just about to remove it. The original version triggered
a bug, as we needed to propagate the new header version first (which
we didn't do, so the subsequent bucket removal failed).

Reviewed-by: Greg Farnum <greg@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
(cherry picked from commit 0373d749cea7d9b532069ba8ebca2f005b2c9f59)
OSD: allow project_pg_history to handle a missing map

If we get a peering message for an old map we don't have, we
can throwit out: the sending OSD will learn about the newer
maps and update itself accordingly, and we don't have the
information to know if the message is valid. This situation
can only happen if the sender was down for a long enough time
to create a map gap and its PGs have not yet advanced from
their boot-up maps to the current ones, so we can rely on it

Fixes: #6712
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit cd0d612e1abdf5c87082eeeccd4ca09dd14fd737)
init, upstart: prevent daemons being started by both

There can be only one init system starting a daemon. If there is a
host entry in ceph.conf for a daemon, sysvinit would try to start it
even if the daemon's directory did not include a sysvinit file. This
preserves backwards compatibility with older installs using sysvinit,
but if an upstart file is present in the daemon's directory, upstart
will try to start them, regardless of host entries in ceph.conf.

If there's an upstart file in a daemon's directory and a host entry
for that daemon in ceph.conf, both sysvinit and upstart would attempt
to manage it.

Fix this by only starting daemons if the marker file for the other
init system is not present. This maintains backwards compatibility
with older installs using neither sysvinit or upstart marker files,
and does not break any valid configurations. The only configuration
that would break is one with both sysvinit and upstart files present
for the same daemon.

Backport: emperor, dumpling
Reported-by: Tim Spriggs <tims@uahirise.org>
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
(cherry picked from commit 5e34beb61b3f5a1ed4afd8ee2fe976de40f95ace)
osd/OSDMonitor: accept 'osd pool set ...' value as string

Newer monitors take this as a CephString.  Accept that so that if we are
mid-upgrade and get a forwarded message using the alternate schema from
a future mon we will handle it properly.

Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
rgw: do not use UNPRIVILEGED_DAEMON option

This is sort of a backport for 4f403c26dc0048ad63e20d20369fa86bfb31c50e
in that we get the same behavior change, but only for rgw.

Signed-off-by: Sage Weil <sage@inktank.com>
osd_types: add osd_perf_stat_t type

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 95f3353a6ab3a4dd2bf28eaca7122487942e913e)
ObjectStore: add ObjectStore::get_cur_stats and impl in FileStore

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit ebde89d5602536b4bc651737e4280fdfb6634c32)
OSDMonitor: add 'osd perf' command to dump recent osd perf information

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 94c3f29a32cbf169d896015da6765febd3c724e0)
rgw: turn swift COPY into PUT

Fixes: #6606
The swift COPY operation is unique in a sense that it's a write
operation that has its destination not set by the URI target, but by a
different HTTP header. This is problematic as there are some hidden
assumptions in the code that the specified bucket/object in the URI is
the operation target. E.g., certain initialization functions, quota,
etc. Instead of creating a specialized code everywhere for this case
just turn it into a regular copy operation, that is, a PUT with
a specified copy source.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
(cherry picked from commit cbf8f9a51737e7d89fb0ec0832b50fd47c35b08d)
rgw: get rid of req_state.bucket_name

No need for this field, as we already have req_state.bucket_name_str.
This saves us some memory allocation / freeing through every request
processing.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
(cherry picked from commit db7eb771226beaa2372ef5860ce742cb3457cd89)

Conflicts:
	src/rgw/rgw_rest_s3.cc
rgw: init src_bucket_name, src_object in the S3 handler

Be consistent and initialize these fields also in the S3 case.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
(cherry picked from commit ccaab2abf9e49ce3e50c15df793a3a3ca6b29bb8)
Validate S3 tokens against Keystone

- Added config option to allow S3 to use Keystone auth
- Implemented JSONDecoder for KeystoneToken
- RGW_Auth_S3::authorize now uses rgw_store_user_info on keystone auth
- Minor fix in get_canon_resource; dout is now after the assignment

Reviewed-by: Yehuda Sadeh<yehuda@inktank.com>
Signed-off-by: Roald J. van Loon <roaldvanloon@gmail.com>
(cherry picked from commit a200e184b15a03a4ca382e94caf01efb41cb9db7)

Conflicts:
	src/rgw/rgw_swift.h
rgw: Allow wildcard in supported keystone roles.

http://tracker.ceph.com/issues/4365 fixes #4365

Signed-off-by: Christophe Courtaut <christophe.courtaut@gmail.com>
(cherry picked from commit 60195581273aee259e8c83d0b471af859d928342)
rgw: don't return data within the librados cb

Fixes: #7030
The callback is running within a single Finisher thread, thus we
shouldn't block there. Append read data to a list and flush it within
the iterate context.

Reviewed-by: Sage Weil <sage@inktank.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
(cherry picked from commit d6a4f6adfaa75c3140d07d6df7be03586cc16183)
rgw: fix use-after-free when releasing completion handle

Backport: emperor, dumpling
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
(cherry picked from commit c8890ab2d46fe8e12200a0d2f9eab31c461fb871)
v0.67.5
Merge pull request #808 from ceph/wip-6152-dumpling

rgw: Fix S3 auth when using response-* query string params
objecter: don't resend paused ops

Paused ops are meant to block on the client side until a new map that
unpauses them is recieved. If we send paused writes when the FULL flag
is set, we'll get -ENOSPC from the osds, which is not what Objecter
users expect. This may cause rbd without caching to produce an I/O
error instead of waiting for the cluster to have capacity.

Fixes: #6725
Backport: dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
(cherry picked from commit c5c399d327cfc0d232d9ec7d49ababa914d0b21a)