rgw: clear bufferlist if write_data() successful

Fixes: #9201
Backport: firefly

We sometimes need to call RGWPutObjProcessor::handle_data() again,
so that we send the pending data. However, we failed to clear the buffer
that was already sent, thus it was resent. This triggers when using non
default pool alignments.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 9181114d6f6062c55ee4b351fc3495345e545c36)
rgw: copy_obj_data() uses atomic processor

Fixes: #9089

copy_obj_data was not using the current object write infrastructure,
which means that the end objects weren't striped.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 800eff24824c0083b8e2441fc34e0bdca5da36dc)
rgw: disable civetweb url decoding

Fixes: #8621

We want to have the raw request uri, as we do the decoding ourselves.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit ffac52b316e7022796d44ae58804d9c20b9c3df9)
rgw: call throttle_data() even if renew_state() failed

Otherwise we're going to leak the aio callback handle.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 7998c322179dd69a1250937321c3c2bb023e0e57)
rgw: fix compilation

RGWRadosPutObj couldn't refer to the ceph context.

Reviewed-by: Sage Weil <sage@redhat.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 6a555434ee3edaf742ee7e5910bcba8dd0de46dd)
rgw: don't try to authenticate a CORS preflight request

Fixes: #8718
Backport: firefly

CORS preflight requests don't need to be authenticated. Treat them as
coming from anonymous user.

Reported-by: Robert Hubbard <bobby.hubbard@garmin.com>
Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 848fcf7871e07fc689bdcd18943ace36b2f4906e)
rgw: fix test to identify whether object has tail

Fixes: #9226
Reported-by: Sylvain Munaut <s.munaut@whatever-company.com>
Backport: firefly

We need to identify whether an object is just composed of a head, or
also has a tail. Test for pre-firefly objects ("explicit objs") was
broken as it was just looking at the number of explicit objs in the
manifest. However, this is insufficient, as we might have empty head,
and in this case it wouldn't appear, so we need to check whether the
sole object is actually pointing at the head.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 751b3e26532932a42ca34f9c062a0a3e29a58cff)
rgw: convert header field underscores into dashes

Fixes: 9206
Backport: firefly

Certain web servers filter out underscores in the header field name.
Convert them into dashes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 11acb7097ce21c6218dd48d0c21e0e04a361eb9a)
civetweb: update submodule

Update submodule to include multiple fixes.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
rgw: separate civetweb log from rgw log

The civetweb log now is independent from the rgw log.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 850242cad749e33e1e6bc008baa75c8ea7eda0c1)

Conflicts:
	src/civetweb
	src/rgw/rgw_main.cc
rgw: push hash calculater deeper

This might have been the culprit for #9307. Before we were calculating
the hash after the call to processor->handle_data(), however, that
method might have spliced the bufferlist, so we can't be sure that the
pointer that we were holding originally is still invalid. Instead, push
the hash calculation down. Added a new explicit complete_hash() call to
the processor, since when we're at complete() it's too late (we need to
have the hash at that point already).

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit d41c3e858c6f215792c67b8c2a42312cae07ece9)

Conflicts:
	src/rgw/rgw_rados.h
crypto: don't hash zero sized buffer

libnss returns an error and we assert in that case.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 7b137246b49a9f0b4d8b8d5cebfa78cc1ebd14e7)
rgw: calculate hash after writing data

Since data is written asynchronously, we should do the hash calculation
while it's pending.

Signed-off-by: Yehuda Sadeh <yehuda@redhat.com>
(cherry picked from commit 5bb94ede19a50543a02a8019ed6c9680b3852d4e)
ReplicatedPG: clean out completed trimmed objects as we go

Also, explicitely maintain a max number of concurrently trimming
objects.

Fixes: 9113
Backport: dumpling, firefly, giant
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: do not queue the snap trimmer constantly

Previously, we continuously requeued the snap trimmer while in
TrimmingObjects.  This is not a good idea now that we try to
limit the number of snap trimming repops in flight and requeue
the snap trimmer directly as those repops complete.

Fixes: #9113
Backport: giant, dumpling, firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: dump snap_trimq on pg query

Signed-off-by: Samuel Just <sam.just@inktank.com>
common: Add cctid meta variable

Fixes: #6228
Signed-off-by: Adam Crume <adamcrume@gmail.com>
(cherry picked from commit bb45621cb117131707a85154292a3b3cdd1c662a)
doc/release-notes: v0.80.6

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit c0dc3a56974a469b61523b67cc032cc5726a3a5f)

Conflicts:

	doc/release-notes.rst
doc/release-notes: fix attributions for 8702 fix

Oops!

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 188370a94353e29fcb8981699022803e23f3fedd)
Locker: accept ctime updates from clients without dirty write caps

The ctime changes any time the inode does. That can happen even without
the file itself having changed, so we'd better accept the update whenever
the auth caps have dirtied, without worrying about the file caps!

Fixes: #9514
Backport: firefly

Signed-off-by: Greg Farnum <greg@inktank.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 0ea20a668cf859881c49b33d1b6db4e636eda18a)