rgw: skip read_policy checks for system_users

A system user should still be able to examine suspended buckets, and
get -ENOENT instead of -EACCESS for a deleted object.

Fixes: #6616
Backport: dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
cls_log: always return final marker from log_list

There's no reason to restrict returning the marker to the case where
less than the whole log is returned, since there's already a truncated
flag to tell the client what happened.

Giving the client the last marker makes it easy to consume when the
log entries do not contain their own marker. If the last marker is not
returned, the client cannot get the last marker without racing with
updates to the log.

Backport: dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
rgw: include marker and truncated flag in data log list api

Consumers of this api need to know their position in the log. It's
readily available when fetching the log, so return it.  Without the
marker in this call, a client could not easily or efficiently figure
out its position in the log, since it would require getting the global
last marker in the log, and then reading all the log entries.

This would be slow for large logs, and would be subject to races that
would cause potentially very expensive duplicate work.

Returning this atomically while fetching the log entries simplifies
all of this.

Fixes: #6615
Backport: dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
rgw: update metadata log list to match data log list

Send the last marker whether the log is truncated in the same format
as data log list, so clients don't have more needless complexity
handling the difference.  Keep bucket index logs the same, since they
contain the marker already, and are not used in exactly the same way
metadata and data logs are.

Backport: dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
rgw: move url escaping to a common place

This is useful outside of the s3 interface. Rename url_escape()
url_encode() for consistency with the exsting common url_decode()
function. This is in preparation for the next commit, which needs
to escape url-unsafe characters in another place.

Backport: dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
rgw: escape bucket and object names in StreamReadRequests

This fixes copy operations for objects that contain unsafe characters,
like a newline, which would return a 403 otherwise, since the GET to
the source rgw would be unable to verify the signature on a partially
valid bucket name.

Fixes: #6604
Backport: dumpling
Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
radosgw-admin: remove unused function escape_str()

This was added before formatters were used for dumping logs.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
rgw: eliminate one unnecessary case statement

0x21 '!' is the first character that doesn't need encoding, so we can
expand the lower bound check.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Merge branch 'wip-rgw-sync-next' into next

Reviewed-by: Yehuda Sadeh <yehuda@inktank.com>