rgw, radosgw-admin: bucket link uses bucket instance id now

Fixes: 7499
We need to link user to a specific bucket instance. This updates both
the radosgw-admin link command, and the RESTful admin api.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Changed the java code example
Update librados-intro.rst
Added Java Example
rgw: fix url escaping

Fixes: #8202
This fixes the radosgw side of issue #8202. Needed to cast value
to unsigned char, otherwise it'd get padded.

Backport: dumpling

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
(cherry picked from commit bcf92c496aba0dfde432290fc2df5620a2767313)
ReplicatedPG: do not preserve op context during flush

Any information stashed in the OpContext may be obsolete by the time we
actually mark the object clean.  Instead, let the start_flush caller
clean up its OpContext and in try_flush_mark_clean we'll create a new
one.  The primary reason to keep the OpContext would have been locking,
but we can set the obc as blocking without holding an OpContext, and
that would allow trimming to happen in the mean time (which is good
since trim_object does not respect rw locks since it doesn't change user
visible state).  In try_flush_mark_clean, we requeue the fop->op along
with (but ahead of) the fop->dup_ops.

Fixes: #8068
Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit d83b8f58513e86c68aa6c25d4f909d9a3c4a103e)
ReplicatedPG: we can get EAGAIN on missing clone flush

Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 060105c313c5b4a777c55f17115eeb95ebb17117)
ECBackend::continue_recovery_op: handle a source shard going down

get_min_avail_to_read_shards might return an error if there are
no longer enough sources to reconstruct the missing shards.
This is possible if osds went down while we were writing the
previous chunk -- we already notice in check_recovery_sources
if a source goes down during a read.

Fixes: #8161
Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 1885792c517670086332a8bab237c58558ee6dda)
rados.h,ReplicatedPG: add CEPH_OSD_FLAG_ENFORCE_SNAPC and use on flush

We need to ensure that even with pool snaps, we use the snapc provided in order
to ensure that the clones are written back correctly.

Fixes: #7941
Signed-off-by: Samuel Just <sam.just@inktank.com>
(cherry picked from commit 499adb1db1cd225c91acce31b5f48fad5145043b)
ReplicatedPG: block scrub on blocked object contexts

Fixes: #8011
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
(cherry picked from commit e66f2e36c06ca00c1147f922d3513f56b122a5c0)
mds: remove mdsdir in the final step of shutdown MDS

Otherwise we may get bad subtree map if we restart the MDS before
the shutdown process finishes.

Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Merge pull request #1751 from ceph/wip-mds-shutdown

mds: remove mdsdir in the final step of shutdown MDS

Reviewed-by: Sage Weil <sage@inktank.com>
osd: track per-pg epochs, min

Add some simple tracking so that we can quickly determine what the min
pg osdmap epoch is.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: prevent pgs from getting too far ahead of the min pg epoch

Bound the range of PG epochs between the slowest and fastest pg
(epoch-wise) with 'osd map max advance'.  This value should be set to
something less than 'osd map cache size' so that the maps we are
processing will be in memory as many PGs advance forward in time in
loose synchrony.

This is part of the solution to #7576.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: ignore MarkMeDown message if we aren't in PREPARING_TO_STOP state

If we aren't waiting for this, ignore it.

Signed-off-by: Sage Weil <sage@inktank.com>
osd: fix 'ack' to be 'request_ack' in MOSDMarkMeDown

This field was passed along but always set to false.  It did not seem to
indicate whether this was/wasn't an ack (the reply message kept the same
value) or whether an ack was requested (it was always false but we always
reply).

Change this by bumping the message encoding, renaming the field, and
making old messages that are decoded force request_ack to true.  Reply
messages will always have request_ack false (although that part doesn't
matter much since the fact that it is a reply is clear from context).

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: do not reply to MOSDMarkMeDown if ack is not requested

If a reply isn't requested, do not bother to send one.  Note that old
clients did not request an ack, but we will infer that they do from the
message encoding (see previous commit).

Signed-off-by: Sage Weil <sage@inktank.com>
sample.ceph.conf update:

 * corrected URLs.
 * added [client] section.
 * more options and descriptions.
 * filestore settings were moved under [osd].
 * cephx settings to reflect lack of support for authentication in RBD client.
 * many minor corrections and updates.

Signed-off-by: Dmitry Smirnov <onlyjob@member.fsf.org>
mon/OSDMonitor: share latest map with osd on dup boot message

If we get a dup boot message, share the newer maps with the osd so that
they know they are living in the past.

Fixes: #8279
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
(cherry picked from commit 2e6b24868da0b203c2d70ac91071166d95d1d851)
mon/PGMonitor: set tid on no-op PGStatsAck

The OSD needs to know the tid.  Both generally, and specifically because
the flush_pg_stats may be blocking on it.

Fixes: #8280
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Joao Eduardo Luis <joao.luis@inktank.com>
(cherry picked from commit 5a6ae2a978dcaf96ef89de3aaa74fe951a64def6)