rgw: fix rgw_dir_suggest_changes() to set header_changed

Apparently we weren't setting header_changed to true in the
case where we handled the CEPH_RGW_UPDATE case and cur_disk.exists
was false. In practice what this created is that in case where
object was created but the index complete call failed (or timed
out), calling rgw_dir_suggest_changes() fixed the entry, however,
we didn't account the new entry. This would lead to negative
stats on the bucket index.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
:doc: Title change for clarity. Initial formatting cleanup.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
doc/config-cluster/*: update recently added/changed config options

Signed-off-by: Sage Weil <sage@inktank.com>
cls_rgw: add client api calls for index suggest

This is needed for unitest creation.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
cls_rgw: client api for bucket index init

create a client api function for bucket index init.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
cpeh.spec.in:  Don't package boto_tool.
test/rgw: add bucket index unitest

Tests objects creation, removal, prepare, complete

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
cls_rgw: configurable tag timeout

we'll be using that for bucket index unitest

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Merge branch 'master' of https://github.com/ceph/ceph
test/rgw: add a test for index suggest

test the bucket index suggest api

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
cls_rgw: change scoping of suggested changes vars

Fixes: #3127
Bad variable scoping made it so that specific variables
weren't initialized between suggested changes iterations.
This specifically affected a case where in a specific
change we had an updated followed by a remove, and the
remove was on a non-existent key (e.g., was already
removed earlier). We ended up re-substracting the
object stats, as the entry wasn't reset between
the iterations (and we didn't read it because the
key didn't exist).

backport:argonaut
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: set exists flag when suggesting existing bucket entry

We weren't setting the 'exists' flag on the bucket entry,
so we ended up not updating the index correctly.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: fix suggested object size

We can't just stat the head object, we need to go through
get_obj_state() to get obj attributes.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
cls_rgw: update index stats based on correct category

only refer to the on-disk category if the object existed.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
cls_rgw: complete_op() only skip update if epoch is non zero

An update shouldn't be skipped if epoch is zero. We'd see a zero
epoch if we tried to read an object and it didn't exist. That
could happen e.g., when a delete object operation failed to
call the complete earlier, and now we're recalling delete on
the (now non-existent object).
However, note that the zero epoch is racy. We may end up racing
with an object creation. This will be taken care of by a new
rados change that will set the returned object version even if
it didn't exist.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: obj_stat() returns object version (epoch), use it

We now pass the object version returned by obj_stat. We use that
epoch for setting the object version through the index suggestion
mechanism. This was broken by a recent change that switched from
reading the obj stats by (wrongly) calling directly to ioctx->stat()
to calling get_obj_state().

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
:doc: Remove legacy usage. Initial format cleanup.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
:doc: Added discussion and stack diagram. Initial format cleanup.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
:doc: Made notice to stop i/o a warning. Initial format clean up.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>
:doc: Changed title to RBD from RADOS. Clearer.

Signed-off-by: John Wilkins <john.wilkins@inktank.com>