rgw: fix rgw_dir_suggest_changes() to set header_changed

Apparently we weren't setting header_changed to true in the
case where we handled the CEPH_RGW_UPDATE case and cur_disk.exists
was false. In practice what this created is that in case where
object was created but the index complete call failed (or timed
out), calling rgw_dir_suggest_changes() fixed the entry, however,
we didn't account the new entry. This would lead to negative
stats on the bucket index.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
cls_rgw: add client api calls for index suggest

This is needed for unitest creation.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
cls_rgw: client api for bucket index init

create a client api function for bucket index init.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
test/rgw: add bucket index unitest

Tests objects creation, removal, prepare, complete

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
cls_rgw: configurable tag timeout

we'll be using that for bucket index unitest

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
test/rgw: add a test for index suggest

test the bucket index suggest api

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
cls_rgw: change scoping of suggested changes vars

Fixes: #3127
Bad variable scoping made it so that specific variables
weren't initialized between suggested changes iterations.
This specifically affected a case where in a specific
change we had an updated followed by a remove, and the
remove was on a non-existent key (e.g., was already
removed earlier). We ended up re-substracting the
object stats, as the entry wasn't reset between
the iterations (and we didn't read it because the
key didn't exist).

backport:argonaut
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: set exists flag when suggesting existing bucket entry

We weren't setting the 'exists' flag on the bucket entry,
so we ended up not updating the index correctly.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: fix suggested object size

We can't just stat the head object, we need to go through
get_obj_state() to get obj attributes.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
cls_rgw: update index stats based on correct category

only refer to the on-disk category if the object existed.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
cls_rgw: complete_op() only skip update if epoch is non zero

An update shouldn't be skipped if epoch is zero. We'd see a zero
epoch if we tried to read an object and it didn't exist. That
could happen e.g., when a delete object operation failed to
call the complete earlier, and now we're recalling delete on
the (now non-existent object).
However, note that the zero epoch is racy. We may end up racing
with an object creation. This will be taken care of by a new
rados change that will set the returned object version even if
it didn't exist.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: obj_stat() returns object version (epoch), use it

We now pass the object version returned by obj_stat. We use that
epoch for setting the object version through the index suggestion
mechanism. This was broken by a recent change that switched from
reading the obj stats by (wrongly) calling directly to ioctx->stat()
to calling get_obj_state().

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw, cls_rgw: a tool to rebuild bucket stats

radosgw-admin bucket check [--fix] --bucket=<bucket>

The command will dump the existing bucket header stats,
and the calculated bucket header stats. If --fix is provided
the bucket stats will be overwritten by the recalculated
stats.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Merge branch 'wip-3127'

Conflicts:
	src/rgw/rgw_rados.cc

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>