osd/osd_types: add num_bytes_hit_set_archive to stats

The tiering agent's life is easier/better if it knows how many bytes these
objects consume (instead of making a conservative guess).

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: account for hit_set_archive bytes

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: scrub and repair hit_set_archive bytes

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: calculate num_user_bytes

This is the bytes stored by the user, excluding hit_sets.

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: adjust avg_size calculation in agent_choose_mode

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
erasure-code: refactor jerasure Liberation parameters checks

Create a virtual function to check for each parameter for Liberation
and its derivations Liber8tion and BlaumRoth. The code is moved around
but is unmodified.

A function reseting the parameters to sane defaults when an error
occurs is introduced. It is slightly different from the code that was
duplicated in Liberation and BlaumRoth. In both cases it now resets w.
It is a noop for Liber8tion because the user cannot modify this
parameter which is always set to the default.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: fix BlaumRoth sanity check on w

For the BlaumRoth technique, w+1 must be a prime. This sanity check is
present in the sample code from the jerasure library but was not
implemented in the jerasure plugin.

The default value for w is also changed to match this constraint. The
unit tests is changed to always use the default w instead of forcing it
to w.

http://tracker.ceph.com/issues/6754 Fixes: #6754

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Objecter::_recalc_linger_op: resend for any acting set change

Fixes: #9220
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
FileStore: report l_os_j_lat as commit latency

l_os_commit_lat is actually the commit cycle latency.

Fixes: #9269
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
osdc/Objecter: drop bad session nref assert

This is a bad assert.  Specifically, handle_osd_op_reply may still be
holding the session ref while it is calling the completion for a previous
request.  This is safe: it is only holding the session ref after it dropped
the global map rwlock because of the per-session completion locks.  The
request in question was already marked completed by the time our thread
took the session lock.

Fixes: #9241
Signed-off-by: Sage Weil <sage@redhat.com>
Change CrushWrapper::crush to private

Currently in CrushWrapper, the member "struct crush_map *crush"  is a public member,
so people can break the encapsulation and manipulate directly to the crush structure.

This is not a good practice for encapsulation and will lead to inconsistent if code
mix use the CrushWrapper API and crush C API.A simple example could be:
1.some code use crush_add_rule(C-API) to add a rule, which will not set the have_rmap flag to false in CrushWrapper
2.another code using CrushWrapper trying to look up the newly added rule by name will get a -ENOENT.

This patch move CrushWrapper::crush to private, together with three reverse map(type_rmap, name_rmap, rule_name_rmap)
and also change codes accessing the CrushWrapper::crush to make it compile.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
osd: set min_read_recency_for_promote to default 1 when doing upgrade

When upgrading from a build without the promotion on 2nd read feature,
should set min_read_recency_for_promote to the default value 1, instead
of 0.

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
Cache tiering: do not allow flush/evict when there are requests pending

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
EC-ISA: add intelligent table cache
EC-ISA: avoid usage of table cache lock outside the class implementation by introducing the setEncodingTable/setEncodingCoefficient methods
osdc/Objecter: fix leak of MStatfsReply

Signed-off-by: Sage Weil <sage@redhat.com>
[rgw][s3] Allow colon ':' in access key

When access key contains ':', e.g. `some_info:for_user',
authorization header looks like:

"AWS some_info:for_user:request_signature"

so `auth_str.find(':')` result in auth_id = "some_info",
auth_sign = "for_user:request_signature".

auth_str.rfind(':') solves this issue.

Signed-off-by: Roman Haritonov <reclosedev@gmail.com>
Merge pull request #2447 from reclosedev/s3_colon_in_access_key

[rgw][s3] Allow colon ':' in access key

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
Merge pull request #2451 from ceph/wip-osdc-leak

osdc/Objecter: fix leak of MStatfsReply

Reviewed-by: John Spray <john.spray@redhat.com>
erasure-code: mon, osd etc. depend on the plugins

Since the erasure code plugin version check has been introduced,
whenever a library/binary that can load plugin needs to be recompiled,
the erasure code plugins must also be considered. If the reason for
recompiling the library/binary is a new commit, the plugins will fail to
load.

The dependency is not based on source compilation and a shared library
dependency on liberasure-code.la is added instead. This library is
uniformly used whenever a plugin is to be loaded and therefore covers
all library/binaries that need it.

http://tracker.ceph.com/issues/9413 Fixes: #9413

Signed-off-by: Loic Dachary <loic-201408@dachary.org>