erasure-code: refactor jerasure Liberation parameters checks

Create a virtual function to check for each parameter for Liberation
and its derivations Liber8tion and BlaumRoth. The code is moved around
but is unmodified.

A function reseting the parameters to sane defaults when an error
occurs is introduced. It is slightly different from the code that was
duplicated in Liberation and BlaumRoth. In both cases it now resets w.
It is a noop for Liber8tion because the user cannot modify this
parameter which is always set to the default.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: fix BlaumRoth sanity check on w

For the BlaumRoth technique, w+1 must be a prime. This sanity check is
present in the sample code from the jerasure library but was not
implemented in the jerasure plugin.

The default value for w is also changed to match this constraint. The
unit tests is changed to always use the default w instead of forcing it
to w.

http://tracker.ceph.com/issues/6754 Fixes: #6754

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Change CrushWrapper::crush to private

Currently in CrushWrapper, the member "struct crush_map *crush"  is a public member,
so people can break the encapsulation and manipulate directly to the crush structure.

This is not a good practice for encapsulation and will lead to inconsistent if code
mix use the CrushWrapper API and crush C API.A simple example could be:
1.some code use crush_add_rule(C-API) to add a rule, which will not set the have_rmap flag to false in CrushWrapper
2.another code using CrushWrapper trying to look up the newly added rule by name will get a -ENOENT.

This patch move CrushWrapper::crush to private, together with three reverse map(type_rmap, name_rmap, rule_name_rmap)
and also change codes accessing the CrushWrapper::crush to make it compile.

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>
Cache tiering: do not allow flush/evict when there are requests pending

Signed-off-by: Zhiqiang Wang <wonzhq@hotmail.com>
EC-ISA: add intelligent table cache
EC-ISA: avoid usage of table cache lock outside the class implementation by introducing the setEncodingTable/setEncodingCoefficient methods
erasure-code: mon, osd etc. depend on the plugins

Since the erasure code plugin version check has been introduced,
whenever a library/binary that can load plugin needs to be recompiled,
the erasure code plugins must also be considered. If the reason for
recompiling the library/binary is a new commit, the plugins will fail to
load.

The dependency is not based on source compilation and a shared library
dependency on liberasure-code.la is added instead. This library is
uniformly used whenever a plugin is to be loaded and therefore covers
all library/binaries that need it.

http://tracker.ceph.com/issues/9413 Fixes: #9413

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
OpTracker: Sharding logic is implemented to improve performance

Enabling OpTracking is degrading performance significantly. The
main reason for this is the contention in the ops_in_flight_lock
during registration/unregistration. Sharding this lock and hence
the xlist (storing the ops) helped to boost the performance.
Sharding is been done based on the op seq number % num_shards.
atomic64_t is used as data type to protect the global sequence
number of OpTracker. Introduced a lock in the OpHistory to
protect its data structure as ealier it was protected by global
ops_in_flight_lock. This is only accessed during unregistration
and thus not contended.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: Removed _dump_op_descriptor_unlocked call for log level < 5

_dump_op_descriptor_unlocked call within the function _mark_event()
was invoked unnecessarily every time for all log levels. This was
degrading performance significantly! The stream is only used during
log level 5 and thus the _dump_op_descriptor_unlocked() is only called
now if user sets the log level to 5 and beyond.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: The TrackedOp::events list was not protected while dumping ops

OpTracker::Mutex::lock is there to protect the event list and it is
only used in the mark_event while pushing the events in the list.
It is not used while dumping the contents of the events list.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OpTracker: Race condition removed while dumping ops through admin socket

OSD was crashing due to a race condition while IO was going on and
user wants to dump in flight ops. This was happening because Message
data and payloads are removed before the op itself is removed from
the in flight list. Calling op->_unregistered() after removing
the op from the in flight list fixes the issue.

Fixes: #9384

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
Merge pull request #2409 from apeters1971/wip-ec-isa-table-cache-refac-master

EC-ISA: add intelligent table cache

Reviewed-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: fix erasure_code_benchmark goop

Using a stringstream that is only displayed on error when calling the
erasure code factory, instead of cerr. The user expects the output to be
clean when there is no error.

http://tracker.ceph.com/issues/9429 Fixes: #9429

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
ReplicatedPG: cancel cb on blacklisted watcher

Fixes: #8315
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
msg: prefetch data when doing recv()

Since that when we're reading the header we do lot's of small IOs, we
can really benefit from doing bigger reads.
Set a limit on the buffer size, so that it should mainly affect header
reads and not the actual data transfer of large IOs.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rbd-replay: Add --anonymize flag to rbd-replay-prep

Signed-off-by: Adam Crume <adamcrume@gmail.com>
librbd: Add missing tracepoints

Signed-off-by: Adam Crume <adamcrume@gmail.com>
rbd-replay: Add rbd-replay-many

Signed-off-by: Adam Crume <adamcrume@gmail.com>
rbd-replay: Add --dump-perf-counters option

Signed-off-by: Adam Crume <adamcrume@gmail.com>
rbd-replay: Add code documentation

Signed-off-by: Adam Crume <adamcrume@gmail.com>