erasure-code: store and compare encoded contents

Introduce ceph_erasure_code_non_regression to check and compare how an
erasure code plugin encodes and decodes content with a given set of
parameters. For instance:

./ceph_erasure_code_non_regression \
      --plugin jerasure \
      --parameter technique=reed_sol_van \
      --parameter k=2 \
      --parameter m=2 \
      --stripe-width 3181 \
      --create \
      --check

Will create an encoded object (--create) and store it into a directory
along with the chunks, one chunk per file. The directory name is derived
from the parameters. The content of the object is a random pattern of 31
bytes repeated to fill the object size specified with --stripe-width.

The check function (--check) reads the object back from the file,
encodes it and compares the result with the content of the chunks read
from the files. It also attempts recover from one or two erasures.

Chunks encoded by a given version of Ceph are expected to be encoded
exactly in the same way by all Ceph versions going forward.

http://tracker.ceph.com/issues/9420 Refs: #9420

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: workunit to check for encoding regression

Clone the archive of encoded objects and decode all archived objects, up
to and including the current ceph version.

http://tracker.ceph.com/issues/9420 Refs: #9420

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: s/alignement/alignment/ typos in jerasure

The jerasure-per-chunk-alignment prameter was mispelled and while
useable that would lead to confusion.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: Makefile.am cosmetics

Cluster benchmark related lines together.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: add corpus verification to make check

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: ceph_erasure_code does not need to avoid dlclose

The only reason for not dlclosing plugins at exit is for callgrind but
ceph_erasure_code has no workload that would require callgrind.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure_code: implement ceph_erasure_code to assert the existence of a plugin

This is handy when scripting in the context of teuthology and only
conditionally run tests for the isa plugin, for instance.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: add erasure_code_benchmark --verbose

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: erasure_code_benchmark exhaustive erasure exploration

Add the --erasure-generation exhaustive flag to try all combinations of
erasures, not just one at random.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
tools: skip up to expire_pos in journal-tool

Previously worked for journals starting from an
object boundary (i.e. freshly created filesystems)

Fixes: #9977
Signed-off-by: John Spray <john.spray@redhat.com>
osdc: fix Journaler write error handling

Since we started wrapping the write error
handler in a finisher, multiple calls to
handle_write_error would hit the assert()
on the second call before the actual
handler had been called (at the other end
of the finisher) from the first call.

The symptom was that the MDS was intermittently
failing to respawn on blacklist, seen in #10011.

Signed-off-by: John Spray <john.spray@redhat.com>
Merge pull request #2853 from ceph/wip-9977

tools: skip up to expire_pos in journal-tool

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Merge pull request #2870 from ceph/wip-blacklist-respawn

osdc: fix Journaler write error handling

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Merge remote-tracking branches 'dachary/wip-9420-erasure-code-non-regression', 'upstream/master', 'upstream/wip-6756', 'upstream/wip-9031-9262', 'upstream/wip-9852', 'upstream/wip-9887', 'upstream/wip-9986', 'upstream/wip-9987', 'upstream/wip-aio-journal', 'upstream/wip-log-client', 'upstream/wip-warn-max-pg' and 'yuandong1222/avoid-unused-pg_to_acting_osds-call' into wip-sam-testing