erasure-code: parse function for the mapping parameter

Each D letter is a data chunk. For instance:

    _DDD_DDD

is going to parse into:

   [ 1, 2, 3, 5, 6, 7 ]

the 0 and 4 positions are not used by chunks and do not show in the
mapping. Implement ErasureCode::parse to support a reasonable default
for the mapping parameter.

Signed-off-by: Loic Dachary <loic@dachary.org>
erasure-code: remap chunks if not sequential

If the remap vector is not empty, use it to figure out the sequence of
data chunks.

http://tracker.ceph.com/issues/9025 Fixes: #9025

Signed-off-by: Loic Dachary <loic@dachary.org>
mon/Paxos: verify all new peons are still contiguous at end of round

During the collect phase we verify that each peon has overlapping or
contiguous versions as us (and can therefore be caught up with some
series of transactions).  However, we *also* assimilate any new states we
get from those peers, and that may move our own first_committed forward
in time.  This means that an early responder might have originally been
contiguous, but a later one moved us forward, and when the round finished
they were not contiguous any more.  This leads to a crash on the peon
when they get our first begin message.

For example:

 - we have 10..20
 - first peon has 5..15
   - ok!
 - second peon has 18..30
   - we apply this state
 - we are now 18..30
 - we finish the round
   - send commit to first peon (empty.. we aren't contiguous)
   - send no commit to second peon (we match)
 - we send a begin for state 31
   - first peon crashes (it's lc is still 15)

Prevent this by checking at the end of the round if we are still
contiguous.  If not, bootstrap.  This is similar to the check we do above,
but reverse to make sure *we* aren't too far ahead of *them*.

Fixes: #9053
Signed-off-by: Sage Weil <sage@redhat.com>
mon/Paxos: share state and verify contiguity early in collect phase

We verify peons are contiguous and share new paxos states to catch peons
up at the end of the round.  Do this each time we (potentially) get new
states via a collect message.  This will allow peons to be pulled forward
and remain contiguous when they otherwise would not have been able to.
For example, if

  mon.0 (leader)  20..30
  mon.1 (peon)    15..25
  mon.2 (peon)    28..40

If we got mon.1 first and then mon.2 second, we would store the new txns
and then boot mon.1 out at the end because 15..25 is not contiguous with
28..40.  However, with this change, we share 26..30 to mon.1 when we get
the collect, and then 31..40 when we get mon.2's collect, pulling them
both into the final quorum.

It also breaks the 'catch-up' work into smaller pieces, which ought to
smooth out latency a bit.

Signed-off-by: Sage Weil <sage@redhat.com>
lttng: Fix "make distcheck"

Signed-off-by: Adam Crume <adamcrume@gmail.com>
lttng: Add distro packaging

Signed-off-by: Adam Crume <adamcrume@gmail.com>
Merge pull request #2213 from dachary/wip-9025-chunk-remapping

erasure-code: chunk remapping

Reviewed-by: Samuel Just <sam.just@inktank.com>
crush: add is_v[23]_rule(ruleid) methods

Add methods to check if a *specific* rule uses v2 or v3 features.  Refactor
the existing checks to use these.

Signed-off-by: Sage Weil <sage@redhat.com>
osd: only require crush features for rules that are actually used

Often there will be a CRUSH rule present for erasure coding that uses the
new CRUSH steps or indep mode.  If these rules are not referenced by any
pool, we do not need clients to support the mapping behavior.  This is true
because the encoding has not changed; only the expected CRUSH output.

Fixes: #8963
Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
Remove Old Wireshark Dissectors

Remove the two old Wireshark plugins.  They do not build and are
superseded by the dissector which is inside Wireshark.

Signed-Off-By: Kevin Cox <kevincox@kevincox.ca>
Merge pull request #2070 from somnathr/wip-sd-filestore-optimization

Wip sd filestore optimization

Reviewed-by: Samuel Just <sam.just@inktank.com>
Merge pull request #2266 from kevincox/removewirehsark

Remove Old Wireshark Dissectors

Reviewed-by: Sage Weil <sage@redhat.com>
unittest_osdmap: test EC rule and pool features

TODO: tiering feature bits.

Signed-off-by: Sage Weil <sage@redhat.com>
Merge pull request #2264 from ceph/wip-crush-features

do not require crush features for rules that aren't being used

Reviewed-by: Loic Dachary <loic@dachary.org>
ReplicatedPG::cancel_copy: clear cop->obc

Otherwise, an objecter callback might still be hanging
onto this reference until after the flush.

Fixes: #8894
Introduced: 589b639af7c8834a1e6293d58d77a9c440107bc3
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG::maybe_handle_cache: do not forward RWORDERED reads

Even with READFORWARD, we can't forward RWORDERED reads.

Fixes: #9119
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge pull request #2268 from ceph/wip-9119

Wip 9119

Reviewed-by: Sage Weil <sage@redhat.com>
osd: fix feature requirement for mons

These features should be set on the client_messenger, not
cluster_messenger.

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
Fix -Wno-format and -Werror=format-security options clash

This causes build failure in latest fedora builds, ceph_test_librbd_fsx adds -Wno-format cflag but the default AM_CFLAGS already contain -Werror=format-security, in previous releases, this was tolerated but in the latest fedora rawhide it no longer is, ceph_test_librbd_fsx builds fine without -Wno-format on x86_64 so there is likely no need for the flag anymore

Signed-off-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>
Merge remote-tracking branch 'gh/next'