tests: mon id is incorrectly skipped

The argument is shifted and $1 should be used to get the mon id instead
of $2.

http://tracker.ceph.com/issues/9836 Fixes: #9836

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
cli: add verbose information related to argument parsing

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
cli: do not parse injectargs arguments twice

The arguments of injectargs being valid ceph arguments, they are.
consumed when the ceph cli calls rados.conf_parse_argv(). It can be
worked around by obsuring them as in:

   ceph tell osd.0 injectargs '--osd_debug_drop_ping_probability 444'

where '--osd_debug_drop_ping_probability 444' is a single argument that
does not match any known argument. The trick is that it will be
evaluated again once it reaches the OSD or the MON and translated into
the expected list of arguments. Although it is clear once explained, it
is obscure and leads to strange combinations such as:

   ceph tell osd.0 injectargs '--osd_debug_op_order '

(note the extra space at the end) to set boolean parameters. A better
workaround is to add a -- marking the end of the options as in:

   ceph tell osd.0 -- injectargs --osd_debug_op_order

this one is unfortunately much less documented and the user does not
usually know the exact semantic of --, let alone where it should be
placed.

The simpler solution is to split the argument list in two if
"injectargs" is found. The arguments that show after the "injectargs"
argument is removed from the list of arguments until parsing is
complete. It implements the more intuitive syntax:

   ceph tell osd.0 injectargs --osd_debug_op_order

and the other forms are still valid for backward compatibility.

http://tracker.ceph.com/issues/9372 Fixes: #9372

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
doc: update injectargs syntax

Modify the examples to use the simpler syntax, not involving -- or
grouping with quotes.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
cli: CEPH_ARGS must be before injectargs

It is incorrect to append the content of CEPH_ARGS to the argument list
when running injectargs. For instance if

  CEPH_ARGS='--log-file the.log' \
    ./ceph tell osd.0 injectargs --no-osd_debug_op_order

translates into

    ./ceph tell osd.0 injectargs --no-osd_debug_op_order \
                                 --log-file the.log

it ends up changing the log file of osd.0 which is probably unintended.
Instead CEPH_ARGS is inserted before injectargs and it translates into:

    ./ceph tell osd.0 --log-file the.log \
                      injectargs --no-osd_debug_op_order

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
common: remove dead code in buffer.cc

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
common: add an aligned buffer with less alignment than a page

SIMD optimized erasure code computation needs aligned memory. Buffers
aligned to a page boundary are not needed though. The buffers used
for the erasure code computation are typical smaller than a page.

The typical alignment requirements SIMD operations are 16 bytes for
SSE2 and NEON and 32 bytes for AVX/AVX2.

Add new prototypes with an align argument, similar to the one enforcing
page alignment. The implementation is exactly the same, except for the
align parameter. The page alignment method are then implemented as calls
to the more generic methods.

The align parameter is an unsigned (same type as CEPH_PAGE_SIZE). The
CEPH_PAGE_MASK value ( ~(CEPH_PAGE_SIZE - 1) ) was only used as
~CEPH_PAGE_MASK, i.e. equivalent of (CEPH_PAGE_SIZE - 1) once the double
~~ is reduced. These occurrence are replaced with (align - 1). The type
of CEPH_PAGE_MASK is an unsigned long which probably because it was
~(CEPH_PAGE_SIZE). When using (align - 1) as a mask for both
CEPH_PAGE_SIZE and SIMD alignment there is no need to use an unsigned
long because there is no risk of overflowing the unsigned value.

The CYGWIN specific code is also modified but not tested.

Unit tests are added for the new methods.

Signed-off-by: Janne Grunau <j@jannau.net>
Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure code: use a function for the chunk mapping index
erasure code: use 32-byte aligned buffers

Requiring page aligned buffers and realigning the input if necessary
creates measurable oberhead. ceph_erasure_code_benchmark is between
10-20% faster depending on the workload.

Also prevents a misaligned buffer when bufferlist::c_str(bufferlist)
has to allocate a new buffer to provide continuous one. See bug #9408

Signed-off-by: Janne Grunau <j@jannau.net>
ceph_erasure_code_benchmark: use 32-byte aligned input

The benchmark is supposed to measure the encoding/decoding speed and
not the overhead of buffer realignments.

Signed-off-by: Janne Grunau <j@jannau.net>
erasure-code: expose ErasureCode::SIMD_ALIGN as a const

For test purposes and it will also be useful for plugins that must
ensure the chunk size is a multiple of SIMD_ALIGN.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: add ErasureCode::encode unit test

Re-create and describe the situation that is fixed by
91a7e18f60bbc9acab3045baaa1b6505474ec4a9 which reworks the buffer
preparation function provided by ErasureCode::encode.

http://tracker.ceph.com/issues/9408 Refs: #9408

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: use ErasureCode::SIMD_ALIGN in ceph_erasure_code_benchmark

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
objecter: Unlock in shutdown before waiting for timer thread

Fixes: #9845

Reviewed-by: Sage Weil <sage@redhat.com>
Signed-off-by: David Zafman <dzafman@redhat.com>
PG:: reset_interval_flush and in set_last_peering_reset

If we have a change in the prior set, but not in the up/acting set, we go back
through Reset in order to reset peering state.  Previously, we would reset
last_peering_reset in the Reset constructor.  This did not, however, reset the
flush_interval, which caused the eventual flush event to be ignored and the
peering messages to not be sent.

Instead, we will always reset_interval_flush if we are actually changing the
last_peering_reset value.

Fixes: #9821
Backport: firefly
Signed-off-by: Samuel Just <sam.just@inktank.com>
Make better error reporting for client's call to rados.Rados

Signed-off-by: Derrick Schneider derrick.schneider@gmail.com
mon: Monitor: MMonGetMap doesn't require caps

We are dropping the requirement for MON_CAP_R for MMonGetMap.

Reason is simple enough: clients may need to contact the monitors and
obtain the latest monmap before authenticating.  This happens, for
instance, when a client calls MonClient::get_monmap_privately().  The
osd uses this function during mkfs, prior to initializing a keyring or
even so much as existing.

Fixes: #9859

Signed-off-by: Joao Eduardo Luis <joao@redhat.com>
Merge pull request #2777 from ceph/wip-9859

mon: Monitor: MMonGetMap doesn't require caps

Reviewed-by: Sage Weil <sage@redhat.com>
qa: use sudo when rsyncing /usr so we can read everything

Signed-off-by: Greg Farnum <greg@inktank.com>
qa: use sudo when rsyncing /usr so we can read everything

Signed-off-by: Greg Farnum <greg@inktank.com>
(cherry picked from commit fa07c04231db2d130de54647957ffab4a7a53733)