FileStore: Race condition during object delete is fixed

There was a race condition (hence OSD crash) between lfn_unlink
and lfn_open. The reason was FDCache lookup was called without
taking index lock from lfn_open. Lookup will increase reference
count and thus Clear will not be able to delete those FDs. FDs
will be leaked. The assert within FDCache clear was hitting
because of this.

Fixes: #9480

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
osd: drop dead shutdown message handler

There is no code anywhere in the code base that generates a MSG_SHUTDOWN.
Kill this.  The get_priv() logic here looks pretty dodgey anyway.

Signed-off-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
osd/ReplicatedPG: handle sessionless op con in do_osd_op_effects

Signed-off-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
osd/Watch: handle con without sessions

Signed-off-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <greg@inktank.com>
Pipe: stop delayed delivery fast_dispatch in stop_and_wait()

If we don't, we can keep fast_dispatching messages after a Pipe has been
mark_down()ed. That breaks things right now.

Fixes: #9295

Signed-off-by: Greg Farnum <greg@inktank.com>
Objecter: disable fast dispatch of CEPH_MSG_OSD_OPREPLY messages

If they fast_dispatch they can deadlock right now. We'll need to change
the semantics around mark_down() or around the Objecter's locking to
fix it.

Sort-of-
Fixes: #9462

Signed-off-by: Greg Farnum <greg@inktank.com>
os/FileJournal: do not request sync while shutting down

FileStore calls should_commit_now() to determine whether it should
loop and do a second sync (among other things).  During shutdown, this
can force us into a livelock: the journal is shutting down, but the
sync_entry loop never completes and repeatedly syncs because the
journal is full.  Since the journal is otherwise stopped, no expire
happens and we never become unfull, and we're stuck.

This seems to be triggered semi-reliably by the ceph_objectstore_tool
import function.

Fix by not requesting a sync while shutting down.

Fixes: #9545
Signed-off-by: Sage Weil <sage@redhat.com>
ceph_test_rados_api_tier: add EvictSnap2 test case

Verify an evict doesn't create a snapdir object.  Reproduces #8629

Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: do not clone or preserve snapdir on cache_evict

If we cache_evict a head in a cache pool, we need to prevent
make_writeable() from cloning the head and finish_ctx() from
preserving the snapdir object.

Fixes: #8629
Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
PG: check full ratio again post-reservation

Otherwise, we might queue 30 pgs for backfill at 0.80 fullness
and then never check again filling the osd after pg 11.

Fixes: #9574
Backport: dumpling, firefly, giant
Signed-off-by: Samuel Just <sam.just@inktank.com>
ReplicatedPG: clean out completed trimmed objects as we go

Also, explicitely maintain a max number of concurrently trimming
objects.

Fixes: 9113
Backport: dumpling, firefly, giant
Signed-off-by: Samuel Just <sam.just@inktank.com>
erasure-code: [ISA] modify get_alignment function to imply a platform/compiler independent alignment constraint of 32-byte aligned buffer addresses & length
erasure-code: isa uses per chunk alignment constraints

Copy code from the jerasure plugin to enforce alignment constraints per
chunk instead of using the total object size. It is simpler and reduces
the size of the chunks. See
https://github.com/ceph/ceph/commit/c7daaaf5e63d0bd1d444385e62611fe276f6ce29
for more information.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: do not use typed tests for isa

Because there only is one type.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: run isa tests via libtool and valgrind

Because running valgrind with no libtool does not test the binary but
the enclosing shell script.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: isa test compare chunks with memcmp instead of strncmp

Because they may contain null characters.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: isa encode tests adapted to per chunk alignment

The encode tests use the alignment constraints. It has been changed to
be aligned on a per chunk basis instead of computing a more expensive
object alignement constraint. The test function is modified to take the
change into account but the logic is otherwise unmodified.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
msg: allow calling dtor immediately after ctor

Asserting on reaper_stop only made sense if the
messenger had ever been started: as it stood,
one couldn't create and destroy a messenger
without also starting and stopping it.

Signed-off-by: John Spray <john.spray@redhat.com>
erasure-code: add test for isa chunk_size method

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
erasure-code: test isa encode/decode with various object sizes

Create an encode_decode() helper method to be called from the
encode_decode test function with various object size arguments. The
helper method is a copy/paste of the previous test that was using a
single object of a fixed size. The test is slightly adapted to
accommodate for different object sizes but the logic is not modified.

The object sizes being tested are chosen to be under the size of the
required size alignment or on multiple pages, size aligned or not.

Signed-off-by: Loic Dachary <loic-201408@dachary.org>