common/Finisher: add queue(list<Context*>&)

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd_types: add pg_log_t::rollback_info_trimmed_to

Signed-off-by: Samuel Just <sam.just@inktank.com>
PGLog: initialize complete_to

Signed-off-by: Samuel Just <sam.just@inktank.com>
os/: add async flush_commit() method to Sequencer and implementations

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: add machinery to temporarily buffer outgoing recovery messages

Signed-off-by: Samuel Just <sam.just@inktank.com>
PG: block outgoing peering messages until the last interval is durable

Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge remote-tracking branch 'upstream/master' into wip-7999-master

Conflicts:
	src/os/KeyValueStore.h
PGLog: fix logging whitespace in trim

Signed-off-by: Samuel Just <sam.just@inktank.com>
PGLog: add rollback_info_trimmed_to_riter to IndexedLog and use in PG

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: plumb rollback_trim_to through interfaces and messages

We simply trim the roll back info up to min_last_complete_ondisk
since we cannot correctly rollback past that point anyway!

ReplicatedPG
-> PGBackend::submit_transaction
-> Backend messages
-> PGBackend::Listener::log_operation
-> append_log

Signed-off-by: Samuel Just <sam.just@inktank.com>
osd/: in scrub, check for and remove obsolete rollback objs

Signed-off-by: Samuel Just <sam.just@inktank.com>
Merge remote-tracking branch 'upstream/master' into wip-8438-master

Conflicts:
	src/osd/ECMsgTypes.cc
doc: dev: cache-pool.rst: fix typo s/objets/objects/

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
test_librbd_fsx: use private RNG context

It is at the core of fsx to be able to reproduce the exact op sequence
that lead to a failure.  Use reentrant glibc RNG functions to make sure
that nothing else has to chance to mess up the sequence.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
map-unmap.sh: drop the get_id() logic

Take advantage of the fact that 'rbd map' will now talk to udev and
output the device that got assigned by the kernel to the newly created
mapping.  Drop the get_id() cruft, udevadm settle and chown calls.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
map-unmap.sh: fail if 'rbd rm' fails

Fail if 'rbd rm' fails - most probably it'd fail with "image still has
watchers" and in that case it's a bug in the kernel client which we do
want to notice.  Also nuke the trap-based error handling - cleanup() is
half-baked and not really necessary here.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
[RGW, memory leaks] Memory leak in RGW initialization (Inserting new connection into connections map w/o check) has been fixed.

Memory leaks detector report:

$ valgrind  --leak-check=full /usr/bin/radosgw -c /etc/ceph/ceph.conf -n
client.radosgw.gateway -f
...
=16986== 1,262 (48 direct, 1,214 indirect) bytes in 1 blocks are definitely lost in loss record 81
of 83
==16986==    at 0x4A075BC: operator new(unsigned long) (vg_replace_malloc.c:298)
==16986==    by 0x618F0D: RGWRados::init_complete() (in /usr/bin/radosgw)
==16986==    by 0x618FE6: RGWRados::initialize() (in /usr/bin/radosgw)
==16986==    by 0x63BB23: RGWRados::initialize(CephContext*, bool) (in /usr/bin/radosgw)
==16986==    by 0x634D20: RGWStoreManager::init_storage_provider(CephContext*, bool) (in
/usr/bin/radosgw)
==16986==    by 0x5B8970: RGWStoreManager::get_storage(CephContext*, bool) (in /usr/bin/radosgw)
==16986==    by 0x5B6D5D: main (in /usr/bin/radosgw)
...

Signed-off-by: Andrey Kuznetsov <Andrey_Kuznetsov@epam.com>
[RGW, memory leak] Memory leak in RGW GC (losing pointer during allocating Ceph-context) has been fixed.

Memory leaks detector report:
...
==117947== 11,725 (200 direct, 11,525 indirect) bytes in 25 blocks are definitely lost in loss
record 82 of 82
==117947==    at 0x4A075BC: operator new(unsigned long) (vg_replace_malloc.c:298)
==117947==    by 0x687CC1: RGWGC::process(int, int) (in /usr/bin/radosgw)
==117947==    by 0x6884D1: RGWGC::process() (in /usr/bin/radosgw)
==117947==    by 0x68875C: RGWGC::GCWorker::entry() (in /usr/bin/radosgw)
==117947==    by 0x55F890A: Thread::_entry_func(void*) (in /usr/lib64/librados.so.2.0.0)
==117947==    by 0x34D46079D0: start_thread (in /lib64/libpthread-2.12.so)
==117947==    by 0x34D42E8B6C: clone (in /lib64/libc-2.12.so)
...

Signed-off-by: Andrey Kuznetsov <Andrey_Kuznetsov@epam.com>
[RGW, memory leak] Memory leak in RGW has been fixed: deletion of allocated pointer to pointer to Log object has been added to "on_exit" handler.

Memory leaks detector report:

$ valgrind  --leak-check=full /usr/bin/radosgw -c /etc/ceph/ceph.conf -n
client.radosgw.gateway -
...
==16986== 8 bytes in 1 blocks are definitely lost in loss record 14 of 83
==16986==    at 0x4A075BC: operator new(unsigned long) (vg_replace_malloc.c:298)
==16986==    by 0x58980B8: ceph::log::Log::set_flush_on_exit() (in /usr/lib64/librados.so.2.0.0)
==16986==    by 0x6BE3CA: global_init(std::vector<char const*, std::allocator<char const*> >*, st
==16986==    by 0x5B6B0A: main (in /usr/bin/radosgw)
...

Signed-off-by: Andrey Kuznetsov <Andrey_Kuznetsov@epam.com>
krbd: rework the unmap retry loop

The retry loop in the unmap path turned out to be insufficient for
doing long fsx -K runs.  Replace it with a single retry and then call
out to 'udevadm settle' if the retry doesn't help.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>