Merge pull request #2811 from ceph/wip-vstart

init-ceph: make ./init-ceph behave from src dir on systemd
Merge pull request #2839 from ceph/wip-9944

osdc/Objecter: fix null dref when pool dne

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Merge pull request #2831 from yuyuyu101/async-kqueue

AsyncMessenger: Add kqueue support
Merge pull request #2835 from leseb/doc-rbd-juno

doc: update RBD for Juno

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
osd: past_interval display bug on acting

The acting array was incorrectly including the primary and up_primary.

http://tracker.ceph.com/issues/9752 Fixes: #9752

Signed-off-by: Loic Dachary <loic-201408@dachary.org>
Merge pull request #2843 from dachary/wip-9752-past-intervals

osd: past_interval display bug on acting

Reviewed-by: Greg Farnum <greg@inktank.com>
Fix rados_shutdown hang forever when using radosstriper

Dear list,

I have met this when I was using radosstriper C API. My program is
roughly like this:

    rados_striper_aio_write
    rados_aio_flush
    rados_aio_wait_for_safe
    rados_aio_release
    rados_striper_destroy
    rados_ioctx_destroy
    rados_shutdown /Hangs here/

In most time, this works well, But the programm occasionally
hangs forever. Output of gstack:

Thread 1 (Thread 0x7fe0afba0760 (LWP 18509)):
0 0x000000330f20822d in pthread_join () from /lib64/libpthread.so.0
1 0x000000347566cea2 in Thread::join(void**) () from
/usr/lib64/librados.so.2
2 0x00000034755ac535 in librados::RadosClient::shutdown() () from
/usr/lib64/librados.so.2
3 0x0000003475592269 in rados_shutdown () from /usr/lib64/librados.so.2
4 0x0000000000402349 in main ()

Thread 4 (Thread 0x7fe0ab14d700 (LWP 18541)):
0 0x000000330f20e264 in __lll_lock_wait () from /lib64/libpthread.so.0
1 0x000000330f209508 in _L_lock_854 () from /lib64/libpthread.so.0
2 0x000000330f2093d7 in pthread_mutex_lock () from
/lib64/libpthread.so.0
3 0x0000003475633af1 in Mutex::Lock(bool) () from
/usr/lib64/librados.so.2
4 0x00000034755abd37 in librados::RadosClient::put() () from
/usr/lib64/librados.so.2
5 0x0000003475592501 in librados::Rados::shutdown() () from
/usr/lib64/librados.so.2
6 0x00007fe0afbba9f7 in
libradosstriper::RadosStriperImpl::CompletionData::~CompletionData() ()
from /usr/lib64/libradosstriper.so.1
7 0x00007fe0afbbaad9 in
libradosstriper::RadosStriperImpl::WriteCompletionData::~WriteCompletionData()
() from /usr/lib64/libradosstriper.so.1
8 0x00007fe0afbc1d75 in RefCountedObject::put() () from
/usr/lib64/libradosstriper.so.1
9 0x00007fe0afbc224d in
libradosstriper::MultiAioCompletionImpl::safe_request(long) () from
/usr/lib64/libradosstriper.so.1
10 0x00000034755c5ce8 in librados::C_AioSafe::finish(int) () from
/usr/lib64/librados.so.2
11 0x00000034755a0e89 in Context::complete(int) () from
/usr/lib64/librados.so.2
12 0x000000347564d4c8 in Finisher::finisher_thread_entry() () from
/usr/lib64/librados.so.2
13 0x000000330f2079d1 in start_thread () from /lib64/libpthread.so.0
14 0x000000330eae886d in clone () from /lib64/libc.so.6

It is obvious that librados::Rados::shutdown is not a thread-safe
function here. It will hang forever. The culprit of this is when
CompletionData is released, it will first notify
"rados_aio_wait_for_safe" to continue, and CompletionData will call
put() to release other data. But if the main thread(Thread 1 here) runs
fast enough, rados_striper_destroy will be executed before other
thread(Thread 4 here)'s releasing refcnf. In this situation, main thread
runs Rados::shutdown() while other thread runs Rados::shutdown() in the
same time.

My suggestion is to let RadosStriperImpl::aio_flush to block until all
the CompletionData has been released. This makes sure other thread will
never call rados_shutdown.
Merge pull request #2813 from ceph/wip-9894

client: fix I_COMPLETE_ORDERED checking

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
Merge pull request #2827 from thesues/fix-hang

Fix rados_shutdown hang forever when using radosstriper

Reviewed-by: Sage Weil <sage@redhat.com>
Merge pull request #2826 from wonzhq/evict-atime-nohitset

osd: tiering: calculate object age during eviction when there is no hit set

Reviewed-by: Sage Weil <sage@redhat.com>
Merge pull request #2796 from ceph/wip-rwtimer

common/Timer: kill RWTimer

Reviewed-by: Yehuda Sadeh <yehuda@redhat.com>
ceph.in: remove '--' from injectargs, if there, as well

Fixes: #9976
Reviewed-by: Sage Weil <sage@redhat.com>
Signed-off-by: Dan Mick <dan.mick@redhat.com>
Merge pull request #2816 from XinzeChi/master

Get the currently atime of the object in cache pool for eviction 

Backport: giant, firefly
Reviewed-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: fix compile error

From 1fef4c3d541cba360738437420ebfa2447d5802e.

Signed-off-by: Sage Weil <sage@redhat.com>
buffer: implement list::get_contiguous

Return a pointer to a contiguous range of the bufferlist, rebuilding
into a contiguous region as needed.  For now, if we need to rebuild,
we just do the whole thing.  We can obviously optimize this later to
rebuild on the necessary region, but this is good enough for the
(presumably) common case where the needed region is already in fact
contiguous.

Signed-off-by: Sage Weil <sage@redhat.com>
JounralingObjectStore: journal->committed_thru after replay

It's possible that the osd stopped between when the filestore
op_seq file was updated and when the journal was trimmed.  In
that case, it's possible that on boot the journal might be
full, and yet not be trimmed because commit_start assumes
there is no work to do.  Calling committed_thru on the journal
ensures that the journal matches committed_seq.

Backport: giant firefly emperor dumpling
Fixes: #6756
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: David Zafman <david.zafman@inktank.com>
mon/PGMap: invalidate cached min_last_epoch_clean from new-style pg keys

We were only invalidating the cache from the legacy apply_incremental(),
which is no longer called on modern clusters.

Fixes: #9987
Signed-off-by: Sage Weil <sage@redhat.com>
mon/PGMap and PGMonitor: update last_epoch_clean cache from new osd keys

We were only invalidating the cached value from apply_incremental, which
is no longer called on modern clusters.

Fix this by storing the update epoch in the key as well (it is not part
of osd_stat_t).

Backport: giant, firefly, dumpling(?)
Fixes: #9987
Signed-off-by: Sage Weil <sage@redhat.com>
osdc/Objecter: e shouldn't be zero in Objecter::handle_osd_map

Signed-off-by: Ding Dinghua <dingdinghua85@gmail.com>
osdc/Objecter: Fix a bug of dead looping in Objecter::handle_osd_map

If current map epoch is less than oldest epoch, current map epoch
should step up to oldest epoch.

Fixes: #9986
Signed-off-by: Ding Dinghua <dingdinghua85@gmail.com>