init: add systemd service files

This patch adds systemd service files. It is possible to start and
enable multiple instances (per monid, osdid, mds name), e.g.

 # systemctl start ceph-mon@node01
 # systemctl enable ceph-mon@node01

 # systemctl start ceph-osd@0
 # systemctl enable ceph-osd@0

The ceph cluster can be set in the system config file:

 /etc/sysconfig/ceph

adding or editing the CLUSTER environment variable.

Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
msg/SimpleMessenger: drop local_conneciton priv link on shutdwon

This breaks ref cycles between the local_connection and session, and let's
us drop the explicit set_priv() calls in OSD::shutdown().

Signed-off-by: Sage Weil <sage@redhat.com>
os/FileJournal: Update the journal header when closing journal

When closing journal, it should check must_write_header and update
journal header if must_write_header alreay set.
It can reduce the nosense journal-replay after restarting osd.

Signed-off-by: Ma Jianpeng <jianpeng.ma@intel.com>
Reviewed-by: Sage Weil <sage@redhat.com>
osd: fix bad Message* defer in C_SendMap and send_map_on_destruct

We were carrying a bare Message*, which could get freed if the op was
canceled (or possibly completed).  Instead, just stash the entity_name_t,
the only piece we need.  The Connection is properly ref counted so no
worries there.

Fixes: #8926
Signed-off-by: Sage Weil <sage@redhat.com>
ceph_test_rados_api_tier: do fewer writes in HitSetWrite

We don't need to do quite so many writes.  It can be slow when we are
thrashing and aren't doing anything in parallel.

Fixes: #8932
Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: fix cache full -> not full requeueing when !active

We only want to do this if is_active().  Otherwise, the normal
requeueing code will do its thing, taking care to get the queue orders
correct.

Backport: firefly
Signed-off-by: Sage Weil <sage@redhat.com>
osd/ReplicatedPG: requeue cache full waiters if no longer writeback

If the cache is full, we block some requests, and then we change the
cache_mode to something else (say, forward), the full waiters don't get
requeued until the cache becomes un-full.  In the meantime, however, later
requests will get processed and redirected, breaking the op ordering.

Fix this by requeueing any full waiters if we see that the cache_mode is
not writeback.

Fixes: #8931
Signed-off-by: Sage Weil <sage@redhat.com>
mds: fix journal reformat failure in standbyreplay

In the 0.82 release, standbyreplay MDS daemons would try
to reformat the jouranl if they saw an older version on
disk, where this should have only been done by the active
MDS for the rank.  Depending on timing, this could cause
fatal corruption of the journal.

This change handles the following cases:
* only do reformat if not in standbyreplay (else raise EAGAIN
to keep trying til an active mds reformats it)
* if journal header goes away while in standbyreplay then raise
EAGAIN (handle rewrite happening in background)
* if journal version is greater than the max supported, suicide

Fixes: #8811

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 5438500af8979fda32e61714ae40b71c7ffdfd15)
Merge remote-tracking branch 'gh/next'

Conflicts:
	src/osdc/Journaler.h
Revert "Merge pull request #2129 from ceph/wip-librbd-oc"

This reverts commit 74b386f03e4ca9970256db72c575589aea077534, reversing
changes made to 36265d0db0d7c0eb31d25a0f77ac233b3fd198f8.

The dirty_or_tx list is used by flush_set, which means we can
resubmit new IOs for writes that are already in progress.  This
has a compounding effect that overwhelms the OSDs with dup IOs
and stalls out the client.

See, for example, teh failues in this run:
  /a/sage-2014-07-25_17:14:20-fs-wip-msgr-testing-basic-plana

The fix is probably pretty simple, but reverting for now to make
the tests pass.

Signed-off-by: Sage Weil <sage@inktank.com>
Reduce ObjectCacher flush overhead

Flush op in ObjectCacher will iterate the whole active object set, each
dirty object also may own several BufferHead. If the object set is large,
it will consume too much time.

Use dirty_bh instead to reduce overhead. Now only dirty BufferHead will
be checked.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
Add rbdcache max dirty object option

Librbd will calculate max dirty object according to rbd_cache_max_size, it
doesn't suitable for every case. If user set image order 24, the calculating
result is too small for reality. It will increase the overhead of trim call
which is called each read/write op.

Now we make it as option for tunning, by default this value is calculated.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
Wait tx state buffer in flush_set

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
ObjectCacher: fix dirty_or_tx_bh logic in bh_set_state()

The else-if chain here was wrong. Handling dirty or tx buffers and
errors should be in independent conditions.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
ObjectCacher: fix bh_{add,remove} dirty_or_tx_bh accounting

tx buffers need to go on the bh_lru_rest as well, and removing erases
(not inserts) them into dirty_or_tx_bh.

Signed-off-by: Josh Durgin <josh.durgin@inktank.com>
Only write bufferhead when it's dirty

The TX state bh should be skipped because the bh should be inflight. We only
need to write dirty bh. And TX and dirty state bh both should be waited until
flushed.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
configure: don't link blkid, udev to everything

These are already explicitly called out for libkrbd; don't need them in
LIBS.

Signed-off-by: Sage Weil <sage@redhat.com>
configure.ac: link libboost_thread only with json-spirit

Signed-off-by: Sage Weil <sage@redhat.com>
Merge pull request #2149 from yuyuyu101/wip-flush-set

Fix dup bh_write for TX state bh

Tested-by: Sage Weil <sage@redhat.com>
Reviewed-by: Haomai Wang <haomaiwang@gmail.com>

Original changeset 

Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
PGMonitor: fix bug in caculating pool avail space

Currently for pools with different rules, "ceph df" cannot report
right available space for them, respectively. For detail assisment
of the bug ,pls refer to bug report #8943

This patch fix this bug and make ceph df works correctlly.

Fixes Bug #8943

Signed-off-by: Xiaoxi Chen <xiaoxi.chen@intel.com>