Add header cache to DBObjectMap

OMap is used by PG to store PGLog, so it's called by each write operations.
Because leveldb don't play well in random read and "header_lock" limit the
concurrence. Add a LRU cache to avoid too much overload on seeking header.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
osdc/Objecter: flag ops that have been redirected

In the future it may be helpful to know whether an op has been redirected
by the OSD.

We don't need it bad enough now to spend a feature bit to know whether this
flag is reliably set.

Signed-off-by: Sage Weil <sage@inktank.com>
ceph-disk: Enable creating multiple osds per dev

This is useful for demos and testing.  Enables
creation of lots of OSDs on a single block device
by simply running ceph-disk prepare more than once,
with a --data-size argument set.

Signed-off-by: John Spray <john.spray@inktank.com>
pybind/rados: Fix timeouts for small t

Previously, if passing a number less than 0.5 for
the timeout, operations would always return
a failure status code.

The same problem would also generate premature timeouts
for operations which completed within 0.5s of
their timeout deadline.

Fix the logic so that we only decrement `countdown`
if the thread has not completed when returning from join().

Signed-off-by: John Spray <john.spray@inktank.com>
crush: add adjust_subtree_weight()

Adjust the weight for all (leaf) items beneath a subtree.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: add 'osd crush reweight-subtree ...'

Reweight all items within a subtree.  For example, you might reweigh tall
OSDs within a rack to a new value.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: simplify 'osd crush reweight ...' flow

Use the goto labels; drop the do/while.

Signed-off-by: Sage Weil <sage@inktank.com>
common: WRITE_{EQ,CMP}_OPERATORS_1

For when a struct is defined to encapsulate a single value and get
control over implicit conversions that would otherwise be impossible
with typedef.

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: factorize shard_id_t/shard_t into a struct

The struct contains a single uint8_t and is preferred to control
conversions that would otherwise be implicit. There is no difference
between shard_id_t and shard_t. The definition is moved in
include/types.h so it is available to both hobject.h and osd_types.h

http://tracker.ceph.com/issues/8254 Fixes: #8254

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: loop over uint8_t instead of shard_id_t

When looping in the range [0,number of shards[ using a uint8_t variable
which is an integer instead of shard_id_t. The alternative would be
define the ++ operator on shard_id_t but since the two are used
interchangeably in a number of places, that seems unnecessary.

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: explicit shard_id_t() and NO_SHARD

Whenever an integer needs to be converted in a shard_id_t,
replace the implicit conversion with an explicit call to the
constructor.

Replace all ghobject_t::NO_SHARD and ghobject_t::no_shard() with
shard_id_t::NO_SHARD

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: replace shard_t usage with shard_id_t

Signed-off-by: Loic Dachary <loic@dachary.org>
common/strtol.cc: strict_strtosi() converts str with SI units to uint64_t

Accepts values with a suffix (B, K, M, G, T, P, E) and returns the
appropriate byte value.

E.g., 10B = 10, while 10K = 10240.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
test/strtol.cc: Test 'strict_strtosi()'

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
common/config.cc: allow integer values to be parsed as SI units

We are allowing this for all and any integer values; that is, OPT_INT,
OPT_LONGLONG, OPT_U32 and OPT_U64.

It's on the user to use appropriate units.  For instance, the user should
not use 'E(xabyte)' when setting a signed int, and use his best judgment
when setting options that, for instance, ought to receive seconds.

Fixes: 8265
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
qa/workunits/cephtool: test setting options using SI units

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
rgw: if extra data pool name is empty, use data pool name instead

Fixes: #8311
An empty pool name could be used as the extra data pool.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: fetch object attrs on multipart completion

Fixes: #8452
Backport: firefly
This fixes a regression following a code cleanup.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Make KeyValueStore support "ceph osd perf" command

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
qa: cp run_xfstests.sh run_xfstests-obsolete.sh

run_xfstests.sh is going to be updated in the next commit to be able to
drive newer xfstests.  Among other things, the new xfstests proper
doesn't support listing tests in ranges, which is what the qemu wrapper
(run_xfstests_qemu.sh) relies on.  So keep a copy of the old
run_xfstests.sh around until the qemu vm image is regenerated and the
up-to-date exclusion list for that kernel is shaken out.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>