Add header cache to DBObjectMap

OMap is used by PG to store PGLog, so it's called by each write operations.
Because leveldb don't play well in random read and "header_lock" limit the
concurrence. Add a LRU cache to avoid too much overload on seeking header.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
ceph-disk: Enable creating multiple osds per dev

This is useful for demos and testing.  Enables
creation of lots of OSDs on a single block device
by simply running ceph-disk prepare more than once,
with a --data-size argument set.

Signed-off-by: John Spray <john.spray@inktank.com>
pybind/rados: Fix timeouts for small t

Previously, if passing a number less than 0.5 for
the timeout, operations would always return
a failure status code.

The same problem would also generate premature timeouts
for operations which completed within 0.5s of
their timeout deadline.

Fix the logic so that we only decrement `countdown`
if the thread has not completed when returning from join().

Signed-off-by: John Spray <john.spray@inktank.com>
common: WRITE_{EQ,CMP}_OPERATORS_1

For when a struct is defined to encapsulate a single value and get
control over implicit conversions that would otherwise be impossible
with typedef.

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: factorize shard_id_t/shard_t into a struct

The struct contains a single uint8_t and is preferred to control
conversions that would otherwise be implicit. There is no difference
between shard_id_t and shard_t. The definition is moved in
include/types.h so it is available to both hobject.h and osd_types.h

http://tracker.ceph.com/issues/8254 Fixes: #8254

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: loop over uint8_t instead of shard_id_t

When looping in the range [0,number of shards[ using a uint8_t variable
which is an integer instead of shard_id_t. The alternative would be
define the ++ operator on shard_id_t but since the two are used
interchangeably in a number of places, that seems unnecessary.

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: explicit shard_id_t() and NO_SHARD

Whenever an integer needs to be converted in a shard_id_t,
replace the implicit conversion with an explicit call to the
constructor.

Replace all ghobject_t::NO_SHARD and ghobject_t::no_shard() with
shard_id_t::NO_SHARD

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: replace shard_t usage with shard_id_t

Signed-off-by: Loic Dachary <loic@dachary.org>
common/strtol.cc: strict_strtosi() converts str with SI units to uint64_t

Accepts values with a suffix (B, K, M, G, T, P, E) and returns the
appropriate byte value.

E.g., 10B = 10, while 10K = 10240.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
test/strtol.cc: Test 'strict_strtosi()'

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
common/config.cc: allow integer values to be parsed as SI units

We are allowing this for all and any integer values; that is, OPT_INT,
OPT_LONGLONG, OPT_U32 and OPT_U64.

It's on the user to use appropriate units.  For instance, the user should
not use 'E(xabyte)' when setting a signed int, and use his best judgment
when setting options that, for instance, ought to receive seconds.

Fixes: 8265
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
qa/workunits/cephtool: test setting options using SI units

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
rgw: if extra data pool name is empty, use data pool name instead

Fixes: #8311
An empty pool name could be used as the extra data pool.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: fetch object attrs on multipart completion

Fixes: #8452
Backport: firefly
This fixes a regression following a code cleanup.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
qa: cp run_xfstests.sh run_xfstests-obsolete.sh

run_xfstests.sh is going to be updated in the next commit to be able to
drive newer xfstests.  Among other things, the new xfstests proper
doesn't support listing tests in ranges, which is what the qemu wrapper
(run_xfstests_qemu.sh) relies on.  So keep a copy of the old
run_xfstests.sh around until the qemu vm image is regenerated and the
up-to-date exclusion list for that kernel is shaken out.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
qa: catch up with xfstests changes

Back in 2013 xfstests were rearranged, which also changed the way
./check parses test lists.  Catch up with those changes.  Note that
tests can no longer be listed in ranges, we only accept individual
tests and test groups (e.g. -g quick).

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
qa: add run_xfstests_krbd.sh wrapper

As run_xfstests.sh does no longer accept test lists, wrap it to provide
an expunge file with a list of tests we want to skip for krbd runs.

Signed-off-by: Ilya Dryomov <ilya.dryomov@inktank.com>
Merge pull request #1857 from ceph/wip-8265

#8265: config: make int parser accept K, M, G, T, P suffix and scale value accordingly

Reviewed-by: Loic Dachary <loic@dachary.org>
Reviewed-by: Samuel Just <sam.just@inktank.com>
0.81
doc: Added a section for ceph df.

Fixes: #8281

Signed-off-by: John Wilkins <john.wilkins@inktank.com>