Add header cache to DBObjectMap

OMap is used by PG to store PGLog, so it's called by each write operations.
Because leveldb don't play well in random read and "header_lock" limit the
concurrence. Add a LRU cache to avoid too much overload on seeking header.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
osdc/Objecter: flag ops that have been redirected

In the future it may be helpful to know whether an op has been redirected
by the OSD.

We don't need it bad enough now to spend a feature bit to know whether this
flag is reliably set.

Signed-off-by: Sage Weil <sage@inktank.com>
ceph-disk: Enable creating multiple osds per dev

This is useful for demos and testing.  Enables
creation of lots of OSDs on a single block device
by simply running ceph-disk prepare more than once,
with a --data-size argument set.

Signed-off-by: John Spray <john.spray@inktank.com>
pybind/rados: Fix timeouts for small t

Previously, if passing a number less than 0.5 for
the timeout, operations would always return
a failure status code.

The same problem would also generate premature timeouts
for operations which completed within 0.5s of
their timeout deadline.

Fix the logic so that we only decrement `countdown`
if the thread has not completed when returning from join().

Signed-off-by: John Spray <john.spray@inktank.com>
osd/: move split_list and split_request into OSD

Signed-off-by: Samuel Just <sam.just@inktank.com>
crush: add adjust_subtree_weight()

Adjust the weight for all (leaf) items beneath a subtree.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: add 'osd crush reweight-subtree ...'

Reweight all items within a subtree.  For example, you might reweigh tall
OSDs within a rack to a new value.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: simplify 'osd crush reweight ...' flow

Use the goto labels; drop the do/while.

Signed-off-by: Sage Weil <sage@inktank.com>
common: WRITE_{EQ,CMP}_OPERATORS_1

For when a struct is defined to encapsulate a single value and get
control over implicit conversions that would otherwise be impossible
with typedef.

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: factorize shard_id_t/shard_t into a struct

The struct contains a single uint8_t and is preferred to control
conversions that would otherwise be implicit. There is no difference
between shard_id_t and shard_t. The definition is moved in
include/types.h so it is available to both hobject.h and osd_types.h

http://tracker.ceph.com/issues/8254 Fixes: #8254

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: loop over uint8_t instead of shard_id_t

When looping in the range [0,number of shards[ using a uint8_t variable
which is an integer instead of shard_id_t. The alternative would be
define the ++ operator on shard_id_t but since the two are used
interchangeably in a number of places, that seems unnecessary.

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: explicit shard_id_t() and NO_SHARD

Whenever an integer needs to be converted in a shard_id_t,
replace the implicit conversion with an explicit call to the
constructor.

Replace all ghobject_t::NO_SHARD and ghobject_t::no_shard() with
shard_id_t::NO_SHARD

Signed-off-by: Loic Dachary <loic@dachary.org>
osd: replace shard_t usage with shard_id_t

Signed-off-by: Loic Dachary <loic@dachary.org>
common/strtol.cc: strict_strtosi() converts str with SI units to uint64_t

Accepts values with a suffix (B, K, M, G, T, P, E) and returns the
appropriate byte value.

E.g., 10B = 10, while 10K = 10240.

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
test/strtol.cc: Test 'strict_strtosi()'

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
common/config.cc: allow integer values to be parsed as SI units

We are allowing this for all and any integer values; that is, OPT_INT,
OPT_LONGLONG, OPT_U32 and OPT_U64.

It's on the user to use appropriate units.  For instance, the user should
not use 'E(xabyte)' when setting a signed int, and use his best judgment
when setting options that, for instance, ought to receive seconds.

Fixes: 8265
Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
qa/workunits/cephtool: test setting options using SI units

Signed-off-by: Joao Eduardo Luis <joao.luis@inktank.com>
rgw: if extra data pool name is empty, use data pool name instead

Fixes: #8311
An empty pool name could be used as the extra data pool.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: fetch object attrs on multipart completion

Fixes: #8452
Backport: firefly
This fixes a regression following a code cleanup.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
ceph-common: Implementation of the sharded threadpool.
Threadpool will only be having a single work queue and internally the
work queue will be having multiple storage data structures.
Based on some logic (which is derived class implementation specific)
the work queue will shard the requests among these storage structures.
Each storage will be guarded by finer grained sunchronization
objects. Sharded threadpool threads will be assigned to work on a
shard based on some algorithm which is again derived class implementation
specific.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>