Add header cache to DBObjectMap

OMap is used by PG to store PGLog, so it's called by each write operations.
Because leveldb don't play well in random read and "header_lock" limit the
concurrence. Add a LRU cache to avoid too much overload on seeking header.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
osd: skip out of order op checks on tiered pools

When we send redirected ops, we do not assign a new tid, which means that
a given client's ops for a pool may not have strictly ordered tids.  Skip
this check if the pool is tiered to avoid false positives.

Fixes: #8380
Signed-off-by: Sage Weil <sage@inktank.com>
osdc/Objecter: flag ops that have been redirected

In the future it may be helpful to know whether an op has been redirected
by the OSD.

We don't need it bad enough now to spend a feature bit to know whether this
flag is reliably set.

Signed-off-by: Sage Weil <sage@inktank.com>
Merge pull request #1825 from ceph/wip-8380

osd: skip out of order op checks on tiered pools

Reviewed-by: Samuel Just <sam.just@inktank.com>
Fix set_alloc_hint op cause KeyValueStore crash problem

Now KeyValueStore doesn't support set_alloc_hit op, the implementation of
_do_transaction need to consider decoding the arguments. Otherwise, the
arguments will be regarded as the next op.

Fix the same problem for MemStore.

Fix #8381

Reported-by: Xinxin Shu <xinxin.shu5040@gmail.com>
Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
README: clean up some whitespace

Remove $ prefix, use tab consistently.

Signed-off-by: Sage Weil <sage@inktank.com>
README: move package dependencies into separate files

Sorted, with a short, simple command to paste.  Avoid the duplication of
the lists.

Signed-off-by: Sage Weil <sage@inktank.com>
Update architecture.rst:Calculating PG IDs

To Calculate PG ID, if I didn't get it wrong, CRUSH calculates the hash modulo 
the number of PGs instead of OSDs, according to osd/osd_types.cc:963 
ceph_stable_mod(pg.ps(), pg_num, pg_num_mask).

Signed-off-by: Kai Zhang <zakir.exe@gmail.com>
ceph-disk: Enable creating multiple osds per dev

This is useful for demos and testing.  Enables
creation of lots of OSDs on a single block device
by simply running ceph-disk prepare more than once,
with a --data-size argument set.

Signed-off-by: John Spray <john.spray@inktank.com>
os/FileStore: fix journal-less operation

A few different fixes here:

- make the op_seq correct on start when there is no replay
- make an explicit 'no journal' case in queue_transaction (one that
  actually works).  Note that the 'trailing journal' mode is probably
  still broken.
- add a -J flag to vstart.sh for journalless operation

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: refactor backend instantiation

Use a factory method for creating the backend so that the logic is not
duplicated in mkfs() and _detect_fs().  Make the other fs-dependent logic
(xattr limits) key off of the fs_type magic.

Signed-off-by: Sage Weil <sage@inktank.com>
os/FileStore: include filestore backend, f_type in osd metadata

Report the FileStore backend and underlying fs type as part of the osd
metadata shared with the monitor.

Signed-off-by: Sage Weil <sage@inktank.com>
Improvements to radosgw docs
- Correct a typo (`rados-admin`) in admin docs
- Reorder sections in config so "above example" is in expected position
osd: fix map advance limit to handle map gaps

The recent change in cf25bdf6b0090379903981fe8cee5ea75efd7ba0 would stop
advancing after some number of epochs, but did not take into consideration
the possibilty that there are missing maps.  In that case, it is impossible
to advance past the gap.

Fix this by increasing the max epoch as we go so that we can always get
beyond the gap.

Signed-off-by: Sage Weil <sage@inktank.com>
pybind/rados: Fix timeouts for small t

Previously, if passing a number less than 0.5 for
the timeout, operations would always return
a failure status code.

The same problem would also generate premature timeouts
for operations which completed within 0.5s of
their timeout deadline.

Fix the logic so that we only decrement `countdown`
if the thread has not completed when returning from join().

Signed-off-by: John Spray <john.spray@inktank.com>
better error reporting on incompatible device requirements

Signed-off-by: Alfredo Deza <alfredo@deza.pe>
crush: add adjust_subtree_weight()

Adjust the weight for all (leaf) items beneath a subtree.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: add 'osd crush reweight-subtree ...'

Reweight all items within a subtree.  For example, you might reweigh tall
OSDs within a rack to a new value.

Signed-off-by: Sage Weil <sage@inktank.com>
mon/OSDMonitor: simplify 'osd crush reweight ...' flow

Use the goto labels; drop the do/while.

Signed-off-by: Sage Weil <sage@inktank.com>
os: ObjectStore::collect_metadata unit tests

And an emacs-executable reminder about how to run a specific test.

Signed-off-by: Loic Dachary <loic@dachary.org>