ceph-common: Implementation of the sharded threadpool.
Threadpool will only be having a single work queue and internally the
work queue will be having multiple storage data structures.
Based on some logic (which is derived class implementation specific)
the work queue will shard the requests among these storage structures.
Each storage will be guarded by finer grained sunchronization
objects. Sharded threadpool threads will be assigned to work on a
shard based on some algorithm which is again derived class implementation
specific.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OSD: Sharded Op worker queue implementation for handling OSD ops
This is the implementation for the client of the
sharded thread pool/sharded workQ. Removed the op_wq class and now OSD
ops are going through sharded workqueue model which is used by the
sharded threadpool. Derived ShardedOpWQ implementation has a data structure
called ShardData which has it's own lock/cond and storage. ShardedOpWQ
holds a vector of that and the size of the vector is a config option.
During enqueue operation on the queue, the ops are sharded across these
ShardData based on pg hash % number of shards.
Similarly, in the _process function the sharded thread pool threads are
divided across ShardData based on thread index % number of shards

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
ceph-common: The Sharded threadpool worker logic changed
Now, the _process() of the derived queue is processing one request
at a time and the outer loop is controlled by the sharded threadpool.
The stop/pause/drain functionalities are controlled by the sharded TP.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OSD:Derived sharded queue implementation is changed
All the threadpool related stuff like stop/pause/drain etc. are not
handled by sharded queue anymore. All it is implementing are related to
processing,enqueue , signaling of waiting threads and shard queue status.
The pg ordering is been taken care of by introducing a map <pg,op> in each
shard.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
WorkQueue: Taking care of potential race condition during pause()
Introduced two variables to keep track of number of threads paused
and drained during threadpool pause/drain. The pause()/drain() call
is waiting till number of pause/drain threads equals to toral number of
thread pool threads.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
WorkQueue: Removed the unused in_process variable

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
WorkQueue: The variable name is corrected.
Modified the variable name from shardedpol_cond->shardedpool_cond

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
rgw: set meta object in extra flag when initializing it

As part of the fix for 8452 we moved the meta object initialization.
Missed moving the extra flag initialization that is needed. This breaks
setups where there's a separate extra pool (needed in ec backends).

Reported-by: Sylvain Munaut <s.munaut@whatever-company.com>
Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
Make KeyValueStore support set_alloc_hint op

Add a new config let KeyValueStore support configurable strip size.
set_alloc_hint op can affect the strip size of the specified object
and the expect write size will become the strip size of the object.

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
Add set_alloc_hint test to ceph_test_objectstore

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
mon: gather perf stats on session adds, removes, counts

I found myself wanting to know this on a running cluster.

Signed-off-by: Sage Weil <sage@inktank.com>
mon: gather perf stats on elections

Signed-off-by: Sage Weil <sage@inktank.com>
Add upper limit to the write size of set_alloc_hint in KeyValueStore

Signed-off-by: Haomai Wang <haomaiwang@gmail.com>
Fix for bug #6700

When preparing OSD disks with colocated journals, the intialization process
fails when using dmcrypt. The kernel fails to re-read the partition table after
the storage partition is created because the journal partition is already in use
by dmcrypt. This fix unmaps the journal partition from dmcrypt and allows the
partition table to be read.

Signed-off-by: Stephen F Taylor <steveftaylor@gmail.com>
ShardedTP: Changes related to conforming to Ceph coding guidelines

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
ShardedTP: The config option changed
The config option for sharded threadpool is changed to
osd_op_num_threads_per_shard instead of osd_op_num_sharded_pool_threads.
Along with osd_op_num_shards this will be much more user friendly while
configuring the number of op threads for the osd.

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
OSD: The thread pool variable name changed
The variable names are more meaningful now. op_tp -> osd_tp and
op_sharded_tp -> osd_op_tp

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
PG: Added a const spg_t member to the PG class
The const spg_t member is been insantiated from constructor
and now get_pgid() can reference this to return a spg_t instance
without the need of pg_info (thus not requiring to acquire pg_lock).

Signed-off-by: Somnath Roy <somnath.roy@sandisk.com>
rgw: cache decoded user info

Instead of accessing the raw user info data, cache the decoded
structure.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>
rgw: cache bucket info

This is really a partial implementation, so should only be used for
testing.

Signed-off-by: Yehuda Sadeh <yehuda@inktank.com>