Commented out the statistics counter update part within throttle class.
Removed the statistical counter update part and only deleting the op from here.
Not doing any statistics and register_inflight_op is also commented. All of these operations are holding global locks and thus massive performance degredation happening.
Tools are showing that this buffer class is one of the most cpu eaters.
Probably, need to redesign it efficiently.
asserts() are consuming some portion of those cpu times, so, as a quick fix I commented out all.
I know asserts are helpfull,but, we need to use it wisely and probably in combination with null check and log print ?
Earlier, this comparison operators was using boost and that was consuming considerable amount of cpu.
Got rid of boost and it boosts the performance/cpu usage.
Some generic optimization here. Return by reference and got rid of vector resize.
Instead of resize, I used reserve and push_back() combination. This is more efficient.
In case of MOSDOpReply(), I used copy constructor in the initialization list instead of assignment operator in the constructor body.
This is the fast io path implementation getting rid of the single dispatch thread bottleneck.
Now, DispatchQueue::enqueue is directly calling dispatcher::ms_fast_dispatch() if dispatcher supports fast dispatching.
Changed conditional broadcast to wake up single thread. Even though one one read/writer thread it is a good practice.
Signing messages was consuming some amount of cpu , so, commented it.
Commented out all throttling related code from here.
This lock was hurting us badly. No need to get this lock here. May be if connection is not there
This is the fast look up path I introduced.This significantly reduced the cpu consumption in the lfn-index look up.
Introduced 4 new functions here. First, I tried to optimized lfn_open but end up rewriting a fast path for read operation.
Now, in the read path getattr_fast, lfn_open_fast, lfn_close_fast, read_fast is been invoked.
optimized the functions here based on the cpu profiler report.
This is the new sharding thread pool implementation. The io is going through this path now instead of the earlier generic thread pool.
BTW, I need to do better job to take the input from conf file and create the thread pool accordingly.
This is POC now, will implement those once approved.
Most of the changes are incorporated to implement a fast io path geting rid of the global OSd lock holding.
The bulk of these changes are done by Sam and I have modified some portion of it..
I commented out the session handling and commented out some checking.
Need to see if we can do some of these checking after holding the PG lock not here.
In the enqueue_op(), I am doing the sharding and putting the request in the corresponding pool.
These are not my changes..But, was there in wip-queue branch created by Sam
Passed param by reference reduces some cpu usage. Also, in case queue_op(), I am not using ceph thread pool in case of osd ops.
Not my changes, inherited from wip-queue branch
Introduced a new path for getting the attribute first..
Not my changes..Inherited from wip-queue