MB-7966 KVShard implementation for Multi-readers and writers

This change introduces the concept of the KVShard class which is
used to allow scheduling for the multi-reader/writer feature. The
KVShard is a simply a logical partitioning of the VBucket map that
allows the engine to easily be able to schedule jobs from different
VBuckets to specific reader and writer threads.

Change-Id: I527b63a2f998020334d0d47dbb8fb3870d8312e6
Reviewed-on: http://review.couchbase.org/25624
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-8175: Fix tap implicit acks stream unit test failure

We should only decrement the transmitted stat if we actually have
the corresponding tap message in the tap log.

Change-Id: I60fd59419043b7e2af87694dbac5980a5f0ecc72
Reviewed-on: http://review.couchbase.org/25977
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-100: Fix warnings in compilation

Change-Id: Ie44ac4f50ed357a915e6455a2a04a63c9f8bd04a
Reviewed-on: http://review.couchbase.org/25990
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-100: Remove some unused variables from EventuallyPersistentStore

Change-Id: Ib5b88d39b3cd1489defa68dfaf0bdd399c7b17e8
Reviewed-on: http://review.couchbase.org/25997
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-8152: Don't write to vbucket file until it exists

We should mark all vbuckets in the vbucket map as in creation mode
until we know that they are actually on disk. This way the flusher
won't write anything to them until we do the initial vbucket
snapshot to make sure that they are there.

Change-Id: I2b8b367b400b9c28723bb94c1469ac849f71c12c
Reviewed-on: http://review.couchbase.org/25988
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-7518: Only snapshot vbuckets from a particular shard

Change-Id: Ic31f97d6f6dfae76af25e3a70fbb0078db03f1d2
Reviewed-on: http://review.couchbase.org/26031
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-8176: Fix all valgrind warnings in unit tests

Change-Id: Idd23b2dc8cdb79602c01309a47e0ea3c387fd7ab
Reviewed-on: http://review.couchbase.org/25978
Tested-by: Michael Wiederhold <mike@couchbase.com>
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
MB-7518: Remove ep_db_init stat

This stat no longer makes sense and always returns 0

Change-Id: Id0d52b50f443419416a3cbbb405cfbc13639b09a
Reviewed-on: http://review.couchbase.org/26033
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-100: fix broken build, missing kvshard file

Change-Id: I586a0b0ec01944c96168038e6c7514b65f6e126d
Reviewed-on: http://review.couchbase.org/26050
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Bin Cui <bin.cui@gmail.com>
MB-7518: Fix snapshotVbuckets logic that was broken by prev change

Change-Id: Ie6b26b4454d2edb301af8bf4a3bb9124d2dfa1ba
Reviewed-on: http://review.couchbase.org/26052
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: abhinav dangeti <abhinav.dangeti@gmail.com>
MB-8203: Make sure we can delete a partially initialized bucket

Change-Id: Icab289f08860b1e3e8363be76eaeecc596ff24c8
Reviewed-on: http://review.couchbase.org/26138
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-7966 IOManager integration with KVShard

Change-Id: I641dda9ea68bdade2f71b5805da439173dc3d678
Reviewed-on: http://review.couchbase.org/25955
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-7729: Fix 100% CPU Issue

The background fetcher jobs todo stat was not being
decremented properly.

Change-Id: I26a35e76074644b63117d348c05123c09691cc44
Reviewed-on: http://review.couchbase.org/26167
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Jin Lim <jin@couchbase.com>
MB-7966 singleton CouchNotifier

This implements singletone CouchNotifier class that is an alternative
approach for synchronization among concurrent notification requests
from mutiple dispatchers.

Change-Id: I13902e9b6161dd177c3e643118b57098ad3cf056
Reviewed-on: http://review.couchbase.org/26034
Tested-by: Jin Lim <jin@couchbase.com>
Tested-by: abhinav dangeti <abhinav.dangeti@gmail.com>
Reviewed-by: abhinav dangeti <abhinav.dangeti@gmail.com>
MB-100: Remove obsolete files from project

Looks like the following file no longer exist in the project.
mutation_log_compactor.cc
statsnap.cc

Change-Id: Idf7fcdb1a6b2e06b1e6e92edd46f03433cd37113
Reviewed-on: http://review.couchbase.org/26199
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Bin Cui <bin.cui@gmail.com>
MB-7966 per bucket singleton notifier

Change-Id: I4e249ca5ccc7fa632b7aaf488e1e89a3b9125fcd
Reviewed-on: http://review.couchbase.org/26202
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-8224: Check bgFetchQueue size before scheduling vkey bgfetch

It's possible that the bgfetch can complete in the other thread
before we do this check and cause the assertion to fail.

Change-Id: Ieda14a787df63a482ed1dd203d78a57aa6a2e5cc
Reviewed-on: http://review.couchbase.org/26207
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-8214 set hasWokenTask true when waking up bgfetcher

Change-Id: I7f152bf12edd30bcc92c0c823d7ac3010666268f
Reviewed-on: http://review.couchbase.org/26183
Tested-by: Jin Lim <jin@couchbase.com>
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
MB-100: Fix build errors caused by missing files

Change-Id: Id6e238c48e0b144cef21e68cd4e610109eb8dc92
Reviewed-on: http://review.couchbase.org/26226
Tested-by: Bin Cui <bin.cui@gmail.com>
Reviewed-by: Bin Cui <bin.cui@gmail.com>
MB-7735: Fix to a race in notifying / releasing TAP connections

Change-Id: I35deeeaab98235500d9ba22c213c413eac1fbc04