MB-10962 Don't snooze the TAP connection notifier unnecessarily.

There is a race in pausing and resuming the TAP connection
notifier upon receiving a new mutation. To address this issue,
This change adds a flag that indicates if there is a pending
notification event for any TAP connections.

Change-Id: Ide00d835ef3d407087c7313f2346394b7f37c630
Reviewed-on: http://review.couchbase.org/36361
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
MB-11083: block shard serial runnable tasks to save CPU

Ready tasks in the task queue that cannot be picked up by
a thread because of shard serialization are now put in
a shard-local pending list to prevent threads from busy looping
Change-Id: I84b999df2da6e769f0da49bfda0409cae564792a
Reviewed-on: http://review.couchbase.org/37056
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
MB-11073: Force schedule snapshot only on deletion

Change-Id: I2e50c5da1684174b74e4b287b664db35ea780c28
Reviewed-on: http://review.couchbase.org/36867
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
Fix sporadic test failure in unit-tests

Change-Id: I2322f0975f7c1ff7f0bbf21b50c844a4abd7ee81
Reviewed-on: http://review.couchbase.org/37090
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
Merge branch '2.5.0'

* 2.5.0:
  MB-10962 Don't snooze the TAP connection notifier unnecessarily.

Conflicts:
	src/tapconnmap.cc
	src/tapconnmap.h

Change-Id: I9e644e4874765e82f5e16cf90677ca6e0e5540ce
MB-10979: set sequence number properly for temporary initial item

When a temporary initial item is being added to the hash table,
the sequence number needs to be set to the max deleted sequence
number as opposed to being zero. There is a possibility that
a set from the client can access this temporary item before
the BG fetcher task can update the metadata with the correct
sequence number.

Change-Id: Iebf5dc70c2cc25ed4a4cf5f6540f1848772491cf
Reviewed-on: http://review.couchbase.org/37104
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
Revert TAP connection notification improvements.

We saw high CPU overhead regression from TAP connection
notification improvements, which was caused by a different
thread scheduling in 3.0.

Change-Id: Icc9ecfcbcee499d1c4ac8b3da67eb337254bb586
Reviewed-on: http://review.couchbase.org/37114
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
Don't print the command name for diskinfo in cbstats

This was likely left in on accident as a debug statement. We don't
print the command for any other stats types and we shouldn't do
it for diskinfo either.

Change-Id: Ied9d55f8cb4d2d4d23e2812f1335c42ec3aea542
Reviewed-on: http://review.couchbase.org/37138
Reviewed-by: David Liao <david.liao@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Use getNKey() instead of getKey().length()

This is a preparation for stop using std::string for the key

Change-Id: I0ccb201cfaa49bbeafd45ad1576e632b300a8106
Reviewed-on: http://review.couchbase.org/36813
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Merge branch '2.5.0'

* 2.5.0:
  Truncate last unaligned bytes of access log
Conflicts:
    src/mutation_log.cc
Change-Id: I021c75f32918d945d06643537e12e7102cab4113
Reviewed-on: http://review.couchbase.org/37128
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
Fix sporadic failure in unit tests when getting upr takeover estimates

The estimates might not be valid immediately because the backfill
task needs to bescheduled onto a background thread. As a result we
should wait for the estimate to be correct as opposed to just
checking it once.

Change-Id: I540cf76194e19018be934f43e097143b01df3e12
Reviewed-on: http://review.couchbase.org/37142
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
MB-10775: Implement all needed upragg stats for use in UI

Implement the following upr stats
(i)   Number of senders: producer count
(ii)  Number of items sent per producer
(iii) Number of items remaining per producer
(iv)  Total bytes sent per producer

Change-Id: I1c5e41ebabd6ccd80008b96545f69762f008c559
Reviewed-on: http://review.couchbase.org/36403
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>
MB-11026 Don't allow the underflow on the dirty queue size stat.

We will find the root cause of the underflow issue separately
soon.

Change-Id: Id5459fe3ac14f850159e9321791137860f265f62
Reviewed-on: http://review.couchbase.org/37153
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>
Move command opcodes to memcached

And rename them all to PROTOCOL_BINARY_

Change-Id: I674cfb8dcf5817d76a973cfda8ef86df32815f51
Reviewed-on: http://review.couchbase.org/37115
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>
Remove compiler warning from comparison

warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     if (blockSize < seek_result) {

Change-Id: Ia014128a362cc192401c411e926e2661071ea178
Reviewed-on: http://review.couchbase.org/37168
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
MB-11136: check for NULL vbucket in getItemsRemaining

When retrieving the number of items remaining to be sent in
the stream, the vbucket should be checked for NULL.

Change-Id: I321d08f222cdc1797c81131a8f4f3cd84abfe5c2
Reviewed-on: http://review.couchbase.org/37211
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>
Don't generate a byseq num for a backfill item on replica vbucket

Change-Id: I46080deebd8e6baae1f7b972b5eaa3259b91a910
Reviewed-on: http://review.couchbase.org/37232
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Reviewed-by: Sriram Ganesan <sriram@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
Use a colon instead of an underscore for the upragg stats seperator

Change-Id: Ia845cc7fa8a213fa6bb64faadeb6978bb12f5525
Reviewed-on: http://review.couchbase.org/37289
Reviewed-by: Sriram Ganesan <sriram@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>
MB-10874 Cleanup of stale db files

Replicated couch_db's cleanup logic of older revisions of db files
and compact files. Compact files are cleanned up during warmup or
after compaction failure. Older revision of db files are cleanned up
during warmup or when bgfetcher thread opens db files.

Change-Id: I207e502f06a16aeaf9e6eb2dfd857c19969bfa2c
Reviewed-on: http://review.couchbase.org/37101
Tested-by: David Liao <david.liao@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
MB-11158: check for NULL stream in getNextItem() of UPR consumer

While retrieving the next item on the UPR consumer, add a NULL check
while accessing a stream for a particular vbucket.

Change-Id: I429337a0194afbe9fd7ee2786a75287498504823
Reviewed-on: http://review.couchbase.org/37306
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: Sundararaman Sridharan <sundar@couchbase.com>