Truncate last unaligned bytes of access log

Truncating unaligned bytes of access log allows use of access log
even if last bytes are unaligned to blocksize.
Avoid throwing a ShortReadException which just disables the access log
Change-Id: I6e82fb0aa4b62e892e092c98654df71bee3f16d6
Reviewed-on: http://review.couchbase.org/36995
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: buildbot <build@couchbase.com>
Fix deadlock in checkpoint persistence command

We need to release the hpChkMutex before notifying memcached in
order to prevent a deadlock. The deadlock occurs when the flusher
trys to notify memcached of checkpoint persistence at the same
time a memcached worker thread is trying to add a new checkpoint
persistence request.

Change-Id: Ida313f5b39ef0e063dee9882410cd0a19ce55292
Reviewed-on: http://review.couchbase.org/41363
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Merge remote-tracking branch 'gerrit/2.5.0' into 3.0.1

Change-Id: I59c45877fc783e30c86dcfd7d303c731cde433fc
MB-11999: Preferential loading of active/replica vbuckets

During warmup, order the vbucket list such that
active vbuckets get 60% preference while replica
vbuckets get 40% preference.

Example:
In a 4 node cluster (DGM), approx. resident ratios:
1. Before warmup
    active:     36%     34%     34%     41%
    replica:    33%     35%     35%     27%
2. After warmup
    active:     40%     42%     41%     46%
    replica:    31%     29%     29%     24%

Change-Id: I60e0427bca58530247086d730135ebb4be70bb84
Reviewed-on: http://review.couchbase.org/41468
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>
Move persistence cursor into list of all cursors

We have a lot of duplicate code for handling the persistence and
tap cursors. All of this code functions in the same way so it's
better if we just keep all of the cursors in the checkpoint manager
in one place.

Change-Id: Ifab7331a7e0ef7b8528a6e3323847acd1e14caa9
Reviewed-on: http://review.couchbase.org/41459
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Remove hasNextForPersistence()

We no longer need this function because the hasNext() function can
be used to figure out if the persistence cursor has another item.

Change-Id: Idf315e4922f1d1c1e6fc7401b5477ae5ef824978
Reviewed-on: http://review.couchbase.org/41506
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Persistence cursor shouldn't be registered to be reset on collapse

If checkpoints are collapsed we shouldn't put the cursor back into
the collapsed checkpoint.

Change-Id: I059a3833defe05cb2218dcb677e732a532511801
Reviewed-on: http://review.couchbase.org/41558
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Fix checkpoint unit test caused by expecting wrong num of cursors

We put all of the cursors in the same structure now so we will
always have at least one cursor since we never remove the
persistence cursor.

Change-Id: Iea9e559ea72e200d71c36fcc20bd5b67c7435d10
Reviewed-on: http://review.couchbase.org/41559
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Remove duplicate code for getting number of items for a cursor

We no longer need this function because the getNumItemsForCursor()
function can be used to figure out how many items are remaining
for the persistence cursor.

Change-Id: Ibe5a515785dbaa2c3d40582d855504079b3a83f2
Reviewed-on: http://review.couchbase.org/41507
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Remove unneeded function getMutationIdByKey()

This function was only used by our testing framework and since each
queued_item instance contains the mutation id (by seqno) we no
longer need this function in order to get this information.

Change-Id: I8e1fbe31380c0cc1837342fe3ac9e65d570a116f
Reviewed-on: http://review.couchbase.org/41508
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Rename checkpoint manager functions with TAP in them

We use the checkpoint maanger for many different cursor types so
we shouldn't have TAP in the names of functions in order to
advertise them as being more general.

Change-Id: I2f17d4db9332b2f5c52cb4e0764bf76bb982864a
Reviewed-on: http://review.couchbase.org/41509
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Remove unused endSeqno in nextItem function

Change-Id: I834734b471be3082e709d958ec8409a626e87dab
Reviewed-on: http://review.couchbase.org/41561
Tested-by: Michael Wiederhold <mike@couchbase.com>
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Remove duplicate code for getting all items from checkpoint

Change-Id: I320cddf9ecd7ae0947abaf33add5065ce50f442c
Reviewed-on: http://review.couchbase.org/41560
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Merge remote-tracking branch 'gerrit/3.0.1' into master

Change-Id: I0a0424c293f085fb83cd6e7c9765994b0f817f45
MB-12117: Release hashtable locks before disk IO

While in access log generation, we'll need to release
all hashtable partition locks before we create new
entries in the mutation log.

Change-Id: Ic3dd0a02452b51ee742e30a0f268b86f9ab6205b
Reviewed-on: http://review.couchbase.org/41572
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>
Account for keylength while allocating buffer for allKeys

While increasing the buffersize (when needed) when the
AllKeysAPI is invoked, we will need to account for two
additional bytes while allocating the key, for its length.

Change-Id: Iba68c4ae7bccae20f97d4e98350d5105093c3487
Reviewed-on: http://review.couchbase.org/41595
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>
Remove unused errinfo field (will be removed from API)

Change-Id: Ib8cd1426d313ea9493076899f826250f07992a9a
Reviewed-on: http://review.couchbase.org/41639
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
MB-12160: Use appropriate CAS for comparison with locked items

In the case of setWithMeta/deleteWithMeta commands, the locked item's
CAS value needs to be compared with the incoming mutation's CAS.

Change-Id: Id12a3c4717b18bc41c3d4b7dded99ea215179e9d
Reviewed-on: http://review.couchbase.org/41594
Tested-by: Sriram Ganesan <sriram@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Merge remote-tracking branch 'gerrit/3.0.1' into Master

Change-Id: Id580f60dabe6e9541cf21fbb52937acfda6efd27
MB-12268: Adding a debug log, when clusterConfig is updated

Change-Id: I0cbb70789f84bddc644bdc4ee59c1e86e1821220
Reviewed-on: http://review.couchbase.org/41746
Reviewed-by: Sundararaman Sridharan <sundar@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>