Simplify the addNewCheckpoint function

Since we keep all cursors in the same list we no longer need to
treat the persistence cursor as special.

Change-Id: Idadb06b873b31587f2eb3d4f57f485165ff1a0e0
Reviewed-on: http://review.couchbase.org/42928
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
No need to close the open checkpoint id, happens automatically

Change-Id: I5b27c26c5858c47d2a5bab29e68123ff1302833c
Reviewed-on: http://review.couchbase.org/42929
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-12598: Add breaks to switch-case to ensure proper behavior

This was leading to a crash since not having breaks meant that we
might execute a transition improperly.

Change-Id: I9c71f01517611edb58883c6d5b120cbafaab22ab
Reviewed-on: http://review.couchbase.org/43057
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-12179: Add the backfill manager

The backfill manager will be used for scheduling backfills on a per
connection basis and in a future commit will be used for managing
the memory of a dcp connections backfills. The backfill manager will
keep track of all of the active backfills for a given connection and
will run them in a round robin fashion. This will ensure that all
backfills for a given connection are treated equally.

Change-Id: Iaf0c7f4b5ff07bb1ca87e2db3684d0146d91354c
Reviewed-on: http://review.couchbase.org/42314
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-12179: Allow backfills to be intervleaved

Provides the structure to pause and resume backfills.

Change-Id: I96df9076885878eea46eff32213b871c7ba3e61d
Reviewed-on: http://review.couchbase.org/42315
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-12179: Pause backfills when after 'x' of items or bytes are read

All dcp backfills are will be paused after reading 4MB or 4096 items
and if any other backfills are scheduled then those will be run before
the original backfill is run again. This change allows us to interleave
backfills so that we no longer have to run a backfill for a single
vbucket to completion before starting the next one.

Change-Id: Ibdb455a5bd931e0276dad61d686f9dbf507ed7af
Reviewed-on: http://review.couchbase.org/42316
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-12179: Add initial max buffer to the BackfillManager

The backfill manager will only allow up to 20MB to be backfilled
from disk without being sent over the wire to the consumer. This
change will help to prevent growing memory usage from backfills
when a vbucket has a large dataset.

Change-Id: Ie505f2a77d0f0fe56a1b63f9a57edc6c11a11780
Reviewed-on: http://review.couchbase.org/42319
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Move decrCursor/Pos out of the CheckpointManager

These functions do not require any state from the CheckpointManager
and should be moved to the CheckpointCursor class.

Change-Id: I9b924b120fde85ce9ecafbee131ab6bf9006e49a
Reviewed-on: http://review.couchbase.org/43071
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Don't require a checkpoint id when closing the open checkpoint

Change-Id: I2780e79351cc4a4f9746f56a6736e45a67ab691e
Reviewed-on: http://review.couchbase.org/42930
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
Fix sporadic unit test failure in rollback test

This issue was caused because pausing the flusher failed. The
reason was that we didn't wait for warmup and it was possible that
the command was sent before the flusher was even initialized.

Change-Id: I52da80c2d6d76f658f1525195ce521ac9dce55d5
Reviewed-on: http://review.couchbase.org/43076
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-12540: Refactor the way we track snapshot start/end seqnos

For the 3.0 release I needed to add snapshot sequence number
quickly and in doing so implemented an approach that was not
very good and prone to errors. The correct approach is to track
sequence numbers on disk and also to track them for each checkpoint
in the checkpoint manager. This change moves us to the better
approach.

Change-Id: Ifbdd6c87b06767ff8c6504c1058ce11195fa2428