MB-5025 disable flushall by default

Flushall operation is disabled and return as noop by default.
In the cases flushall is necessary, this change-set introduces
an engine parameter to enable the operation.

Reviewed-on: http://review.couchbase.org/15002
Reviewed-by: Dustin Sallings <dustin@spy.net>
Tested-by: Dustin Sallings <dustin@spy.net>
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>

Conflicts:

	Makefile.am
	docs/engine-params.org
	ep_engine.cc
	ep_testsuite.cc

Change-Id: Iaf86ced50723bf11f88473d0b3f9595011e6553b
Reviewed-on: http://review.couchbase.org/15258
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Remove definitions for start/stop replication

These are not used anywhere in ep-engine and probably should have
been removed a while ago.

Reviewed-on: http://review.couchbase.org/15046
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>

Conflicts:

	command_ids.h
	management/mc_bin_client.py
	management/memcacheConstants.py

Change-Id: If3443878433876e4606c91faede684cc294dc3b1
Reviewed-on: http://review.couchbase.org/15259
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
MB-4889: Don't notify tap when shutting down

On shutdown we delete the tapconnmap and tap connections before
stopping the dispatcher threads. In this particular case the
nio-dispatcher thread was notifying tap during shutdown. This fix
causes us to check for shutdown before notifying tap.

Reviewed-on: http://review.couchbase.org/14320
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>

Conflicts:

	ep_engine.cc
	ep_engine.h

Change-Id: I87ae317e21953f0ea71c64e757eb2b9c339d341e
Reviewed-on: http://review.couchbase.org/15260
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
MB-5136 Don't sleep if there are any new pending notifications.

Because the notification thread always sleeps for one second
through TapConnMap.wait(), it may result in one second delay even
if a new notification occurs when the notification thread is not
"parked" in wait. This one sec delay was also frequently observed
in vbucket takeover.

This change maintains cas-like notification counter variable that
is incremented per new notification. The notification thread
will then sleep for one sec iff the current notification counter
is different from its previous value. This change provides faster
rebalance operations.

Reviewed-on: http://review.couchbase.org/15119
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>

Conflicts:

	ep_engine.cc
	tapconnmap.hh

Change-Id: I9f307b3761ec4ba8ecf9bc00d546b633cb67b54e
Reviewed-on: http://review.couchbase.org/15261
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Remove unused connection reserve / release functions

Reviewed-on: http://review.couchbase.org/15040
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

Conflicts:

	backfill.hh

Change-Id: Ie7da3f4f206b9f550f0b1b9d343d3263fa8f4982
Reviewed-on: http://review.couchbase.org/15262
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Correct the format of the backfill completion log message.

Change-Id: I5a57765c03f75c9680900f650124fa7b26270bdc
Reviewed-on: http://review.couchbase.org/15132
Tested-by: Trond Norbye <trond.norbye@gmail.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Reviewed-on: http://review.couchbase.org/15263
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
MB-5092 Use std::set to maintain the list of VB ids in VBFilter

Using std::vector introduced some bugs in maintaining the list
of sorted unique vbucket ids in VBFilter instance. This change
fixes this issue by using std::set instead.

Change-Id: I531a080cddae356b897a4feab1d5a74dc2de1ced
Reviewed-on: http://review.couchbase.org/15179
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-on: http://review.couchbase.org/15264
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Merge branch 'branch-18'

* branch-18:
  MB-5092 Use std::set to maintain the list of VB ids in VBFilter
  Correct the format of the backfill completion log message.
  Remove unused connection reserve / release functions
  MB-5136 Don't sleep if there are any new pending notifications.
  MB-4889: Don't notify tap when shutting down
  Remove definitions for start/stop replication
  MB-5025 disable flushall by default

Change-Id: I60dd145e2dc3338419455df968a2bb4207a214ad
MB-5171 do not decrement numNonResident during initial warmup

Decrementing numNonResident during the warmup caused the total resident
ratio and cbstats' num_non_resident attributes to become negative. Each
initial insert of items during the warmup should never decrement the
numNonResident.

Change-Id: I514f1b1edbdb8fe3bdc95650b890b383a7b5e6a6
Reviewed-on: http://review.couchbase.org/15308
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Return nothing for memory allocation stats if disabled

Change-Id: I7f0e9ff0f3a3f1bacf47a4fe0f78268256540312
Reviewed-on: http://review.couchbase.org/15315
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Remove unused tap connection validity functions.

Change-Id: I541419cb948d892717ff95bb539b1dda9d1281bf
Reviewed-on: http://review.couchbase.org/15347
Tested-by: Trond Norbye <trond.norbye@gmail.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Reviewed-on: http://review.couchbase.org/15395
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
MB-5092 Queue non-existent item deletion into checkpoints.

An item might not exist in the replica vbucket when its deletion
is received from the active vbucket, if that item was already
expired and deleted from the replica vbucket. In this case,
an item deletion should be still pushed into a checkpoint for
replication into another replica vbucket.

Change-Id: I2bd079c09ffcb0c7c54a4898ba7a497c279f74c6
Reviewed-on: http://review.couchbase.org/15348
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-on: http://review.couchbase.org/15396
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Release the lock after releasing the dead TAP connections.

Change-Id: Ifd86dcc7a58a4f34ba223d9cb166a275ab92d0ce
Reviewed-on: http://review.couchbase.org/15360
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-on: http://review.couchbase.org/15397
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Add an extra reference to the tap connection during notifications

This change is used to try to determine where the race condition
problem occurs.

Change-Id: I93ad82715e8b334a3c883dc59525dadaaaeb1cac
Reviewed-on: http://review.couchbase.org/15349
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-on: http://review.couchbase.org/15398
Remove unnecessary TAP notify calls.

Change-Id: Idadfc3eaac92f124832840fcffb6206b63d393bc
Reviewed-on: http://review.couchbase.org/15363
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-on: http://review.couchbase.org/15399
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
MB-5092 Don't drop items from dead vbucket checkpoints.

With the new swap rebalance algorithm, there is a very small time
window between (1) the active vbucket becomes a dead state after
takeover, and (2) then reset to replica state. This change doesn't
drop items from dead vbuckets for more reliable replication with
the new rebalance algorithm.

Change-Id: Ieb8a97314a6f567c5c2ae5c853db6b1271f8b971
Reviewed-on: http://review.couchbase.org/15364
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-on: http://review.couchbase.org/15400
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Queue non-existent replica item deletion into checkpoints

We shouldn't push non-existence active item deletion into
checkpoints. This change resolves the issue from the
previous commit ee319a9093bb766a05294b78a2b45e9584fb1ac6.

Change-Id: I78ea5a4df58a1117e5a494cdbcfcb251d2b434e8
Reviewed-on: http://review.couchbase.org/15369
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-on: http://review.couchbase.org/15401
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Merge branch 'branch-18'

* branch-18:
  Queue non-existent replica item deletion into checkpoints
  MB-5092 Don't drop items from dead vbucket checkpoints.
  Remove unnecessary TAP notify calls.
  Add an extra reference to the tap connection during notifications
  Release the lock after releasing the dead TAP connections.
  MB-5092 Queue non-existent item deletion into checkpoints.
  Remove unused tap connection validity functions.

Change-Id: Iac4a7d86e0dde4776b4a3e6c3590d0cfd18877ec
Fix warning about unused variable

Happens on Linux with gcc 4.6.1.

Change-Id: Ic72369f342a1cfa6a6e9c881f8e06bebd2c8ddfd
Reviewed-on: http://review.couchbase.org/15431
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Fix win make

Change-Id: I13b5c99fc1f7c0ffa3ceff7cf2c65f0bb1113a1c
Reviewed-on: http://review.couchbase.org/15429
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>