MB-5000 Eject master as late as possible.

Since ns_orchestrator is responsible for setting rebalance_status
after rebalance is completed (due to any reason), it's not completely
correct to eject master in ns_rebalancer. Because in this case these
final config changes can be left unreplicated. To prevent this master
is ejected now only after all the other final work is done by
ns_orchestrator.

Change-Id: Ic3ab163d699892acd58d754b80ef789f04310ec3
Reviewed-on: http://review.couchbase.org/14853
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Remove to verbose logging from ns_bucket_sup.

These messages are mostly redundant since the same information can be
acquired from ns_config_log logging. And since they generate enormous
amount of message during rebalance, we just remove them.

Change-Id: I535af53e84ea0204af75635f4460aabcc1018a93
Reviewed-on: http://review.couchbase.org/14845
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Convert ns_config_log to gen_server and remove unused code.

Change-Id: I5ba40f3eed7a563a1b9e9897828a30cfae013a03
Reviewed-on: http://review.couchbase.org/14843
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Special case buckets key when logging config changes.

Since a value corresponding to 'buckets' key can be changed very often
during rebalance (which generates huge log files), we won't print full
vbucket map and fast forward map to the log. Instead, only diffs of
those will be printed.

Change-Id: Icfc0464317e41b49de9f27c98092b88ec37d4dc6
Reviewed-on: http://review.couchbase.org/14844
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-5087 - moved statstore URL

Change-Id: I60598cffa296354241f6f9d506fd77523a5d1ddb
Reviewed-on: http://review.couchbase.org/14923
Tested-by: Farshid Ghods <farshid@couchbase.com>
Reviewed-by: Farshid Ghods <farshid@couchbase.com>
Report failure to connect to memcached correctly.

Seen in MB-5080.

Change-Id: I22adf90a583b7dcdd0a987fbfa86d792649a120d
Reviewed-on: http://review.couchbase.org/14902
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Handle rebalance request correctly when it's already running.

Seen in MB-5080.

Change-Id: Id7a64c724003f8a5b916e5d2379785f738b4fad9
Reviewed-on: http://review.couchbase.org/14903
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-5060 Remove bucket from remaining_buckets upon deletion.

If user request bucket deletion when janitor run is in progress, we
shut janitor down but we used to keep remaining_bucket list intact. It
means that in certain circumstances subsequent janitor run would try
to cleanup the bucket that had just been deleted.

Change-Id: Ic5a12b4a320ea7fe547d0e54ee0d8c695848e06b
Reviewed-on: http://review.couchbase.org/14957
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
don't allow attempts to "rebalance out" unknown nodes.MB-5062

It was previously possible to ask rebalancing out of node that's not
part of cluster anymore. This fix will return nodes mismatch if
somebody tries to rebalance out unknown node.

Actual fix is verifying that EjectedNodes are subset of KnownNodes.

Change-Id: Ie8a6713070850e5b705f91a33d6a8abd31ccd07c
Reviewed-on: http://review.couchbase.org/14969
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
force-refresh pool details before creating bucket. MB-5019

We're generally supposed to have correct and automagically updated
pool details all the time. But there are some issues correctly
spotting that all node's status is updated after bucket
deletion. We're often seeing pool details being updated after first
node completes bucket deletion, and subsequent updates do not
currently initiate pool details update.

Simple workaround is to force update pool details right before
creating bucket. Simple and effective.

Change-Id: I5a94251e6e1e0b44cc0871723c57b6148c744e34
Reviewed-on: http://review.couchbase.org/14971
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
do enter gen_server loop when no vbuckets are ready.MB-5052

Otherwise changing filter doesn't work.

Change-Id: I26895af89d8cc17ea1349c6343f777c763cb92bc
Reviewed-on: http://review.couchbase.org/14970
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
added "not ready vbuckets" ebucketmigrator master event.

As can be seen in MB-5052 we're having one of them here but it should
not happen. Also currently ebucketmigrator events log only ready
vbuckets (i.e. vbuckets actually used) without any way to communicate
full set of vbuckets. Now we have both ready and not ready (if any)
vbuckets visible through master events.

Change-Id: Ie868803c2c7d7383c438c7ec2deb0f32f9d8f415
Reviewed-on: http://review.couchbase.org/15005
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Unexport confusing and unused ns_config:reload. MB-5110

Change-Id: I2fed607a4d5ffff6fd8389a1a1b94d43195a17ab
Reviewed-on: http://review.couchbase.org/15006
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Merge remote-tracking branch 'membase/branch-181' into merge

* membase/branch-181:
  Unexport confusing and unused ns_config:reload. MB-5110
  added "not ready vbuckets" ebucketmigrator master event.
  do enter gen_server loop when no vbuckets are ready.MB-5052
  force-refresh pool details before creating bucket. MB-5019
  don't allow attempts to "rebalance out" unknown nodes.MB-5062
  MB-5060 Remove bucket from remaining_buckets upon deletion.
  Handle rebalance request correctly when it's already running.
  Report failure to connect to memcached correctly.
  MB-5087 - moved statstore URL
  Special case buckets key when logging config changes.
  Convert ns_config_log to gen_server and remove unused code.
  Remove to verbose logging from ns_bucket_sup.
  MB-5000 Eject master as late as possible.

Conflicts:
	src/ns_rebalancer.erl

Change-Id: I0309238026a397f4201231fb27c96e0ae89490fd