MB-9343 allow to stop rebalance if it hanged in some janitor_agent

.. calls

Change-Id: I1ae9b3bca9368a3d8bda53fe9242bc6d1c278222
Reviewed-on: http://review.couchbase.org/34691
Tested-by: Artem Stemkovski <artem@couchbase.com>
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-10515: made 2.5.1 nodes take over mastership over 2.5.0 nodes

I.e. in order for MB-10220 fixes to be effective.

Change-Id: Ic74bd90cd5e8d7f0edb1e7386c06baad898440b2
Reviewed-on: http://review.couchbase.org/34729
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Merge remote-tracking branch 'origin/for-rackaware' into master

* origin/for-rackaware:
  MB-10515: made 2.5.1 nodes take over mastership over 2.5.0 nodes

The change is empty because 3.0 is advertising 3.0 as mb_master
version.

Change-Id: I4432b7955eeb6d0cba9068a32c531e0d049edab9
MB-10459: added sslServerList to bucket info

So that clients can discover ssl ports in accordance with our
agreement with sdk people.

Change-Id: If8d9a03e81dffa567269daf4c46753a568bb479b
Reviewed-on: http://review.couchbase.org/34686
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Matt Ingenthron <matt@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
MB-10438: dont swap args order in wait_for_data_move_on_one_node

Change-Id: Id8f2ac0faff2ce56f08c5ee19a09e4435d030604
Reviewed-on: http://review.couchbase.org/34734
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
bp:MB-10051: configurable disk and metadata limits

Change-Id: Icfdd2e580149e54b9627b2da460407eae0a8285b
Orig-Author: Brent Woodruff <brent@fprimex.com>
Orig-Reviewed-on: http://review.couchbase.org/34265
Reviewed-on: http://review.couchbase.org/34742
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Merge remote-tracking branch 'origin/for-rackaware' into master

* origin/for-rackaware:
  bp:MB-10051: configurable disk and metadata limits

This merge is empty because it's merging back a backport

Change-Id: Ie233b37e576684e194c2d6837e02493521944fa0
MB-100 unbreak ns_config:test_cas_config

Change-Id: Ic21649e1791743c7ff5d611576e5966a97d2cdf4
Reviewed-on: http://review.couchbase.org/34754
Tested-by: Artem Stemkovski <artem@couchbase.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
MB-10400 notifier open upr connection and stream_request commands

Change-Id: I0a3bea32d0cba0ad6cea355d451cd2977a2df799
Reviewed-on: http://review.couchbase.org/34453
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-10400 upr notifier

unblocks subscribe call if the partition gets some data
after the start seqno

supports multiple subscribers per partition

Change-Id: I408178cc8755e4e720c65de6a1b068515e278c77
Reviewed-on: http://review.couchbase.org/34444
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-10118 Eliminate old _all_docs.

Change-Id: I374c0a4cbd747efe9c48223c772a957f69c4f80a
Reviewed-on: http://review.couchbase.org/34753
Reviewed-by: Artem Stemkovski <artem@couchbase.com>
Tested-by: Artem Stemkovski <artem@couchbase.com>
MB-10118 Reimplement docs REST endpoint using get_keys command.

Our approach is as follows:

  1. The node that handles the all docs request sends subrequests via
     ns_memcached to all the nodes in the cluster. It passes a list of
     vbuckets it expects destination node to have and all request
     parameters with two exceptions: 'skip' is not sent, 'limit' is
     set to 'original limit' + 'skip'. After receiving all the nodes'
     responses, the coordinating node skips 'skip' smallest items and
     returns next 'limit' number of items.

     Coordinating node will retry the request fixed amount of times
     with some delay upon receiving not_my_vbucket error from any of
     the nodes.

  2. Each node that handles all docs subrequest sends a get_keys
     command for each of the vbuckets. Assuming that keys span
     vbuckets more or less evenly, the limit it uses is
     2 * ('original limit' / 'number of vbuckets' + 1). Then it gets
     'original limit' number of keys from results. If for any of the
     vbuckets the list of keys gets exhausted, another request is sent
     to receive more keys. After this is done, as a separate stage the
     values for the selected keys are retrieved (as long as
     include_docs parameter is set to true). Care is taken to select
     more keys if any of the items happen to be deleted by the time
     we try to get their bodies.

Change-Id: I7f3f981e90cb190b44b21a8a6949c3acf7f0989c
Reviewed-on: http://review.couchbase.org/34752
Reviewed-by: Artem Stemkovski <artem@couchbase.com>
Tested-by: Artem Stemkovski <artem@couchbase.com>
MB-10544: handle https sockets in menelaus_web:handle_streaming

Particularly inet:setopts on ssl socket does not work. Correct
solution is to use mochiweb_socket layer that deals with plain and ssl
sockets transparently.

Change-Id: Ib55538f60b2a3e81b466c01ee59bd57aba9d250b
Reviewed-on: http://review.couchbase.org/34861
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-10465 eliminate races between producer and consumer sockets

... during takeover

1. obtain stream opaque from stream_req because it for sure will
   come earlier than set_vbucket_state
2. change set_vbucket_state from cast to call to guarantee that
   set_vbucket_state won't be proxied on producer before the consumer
   changed its state

Change-Id: I425373777aea5424c2d6ec172608c922d865b434
Reviewed-on: http://review.couchbase.org/34881
Tested-by: Artem Stemkovski <artem@couchbase.com>
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-10400: use wait_for_seqno_persistence for xdcr checkpointing

Change-Id: Id9d83497e3d5aa914832266f704b3704c9ec3c8d
Reviewed-on: http://review.couchbase.org/34806
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-10400: added ns_server_testrunner_api:shutdown_nicely/0

I.e. for testing clean shutdown of couchbase

Change-Id: I6830af486880a823be8ac6b958a9e03bb7b53645
Reviewed-on: http://review.couchbase.org/34807
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-10400: don't import a bunch of couch_util functions in header file

Change-Id: I2472efedca9c0789d4648678a1260901707928ed
Reviewed-on: http://review.couchbase.org/34808
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-10400: commented out "behind purger check"

which we cannot yet do in upr world

Change-Id: Iab4a041e5d0c98e619b7339ffeee37fa370ca4a2
Reviewed-on: http://review.couchbase.org/34809
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-10400: use better api for opening src master vbucket

Change-Id: I93ce7bab82c53b56cbc4ea5c0606a9c2fe370b60
Reviewed-on: http://review.couchbase.org/34810
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-10400: removed duplication in queue_fetch_loop

Change-Id: I930e163f8d5b39d4f1e327b22ead24425cdb036b
Reviewed-on: http://review.couchbase.org/34811
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>