MB-12642:bp:MB-10437:prefer oldest waiter in concurrency_throttle

Old behavior could choose same pid(s) over and over again. Which
caused particular problem for initial replication under slight udpates
load. I.e. it would repeatedly pick already mostly up-to-date vbuckets
over vbuckets that still need to be replicated. Which would cause very
short batches which in turn would cause very slow rebalance.

By picking oldest waiter (modulo system clock jumps which should be
infrequent) we avoid this problem. It should also help at least
somewhat for the case of ongoing replication under somewhat heavy but
not too heavy load, due to better avoidance of too short replication
batches.

Change-Id: Ie6dcafabd510367015c1e5897a620e37eb7a17f5
Orig-Reviewed-on: http://review.couchbase.org/38368
Reviewed-on: http://review.couchbase.org/43227
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-12643:bp:MB-11631 Ensure terminate is called if ensure_bucket succeeds.

This is backported as part of backporting MB-12156.

Change-Id: Ib03c6034c8d63e7c6a1d027f34992c9c63608183
Orig-Reviewed-on: http://review.couchbase.org/39197
Reviewed-on: http://review.couchbase.org/43228
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-12643:bp:MB-12156 Don't delete buckets when changing datapath.

Change-Id: I0039e0bfa7353cd784791e6f7aca7d73a89778b7
Orig-Reviewed-on: http://review.couchbase.org/41332
Reviewed-on: http://review.couchbase.org/43229
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-12643:bp:MB-12156 Don't trap exits while change disk paths.

It's not needed and prevents us from a situation where process gets
stuck and wakes up much later (potentially after node became part of a
cluster) and thus invalidates the check that node is not provisioned.

Change-Id: Ic0d5b9a05088b0d376002338109d80cabc44f0ed
Orig-Reviewed-on: http://review.couchbase.org/41333
Reviewed-on: http://review.couchbase.org/43230
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Merge commit 'cc8cca2b4df9e58568289984de9da15daf073e1b' into rel-3.0.0++

* commit 'cc8cca2b4df9e58568289984de9da15daf073e1b':
  MB-12643:bp:MB-12156 Don't trap exits while change disk paths.
  MB-12643:bp:MB-12156 Don't delete buckets when changing datapath.
  MB-12643:bp:MB-11631 Ensure terminate is called if ensure_bucket succeeds.
  MB-12642:bp:MB-10437:prefer oldest waiter in concurrency_throttle

This is empty merge of backported commits.

Conflicts:
	src/concurrency_throttle.erl

Change-Id: Ied5dd16f7896636ea3d9a39abb616c61a0925b1e