MB-5968: re-impl. vbucket states & replications orchestration

Part of this commit is implementation of new backwards compatibility
handling approach. There's no ns_vbm_new_sup anymore. And there are
no cb_gen_vbm_sup too.

New backwards compat handling approach is implemented in
cluster_compat_mode module. Basic idea is to store cluster
compatibility mode in ns_config. When it's safe (before janitor runs
and rebalance runs) orchestrator process on master node will check if
cluster can increase it's compat mode. This happens if all nodes
support higher mode. If it can it does, orchestrator then can
restart/kill services. Currently there's two modes 'undefined' (1.8.x)
and 2.0. Cluster _does not_ automagically switches to lower compat
mode. And disallows older nodes to join cluster.

If somebody needs to do rolling downgrade, there's half supported
cluster_compat_mode:force_compat_version/1 function that can be called
via /diag/eval. Calling this function will force cluster into specific
compat mode and disallow any upgrades. Calling
cluster_compat_mode:un_force_compat_version/0 will removed that
'disallow'-ance.

Most benefit of this approach is simplicity and safety of checking
cluster mode and doing things depending on current version. I.e. new
behavior is enabled when we know entire cluster supports it, and
cannot easily 'undo' this support as rolling downgrade is not directly
supported and requires careful manual action.

1.8.x compat mode is doing things compatibly with 1.8.x
nodes. Particularly, we're now using mostly 1.8.1 version of
ns_vbm_sup (because cb_gen_vbm_sup is gone). That means of course,
'replicator on source'. And most importantly 'direct' replication and
vbucket states changes. I.e. janitor/vbucket_mover are doing remote
calls to ns_vbm_sup and ns_memcached directly.

It should be noted that failover safeness computation code is not
handling 1.8.x backwards compat at all. I.e. trying to failover 2.0
node in mixed versions cluster will always present user this somewhat
scary warning "some data is not replicated".

2.0 compat mode is designed to be more robust and more future
compatible. And it makes index configuration management much
easier.

All changes are performed via janitor_agent service that runs on each
node. That orders 'vbucket state in broad sense' operations making
things less error prone. I.e. call to return current state of vbuckets
and replications can only succeed after previous mutation calls are
completed.

janitor_agent also drives capi_set_view_manager directly. That's most
important user-visible change. Previously capi_set_view_manager was
trying to infer correct index states. I.e. if it saw vbucket in
fast-forward map as master it'll add it to pending index state if and
when possible. Old behavior was quite complex.

Because we're now having explicitly designed API it's easy to pass
index states in calls to janitor_agent. Which will then make sure
they're passed to capi_set_view_manager.

Because janitor mutations are now via janitor_agent I've also
separated main logic of janitor into purely functional code. And
cleaned up janitor a bit.

Change-Id: If58f4508872b58ade6e0e690537b076c77df0f25
Reviewed-on: http://review.couchbase.org/18637
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-5307: ignore {error, view_undefined} for set views

Somehow sometimes it throws {error, view_undefined} when we only
expect view_undefined to be thrown. Let's try to just ignore it like
plain view_undefined 'throw'.

Change-Id: I37695277787c4bdfc2da7a1ab8c5d4854ed2c5a7
Reviewed-on: http://review.couchbase.org/18752
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-5307 Introduce a means of upgrading global keys in config.

It was not possible before for two reasons. First, ns_config_default
upgrade path is executed only during off-line upgrade. Second, it's
safe to upgrade only per-node keys there. Thus
ns_online_config_upgrader is introduced. It's called by
ns_orchestrator as part of switching compatibility mode procedure.

Change-Id: I49c2779779aa93cdf8d7476d82887d391beb1258
Reviewed-on: http://review.couchbase.org/18825
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-5307 Add UUIDs to already existent buckets.

Change-Id: I48a8dbef4bd3e83d755e30038427dd9a7b7e91fa
Reviewed-on: http://review.couchbase.org/18827
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-5307: implemented misc:is_binary_ends_with

Change-Id: I270d84d0d72219bb0395a3f4b97330579f15cfdb
Reviewed-on: http://review.couchbase.org/18796
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-5307: made ebucketmigrator_srv:had_backfill reply after ack

Previous implementation was incorrectly assuming that seeing
backfill-ness evidence message is enough. But at least of initial
stream message, which is positive evidence of backfill, it's possible
to have result of had_backfill before resetVbucket on destination is
really done. Which we don't want as we want 'stable' checkpoint ids in
destination after had_backfill returns.

Change-Id: Ibeb496b592f8668ff28c22d167dc3237572fbb15
Reviewed-on: http://review.couchbase.org/18797
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>