MB-5307: removed obsolete ns_server supervision tree image

Change-Id: I6ff3372703824bf05404d31728dd6b6237b6ac93
MB-5307: updated cluster_run ports info in README

Change-Id: Ie9764a9e5edcf23af37ecc81fca6993a2f1d97ef
MB-5968: re-impl. vbucket states & replications orchestration

Part of this commit is implementation of new backwards compatibility
handling approach. There's no ns_vbm_new_sup anymore. And there are
no cb_gen_vbm_sup too.

New backwards compat handling approach is implemented in
cluster_compat_mode module. Basic idea is to store cluster
compatibility mode in ns_config. When it's safe (before janitor runs
and rebalance runs) orchestrator process on master node will check if
cluster can increase it's compat mode. This happens if all nodes
support higher mode. If it can it does, orchestrator then can
restart/kill services. Currently there's two modes 'undefined' (1.8.x)
and 2.0. Cluster _does not_ automagically switches to lower compat
mode. And disallows older nodes to join cluster.

If somebody needs to do rolling downgrade, there's half supported
cluster_compat_mode:force_compat_version/1 function that can be called
via /diag/eval. Calling this function will force cluster into specific
compat mode and disallow any upgrades. Calling
cluster_compat_mode:un_force_compat_version/0 will removed that
'disallow'-ance.

Most benefit of this approach is simplicity and safety of checking
cluster mode and doing things depending on current version. I.e. new
behavior is enabled when we know entire cluster supports it, and
cannot easily 'undo' this support as rolling downgrade is not directly
supported and requires careful manual action.

1.8.x compat mode is doing things compatibly with 1.8.x
nodes. Particularly, we're now using mostly 1.8.1 version of
ns_vbm_sup (because cb_gen_vbm_sup is gone). That means of course,
'replicator on source'. And most importantly 'direct' replication and
vbucket states changes. I.e. janitor/vbucket_mover are doing remote
calls to ns_vbm_sup and ns_memcached directly.

It should be noted that failover safeness computation code is not
handling 1.8.x backwards compat at all. I.e. trying to failover 2.0
node in mixed versions cluster will always present user this somewhat
scary warning "some data is not replicated".

2.0 compat mode is designed to be more robust and more future
compatible. And it makes index configuration management much
easier.

All changes are performed via janitor_agent service that runs on each
node. That orders 'vbucket state in broad sense' operations making
things less error prone. I.e. call to return current state of vbuckets
and replications can only succeed after previous mutation calls are
completed.

janitor_agent also drives capi_set_view_manager directly. That's most
important user-visible change. Previously capi_set_view_manager was
trying to infer correct index states. I.e. if it saw vbucket in
fast-forward map as master it'll add it to pending index state if and
when possible. Old behavior was quite complex.

Because we're now having explicitly designed API it's easy to pass
index states in calls to janitor_agent. Which will then make sure
they're passed to capi_set_view_manager.

Because janitor mutations are now via janitor_agent I've also
separated main logic of janitor into purely functional code. And
cleaned up janitor a bit.

Change-Id: If58f4508872b58ade6e0e690537b076c77df0f25
MB-5307: ignore {error, view_undefined} for set views

Somehow sometimes it throws {error, view_undefined} when we only
expect view_undefined to be thrown. Let's try to just ignore it like
plain view_undefined 'throw'.

Change-Id: I37695277787c4bdfc2da7a1ab8c5d4854ed2c5a7
MB-5307 Introduce a means of upgrading global keys in config.

It was not possible before for two reasons. First, ns_config_default
upgrade path is executed only during off-line upgrade. Second, it's
safe to upgrade only per-node keys there. Thus
ns_online_config_upgrader is introduced. It's called by
ns_orchestrator as part of switching compatibility mode procedure.

Change-Id: I49c2779779aa93cdf8d7476d82887d391beb1258
MB-5307 Introduce per-bucket UUID.

Add the UUID to the bucket details output. And add it to certain
bucket related URLs. And when such URL is requested ensure that UUID
in the URL matches the UUID of the bucket.

Change-Id: I439615243c0731c6afe024a037f4ea83fc398b8f
MB-5307 Add UUIDs to already existent buckets.

Change-Id: I48a8dbef4bd3e83d755e30038427dd9a7b7e91fa
MB-5307 Check bucket UUID when handling per-vbucket replications.

This is needed to prevent races when remote replication manager has
not yet retrieved updated vbucket map and tries to replicate (say) to
the bucket with the same name on a different cluster.

Bucket UUID is encoded in the URL as in the following example:

  http://user:pwd@lh:8092/default%2f0%3bfe919bda0242eac3ddf9e47586c3e67b

Change-Id: Id69db4ed461dfc76a606ec985b43dcefdc583b6f
MB-5307 Include cluster and bucket UUIDs in more advertised URLs.

This ensures that any information retrieved via such an URL is an
information relevant to the same instance of the cluster and bucket as
URL. And not, for instance, an information from the other cluster that
was configured on the same address or recreated bucket with the same
name.

Change-Id: I37266fd685c2b750b73e8d74826ee1e07a441f42
MB-5307 Export menelaus_web_remote_clusters:cas_remote_clusters.

Will be used later.

Change-Id: Id9f5006f2aa1be48fde1f0b936a62319def3434c
CBD-378 A service to access remote clusters information.

Provides several functions to fetch information about remote clusters
and remote clusters' buckets. Whenever this information is inquired
it's cached locally. So that next queries could be served from this
cache. In case the client finds out the provided information to be
inconsistent it can force remote_cluster_info to go to the remote
cluster to get up-to-date data.

Change-Id: Ie3e27c992fd065151abcb977cfa20dff5f8460b5
CBD-378 Update remote cluster reference creation code.

Now it uses newly introduced remote_clusters_info service to fetch the
information about remote cluster right away. Thus more errors might be
detected early. Additionally UUID of remote cluster is added to the
remote cluster config so that replication will be performed only into
correct cluster identified by that UUID.

Change-Id: I155c9174f46dce7f27e2f5e8eed4dbb1f549ab0c
CBD-378 Update replication creation code.

Most of the code removed since similar functionality's now available
through remote_clusters_info. Structure of replication document has
been changed slightly. "target" is just an abstract reference to
remote cluster's bucket instead of url
(e.g. /remoteCluster/remote/buckets/default). "targetUUID" field
containing UUID of remote cluster added.

Change-Id: Ic86f009e77afc4c7d840e039ae23e62c6543de0e
CBD-378 Basic integration of XDCR code and remote_clusters_info.

I just made it work in the most straightforward way. Currently XDCR
code doesn't use remote_cluster_info ability of caching vbucket maps
and just always fetches vbucket map from remote cluster. Additionally,
vbucket map can contain 'undefined' values which are not handled by
XDCR code.

Change-Id: Ie66f61925810d1e058d95d689c0571561671dabf
MB-5307: implemented misc:is_binary_ends_with

Change-Id: I270d84d0d72219bb0395a3f4b97330579f15cfdb
MB-5307: made ebucketmigrator_srv:had_backfill reply after ack

Previous implementation was incorrectly assuming that seeing
backfill-ness evidence message is enough. But at least of initial
stream message, which is positive evidence of backfill, it's possible
to have result of had_backfill before resetVbucket on destination is
really done. Which we don't want as we want 'stable' checkpoint ids in
destination after had_backfill returns.

Change-Id: Ibeb496b592f8668ff28c22d167dc3237572fbb15
WIP: ensure persistence of items during rebalance

We make sure new master and replicas has at least current
persisted_checkpoint_id + 1 of master (+1 because some items of
persisted_checkpoint_id + 1 may have been saved).

Change-Id: Ifd53c4f14af4970183366c0c66474216940296d0