unbreak warmup completion detection

This was recently broken by ep-engine stats change.

Change-Id: I07c4e59767c755a59c6c8fdbd039946f49aadd5b
don't register/unregister ns_memcached to late/soon

There's seeminly no need anymore. And it allows us to check if bucket
is still running by looking at registered processes.

Change-Id: Iecef1bed6690e20cc14d3f8caed2dbdcd27bf6ab
actually delete buckets when node is failed over

Not just shutdown ns_memcached. Right now we shutdown ns_memcached,
delete bucket without force option but leave data files. Those files
are deleted when node is finally ejected, but if user does add-back
instead it leads to lengthy warmup of those data files. Essentially
not allowing rebalance without good reason.

Change-Id: Iba2a82e35471b496e04ec2c433a3868a4f886176
try to delete data files even if delete bucket failed

So that if memcached crashes/dies when we're force deleteting bucket
we still delete data files.

Change-Id: I2ab6d7212091351f4f04cc0d2b81ef4455176237
allow port_servers to send arbitrary data to port's stdin

Change-Id: I3de786db1d573fa8abf93d7dc73f1dc4291847fb
made supervisor_cushion exit with same reason as child

So that termination with reason normal is normal.

Change-Id: I74defba6ddb9f114671cfd448d7278b2402b10ef
ask memcached to die real quick on failover

We send die! message that we expect memcached to just do _exit. We'll
then restart it, but the idea is that per bucket supervisors are
already shutting down for this node.

Change-Id: If626710ecf1abe9cbaef3b9f343e2adbe0a97039
kill only memcached only on transition

Change-Id: I88a6433f5645bec125fc06ecb185867c2b34f96b
deactivate node now marks it as failed over

Change-Id: Ifeae193e775f6324dc9fa46b603be6ec6b913636
try not creating bucket if it's running on any cluster nodes

We'll do best effort check if ns_memcached for this bucket still
exists anywhere and refuse creating it.

Best-effort here means that we'll try to reach other nodes, but with
sane timeout and if we cannot we'll assume it's fine. It is consistent
with current behavior and don't want to change that.

It is intended to prevent issues when somebody tries to re-create
bucket too soon after it was deleted. And to protect us from any
issues in ep-engine/bucket-engine that may delay bucket deletion. See
for example http://www.couchbase.com/issues/browse/MB-4329

Change-Id: I8d66cb5dc9603b4da8ad9ca430a0cd112e2a422f
handle errors_500 from do_bucket_create in setup default call

Change-Id: I0f005254ce8aef10e1c7b501e4310ef2e565d21d
fixed filter_ready_buckets support of memcached buckets

Without this fix memcached buckets that are not yet "rebalanced" will
not be actually started on new nodes and that will cause periodic
crashes of hot keys keeper.

Change-Id: I3c366efc3e75d7ab2e9f2c4f1eef3318d14f8d4e
don't crash but return 400 when new bucket name is missing

Change-Id: I8ec90b8a00891183efb6772e4eb615de58abb3f2
reset node's cookie before joining cluster.MB-4476

Change-Id: I958731daad0f85deb28990b1b1a2040e3c65fc04
use past vbucket maps to optimize vbucket movements

New code picks closest of past but matching vbucket maps and newly
generated vbucket map.

Unit tests particularly ensure that failover/add-back case is handled
with minimal movements. Also simultaneous removal of N nodes and
addition of N other nodes performs minimal movements as well (transfer
of vbuckets between this nodes).

Change-Id: Iccaafb26e0d75cedfa161b741847a82bd02bdf90
log distance between current and future maps

Change-Id: If4f97b496ea07678373f81370f8f6beab240f957
use proc_lib:spawn_link to spawn single vbucket mover

Change-Id: I547724669cc5ed46c058d7a0e6d15df9db93ba62
log more janitor state when setting dead state

Change-Id: Ic82b033b050f5b350111f96e32034e8018316f72
skip moves for completely same new/old chains

Change-Id: I36ec7cc471dbbdeac92602dbbd8919aa238cb2de
make sure we're replicated latest config before leaving

ns_config_rep:synchronize is not enough because ns_config_rep may yet
to see latest config. We need to sync on config announcements,
queueing config replication, first. And then wait for all config
replications to be done.

Change-Id: Iac70fced38cc70c52667c257b6cd560519eb8fb6