MB-6918 Remove memcached dbdir from default config.

It's not needed in 2.0. And with recent change that synchronizes it to
database_dir and view_index_dir it may be confused with genuine dbdir
left from the older version of couchbase server. Particularly, if some
node is removed from the cluster its config gets cleared and dbdir
will again become part of memcached config. It means if this node had
a custom data path it will be overwritten on the next restart.

Change-Id: I158309323506863790044285a9fb04cd2d3718da
Reviewed-on: http://review.couchbase.org/21696
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
MB-6936: log db and index path changes

Change-Id: I08c89df47c76d9cd33167eea90946ab88115f304
Reviewed-on: http://review.couchbase.org/21698
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Farshid Ghods <farshid@couchbase.com>
Tested-by: Farshid Ghods <farshid@couchbase.com>
MB-5307: un-mark 2.0 prerelease in config

It was intended to mark specific pre-release version for costumer C.

Change-Id: I395259fcd7714d2b5471516188e364fc0b5ca8cd
Reviewed-on: http://review.couchbase.org/21699
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-6937: remove spamful log messages from smart clients activity

Because we anticipate much more smart clients activity than in 1.6 and
1.7 period, and because we don't really need those messages, we're
removing them.

Change-Id: Ief6c0a039e3ac7b1e228e967c08c5a7a38444141
Reviewed-on: http://review.couchbase.org/21700
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
MB-6951: don't log subscribed/unsubscribed messages in ns_pubsub

Change-Id: I2845aa57674d07e002120b836a77322c8762c575
Reviewed-on: http://review.couchbase.org/21732
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
MB-6494: handle empty ready vbuckets set in ebucketmigrator

We cannot send empty vbucket filter to ep-engine. If we do, ep-engine
will send all vbuckets, instead of none.

Thus when all vbuckets are not ready to be replicated from, we have to
nothing. We cannot die, because it will cause crashes above.

We had such behavior previously, but apprently recent code for
handling new-style vbucket filter change or something else removed it.

Note, there's also effect on vbucket filter change. When we hit this
condition, we force old-style vbucket filter change. That's because
there's no existing tap producer with our tap name.

Because no tap stream is going to be used while we do nothing (waiting
restart message), we kill tap stream with our name (if any).

Change-Id: Ic868a059fddf5055290b783882031453e3fb2ceb
Reviewed-on: http://review.couchbase.org/21733
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
MB-6953: anticipate exit from parent while waiting index update

Doing blocking janitor agent call for waiting index update implies we
were insensitive to shutdown signal from parent. Which means if
indexing is too slow or deadlocked somewhere, we cannot really stop
rebalance.

We mitigate it by using tried-and-tested spawn_and_wait function of
single vbucket mover ther performs work on new process while waiting
for both this process and exit from parent.

Change-Id: Ic5ed1b7f30b624473ce1c300327782b937bec2a9
Reviewed-on: http://review.couchbase.org/21751
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Reviewed-by: Steve Yen <steve.yen@gmail.com>
MB-6955: only react on our bucket mccouch events

Quite embarrassingly we were reacting on any bucket's mccouch
event. Refresh vbuckets event is harmless, but vbucket deletion will
obviously cause _other_ bucket's capi_set_view_manager to delete this
vbucket from it's indexes. Lame indeed :)

Fix is trivial.

Change-Id: I3f3a8be0ce9233b4268df97041a805c450f76eee
Reviewed-on: http://review.couchbase.org/21752
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Reviewed-by: Steve Yen <steve.yen@gmail.com>
MB-6919: expose more XDCR stats to UI

This change-set provides more XDCR stats on UI to help QE, performance
team and customers understand the XDCR operations better. We add 3 new
outbound operation stats, namely, 1) data replicated, which is amount
of data XDCR have replicated to the remote data center through inter-
cluster network; 2) active vb replications, which shows the number of
ongoing active replications to the remote clusters; 3) waiting vb
replications, which is the number of vb replications in the waiting
pool to wait for turn to start replication. These stats will be useful
to performance monitoring and diagnosis.

Change-Id: If9d42cde5dd715404f0b1bcf26ea143493ae9f75
Reviewed-on: http://review.couchbase.org/21613
Reviewed-by: Junyi Xie <junyi.couchbase@gmail.com>
Tested-by: Junyi Xie <junyi.couchbase@gmail.com>
MB-6638: disable async threads

Our empirical evidence so far suggests that those weird crashes are
caused by async threads which is somewhat experimental erlang VM
feature.

Thus we turn it off. We compensate by generously bumping up number of
scheduler threads.

Change-Id: I3ff3da7ec8190b719df8e7a06e40bdf2154f360d
Reviewed-on: http://review.couchbase.org/21884
Reviewed-by: Steve Yen <steve.yen@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
MB-6934: separate timestamp from error msg within XDCR

Today we include timestamp as part of error msg grabbed by ns_server.
For better msg filtering based on time stamps, it is better to return
them separately as {Timestamp, Error}, where Timestamp is the raw
time returned by erlang:localtime(). Another commit is needed to sort
and filter error msgs on UI.

Change-Id: Ic80121d53782379d746e6104b985b14c1a456e6d
Reviewed-on: http://review.couchbase.org/21694
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Junyi Xie <junyi.couchbase@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Steve Yen <steve.yen@gmail.com>
MB-6944: clean statistics if replicator terminated

When vb replicator terminated, we need to clear the statistics
num_change_left, docs_changes_queue and size_changes_queue, and
update them to the bucket replicator since the changes queue for
this particular vb replicator no longer exists. The vb replicator
supervisor will start a new vb replicator and populate these
stats to bucket replicator.

Change-Id: I4eb5796a43f6cf89b40e9ce80cf96c73f403401f
Reviewed-on: http://review.couchbase.org/21794
Reviewed-by: Damien Katz <damien@couchbase.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Steve Yen <steve.yen@gmail.com>
MB-6939: increase XDCR checkpoint interval to 30 min

XDCR will use priority checkpoint intead of regular checkpoints. To
address the overhead concern, we increase the checkpoint interval from
5 min to 30 min.

Change-Id: I128a2c826e3823084912bb6049a6390e2aec25d6
Reviewed-on: http://review.couchbase.org/21799
Reviewed-by: Damien Katz <damien@couchbase.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Steve Yen <steve.yen@gmail.com>
MB-6925 Don't try to cleanup old indexes in vbucket databases.

It's meaningless because there won't be any indexes.

Change-Id: I1fde741456f932a37244b7573b49f96f3af1d22d
Reviewed-on: http://review.couchbase.org/21885
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Steve Yen <steve.yen@gmail.com>
MB-6993 Log PID even for named processes.

Change-Id: Ie06994641093cf218cb95e6337b1a60a1fe8d24c
Reviewed-on: http://review.couchbase.org/21898
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
MB-6939: use priority checkpointing in ensure_full_commit

Today we use regular ep_engine checkpoint to persist XDCR checkpoint
which in most cases failed because the disk drain queue build up over
time. The failed XDCR checkpoints caused XDCR to pay restart time per
vb replicator and XDCR may have to resume replication from the last
successful checkpoint which was persited long time back. This change-
set uses priority ep_engine checkpoint which is less reliant on the
disk write queue to improve the chance of successful XDCR checkpoint.

Change-Id: I929274533a0586f6e6ca584593bb5da7228c4f56
Reviewed-on: http://review.couchbase.org/21730
Tested-by: Junyi Xie <junyi.couchbase@gmail.com>
Reviewed-by: Junyi Xie <junyi.couchbase@gmail.com>
Reviewed-by: Damien Katz <damien@couchbase.com>
MB-6934: show xdcr errors in less panic color

Part of 6934 improvements suggested by Dipti is to remove red color
from last 10 errors link.

Change-Id: Ic93e1fabb7e80218bb80aeabf04378c199af3bbf
Reviewed-on: http://review.couchbase.org/21903
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
MB-6934: omit too old xdcr errors

XDCR keeps errors forever. Even if errors happened last year. We want
to omit too old errors. By default, stale error is error that happened
more than 2 hours ago.

Change-Id: I06223fa328a9fee201ed9336bbf685f6b0f16d71
Reviewed-on: http://review.couchbase.org/21904
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
MB-6974: set reasonable ERL_FULLSWEEP_AFTER

Erlang's default leads to massive memory blowup. Our perf developers
found that 512 is a reasonable value.

Change-Id: I50822ecb752d6667af3a6f8b9bac283e3d5aef3a
Reviewed-on: http://review.couchbase.org/21902
Reviewed-by: Steve Yen <steve.yen@gmail.com>
Reviewed-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Typo - COUCHBASE_NUM_VBUCKET(S) to set vbucket cnt

Instead of COUCHBASE_NUM_VBUCKET

Change-Id: I1b7f1eca03924089a02ac631787532148793017d
Reviewed-on: http://review.couchbase.org/21813
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Steve Yen <steve.yen@gmail.com>