Separate barriers for main and active index updates

Replica index updates can prevent main index updates from
running sooner, causing clients to wait for a very long
time to get responses to view queries. Therefore use a
separate, and smaller, pool for replica index updaters.

This relates to MB-4848.

Change-Id: If356994fc35ac24e5b09199b18304244ca7af777
Reviewed-on: http://review.couchbase.org/13704
Reviewed-by: Damien Katz <damien@couchbase.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
Reviewed-on: http://review.couchbase.org/13741
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Better updater's doc loader error reporting

Change-Id: I38ec2d733796eb599e779dbf10151e208cef8be2
Reviewed-on: http://review.couchbase.org/13720
Reviewed-by: Damien Katz <damien@couchbase.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
Reviewed-on: http://review.couchbase.org/13742
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Avoid unnecessary doc encoding

This speeds up map queries with ?include_docs=true

Change-Id: I395df41bd92de5b3b9ff826ef7ba84508547c33a
Reviewed-on: http://review.couchbase.org/13862
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Damien Katz <damien@couchbase.com>
CouchDB changes towards fixing xdcr

Change-Id: Ie355e8f67f48310995ff52c00ea06bfe25cfda89
Reviewed-on: http://review.couchbase.org/12291
Reviewed-by: Damien Katz <damien@couchbase.com>
Tested-by: Srinivas Vadlamani <srinivas@couchbase.com>
Tested-by: buildbot <build@couchbase.com>
Modify replicator to support xdcr checkpointing

The following changes have been made:
1. We're going to bypass ep-engine for opening and updating checkpoint
   documents. For this reason, we will store the documents in the
   master database instead of the individual vbucket databases.
2. The document ids will be prepended with the vbucke id to avoid
   potential conflicts since they're all going to be stored in the
   master database.

The replicator now generates local and remote URIs for the checkpoint
docs per the above.

Also increased the checkpointing interval from 5s to 60s.

Change-Id: Id62a99d4bf129ebc2275776fd53d760c50f00b2b
Reviewed-on: http://review.couchbase.org/13774
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Damien Katz <damien@couchbase.com>
Add new option --disable-spidermonkey

This option allows us to build Couchbase 2.0 on platforms where
the ancient version of spidermonkey we support doesn't build.
The CouchDB version will of course not function as expected in some
areas, but it allows us to store/retrieve values from the memcached
interface used in Couchbase 2.0.

Change-Id: I982166c99bffe53a6b81d150a290eab4b1245976
Reviewed-on: http://review.couchbase.org/13945
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: buildbot <build@couchbase.com>
implemented better diagnostics for writer_loop fd reopen problems

Instead of badarg we'll hopefully have much better diagnostics now.

Change-Id: I852b57c386bd15120dd6c959dcf9f40c6e09fe48
Reviewed-on: http://review.couchbase.org/13974
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Reviewed-by: Damien Katz <damien@couchbase.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
message about switching files in compaction deserves LOG_INFO

Change-Id: Iabd05c3ce715f8b47ed44b375946048d393f759a
Reviewed-on: http://review.couchbase.org/13975
Reviewed-by: Damien Katz <damien@couchbase.com>
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
don't die enabling snapshot reads in terminate

Exception from terminate will replace exit reason of gen_server. So
we'll lose important piece of information if that happens.

So in order to keep original reason we should just log and swallow
problems in terminate.

Change-Id: I7e8071cd795d97fe5401f837f967a1e20cd7c1fd
Reviewed-on: http://review.couchbase.org/13973
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Merge branch 'branch-20dp4'

* branch-20dp4:
  don't die enabling snapshot reads in terminate
  message about switching files in compaction deserves LOG_INFO
  implemented better diagnostics for writer_loop fd reopen problems
  Better updater's doc loader error reporting
  Separate barriers for main and active index updates

Change-Id: Ice852f3069ec838507b6f70491dcf1608834b69b
Fix for occasional eacces error on windows.

For reasons unknown, opening a file on windows sometimes returns
{error, eacces}. Fix by retrying open for a maximum of 10 seconds.
Max retry time now also applies to emfile errors.

Change-Id: I915416a2512def52ecbc615c058deff7e630922f
Reviewed-on: http://review.couchbase.org/14085
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: Damien Katz <damien@couchbase.com>
Reviewed-by: Damien Katz <damien@couchbase.com>
Merge branch 'branch-20dp4'

* branch-20dp4:
  Fix for occasional eacces error on windows.

Change-Id: Ic179c23ae249a09cce7bd40aefbf0419c2cc40bb
Set index fd to read mode only on group shutdown

It's possible that the group shutdowns while there are still
clients streaming the view (keeping the fd ref counter > 0) and
that when it's open again, these clients are still folding the
index - this prevents the view group to open the file again.
Closes MB-4869.

Change-Id: I726d11c9642b9c5581e6d3e5acca0aa2f37a3e8b
Reviewed-on: http://review.couchbase.org/13992
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Damien Katz <damien@couchbase.com>
Ensure right cleanup bitmask when updater is stopped

Issue MB-4900.

Change-Id: I41caca9cef145a676fa6747e347977c5fd1d547c
Reviewed-on: http://review.couchbase.org/13991
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Damien Katz <damien@couchbase.com>
MB-4523 - CURL/ICU_CONFIG overridability during configure

Originally from Xinchen Hui.

Change-Id: If9ff42bbdd9c57f3aab5971b1c811f6455978ebf
Reviewed-on: http://review.couchbase.org/14172
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Tested-by: buildbot <build@couchbase.com>
Don't delete index file on POSIX error emfile

If we can't open the file because too many file descriptors
are open, just log an explicit message error and don't delete
the index file.

Change-Id: I0021fd83c67e854705da9842fc7ca40d90d8acd4
Reviewed-on: http://review.couchbase.org/14352
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Damien Katz <damien@couchbase.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
Account for whitespaces in couch_doc:to_raw_json_binary/2

Change-Id: I32bcf3aaf13b5188afb39dd8cd191f68d01ac603
Reviewed-on: http://review.couchbase.org/14359
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Damien Katz <damien@couchbase.com>
Add MapReduce subsystem implemented as an Erlang NIF

It uses Google's V8 JavaScript engine for executing the
map and reduce functions.

Change-Id: I982c2f37ddf499daaa690e76e7435f3e852ab62a
Reviewed-on: http://review.couchbase.org/13831
Reviewed-by: Damien Katz <damien@couchbase.com>
Tested-by: Damien Katz <damien@couchbase.com>
Make set view component use the V8 based MapReduce subsystem

Change-Id: I90b26eeb78f50a6bac4f71e7ff90cbe0e92b8f42
Reviewed-on: http://review.couchbase.org/13857
Reviewed-by: Damien Katz <damien@couchbase.com>
Tested-by: Damien Katz <damien@couchbase.com>
Add preemption to MapReduce component

This ensures that functions that take too long, or just go
into an infinite loop, are stopped after some defined elapsed
period of time (defaults to 5 seconds). If their execution
is not stopped, an Erlang scheduler thread is kept executing
that bad JavaScript function forever (or too long).

Change-Id: I62340ab912561303cddf75855e23b6d32096bf47
Reviewed-on: http://review.couchbase.org/14028
Reviewed-by: Damien Katz <damien@couchbase.com>
Tested-by: Damien Katz <damien@couchbase.com>