MB-9467 On JSON.stringify failure, avoid throwing exception with null value

While not a problem, the logged error reason ended up being a "null",
not very informative. Was easy to get via for example:

function(doc, meta) {
    while (true) {
        emit(doc, doc);
    }
}

Change-Id: Ic3757884398c6a87f7530d351447b94aac2aa7b3
Reviewed-on: http://review.couchbase.org/30052
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
MB-9467 Don't process documents with too large bodies

Too large documents slow down indexing and make the JavaScript engine
use too much cpu time (JSON parsing/validation), causing the whole
system to be inneficient during rebalance and failover operations or
not to be able to accomplish such operations at all.

Therefore limit the maximum size of documents passed to the JavaScript
engine, configurable via the internal parameter "indexer_max_doc_size".

Change-Id: I8e130d4c419c63f495452dab10f3e5ea96dd0db1
Reviewed-on: http://review.couchbase.org/30053
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
MB-9467 Raise explicit error if list of emitted keys exceeds limit

If more than 64k keys are emitted for a single document, raise an
explicit error informing about this fact, instead of having a badmatch
later when updating the key list for a document.

Change-Id: I61f5aa040ce026777aac52217a696f220da3d3ae
Reviewed-on: http://review.couchbase.org/30054
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
MB-9467 Enforce max total KV size that can be emitted per doc/view

To avoid consuming too much memory while running a single map function,
enforce a maximum limit for the total amount of data that can be emitted
per document and per view. This limit is configurable via the couch_config
option "max_kv_size_per_doc", section "mapreduce", and corresponds to the
sum of the byte size of all emitted keys and values.

A value of 0 for this new setting, disables the limit (meaning unlimited,
as before this change).

So far, before any memory blowup could happen, the function execution was
aborted due to the trigger of the timeout, such as in the following example
case:

function(doc, meta) {
    for (var i = 0; i < 1000000; i++) {
        for (var j = 0; j < 1000000; j++) {
            emit(doc.value, null);
        }
    }
}

However, one user was able to get V8 heap exhaustion while running a map
function. Despite unsuccessful attempts so far to reproduce the heap
exhaustion issue, this change allows to enforce sane limits.

Change-Id: I4af03208027868a461c568dcd20ff49bec27ed76
Reviewed-on: http://review.couchbase.org/30055
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
Merge remote-tracking branch 'origin/2.1.1'

* origin/2.1.1:
  MB-9467 Enforce max total KV size that can be emitted per doc/view
  MB-9467 Raise explicit error if list of emitted keys exceeds limit
  MB-9467 Don't process documents with too large bodies
  MB-9467 On JSON.stringify failure, avoid throwing exception with null value

Conflicts:
	etc/couchdb/default.ini.tpl.in
	src/mapreduce/mapreduce.cc

Change-Id: I22d8e6435079c4b1e759ee4048e5ef3c7f5c94ba
Fix test couch_set_view/test/12-errors.t after merge

Master branch has different view group APIs, so the test needs
to use the master APIs instead of those in the 2.1.1 branch.

Change-Id: I23fb381e6a7ed59c9ac9bbd23298463b07931518
Reviewed-on: http://review.couchbase.org/30137
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: buildbot <build@couchbase.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
MB-8454 Improve indexing progress stats update

Instead of updating indexing progress stats (`http://localhost:5984/_active_tasks`)
after each batch write to btrees, update the stats simultanously while items
are being loaded to Mapper Queue. Hence, it will produce an incremental
progress rather than abruptly moving by batch progress.

Change-Id: Ib82260aaa17819efb9a5b51f75a93ff40df8f6b0
Reviewed-on: http://review.couchbase.org/30201
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
MB-8454 Fix initial index build progress stats

During initial index build, calculation of total number of changes is
inaccurate since it is assumed to be equal to the sum of current seqs
for vbuckets. This results in displaying progress stats having total_changes
much more than the actual number of changes to be processed.

For initial build case, total changes should be estimated by summation
of non-deleted docs count from each of the vbuckets belonging to a bucket.

Change-Id: Iea4e2a6ebcd614d973ac5551e46c565e3be46f7c
Reviewed-on: http://review.couchbase.org/30202
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
CBD-485 jump_to_another_version to adjust the vbucket

version in couchdb after the ep-engine compacted it

Change-Id: Id1c1e375d142788e473b32e618feb5ae97963f15
Reviewed-on: http://review.couchbase.org/30326
Tested-by: Artem Stemkovski <artem@couchbase.com>
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
MB-8903: Add UPR project

In couch_upr will be all the code that is needed to get
changes from an UPR stream.

Change-Id: I1c71c64819e4369a38b32a20128e9d21fd7bd022
Reviewed-on: http://review.couchbase.org/29401
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
MB-8903: First version of couch_upr

See test/02-couch-upr.t for the API usage.

This commit also adds the fake UPR server.

Change-Id: I670f8e36196057399b4f942c7ffebfbdb9295066
Reviewed-on: http://review.couchbase.org/29402
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
MB-9058: Consume from fake UPR server

Instead of reading the changes from files, get them
via TCP from the fake UPR server.

Change-Id: If69433c4863de9a7ec2684da1c72d070c5406205
Reviewed-on: http://review.couchbase.org/29403
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
MB-9085: Don't hard-code UPR port

Instead of hard-coding the UPR port, use a configuration
setting for it. This makes it also possible to start the
fake UPR server on a random port (assigned by the
operating system), when the value is set to "0".

Hence it's now possible to run the set_view_tests in
parallel again.

Change-Id: I6c319dbb6fcac8b4f20f38ce8a44c6abed24658d
Reviewed-on: http://review.couchbase.org/29404
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
MB-9058: Remove all references to the database files

Access to the database files is no longer needed as
the information now gets in through the UPR stream.

A check for purging was removed. That is OK since with
UPR the logic to deal with purges needs to be re-done
anyway.

Change-Id: I3b751905867cda4453ebd98cbf48bd8144fbae0b
Reviewed-on: http://review.couchbase.org/29405
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
MB-8903: Update according to transfer protocol changes

The transfer protocol changed quite heavily. This commits
makes the corresponding changes.

Change-Id: I860b072c96258f3045a5fcc94b34eaa91f65f4f0
Reviewed-on: http://review.couchbase.org/29406
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
MB-8903: Switch defines from bytes to bits

Change-Id: Ic136a651c4e0f2563084b637693c9a25cd082cc6
Reviewed-on: http://review.couchbase.org/29407
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
MB-8903: Simplify header parsing

Change-Id: I9e24502b8aea7a589f1edbcc19c063cf9daa605c
Reviewed-on: http://review.couchbase.org/29408
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
Fix dialyzer warnings

Change-Id: I440acf09dc913f3b56b68f7ce3ce4077fda07da2
Reviewed-on: http://review.couchbase.org/29409
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
MB-9307: Add rollback to couch_upr

Add the rollback response to the couch_upr_fake_server when
the sequence number is too high. Also handle that case in
couch_upr.

Change-Id: Idf09828f29e6edc766d004a1f62f98f789d29f48
Reviewed-on: http://review.couchbase.org/29416
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
MB-8903: Update according to transfer protocol changes

Change-Id: I2ff866f228c72f6e7aa2763a6c1495a294afed88
Reviewed-on: http://review.couchbase.org/29734
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>