MB-9919: expose DbInfo.file_size via couchstore_db_info

Change-Id: Iae8c76b7bf6b1a305be76b401f383a47766241e5
Reviewed-on: http://review.couchbase.org/32468
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
MB-9661: API to count changes between two sequence numbers

Change-Id: Icf62266bf29b5981d1e9dd6d33aa784a5a90d04f
Reviewed-on: http://review.couchbase.org/32412
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
MB-7629 Add native viewgroup compactor

Implementation of native view index compactor.
The native program receives viewgroup header info through stdin
and it creates a new index file and recreates all view btrees
and id btree. The resulting index header is written out to stdout.

Change-Id: I0c9c9ff33d3e005f22cc848c85d708b158d806e4
Reviewed-on: http://review.couchbase.org/31666
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
MB-7629 Add filter support for view compactor

A filter bitmask can be used to optionally ignore kvs
from certain vbuckets while rewriting the btrees to
the target index file during compaction.

Change-Id: I1e37834f062de10da29208a88619c5e3580f4de6
Reviewed-on: http://review.couchbase.org/31776
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
MB-7629 Add option to skip sorting in native btree updater

For compactor delta logs update, we do not want to
sort the log files since they are already sorted.

Change-Id: I298db564db2ebba677f0aa58924d97461348a0d5
MB-7629 Implement view compactor incremental progress stats

When view compactor makes progress, a stats callback
is used to update the current progress of inserted kvs.
For every percentage progress, the compactor program sends
inserted count stats through stdout.

Change-Id: I550a8ed077e6dda78753f4a944d0bff36a624540
Replace few string allocs and copy with strdup

Change-Id: I33cba00877556171693989e07af23ace7c0eba90
Add exit message handler for native index builder

Change-Id: If60d7eeef715d72caa89ccfbd3caac0daf28af59
Add kvs sorting step in initial index builder

Change-Id: I7e79d7cd13fb53cb17db4f5fad83f6c1d5963aa0
Add feed callback and skip_writeback option in file sorter

For the file sorter, we write out intermediate sorted files
to disk and perform a k-way merge to produce final sorted
records file. The final records file is consumed by index
builder to build btree indexes. But, ideally we could skip
writing the final result of k-way merge to disk if we could
pipeline the results coming out of k-way merge directly into
index builder. A callback mechanism has been added to the file
sorter/merger to call a feed_record method during the final
phase of file merger. Also added a flag to skip writing final
sorted results to a file.

The consumer of file sorter can save a lot of disk IO
by avoiding unnecessary write-read of final sort results.

Change-Id: If9b15092bdc7242772981859344f4f88dec45998
Add tests for file sorter callback and skip_writeback option

Change-Id: I87246cf28fdb3b524d219d7f92c82ffb425c4326
Use file sorter callback to build initial index

Initial btree index was built by reading the records
from a sorted records file. File sorter now supports
a callback which gets invoked as soon as final sorted
records are available from final sort-merge phase.

Initial index builder is refactored to build the btree
by avoiding final write phase of sorter and it builds
up through the sorted result stream.

Change-Id: I67f46ee71f5c9afed6ca321c1da8dd265c82ce42