Remove merge_index dependency and unused erl_first_file.
Add bulk vnode start function to allow parallelism
Merge pull request #511 from basho/eas-add-kv-vnode-parallel-init

riak_kv 1.3.1 using parallel vnode init
support for handling legacy sext encoding of 2i keys

This commit is a squashed version of the branch:
jrw-eas-2i-fix-keys-backport.

The version of sext used by riak 1.3 and prior did not
preserve the sort order of bigints when serializing erlang
terms resulting in 2i queries that returned missing or
invalid results (riak_kv#499). The updated version of sext fixes the issue
with an incompatible change to the encoding of bigints. Special
handling of this incompatability is necessary to provide a path
for indexes to be fixed (as they are written to disk).

A utility function for reformatting the on-disk indexes (per-node)
is provided. This allows indexes to be upgraded on a running node.
The utility folds over all indexes, per partition, and rewrites any
invalid index (those containing bigints encoded using the old version
of sext). In order to safely migrate indexes, until the fold that
rewrites the indexes in the new format completes, all riak_object
puts/deletes must delete the legacy index format in addition to the new format

Upon successful completion or if the partition is empty a flag is
written to the leveldb backend to ensure that future writes/deletes
to not inccur the additional cost of deleting the legacy index format

add support for downgrading back to legacy index format

Fix eleveldb fold_objects to start at 1st object

Also fixed special bucket option so it starts at the beginning of the
given bucket. But that doesn't appear to be used anywhere...

Add key namespace for special leveldb markers

My argument is that this may make it less messy down the road if others
start adding other special markers in leveldb partitions. They now have
a separate namespace, like objects and index keys do.

add console support for reformatting indices

Fix 2i bigint eq query during transition

This causes a double scan while in the transition from the legacy
(broken for big int) sext encoding for 2i keys. Once starting at the old
key value, later at the new one. Once the upgrade or
downgrade is finished, it should return to a normal scan.

add support for multi-backend index reformatting

the multibackend supports indexing on buckets assigned to sub-backends
that support indexing. LevelDB sub-backends must be reformatted.
Merge pull request #516 from basho/jrw-eas-2i-fix-keys-backport-squashed

support for handling legacy sext encoding of 2i keys
Change AAE to use incremental crypto:sha calculations

The crypto library is implemented as a NIF, and long running NIFs can
negatively impact the work balancing algorithm of the Erlang
scheduler. Using normal crypto:sha(object) is unbounded in duration as
the size of an object is unbounded. This commit therefore changes AAE
to use an incremental SHA calculation (sha_init/sha_update/sha_final)
with a set chunk size (defaults to 4096; configurable app var
riak_kv/anti_entropy_sha_chunk).
Merge pull request #514 from basho/jdb-aae-incremental-sha-1.3
Re-use the stat calc funs in riak_core

Use the cached riak_pipe stats for legacy stats

When a broken stat is detected, register it.

Fix bug where stats endpoints were calculating _all_ riak_kv stats

Since adding many more stats, and most of the infrastructure for
ad hoc querying of stats, the stat calculation code for the
(not yet legacy) endpoints was calculating all stats for riak_kv.

As there are about (ring_size * vnode stats) + (fsm stages * fsm stats)
more stats now, this calculation, understandbly, took a long time.

This patch instead only calculates the minimum subset of stats
needed to support the (not yet legacy) stats endpoints.
Since stats now get repaired when an update fails, log as `warning`
Merge pull request #517 from basho/kv508-stats-warn

Since stats now get repaired when an update fails, log as `warning`
Roll riak_kv version 1.3.1
Change to support the pinning of secondary indexes into memory within leveldb backend using ets sorted-set in a similar way to the existing memory backend
Tidy-up log entries and add TODOs
Add some explanatory notes as to what is expected to be achieved through the 2i changes, and how it will be achieved
Correct typographical errors in docs
Merge branch '1.3'

Conflicts:
	rebar.config
	src/riak_kv_stat.erl
	src/riak_kv_stat_bc.erl
	src/riak_kv_vnode.erl
Add clarification over use of bloom filters
Changes needed to expose FSM timeouts to the clients:
  - change delete in riak_client to honor timeouts passed
    in Options
  - change the various object interfaces to collect and
    pass on the timeout values
Added support for bloom filter guarding of exact term index queries
spell badarg correctly

Spelled incorrectly, the catch clause fails to prevent the process from
blowing up on error. Spelled correctly, the user receives an error
message describing what the problem is.