Add bulk vnode start function to allow parallelism
Merge pull request #511 from basho/eas-add-kv-vnode-parallel-init

riak_kv 1.3.1 using parallel vnode init
support for handling legacy sext encoding of 2i keys

This commit is a squashed version of the branch:
jrw-eas-2i-fix-keys-backport.

The version of sext used by riak 1.3 and prior did not
preserve the sort order of bigints when serializing erlang
terms resulting in 2i queries that returned missing or
invalid results (riak_kv#499). The updated version of sext fixes the issue
with an incompatible change to the encoding of bigints. Special
handling of this incompatability is necessary to provide a path
for indexes to be fixed (as they are written to disk).

A utility function for reformatting the on-disk indexes (per-node)
is provided. This allows indexes to be upgraded on a running node.
The utility folds over all indexes, per partition, and rewrites any
invalid index (those containing bigints encoded using the old version
of sext). In order to safely migrate indexes, until the fold that
rewrites the indexes in the new format completes, all riak_object
puts/deletes must delete the legacy index format in addition to the new format

Upon successful completion or if the partition is empty a flag is
written to the leveldb backend to ensure that future writes/deletes
to not inccur the additional cost of deleting the legacy index format

add support for downgrading back to legacy index format

Fix eleveldb fold_objects to start at 1st object

Also fixed special bucket option so it starts at the beginning of the
given bucket. But that doesn't appear to be used anywhere...

Add key namespace for special leveldb markers

My argument is that this may make it less messy down the road if others
start adding other special markers in leveldb partitions. They now have
a separate namespace, like objects and index keys do.

add console support for reformatting indices

Fix 2i bigint eq query during transition

This causes a double scan while in the transition from the legacy
(broken for big int) sext encoding for 2i keys. Once starting at the old
key value, later at the new one. Once the upgrade or
downgrade is finished, it should return to a normal scan.

add support for multi-backend index reformatting

the multibackend supports indexing on buckets assigned to sub-backends
that support indexing. LevelDB sub-backends must be reformatted.
Merge pull request #516 from basho/jrw-eas-2i-fix-keys-backport-squashed

support for handling legacy sext encoding of 2i keys
Change AAE to use incremental crypto:sha calculations

The crypto library is implemented as a NIF, and long running NIFs can
negatively impact the work balancing algorithm of the Erlang
scheduler. Using normal crypto:sha(object) is unbounded in duration as
the size of an object is unbounded. This commit therefore changes AAE
to use an incremental SHA calculation (sha_init/sha_update/sha_final)
with a set chunk size (defaults to 4096; configurable app var
riak_kv/anti_entropy_sha_chunk).
Merge pull request #514 from basho/jdb-aae-incremental-sha-1.3
Re-use the stat calc funs in riak_core

Use the cached riak_pipe stats for legacy stats

When a broken stat is detected, register it.

Fix bug where stats endpoints were calculating _all_ riak_kv stats

Since adding many more stats, and most of the infrastructure for
ad hoc querying of stats, the stat calculation code for the
(not yet legacy) endpoints was calculating all stats for riak_kv.

As there are about (ring_size * vnode stats) + (fsm stages * fsm stats)
more stats now, this calculation, understandbly, took a long time.

This patch instead only calculates the minimum subset of stats
needed to support the (not yet legacy) stats endpoints.
Since stats now get repaired when an update fails, log as `warning`
Merge pull request #517 from basho/kv508-stats-warn

Since stats now get repaired when an update fails, log as `warning`
Roll riak_kv version 1.3.1
Merge branch '1.3'

Conflicts:
	rebar.config
	src/riak_kv_stat.erl
	src/riak_kv_stat_bc.erl
	src/riak_kv_vnode.erl
spell badarg correctly

Spelled incorrectly, the catch clause fails to prevent the process from
blowing up on error. Spelled correctly, the user receives an error
message describing what the problem is.
Merge pull request #522 from branch 'bwf-badard' into 1.3
Fix perf problems and bug in 2i reformat

Add batch size parameter to control how many keys
are fixed at a time.
Add backpressure to query of bad idx entries to avoid
having the reformat process flooded with msgs.
Fixed problem with list keys and code triggering a second
scan during 2i reformat transition.
Make multi backend work with batched 2i reformats

This is WIP. Testing is not conclusive on whether it works, but it runs
to completion now.
Fix 2i reformat batch for multibackend
Make reformat-indexes print default parameters
move querying of fixed index status to seperate backend function

makes top-level fixed_indexes status on multi-backend,
which messes with stats, unncessary allowing it to be removed
Merge pull request #525 from basho/jrw-2i-reformat-status-change

move querying of fixed index status to seperate backend function
Merge pull request #523 from basho/eas-perf-fixes-to-2i-reformat

Fix perf problems and bug in 2i reformat