Add active anti-entropy to Riak

Active anti-entropy is a process through which multiple replicas
periodically sync with each other and automatically repair any keys
that are missing or have divergent data. For example, a user could
delete all data from a node and Riak would automatically regenerate
the data from the other replicas.

This implementation uses hash trees / Merkle trees to perform
lightweight key exchange, with work proportional to the number of
divergent keys rather than the size of the overall keyspace.

This implementation meets several design goals:

1. The underlying hash trees are on-disk structures, and can scale to
   billions+ keys. This is in contrast to in-memory trees that require
   significant RAM to support massive keyspaces.

2. The underlying hash trees are persistent. Riak nodes can be
   restarted without fear of hash tree data being lost and needing to
   be rebuilt.

3. As new data is written to Riak, the hash trees associated with the
   various partitions are kept up-to-date. Each write in Riak triggers
   an asynchronous write to one or more hash trees. Combined with #2,
   this enables trees to be built once through a scan over existing
   data, and then maintained in real-time. In reality, trees are expired
   over time and rebuilt to ensure the hash tree and backend data stay
   in sync, and also to identify bit rot / disk failure.

4. The entire implementation is designed to be non-blocking. For
   example, a snapshot of a hash tree is generated before performing
   an exchange with other replicas, therefore allowing concurrent
   inserts of new key/hash pairs as new writes occur.

The current implementation triggers read repair for each key difference
identified through the hash exchange. This is a reasonable approach as
read repair is stable, production-tested mechanism in Riak. However,
the read repair approach leads to slower replica repair in cases where
there are a large number of key differences. This is an area for future
improvement.
Change several gen_server:calls to have infinity timeout
Remove AAE debug information stored in process dictionary
Make entropy manager handle remote locking, not remote vnode

Change the start_exchange_remote logic to route through the remote node
entropy manager rather than the remote vnode. Also, change the code to
rely upon chained casts rather than calls. Specifically, first ask the
remote entropy manager to acquire a concurrency lock. If successful, the
entropy manager than forwards the message to the relevant index hashtree
to acquire the tree lock. If successful, the index hashtree sends a reply
back to the originating exchange FSM.

Make entropy manager always run, even when AAE is disabled. This is
necessary to handle the new remote exchange process, as well as sets the
stage for runtime enable/disable of AAE.
Make AAE exchange FSM wait indefinitely for trees to update

The exchange FSM will now wait for both the local and remote hashtrees
to perform a tree update, rather than potentially timing out. Previously,
the exchange FSM would time out, while the updates would continue to run
asynchronously. However, since the exchange FSM is the process holding
both the concurrency and hashtree locks, future exchanges could be
triggered that initiate additional updates. If an update was started for
a tree already performing an update, the previous update would be killed
and you run the risk of a tree never successfully finishing an update.
Even if the update is for different trees, a potentially unbounded number
of concurrent updates could occur that are not limited by the concurrency
setting.

Also, changed the riak_kv_exchange_fsm:as_event to use spawn_link rather
than spawn to ensure call failures trigger crashes of the exchange FSM.
Change index_hashtree:do_compare to use spawn_link
Make the entropy manager not block to get hashtree PIDs

Instead of having the entropy manager use riak_kv_vnode:hashtree_pid,
which uses a synchronous vnode request, have the entropy manager use
the new riak_kv_vnode:request_hashtree_pid which sends an asynchronous
request to the vnode. The vnode will eventually reply by sending a
message back to the entropy manager which is handled by handle_info.
These changes ensure the entropy manager never blocks when there are
slow vnodes.
Ensure that index hashtree failures do not crash vnodes

Previously, riak_kv_vnode would create riak_kv_index_hashtree PIDs
using spawn_link, which would force the vnode to crash if the hashtree
ever crashed. Now, the vnode spawns and monitors the hashtree PID and
will recreate the tree if it ever dies. If creating a new tree fails,
the vnode will retry to create the tree periodically. This handles the
case where a lingering LevelDB lock has yet to be released between
when a previous tree exited and the new tree was created. The first
create may fail, but eventually the lock is released and a new tree
started.
Add build tokens to limit hashtree builds/rebuilds
Make hashtree rehashing not require a build token

After a node requests, already built hashtrees are not rebuilt but
instead rehashed. The new build token system limited both actions, but
tree rehashing is relatively cheap and something we should not block.
Otherwise, a node that restarts will take a long time before it can
initiate AAE exchanges.
Support enabling/disabling of AAE without node restart
Make AAE exchange check riak_kv service status

Exchanges will now only occur if both the local and remote nodes are
up and have the riak_kv service running. This prevents AAE from trying
to do exchanges with nodes that are offline, or performing exchanges
with a node that is up but not handling K/V requests and therefore
would not repair anything even if read-repair was triggered.
Make AAE data root configurable
Make preflist_siblings not include specified index
Rewrite determine_max_n to reuse determine_all_n
Various minor changes to AAE

-- Fix typo in riak_kv_entropy_manager:get_lock edoc.
-- Make AAE config use proplist functions for clarity of intent.
-- Rename riak_kv_entropy_manager:trigger_exchange to enqueue_exchange.
-- Change AAE config from on/off/manual to on/off, as manual already
   works as a config modifier similar to debug.
-- Fix -spec for riak_kv_entropy_manager:do_get_lock.
-- Add additional information to the "repaired X keys" info message.
Fix build_tokens spec
Fix hashtree_eqc
Increase AAE default timeouts/limits
Enable AAE by default in riak_kv.app.src