Change riak_core_ring_manager to use gen_server
Hybrid ring/mochiglobal approach

Numerous processes concurrently read and access the ring in a
variety of time sensitive code paths. To make this efficient,
riak_core uses mochiglobal which exploits the Erlang constant
pool to provide constant-time access to the ring without needing
to copy data into individual process heaps.

However, updating a mochiglobal value is very slow, and becomes slower
the larger the item being stored. In Riak clusters with large rings and
many nodes, storing a ring in mochiglobal can take seconds. This delay
is too long during periods of high gossip, where hundreds of ring events
are being triggered a second. The riak_core_ring_manager will stall and
start to exhibit message queue growth.

This commit uses a hybrid approach to solve this problem. When a ring is
first written, it is written to a shared ETS table. If no ring events
have occurred for 90 seconds, the ring is then promoted to mochiglobal.
This provides fast updates during periods of ring churn, while eventually
providing very fast reads after the ring stabilizes. The downside is that
reading from the ETS table before promotion is slower than mochiglobal,
and requires copying the ring into individual process heaps.
Enable warning_as_errors build option and verbose eunit

Several minor changes made to fix pre-existing warnings that would now
prevent the build given the warning_as_errors option.
Add chashbin and bucket metadata

The hybrid ETS/mochiglobal approach solves the fast update problem,
but hurts ring read performance during ring transitions. This issue
can be avoided by exploiting the fact that the majority of Riak only
reads the ring in order to determine index owners, preference lists,
and bucket properties. This commit provides an alternate approach to
getting access to these values without reading the entire ring.

First, this commit creates a binary version of the chash structure
which is stored in the ring manager's ETS table. This chashbin
structure is fast to copy between processes due to off-heap binary
sharing. Furthermore, this structure provides a secondary benefit
of being much faster than the traditional chash structure for normal
operations.

Second, this commit extracts bucket metadata from the ring, storing
the values in the ring manager's ETS table. This allows processes
to directly query a given bucket's metadata without reading the ring.
Optimize riak_core_apl using chashbin

This commit changes riak_core_apl to use the new chashbin structure,
avoiding the need to read the ring in order to compute preflists. This
change also makes preflist calculation significantly faster given the
better performance of the chashbin structure over the traditional chash
structure. Specifically, traditional preflist calculation time scales
with the ring size whereas the new approach is essentially constant
regardless of ring size.
Optimize the "up nodes" logic of riak_core_apl

Change riak_core_apl to use an unordered list and lists:member rather
than ordsets and ordsets:is_element for determining up nodes. This
approach is much faster in practice. Both ordsets:is_element and
lists:member are O(N); however, lists:member is a BIF and has a
significantly lower constant factor. Likewise, this approach avoids
the lists:usort performed by ordsets:from_list.
Optimize coverage plan using chashbin
Update vnode manager to use chashbin
Change vnode + capability managers to use intelligent ring polling

The standard ring event handler sends an event for every ring received.
The vnode manager and capability manager do not care about every ring
received, they only care if the local ring actually changes. In other
words, if a ring is received that provides no new information, and
therefore the local ring does not change, there is no reason to trigger
an event.

This commit changes these managers to no longer listen to ring events,
but instead poll the ring manager at a fixed frequency. This not only
solves the above problem, but also allows the managers to ignore high
frequency ring events.

Given that the common case is for the ring to remain stable, this change
also introduces the notion of the "ring id" which is a monotonic sequence
number for the ring maintained by the ring manager. The periodic polling
then simplifies to checking if the ring id has changed, and only
performing the more expensive state update if so.
Make vnode manager responsible for ensuring vnodes started
Change handoff manager to use sets instead of ordsets
Reduce file server bottleneck when writing ring to disk
Optimize vnode manager

Change certain uses of orddict to dict where the size of the structure
depends on either the ring size or number of nodes in a cluster.

Change update_never_started to only query ring indices when there
are actually unknown modules.
Optimize riak_core_vnode_manager:compute_forwarding

Change from O(n*m) to O(m) + O(n*log32(m))
Change how vnode manager uses ETS to avoid table scans
Randomize vnode inactivity timeout

The goal is to prevent multiple vnodes timing out simultaneously
and overloading the vnode manager with tons of messages.
Permanently disable legacy gossip

This commit hardcodes all checks for legacy gossip to return
false. This is the first step towards removing legacy gossip
from riak_core entirely. A future commit will completely remove
all relevant code as well as adjust the callsites that currently
still call into these now hardcoded functions.
add support for resizable ring

Allows the ring to be resized as a cluster operation similar to join/leave.
Expansion is only allowed, but the restriction is temprorary, and shrinking
requires only small modifications.

When the operation is staged/commited the claimant calulates a resized ring,
running it through claim. It then schedules a "resize operation" for each
partition, via the next list. Each partition is scheduled for the operation because
during resize, each index potentially has data to send to others. Per partition,
the resize operation consists of several "resize transfers", each of which must be
completed to complete the entire operation. The first of these resize transfers is
scheduled by the claimant. Subsequent transfers are scheduled by the vnode that
owns the partition. When all of the transfers for a partition of have completed,
the operation is marked as such in the next list. Once the operation has completed
for *all* partitions the claimant installs the resized ring.

The handoff subsystem has been modified to add a new transfer type: resize_transfer.
In addition, the handoff sender now supports applying a function to keys that are
not sent (due to the handoff filter). The function operates on the unsent key and
a per-sender defined accumulator. Resize transfers are triggered like hinted and
ownership handoff by the vnode manager and vnode. The primary difference between
those transfers and a resize_transfer is that the latter includes a filter
that only sends keys between the source and target partitions if, for both, the
key hashes to the same position in the preflist. Resize transfers also include an
unsent keys function and accumulator that determines which partitions the key
would be destined for. The vnode uses this information to schedule its subsequent
resize transfers. A partition which has not been transferred to yet and is in the
returned list will be scheduled. No more resize transfers will be schedueld when
the returned list contains no partitions or only partitions that have already been
scheduled.

One major difference between typical transfers and a resize operation is the partition
does not delete its data (or unregister) after completing. For some partitions, they will
remain running but a portion of their data should be removed. Other partitions will no
longer live on their original owner should have all of their data deleted. This is
not addressed in this commit.

During resizing, how vnodes forward has been modified, as well. Unlike regular handoff
forwarding is performed only explicitly (when the vnode returns {forward,...} from
handle_handoff_command). The difference is that since the operation consists of multiple
transfers, and will support the ability to abort the operation, all requests are handled
locally and may be forwarded explicitly if they have a defined "request hash". To determine
the hash of the request, a new vnode callback has been added: request_hash. This callback is
optional, unless the application supports ring resizing. The callback can also return undefined
instead of a hash. In which case, the message will be always be handled locally only. Special care
must be taken to handle messages on a vnode forwarded from another, because primaries take part
in forwarding during resize. Coverage requests are not forwarded during a resize.
cleanup after resized ring

After a ring resizes there are several cleanup operations that must
be performed. In some cases a partition no longer exists (if the ring
shrinks) or its owner has been moved to another node. In these cases the
vnode must be unregistered and all its data deleted. In other cases, where
the vnode does continue to exist and has not moved, some data must be removed
but the vnode continues to run. This commit addresses the former.

After completion, the resized ring will have deletions scheduled for the
appropriate indexes. The deletions are scheduled like any other ring transition.
Although this is considered part of the resize operation, at this time the
future ring has been installed and the vnodes targetted for cleanup appear as fallbacks.
After each deletion is complete, the vnode unregisters. After all deletions are
complete the full resize operation is considered complete.

Also move starting of extra vnode proxies to ring handler
ability to shrink ring

not many changes were necessary with the exception that when the ring shrinks
some indexes disappear, which needs to be handled specially. Note: this only works
assuming NewRingSize < MaxN, where MaxN is the largest N value used on any bucket
in the cluster. Also, reduce impact on gossip during resize using 'set_only'.