cherry pick 5de7972 into 1.3 branch (backporting)

It was desired to have the riak_core_vnode_worker_pool race from
basho/riak_core#300 backported to the 1.3 branch. The parent commit
containing the regression tests for this race has been elided, since
it requires changes to poolboy that are not otherwise required to
fix the issue (so there is no need to roll a new poolboy version).

Original commit message:

fix races between riak_core_vnode_worker_pool and poolboy

This patch fixes both races demonstrated by the test in the previous
commit, and described in #298.

The fix for the checkin race is for the worker to only checkin to
riak_core_vnode_worker_pool, and allow riak_core_vnode_worker_pool to
decide when to checkin to poolboy. If there is outstanding work in the
queue, the worker will just be reused immediately instead of being
checked in.

The fix for the DOWN race is for the worker to send a 'worker_started'
message to riak_core_vnode_worker_pool, when the worker starts. While
the worker is sending this message, poolboy is blocking, so
riak_core_vnode_worker_pool can send poolboy a checkout message
immediately, and still be guaranteed that it will be received after
poolboy has put the worker in its pool.

Documentation about how this works, and about how
riak_core_vnode_worker_pool is acting as a queue wrapper around the
pool, is also included.
Implement vnode overload protection
Rewrite and extend the vnode overload protection logic

The primary change is the addition of a fast path method for adjusting
the estimated mailbox size of a vnode. The vnode proxy now sends a ping
to the vnode after a configurable number of messages.  If the vnode
replies, the vnode proxy knows that all messages sent before the ping
have been handled and therefore it can adjust the mailbox estimate
accordingly. In the common case, this ping/pong approach keeps the
mailbox estimate up-to-date without ever falling back to the more
expensive process_info call.

Minor changes:
-- When a message is dropped, the `dropped_vnode_requests_total` counter
   is incremented.
-- When a vnode exits, the proxy resets its overload state.
-- Overload parameters are now configurable via application variables.

The `vnode_overload_threshold` setting determines the overload threshold.
If a vnode mailbox ever reaches this number, the corresponding vnode proxy
starts dropping messages. When set to `undefined`, overload protection is
disabled and the vnode proxy uses the older/more efficient code path.

The `vnode_check_request_interval` setting sets the frequency of mailbox
pings. After this number of messages have been forwarded by the proxy,
the proxy will send a ping message.

The `vnode_check_interval` setting sets the maximum number of messages
that can be sent before the proxy will directly query the vnode mailbox
size using process_info.
Merge branch 'jdb-vnode-proxy-overload-1.2' into jdb-vnode-proxy-overload-1.3
First draft (3rd edition), ready for some review

Vnode replies always go via reply(), and reply() always uses unreliable
messaging.  (As opposed to the usual (and more reliable) send-and-pray
messaging.)

During handoff, all forwarding requests use unreliable vnode master
commands to avoid net_kernel blocking interference.
Review: fix return value of bang_unreliable/2
Add PULSE instrumentation to riak_core_send_msg.erl
Merge pull request #307 from basho/bwf-pool-race-1.3

Backport issue 300 to 1.3 branch
Merge pull request #314 from basho/jdb-vnode-proxy-overload-1.3
Merge pull request #317 from basho/jrw-gh274-13backport

Backport of #299 to 1.3
Roll riak_core version 1.3.2
Change riak_core_ring_manager to use gen_server
Hybrid ring/mochiglobal approach

Numerous processes concurrently read and access the ring in a
variety of time sensitive code paths. To make this efficient,
riak_core uses mochiglobal which exploits the Erlang constant
pool to provide constant-time access to the ring without needing
to copy data into individual process heaps.

However, updating a mochiglobal value is very slow, and becomes slower
the larger the item being stored. In Riak clusters with large rings and
many nodes, storing a ring in mochiglobal can take seconds. This delay
is too long during periods of high gossip, where hundreds of ring events
are being triggered a second. The riak_core_ring_manager will stall and
start to exhibit message queue growth.

This commit uses a hybrid approach to solve this problem. When a ring is
first written, it is written to a shared ETS table. If no ring events
have occurred for 90 seconds, the ring is then promoted to mochiglobal.
This provides fast updates during periods of ring churn, while eventually
providing very fast reads after the ring stabilizes. The downside is that
reading from the ETS table before promotion is slower than mochiglobal,
and requires copying the ring into individual process heaps.
Enable warning_as_errors build option and verbose eunit

Several minor changes made to fix pre-existing warnings that would now
prevent the build given the warning_as_errors option.
Add chashbin and bucket metadata

The hybrid ETS/mochiglobal approach solves the fast update problem,
but hurts ring read performance during ring transitions. This issue
can be avoided by exploiting the fact that the majority of Riak only
reads the ring in order to determine index owners, preference lists,
and bucket properties. This commit provides an alternate approach to
getting access to these values without reading the entire ring.

First, this commit creates a binary version of the chash structure
which is stored in the ring manager's ETS table. This chashbin
structure is fast to copy between processes due to off-heap binary
sharing. Furthermore, this structure provides a secondary benefit
of being much faster than the traditional chash structure for normal
operations.

Second, this commit extracts bucket metadata from the ring, storing
the values in the ring manager's ETS table. This allows processes
to directly query a given bucket's metadata without reading the ring.
Optimize riak_core_apl using chashbin

This commit changes riak_core_apl to use the new chashbin structure,
avoiding the need to read the ring in order to compute preflists. This
change also makes preflist calculation significantly faster given the
better performance of the chashbin structure over the traditional chash
structure. Specifically, traditional preflist calculation time scales
with the ring size whereas the new approach is essentially constant
regardless of ring size.
Optimize the "up nodes" logic of riak_core_apl

Change riak_core_apl to use an unordered list and lists:member rather
than ordsets and ordsets:is_element for determining up nodes. This
approach is much faster in practice. Both ordsets:is_element and
lists:member are O(N); however, lists:member is a BIF and has a
significantly lower constant factor. Likewise, this approach avoids
the lists:usort performed by ordsets:from_list.
Optimize coverage plan using chashbin
Update vnode manager to use chashbin
Change vnode + capability managers to use intelligent ring polling

The standard ring event handler sends an event for every ring received.
The vnode manager and capability manager do not care about every ring
received, they only care if the local ring actually changes. In other
words, if a ring is received that provides no new information, and
therefore the local ring does not change, there is no reason to trigger
an event.

This commit changes these managers to no longer listen to ring events,
but instead poll the ring manager at a fixed frequency. This not only
solves the above problem, but also allows the managers to ignore high
frequency ring events.

Given that the common case is for the ring to remain stable, this change
also introduces the notion of the "ring id" which is a monotonic sequence
number for the ring maintained by the ring manager. The periodic polling
then simplifies to checking if the ring id has changed, and only
performing the more expensive state update if so.