basho/riak_core — pull request #457 by jrwest (2013-11-21 21:55:42+00:00)
Title: Yet Another Round of Cluster Metadata Improvements (YAROCMI #1)

This PR includes several minor changes to Cluster Metadata to address outstanding issues:

* #450
* #440
* #438

In addition, this PR adds two additional functions to `riak_core_metadata`. `to_list/1,2` are primarily useful for testing and debugging.

The commits contain details about the motivations for each change but I've put some extra notes below.

### Increasing Eager Set Size for Larger Clusters

Prior to these changes the initial size of the eager peer set for all nodes had a maximum of two. This increased the chances of relying on Cluster Metadata's anti-entropy mechanism. Ideally, we want to reduce this chance as much as possible. These changes increase that maximum depending on the size of the cluster. In addition, as the number of nodes in the cluster increases or decreases, because the tree is re-initialized at each membership change, it will be re-optimized for the new size.

The following table shows the average percentage of failed nodes before an exchange is needed for 10, 20 and 40 nodes. These numbers were obtained by running a controlled test against the actual implementation.

# nodes
before
after

10
64%
85%

20
44%
78%

20
40%
76%

### allow_put

What this option does is described in the commit and edocs added as part of it. However, the motivation for it is roughly explored [here](https://gist.github.com/jrwest/f8c0d49174f4db1c4c88)