basho/riak_kv — pull request #586 by engelsanchez (2013-06-25 02:36:53+00:00)
Title: Stop fold when a vnode reaches page size

Currently, 2i pagination may, with a sufficiently slow participating
node, receive many, many more keys than the page size (max_results) from
all the other participating nodes.
This change will track how many results have been received per vnode,
and send a stop fold message to a vnode as soon as it has sent more than
max_results.
Note, however, that vnodes will send in chunks possibly much larger than
the page size (100 items at a time I believe). It would be ideal to pass
max_results to the folding function and have it stop when it reaches
that number. But that will have to wait until later. Release is near.
This should at least help prevent a worst case of unbounded growth that could
consume all the node memory simply because one node is too slow.

/cc @russelldb @jtuple @evanmcc @jrwest