basho/riak_test — pull request #505 by kellymclaughlin (2014-01-10 23:55:35+00:00)
Title: Changes to loaded_upgrade test to improve reliability

* Wait for services after upgrading a node
* Remove hardcoded sleep call
* Reduce time between upgrades from 300 to 120 seconds
* Refactor worker functions to accept a report pid parameter instead
of assuming a loaded_upgrade named process.
* Remove the mapreduce load. The results of the mapreduce queries
during node shutdown are not predictable enough at this time to rely
on for this test.

Regarding the removal of the mapreduce load, I believe I have identified an issue with riak_pipe + mapreduce that can lead to work being sent to nodes where KV is already in the process of shutting down. The queries still return successfully, but it leads to unpredictable results and makes it inappropriate to rely on for this test. I think this could be improved by changing `riak_pipe` to allow the service used determining which nodes are available during coverage planning to be configurable. Currently it is hard coded to `riak_pipe`, but `riak_pipe` may shutdown after `riak_kv` and I think `riak_kv` should really be the service of more interest to mapreduce. This needs more investigation so for now the answer is to remove the mapreduce load.