scikit-learn/scikit-learn — pull request #3196 by andrewclegg (2014-05-25 09:51:04+00:00)
Title: Fix for Python 3.x race condition causing "RuntimeError: dictionary changed size during iteration"

I was running the test suite on a heavily loaded server via `nosetests` -- it was testing a load of other packages at the same time, but in completely separate processes.

I got a load of test failures with "RuntimeError: dictionary changed size during iteration" in a test helper function:

https://gist.github.com/andrewclegg/e3060f4e49e04ee66dac

The function in question (`clean_warning_registry`) loops through `sys.modules.values()`. In Python 3 this is not a static list as before, but an iterator over a dynamic view of the dict (http://blog.labix.org/2008/06/27/watch-out-for-listdictkeys-in-python-3). So, another thread within sklearn's `nosetests` process (culprit unknown) must have added/removed a module at a bad time.

Calling `.copy()` first fixes this as it's atomic w.r.t. the GIL, and it won't do any harm in Python 2.x. The additional overhead of shallow-copying a small data structure in a test method is negligible compared to the rest of sklearn...