joblib/joblib — pull request #21 by cavorite (2011-12-24 00:40:15+00:00)
Title: Additional argument to Parallel for reporting the progress every certain number of elements

I added a new argument to Parallel: report_every. It allows to limit the output of the verbose mode
so that it prints a line every certain number of elements. This might be useful for processing a large sequence of elements. For example:

>>> r = Parallel(n_jobs=2, verbose=1, report_every=50)(delayed(sleep)(.1) for _ in range(10))
[Parallel(n_jobs=2)]: Done 1 out of 1000 |elapsed: 0.1s remaining: 0.9s
[Parallel(n_jobs=2)]: Done 101 out of 1000 |elapsed: 0.1s remaining: 0.9s
[Parallel(n_jobs=2)]: Done 201 out of 1000 |elapsed: 0.1s remaining: 0.9s
# ...