ipython/ipython — Issue #1277: document tuning IPython.parallel
Author: minrk
Created: 2012-01-17 01:16:46+00:00
State: open
Labels: docs, parallel

The parallel docs fall short in a number of ways, largely due to being a straight translation of the old IPython.kernel docs.

My [scipy tutorial](https://minrk.github.com/scipy-tutorial-2011) is a bit more complete in some areas, but one thing particularly lacking is performance tuning.

The principal knobs for tuning IPython.parallel are:

* TaskScheduler.scheme
* TaskScheduler.hwm
* HubFactory.db_class
* use of References to avoid data movement
* message serialization
* setting up inter-engine communication

And the typical workload characteristics that are important to how you would tune IPython are:

* task atomicity/dependency
* data movement
* task heterogeneity