enthought/distarray — pull request #467 by cowlicks (2014-06-25 20:48:08+00:00)
Title: Add a MPI only mode so distarray programs can be run without IPython.parallel.

*This is not ready to merge*

The interface:

* write a program using Distarray, call it `prog.py`. But instead of using `Context` use `MPIConetxt`.
* Then you can run the program with as many engines as you want, using `mpiexec -n 1 python prog.py: -n engine.py`.

You cannot run `prog.py` interactively, only with mpiexec. We should merge `Context` and `MPIContext` so the same program can be run in either mode.

The implementation:
I copied the `Context` class and re-implemented the parts that used IPython (`push, pull, apply, ...`). I then added a module called `engine.py` which runs a loop waiting for messages from rank 0. It reads these messages and acts on them. So now the program being run acts as the client, it is rank 0. It is an intercomm with engines as the other group.

Todo:
* Make the test suite agnostic of IPython
* refactor new additions and rebase for commit clarity
* fix whatever bugs come up when running the test suite with mpi-only mode
* Unify contetx classes so the same code can be run with MPI-only or with IPython too