Theano/Theano — pull request #916 by mrocklin (2012-09-11 00:37:05+00:00)
Title: WIP - Add MPI operations to Theano

This adds operations for MPISend and MPIRecv as well as the associated waiting operations. These are accessible through the functoins send, recv, isend, irecv.

This allows theano functions to be used as parts of an MPI communication; it isn't of much use to pure-Theano users.

There are some issues:
1. Some of the optimizations don't like these nodes. I have to use a mode without an optimization to compile without errors
2. If you look at theano/tensor/tests/_test_mpi_roundtrip.py I've had to comment out a very slightly more complex test case (look for "y = x + x" in the top branch). With this added the computation never finishes. The processes are stuck waiting for each other. This doesn't make sense to me
3. _test_mpi_roundtrip.py is a simple test. I'm not sure how best to integrate this test into theano's testing framework. In my own code I have a test that calls `os.popen("mpiexec -np 2 .....").read() == "True"``. This works great for me but isn't (I assume) cross platorm.