biocore/qiime — pull request #1616 by wasade (2014-07-08 22:37:48+00:00)
Title: ENH: parallel context

Currently don't have support for mapping yet, but we can submit system calls or function calls now via IPython. The latter may require state updates on the engines, which is feasible from the current context. It is likely that mapping will go in relatively easily.

From the shell:

```bash
$ echo -e "profile\tdefault" >> ~/.qiime_config
$ echo -e "n_engines\t2" >> ~/.qiime_config
$ parallel start
```

From within Python:

```python
>>> from qiime.parallel.context import context
>>> context.submit_sync('hostname')
('rl1-1-220-221-dhcp.int.colorado.edu\n', '', 0)
>>> def foo():
... return "bar"
...

>>> context.submit_sync(foo)
'bar'
>>> def foo(a):
... return a**2
...
>>> res = context.submit_async(foo, 5)
>>> res.result
25
```

From the shell:

```bash
$ parallel stop
```