bipy/pyqi — pull request #201 by wasade (2013-10-15 06:01:01+00:00)
Title: Functional interface and IPython parallel support (scroll down...)

This uses the pending changes in my outparams branch. This is not safe to merge yet (also because I did not add test code...) but it is a proof of concept interface. From this exercise, it is clear that we're missing a general way to represent the "controller" or driver, essentially the analog to the ``pyqi`` command. But, that is another discussion... perhaps those should be per interface.

This functional interface can handle an arbitrary number of positional arguments or a single argument; it cannot handle zero arguments right now although that would not be hard to add. Either a single or multiple positional arguments can be accompanied by keyword arguments. Only a single output can be returned by the interface, though the ``Command`` is of course free to return whatever it wants. Only the positional arguments (single or multiple) and the single output need to be described in the configuration.

Anyway, below is an example of a toy adder. This code is very light, supports both positional and keyword arguments, can be used in a ``reduce`` and a ``map``, and is meant as an example of what is possible with the functional interface. This interface assumes everything is already in memory:

```python
>>> from pyqi.interfaces.functional.config.adder import interface_object
>>> x = interface_object()
>>> x(10.0, 20.0)
30.0
>>> reduce(x,[10.0, 20.0, 30.0, 40.0])
100.0
>>> reduce(x,[10.0,20.0,30.0,40])
Traceback (most recent call last):
File "", line 1, in
File "/Users/mcdonadt/ResearchWork/software/pyqi/pyqi/core/interface.py", line 49, in __call__
cmd_input = self._input_handler(in_, *args, **kwargs)
File "/Users/mcdonadt/ResearchWork/software/pyqi/pyqi/core/interfaces/functional/__init__.py", line 52, in _input_handler
"type!" % interface_in.Name)
ValueError: Input y doesn't appear to be the correcttype!
>>> map(lambda x: f(x, y=5.0), [1.0, 2.0, 3.0])
[6.0, 7.0, 8.0]
```

Notice that it is specific to the type of float and uses strict evaluation.

With this in place, I thought I'd see if IPython's parallel code works :)

```python
>>> # prior to interpreter, ran "ipcluster start -n 4"
>>> # mostly based off of: http://ipython.org/ipython-doc/stable/parallel/parallel_intro.html
>>> from IPython.parallel import Client
>>> c = Client()
>>> c.ids
[0, 1, 2, 3]
>>> dview = c[:]
>>> from pyqi.interfaces.functional.config.adder import interface_object
>>> c[:].execute("from pyqi.interfaces.functional.config.adder import interface_object; f = interface_object()")

>>> f = interface_object()
> dview.map_sync(lambda x: f(x, y=10.0), arange(32.0))
[10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.0, 29.0, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0]
```