ufo-kit/ufo-core — pull request #41 by matze (2014-10-15 14:16:53+00:00)
Title: Fix #36: Add fixed scheduler for more user control

This change adds a new scheduler that does not automatically expand the task graph for the current compute environment but rather leaves this decision to the user. Moreover, this change allows users to specify arbitrary connections from single sources to multiple destinations thus fixing #36.

In detail, this is how a user would work with the new scheduler. First some imports and a dot printer for later:

``` python
import sys
from gi.repository import Ufo

class DotPrinter(object):
def __init__(self, char):
self.n = 0
self.char = char

def __call__(self, obj):
if self.n % 10 == 0:
sys.stdout.write(self.char)
sys.stdout.flush()

self.n += 1
```

Now let's create a new graph, a fixed scheduler and also get the available GPU nodes.

``` python
pm = Ufo.PluginManager()
graph = Ufo.TaskGraph()
scheduler = Ufo.FixedScheduler()
gpus = scheduler.get_arch().get_gpu_nodes()
```

This function connects a sub-graph and assigns the passed `gpu` node:

``` python
def make_branch(gpu, filename, axis=413.0, dot_char='.'):
fft = pm.get_task('fft')
ifft = pm.get_task('ifft')
fltr = pm.get_task('filter')
bp = pm.get_task('backproject')
writer = pm.get_task('writer')

bp.set_properties(axis_pos=axis, angle_step=0.01256637)
ifft.set_properties(crop_width=843)
writer.set_properties(filename=filename)

fft.set_proc_node(gpu)
ifft.set_proc_node(gpu)
fltr.set_proc_node(gpu)
bp.set_proc_node(gpu)

graph.connect_nodes(fft, fltr)
graph.connect_nodes(fltr, ifft)
graph.connect_nodes(ifft, bp)
graph.connect_nodes(bp, writer)

bp.connect('processed', DotPrinter(dot_char))
return fft
```

Now we hook up the rest. The `UfoCopyTask` is a task that copies data from input 0 to its output. By connecting this several times, we emulate broadcasting behaviour.

``` python
reader = pm.get_task('reader')
reader.set_properties(path='./sinograms/sino-*.tif')
broadcast = Ufo.CopyTask()

graph.connect_nodes(reader, broadcast)
graph.connect_nodes(broadcast, make_branch(gpus[0], './slices/slice-%05i.tif'))
graph.connect_nodes(broadcast, make_branch(gpus[1], './tmp/slice-%05i.tif', axis=400.0, dot_char='+'))

scheduler.run(graph)
print("\nFinished in {} s".format(scheduler.props.time))
```

That's it. Suggestions and comments are welcome.

@tomago and @kamer16 this might be interesting for you.