ContinuumIO/blaze — pull request #636 by mrocklin (2014-09-16 20:19:16+00:00)
Title: Split

Another approach to parallelizing blaze expressions. Replaces #549

Might be of interest to @hhuuggoo

This introduces a function, `split` that transforms an computation `t -> expr` into two computations `chunk -> chunkexpr` and `aggregate -> aggregate_expr`

```Python
>>> t = TableSymbol('t', '{name: string, amount: int, id: int}')
>>> expr = t.id.count()
>>> split(t, expr)
((chunk, count(child=chunk['id'])), (aggregate, sum(child=aggregate)))
```

The first TableSymbol, expr pair should be applied to each component of the data. The results of these computations should be assembled and then the second TableSymbol, expr pair should be applied to the concatenation.

This supports reductions, distincts, and split-apply-combine.