ContinuumIO/blaze — pull request #484 by mrocklin (2014-08-19 15:45:09+00:00)
Title: [WIP] - `dplyr` interface`

DPlyr is a fine project in R for tabular computations. Those interested might want to look through this vignette http://cran.rstudio.com/web/packages/dplyr/vignettes/introduction.html .

I'd like to start experimenting with new frontends for Blaze. I've mapped something that approximates the dplyr interface to Blaze expressions. Here is a small demo.

```Python
In [1]: from blaze import *

In [2]: csv = CSV('/home/mrocklin/data/iris/iris.csv')

In [3]: t = Table(csv)

In [4]: t
Out[4]:
SepalLength SepalWidth PetalLength PetalWidth Species
0 5.1 3.5 1.4 0.2 setosa
1 4.9 3.0 1.4 0.2 setosa
2 4.7 3.2 1.3 0.2 setosa
3 4.6 3.1 1.5 0.2 setosa
4 5.0 3.6 1.4 0.2 setosa
5 5.4 3.9 1.7 0.4 setosa
6 4.6 3.4 1.4 0.3 setosa
7 5.0 3.4 1.5 0.2 setosa
8 4.4 2.9 1.4 0.2 setosa
9 4.9 3.1 1.5 0.1 setosa
...

In [5]: from blaze.api.dplyr import *

In [6]: inject(t) # inject columns into namespace to approximate R's macros

In [7]: PetalWidth / PetalLength
Out[7]:
0
0 0.142857
1 0.142857
2 0.153846
3 0.133333
4 0.142857
5 0.235294
6 0.214286
7 0.133333
8 0.142857
9 0.066667
...

In [8]: select(t, PetalWidth, Species) # dplyr term for projection
Out[8]:
PetalWidth Species
0 0.2 setosa
1 0.2 setosa
2 0.2 setosa
3 0.2 setosa
4 0.2 setosa
5 0.4 setosa
6 0.3 setosa
7 0.2 setosa
8 0.2 setosa
9 0.1 setosa
...

In [11]: mutate(t, sepal_ratio=SepalLength / SepalWidth) # Define new column
Out[11]:
SepalLength SepalWidth PetalLength PetalWidth Species sepal_ratio
0 5.1 3.5 1.4 0.2 setosa 1.457143
1 4.9 3.0 1.4 0.2 setosa 1.633333
2 4.7 3.2 1.3 0.2 setosa 1.468750
3 4.6 3.1 1.5 0.2 setosa 1.483871
4 5.0 3.6 1.4 0.2 setosa 1.388889
5 5.4 3.9 1.7 0.4 setosa 1.384615
6 4.6 3.4 1.4 0.3 setosa 1.352941
7 5.0 3.4 1.5 0.2 setosa 1.470588
8 4.4 2.9 1.4 0.2 setosa 1.517241
9 4.9 3.1 1.5 0.1 setosa 1.580645
...

In [12]: summarize(t, pl=PetalLength.max(), pw=PetalWidth.max()) # Named reductions
Out[12]: (6.9, 2.5)

In [13]: g = group_by(t, Species) # Split-apply-combine pattern

In [14]: summarize(g, max=max(PetalLength), sum=sum(PetalLength))
Out[14]:
Species max sum
0 setosa 1.9 73.2
1 versicolor 5.1 213.0
2 virginica 6.9 277.6

```

I really like some parts of this, notably the use of keyword arguments to force users to label columns, and the `mutate` function (which may interest @chdoig).

Mostly though I want people to start thinking about Blaze frontends. Our current front-end just mirrors our expression system which was designed to map cleanly to backends. It wasn't designed with human usability in mind. Discussion on this broader topic of user intrefaces is most welcome.