ContinuumIO/blaze — pull request #245 by mrocklin (2014-04-25 01:34:21+00:00)
Title: Table

OK, here is a tiny draft of a purely abstract Table object. The `Table` itself is pure syntax, it doesn't attach itself to any data descriptor, it doesn't assume any underlying storage technique (i.e. isn't a row or column store). It only specifies a syntax and builds out an expression tree. So far I'm stealing syntax from Pandas.

```Python

>>> from blaze.objects.table import Table
>>> from blaze.compute.python import compute

>>> accounts = Table('{name: string, amount: int}')
>>> deadbeats = accounts['name'][accounts['amount'] < 0]
```

This produces a tree that we can now optimize and crunch down on a variety of backends. Notably I've produced proof of concept backends for streaming Python, Pandas, and SQLAlchemy. One could imagine adding PyTables, ColumnStore (abstract backend that leverages `blaze.array`) or some HDFS solution.

### Target Python

```Python
>>> data = [['Alice', 100], ['Bob', -50], ['Charlie', -20]]

>>> list(compute(deadbeats, data))
['Bob', 'Charlie']
```

### Target Pandas

```Python
>>> from pandas import DataFrame
>>> data = [['Alice', 100], ['Bob', -50], ['Charlie', -20]]
>>> df = DataFrame(data, columns=['name', 'amount'])

>>> compute(deadbeats, df)
1 Bob
2 Charlie
Name: name, dtype: object
```

### Target SQLAlchemy

```Python
>>> from sqlalchemy import Table, Column, MetaData, Integer, String
>>> t = Table('accounts', MetaData(),
... Column('name', String, primary_key = True),
... Column('amount', Integer))

>>> print(compute(deadbeats, t))
SELECT accounts.name
FROM accounts
WHERE accounts.amount < :amount_1
```

The only functionality written is the functionality that you're seeing. This didn't take me very long to do, and I think that exploring further along this direction is cheap. Still, I'd like to get feedback on this idea as it's very different from what was previously proposed.