wesm/pandas — Issue #611: TypeError in Cython with tuple subclasses in DataFrame.from_records
Author: wesm
Created: 2012-01-11 21:25:49+00:00
State: open
Labels: Bug

From user email

Hi there,

Just a quick bug report. I'm using psycopg2 with a NamedTupleCursor, which returns namedtuples for rows. I'm trying to use DataFrame.from_records like so:

df = DataFrame.from_records(rows, rows[0]._fields)

I get the following error:

```
TypeError Traceback (most recent call last)
/Users/kdevens/source/(etc.) in ()
----> 1 df = DataFrame.from_records(rows, rows[0]._fields)

/Library/Python/2.6/site-packages/pandas-0.6.1-py2.6-macosx-10.6-universal.egg/pandas/core/frame.pyc in from_records(cls, data, index, exclude, columns, names)
533 columns, sdict = _rec_to_dict(data)
534 else:
--> 535 sdict, columns = _list_to_sdict(data, columns)
536
537 if exclude is None:

/Library/Python/2.6/site-packages/pandas-0.6.1-py2.6-macosx-10.6-universal.egg/pandas/core/frame.pyc in _list_to_sdict(data, columns)
3547 def _list_to_sdict(data, columns):
3548 if isinstance(data[0], tuple):
-> 3549 content = list(lib.to_object_array_tuples(data).T)
3550 else:
3551 # list of lists

/Library/Python/2.6/site-packages/pandas-0.6.1-py2.6-macosx-10.6-universal.egg/pandas/_tseries.so in pandas._tseries.to_object_array_tuples (pandas/src/tseries.c:52160)()

TypeError: Expected tuple, got Record
```

So it looks like in tseries.c it's doing something like "type(row) == tuple" instead of "isinstance(row, tuple)". For what it's worth, "isinstance(rows[0], tuple)" is true, so the tuples returned by psycopg2 are definitely real namedtuples. Additionally, the following works fine:

```
In [49]: dictrows = [r._asdict() for r in rows]

In [50]: df = DataFrame.from_dict(dictrows)
```

So there's no problem with the data, or with DataFrame, just the type check in tseries.c.

Let me know if I can provide any more information! Thanks.