Repository: pandas-dev/pandas

Issue:
ENH: make itertuples() return namedtuples
I propose that `itertuples()` should return `collections.namedtuple` objects, a drop-in replacement for the standard tuple but with the benefit of having named fields. I have tested the following with Python 3.4 (only slight changes compared to the current implementation).

``` python
def itertuples(self, index=True):
    arrays = []
        if index:
            arrays.append(self.index)
            fields = ["Index"] + list(self.columns)
        else:
            fields = self.columns

        itertuple = collections.namedtuple("Itertuple", fields, rename=True)

        # use integer indexing because of possible duplicate column names
        arrays.extend(self.iloc[:, k] for k in range(len(self.columns)))

        return (itertuple(*row) for row in zip(*arrays))
```

Example

```
In [3]: df = pd.DataFrame({'col1': [1, 2], 'col2': [0.1, 0.2]}, index=['a', 'b'])

In [4]: for row in df.itertuples():
   ...:     print(row)
   ...:     
Itertuple(Index='a', col1=1, col2=0.10000000000000001)
Itertuple(Index='b', col1=2, col2=0.20000000000000001)

In [5]: row.Index, row.col1, row.col2
Out[5]: ('b', 2, 0.20000000000000001)
```

There is no performance overhead. I'm not sure about the compatibility for older versions of Python, though. The `rename` parameter is needed for renaming disallowed field names and duplicate identifiers to standard position-based identifiers, and this feature was added in Python 2.7/3.1.



Discussion:
sure this would make sense. I would prob name the tuple `Pandas` or somesuch.

pull-requests welcome.

I believe this should work on all versions (inc 2.6) as `namedtuple` was back-ported.

Just checked Python 2.6.6, and there is no support for the `rename` parameter, so this would require some compatibility handling.

oh, why do you need `rename`?

`rename=False` will raise a `ValueError` for invalid field names, while `rename=True` will rename them to `_pos`:

``` python
In [7]: itertuple = namedtuple("Pandas", ["def", "return"], rename=True)

In [8]: itertuple(1,2)
Out[8]: Pandas(_0=1, _1=2)

In [9]: itertuple = namedtuple("Pandas", ["def", "return"], rename=False)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-9-098cde3289d7> in <module>()
----> 1 itertuple = namedtuple("Pandas", ["def", "return"], rename=False)

/usr/lib/python3.5/collections/__init__.py in namedtuple(typename, field_names, verbose, rename)
    394         if _iskeyword(name):
    395             raise ValueError('Type names and field names cannot be a '
--> 396                              'keyword: %r' % name)
    397     seen = set()
    398     for name in field_names:

ValueError: Type names and field names cannot be a keyword: 'def'
```

ahh I see, ok, you can just do this conditionally and raise in py2.6 if its not supported

This was also discussed in #7958.

I'm not sure how to handle the `rename` bit; if I just set `rename=True`, this will make `itertuple` unusable for anyone still using Python 2.6, and `rename=False` will raise an error for any disallowed field name. Should we wait until 2.6 support is dropped?

@mjoud as I said above, you can pass the `rename` argument in a try/except. If it doesn't work (iow in py2.6), then try w/o the rename, if that blows up, then you give up.


Resolving patch:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -584,7 +584,7 @@ def iteritems(self):
         See also
         --------
         iterrows : Iterate over the rows of a DataFrame as (index, Series) pairs.
-        itertuples : Iterate over the rows of a DataFrame as tuples of the values.
+        itertuples : Iterate over the rows of a DataFrame as namedtuples of the values.
 
         """
         if self.columns.is_unique and hasattr(self, '_item_cache'):
@@ -617,7 +617,7 @@ def iterrows(self):
            int64
 
            To preserve dtypes while iterating over the rows, it is better
-           to use :meth:`itertuples` which returns tuples of the values
+           to use :meth:`itertuples` which returns namedtuples of the values
            and which is generally faster as ``iterrows``.
 
         2. You should **never modify** something you are iterating over.
@@ -632,7 +632,7 @@ def iterrows(self):
 
         See also
         --------
-        itertuples : Iterate over the rows of a DataFrame as tuples of the values.
+        itertuples : Iterate over the rows of a DataFrame as namedtuples of the values.
         iteritems : Iterate over (column name, Series) pairs.
 
         """
@@ -641,15 +641,23 @@ def iterrows(self):
             s = Series(v, index=columns, name=k)
             yield k, s
 
-    def itertuples(self, index=True):
+    def itertuples(self, index=True, name="Pandas"):
         """
-        Iterate over the rows of DataFrame as tuples, with index value
+        Iterate over the rows of DataFrame as namedtuples, with index value
         as first element of the tuple.
 
         Parameters
         ----------
         index : boolean, default True
             If True, return the index as the first element of the tuple.
+        name : string, default "Pandas"
+            The name of the returned namedtuple.
+
+        Notes
+        -----
+        The columns names will be renamed to positional names if they are
+        invalid Python identifiers, repeated, or start with an underscore.
+        With a large number of columns (>255), regular tuples are returned.
 
         See also
         --------
@@ -666,16 +674,32 @@ def itertuples(self, index=True):
         b     2   0.2
         >>> for row in df.itertuples():
         ...     print(row)
-        ('a', 1, 0.10000000000000001)
-        ('b', 2, 0.20000000000000001)
+        ...
+        Pandas(Index='a', col1=1, col2=0.10000000000000001)
+        Pandas(Index='b', col1=2, col2=0.20000000000000001)
 
         """
         arrays = []
+        fields = []
         if index:
             arrays.append(self.index)
+            fields.append("Index")
 
         # use integer indexing because of possible duplicate column names
         arrays.extend(self.iloc[:, k] for k in range(len(self.columns)))
+
+        # Python 3 supports at most 255 arguments to constructor, and
+        # things get slow with this many fields in Python 2
+        if len(self.columns) + index < 256:
+            # `rename` is unsupported in Python 2.6
+            try:
+                itertuple = collections.namedtuple(
+                    name, fields+list(self.columns), rename=True)
+                return (itertuple(*row) for row in zip(*arrays))
+            except:
+                pass
+
+        # fallback to regular tuples
         return zip(*arrays)
 
     if compat.PY3:  # pragma: no cover
@@ -1213,7 +1237,7 @@ def to_panel(self):
 
     def to_csv(self, path_or_buf=None, sep=",", na_rep='', float_format=None,
                columns=None, header=True, index=True, index_label=None,
-               mode='w', encoding=None, compression=None, quoting=None, 
+               mode='w', encoding=None, compression=None, quoting=None,
                quotechar='"', line_terminator='\n', chunksize=None,
                tupleize_cols=False, date_format=None, doublequote=True,
                escapechar=None, decimal='.', **kwds):
@@ -1251,7 +1275,7 @@ def to_csv(self, path_or_buf=None, sep=",", na_rep='', float_format=None,
             A string representing the encoding to use in the output file,
             defaults to 'ascii' on Python 2 and 'utf-8' on Python 3.
         compression : string, optional
-            a string representing the compression to use in the output file, 
+            a string representing the compression to use in the output file,
             allowed values are 'gzip', 'bz2',
             only used when the first argument is a filename
         line_terminator : string, default '\\n'


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -5545,6 +5545,27 @@ def test_itertuples(self):
         dfaa = df[['a', 'a']]
         self.assertEqual(list(dfaa.itertuples()), [(0, 1, 1), (1, 2, 2), (2, 3, 3)])
 
+        tup = next(df.itertuples(name='TestName'))
+
+        # no support for field renaming in Python 2.6, regular tuples are returned
+        if sys.version >= LooseVersion('2.7'):
+            self.assertEqual(tup._fields, ('Index', 'a', 'b'))
+            self.assertEqual((tup.Index, tup.a, tup.b), tup)
+            self.assertEqual(type(tup).__name__, 'TestName')
+
+        df.columns = ['def', 'return']
+        tup2 = next(df.itertuples(name='TestName'))
+        self.assertEqual(tup2, (0, 1, 4))
+
+        if sys.version >= LooseVersion('2.7'):
+            self.assertEqual(tup2._fields, ('Index', '_1', '_2'))
+
+        df3 = DataFrame(dict(('f'+str(i), [i]) for i in range(1024)))
+        # will raise SyntaxError if trying to create namedtuple
+        tup3 = next(df3.itertuples())
+        self.assertFalse(hasattr(tup3, '_fields'))
+        self.assertIsInstance(tup3, tuple)
+
     def test_len(self):
         self.assertEqual(len(self.frame), len(self.frame.index))