Repository: pandas-dev/pandas

Issue:
groupby enumerate method
I'm not sure what a good word for this is (count is taken, order means sort)!

But it's quite an often used thing to create a column which enumerates the items in each group / counts their occurrences.

You can hack it:

```
In [1]: df = pd.DataFrame([[1, 2], [2, 3], [1, 4], [1, 5], [2, 6]])

In [2]: g = df.groupby(0)

In [3]: g.apply(lambda x: pd.Series(np.arange(len(x)), x.index))
Out[3]:
0    0
1    0
2    1
3    2
4    1
dtype: int64

In [5]: df['order'] = _

In [6]: df
Out[6]:
   0  1  order
0  1  2      0
1  2  3      0
2  1  4      1
3  1  5      2
4  2  6      1
```

_I've seen this in a few SO questions, here's just [one](http://stackoverflow.com/questions/18392283/pandas-return-first-item-in-dataframe-grouped-by-user)._

cc @cpcloud (and I've seen @jreback answer a question with this)



Discussion:
counter() ?

I kind of favour `enumerate()` as it's more in line with python (and it reminds me of Demolition Man).

_I was wrong: count isn't taken for groupby, it's size, but I'm -1 on same/similarly named methods with different meaning across groupby/DataFrame._

plus with enumerate maybe you could provide an iterator or column to cycle through..

@cpcloud Interesting idea, not sure how it'd work (if you pass in an iterator it'd be eaten by the first thing/start part way through?). But perhaps mapping after the enumeration would be the way to go (mod and apply a getitem)?

```
In [21]: df['order'].apply(list('abc').__getitem__)  # continued from above
Out[21]:
0    a
1    a
2    b
3    c
4    b
Name: order, dtype: object

In [22]: (df['order'] % 2).apply(list('ab').__getitem__)  # I guess this is what you mean by cycle
Out[22]:
0    a
1    a
2    b
3    a
4    b
Name: order, dtype: object
```

To implement is there a more efficient way? Or should it just be a shortcut:

```
g.apply(lambda x: pd.Series(np.arange(len(x)), x.index))
```

I wonder if enumerate could also mean enumerate by the group it's in (or does that not make sense?) maybe that could be a possible kwarg

enumerate is something specific in Python (iterator with index), might want
to pick a different name.

As I read through I had this question: how is this different from
value_counts()?

it's enumerating the integer index of the elements in a group

sort of like a `cumvalue_counts()`

(which is a _horrible_ name for this)

I think enumerate is the right name for this _because_ it's something specific to python... I really like this name.

Was wondering if another potentially useful/expected thing for enumerate to do is enumerate the groups (I'm not sure), a little bit like this:

```
from itertools import count
c = count(-1)
df = pd.DataFrame([['a', 2], ['b', 3], ['a', 4], ['a', 5], ['b', 6]])

In [4]: df
Out[4]: 
   0  1
0  a  2
1  b  3
2  a  4
3  a  5
4  b  6

In [5]: g = df.groupby(0)

In [6]: g.apply(lambda x: pd.Series(c.next(), x.index))
Out[23]: 
0    0
1    1
2    0
3    0
4    1
dtype: int64
```

Although potentially the groups are not well ordered anyway....

(I think they are ordered, so this should be ok, I wonder if these two alternatives could be distinguished with a well chosen kwarg...)

My point of view,
as an average python/pandas user,

a name with the verb 'to count' is the most intuitive
because you count the apparitions
I take the first value in a Series, let's say value is 6,
and then I parse visually the Series counting the apparitions of 6

enumerate in python is not counting
it gets every element with its index
so it's not intuitive at all from my point of view
it's quite confusing actually 

I take the point that it is somewhat confusing, but thing that any other option will be significantly more so.

This is enumerating _each_ of the "apparitions". count is the wrong word since it usually has the meaning (elsewhere in pandas, e.g. DataFrame .count) of counting the total occurrences (i.e. groupby .size). We are not counting, we are enumerating.

> enumerate in python is not counting it gets every element with its index

If you replace index with occurrence, this is exactly what we are doing.

how about `tally`, or `number` ?

I like `tally`, -1 on `number`.

other names: 
counter
Numerate
Apparencies
Count_apparencies
Count_occurences
Occurences
Rolling_count

That's all i can think now of 

I checked English dictionary 
Apparitions and apparencies do not make any sense in English, it seems 

sorry, English is not my first language 

maybe "appearances" is close to what you were thinking?

-1 on count or counter (as mentioned above). Also -1 on rolling_count since those function have windows rather than cumulative, and actually it does make sense to apply these.

I wonder if cumcount makes sense, inline with already existing cumsum etc. :s (though I dislike it mathematically)

I'm unsure about tally, it could be ok but I'm unsure if it's a bit of colloquialism (I think it's less clear than enumerate)...

I like `tally`. `cumcount` is good too.

EDIT: Actually, after thinking a bit more about `cumcount` I like it less. `cumsum` is always increasing. `cumcount` wouldn't be since the "current" cumulative count is jumping between groups...

@ifmihai I like _apparitions_ :ghost: :smile: 

@TomAugspurger It's always increasing _within each group_, just like cumcount/enumerate.

cumsum is already available:

```
In [31]: df = pd.DataFrame([[1, 1], [2, 1], [1, 2]], columns=['A', 'B'])

In [32]: g = df.groupby('B')

In [33]: g.cumsum()
Out[33]: 
   A  B
0  1  1
1  3  2
2  1  2

In [34]: g.A.cumsum()
Out[34]: 
0    1
1    3
2    1
dtype: int64
```

(and is not increasing)

Actually I think tally suffers from one of issues that count etc. does, it usually means total (and not cumulative total).

If we were to go with cumcount etc we should make cumsum etc. a bit more visible...

Yep, I was wrong. Not sure what I was thinking.  Should we split the difference and go with `cumtally`?  Programming is hard.

@jtratner YES! appearances was the word I was searching for, 
but it seems <b>cumcount, enumerate, tally</b> are preferred 
(I put them in the order of my own preference)

From my perspective (of a foreigner) tally doesn't mean anything, especially in programming
<b>cumcount</b> appears the most logical up to this point, to be related to pandas also (or statistics)
but <b>count</b> gives me also the impression of a total, not cummulative

@cpcloud ha ha! anyway, apparitions are interesting :)

some other words that can be used (maybe): 
track
reckon(?)
register(?)
mark
score
account

personally I prefer <b>appearances</b> or <b>track_appearances</b>

I guess it will not be so much a used function, so I guess the name can be longer if needed, right?

-1 for `tally`/`cumtally`. As a non-native English speaking person, I had never heard of that word, and I asked my colleagues and I am not alone. 

If I would describe the action, I would say I number the items within the group (give them a number), so maybe `number` or `numbering` (but number has also other nuances, so maybe not so clear). 
For the rest, I think `cumcount` is clear and in the same line with cumsum etc, so +1.  And I think I like 'occurence' more than 'appearance'.


Resolving patch:
diff --git a/pandas/core/groupby.py b/pandas/core/groupby.py
--- a/pandas/core/groupby.py
+++ b/pandas/core/groupby.py
@@ -468,6 +468,7 @@ def ohlc(self):
         Compute sum of values, excluding missing values
 
         For multiple groupings, the result index will be a MultiIndex
+
         """
         return self._cython_agg_general('ohlc')
 
@@ -480,9 +481,49 @@ def picker(arr):
                 return np.nan
         return self.agg(picker)
 
+    def cumcount(self):
+        '''
+        Number each item in each group from 0 to the length of that group.
+
+        Essentially this is equivalent to
+        
+        >>> self.apply(lambda x: Series(np.arange(len(x)), x.index)).
+
+        Example
+        -------
+
+        >>> df = pd.DataFrame([['a'], ['a'], ['a'], ['b'], ['b'], ['a']], columns=['A'])
+        >>> df
+           A
+        0  a
+        1  a
+        2  a
+        3  b
+        4  b
+        5  a
+        >>> df.groupby('A').cumcount()
+        0    0
+        1    1
+        2    2
+        3    0
+        4    1
+        5    3
+        dtype: int64
+
+        '''
+        index = self.obj.index
+        cumcounts = np.zeros(len(index), dtype='int64')
+        for v in self.indices.values():
+            cumcounts[v] = np.arange(len(v), dtype='int64')
+        return Series(cumcounts, index)
+
+
     def _try_cast(self, result, obj):
-        """ try to cast the result to our obj original type,
-        we may have roundtripped thru object in the mean-time """
+        """
+        try to cast the result to our obj original type,
+        we may have roundtripped thru object in the mean-time
+
+        """
         if obj.ndim > 1:
             dtype = obj.values.dtype
         else:


Test patch:
diff --git a/pandas/tests/test_groupby.py b/pandas/tests/test_groupby.py
--- a/pandas/tests/test_groupby.py
+++ b/pandas/tests/test_groupby.py
@@ -2560,6 +2560,57 @@ def test_groupby_with_empty(self):
         grouped = series.groupby(grouper)
         assert next(iter(grouped), None) is None
 
+    def test_cumcount(self):
+        df = DataFrame([['a'], ['a'], ['a'], ['b'], ['a']], columns=['A'])
+        g = df.groupby('A')
+        sg = g.A
+
+        expected = Series([0, 1, 2, 0, 3])
+
+        assert_series_equal(expected, g.cumcount())
+        assert_series_equal(expected, sg.cumcount())
+
+    def test_cumcount_empty(self):
+        ge = DataFrame().groupby()
+        se = Series().groupby()
+
+        e = Series(dtype='int')  # edge case, as this is usually considered float
+
+        assert_series_equal(e, ge.cumcount())
+        assert_series_equal(e, se.cumcount())
+
+    def test_cumcount_dupe_index(self):
+        df = DataFrame([['a'], ['a'], ['a'], ['b'], ['a']], columns=['A'], index=[0] * 5)
+        g = df.groupby('A')
+        sg = g.A
+
+        expected = Series([0, 1, 2, 0, 3], index=[0] * 5)
+
+        assert_series_equal(expected, g.cumcount())
+        assert_series_equal(expected, sg.cumcount())
+
+    def test_cumcount_mi(self):
+        mi = MultiIndex.from_tuples([[0, 1], [1, 2], [2, 2], [2, 2], [1, 0]])
+        df = DataFrame([['a'], ['a'], ['a'], ['b'], ['a']], columns=['A'], index=mi)
+        g = df.groupby('A')
+        sg = g.A
+
+        expected = Series([0, 1, 2, 0, 3], index=mi)
+
+        assert_series_equal(expected, g.cumcount())
+        assert_series_equal(expected, sg.cumcount())        
+
+    def test_cumcount_groupby_not_col(self):
+        df = DataFrame([['a'], ['a'], ['a'], ['b'], ['a']], columns=['A'], index=[0] * 5)
+        g = df.groupby([0, 0, 0, 1, 0])
+        sg = g.A
+
+        expected = Series([0, 1, 2, 0, 3], index=[0] * 5)
+
+        assert_series_equal(expected, g.cumcount())
+        assert_series_equal(expected, sg.cumcount())
+
+
     def test_filter_series(self):
         import pandas as pd
         s = pd.Series([1, 3, 20, 5, 22, 24, 7])
@@ -3180,7 +3231,7 @@ def test_tab_completion(self):
             'min','name','ngroups','nth','ohlc','plot', 'prod',
             'size','std','sum','transform','var', 'count', 'head', 'describe',
             'cummax', 'dtype', 'quantile', 'rank', 'cumprod', 'tail',
-            'resample', 'cummin', 'fillna', 'cumsum'])
+            'resample', 'cummin', 'fillna', 'cumsum', 'cumcount'])
         self.assertEqual(results, expected)
 
 def assert_fp_equal(a, b):