Repository: pandas-dev/pandas

Issue:
Performance decrease of groupby.first for datetime64 in 0.14
I have seen a dramatic (around 100x) slowdown of `groupby.first` method for `datetime64` type after updating to 0.14.

Here is an example:

``` python
from time import time
d = pd.DataFrame({'a': pd.date_range('1/1/2011', periods=100000, freq='s'), 'b': range(100000)})
t = time()
d.groupby('b').first()
print(time() - t)
```

Result:

```
3.87164497375
```

Same for `int` type

``` python
from time import time
d = pd.DataFrame({'a': range(100000), 'b': range(100000)})
t = time()
d.groupby('b').first()
print(time() - t)
```

Result:

```
0.0148799419403
```



Discussion:
what version were you using? 

This is from 0.14.0, but I have experienced the same on master yesterday. 
UPDATE: tested again on current master. Still the same.

I see that

did u have a version where it was fast?

Latest 0.13 was definitely fine. I will check it again just to make sure.

UPDATE: Just tested 0.13.1 and it takes even longer. Maybe the bug is on the numpy side? I will try to go deeper into that and post the findings here.


Resolving patch:
diff --git a/pandas/core/groupby.py b/pandas/core/groupby.py
--- a/pandas/core/groupby.py
+++ b/pandas/core/groupby.py
@@ -2332,17 +2332,16 @@ def _cython_agg_blocks(self, how, numeric_only=True):
             data = data.get_numeric_data(copy=False)
 
         for block in data.blocks:
-            values = block.values
 
-            is_numeric = is_numeric_dtype(values.dtype)
+            values = block._try_operate(block.values)
 
-            if is_numeric:
+            if block.is_numeric:
                 values = com.ensure_float(values)
 
             result, _ = self.grouper.aggregate(values, how, axis=agg_axis)
 
             # see if we can cast the block back to the original dtype
-            result = block._try_cast_result(result)
+            result = block._try_coerce_and_cast_result(result)
 
             newb = make_block(result, placement=block.mgr_locs)
             new_blocks.append(newb)
diff --git a/pandas/core/internals.py b/pandas/core/internals.py
--- a/pandas/core/internals.py
+++ b/pandas/core/internals.py
@@ -412,6 +412,11 @@ def _try_coerce_result(self, result):
         """ reverse of try_coerce_args """
         return result
 
+    def _try_coerce_and_cast_result(self, result, dtype=None):
+        result = self._try_coerce_result(result)
+        result = self._try_cast_result(result, dtype=dtype)
+        return result
+
     def _try_fill(self, value):
         return value
 
@@ -513,8 +518,7 @@ def setitem(self, indexer, value):
                 dtype, _ = _infer_dtype_from_scalar(value)
             else:
                 dtype = 'infer'
-            values = self._try_coerce_result(values)
-            values = self._try_cast_result(values, dtype)
+            values = self._try_coerce_and_cast_result(values, dtype)
             return [make_block(transf(values),
                                ndim=self.ndim, placement=self.mgr_locs,
                                fastpath=True)]
diff --git a/vb_suite/groupby.py b/vb_suite/groupby.py
--- a/vb_suite/groupby.py
+++ b/vb_suite/groupby.py
@@ -244,6 +244,13 @@ def f():
 groupby_last_float32 = Benchmark('data2.groupby(labels).last()', setup,
                                  start_date=datetime(2013, 1, 1))
 
+# with datetimes (GH7555)
+setup = common_setup + """
+df = DataFrame({'a' : date_range('1/1/2011',periods=100000,freq='s'),'b' : range(100000)})
+"""
+
+groupby_mixed_first = Benchmark('df.groupby("b").first()', setup,
+                                 start_date=datetime(2013, 5, 1))
 
 #----------------------------------------------------------------------
 # groupby_indices replacement, chop up Series


Test patch: