Repository: pandas-dev/pandas

Issue:
GroupBy.apply type upcasting regression
Blocker for 0.11.1. This got broken sometime between 0.11.0 and master; I haven't had time to bisect yet. Essentially, write a mixed type DataFrame and a groupby function that extracts a row. This used to do type inference and convert object back to numeric in the columns of the resulting DataFrame. Didn't have a test so I don't blame anyone for breaking it by accident =)

```
In [151]: cafdata.dtypes
Out[151]:
id            int64
food         object
fgroup       object
nutrient     object
ngroup       object
units        object
value       float64
dtype: object

In [152]:
def max_value(group):
    return group.ix[group['value'].idxmax()]

max_value(cafdata)

Out[152]:
id                                      14366
food        Tea, instant, unsweetened, powder
fgroup                              Beverages
nutrient                             Caffeine
ngroup                                  Other
units                                      mg
value                                    3680
Name: 336702, dtype: object

In [153]: cafdata.groupby('fgroup').apply(max_value).dtypes
Out[153]:
id          object
food        object
fgroup      object
nutrient    object
ngroup      object
units       object
value       object
dtype: object
```



Discussion:
Actually not sure this is a regression, here's with 0.11.0:

```
In [166]:

grouped = data.groupby(['nutrient', 'fgroup'])
results = grouped.apply(max_value)
results.dtypes
Out[166]:
id          object
food        object
fgroup      object
nutrient    object
ngroup      object
units       object
value       object
dtype: object
```

for some reason making a bar plot of the value column fails in master. i will investigate when i can

is the failure a type error? plotting `object` `Series` was deprecated by https://github.com/pydata/pandas/pull/3572. should do `results.value.convert_objects().plot(kind='bar')`

Um, really? In this case the resulting plot is totally fine because all of the objects inside are floats. Why not attempt to convert_objects for dtype object and raise if it can't be converted to numeric?

I'm strongly :-1: on that change

ok then will change it now

@wesm cpcloud/pandas@fe3f01f if u want to check while waiting for travis...

@wesm also is that the only issue here? not clear to me if there's still a `groupby` issue from ur comments above

oh i c. still looks like casting back to original types is an issue


Resolving patch:
diff --git a/pandas/core/groupby.py b/pandas/core/groupby.py
--- a/pandas/core/groupby.py
+++ b/pandas/core/groupby.py
@@ -1928,7 +1928,7 @@ def _wrap_applied_output(self, keys, values, not_indexed_same=False):
                     return Series(values, index=key_index)
 
                 return DataFrame(stacked_values, index=index,
-                                 columns=columns)
+                                 columns=columns).convert_objects()
 
             else:
                 return Series(values, index=key_index)


Test patch:
diff --git a/pandas/tests/test_groupby.py b/pandas/tests/test_groupby.py
--- a/pandas/tests/test_groupby.py
+++ b/pandas/tests/test_groupby.py
@@ -261,6 +261,20 @@ def test_groupby_nonobject_dtype(self):
         expected = self.mframe.groupby(key.astype('O')).sum()
         assert_frame_equal(result, expected)
 
+        # GH 3911, mixed frame non-conversion
+        df = self.df_mixed_floats.copy()
+        df['value'] = range(len(df))
+
+        def max_value(group):
+            return group.ix[group['value'].idxmax()]
+
+        applied = df.groupby('A').apply(max_value)
+        result = applied.get_dtype_counts()
+        result.sort()
+        expected = Series({ 'object' : 2, 'float64' : 2, 'int64' : 1 })
+        expected.sort()
+        assert_series_equal(result,expected)
+
     def test_groupby_return_type(self):
 
         # GH2893, return a reduced type