Repository: pandas-dev/pandas

Issue:
Lost timezone information after groupby transform
#### Code Sample, a copy-pastable example if possible

Similar issue as #24198 in pandas 0.25.0

```python
import pandas as pd

df = pd.DataFrame({'time': [pd.Timestamp('2010-07-15 03:14:45'), 
                            pd.Timestamp('2010-11-19 18:47:06')],
                   'timezone': ['Etc/GMT+4', 'US/Eastern']})

df['time_tz'] = (df.groupby(['timezone'])['time']
                   .transform(lambda x: x.dt.tz_localize(x.name, ambiguous='NaT')))

df['time_tz_desired'] = (df.groupby(['timezone'])['time']
                   .apply(lambda x: x.dt.tz_localize(x.name, ambiguous='NaT')))
print(df)
```
#### Problem description

The timezone in the transform should be preserved but it is not. 
For the apply the datetime returned is an object type

`df.info()` shows
```
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 2 entries, 0 to 1
Data columns (total 4 columns):
time               2 non-null datetime64[ns]
timezone           2 non-null object
time_tz            2 non-null datetime64[ns]
time_tz_desired    2 non-null object
dtypes: datetime64[ns](2), object(2)
memory usage: 192.0+ bytes
```

#### Expected Output
Should preserve timezone
```python
df.groupby(['timezone'])['time'].transform(lambda x: x.dt.tz_localize(x.name, ambiguous='NaT'))
```
And should return datetime
```python
df.groupby(['timezone'])['time'].apply(lambda x: x.dt.tz_localize(x.name, ambiguous='NaT'))
```

#### Output of ``pd.show_versions()``

<details>

[paste the output of ``pd.show_versions()`` here below this line]
INSTALLED VERSIONS
------------------
commit           : None
python           : 3.7.3.final.0
python-bits      : 64
OS               : Windows
OS-release       : 10
machine          : AMD64
processor        : Intel64 Family 6 Model 78 Stepping 3, GenuineIntel
byteorder        : little
LC_ALL           : None
LANG             : None
LOCALE           : None.None

pandas           : 0.25.0
numpy            : 1.16.3
pytz             : 2019.1
dateutil         : 2.8.0
pip              : 19.1.1
setuptools       : 40.8.0
Cython           : 0.29.10
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : 4.3.4
html5lib         : 1.0.1
pymysql          : 0.9.3
psycopg2         : 2.8.2 (dt dec pq3 ext lo64)
jinja2           : 2.10.1
IPython          : 7.5.0
pandas_datareader: None
bs4              : 4.7.1
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : 4.3.4
matplotlib       : 3.0.3
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
s3fs             : None
scipy            : 1.3.0
sqlalchemy       : 1.3.3
tables           : None
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None

</details>



Discussion:
> For the apply the datetime returned is an object type

That is in general what happens when you mix timezones inside of a container:

```python
In [8]: pd.Series([pd.Timestamp('today').tz_localize('US/Eastern'), 
    pd.Timestamp('today').tz_localize('US/Central')])
Out[8]:
0    2019-07-20 18:00:52.167252-04:00
1    2019-07-20 18:00:52.167459-05:00
dtype: object
```

So *I think* the issue may just be that transform tries to infer back to datetime when it probably should just leave as object, though cc @mroeschke for thoughts

Hmm, the end result desired is to get everything into one format (utc) so doing:

```python
df['time_tz'] = (df.groupby(['timezone'])['time']
                   .transform(lambda x: x.dt.tz_localize(x.name, ambiguous='NaT').dt.tz_convert('UTC')))

df['time_tz_desired'] = (df.groupby(['timezone'])['time']
                   .apply(lambda x: x.dt.tz_localize(x.name, ambiguous='NaT').dt.tz_convert('UTC')))
```

Returns the correct values for the apply but not the transform once again.

```
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 2 entries, 0 to 1
Data columns (total 4 columns):
time               2 non-null datetime64[ns]
timezone           2 non-null object
time_tz            2 non-null datetime64[ns]
time_tz_desired    2 non-null datetime64[ns, UTC]
dtypes: datetime64[ns, UTC](1), datetime64[ns](2), object(1)
memory usage: 192.0+ bytes
```

Resolving patch:
diff --git a/pandas/core/groupby/generic.py b/pandas/core/groupby/generic.py
--- a/pandas/core/groupby/generic.py
+++ b/pandas/core/groupby/generic.py
@@ -42,7 +42,7 @@
 from pandas.core.base import DataError, SpecificationError
 import pandas.core.common as com
 from pandas.core.frame import DataFrame
-from pandas.core.generic import NDFrame, _shared_docs
+from pandas.core.generic import ABCDataFrame, ABCSeries, NDFrame, _shared_docs
 from pandas.core.groupby import base
 from pandas.core.groupby.groupby import GroupBy, _apply_docs, _transform_template
 from pandas.core.index import Index, MultiIndex
@@ -1025,8 +1025,8 @@ def transform(self, func, *args, **kwargs):
             object.__setattr__(group, "name", name)
             res = wrapper(group)
 
-            if hasattr(res, "values"):
-                res = res.values
+            if isinstance(res, (ABCDataFrame, ABCSeries)):
+                res = res._values
 
             indexer = self._get_index(name)
             s = klass(res, indexer)


Test patch:
diff --git a/pandas/tests/groupby/test_transform.py b/pandas/tests/groupby/test_transform.py
--- a/pandas/tests/groupby/test_transform.py
+++ b/pandas/tests/groupby/test_transform.py
@@ -1001,3 +1001,27 @@ def test_ffill_not_in_axis(func, key, val):
     expected = df
 
     assert_frame_equal(result, expected)
+
+
+def test_transform_lambda_with_datetimetz():
+    # GH 27496
+    df = DataFrame(
+        {
+            "time": [
+                Timestamp("2010-07-15 03:14:45"),
+                Timestamp("2010-11-19 18:47:06"),
+            ],
+            "timezone": ["Etc/GMT+4", "US/Eastern"],
+        }
+    )
+    result = df.groupby(["timezone"])["time"].transform(
+        lambda x: x.dt.tz_localize(x.name)
+    )
+    expected = Series(
+        [
+            Timestamp("2010-07-15 03:14:45", tz="Etc/GMT+4"),
+            Timestamp("2010-11-19 18:47:06", tz="US/Eastern"),
+        ],
+        name="time",
+    )
+    assert_series_equal(result, expected)