Repository: pydata/xarray

Issue:
Potential regression in Dataset.from_dataframe() not preserving timezone
### What happened?

Converting pandas DataFrame that has a datetime column with timezone to an xarray dataset does not preserve the timezone, this only breaks in version 2024.5

### What did you expect to happen?

I would expect the timezone info to be preserved, as it was the case before.

### Minimal Complete Verifiable Example

```Python
import pandas as pd
import xarray as xr

df1 = pd.DataFrame(
    {"A": pd.date_range("20130101", periods=4, tz="US/Eastern"), "B": [1, 2, 3, 4]}
)
dataset = xr.Dataset.from_dataframe(df1)
df2 = dataset.to_dataframe()

print(df1.dtypes, dataset.dtypes, df2.dtypes, sep="\n\n")
```


### MVCE confirmation

- [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- [X] Complete example — the example is self-contained, including all data and the text of any traceback.
- [X] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result.
- [X] New issue — a search of GitHub Issues suggests this is not a duplicate.
- [X] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

### Relevant log output

```Python
# On xarary 2024.5.0:
A    datetime64[ns, US/Eastern]
B                         int64
dtype: object

Frozen({'A': dtype('<M8[ns]'), 'B': dtype('int64')})

A    datetime64[ns]
B             int64
dtype: object

# ---------------------------
#  On previous versions:

A    datetime64[ns, US/Eastern]
B                         int64
dtype: object

Frozen({'A': dtype('O'), 'B': dtype('int64')})

A    datetime64[ns, US/Eastern]
B                         int64
dtype: object
```


### Anything else we need to know?

_No response_

### Environment

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 3.12.1 (tags/v3.12.1:2305ca5, Dec  7 2023, 22:03:25) [MSC v.1937 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: AMD64 Family 23 Model 113 Stepping 0, AuthenticAMD
byteorder: little
LC_ALL: None
LANG: None
LOCALE: ('English_United States', '1252')
libhdf5: 1.14.2
libnetcdf: None

xarray: 2024.5.0
pandas: 2.2.2
numpy: 1.26.4
scipy: 1.12.0
netCDF4: None
pydap: None
h5netcdf: None
h5py: 3.10.0
zarr: None
cftime: None
nc_time_axis: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: 3.8.3
cartopy: None
seaborn: None
numbagg: None
fsspec: 2024.3.1
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 69.2.0
pip: 24.0
conda: None
pytest: None
mypy: None
IPython: 8.22.2
sphinx: 7.2.6

</details>



Discussion:
Thanks for opening your first issue here at xarray! Be sure to follow the issue template!
If you have an idea for a solution, we would really welcome a Pull Request with proposed changes.
See the [Contributing Guide](https://docs.xarray.dev/en/latest/contributing.html) for more.
It may take us a while to respond here, but we really value your contribution. Contributors like you help make xarray better.
Thank you!

@ilan-gold are you able to take a look here please? I suspect it's related to extension array stuff
Is `dtype('O')` from previous versions correct though?
Ah, ok, I see, previously this was an array of `TimeStamp` objects and now is being converted in a numpy array with a "proper" datatype
It's possible the previous behaviour was unintentional and this one is more "correct"/consistent ... Some exploration and reporting would be very helpful.
Ok, so the problem is that `DateTime` is an extension array `dtype`: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.api.types.is_extension_array_dtype.html

I will look into properly preserving the dtype then, although I suspect there is something else going on regarding datetimes (or the testing is not specific enough to cover this case)



Patch:
diff --git a/doc/whats-new.rst b/doc/whats-new.rst
index 96005e17f78..f0778c1e021 100644
--- a/doc/whats-new.rst
+++ b/doc/whats-new.rst
@@ -40,6 +40,9 @@ Deprecations
 
 Bug fixes
 ~~~~~~~~~
+- Preserve conversion of timezone-aware pandas Datetime arrays to numpy object arrays
+  (:issue:`9026`, :pull:`9042`).
+  By `Ilan Gold <https://github.com/ilan-gold>`_.
 
 - :py:meth:`DataArrayResample.interpolate` and :py:meth:`DatasetResample.interpolate` method now
   support aribtrary kwargs such as ``order`` for polynomial interpolation. (:issue:`8762`).
diff --git a/xarray/core/dataset.py b/xarray/core/dataset.py
index 09597670573..9b5f2262b6d 100644
--- a/xarray/core/dataset.py
+++ b/xarray/core/dataset.py
@@ -7420,7 +7420,9 @@ def from_dataframe(cls, dataframe: pd.DataFrame, sparse: bool = False) -> Self:
         arrays = []
         extension_arrays = []
         for k, v in dataframe.items():
-            if not is_extension_array_dtype(v):
+            if not is_extension_array_dtype(v) or isinstance(
+                v.array, (pd.arrays.DatetimeArray, pd.arrays.TimedeltaArray)
+            ):
                 arrays.append((k, np.asarray(v)))
             else:
                 extension_arrays.append((k, v))


Test patch:
diff --git a/properties/test_pandas_roundtrip.py b/properties/test_pandas_roundtrip.py
index 3d87fcce1d9..ca5490bcea2 100644
--- a/properties/test_pandas_roundtrip.py
+++ b/properties/test_pandas_roundtrip.py
@@ -30,6 +30,16 @@
 )
 
 
+datetime_with_tz_strategy = st.datetimes(timezones=st.timezones())
+dataframe_strategy = pdst.data_frames(
+    [
+        pdst.column("datetime_col", elements=datetime_with_tz_strategy),
+        pdst.column("other_col", elements=st.integers()),
+    ],
+    index=pdst.range_indexes(min_size=1, max_size=10),
+)
+
+
 @st.composite
 def datasets_1d_vars(draw) -> xr.Dataset:
     """Generate datasets with only 1D variables
@@ -98,3 +108,15 @@ def test_roundtrip_pandas_dataframe(df) -> None:
     roundtripped = arr.to_pandas()
     pd.testing.assert_frame_equal(df, roundtripped)
     xr.testing.assert_identical(arr, roundtripped.to_xarray())
+
+
+@given(df=dataframe_strategy)
+def test_roundtrip_pandas_dataframe_datetime(df) -> None:
+    # Need to name the indexes, otherwise Xarray names them 'dim_0', 'dim_1'.
+    df.index.name = "rows"
+    df.columns.name = "cols"
+    dataset = xr.Dataset.from_dataframe(df)
+    roundtripped = dataset.to_dataframe()
+    roundtripped.columns.name = "cols"  # why?
+    pd.testing.assert_frame_equal(df, roundtripped)
+    xr.testing.assert_identical(dataset, roundtripped.to_xarray())