Repository: unionai-oss/pandera

Issue:
Coercing DateTimeIndex fails starting with Pandera v0.13.4
- [X] I have checked that this issue has not already been reported.
- [X] I have confirmed this bug exists on the latest version of pandera.
- [] (optional) I have confirmed this bug exists on the master branch of pandera.

**Describe the bug**

When passed the following date time index to [DateTime._coerce(...)'s inline function _to_datetime](https://github.com/unionai-oss/pandera/blob/5aa77957615720741dd817b668154e9a72cf6c73/pandera/engines/pandas_engine.py#L816)
```
col = DatetimeIndex(['2021-10-24 00:00:00+00:00', '2021-10-25 00:00:00+00:00',
               '2021-10-26 00:00:00+00:00', '... 00:00:00+00:00', '2021-10-31 00:00:00+00:00'],
              dtype='datetime64[ns, UTC]', name='timestamp', freq=None)

```

with schema instance built from 

```python
class PriceDFSchema(pandera.SchemaModel):
    timestamp: Index[pd.DatetimeTZDtype] = Field(
        coerce=True, dtype_kwargs={"unit": "ns", "tz": "UTC"}
    )
    product_id: Series[pd.Int64Dtype] = Field(coerce=True)
    ...
```

and **starting with** #972, Pandera fails to coerce <class 'pandas.core.indexes.datetimes.DatetimeIndex'> data_container into type datetime64[ns, UTC] due to missing `dt` called here:

https://github.com/unionai-oss/pandera/blob/5aa77957615720741dd817b668154e9a72cf6c73/pandera/engines/pandas_engine.py#L816

**Pandera/Pandas/Bug Status matrix:**

Pandera 0.12.0 / Pandas 1.4.4-1.5.2 => coercion success
Pandera 0.13.4 / Pandas 1.4.4-1.5.2 => coercion failure

The difference seems to be that the coercion happened on a series on #972 while in the code sample above it is on an index.

#### Workaround 

Use Pandera 0.12.0

#### Code Sample, a copy-pastable example

See above.

#### Expected behavior
Coercing succeeds

#### Desktop (please complete the following information):

 - OS: macOS 15.6
 - Version Pandera: 0.13.4
 - Version Pandas: 1.5.2
 - Python 3.9.x

#### Screenshots
NA
#### Additional context
NA


Discussion:
> The difference seems to be that the coercion happened on a series on https://github.com/unionai-oss/pandera/pull/972 while in the code sample above it is on an index.

@cristianmatache As #972 submitter, and in light with the text above, any idea why Index (vs Series) seems to have no 'dt' field ?
@davidandreoletti it looks like i haven't handled that case properly, only Series' have `.dt`, indexes don't.

We should add an if statement to check whether `hasattr(col, 'dt')`, I think something like this should do
```python
            if (
                hasattr(pandas_dtype, "tz")
                and pandas_dtype.tz is not None
            ):
                # localize datetime column so that it's timezone-aware
                if hasattr(col, 'dt') and col.dt.tz is None:  # Series
                    col = col.dt.tz_localize(pandas_dtype.tz, **_tz_localize_kwargs)
                elif col.tz is None:  # Index
                   col = col.tz_localize(pandas_dtype.tz, **_tz_localize_kwargs)
            return col.astype(pandas_dtype)
```
I can have a look at this early next week, if you need the functionality before that, please feel free to submit a PR.

Resolving patch:
diff --git a/pandera/engines/pandas_engine.py b/pandera/engines/pandas_engine.py
--- a/pandera/engines/pandas_engine.py
+++ b/pandera/engines/pandas_engine.py
@@ -808,16 +808,23 @@ def _coerce(
 
         def _to_datetime(col: PandasObject) -> PandasObject:
             col = to_datetime_fn(col, **self.to_datetime_kwargs)
-            if (
-                hasattr(pandas_dtype, "tz")
-                and pandas_dtype.tz is not None
-                and col.dt.tz is None
-            ):
-                # localize datetime column so that it's timezone-aware
-                col = col.dt.tz_localize(
-                    pandas_dtype.tz,
-                    **_tz_localize_kwargs,
-                )
+            if hasattr(pandas_dtype, "tz") and pandas_dtype.tz is not None:
+                if hasattr(col, "dt") and col.dt.tz is None:
+                    # localize datetime column so that it's timezone-aware
+                    col = col.dt.tz_localize(
+                        pandas_dtype.tz,
+                        **_tz_localize_kwargs,
+                    )
+                elif (
+                    hasattr(col, "tz")
+                    and col.tz != pandas_dtype.tz
+                    and hasattr(col, "tz_localize")
+                ):
+                    # localize datetime index sso that it's timezone-aware
+                    col = col.tz_localize(
+                        pandas_dtype.tz,
+                        **_tz_localize_kwargs,
+                    )
             return col.astype(pandas_dtype)
 
         if isinstance(data_container, pd.DataFrame):


Test patch:
diff --git a/tests/core/test_dtypes.py b/tests/core/test_dtypes.py
--- a/tests/core/test_dtypes.py
+++ b/tests/core/test_dtypes.py
@@ -15,7 +15,7 @@
 from _pytest.mark.structures import ParameterSet
 from _pytest.python import Metafunc
 from hypothesis import strategies as st
-from pandas import DatetimeTZDtype
+from pandas import DatetimeTZDtype, to_datetime
 
 import pandera as pa
 from pandera.engines import pandas_engine
@@ -419,6 +419,49 @@ def test_try_coerce(examples, type_, failure_indices):
         assert exc.failure_cases["index"].to_list() == failure_indices
 
 
+@pytest.mark.parametrize(
+    "examples, type_, has_tz",
+    [
+        (
+            ["2022-04-30T00:00:00Z", "2022-04-30T00:00:01Z"],
+            DatetimeTZDtype(tz="UTC"),
+            True,
+        ),
+        (
+            ["2022-04-30T00:00:00", "2022-04-30T00:00:01"],
+            DatetimeTZDtype(tz="UTC"),
+            True,
+        ),
+        (
+            ["2022-04-30T00:00:00Z", "2022-04-30T00:00:01Z"],
+            pandas_engine.DateTime,
+            False,
+        ),
+        (
+            ["2022-04-30T00:00:00", "2022-04-30T00:00:01"],
+            pandas_engine.DateTime,
+            False,
+        ),
+    ],
+)
+@pytest.mark.parametrize("is_index", [True, False])
+def test_coerce_dt(examples, type_, has_tz, is_index):
+    """Test coercion of Series and Indexes to DateTime with and without Time Zones."""
+    data = pd.Index(examples) if is_index else pd.Series(examples)
+    data_type = pandas_engine.Engine.dtype(type_)
+    if has_tz:
+        expected = [
+            to_datetime("2022-04-30 00:00:00", utc=True),
+            to_datetime("2022-04-30 00:00:01", utc=True),
+        ]
+    else:
+        expected = [
+            to_datetime("2022-04-30 00:00:00"),
+            to_datetime("2022-04-30 00:00:01"),
+        ]
+    assert data_type.try_coerce(data).to_list() == expected
+
+
 def test_coerce_string():
     """Test that strings can be coerced."""
     data = pd.Series([1, None], dtype="Int32")