Repository: pandas-dev/pandas
Pull request: BUG: assert_frame_equal exception for datetime #37609
closes #37609 

- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Working with @richardwong8 on this issue.

This addresses issue #37609 where a check `needs_i8_conversion(left.dtype) or needs_i8_conversion(right.dtype)` in assert_series_equal caused an assert_extension_array on left._values and right._values when DatetimeArray or TimedeltaArray were used. This caused an exception to be raised when either the right or left was not an instance of ExtensionArray but was used with DatetimeArray or TimedeltaArray. 

This would have been covered in the earlier check `is_extension_array_dtype(left.dtype) and is_extension_array_dtype(right.dtype)`, however, DatetimeArray and TimedeltaArray are still experimental and their dtypes are not instances of ExtensionDtype subclass.

Problem statement:
BUG: pandas.testing.assert_frame_equal with check_dtype=False unexpectedly raises an exception
This code

```
import pandas
from datetime import datetime
df1 = pandas.DataFrame({'a': []}) 
df2 = pandas.DataFrame({'a': []})
df1['a'] = pandas.to_datetime(df1['a'])
pandas.testing.assert_frame_equal(df1, df2, check_dtype=False)
```

raises: 
```
AssertionError: right is not an ExtensionArray

```

But should it raise instead no exception as df1 and df2 are identical, except for their dtypes ?

Hints from issue discussion:
I think you're right, will look
Hi. I'm new and looking to help out on this project! Is this an issue that I'd be able to pick up and work on?

Commit message: add issue number in docstring

Patch to review:
diff --git a/pandas/_testing.py b/pandas/_testing.py
index bfff4301c2220..b03d9d8bbe768 100644
--- a/pandas/_testing.py
+++ b/pandas/_testing.py
@@ -1456,7 +1456,14 @@ def assert_series_equal(
             check_dtype=check_dtype,
             index_values=np.asarray(left.index),
         )
-    elif needs_i8_conversion(left.dtype) or needs_i8_conversion(right.dtype):
+    elif is_ExtensionArrayDtype_and_needs_i8_conversion(left.dtype, right.dtype):
+        assert_extension_array_equal(
+            left._values,
+            right._values,
+            check_dtype=check_dtype,
+            index_values=np.asarray(left.index),
+        )
+    elif needs_i8_conversion(left.dtype) and needs_i8_conversion(right.dtype):
         # DatetimeArray or TimedeltaArray
         assert_extension_array_equal(
             left._values,
@@ -1866,6 +1873,18 @@ def assert_copy(iter1, iter2, **eql_kwargs):
         assert elem1 is not elem2, msg
 
 
+def is_ExtensionArrayDtype_and_needs_i8_conversion(left_dtype, right_dtype):
+    """
+    Checks that we have the combination of an ExtensionArraydtype and
+    a dtype that should be converted to int64
+
+    Related to issue #37609
+    """
+    return (
+        is_extension_array_dtype(left_dtype) and needs_i8_conversion(right_dtype)
+    ) or (is_extension_array_dtype(right_dtype) and needs_i8_conversion(left_dtype))
+
+
 def getCols(k):
     return string.ascii_uppercase[:k]
 
diff --git a/pandas/tests/util/test_assert_frame_equal.py b/pandas/tests/util/test_assert_frame_equal.py
index 40d2763a13489..8034ace479a62 100644
--- a/pandas/tests/util/test_assert_frame_equal.py
+++ b/pandas/tests/util/test_assert_frame_equal.py
@@ -272,6 +272,20 @@ def test_assert_frame_equal_ignore_extension_dtype_mismatch(right_dtype):
     tm.assert_frame_equal(left, right, check_dtype=False)
 
 
+@pytest.mark.parametrize(
+    "dtype",
+    [
+        ("timedelta64[ns]"),
+        ("datetime64[ns, UTC]"),
+        ("Period[D]"),
+    ],
+)
+def test_assert_frame_equal_datetime_like_dtype_mismatch(dtype):
+    df1 = DataFrame({"a": []}, dtype=dtype)
+    df2 = DataFrame({"a": []})
+    tm.assert_frame_equal(df1, df2, check_dtype=False)
+
+
 def test_allows_duplicate_labels():
     left = DataFrame()
     right = DataFrame().set_flags(allows_duplicate_labels=False)


Code review comments:
diff_hunk: @@ -1866,6 +1873,18 @@ def assert_copy(iter1, iter2, **eql_kwargs):
         assert elem1 is not elem2, msg
 
 
+def is_ExtensionArrayDtype_and_needs_i8_conversion(left_dtype, right_dtype):
path: pandas/_testing.py
text: @user1:
can you rename to 

is_extension_array_dtype_and_needs_i8_conversion (e.g. all lowercase)

ideally type left_dtype, right_dtype (I think we type these in other places)

@user1:
and type the return type as bool

diff_hunk: @@ -1866,6 +1873,18 @@ def assert_copy(iter1, iter2, **eql_kwargs):
         assert elem1 is not elem2, msg
 
 
+def is_ExtensionArrayDtype_and_needs_i8_conversion(left_dtype, right_dtype):
+    """
+    Checks that we have the combination of an ExtensionArraydtype and
path: pandas/_testing.py
text: @user1:
i would *just* check a single dtype and do the or condition in on L1459