Add `strict_na` keyword to the assert_.._equal methods for object dtype to help with deprecation
State: open

Adding a `strict_na=False` option to our assert functions (the default of `True` keeps the new behaviour enforced on main). This keyword is _only_ relevant for object dtype (or when using that code path when specifying `check_dtype=False` and having different dtypes).

This was brought up in the original PR, see https://github.com/pandas-dev/pandas/pull/52081#issuecomment-1620648937, and also discussed in the original issue (https://github.com/pandas-dev/pandas/issues/18463)

This obviously needs some more docstring and test updates, but first wanted to see how this is received


- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Stale

Comments:
Is the idea to keep this forever or just a lengthened deprecation cycle?

could be modestly be more strict by disallowing egregiously mismatched cases e.g. np.dt64(nat) vs np.td64(nat)?
>  or just a lengthened deprecation cycle?

No, it's actually just to make it easier to use, because for certain use cases the previous behaviour was a lot more convenient (so the title "help with deprecation" is a bit wrong, it's not to help with the deprecation process itself, but just to help live with the new enforced behaviour).


> could be modestly be more strict by disallowing egregiously mismatched cases e.g. np.dt64(nat) vs np.td64(nat)?

That again would then make the keyword logic and implementation more complex. So unless there is a use case for this, I personally wouldn't do that. 
(in the end, the keyword that I am adding in this PR needs to be explicitly enabled, so the user of this typically knows why they are specifying it)
What is the envisioned scope of this keyword? Does it treat None, pd.NA, pa.null and np.nan all the same? Or just a subset of those?
It will treat anything that `pd.isna(..)` considers as missing (for scalar input) as equal. 
(so not a subset, as all those scalars can occur in an object-dtype column)
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
docstring request, otherwise LGTM pending green
It would be practical I think if the `strict_na` parameter whould be backported to 2.* (possibly with `strict_na=False` just raising a `ValueError` if this make implementation easier) so it can be used in tests that target both pandas 2 and pandas 3.