Datetime parsing (PDEP-4): allow mixture of ISO formatted strings
State: closed

- [ ] closes #50411 (Replace xxxx with the GitHub issue number)
- [ ] closes #50972
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype

Comments:
It's ignored - good point, I'll make sure that's in the docstring
>> How does `format="ISO8601"` interact with `exact=True/False`?
>
> It's ignored - good point, I'll make sure that's in the docstring

Could we raise an error instead? (since the default is OK, it's only when the user explicitly passes `exact=False` that this can occur, so an error if you pass both `format="ISO8601", exact=False` seems fine?)


One question that is not directly clear to me from the doc update (without diving into the code): you can pass `format="ISO8601"` explicitly, but do we also "infer" this format? (i.e. if the inferred format is a subset of ISO8601, use the flexible one as inferred format) 

I assume not (and we probably also wouldn't want to do that, I think? Although it would be more backwards compatible), but I was wondering if we can make this clearer for the cases that would benefit from this (like `pd.to_datetime(["2022-01-01T09:00:00", "2022-01-02T09:00:00.123"])`) that work fine on pandas < 2, but error now. 
Right now, that case gives a "... doesn't match format ..." error message. _Ideally_ we could point people to this option. In theory we could check if the inferred format is ISO-like and infer the format from the non-matching string, and if that is also ISO like, append the error message. But that's maybe making it a bit too complicated not worth this corner case ;)
> but do we also "infer" this format? (i.e. if the inferred format is a subset of ISO8601, use the flexible one as inferred format)
> I assume not 

I wasn't planning on doing that, as it would strike me as value-dependent behaviour

Good point on the error message though, probably worth putting extra effort in to making sure that's as clear and actionable as possible
Thanks for your review!

1. it'll still try the ISO8601 fastpath first, so it shouldn't be a loss in performance. However, they will no longer get the validation that the elements are all ISO8601 - e.g. if they had an element that was %Y-%d-%m, then `to_datetime(..., format='mixed')` would parse it, whereas `format='ISO8601'` wouldn't
2. that's right, this doesn't affect non-string inputs
> LGTM

Thanks for your review!

@jorisvandenbossche any objections?
Does this close https://github.com/pandas-dev/pandas/issues/50972 as well?
yup, thanks!
Couple of small questions, none of them blockers.  LGTM.
Looks like a merge conflict then we can merge
> Looks like a merge conflict then we can merge

resolved, thanks for your reviews!
@MarcoGorelli merged in main once more and then I think we are good to merge. If there are any outstanding issues they can be addressed during the rc period
sounds good, thanks!
merging then so this doesn't block the RC, but if there's further requests please do let me know and I'll address them promptly
Thanks @MarcoGorelli !