ENH: to_datetime support iso week year (16607) Updated
State: closed

- [x] closes #16607 
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

I found the issue stagnant, but since the fix was already in place, I manually took the code @rosygupta made and applied it onto latest master. Rebase wasn't an option that I found would work since the file in question has been split into multiple. Let me know what else needs to be updated.

Question: Given the code was created from a reference of cpython https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/_strptime.py#L321 
 I'm wondering why we can't use cpython's strptime directly and have to make our own implementation?  

Labels:
name: Datetime
description: Datetime data dtype
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=h1) Report
> Merging [#24844](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/f4458c18287288562b21adece524fc1b046e9724?src=pr&el=desc) will **decrease** coverage by `49.48%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24844/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           master   #24844       +/-   ##
===========================================
- Coverage   92.39%    42.9%   -49.49%     
===========================================
  Files         166      166               
  Lines       52378    52378               
===========================================
- Hits        48393    22472    -25921     
- Misses       3985    29906    +25921
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `?` | |
| #single | `42.9% <ø> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/latex.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvbGF0ZXgucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/core/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY2F0ZWdvcmljYWwucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/sas/sas\_constants.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNfY29uc3RhbnRzLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/tseries/plotting.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvcGxvdHRpbmcucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/tseries/converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvY29udmVydGVyLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/formats/html.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvaHRtbC5weQ==) | `0% <0%> (-99.35%)` | :arrow_down: |
| [pandas/core/groupby/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS9jYXRlZ29yaWNhbC5weQ==) | `0% <0%> (-95.46%)` | :arrow_down: |
| [pandas/io/sas/sas7bdat.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXM3YmRhdC5weQ==) | `0% <0%> (-91.17%)` | :arrow_down: |
| [pandas/io/sas/sas\_xport.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNfeHBvcnQucHk=) | `0% <0%> (-90.15%)` | :arrow_down: |
| [pandas/core/tools/numeric.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvdG9vbHMvbnVtZXJpYy5weQ==) | `10.44% <0%> (-89.56%)` | :arrow_down: |
| ... and [123 more](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=footer). Last update [f4458c1...f3183f0](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=h1) Report
> Merging [#24844](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/2b16e2e6c5a298396727fc2e66a60edf1eb13bf9?src=pr&el=desc) will **decrease** coverage by `0.62%`.
> The diff coverage is `92.59%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24844/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #24844      +/-   ##
==========================================
- Coverage   92.38%   91.75%   -0.63%     
==========================================
  Files         166      173       +7     
  Lines       52398    52960     +562     
==========================================
+ Hits        48406    48595     +189     
- Misses       3992     4365     +373
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.33% <92.5%> (-0.48%)` | :arrow_down: |
| #single | `41.71% <29.35%> (-1.18%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/reshape/reshape.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9yZXNoYXBlLnB5) | `99.35% <ø> (ø)` | :arrow_up: |
| [pandas/core/dtypes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Jhc2UucHk=) | `100% <ø> (ø)` | :arrow_up: |
| [pandas/core/accessor.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYWNjZXNzb3IucHk=) | `98.79% <ø> (ø)` | :arrow_up: |
| [pandas/core/groupby/ops.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS9vcHMucHk=) | `94.15% <ø> (-2.58%)` | :arrow_down: |
| [pandas/core/reshape/tile.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS90aWxlLnB5) | `97.67% <ø> (+2.73%)` | :arrow_up: |
| [pandas/io/html.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2h0bWwucHk=) | `92.66% <ø> (ø)` | :arrow_up: |
| [pandas/core/reshape/melt.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9tZWx0LnB5) | `97.5% <ø> (ø)` | :arrow_up: |
| [pandas/core/config.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29uZmlnLnB5) | `87.04% <ø> (ø)` | :arrow_up: |
| [pandas/io/parquet.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnF1ZXQucHk=) | `84.61% <ø> (ø)` | :arrow_up: |
| [pandas/core/groupby/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS9fX2luaXRfXy5weQ==) | `100% <ø> (ø)` | :arrow_up: |
| ... and [108 more](https://codecov.io/gh/pandas-dev/pandas/pull/24844/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=footer). Last update [2b16e2e...f73952d](https://codecov.io/gh/pandas-dev/pandas/pull/24844?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Failures in travisci build for python 3.5? 
https://travis-ci.org/pandas-dev/pandas/jobs/481950579 maybe this reduces in the project codecpv down to zero? Hth
Hey so I am currently stuck on making the checks pass. From what I can understand from the CI checks, it seems on Linux builds the `test_ValueError_iso_week_year` is returning a different error message than expected. Somehow the error message is being caught in https://github.com/pandas-dev/pandas/blob/e99049a820bf6a914513f715ce7ed0b01224d826/pandas/_libs/tslibs/strptime.pyx#L157 instead of https://github.com/pandas-dev/pandas/blob/e99049a820bf6a914513f715ce7ed0b01224d826/pandas/_libs/tslibs/strptime.pyx#L282 and I'm unclear why this happens when a year ago the changes did pass that time's tests.
> a year ago the changes did pass that time's tests.

@RjLi13 : Our codebase changes relatively quickly, so a year is essentially an eternity in our world.  A lot could have changed since then. 🙂 Here's what I would do:

* Take the code for your test setup (and the particular `pytest` case that it's failing) and run the code for that test in your Python console.

* Look at the stack-trace to see how the logic is being traced.  It could be that the logic isn't even getting to your error condition (because this `strptime` error happens earlier), or it is, but erroneously passing over it, which could indicate a bug in your implementation.

If you're having issues, feel free to share the stacktrace, and we can have a look.
@gfyoung  So I have a Mac running python3.7 and the Azure pipelines show that only Linux builds seems to fail. I don't know what OS travisCI runs on but I don't have this specific environment currently on my machine
```
Python: 3.5
JOB="3.6, locale" ENV_FILE="ci/deps/travis-36-locale.yaml" PATTERN="((not slow and not network) or (single and db))" LOCALE_OVERRIDE="zh_CN.UTF-8"
```
Because of this I can't reproduce the test failures. 

Could you give me some steps on how to recreate such an env or reproduce the failures?
> Could you give me some steps on how to recreate such an env or reproduce the failures?

Generally, such failures can be reproduced locally without having to follow the exact setup for the test.  Thus, try running the code in your development environment.

If you can't reproduce, ping us again.
Here's my console output on my dev environment. I followed the instructions to run my tests from the doc (they contain 'iso_week_year')

```
(pandas-dev) Ruijings-MacBook-Pro:pandas ruijing$ pytest pandas/tests/indexes/datetimes/test_tools.py -v -k iso_week_year
======================================= test session starts ========================================
platform darwin -- Python 3.7.2, pytest-4.1.1, py-1.7.0, pluggy-0.8.0 -- /Users/ruijing/anaconda3/envs/pandas-dev/bin/python
cachedir: .pytest_cache
hypothesis profile 'ci' -> deadline=500, timeout=unlimited, suppress_health_check=[HealthCheck.too_slow], database=DirectoryBasedExampleDatabase('/Users/ruijing/pandas/.hypothesis/examples')
rootdir: /Users/ruijing/pandas, inifile: setup.cfg
plugins: xdist-1.26.0, forked-0.2, cov-2.6.1, hypothesis-4.0.1
collected 379 items / 363 deselected                                                               

pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_to_datetime_iso_week_year_format[2015-1-1-%G-%V-%u-dt0] PASSED [  6%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_to_datetime_iso_week_year_format[2015-1-4-%G-%V-%u-dt1] PASSED [ 12%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_to_datetime_iso_week_year_format[2015-1-7-%G-%V-%u-dt2] PASSED [ 18%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[ISO week directive '%V' must be used with the ISO year directive '%G' and a weekday directive '%A', '%a', '%w', or '%u'.-1999 50-%Y %V] PASSED [ 25%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[ISO year directive '%G' must be used with the ISO week directive '%V' and a weekday directive '%A', '%a', '%w', or '%u'.-1999 51-%G %V] PASSED [ 31%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[ISO year directive '%G' must be used with the ISO week directive '%V' and a weekday directive '%A', '%a', '%w', or '%u'.-1999 Monday-%G %A] PASSED [ 37%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[ISO year directive '%G' must be used with the ISO week directive '%V' and a weekday directive '%A', '%a', '%w', or '%u'.-1999 Mon-%G %a] PASSED [ 43%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[ISO year directive '%G' must be used with the ISO week directive '%V' and a weekday directive '%A', '%a', '%w', or '%u'.-1999 6-%G %w] PASSED [ 50%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[ISO year directive '%G' must be used with the ISO week directive '%V' and a weekday directive '%A', '%a', '%w', or '%u'.-1999 6-%G %u] PASSED [ 56%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[ISO year directive '%G' must be used with the ISO week directive '%V' and a weekday directive '%A', '%a', '%w', or '%u'.-2051-%G] PASSED [ 62%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[Day of the year directive '%j' is not compatible with ISO year directive '%G'. Use '%Y' instead.-1999 51 6 256-%G %V %u %j] PASSED [ 68%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[ISO week directive '%V' is incompatible with the year directive '%Y'. Use the ISO year '%G' instead.-1999 51 Sunday-%Y %V %A] PASSED [ 75%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[ISO week directive '%V' is incompatible with the year directive '%Y'. Use the ISO year '%G' instead.-1999 51 Sun-%Y %V %a] PASSED [ 81%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[ISO week directive '%V' is incompatible with the year directive '%Y'. Use the ISO year '%G' instead.-1999 51 1-%Y %V %w] PASSED [ 87%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[ISO week directive '%V' is incompatible with the year directive '%Y'. Use the ISO year '%G' instead.-1999 51 1-%Y %V %u] PASSED [ 93%]
pandas/tests/indexes/datetimes/test_tools.py::TestToDatetime::test_ValueError_iso_week_year[ISO week directive '%V' must be used with the ISO year directive '%G' and a weekday directive '%A', '%a', '%w', or '%u'.-20-%V] PASSED [100%]

============================ 16 passed, 363 deselected in 0.17 seconds =============================
(pandas-dev) Ruijings-MacBook-Pro:pandas ruijing$ 
```

In addition, I ran `pytest pandas` and `pytest pandas/tests/indexes/datetimes/test_tools.py -v` with no failures reported. Please advise next steps.
@RjLi13 : Cool!  Thanks for doing that.  Alright, so what I'm observing is that all of the relevant failures occurred on Python 3.x environments with a locale override, which are:

* `zh_CN.UTF-8`
* `it_IT.UTF-8`

Do you mind [overriding](https://github.com/pandas-dev/pandas/blob/00f5abab2150a4e9e1798d07bafc40e734941611/ci/run_tests.sh#L15-L18) your locale first, double check that the override works, and then run those same test commands again?
@gfyoung  can you explain how to override the locale? I'm not quite understanding what you linked.

Edit: From my understanding, I just run the 'ci/run_test.sh` script with -n and one of the locale options you provided?
@RjLi13 : Before you run `pytest`, you should run those three commands (lines 16 - 18 of the highlight) in your terminal in order to:

1) Change locale in your session
2) Confirm that locale was changed

Just do that and then run your original `pytest` command.
@gfyoung I'm having trouble having pandas detect my system locale ovveride. Running lines 16-17 did change my system locale, I can confirm that with the `locale` command. But running `pandas.get_option("display.encoding")` still gives me 'utf-8'
> But running pandas.get_option("display.encoding") still gives me 'utf-8'

Hmmm...I see...since it is successfully changed after you check with the `locale` command, can you try running tests anyway and see what happens?
I did run with the two new locales given on the `iso_week_year` tests added. But they all pass, so I'm not quite sure what to do now.
> so I'm not quite sure what to do now.

Hmm...that is very weird.  Let's see if updating your branch might do the trick then.  That would entail:

* Addressing all unresolved comments (there's at least one)
* Rebasing your changes on `master`
* Moving your `whatsnew` entry to `0.25.0`

Let's make all of those changes and re-trigger CI and see what happens.
Before that, I ran the full pytest pandas suite using just `pytest pandas` after overriding the locale to `zh_CN.UTF-8`. Interestingly, I do see 6 errors. The stacktrace is rather long and I see tests other than mine seem to have failed. It doesn't seem like the stacktrace that the CI test shows. This is part of the stacktrace of the failure that happened on my test.

```
============================================= FAILURES =============================================
 TestToDatetime.test_ValueError_iso_week_year[ISO year directive '%G' must be used with the ISO week directive '%V' and a weekday directive '%A', '%a', '%w', or '%u'.-1999 Monday-%G %A] 

arg = array(['1999 Monday'], dtype=object), box = True, format = '%G %A', name = None, tz = None
unit = None, errors = 'raise', infer_datetime_format = False, dayfirst = False, yearfirst = False
exact = True

    def _convert_listlike_datetimes(arg, box, format, name=None, tz=None,
                                    unit=None, errors=None,
                                    infer_datetime_format=None, dayfirst=None,
                                    yearfirst=None, exact=None):
        """
        Helper function for to_datetime. Performs the conversions of 1D listlike
        of dates
    
        Parameters
        ----------
        arg : list, tuple, ndarray, Series, Index
            date to be parced
        box : boolean
            True boxes result as an Index-like, False returns an ndarray
        name : object
            None or string for the Index name
        tz : object
            None or 'utc'
        unit : string
            None or string of the frequency of the passed data
        errors : string
            error handing behaviors from to_datetime, 'raise', 'coerce', 'ignore'
        infer_datetime_format : boolean
            inferring format behavior from to_datetime
        dayfirst : boolean
            dayfirst parsing behavior from to_datetime
        yearfirst : boolean
            yearfirst parsing behavior from to_datetime
        exact : boolean
            exact format matching behavior from to_datetime
    
        Returns
        -------
        ndarray of parsed dates
            Returns:
    
            - Index-like if box=True
            - ndarray of Timestamps if box=False
        """
        from pandas import DatetimeIndex
        from pandas.core.arrays import DatetimeArray
        from pandas.core.arrays.datetimes import (
            maybe_convert_dtype, objects_to_datetime64ns)
    
        if isinstance(arg, (list, tuple)):
            arg = np.array(arg, dtype='O')
    
        # these are shortcutable
        if is_datetime64tz_dtype(arg):
            if not isinstance(arg, (DatetimeArray, DatetimeIndex)):
                return DatetimeIndex(arg, tz=tz, name=name)
            if tz == 'utc':
                arg = arg.tz_convert(None).tz_localize(tz)
            return arg
    
        elif is_datetime64_ns_dtype(arg):
            if box and not isinstance(arg, (DatetimeArray, DatetimeIndex)):
                try:
                    return DatetimeIndex(arg, tz=tz, name=name)
                except ValueError:
                    pass
    
            return arg
    
        elif unit is not None:
            if format is not None:
                raise ValueError("cannot specify both format and unit")
            arg = getattr(arg, 'values', arg)
            result = tslib.array_with_unit_to_datetime(arg, unit,
                                                       errors=errors)
            if box:
                if errors == 'ignore':
                    from pandas import Index
                    result = Index(result, name=name)
                    # GH 23758: We may still need to localize the result with tz
                    try:
                        return result.tz_localize(tz)
                    except AttributeError:
                        return result
    
                return DatetimeIndex(result, tz=tz, name=name)
            return result
        elif getattr(arg, 'ndim', 1) > 1:
            raise TypeError('arg must be a string, datetime, list, tuple, '
                            '1-d array, or Series')
    
        # warn if passing timedelta64, raise for PeriodDtype
        # NB: this must come after unit transformation
        orig_arg = arg
        arg, _ = maybe_convert_dtype(arg, copy=False)
    
        arg = ensure_object(arg)
        require_iso8601 = False
    
        if infer_datetime_format and format is None:
            format = _guess_datetime_format_for_array(arg, dayfirst=dayfirst)
    
        if format is not None:
            # There is a special fast-path for iso8601 formatted
            # datetime strings, so in those cases don't use the inferred
            # format because this path makes process slower in this
            # special case
            format_is_iso8601 = _format_is_iso(format)
            if format_is_iso8601:
                require_iso8601 = not infer_datetime_format
                format = None
    
        tz_parsed = None
        result = None
    
        if format is not None:
            try:
                # shortcut formatting here
                if format == '%Y%m%d':
                    try:
                        # pass orig_arg as float-dtype may have been converted to
                        # datetime64[ns]
                        orig_arg = ensure_object(orig_arg)
                        result = _attempt_YYYYMMDD(orig_arg, errors=errors)
                    except (ValueError, TypeError, tslibs.OutOfBoundsDatetime):
                        raise ValueError("cannot convert the input to "
                                         "'%Y%m%d' date format")
    
                # fallback
                if result is None:
                    try:
                        result, timezones = array_strptime(
                            arg, format, exact=exact, errors=errors)
                        if '%Z' in format or '%z' in format:
                            return _return_parsed_timezone_results(
                                result, timezones, box, tz, name)
                    except tslibs.OutOfBoundsDatetime:
                        if errors == 'raise':
                            raise
                        elif errors == 'coerce':
                            result = np.empty(arg.shape, dtype='M8[ns]')
                            iresult = result.view('i8')
                            iresult.fill(tslibs.iNaT)
                        else:
                            result = arg
                    except ValueError:
                        # if format was inferred, try falling back
                        # to array_to_datetime - terminate here
                        # for specified formats
                        if not infer_datetime_format:
                            if errors == 'raise':
                                raise
                            elif errors == 'coerce':
                                result = np.empty(arg.shape, dtype='M8[ns]')
                                iresult = result.view('i8')
                                iresult.fill(tslibs.iNaT)
                            else:
                                result = arg
            except ValueError as e:
                # Fallback to try to convert datetime objects if timezone-aware
                #  datetime objects are found without passing `utc=True`
                try:
>                   values, tz = conversion.datetime_to_datetime64(arg)

pandas/core/tools/datetimes.py:291: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   raise TypeError('Unrecognized value type: %s' % type(val))
E   TypeError: Unrecognized value type: <class 'str'>

pandas/_libs/tslibs/conversion.pyx:200: TypeError

During handling of the above exception, another exception occurred:

self = <pandas.tests.indexes.datetimes.test_tools.TestToDatetime object at 0x1c5894e8d0>
msg = "ISO year directive '%G' must be used with the ISO week directive '%V' and a weekday directive '%A', '%a', '%w', or '%u'."
s = '1999 Monday', _format = '%G %A'

    @pytest.mark.parametrize("msg, s, _format", [
        ["ISO week directive '%V' must be used with the ISO year directive "
         "'%G' and a weekday directive '%A', '%a', '%w', or '%u'.", "1999 50",
         "%Y %V"],
        ["ISO year directive '%G' must be used with the ISO week directive "
         "'%V' and a weekday directive '%A', '%a', '%w', or '%u'.", "1999 51",
         "%G %V"],
        ["ISO year directive '%G' must be used with the ISO week directive "
         "'%V' and a weekday directive '%A', '%a', '%w', or '%u'.", "1999 "
         "Monday", "%G %A"],
        ["ISO year directive '%G' must be used with the ISO week directive "
         "'%V' and a weekday directive '%A', '%a', '%w', or '%u'.", "1999 Mon",
         "%G %a"],
        ["ISO year directive '%G' must be used with the ISO week directive "
         "'%V' and a weekday directive '%A', '%a', '%w', or '%u'.", "1999 6",
         "%G %w"],
        ["ISO year directive '%G' must be used with the ISO week directive "
         "'%V' and a weekday directive '%A', '%a', '%w', or '%u'.", "1999 6",
         "%G %u"],
        ["ISO year directive '%G' must be used with the ISO week directive "
         "'%V' and a weekday directive '%A', '%a', '%w', or '%u'.", "2051",
         "%G"],
        ["Day of the year directive '%j' is not compatible with ISO year "
         "directive '%G'. Use '%Y' instead.", "1999 51 6 256", "%G %V %u %j"],
        ["ISO week directive '%V' is incompatible with the year directive "
         "'%Y'. Use the ISO year '%G' instead.", "1999 51 Sunday", "%Y %V %A"],
        ["ISO week directive '%V' is incompatible with the year directive "
         "'%Y'. Use the ISO year '%G' instead.", "1999 51 Sun", "%Y %V %a"],
        ["ISO week directive '%V' is incompatible with the year directive "
         "'%Y'. Use the ISO year '%G' instead.", "1999 51 1", "%Y %V %w"],
        ["ISO week directive '%V' is incompatible with the year directive "
         "'%Y'. Use the ISO year '%G' instead.", "1999 51 1", "%Y %V %u"],
        ["ISO week directive '%V' must be used with the ISO year directive "
         "'%G' and a weekday directive '%A', '%a', '%w', or '%u'.", "20", "%V"]
    ])
    def test_ValueError_iso_week_year(self, msg, s, _format):
        # See GH#16607
        with pytest.raises(ValueError, match=msg):
>           to_datetime(s, format=_format)

pandas/tests/indexes/datetimes/test_tools.py:297: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
```

I have a full copy of the failures stacktrace. It is 3000+ lines long. I'm not sure where to start debugging with it or should I even try posting it here.
I'll push the latest commit I have, and then let's think of what should be the next steps?

Edit: I also have `test-data-multiple.xml` and `test-data-single.xml`. Not sure if these files generated help at all. They are 20k plus lines it seem.
@RjLi13 can you merge master and address comments?
Hey, sorry for the silence. Been busy these past few weeks. Will be pushing the rebased version and addressing changes this weekend.
Rebase gone wrong, not sure how to fix. I'm closing the PR and remaking a new one which I will reference here.