ENH: to_datetime support iso week year (16607)
State: closed

- [x] closes #16607
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

I messed up the rebase of the old PR #24844 and started this again on a clean state. Reviewers please review on this PR! 

#### From old PR

I found the issue stagnant, but since the fix was already in place, I manually took the code @rosygupta made and applied it onto latest master. Rebase wasn't an option that I found would work since the file in question has been split into multiple. Let me know what else needs to be updated.

Labels:
name: Datetime
description: Datetime data dtype
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
Just some things from the previous PR. Up to date with master. Moved to 0.25 with up to date description. Added docstrings to the function calc_julian.

#### Things needed to be addressed
- Locale tests not passing on Linux on Python 3.x environments with a locale override
  - zh_CN.UTF-8
  - it_IT.UTF-8
  - I need some help figuring out where the issue lies. It seems to be triggering the wrong error message @gfyoung 
- @jreback on splitting the test_tools testcase? I didn't quite understand the conversation.
- Other comments made by @jreback 

@gfyoung @jreback @WillAyd
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=h1) Report
> Merging [#25541](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/076b5a85bcbed979c3b8df5dd825b1b2771894ff?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25541/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25541      +/-   ##
==========================================
- Coverage   91.75%   91.75%   -0.01%     
==========================================
  Files         173      173              
  Lines       52960    52960              
==========================================
- Hits        48596    48595       -1     
- Misses       4364     4365       +1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.33% <ø> (ø)` | :arrow_up: |
| #single | `41.71% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25541/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `87.57% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=footer). Last update [076b5a8...2758257](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=h1) Report
> Merging [#25541](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/65c0441a41b2dcaeebb648274d30978419a8661a?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25541/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25541      +/-   ##
==========================================
- Coverage   91.25%   91.25%   -0.01%     
==========================================
  Files         172      172              
  Lines       52977    52977              
==========================================
- Hits        48343    48342       -1     
- Misses       4634     4635       +1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.82% <ø> (ø)` | :arrow_up: |
| #single | `41.74% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/tools/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/25541/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvdG9vbHMvZGF0ZXRpbWVzLnB5) | `84.63% <ø> (ø)` | :arrow_up: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25541/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `88.98% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=footer). Last update [65c0441...eedf0e6](https://codecov.io/gh/pandas-dev/pandas/pull/25541?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@RjLi13 : With the branch rebased, are you now by any chance able to reproduce any of the test failures locally using the override methods that we discussed in the old PR?
@gfyoung Yes I get the same error as what the CI checks, using the `zh_CN.UTF-8` locale override.
Okay, that's good.  At least you can debug locally this time around.
The code was not passing the CI checks for the locale builds because the regex could not match the correct string with the pattern generated from the passed format, and it threw a different error ending the code before it got to the correct error. See this code here: https://github.com/pandas-dev/pandas/blob/947c346e860e88134f4b5032f410265f13f7a0ad/pandas/_libs/tslibs/strptime.pyx#L152-L158 
My tests for failures were not expecting to have that error thrown (see test_tools.py changes)

As for why the regex could not match the pattern, my guess is that the locale could not detect English, given the string to search for was '1999 Monday' and the regex pattern for 'zh_CN.utf8' was `re.compile('(?P<G>\\d\\d\\d\\d)\\s+(?P<A>星期一|星期二|星期三|星期四|星期五|星期六|星期日)', re.IGNORECASE)`.
As for the fix, I'm not quite sure what to do. The code works correctly, independent of the locale, because the format wanted for iso_time is %G-%V-%u, which given this snippet https://github.com/pandas-dev/pandas/blob/947c346e860e88134f4b5032f410265f13f7a0ad/pandas/_libs/tslibs/strptime.pyx#L546-L574

shows the regex is static. 

The iso code references the strptime function written for cpython [here](https://github.com/python/cpython/blob/master/Lib/_strptime.py) quite liberally.

My solution was essentially to check for the locales currently being overridden, and not run the ValueError tests for them. Alternative solutions include getting rid of checking the message matches for the ValueError, since both throw ValueErrors, or deleting the tests that use a format dependent on locale. I'm not sure how to proceed further. 

@gfyoung @jreback @WillAyd
@jreback I addressed comments and rebased my branch to latest.
RR (Review Ready)
@jreback @gfyoung @WillAyd could you continue your reviews before I rebase to latest master?
@jreback @gfyoung @WillAyd RR
thanks @RjLi13 really nice patch! keep em coming!