Repository: pandas-dev/pandas

Issue:
to_datetime should support ISO week year
`to_datetime` does not currently seem to support `ISO week year` like `strptime` does:

```
In [38]: datetime.date(2016, 1, 1).strftime('%G-%V')
Out[38]: '2015-53'

In [39]: datetime.datetime.strptime(datetime.date(2016, 1, 1).strftime('%G-%V')+'-1', '%G-%V-%u')
Out[39]: datetime.datetime(2015, 12, 28, 0, 0)

In [41]: pd.to_datetime(datetime.date(2016, 1, 1).strftime('%G-%V')+'-1', format='%G-%V-%u')
        ---------------------------------------------------------------------------
        TypeError                                 Traceback (most recent call last)
        /Users/Robin/.pyenv/versions/3.6.1/lib/python3.6/site-packages/pandas/core/tools/datetimes.py in _convert_listlike(arg, box, format, name, tz)
            443             try:
        --> 444                 values, tz = tslib.datetime_to_datetime64(arg)
            445                 return DatetimeIndex._simple_new(values, name=name, tz=tz)

        pandas/_libs/tslib.pyx in pandas._libs.tslib.datetime_to_datetime64 (pandas/_libs/tslib.c:33275)()

        TypeError: Unrecognized value type: <class 'str'>

        During handling of the above exception, another exception occurred:

        ValueError                                Traceback (most recent call last)
        <ipython-input-41-7ce30c959690> in <module>()
        ----> 1 pd.to_datetime(datetime.date(2016, 1, 1).strftime('%G-%V')+'-1', format='%G-%V-%u')

        /Users/Robin/.pyenv/versions/3.6.1/lib/python3.6/site-packages/pandas/core/tools/datetimes.py in to_datetime(arg, errors, dayfirst, yearfirst, utc, box, format, exact, unit, infer_datetime_format, origin)
            516         result = _convert_listlike(arg, box, format)
            517     else:
        --> 518         result = _convert_listlike(np.array([arg]), box, format)[0]
            519 
            520     return result

        /Users/Robin/.pyenv/versions/3.6.1/lib/python3.6/site-packages/pandas/core/tools/datetimes.py in _convert_listlike(arg, box, format, name, tz)
            445                 return DatetimeIndex._simple_new(values, name=name, tz=tz)
            446             except (ValueError, TypeError):
        --> 447                 raise e
            448 
            449     if arg is None:

        /Users/Robin/.pyenv/versions/3.6.1/lib/python3.6/site-packages/pandas/core/tools/datetimes.py in _convert_listlike(arg, box, format, name, tz)
            412                     try:
            413                         result = tslib.array_strptime(arg, format, exact=exact,
        --> 414                                                       errors=errors)
            415                     except tslib.OutOfBoundsDatetime:
            416                         if errors == 'raise':

        pandas/_libs/tslib.pyx in pandas._libs.tslib.array_strptime (pandas/_libs/tslib.c:63124)()

        pandas/_libs/tslib.pyx in pandas._libs.tslib.array_strptime (pandas/_libs/tslib.c:63003)()

        ValueError: 'G' is a bad directive in format '%G-%V-%u'

```

<details>
INSTALLED VERSIONS
------------------
commit: None

pandas: 0.20.1
pytest: 3.1.0
pip: 9.0.1
setuptools: 28.8.0
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.19.0
xarray: None
IPython: 6.0.0
sphinx: None
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 0.999999999
sqlalchemy: 1.1.10
pymysql: None
psycopg2: 2.7.1 (dt dec pq3 ext lo64)
jinja2: 2.9.6
s3fs: None
pandas_gbq: None
pandas_datareader: None
</details>



Discussion:
sure could be added.

pull-requests are welcome.
@jreback can you help me start working on this pull request? Like where to look to fix it?
The relevant part should be in [`array_strptime`](https://github.com/pandas-dev/pandas/blob/ce4eef3750052cec62ca0fe6536521dec523cd64/pandas/_libs/tslib.pyx#L3585) in `/_libs/tslib.pyx`. 

A good ref impl is `datetime`s `strptime`: https://github.com/python/cpython/blob/6f0eb93183519024cb360162bdd81b9faec97ba6/Lib/_strptime.py#L321
@buyology  Could I be assigned this task so that I can work on its pull request?
@rosygupta feel free to work on this :-) 
@buyology This will be my first attempt to make a PR here. Do you suggest if this would be the right task to take on ?
@rosygupta it involves some `cython`, but should be pretty straightforward as long as you get the environment up and running properly. otherwise, if you want something more lightweight, go and look for `novice`-labeled issues :rocket:
@buyology Seems achievable. Where do I look for the tests for this particular piece to check my code?
@rosygupta this is a new feature request, so there aren't existing tests for it. Similar tests to what you would need to add are in https://github.com/pandas-dev/pandas/blob/73930c58e8eac4031608bb8c4bf624d77e1d1dcb/pandas/tests/indexes/datetimes/test_tools.py
Hey, I'm not sure why tests of all classes are not being executed when testing. Only 6 classes being executed. Can someone clear out?
@buyology @TomAugspurger Pulled a PR. Need some guidance.
Hey I'm curious what's happening with this issue. I took a look at the referenced PR @rosygupta made and it looks like the PR fixes the issue, just needs to be rebased / fix any merge conflicts now? Any update into looking to get it merged in?
> just needs to be rebased / fix any merge conflicts now? 

you are welcome to do this
Sure I can tackle this. 
So the datetime code has been moved out of `_libs/tslib.pyx` making a rebase hard to do. Is it better to move her changes out and reapply them manually?

Resolving patch:
diff --git a/pandas/core/tools/datetimes.py b/pandas/core/tools/datetimes.py
--- a/pandas/core/tools/datetimes.py
+++ b/pandas/core/tools/datetimes.py
@@ -455,6 +455,8 @@ def to_datetime(arg, errors='raise', dayfirst=False, yearfirst=False,
     format : string, default None
         strftime to parse time, eg "%d/%m/%Y", note that "%f" will parse
         all the way up to nanoseconds.
+        See strftime documentation for more information on choices:
+        https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior
     exact : boolean, True by default
 
         - If True, require an exact format match.


Test patch:
diff --git a/pandas/tests/indexes/datetimes/test_tools.py b/pandas/tests/indexes/datetimes/test_tools.py
--- a/pandas/tests/indexes/datetimes/test_tools.py
+++ b/pandas/tests/indexes/datetimes/test_tools.py
@@ -244,6 +244,63 @@ def test_to_datetime_parse_timezone_keeps_name(self):
 
 
 class TestToDatetime(object):
+    @pytest.mark.parametrize("s, _format, dt", [
+        ['2015-1-1', '%G-%V-%u', datetime(2014, 12, 29, 0, 0)],
+        ['2015-1-4', '%G-%V-%u', datetime(2015, 1, 1, 0, 0)],
+        ['2015-1-7', '%G-%V-%u', datetime(2015, 1, 4, 0, 0)]
+    ])
+    def test_to_datetime_iso_week_year_format(self, s, _format, dt):
+        # See GH#16607
+        assert to_datetime(s, format=_format) == dt
+
+    @pytest.mark.parametrize("msg, s, _format", [
+        ["ISO week directive '%V' must be used with the ISO year directive "
+         "'%G' and a weekday directive '%A', '%a', '%w', or '%u'.", "1999 50",
+         "%Y %V"],
+        ["ISO year directive '%G' must be used with the ISO week directive "
+         "'%V' and a weekday directive '%A', '%a', '%w', or '%u'.", "1999 51",
+         "%G %V"],
+        ["ISO year directive '%G' must be used with the ISO week directive "
+         "'%V' and a weekday directive '%A', '%a', '%w', or '%u'.", "1999 "
+         "Monday", "%G %A"],
+        ["ISO year directive '%G' must be used with the ISO week directive "
+         "'%V' and a weekday directive '%A', '%a', '%w', or '%u'.", "1999 Mon",
+         "%G %a"],
+        ["ISO year directive '%G' must be used with the ISO week directive "
+         "'%V' and a weekday directive '%A', '%a', '%w', or '%u'.", "1999 6",
+         "%G %w"],
+        ["ISO year directive '%G' must be used with the ISO week directive "
+         "'%V' and a weekday directive '%A', '%a', '%w', or '%u'.", "1999 6",
+         "%G %u"],
+        ["ISO year directive '%G' must be used with the ISO week directive "
+         "'%V' and a weekday directive '%A', '%a', '%w', or '%u'.", "2051",
+         "%G"],
+        ["Day of the year directive '%j' is not compatible with ISO year "
+         "directive '%G'. Use '%Y' instead.", "1999 51 6 256", "%G %V %u %j"],
+        ["ISO week directive '%V' is incompatible with the year directive "
+         "'%Y'. Use the ISO year '%G' instead.", "1999 51 Sunday", "%Y %V %A"],
+        ["ISO week directive '%V' is incompatible with the year directive "
+         "'%Y'. Use the ISO year '%G' instead.", "1999 51 Sun", "%Y %V %a"],
+        ["ISO week directive '%V' is incompatible with the year directive "
+         "'%Y'. Use the ISO year '%G' instead.", "1999 51 1", "%Y %V %w"],
+        ["ISO week directive '%V' is incompatible with the year directive "
+         "'%Y'. Use the ISO year '%G' instead.", "1999 51 1", "%Y %V %u"],
+        ["ISO week directive '%V' must be used with the ISO year directive "
+         "'%G' and a weekday directive '%A', '%a', '%w', or '%u'.", "20", "%V"]
+    ])
+    def test_error_iso_week_year(self, msg, s, _format):
+        # See GH#16607
+        # This test checks for errors thrown when giving the wrong format
+        # However, as discussed on PR#25541, overriding the locale
+        # causes a different error to be thrown due to the format being
+        # locale specific, but the test data is in english.
+        # Therefore, the tests only run when locale is not overwritten,
+        # as a sort of solution to this problem.
+        if (locale.getlocale() != ('zh_CN', 'UTF-8') and
+           locale.getlocale() != ('it_IT', 'UTF-8')):
+            with pytest.raises(ValueError, match=msg):
+                to_datetime(s, format=_format)
+
     @pytest.mark.parametrize('tz', [None, 'US/Central'])
     def test_to_datetime_dtarr(self, tz):
         # DatetimeArray