Repository: pandas-dev/pandas
Pull request: ERR: Raise ValueError when week is passed in to_datetime format witho…
…ut day or year (#16774)

- [x] closes #16774
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Problem statement:
to_datetime parses dates incorrectly when the format includes '%W' and does not include day of week plus calendar year
#### Code Sample, a copy-pastable example if possible

```python
import pandas as pd

# I'm not sure how a bare week should be interpreted.  But the result should
# probably still be of the same week number
print(pd.to_datetime('20', format='%W').strftime('%W'))  # 01

# Parsing is broken even when the week is fully specified
print(pd.to_datetime('2017-20', format='%Y-%W').strftime('%Y-%W'))  # 2017-00
```
#### Problem description

to_datetime parses a week XX as 0 for any valid XX.  Interestingly, an impossible XX (ex: '70') throws a ValueError as expected.

#### Expected Output
20
2017-20

#### Output of ``pd.show_versions()``

<details>
INSTALLED VERSIONS
------------------
commit: None
python: 3.4.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-79-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.20.2
pytest: None
pip: 9.0.1
setuptools: 36.0.1
Cython: 0.24.1
numpy: 1.13.0
scipy: 0.18.1
xarray: None
IPython: 6.1.0
sphinx: None
patsy: None
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.0.0
openpyxl: 2.4.0
xlrd: 1.0.0
xlwt: None
xlsxwriter: None
lxml: 3.3.3
bs4: 4.2.1
html5lib: 0.999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
pandas_gbq: None
pandas_datareader: None
</details>

Hints from issue discussion:
In the docs for [datetime.datetime.strptime](https://docs.python.org/3.6/library/datetime.html#datetime.datetime.strptime):

> When used with the strptime() method, %U and %W are only used in calculations when the day of the week and the calendar year (%Y) are specified.

Turns out this is true with to_datetime, too.  It'd also be a totally-valid resolution to this issue to include this note in the to_datetime docstring.
When specifying the day of the week, it indeed seems to work (my locale is set to nl_BE here):

```
In [83]: pd.to_datetime('2017-20 Zondag', format='%Y-%W %A')
Out[83]: Timestamp('2017-05-21 00:00:00')
```
I should have phrased the bug more clearly: "to_datetime /only/ parses %W when both day and year are also specified."

Ideal behavior would be for a timestamp to be generated for, say, the beginning of the week (as a timestamp is analagously generated for the beginning of a day when no time is specified).  Alternate behavior would be for to_datetime to refuse (like datetime.datetime.strptime) to parse such dates but to document this behavior.
I would raise on this if ``%W`` is passed but not also day and year.

note #16661/ #16607 as well which does this type of checking (for ISO week)

Commit message: ERR: Raise ValueError when week is passed in to_datetime format without day or year (#16774)

Patch to review:
diff --git a/doc/source/whatsnew/v0.21.0.txt b/doc/source/whatsnew/v0.21.0.txt
index 1e9c402dac73e..033e428bcbbb0 100644
--- a/doc/source/whatsnew/v0.21.0.txt
+++ b/doc/source/whatsnew/v0.21.0.txt
@@ -701,6 +701,7 @@ Other API Changes
 - :func:`Series.argmin` and :func:`Series.argmax` will now raise a ``TypeError`` when used with ``object`` dtypes, instead of a ``ValueError`` (:issue:`13595`)
 - :class:`Period` is now immutable, and will now raise an ``AttributeError`` when a user tries to assign a new value to the ``ordinal`` or ``freq`` attributes (:issue:`17116`).
 - :func:`to_datetime` when passed a tz-aware ``origin=`` kwarg will now raise a more informative ``ValueError`` rather than a ``TypeError`` (:issue:`16842`)
+- :func:`to_datetime` now raises a ``ValueError`` when format includes ``%W`` or ``%U`` without also including day of the week and calendar year (:issue:`16774`)
 - Renamed non-functional ``index`` to ``index_col`` in :func:`read_stata` to improve API consistency (:issue:`16342`)
 - Bug in :func:`DataFrame.drop` caused boolean labels ``False`` and ``True`` to be treated as labels 0 and 1 respectively when dropping indices from a numeric index. This will now raise a ValueError (:issue:`16877`)
 - Restricted DateOffset keyword arguments.  Previously, ``DateOffset`` subclasses allowed arbitrary keyword arguments which could lead to unexpected behavior.  Now, only valid arguments will be accepted. (:issue:`17176`).
diff --git a/pandas/core/tools/datetimes.py b/pandas/core/tools/datetimes.py
index e335dfe3a4142..f4e68dc1bcc27 100644
--- a/pandas/core/tools/datetimes.py
+++ b/pandas/core/tools/datetimes.py
@@ -254,6 +254,16 @@ def _convert_listlike(arg, box, format, name=None, tz=tz):
                 require_iso8601 = not infer_datetime_format
                 format = None
 
+        if format is not None:
+            if '%W' in format or '%U' in format:
+                if '%Y' not in format and '%y' not in format:
+                    raise ValueError("Cannot use '%W' or '%U' without "
+                                     "day and year")
+                if ('%A' not in format and '%a' not in format and '%w' not
+                        in format):
+                    raise ValueError("Cannot use '%W' or '%U' without "
+                                     "day and year")
+
         try:
             result = None
 
diff --git a/pandas/tests/indexes/datetimes/test_tools.py b/pandas/tests/indexes/datetimes/test_tools.py
index b8ce1f0af6ea8..1157d13f91e82 100644
--- a/pandas/tests/indexes/datetimes/test_tools.py
+++ b/pandas/tests/indexes/datetimes/test_tools.py
@@ -372,6 +372,19 @@ def test_datetime_invalid_datatype(self):
         with pytest.raises(TypeError):
             pd.to_datetime(pd.to_datetime)
 
+    @pytest.mark.parametrize('date, format',
+                             [('2017-20', '%Y-%W'),
+                              ('20 Sunday', '%W %A'),
+                              ('20 Sun', '%W %a'),
+                              ('2017-21', '%Y-%U'),
+                              ('20 Sunday', '%U %A'),
+                              ('20 Sun', '%U %a')])
+    def test_week_without_day_and_calendar_year(self, date, format):
+        # GH16774
+
+        with pytest.raises(ValueError):
+            pd.to_datetime(date, format=format)
+
 
 class TestToDatetimeUnit(object):
 


Code review comments:
diff_hunk: @@ -372,6 +372,19 @@ def test_datetime_invalid_datatype(self):
         with pytest.raises(TypeError):
             pd.to_datetime(pd.to_datetime)
 
+    @pytest.mark.parametrize('date, format',
+                             [('2017-20', '%Y-%W'),
+                              ('20 Sunday', '%W %A'),
+                              ('20 Sun', '%W %a'),
+                              ('2017-21', '%Y-%U'),
+                              ('20 Sunday', '%U %A'),
+                              ('20 Sun', '%U %a')])
+    def test_week_without_day_and_calendar_year(self, date, format):
+        # GH16774
+
+        with pytest.raises(ValueError):
path: pandas/tests/indexes/datetimes/test_tools.py
text: @user1:
Instead of `pytest.raises`, use `tm.assert_raises_regex` to check for the error message as well.

@author:
Thanks. Done.