Repository: pandas-dev/pandas

Issue:
pd.to_datetime ValueError or TypeError
Hi,

I am using pandas 0.12.0 and numpy 1.7.1.

Not sure if this is the expected functionality or not.  I thought if coerce = True or errors='ignore' I wouldn't get this result:

``` python
In [372]: td = pd.Series(['May 04', 'Jun 02', 'Dec 11'], index=[1,2,3])

In [373]: td.apply(pd.to_datetime, format='%b %y', errors='ignore', coerce=True)
Out[373]:
1   2004-05-01 00:00:00
2   2002-06-01 00:00:00
3   2011-12-01 00:00:00
dtype: datetime64[ns]

In [374]: td = pd.Series(['May 04', 'Jun 02', ''], index=[1,2,3])

In [375]: td.apply(pd.to_datetime, format='%b %y', errors='ignore', coerce=True)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-375-0c44a3c36f8e> in <module>()
----> 1 td.apply(pd.to_datetime, format='%b %y', errors='ignore', coerce=True)

C:\Python27\lib\site-packages\pandas\core\series.pyc in apply(self, func, convert_dtype, args, **kwds)
   2534             values = lib.map_infer(values, lib.Timestamp)
   2535
-> 2536         mapped = lib.map_infer(values, f, convert=convert_dtype)
   2537         if isinstance(mapped[0], Series):
   2538             from pandas.core.frame import DataFrame

C:\Python27\lib\site-packages\pandas\lib.pyd in pandas.lib.map_infer (pandas\lib.c:42320)()

C:\Python27\lib\site-packages\pandas\core\series.pyc in <lambda>(x)
   2523
   2524         if kwds or args and not isinstance(func, np.ufunc):
-> 2525             f = lambda x: func(x, *args, **kwds)
   2526         else:
   2527             f = func

C:\Python27\lib\site-packages\pandas\tseries\tools.pyc in to_datetime(arg, errors, dayfirst, utc, box, format, coerce, u
nit)
    127         return _convert_listlike(arg, box=box)
    128
--> 129     return _convert_listlike(np.array([ arg ]), box=box)[0]
    130
    131 class DateParseError(ValueError):

C:\Python27\lib\site-packages\pandas\tseries\tools.pyc in _convert_listlike(arg, box)
    115                 return DatetimeIndex._simple_new(values, None, tz=tz)
    116             except (ValueError, TypeError):
--> 117                 raise e
    118
    119     if arg is None:

ValueError: time data '' does not match format '%b %y'

In [376]:

```



Discussion:
The problem with pd.to_datetime(td) is it converts "May 04" to May 4, 2013
instead of May 1, 2004.

On Sat, Oct 12, 2013 at 5:49 PM, jreback notifications@github.com wrote:

> that looks like a bug, of course you don't need the apply in any event
> 
> In [25]: pd.to_datetime(td)
> Out[25]:
> 1   2013-05-04 00:00:00
> 2   2013-06-02 00:00:00
> 3                   NaT
> dtype: datetime64[ns]
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/issues/5195#issuecomment-26206785
> .


Resolving patch:
diff --git a/pandas/tseries/tools.py b/pandas/tseries/tools.py
--- a/pandas/tseries/tools.py
+++ b/pandas/tseries/tools.py
@@ -112,7 +112,7 @@ def _convert_listlike(arg, box):
 
                 # fallback
                 if result is None:
-                    result = tslib.array_strptime(arg, format)
+                    result = tslib.array_strptime(arg, format, coerce=coerce)
             else:
                 result = tslib.array_to_datetime(arg, raise_=errors == 'raise',
                                                  utc=utc, dayfirst=dayfirst,


Test patch:
diff --git a/pandas/tseries/tests/test_timeseries.py b/pandas/tseries/tests/test_timeseries.py
--- a/pandas/tseries/tests/test_timeseries.py
+++ b/pandas/tseries/tests/test_timeseries.py
@@ -879,6 +879,29 @@ def test_to_datetime_on_datetime64_series(self):
         result = to_datetime(s)
         self.assertEquals(result[0], s[0])
 
+    def test_to_datetime_with_apply(self):
+
+        # this is only locale tested with US/None locales
+        import locale
+        (lang,encoding) = locale.getlocale()
+        if lang is not None:
+            raise nose.SkipTest("format codes cannot work with a locale of {0}".format(lang))
+
+        # GH 5195
+        # with a format and coerce a single item to_datetime fails
+        td = Series(['May 04', 'Jun 02', 'Dec 11'], index=[1,2,3])
+        expected = pd.to_datetime(td, format='%b %y')
+        result = td.apply(pd.to_datetime, format='%b %y')
+        assert_series_equal(result, expected)
+
+        td = pd.Series(['May 04', 'Jun 02', ''], index=[1,2,3])
+        self.assertRaises(ValueError, lambda : pd.to_datetime(td,format='%b %y'))
+        self.assertRaises(ValueError, lambda : td.apply(pd.to_datetime, format='%b %y'))
+        expected = pd.to_datetime(td, format='%b %y', coerce=True)
+
+        result = td.apply(lambda x: pd.to_datetime(x, format='%b %y', coerce=True))
+        assert_series_equal(result, expected)
+
     def test_nat_vector_field_access(self):
         idx = DatetimeIndex(['1/1/2000', None, None, '1/4/2000'])