COMPAT: dateutil fixups for 2.3 (GH9021, GH8639)
State: closed

fixes #9021 
but going to reopen #8639 


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
ok, this shows the failures (scroll way to the end): https://travis-ci.org/jreback/pandas/jobs/43281972

cc @dbew

this mostly fixes the issues so going to merge this.

I get massive test failures similar to travis. Any ETA for a fix of those?

are u sure that u updated to master and recompiled
their should only be a single test failure ATM 

- Fetching https://github.com/pydata/pandas.git ...
- Checking out https://github.com/pydata/pandas.git to /var/tmp/portage/dev-python/pandas-9999/work/pandas-9999 ...
  git checkout --quiet master
  GIT update -->
  repository:               https://github.com/pydata/pandas.git
  at the commit:            67ec0a883c13ce8eadc7f8e250d44d2954522913

......

# 

## ERROR: test_timestamp_tz_convert (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 341, in test_timestamp_tz_convert
    conv = idx[0].tz_convert(self.tzstr('US/Pacific'))
  File "pandas/tslib.pyx", line 481, in pandas.tslib.Timestamp.tz_convert (pandas/tslib.c:10835)
TypeError: Cannot convert tz-naive Timestamp, use tz_localize to localize

# 

## ERROR: test_utc_with_system_utc (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 849, in test_utc_with_system_utc
    self.assertEqual(ts, ts.tz_convert(dateutil.tz.tzutc()))
  File "pandas/tslib.pyx", line 481, in pandas.tslib.Timestamp.tz_convert (pandas/tslib.c:10835)
TypeError: Cannot convert tz-naive Timestamp, use tz_localize to localize

# 

## ERROR: test_with_tz (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 382, in test_with_tz
    central = dr.tz_convert(tz)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/index.py", line 1609, in tz_convert
    raise TypeError('Cannot convert tz-naive timestamps, use '
TypeError: Cannot convert tz-naive timestamps, use tz_localize to localize

# 

## ERROR: test_tz_convert_roundtrip (test_timezones.TestTimeZones)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 1035, in test_tz_convert_roundtrip
    reset = converted.tz_convert(None)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/index.py", line 1609, in tz_convert
    raise TypeError('Cannot convert tz-naive timestamps, use '
TypeError: Cannot convert tz-naive timestamps, use tz_localize to localize

# 

## FAIL: test_ambiguous_flags (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 471, in test_ambiguous_flags
    self.assert_numpy_array_equal(dr, localized)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/util/testing.py", line 638, in assert_numpy_array_equal
    '{0} is not equal to {1}.'.format(np_array, assert_equal))
AssertionError: <class 'pandas.tseries.index.DatetimeIndex'>
[2011-11-06 00:00:00, ..., 2011-11-06 04:00:00]
Length: 5, Freq: H, Timezone: None is not equal to <class 'pandas.tseries.index.DatetimeIndex'>
[2011-11-06 00:00:00, ..., 2011-11-06 03:00:00]
Length: 5, Freq: None, Timezone: None.

# 

## FAIL: test_ambiguous_infer (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 432, in test_ambiguous_infer
    self.assertRaises(pytz.AmbiguousTimeError, dr.tz_localize, tz)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/util/testing.py", line 1562, in assertRaises
    _callable(_args, *_kwargs)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/util/testing.py", line 1626, in __exit__
    raise AssertionError("{0} not raised.".format(name))
AssertionError: AmbiguousTimeError not raised.

# 

## FAIL: test_ambiguous_nat (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 517, in test_ambiguous_nat
    self.assert_numpy_array_equal(di_test, localized)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/util/testing.py", line 638, in assert_numpy_array_equal
    '{0} is not equal to {1}.'.format(np_array, assert_equal))
AssertionError: <class 'pandas.tseries.index.DatetimeIndex'>
[2011-11-06 00:00:00-04:00, ..., 2011-11-06 03:00:00-05:00]
Length: 5, Freq: None, Timezone: US/Eastern is not equal to <class 'pandas.tseries.index.DatetimeIndex'>
[2011-11-06 00:00:00, ..., 2011-11-06 03:00:00]
Length: 5, Freq: None, Timezone: None.

# 

## FAIL: test_datetimeindex_tz_nat (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 812, in test_datetimeindex_tz_nat
    self.assertTrue(idx[0].tzinfo is not None)
AssertionError: False is not true

# 

## FAIL: test_frame_no_datetime64_dtype (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 673, in test_frame_no_datetime64_dtype
    self.assertEqual(e['B'].dtype, 'O')
AssertionError: dtype('<M8[ns]') != 'O'

# 

## FAIL: test_index_with_timezone_repr (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 562, in test_index_with_timezone_repr
    self.assertIn('2010-04-13 00:00:00', rng_repr)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/util/testing.py", line 572, in assertIn
    assert a in b, "%s: %r is not in %r" % (msg.format(a, b), a, b)
AssertionError: : '2010-04-13 00:00:00' is not in "<class 'pandas.tseries.index.DatetimeIndex'>\n[2010-04-13, ..., 2010-05-06]\nLength: 24, Freq: D, Timezone: None"

# 

## FAIL: test_infer_tz (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 538, in test_infer_tz
    self.assertRaises(Exception, tools._infer_tzinfo, start, end)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/util/testing.py", line 1562, in assertRaises
    _callable(_args, *_kwargs)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/util/testing.py", line 1626, in __exit__
    raise AssertionError("{0} not raised.".format(name))
AssertionError: Exception not raised.

# 

## FAIL: test_localize_utc_conversion (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 107, in test_localize_utc_conversion
    self.assert_numpy_array_equal(converted.asi8, expected_naive.asi8)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/util/testing.py", line 638, in assert_numpy_array_equal
    '{0} is not equal to {1}.'.format(np_array, assert_equal))
AssertionError: [1331337600000000000 1331339400000000000 1331341200000000000
 1331343000000000000 1331344800000000000 1331346600000000000
 1331348400000000000 1331350200000000000 1331352000000000000
 1331353800000000000 1331355600000000000 1331357400000000000
 1331359200000000000 1331361000000000000 1331362800000000000
 1331364600000000000 1331366400000000000 1331368200000000000
 1331370000000000000 1331371800000000000 1331373600000000000
 1331375400000000000 1331377200000000000 1331379000000000000
 1331380800000000000 1331382600000000000 1331384400000000000
 1331386200000000000 1331388000000000000 1331389800000000000
 1331391600000000000 1331393400000000000 1331395200000000000
 1331397000000000000 1331398800000000000 1331400600000000000
 1331402400000000000 1331404200000000000 1331406000000000000
 1331407800000000000 1331409600000000000 1331411400000000000
 1331413200000000000 1331415000000000000 1331416800000000000
 1331418600000000000 1331420400000000000 1331422200000000000
 1331424000000000000] is not equal to [1331355600000000000 1331357400000000000 1331359200000000000
 1331361000000000000 1331362800000000000 1331364600000000000
 1331366400000000000 1331368200000000000 1331370000000000000
 1331371800000000000 1331373600000000000 1331375400000000000
 1331377200000000000 1331379000000000000 1331380800000000000
 1331382600000000000 1331384400000000000 1331386200000000000
 1331388000000000000 1331389800000000000 1331391600000000000
 1331393400000000000 1331395200000000000 1331397000000000000
 1331398800000000000 1331400600000000000 1331402400000000000
 1331404200000000000 1331406000000000000 1331407800000000000
 1331409600000000000 1331411400000000000 1331413200000000000
 1331415000000000000 1331416800000000000 1331418600000000000
 1331420400000000000 1331422200000000000 1331424000000000000
 1331425800000000000 1331427600000000000 1331429400000000000
 1331431200000000000 1331433000000000000 1331434800000000000
 1331436600000000000 1331438400000000000 1331440200000000000
 1331442000000000000].

# 

## FAIL: test_localize_utc_conversion_explicit (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 122, in test_localize_utc_conversion_explicit
    self.assert_(np.array_equal(converted.asi8, expected_naive.asi8))
AssertionError: False is not true

# 

## FAIL: test_timedelta_push_over_dst_boundary (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 184, in test_timedelta_push_over_dst_boundary
    self.assertEquals(result, expected)
AssertionError: Timestamp('2012-03-11 04:00:00') != Timestamp('2012-03-11 05:00:00')

# 

## FAIL: test_timedelta_push_over_dst_boundary_explicit (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 197, in test_timedelta_push_over_dst_boundary_explicit
    self.assertEqual(result, expected)
AssertionError: Timestamp('2012-03-11 04:00:00') != Timestamp('2012-03-11 05:00:00')

# 

## FAIL: test_tz_localize_dti (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 210, in test_tz_localize_dti
    self.assert_numpy_array_equal(dti2.values, dti_utc.values)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/util/testing.py", line 638, in assert_numpy_array_equal
    '{0} is not equal to {1}.'.format(np_array, assert_equal))
AssertionError: ['2005-01-01T01:00:00.000000000+0100' '2005-01-01T01:00:00.001000000+0100'
 '2005-01-01T01:00:00.002000000+0100' ...,
 '2005-01-01T01:00:30.254000000+0100' '2005-01-01T01:00:30.255000000+0100'
 '2005-01-01T01:00:30.256000000+0100'] is not equal to ['2005-01-01T06:00:00.000000000+0100' '2005-01-01T06:00:00.001000000+0100'
 '2005-01-01T06:00:00.002000000+0100' ...,
 '2005-01-01T06:00:30.254000000+0100' '2005-01-01T06:00:30.255000000+0100'
 '2005-01-01T06:00:30.256000000+0100'].

# 

## FAIL: test_utc_box_timestamp_and_localize (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 335, in test_utc_box_timestamp_and_localize
    self.assert_('EDT' in repr(rng_eastern[0].tzinfo) or 'tzfile' in repr(rng_eastern[0].tzinfo))
AssertionError: False is not true

# 

## FAIL: test_with_tz_ambiguous_times (test_timezones.TestTimeZoneSupportDateutil)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 411, in test_with_tz_ambiguous_times
    self.assertRaises(pytz.NonExistentTimeError, dr.tz_localize, tz)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/util/testing.py", line 1562, in assertRaises
    _callable(_args, *_kwargs)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/util/testing.py", line 1626, in __exit__
    raise AssertionError("{0} not raised.".format(name))
AssertionError: NonExistentTimeError not raised.

# 

## FAIL: test_tz_localize_roundtrip (test_timezones.TestTimeZones)

Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/tseries/tests/test_timezones.py", line 964, in test_tz_localize_roundtrip
    localized.tz_localize(tz)
  File "/var/tmp/portage/dev-python/pandas-9999/work/pandas-9999-python2_7/lib/pandas/util/testing.py", line 1626, in **exit**
    raise AssertionError("{0} not raised.".format(name))
AssertionError: TypeError not raised.

---

Ran 135 tests in 0.524s

FAILED (SKIP=2, errors=4, failures=15)

## INSTALLED VERSIONS

commit: None
python: 2.7.8.final.0
python-bits: 64
OS: Linux
OS-release: 3.17.4-lh
machine: x86_64
processor: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
byteorder: little
LC_ALL: None
LANG: en_GB.utf8

pandas: 0.15.1-213-g67ec0a8
nose: 1.3.4
Cython: 0.21.1
numpy: 1.8.2
scipy: 0.14.0
statsmodels: 0.5.0
IPython: 2.3.1
sphinx: 1.2.3
patsy: 0.3.0
dateutil: 2.3
pytz: 2014.10
bottleneck: 0.8.0
tables: 3.1.1
numexpr: 2.4
matplotlib: 1.4.2
openpyxl: 2.1.0
xlrd: 0.9.3
xlwt: 0.7.5
xlsxwriter: None
lxml: 3.4.1
bs4: 4.3.2
html5lib: 0.999
httplib2: None
apiclient: None
rpy2: None
sqlalchemy: None
pymysql: None
psycopg2: None

you need to

`python setup.py build_ext --inplace`

you didn't recompile

I was able to repro the problem, and the single original test failure remains (but is actually a larger problem).

datetutil 2.3 broke the API (again :()), this was release a few days ago

easiest is simply to downgrade to 2.2. or use pytz timezones which are the default anyhow

I did recompile, but I didn't paste the log here.
In case you would like to see it, it is here http://dev.gentoo.org/~jlec/paste/build.log

you are showing me a build log for 3.3.
not really sure what that is anyhow.

Pls humor me. Just clone the repo in a new directly, compile and test on 2.7.

Where are you seeing a travis failure (again except for the final build, which has 2 test failures)?

Just scroll up, there is the test failure for 2.7.

And believe me, I am experienced enough in building packages that you don't need to treat me as a stupid noob. The failure is there on python 2.7, 3.3 and 3.4. 

@jlec I appreciate you are trying to help. Pls downgrade to 2.2
and await fix when someone gets around to it. Complain to dateutil that they broke the API.

This is plain python-2.7 with current HEAD and python-dateutils-2.2

https://gist.github.com/jlec/5dc56c919e5a0506dcbb

so since this is not happening on any other linux (for 2.2), then something different between what you are doing and say travis. maybe you can help figure that out. 

IIRC you had this problem with 0.15.0 as well?

With 0.15\* I had #8639

But I am retesting right now.

So current HEAD and 0.15.1 show different problems using dateutil-2.2