REF: require listlike in maybe_cast_to_datetime
State: closed


Labels:
name: Refactor
description: Internal refactoring of code

Comments:
are the fails unrelated?
> are the fails unrelated?

no, there are some int32 wrinkles to figure out
@simonjayhawkins IIRC you use windows.  can you reproduce the failures here?
> @simonjayhawkins IIRC you use windows. can you reproduce the failures here?

Not been successful running test suite locally on Windows for a long time. now use Ubuntu in WSL.

last time I tried to debug Windows issues, I seemed to be hit with #38214. (also concerns listlike)
can confirm running full test suite still failing for me (with master)

here as a baseline. note the duration. although I think it had hung and I may have interrupted. will run on same hardware with Ubuntu for comparison and also with this branch on Windows shortly.

```
=========================== short test summary info ===========================
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other3]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other7]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other0]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other4]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other8]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other2]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other6]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other1]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other5]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other9]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_mixed_invalid[tzlocal()]
FAILED pandas/tests/dtypes/cast/test_promote.py::test_maybe_promote_datetimetz_with_datetimetz[tzlocal()-tzlocal()]
FAILED pandas/tests/extension/json/test_json.py::TestConstructors::test_series_constructor_no_data_with_index
FAILED pandas/tests/extension/json/test_json.py::TestConstructors::test_series_constructor_scalar_na_with_index
FAILED pandas/tests/plotting/test_backend.py::test_register_entrypoint - pkg_...
FAILED pandas/tests/plotting/frame/test_frame_subplots.py::TestDataFramePlotsSubplots::test_bar_barwidth_position_int
ERROR pandas/tests/io/excel/test_readers.py::TestReaders::test_read_from_py_localpath[engine_and_read_ext1]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
C:\Users\simon\anaconda3\envs\pandas-dev\lib\threading.py:306: KeyboardInterrupt
(to show a full traceback on KeyboardInterrupt use --full-trace)
= 16 failed, 147074 passed, 3542 skipped, 1074 xfailed, 6 xpassed, 115 warnings, 1 error in 4577.43s (1:16:17) =
```

with a splattering of...
```
.......................................................Windows fatal exception: stack overflow

Thread 0x000031ac (most recent call first):
  File "C:\Users\simon\anaconda3\envs\pandas-dev\lib\site-packages\execnet\gateway_base.py", line 400 in read
  File "C:\Users\simon\anaconda3\envs\pandas-dev\lib\site-packages\execnet\gateway_base.py", line 432 in from_io
  File "C:\Users\simon\anaconda3\envs\pandas-dev\lib\site-packages\execnet\gateway_base.py", line 967 in _thread_receiver
  File "C:\Users\simon\anaconda3\envs\pandas-dev\lib\site-packages\execnet\gateway_base.py", line 220 in run
  File "C:\Users\simon\anaconda3\envs\pandas-dev\lib\site-packages\execnet\gateway_base.py", line 285 in _perform_spawn

Current thread 0x00002fe8 (most recent call first):
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 652 in create_series_with_explicit_dtype
  File "C:\Users\simon\pandas\pandas\core\series.py", line 382 in _init_dict
  File "C:\Users\simon\pandas\pandas\core\series.py", line 289 in __init__
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 656 in create_series_with_explicit_dtype
  File "C:\Users\simon\pandas\pandas\core\series.py", line 382 in _init_dict
  File "C:\Users\simon\pandas\pandas\core\series.py", line 289 in __init__
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 656 in create_series_with_explicit_dtype
  File "C:\Users\simon\pandas\pandas\core\series.py", line 382 in _init_dict
  File "C:\Users\simon\pandas\pandas\core\series.py", line 289 in __init__
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 656 in create_series_with_explicit_dtype
```


same hardware. WSL...

```
=========================================================== short test summary info ============================================================
FAILED pandas/tests/io/test_clipboard.py::test_raw_roundtrip[\U0001f44d...] - pandas.io.clipboard.PyperclipException:
FAILED pandas/tests/io/test_clipboard.py::test_raw_roundtrip[\u03a9\u0153\u2211\xb4...] - pandas.io.clipboard.PyperclipException:
FAILED pandas/tests/io/test_clipboard.py::test_raw_roundtrip[abcd...] - pandas.io.clipboard.PyperclipException:
====================== 3 failed, 148578 passed, 3410 skipped, 1077 xfailed, 2 xpassed, 105 warnings in 607.40s (0:10:07) =====================
```
> can you reproduce the failures here?

appears so. (95-16 = 79)

```
=========================== short test summary info ===========================
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other3]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other7]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other0]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other4]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other8]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other2]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other6]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other1]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other5]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other9]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_mixed_invalid[tzlocal()]
FAILED pandas/tests/dtypes/cast/test_promote.py::test_maybe_promote_datetimetz_with_datetimetz[tzlocal()-tzlocal()]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[Int8Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[Int16Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[Int32Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[Int64Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[UInt8Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[UInt16Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[UInt32Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[UInt64Dtype]
FAILED pandas/tests/extension/test_interval.py::TestGrouping::test_groupby_extension_transform
FAILED pandas/tests/frame/test_stack_unstack.py::TestDataFrameReshape::test_stack_mixed_level
FAILED pandas/tests/extension/test_numpy.py::TestGroupby::test_groupby_extension_transform[float]
FAILED pandas/tests/extension/test_numpy.py::TestGroupby::test_groupby_extension_transform[object]
FAILED pandas/tests/extension/test_boolean.py::TestGroupby::test_groupby_extension_transform
FAILED pandas/tests/extension/test_datetime.py::TestGroupby::test_groupby_extension_transform[US/Central]
FAILED pandas/tests/extension/test_period.py::TestGroupby::test_groupby_extension_transform
FAILED pandas/tests/extension/test_floating.py::TestGroupby::test_groupby_extension_transform[Float32Dtype]
FAILED pandas/tests/extension/test_floating.py::TestGroupby::test_groupby_extension_transform[Float64Dtype]
FAILED pandas/tests/extension/test_integer.py::TestDtype::test_check_dtype[Int32Dtype]
FAILED pandas/tests/extension/test_integer.py::TestDtype::test_check_dtype[Int64Dtype]
FAILED pandas/tests/frame/methods/test_count.py::TestDataFrameCount::test_count
FAILED pandas/tests/frame/methods/test_count.py::TestDataFrameCount::test_count_level_corner
FAILED pandas/tests/frame/methods/test_dtypes.py::TestDataFrameDataTypes::test_empty_frame_dtypes
FAILED pandas/tests/extension/test_string.py::TestGroupBy::test_groupby_extension_transform[StringDtype]
FAILED pandas/tests/extension/test_string.py::TestGroupBy::test_groupby_extension_transform[ArrowStringDtype]
FAILED pandas/tests/extension/decimal/test_decimal.py::TestGroupby::test_groupby_extension_transform
FAILED pandas/tests/frame/methods/test_get_numeric_data.py::TestGetNumericData::test_get_numeric_data
FAILED pandas/tests/extension/json/test_json.py::TestConstructors::test_series_constructor_no_data_with_index
FAILED pandas/tests/indexes/datetimes/test_partial_slicing.py::TestSlicing::test_monotone_DTI_indexing_bug
FAILED pandas/tests/series/indexing/test_delitem.py::TestSeriesDelItem::test_delitem_object_index
FAILED pandas/tests/series/indexing/test_setitem.py::TestSetitemWithExpansion::test_setitem_empty_series
FAILED pandas/tests/series/indexing/test_setitem.py::TestSetitemWithExpansion::test_setitem_empty_series_datetimeindex_preserves_freq
FAILED pandas/tests/resample/test_deprecated.py::test_resample_loffset_count
FAILED pandas/tests/resample/test_deprecated.py::test_resample_float_base - A...
FAILED pandas/tests/series/methods/test_unstack.py::test_unstack_tuplename_in_multiindex
FAILED pandas/tests/series/methods/test_unstack.py::test_unstack_mixed_type_name_in_multiindex[unstack_idx0-expected_values0-expected_index0-expected_columns0]
FAILED pandas/tests/series/methods/test_unstack.py::test_unstack_mixed_type_name_in_multiindex[unstack_idx1-expected_values1-expected_index1-expected_columns1]
FAILED pandas/tests/resample/test_time_grouper.py::test_upsample_sum[sum-method_args0-expected_values0]
FAILED pandas/tests/resample/test_time_grouper.py::test_upsample_sum[sum-method_args1-expected_values1]
FAILED pandas/tests/resample/test_time_grouper.py::test_upsample_sum[prod-method_args4-expected_values4]
FAILED pandas/tests/resample/test_time_grouper.py::test_upsample_sum[prod-method_args5-expected_values5]
FAILED pandas/tests/reshape/test_crosstab.py::TestCrosstab::test_crosstab_tuple_name[names0]
FAILED pandas/tests/reshape/test_crosstab.py::TestCrosstab::test_crosstab_tuple_name[names1]
FAILED pandas/tests/frame/methods/test_to_csv.py::TestDataFrameToCSV::test_to_csv_with_dst_transitions
FAILED pandas/tests/reshape/merge/test_join.py::TestJoin::test_join_index_mixed_overlap
FAILED pandas/tests/reshape/merge/test_join.py::test_join_inner_multiindex_deterministic_order
FAILED pandas/tests/reshape/merge/test_merge.py::TestMerge::test_merge_nocopy
FAILED pandas/tests/groupby/test_apply.py::test_time_field_bug - AssertionErr...
FAILED pandas/tests/groupby/test_categorical.py::test_apply[True] - Assertion...
FAILED pandas/tests/groupby/test_categorical.py::test_apply[False] - Assertio...
FAILED pandas/tests/reshape/merge/test_multi.py::TestMergeMulti::test_merge_multiple_cols_with_mixed_cols_index
FAILED pandas/tests/groupby/test_filters.py::test_filter_multiple_timestamp
FAILED pandas/tests/groupby/test_filters.py::test_filter_and_transform_with_non_unique_int_index
FAILED pandas/tests/groupby/test_filters.py::test_filter_and_transform_with_multiple_non_unique_int_index
FAILED pandas/tests/groupby/test_filters.py::test_filter_and_transform_with_non_unique_float_index
FAILED pandas/tests/groupby/test_filters.py::test_filter_and_transform_with_non_unique_timestamp_index
FAILED pandas/tests/groupby/test_filters.py::test_filter_and_transform_with_non_unique_string_index
FAILED pandas/tests/series/test_arithmetic.py::TestSeriesArithmetic::test_arithmetic_with_duplicate_index
FAILED pandas/tests/groupby/test_groupby_dropna.py::test_slice_groupby_then_transform[False-df_expected1-s_expected1]
FAILED pandas/tests/groupby/test_grouping.py::TestGrouping::test_grouper_getting_correct_binner
FAILED pandas/tests/extension/json/test_json.py::TestConstructors::test_series_constructor_scalar_na_with_index
FAILED pandas/tests/indexes/datetimes/test_shift.py::TestDatetimeIndexShift::test_dti_shift_near_midnight[0-2014-11-14 00:00:00]
FAILED pandas/tests/indexes/datetimes/test_shift.py::TestDatetimeIndexShift::test_dti_shift_near_midnight[-1-2014-11-13 23:00:00]
FAILED pandas/tests/indexes/datetimes/test_shift.py::TestDatetimeIndexShift::test_dti_shift_near_midnight[1-2014-11-14 01:00:00]
FAILED pandas/tests/io/json/test_normalize.py::TestJSONNormalize::test_nonetype_record_path[None]
FAILED pandas/tests/io/json/test_normalize.py::TestJSONNormalize::test_nonetype_record_path[nan0]
FAILED pandas/tests/io/json/test_normalize.py::TestJSONNormalize::test_nonetype_record_path[NaT]
FAILED pandas/tests/io/json/test_normalize.py::TestJSONNormalize::test_nonetype_record_path[nan1]
FAILED pandas/tests/io/json/test_normalize.py::TestJSONNormalize::test_nonetype_record_path[<NA>]
FAILED pandas/tests/indexing/test_datetime.py::TestDatetimeIndex::test_indexing_with_datetimeindex_tz
FAILED pandas/tests/indexing/test_datetime.py::TestDatetimeIndex::test_nanosecond_getitem_setitem_with_tz
FAILED pandas/tests/indexing/test_floats.py::TestFloatIndexers::test_scalar_integer[DataFrame-makeIntIndex-<lambda>-True]
FAILED pandas/tests/indexing/test_floats.py::TestFloatIndexers::test_scalar_integer[DataFrame-makeRangeIndex-<lambda>-True]
FAILED pandas/tests/indexing/test_iloc.py::TestiLoc2::test_iloc_setitem - Ass...
FAILED pandas/tests/indexing/test_loc.py::TestLocSetitemWithExpansion::test_loc_setitem_incremental_with_dst
FAILED pandas/tests/indexing/test_scalar.py::TestScalar2::test_imethods_with_dups
FAILED pandas/tests/frame/test_constructors.py::TestDataFrameConstructors::test_constructor_scalar_inference
FAILED pandas/tests/frame/test_constructors.py::TestDataFrameConstructors::test_constructor_list_of_series_aligned_index
FAILED pandas/tests/frame/test_constructors.py::TestDataFrameConstructors::test_constructor_with_datetimes
FAILED pandas/tests/frame/test_constructors.py::TestDataFrameConstructors::test_constructor_for_list_with_dtypes
FAILED pandas/tests/io/pytables/test_timezones.py::test_py2_created_with_datetimez
FAILED pandas/tests/plotting/test_backend.py::test_register_entrypoint - pkg_...
FAILED pandas/tests/plotting/frame/test_frame_subplots.py::TestDataFramePlotsSubplots::test_bar_barwidth_position_int
FAILED pandas/tests/resample/test_datetime_index.py::test_resample_upsampling_picked_but_not_correct
ERROR pandas/tests/io/excel/test_readers.py::TestExcelFileRead::test_excel_passes_na_filter[engine_and_read_ext0-False]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
C:\Users\simon\anaconda3\envs\pandas-dev\lib\threading.py:306: KeyboardInterrupt
(to show a full traceback on KeyboardInterrupt use --full-trace)
= 95 failed, 147109 passed, 3555 skipped, 1074 xfailed, 6 xpassed, 115 warnings, 1 error in 1912.78s (0:31:52) =
```
> Windows fatal exception: stack overflow

yikes.  Thanks for taking a look.  I'll keep poking at this on the CI for now.

> note the duration [4577.43s]

yikes

> same hardware. WSL...

re test_clipboard, I saw the same failures on a headless Ubuntu machine that I sometimes use, fixed that by

1) adding `export DISPLAY=":0"` to .bash_profile
2) amending the alias I use for tests to `alias ptest="xvfb-run python3 -m pytest pandas/tests --skip-slow --skip-db"`

YMMV

> re test_clipboard, I saw the same failures on a headless Ubuntu machine that I sometimes use, fixed that by

there's an open issue https://github.com/pandas-dev/pandas/issues/38527 and a potential fix https://github.com/pandas-dev/pandas/pull/38546

I'm using an (headless) desktop machine to run the tests (in WSL) using Windows remote desktop (from laptop).

i've not been too concerned since when I used to use the machine as my primary desktop machine, I always had clipboard errors. (the only difference being is that reruning the tests with --lf and they always then passed)
@jbrockmendel good on this?
yep