Question: I have a csv with a datetime column. However, dates go all the way from 1 AD to 3000 AD. They are out of range for a normal datetime64[ns] dtype. In pandas < 2, my workaround was to have this column with a "period[H]" dtype. I had a custom function doing this, which I passed as a date_parser in pd.read_csv. In pandas >= 2, an option to have timestamps with other resolutions has been introduced. The date_parser argument has been deprecated, so the previous solution is also not doable. I thought I could simply cast the whole column of strings with pd.to_datetime. But that doesn't work. Full example: example.csv: index,date A,"0004-04-04T12:30" B,"2004-04-04T12:30" C,"3004-04-04T12:30"  1. Clueless optimism - Fails Dates are ISO8601, easiest of the formats. df = pd.read_csv('example.csv', parse_dates=['date']) print(df.date.dtype, '|', type(df.date.iloc[0]))  Got a warning: <ipython-input-103-059359fbaeab>:1: UserWarning: Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format  And output: object | <class 'str'> If example.csv only contains datetime64[ns]-valid dates, this works perfectly. 2. As documented - Fails Doc says: "read in as object and then apply to_datetime() as-needed." df = pd.read_csv('example.csv') pd.to_datetime(df.date)  Fails with OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 0004-04-04T12:30, at position 0  pd.to_datetime(df.date, unit='s') gives : ValueError: non convertible value 0004-04-04T12:30 with the unit 's', at position 0  so I guess the unit argument here doesn't have the meaning I expected. 3. Per-element conversion - Partially works def to_timestamp(val): if pd.isna(val): return pd.Timestamp('NaT', unit='s') return pd.Timestamp(val, unit='s') df = pd.read_csv('example.csv') df['date'] = df.date.apply(_to_timestamp) print(df.date.dtype, '|', type(df.date.iloc[0]))  Prints: object | <class 'pandas._libs.tslibs.timestamps.Timestamp'> So indeed, the individual elements have been converted to timestamps, yay! But the column itself still has an "object" dtype, not a "datetime" one. df.date.astype('datetime64[s]') fails with OutOfBoundsDatetime: Cannot cast 0004-04-04 12:30:00 to unit='ns' without overflow., at position 0  even though I specified a "s" resolution ? Same with pd.to_datetime(df.date). 4. Step outside pandas - Works import numpy as np def to_d64(val): if pd.isna(val): return np.datetime64('') return np.datetime64(val) df = pd.read_csv('example.csv') df['date'] = np.array(df.date.apply(to_d64), dtype='M8[s]') print(df.date.dtype, '|', type(df.date.iloc[0]))  Success! It prints: datetime64[s] | <class 'pandas._libs.tslibs.timestamps.Timestamp'>  But at what price? In the real code, my csv is thousands of lines long. The pandas < 2 code made sure to avoid converting element-per-element, as this had performance issues. I was hoping that the support for "s" resolution would be accessible directly from the string-parsing step. I also note that my final example works because my strings are in a ISO8601 format, which is support by numpy. That is not a general solution at all. Did I miss something? Does anybody have a cleaner solution? EDIT: I am using pandas 2.0.3 and numpy 1.24.4, Python 3.11.4 on Linux (Fedora 37).

Answer: I think unit in to_datetime only takes effect if your input is integers - but I think there is work to make it take effect even if the input is strings. Non-nanosecond support is quite new, so not yet too mature. Anyway, here's a solution which you can use until to_datetime can parse non-nanosecond range strings, which involves going via numpy datetime64[s]: In [22]: data = """\ ...: index,date ...: A,"0004-04-04T12:30" ...: B,"2004-04-04T12:30" ...: C,"3004-04-04T12:30" ...: """ In [23]: df = pd.read_csv(io.StringIO(data)) In [24]: df['date'] = df['date'].to_numpy().astype('datetime64[s]') In [25]: df Out[25]: index date 0 A 4-04-04 12:30:00 1 B 2004-04-04 12:30:00 2 C 3004-04-04 12:30:00