Fixes #45506 Catch overflow error when converting to datetime
State: closed

Exception is raised when date part (like day, month, year) is greater than 32 bit signed integer

- [x] xref #45506
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Datetime
description: Datetime data dtype
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
Hello @Compro-Prasad! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-01-31 14:17:45 UTC
Could you add a unit test?

Also I don't think this closes the original issue. I think in the original issue the strings should be just compared as strings not datetime-like objects.
Yeah I was hoping the fix would result in a boolean series.

```python
import pandas as pd

data = [
    {"f_0": "2015-07-01", "f_2": "08335394550"},
    {"f_0": "2015-07-02", "f_2": "+49 (0) 0345 300033"},
    {"f_0": "2015-07-03", "f_2": "+49(0)2598 04457"},
    {"f_0": "2015-07-04", "f_2": "0741470003"},
    {"f_0": "2015-07-05", "f_2": "04181 83668"},
]

dtypes = dict(f_0='datetime64[ns]', f_2='string')
df = pd.DataFrame(data=data).astype(dtypes)
df['f_0'].eq(df['f_2'])
```

```
0    False
1    False
2    False
3    False
4    False
dtype: bool
```
> Could you add a unit test?

Sure. On it.

> Also I don't think this closes the original issue. I think in the original issue the strings should be just compared as strings not datetime-like objects.

Guessing datatypes isn't the most pleasant things to work with. For efficiency its nice to convert both into strings. But if one is already a datetime like object then its more convenient to prefer datetime over string. If that fails, string should be preferred. This case from OP is not a real scenario anyway.

> Yeah I was hoping the fix would result in a boolean series.

That is what I get after applying the fix:

```
Python 3.10.1 (main, Dec 18 2021, 23:53:45) [GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> 
>>> data = [
...     {"f_0": "2015-07-01", "f_2": "08335394550"},
...     {"f_0": "2015-07-02", "f_2": "+49 (0) 0345 300033"},
...     {"f_0": "2015-07-03", "f_2": "+49(0)2598 04457"},
...     {"f_0": "2015-07-04", "f_2": "0741470003"},
...     {"f_0": "2015-07-05", "f_2": "04181 83668"},
... ]
>>> 
>>> dtypes = dict(f_0='datetime64[ns]', f_2='string')
>>> df = pd.DataFrame(data=data).astype(dtypes)
>>> df['f_0'].eq(df['f_2'])
0    False
1    False
2    False
3    False
4    False
dtype: bool
```

> pls always start with a test that demonstrates the issue (from the OP)

Sure thing. I am very new to pandas development. Thanks for the suggestion. I will add a test.
Have added a test in `pandas/tests/arrays/test_array.py`. Is this the correct place where the test should go?
I found the following error from `mypy`:

```
pandas/io/parquet.py:184: error: "RawIOBase" has no attribute "name"  [attr-defined]
```

IMO using `.name` instead of `.raw.name` should be a good enough solution.

Shall I make the changes?
> I found the following error from `mypy`:
> 
> ```
> pandas/io/parquet.py:184: error: "RawIOBase" has no attribute "name"  [attr-defined]
> ```
> 
> IMO using `.name` instead of `.raw.name` should be a good enough solution.
> 
> Shall I make the changes?

ignore as this is taken care of elsewhere
@jreback Have made the changes.
@jreback Have added the whatsnew entry.
thanks @Compro-Prasad