DEPR/API: Non-ns precision in Index constructors
State: closed

This deprecates passing dtypes without a precision to DatetimeIndex
and TimedeltaIndex

```python
In [2]: pd.DatetimeIndex(['2000'], dtype='datetime64')
/Users/taugspurger/.virtualenvs/pandas-dev/bin/ipython:1: FutureWarning: Passing in 'datetime64' dtype with no precision is deprecated
and will raise in a future version. Please pass in
'datetime64[ns]' instead.
  #!/Users/taugspurger/Envs/pandas-dev/bin/python3
Out[2]: DatetimeIndex(['2000-01-01'], dtype='datetime64[ns]', freq=None)
```

Closes https://github.com/pandas-dev/pandas/issues/24739
Closes https://github.com/pandas-dev/pandas/issues/24753

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Datetime
description: Datetime data dtype
name: Timedelta
description: Timedelta data type
name: Deprecate
description: Functionality to remove in pandas

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=h1) Report
> Merging [#24806](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/98f46f7f08e9f15374fd840507438cb237313a9e?src=pr&el=desc) will **decrease** coverage by `49.45%`.
> The diff coverage is `57.69%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24806/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           master   #24806       +/-   ##
===========================================
- Coverage   92.38%   42.92%   -49.46%     
===========================================
  Files         166      166               
  Lines       52382    52397       +15     
===========================================
- Hits        48395    22494    -25901     
- Misses       3987    29903    +25916
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `?` | |
| #single | `42.92% <57.69%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `56.49% <100%> (-39.82%)` | :arrow_down: |
| [pandas/core/arrays/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2RhdGV0aW1lcy5weQ==) | `65.88% <40%> (-31.91%)` | :arrow_down: |
| [pandas/core/arrays/timedeltas.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL3RpbWVkZWx0YXMucHk=) | `41.81% <60%> (-46.41%)` | :arrow_down: |
| [pandas/io/formats/latex.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvbGF0ZXgucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/core/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY2F0ZWdvcmljYWwucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/sas/sas\_constants.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNfY29uc3RhbnRzLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/tseries/plotting.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvcGxvdHRpbmcucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/tseries/converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvY29udmVydGVyLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/formats/html.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvaHRtbC5weQ==) | `0% <0%> (-99.35%)` | :arrow_down: |
| [pandas/core/groupby/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS9jYXRlZ29yaWNhbC5weQ==) | `0% <0%> (-95.46%)` | :arrow_down: |
| ... and [126 more](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=footer). Last update [98f46f7...37d6779](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=h1) Report
> Merging [#24806](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0f84b4092f7a78344e06933673b664dbe73096b8?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24806/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #24806      +/-   ##
==========================================
+ Coverage   92.39%   92.39%   +<.01%     
==========================================
  Files         166      166              
  Lines       52391    52401      +10     
==========================================
+ Hits        48407    48417      +10     
  Misses       3984     3984
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.82% <100%> (ø)` | :arrow_up: |
| #single | `42.89% <60.86%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/arrays/timedeltas.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL3RpbWVkZWx0YXMucHk=) | `88.3% <100%> (+0.09%)` | :arrow_up: |
| [pandas/core/arrays/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2RhdGV0aW1lcy5weQ==) | `97.79% <100%> (+0.01%)` | :arrow_up: |
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/24806/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `96.32% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=footer). Last update [0f84b40...a9a9588](https://codecov.io/gh/pandas-dev/pandas/pull/24806?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Fixed the merge conflict.

I'm inclined to keep the change of `DatetimeIndex([1, 2], dtype='datetime64[us])` raising, as the user is unambiguously asking for something wrong (in comparison with `'datetime64'`, which is ambiguous, so we deprecate it).
All green. Anyone else (@jorisvandenbossche @jbrockmendel) have an opinion on whether this should warn or raise?

```python
In [3]: pd.DatetimeIndex(['2000'], dtype='datetime64[us]')
Out[3]: DatetimeIndex(['2000-01-01'], dtype='datetime64[ns]', freq=None)
```

On 0.23 that silently ignored the precision.

On master (and this PR) we raise.
No strong opinion on raise vs warn for datetime64[us]
> On 0.23 that silently ignored the precision.

Pedantic note: it doesn't really "ignores the precision" in this example case, as it is doing "datetime string parsing", even if you pass `dtype=datetime64[ns]` it doesn't use the ns precision to interpret the string value '2000'. 
But we do ignore it for integer input:

```
In [10]: pd.DatetimeIndex([2000], dtype='datetime64[s]')
Out[10]: DatetimeIndex(['1970-01-01 00:00:00.000002'], dtype='datetime64[ns]', freq=None)
```

so in the end my note doesn't matter :), because here it is indeed being ignored, and fine to raise an error here.

> fine to raise an error here.

Actually, this already seems to be failing on master:

```
In [86]: pd.DatetimeIndex([2000], dtype='datetime64[s]')                                                                                                                        
...
ValueError: Unexpected value for 'dtype': 'datetime64[s]'. Must be 'datetime64[ns]' or DatetimeTZDtype'.
```

So we already changed this (by accident?) compared to 0.23.4.
@jorisvandenbossche can you explain your conclusion in 
https://github.com/pandas-dev/pandas/pull/24806#issuecomment-455458403 and 
https://github.com/pandas-dev/pandas/pull/24806#issuecomment-455458041 a bit further? I don't see where the precision is being used for parsing (strings or ints), though maybe that's what you're saying?

On 0.23.4

```python
In [3]: pd.DatetimeIndex([2000], dtype='datetime64[s]')
Out[3]: DatetimeIndex(['1970-01-01 00:00:00.000002'], dtype='datetime64[ns]', freq=None)
```

i.e. the same a `datetime64[ns]`. You could imagine it being used as the "unit" parameter for Timestamp for interpreting the integers, but that's not what we did on 0.23.4.

---

So to summarize *my* understanding, which may be wrong

1. In 0.23.4 DatetimeIndex completely ignored the precision.
2. Master currently raises for anything that is non-ns, including no precision.
3. I think we should warn for `dtype='datetime64'` (no precision) since it's ambiguous.
4. I think we should 4raise for `dtype='datetime64[s]'` (non-ns precision) since it's wrong.

those all apply to TimedeltaIndex and Index as well.
Specifically

> Pedantic note: it doesn't really "ignores the precision" in this example case, as it is doing "datetime string parsing", even if you pass dtype=datetime64[ns] it doesn't use the ns precision to interpret the string value '2000'.


Two things there confused me

1. I'm not sure how the precision would affect parsing of strings at all. To me, those don't have an *input* precision (but end up with a precision when they're a Timestamp / array)
2. The `dtype` parameter to Index/DatetimeIndex/TimedeltaIndex means the *desired output dtype*. Not anything to do with parsing, or the input dtype of `data`.
Yes, the confusing part (or at least why my comment was also confusing / wrong), is the difference of input vs output dtype. About my comment that the precision is ignored for string parsing, that was indeed assuming that the `dtype` argument would give an indication about the provided input data. But of course, that is not the case. 

Basically, the whole `dtype` argument is just completely useless (apart from consistency with the other constructors, the reason we have it I assume), as there is only one possible dtype (note: this is only true for tz naive of course, but for that there is also the `tz` keyword).

lgtm merge on green.
thanks @TomAugspurger