Repository: pandas-dev/pandas
Pull request: [BUG] Validate dtype when Int64Index, UInt64Index, or Float64Index are cons…
…tructed

- [x] closes #29539 
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Problem statement:
Numeric Index constructors should validate `dtype` parameter
The `Int64Index`, `UInt64Index`, and `Float64Index` constructors do not validate the `dtype` parameter and instead ignore it when it conflicts with the default dtype:
```python
In [1]: import pandas as pd; pd.__version__
Out[1]: '0.26.0.dev0+842.g78c3c1640'

In [2]: pd.Int64Index([1, 2, 3], dtype="float64")
Out[2]: Int64Index([1, 2, 3], dtype='int64')

In [3]: pd.UInt64Index([1, 2, 3], dtype="categorical")
Out[3]: UInt64Index([1, 2, 3], dtype='uint64')

In [4]: pd.Float64Index([1, 2, 3], dtype="datetime64")
Out[4]: Float64Index([1.0, 2.0, 3.0], dtype='float64')
```

Other index types do not allow non-standard dtypes:
```python
In [5]: pd.CategoricalIndex(list('abc'), dtype='timedelta64')
---------------------------------------------------------------------------
ValueError: Unknown dtype 'timedelta64'

In [6]: pd.DatetimeIndex(['2018-01-01'], dtype='int64')
---------------------------------------------------------------------------
ValueError: Unexpected value for 'dtype': 'int64'. Must be 'datetime64[ns]' or DatetimeTZDtype'.

In [7]: pd.IntervalIndex([pd.Interval(0, 1)], dtype=object)
---------------------------------------------------------------------------
TypeError: dtype must be an IntervalDtype, got object

In [8]: pd.RangeIndex(0, 3, dtype='float64')
---------------------------------------------------------------------------
TypeError: Invalid to pass a non-int64 dtype to RangeIndex
```
#### Expected Output
I'd expect `In [2]`/`In [3]`/`In [4]` to raise a `ValueError`.

Hints from issue discussion:
Sure, I could work on this if you like (though if any new contributors following along would like to take it, please do go ahead)

Commit message: Don't allow Float64Index to be constructued with integer dtype, instead remove dtype from call to Float64Index in the constructor of Index

Patch to review:
diff --git a/doc/source/whatsnew/v1.0.0.rst b/doc/source/whatsnew/v1.0.0.rst
index cd012fe755337..0253c3237046d 100644
--- a/doc/source/whatsnew/v1.0.0.rst
+++ b/doc/source/whatsnew/v1.0.0.rst
@@ -336,6 +336,7 @@ Numeric
 - Bug in :class:`DataFrame` logical operations (`&`, `|`, `^`) not matching :class:`Series` behavior by filling NA values (:issue:`28741`)
 - Bug in :meth:`DataFrame.interpolate` where specifying axis by name references variable before it is assigned (:issue:`29142`)
 - Improved error message when using `frac` > 1 and `replace` = False (:issue:`27451`)
+- Bug in :class:`NumericIndex` resulted in it being possible to instantiate an :class:`Int64Index`, :class:`UInt64Index`, or :class:`Float64Index` with an invalid dtype (e.g. datetime-like) (:issue:`29539`)
 -
 
 Conversion
diff --git a/pandas/core/indexes/base.py b/pandas/core/indexes/base.py
index c9697c530628a..346b8bcf56260 100644
--- a/pandas/core/indexes/base.py
+++ b/pandas/core/indexes/base.py
@@ -380,7 +380,7 @@ def __new__(
                             pass
 
                         # Return an actual float index.
-                        return Float64Index(data, copy=copy, dtype=dtype, name=name)
+                        return Float64Index(data, copy=copy, name=name)
 
                     elif inferred == "string":
                         pass
diff --git a/pandas/core/indexes/numeric.py b/pandas/core/indexes/numeric.py
index 46bb8eafee3b9..78072ef2ea6d7 100644
--- a/pandas/core/indexes/numeric.py
+++ b/pandas/core/indexes/numeric.py
@@ -15,6 +15,8 @@
     is_float_dtype,
     is_integer_dtype,
     is_scalar,
+    is_signed_integer_dtype,
+    is_unsigned_integer_dtype,
     needs_i8_conversion,
     pandas_dtype,
 )
@@ -45,7 +47,7 @@ class NumericIndex(Index):
     _is_numeric_dtype = True
 
     def __new__(cls, data=None, dtype=None, copy=False, name=None, fastpath=None):
-
+        cls._validate_dtype(cls, dtype)
         if fastpath is not None:
             warnings.warn(
                 "The 'fastpath' keyword is deprecated, and will be "
@@ -72,6 +74,25 @@ def __new__(cls, data=None, dtype=None, copy=False, name=None, fastpath=None):
             name = data.name
         return cls._simple_new(subarr, name=name)
 
+    def _validate_dtype(cls, dtype):
+        if dtype is None:
+            return
+        if cls._typ == "int64index" and not is_signed_integer_dtype(dtype):
+            raise ValueError(
+                "Incorrect `dtype` passed: expected signed integer, received {}".format(
+                    dtype
+                )
+            )
+        elif cls._typ == "uint64index" and not is_unsigned_integer_dtype(dtype):
+            raise ValueError(
+                "Incorrect `dtype` passed: expected unsigned integer"
+                ", received {}".format(dtype)
+            )
+        elif cls._typ == "float64index" and not is_float_dtype(dtype):
+            raise ValueError(
+                "Incorrect `dtype` passed: expected float, received {}".format(dtype)
+            )
+
     @Appender(_index_shared_docs["_maybe_cast_slice_bound"])
     def _maybe_cast_slice_bound(self, label, side, kind):
         assert kind in ["ix", "loc", "getitem", None]
diff --git a/pandas/tests/indexes/test_numeric.py b/pandas/tests/indexes/test_numeric.py
index e424b3601a4b2..fdd4257466006 100644
--- a/pandas/tests/indexes/test_numeric.py
+++ b/pandas/tests/indexes/test_numeric.py
@@ -167,6 +167,22 @@ def test_constructor(self):
         result = Index(np.array([np.nan]))
         assert pd.isna(result.values).all()
 
+    @pytest.mark.parametrize(
+        "index, dtype",
+        [
+            (pd.Int64Index, "float64"),
+            (pd.UInt64Index, "categorical"),
+            (pd.Float64Index, "datetime64"),
+        ],
+    )
+    def test_invalid_dtype(self, index, dtype):
+        # GH 29539
+        with pytest.raises(
+            ValueError,
+            match=r"Incorrect `dtype` passed: expected .*, received {}".format(dtype),
+        ):
+            index([1, 2, 3], dtype=dtype)
+
     def test_constructor_invalid(self):
 
         # invalid
diff --git a/pandas/tests/series/indexing/test_numeric.py b/pandas/tests/series/indexing/test_numeric.py
index bcddcf843df06..60b89c01cc22d 100644
--- a/pandas/tests/series/indexing/test_numeric.py
+++ b/pandas/tests/series/indexing/test_numeric.py
@@ -86,8 +86,7 @@ def test_get():
                 1764.0,
                 1849.0,
                 1936.0,
-            ],
-            dtype="object",
+            ]
         ),
     )
 


Code review comments:
diff_hunk: @@ -72,6 +74,25 @@ def __new__(cls, data=None, dtype=None, copy=False, name=None, fastpath=None):
             name = data.name
         return cls._simple_new(subarr, name=name)
 
+    def _validate_dtype(cls, dtype):
+        if dtype is None:
+            return
+        if cls._typ == "int64index" and not is_signed_integer_dtype(dtype):
+            raise ValueError(
+                "Incorrect `dtype` passed: expected signed integer, received {}".format(
+                    dtype
+                )
+            )
+        elif cls._typ == "uint64index" and not is_unsigned_integer_dtype(dtype):
+            raise ValueError(
+                "Incorrect `dtype` passed: expected unsigned integer"
+                ", received {}".format(dtype)
+            )
+        elif cls._typ == "float64index" and not is_float_dtype(dtype):
+            raise ValueError(
+                "Incorrect `dtype` passed: expected float, received {}".format(dtype)
+            )
path: pandas/core/indexes/numeric.py
text: @user1:
Could reduce some code duplication here with a pattern like:
```python
validation_metadata = {
    "int64index": (is_signed_integer_dtype, "signed integer"),
    "uint64index": (is_unsigned_integer_dtype, "unsigned integer"),
    "float64index": (is_float_dtype, "float"),
}

validation_func, expected = validation_metadata[cls._type]
if not validation_func(dtype):
    msg = f"Incorrect `dtype` passed: expected {expected}, received {dtype}"
    raise ValueError(msg)
```
A little more concise but maybe a little less transparent.  Up to you if you want to make this change or not.  If not, then convert to f-strings for the error messages.

diff_hunk: @@ -336,6 +336,7 @@ Numeric
 - Bug in :class:`DataFrame` logical operations (`&`, `|`, `^`) not matching :class:`Series` behavior by filling NA values (:issue:`28741`)
 - Bug in :meth:`DataFrame.interpolate` where specifying axis by name references variable before it is assigned (:issue:`29142`)
 - Improved error message when using `frac` > 1 and `replace` = False (:issue:`27451`)
+- Bug in :class:`NumericIndex` resulted in it being possible to instantiate an :class:`Int64Index`, :class:`UInt64Index`, or :class:`Float64Index` with an invalid dtype (e.g. datetime-like) (:issue:`29539`)
path: doc/source/whatsnew/v1.0.0.rst
text: @user1:
I don't think `NumericIndex` is a public class, so maybe just say "numeric indexes" instead.

diff_hunk: @@ -72,6 +74,25 @@ def __new__(cls, data=None, dtype=None, copy=False, name=None, fastpath=None):
             name = data.name
         return cls._simple_new(subarr, name=name)
 
+    def _validate_dtype(cls, dtype):
path: pandas/core/indexes/numeric.py
text: @user1:
looks ok, can you type this

diff_hunk: @@ -45,7 +47,7 @@ class NumericIndex(Index):
     _is_numeric_dtype = True
 
     def __new__(cls, data=None, dtype=None, copy=False, name=None, fastpath=None):
-
+        cls._validate_dtype(cls, dtype)
path: pandas/core/indexes/numeric.py
text: @user2:
make this a ```@user1``` then you don't need to pass cls

diff_hunk: @@ -167,6 +167,22 @@ def test_constructor(self):
         result = Index(np.array([np.nan]))
         assert pd.isna(result.values).all()
 
+    @pytest.mark.parametrize(
+        "index, dtype",
+        [
+            (pd.Int64Index, "float64"),
+            (pd.UInt64Index, "categorical"),
+            (pd.Float64Index, "datetime64"),
+        ],
+    )
+    def test_invalid_dtype(self, index, dtype):
+        # GH 29539
+        with pytest.raises(
+            ValueError,
+            match=r"Incorrect `dtype` passed: expected .*, received {}".format(dtype),
path: pandas/tests/indexes/test_numeric.py
text: @user1:
This can be an f-string since 3.6 is the minimum Python version now.