Repository: scikit-learn/scikit-learn
Pull request: ENH Allows target to be pandas nullable dtypes
<!--
Thanks for contributing a pull request! Please ensure you have taken a look at
the contribution guidelines: https://github.com/scikit-learn/scikit-learn/blob/main/CONTRIBUTING.md
-->

#### Reference Issues/PRs
<!--
Example: Fixes #1234. See also #3456.
Please use keywords (e.g., Fixes) to create link to the issues or pull requests
you resolved, so that they will automatically be closed when your pull request
is merged. See https://github.com/blog/1506-closing-issues-via-pull-requests
-->
Closes #25637
Closes #25635
Closes #25634

#### What does this implement/fix? Explain your changes.
This PR makes use of `check_array` to convert `y` in the `multiclass` module. This allows `type_of_target` and it's callers to handle pandas nullable dtypes.

<!--
Please be aware that we are a loose team of volunteers so patience is
necessary; assistance handling other issues is very welcome. We value
all user contributions, no matter how minor they are. If we are slow to
review, either the pull request needs some benchmarking, tinkering,
convincing, etc. or more likely the reviewers are simply busy. In either
case, we ask for your understanding during the review process.
For more information, see our FAQ on this topic:
http://scikit-learn.org/dev/faq.html#why-is-my-pull-request-not-getting-any-attention.

Thanks for contributing!
-->


Problem statement:
Support nullable pandas dtypes in `unique_labels`
### Describe the workflow you want to enable

I would like to be able to pass the nullable pandas dtypes ("Int64", "Float64", "boolean") into sklearn's `unique_labels` function. Because the dtypes become `object` dtype when converted to numpy arrays we get `ValueError: Mix type of y not allowed, got types {'binary', 'unknown'}`:

Repro with sklearn 1.2.1
```py 
    import pandas as pd
    import pytest
    from sklearn.utils.multiclass import unique_labels
    
    for dtype in ["Int64", "Float64", "boolean"]:
        y_true = pd.Series([1, 0, 0, 1, 0, 1, 1, 0, 1], dtype=dtype)
        y_predicted = pd.Series([0, 0, 1, 1, 0, 1, 1, 1, 1], dtype="int64")

        with pytest.raises(ValueError, match="Mix type of y not allowed, got types"):
            unique_labels(y_true, y_predicted)
```

### Describe your proposed solution

We should get the same behavior as when `int64`, `float64`, and `bool` dtypes are used, which is no error:  

```python
    import pandas as pd
    from sklearn.utils.multiclass import unique_labels
    
    for dtype in ["int64", "float64", "bool"]:
        y_true = pd.Series([1, 0, 0, 1, 0, 1, 1, 0, 1], dtype=dtype)
        y_predicted = pd.Series([0, 0, 1, 1, 0, 1, 1, 1, 1], dtype="int64")

        unique_labels(y_true, y_predicted)
```

### Describe alternatives you've considered, if relevant

Our current workaround is to convert the data to numpy arrays with the corresponding dtype that works prior to passing it into `unique_labels`.

### Additional context

_No response_

Support nullable pandas dtypes in `confusion_matrix`
### Describe the workflow you want to enable

I would like to be able to pass the nullable pandas dtypes ("Int64", "Float64", "boolean") into sklearn's `confusion_matrix` function. Because the dtypes become object dtype when converted to numpy arrays we get `ValueError: Classification metrics can't handle a mix of unknown and binary targets`:

Repro with sklearn 1.2.1:
```python
    import pandas as pd
    import pytest
    from sklearn.metrics import confusion_matrix

    for dtype in ["Int64", "Float64", "boolean"]:
        y_true = pd.Series([1, 0, 0, 1, 0, 1, 1, 0, 1], dtype=dtype)
        y_predicted = pd.Series([0, 0, 1, 1, 0, 1, 1, 1, 1], dtype="int64")

        with pytest.raises(ValueError, match="Classification metrics can't handle a mix of unknown and binary targets"):
            confusion_matrix(y_true, y_predicted)
```

### Describe your proposed solution

We should get the same behavior as when int64, float64, and bool dtypes are used, which is no error:

```python
    import pandas as pd
    from sklearn.metrics import confusion_matrix

    for dtype in ["int64", "float64", "bool"]:
        y_true = pd.Series([1, 0, 0, 1, 0, 1, 1, 0, 1], dtype=dtype)
        y_predicted = pd.Series([0, 0, 1, 1, 0, 1, 1, 1, 1], dtype="int64")

        confusion_matrix(y_true, y_predicted)
```

### Describe alternatives you've considered, if relevant

Our current workaround is to convert the data to numpy arrays with the corresponding dtype that works prior to passing it into `confusion_matrix`

### Additional context

_No response_

Support nullable pandas dtypes in LabelBinarizer
### Describe the workflow you want to enable

I would like to be able to pass the nullable pandas dtypes ("Int64", "Float64", "boolean") into sklearn's LabelBinarizer. Because the dtypes become object dtype when converted to numpy arrays we get `ValueError: Unknown label type:`:

Repro with sklearn 1.2.1:

```python
    import pandas as pd
    import pytest
    from sklearn.preprocessing import LabelBinarizer

    for dtype in ["Int64", "Float64", "boolean"]:

        y_true = pd.Series([1, 0, 0, 1, 0, 1, 1, 0, 1], dtype=dtype)

        lb = LabelBinarizer()

        with pytest.raises(ValueError, match="Unknown label type:"):
            lb.fit(y_true.unique())

```

### Describe your proposed solution

We should get the same behavior as when int64, float64, and bool dtypes are used, which is no error:

```python
    import pandas as pd
    from sklearn.preprocessing import LabelBinarizer

    for dtype in ["int64", "float64", "bool"]:
        y_true = pd.Series([1, 0, 0, 1, 0, 1, 1, 0, 1], dtype=dtype)

        lb = LabelBinarizer()

        lb.fit(y_true.unique())
        y_one_hot_true = lb.transform(y_true)
```

### Describe alternatives you've considered, if relevant

Our current workaround is to convert the data to numpy arrays with the corresponding dtype that works prior to passing it into LabelBinarizer

### Additional context

_No response_

Hints from issue discussion:
Thank you for opening the issue! The underlying issue comes from how pandas exposes `__array__` to be object dtypes with nullable dtypes:

```python
import pandas as pd

y_true = pd.Series([1, 0, 0, 1, 0, 1, 1, 0, 1], dtype="Int64")
print(np.asarray(y_true).dtype)
# object

y_pred = pd.Series([1, 0, 0, 1, 0, 1, 1, 0, 1], dtype="int64")
print(np.asarray(y_pred).dtype)
# int64
```

Scikit-learn already has `check_array` that handles converting pandas nullable dtypes into their corresponding NumPy dytpes, so I opened #25638 to use `check_array` to fix this issue.

As noted in https://github.com/scikit-learn/scikit-learn/issues/25634#issuecomment-1435341844, I opened #25638 to resolve this issue.
As noted in https://github.com/scikit-learn/scikit-learn/issues/25634#issuecomment-1435341844, I opened #25638 to resolve this issue.

Commit message: DOC Adds PR number

Patch to review:
diff --git a/doc/whats_new/v1.3.rst b/doc/whats_new/v1.3.rst
index 08ebf4abc92c3..2af49651b8a8b 100644
--- a/doc/whats_new/v1.3.rst
+++ b/doc/whats_new/v1.3.rst
@@ -222,6 +222,9 @@ Changelog
   during `transform` with no prior call to `fit` or `fit_transform`.
   :pr:`25190` by :user:`Vincent Maladière <Vincent-Maladiere>`.
 
+- |Enhancement| :func:`utils.multiclass.type_of_target` can identify Pandas
+  nullable data types as classification targets. :pr:`25638` by `Thomas Fan`_.
+
 :mod:`sklearn.semi_supervised`
 ..............................
 
diff --git a/sklearn/metrics/tests/test_classification.py b/sklearn/metrics/tests/test_classification.py
index 704b79add5414..62d67bef1f00b 100644
--- a/sklearn/metrics/tests/test_classification.py
+++ b/sklearn/metrics/tests/test_classification.py
@@ -1079,6 +1079,24 @@ def test_confusion_matrix_dtype():
     assert cm[1, 1] == -2
 
 
+@pytest.mark.parametrize("dtype", ["Int64", "Float64", "boolean"])
+def test_confusion_matrix_pandas_nullable(dtype):
+    """Checks that confusion_matrix works with pandas nullable dtypes.
+
+    Non-regression test for gh-25635.
+    """
+    pd = pytest.importorskip("pandas")
+
+    y_ndarray = np.array([1, 0, 0, 1, 0, 1, 1, 0, 1])
+    y_true = pd.Series(y_ndarray, dtype=dtype)
+    y_predicted = pd.Series([0, 0, 1, 1, 0, 1, 1, 1, 1], dtype="int64")
+
+    output = confusion_matrix(y_true, y_predicted)
+    expected_output = confusion_matrix(y_ndarray, y_predicted)
+
+    assert_array_equal(output, expected_output)
+
+
 def test_classification_report_multiclass():
     # Test performance report
     iris = datasets.load_iris()
diff --git a/sklearn/preprocessing/tests/test_label.py b/sklearn/preprocessing/tests/test_label.py
index bd5ba6d2da9ee..b90218a97016b 100644
--- a/sklearn/preprocessing/tests/test_label.py
+++ b/sklearn/preprocessing/tests/test_label.py
@@ -117,6 +117,22 @@ def test_label_binarizer_set_label_encoding():
     assert_array_equal(lb.inverse_transform(got), inp)
 
 
+@pytest.mark.parametrize("dtype", ["Int64", "Float64", "boolean"])
+def test_label_binarizer_pandas_nullable(dtype):
+    """Checks that LabelBinarizer works with pandas nullable dtypes.
+
+    Non-regression test for gh-25637.
+    """
+    pd = pytest.importorskip("pandas")
+    from sklearn.preprocessing import LabelBinarizer
+
+    y_true = pd.Series([1, 0, 0, 1, 0, 1, 1, 0, 1], dtype=dtype)
+    lb = LabelBinarizer().fit(y_true)
+    y_out = lb.transform([1, 0])
+
+    assert_array_equal(y_out, [[1], [0]])
+
+
 @ignore_warnings
 def test_label_binarizer_errors():
     # Check that invalid arguments yield ValueError
diff --git a/sklearn/utils/multiclass.py b/sklearn/utils/multiclass.py
index 5eaef2fde87e4..4cce3a67fa443 100644
--- a/sklearn/utils/multiclass.py
+++ b/sklearn/utils/multiclass.py
@@ -155,14 +155,25 @@ def is_multilabel(y):
     if hasattr(y, "__array__") or isinstance(y, Sequence) or is_array_api:
         # DeprecationWarning will be replaced by ValueError, see NEP 34
         # https://numpy.org/neps/nep-0034-infer-dtype-is-object.html
+        check_y_kwargs = dict(
+            accept_sparse=True,
+            allow_nd=True,
+            force_all_finite=False,
+            ensure_2d=False,
+            ensure_min_samples=0,
+            ensure_min_features=0,
+        )
         with warnings.catch_warnings():
             warnings.simplefilter("error", np.VisibleDeprecationWarning)
             try:
-                y = xp.asarray(y)
-            except (np.VisibleDeprecationWarning, ValueError):
+                y = check_array(y, dtype=None, **check_y_kwargs)
+            except (np.VisibleDeprecationWarning, ValueError) as e:
+                if str(e).startswith("Complex data not supported"):
+                    raise
+
                 # dtype=object should be provided explicitly for ragged arrays,
                 # see NEP 34
-                y = xp.asarray(y, dtype=object)
+                y = check_array(y, dtype=object, **check_y_kwargs)
 
     if not (hasattr(y, "shape") and y.ndim == 2 and y.shape[1] > 1):
         return False
@@ -180,7 +191,7 @@ def is_multilabel(y):
         labels = xp.unique_values(y)
 
         return len(labels) < 3 and (
-            y.dtype.kind in "biu" or _is_integral_float(labels)  # bool, int, uint
+            y.dtype.kind in "Obiu" or _is_integral_float(labels)  # bool, int, uint
         )
 
 
@@ -302,15 +313,27 @@ def type_of_target(y, input_name=""):
     # https://numpy.org/neps/nep-0034-infer-dtype-is-object.html
     # We therefore catch both deprecation (NumPy < 1.24) warning and
     # value error (NumPy >= 1.24).
+    check_y_kwargs = dict(
+        accept_sparse=True,
+        allow_nd=True,
+        force_all_finite=False,
+        ensure_2d=False,
+        ensure_min_samples=0,
+        ensure_min_features=0,
+    )
+
     with warnings.catch_warnings():
         warnings.simplefilter("error", np.VisibleDeprecationWarning)
         if not issparse(y):
             try:
-                y = xp.asarray(y)
-            except (np.VisibleDeprecationWarning, ValueError):
+                y = check_array(y, dtype=None, **check_y_kwargs)
+            except (np.VisibleDeprecationWarning, ValueError) as e:
+                if str(e).startswith("Complex data not supported"):
+                    raise
+
                 # dtype=object should be provided explicitly for ragged arrays,
                 # see NEP 34
-                y = xp.asarray(y, dtype=object)
+                y = check_array(y, dtype=object, **check_y_kwargs)
 
     # The old sequence of sequences format
     try:
@@ -340,8 +363,10 @@ def type_of_target(y, input_name=""):
             return "binary"  # []
         # 2-D empty array: [[]]
         return "unknown"
-    if not issparse(y) and y.dtype == object and not isinstance(y.flat[0], str):
-        # [obj_1] and not ["label_1"]
+
+    y_is_sparse = issparse(y)
+    if not y_is_sparse and y.dtype == object and not isinstance(y.flat[0], str):
+        # [obj_1] and not ["label_1"] or not [1] or not [1.0]
         return "unknown"
 
     # Check if multioutput
diff --git a/sklearn/utils/tests/test_multiclass.py b/sklearn/utils/tests/test_multiclass.py
index cf5858d0f52f9..731edbefc3925 100644
--- a/sklearn/utils/tests/test_multiclass.py
+++ b/sklearn/utils/tests/test_multiclass.py
@@ -346,6 +346,42 @@ def test_type_of_target_pandas_sparse():
         type_of_target(y)
 
 
+def test_type_of_target_pandas_nullable():
+    """Check that type_of_target works with pandas nullable dtypes."""
+    pd = pytest.importorskip("pandas")
+
+    for dtype in ["Int32", "Float32"]:
+        y_true = pd.Series([1, 0, 2, 3, 4], dtype=dtype)
+        assert type_of_target(y_true) == "multiclass"
+
+        y_true = pd.Series([1, 0, 1, 0], dtype=dtype)
+        assert type_of_target(y_true) == "binary"
+
+    y_true = pd.DataFrame([[1.4, 3.1], [3.1, 1.4]], dtype="Float32")
+    assert type_of_target(y_true) == "continuous-multioutput"
+
+    y_true = pd.DataFrame([[0, 1], [1, 1]], dtype="Int32")
+    assert type_of_target(y_true) == "multilabel-indicator"
+
+    y_true = pd.DataFrame([[1, 2], [3, 1]], dtype="Int32")
+    assert type_of_target(y_true) == "multiclass-multioutput"
+
+
+@pytest.mark.parametrize("dtype", ["Int64", "Float64", "boolean"])
+def test_unique_labels_pandas_nullable(dtype):
+    """Checks that unique_labels work with pandas nullable dtypes.
+
+    Non-regression test for gh-25634.
+    """
+    pd = pytest.importorskip("pandas")
+
+    y_true = pd.Series([1, 0, 0, 1, 0, 1, 1, 0, 1], dtype=dtype)
+    y_predicted = pd.Series([0, 0, 1, 1, 0, 1, 1, 1, 1], dtype="int64")
+
+    labels = unique_labels(y_true, y_predicted)
+    assert_array_equal(labels, [0, 1])
+
+
 def test_class_distribution():
     y = np.array(
         [


Code review comments:
diff_hunk: @@ -340,8 +363,10 @@ def type_of_target(y, input_name=""):
             return "binary"  # []
         # 2-D empty array: [[]]
         return "unknown"
-    if not issparse(y) and y.dtype == object and not isinstance(y.flat[0], str):
-        # [obj_1] and not ["label_1"]
+
+    y_is_sparse = issparse(y)
+    if not y_is_sparse and y.dtype == object and not isinstance(y.flat[0], str):
path: sklearn/utils/multiclass.py
text: @user1:
I don't see the benefit of defining this intermediate variable
```suggestion
    if not issparse(y) and y.dtype == object and not isinstance(y.flat[0], str):
```

@author:
Ah, this was a mistake, while I was experimenting with another approach. I'll revert the diff.

diff_hunk: @@ -180,7 +191,7 @@ def is_multilabel(y):
         labels = xp.unique_values(y)
 
         return len(labels) < 3 and (
-            y.dtype.kind in "biu" or _is_integral_float(labels)  # bool, int, uint
+            y.dtype.kind in "Obiu" or _is_integral_float(labels)  # bool, int, uint
path: sklearn/utils/multiclass.py
text: @user1:
Is this necessary ? I don't think it's impacted by the pandas dtypes thing.

@author:
Yea, this is not needed. This was left over from when I was experimenting with another approach. I reverted the change.