Talip7/scikit-learn-issues-multilabel huggingface dataset:
Support pandas nullable dtypes for scoring metrics

### Describe the workflow you want to enable

I would like to be able to pass data with the nullable pandas dtypes (`Int64`, `Float64`, and `boolean`) into sklearn metrics such as `matthews_corrcoef`, `accuracy_score`, and `f1_score` (and more) even if the data does not contain any nans. Currently, they result in one of several errors:

- If `y_true` and `y_pred` are both nullable types: `ValueError: unknown is not supported`
-  it only one of `y_true` or `y_pred` is nullable and the other is non nullable : `ValueError: Classification metrics can't handle a mix of unknown and binary [or multiclass] targets`
- Some metrics such as `log_loss` result in a different error when `y_true` is nullable:  `ValueError: Unknown label type: (0    1`

Repro with sklearn 1.2.1 and pandas 1.5.3:

```python
    import pandas as pd
    import pytest
    from sklearn import metrics

    for dtype in ['Int64', 'Float64', 'boolean']:
        # Error if only target uses nullable types
        X = pd.DataFrame({"a": pd.Series([1, 2, 3, 4]), 
                          "b": pd.Series([9,8,7,6])})

        # Two nullable dtypes used 
        y_true = pd.Series([1, 0, 1, 0], dtype=dtype)
        y_predicted = pd.Series([1, 0, 1, 0], dtype=dtype)
        with pytest.raises(ValueError, match="unknown is not supported"):
            metrics.accuracy_score(
                    y_true,
                    y_predicted,
                )

        # Only one nullable dtype used 
        y_predicted = pd.Series([1, 0, 1, 0], dtype="float64")
        with pytest.raises(ValueError, match="Classification metrics can't handle a mix of unknown and binary targets"):
            metrics.accuracy_score(
                    y_true,
                    y_predicted,
                )
``` 



### Describe your proposed solution

Sklearn should recognize the pandas nullable dtypes as the correct type of target for their scoring metrics like it does with the non nullable dtypes.

### Describe alternatives you've considered, if relevant

As this data doesn't have null values, we can convert to the non nullable dtype prior to passing to sklearn, but doing that will make it cumbersome to build software that leverages both the latest pandas dtypes and sklearn.

### Additional context

_No response_

COMMENT:
Currently, https://github.com/scikit-learn/scikit-learn/pull/25638 is targeted for `v1.3`, but I think we can consider it for a bug patch release.

WDYT @lorentzenchr @jeremiedbb ?

Labels: New Feature, Needs Triage
Support pandas nullable dtypes for scoring metrics

### Describe the workflow you want to enable

I would like to be able to pass data with the nullable pandas dtypes (`Int64`, `Float64`, and `boolean`) into sklearn metrics such as `matthews_corrcoef`, `accuracy_score`, and `f1_score` (and more) even if the data does not contain any nans. Currently, they result in one of several errors:

- If `y_true` and `y_pred` are both nullable types: `ValueError: unknown is not supported`
-  it only one of `y_true` or `y_pred` is nullable and the other is non nullable : `ValueError: Classification metrics can't handle a mix of unknown and binary [or multiclass] targets`
- Some metrics such as `log_loss` result in a different error when `y_true` is nullable:  `ValueError: Unknown label type: (0    1`

Repro with sklearn 1.2.1 and pandas 1.5.3:

```python
    import pandas as pd
    import pytest
    from sklearn import metrics

    for dtype in ['Int64', 'Float64', 'boolean']:
        # Error if only target uses nullable types
        X = pd.DataFrame({"a": pd.Series([1, 2, 3, 4]), 
                          "b": pd.Series([9,8,7,6])})

        # Two nullable dtypes used 
        y_true = pd.Series([1, 0, 1, 0], dtype=dtype)
        y_predicted = pd.Series([1, 0, 1, 0], dtype=dtype)
        with pytest.raises(ValueError, match="unknown is not supported"):
            metrics.accuracy_score(
                    y_true,
                    y_predicted,
                )

        # Only one nullable dtype used 
        y_predicted = pd.Series([1, 0, 1, 0], dtype="float64")
        with pytest.raises(ValueError, match="Classification metrics can't handle a mix of unknown and binary targets"):
            metrics.accuracy_score(
                    y_true,
                    y_predicted,
                )
``` 



### Describe your proposed solution

Sklearn should recognize the pandas nullable dtypes as the correct type of target for their scoring metrics like it does with the non nullable dtypes.

### Describe alternatives you've considered, if relevant

As this data doesn't have null values, we can convert to the non nullable dtype prior to passing to sklearn, but doing that will make it cumbersome to build software that leverages both the latest pandas dtypes and sklearn.

### Additional context

_No response_

COMMENT:
I‘m not the release manager. If we do a further bugfix release, why not.

Labels: New Feature, Needs Triage
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
I am +1 on stopping the use of `DTYPE` and `ITYPE` aliases whose meaning is not well defined. When a given piece of code works for a given precision, let's directly use the explicit Cython names `cnp.float64_t` or `cnp.float32_t`.

When this is a fused type to work with several precision levels, then I think our usual `floating` convention is fine, but we could change it when we want to make it explicit that this a numerical feature values (or result of arithmetic computations derived from them and internal model parameters that typically use the same type).

For tempita variable names, maybe we could also reuse the `floating` name (if it does not conflict with the fused type name in the same Cython source file). Alternatively we could use `float_precision` in `{32, 64}` as template variable and the use `cnp.float{float_precision}_t` for the type declarations.

For integers, when they are used for data in our Cython code, they are mostly used as contiguous category or class indices among a precomputed list of possible values, typically the `classes_` fit attribute computed by a `LabelEncoder` or similarly the outcome of calling `OrdinalEncoder` on categorical features. But they might also be used to represent count data (e.g. bin counts in HGBDT histograms & in `KBinsDiscretizer` in text / dict / hashing vectorizers).

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
I am against using type names with implicit precision levels (`float`, `double`, `int`, `long`...) as it's harder to reason about and less explicit whether or not the code behavior ( numerical precision of the computation, maximum count values before integer overflows) becomes platform dependent as a result.

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
I also want to stop using all the `DTYPE`, `ITYPE`, etc. aliases and use names that represents the precision.

My preference is to avoid `cimporting` numpy when we can avoid it and used the `C` types directly. Functionally, that means copying the typedefs we need from `numpy` [defined here](https://github.com/numpy/numpy/blob/6dadb8c40451e934075904f6acdfe341d3b8762e/numpy/__init__.cython-30.pxd#L324-L325), but with our names:

```python
# sklearn/utils/_typedefs.pyd

ctypedef Py_ssize_t         intp_t
ctypedef unsigned char      bool_t
ctypedef signed int         int32_t
ctypedef signed long long   int64_t
ctypedef float              float32_t
ctypedef double             float64_t
# etc
```

and use those typedefs everywhere. This means our Cython code would look like:

```python
import numpy as np
from sklearn.utils._typedefs cimport float32_t

cdef float32_t[:] X = np.zeros(10, dtype=np.float32)
```

I like having the precision levels explicit in the type, but I also want to avoid including NumPy headers just for the types. The cost is copying some of the typedefs from NumPy.

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
I agree with what @ogrisel and @thomasjpfan propose.

In `sklearn.tree._tree`, types definitions are misleading as some names are clashing with other types definitions somewhere else. For instance, `DTYPE_t` in this module is `float`:

https://github.com/scikit-learn/scikit-learn/blob/99562100e941f0972a5a65484ff80f407eeb5137/sklearn/tree/_tree.pxd#L16-L20

Yet, it is `double` in all other places (see the [the global search in the codebase](https://github.com/search?q=repo%3Ascikit-learn%2Fscikit-learn%20ctypedef%20cnp.float64_t%20DTYPE_t&type=code)).

Moreover, should we create our own fused types?

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
I think we have an agreement with @thomasjpfan proposal in https://github.com/scikit-learn/scikit-learn/issues/25572#issuecomment-1426159759. Now, we need a volunteer...

One remark about predefined fused types like `cython.floating`. This works fine as long as everything has the same type. But consider
```cython
def fun(floating a[:], floating b[:]):
    cdef:
        int i
        int n = a.shape[0]
    for i in range(n):
        a[i] += b[i]
```
This enforces `a` and `b` to always have the same (d)type. And there are situations where we do not want that.

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
For fused types, I prefer using Cython's `floating` when we can. But as mentioned in https://github.com/scikit-learn/scikit-learn/issues/25572#issuecomment-1442541931 there are cases where we have to create our own.

For commonly fused types, I can see doing something like scikit-image:

https://github.com/scikit-image/scikit-image/blob/main/skimage/_shared/fused_numerics.pxd

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
I agree with @thomasjpfan's proposal, and regarding fused types I'd like to echo my sentiment from a prior discussion: https://github.com/scikit-learn/scikit-learn/pull/24153#issuecomment-1291234002

TL;DR
Explicit is better than implicit. Explicit library-level aliases as suggested by @thomasjpfan are very helpful. I am not a fan of working with custom library-defined fused types. I think we should avoid making global fused types and instead defer to creating per-file/module fused types as needed.

With that being said, I think Cython standard fused types (e.g. `floating`) are fine since they're usually pretty helpful, and very obvious.

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
I'd also be glad to get rid of all the DTYPE_t and friends.
I opened https://github.com/scikit-learn/scikit-learn/pull/25739 to start the work.

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
Note that we won't be able to complete this until we require cython 3 due to a bug I just found where cython doesn't properly infer types:
```py
%%cython
cimport numpy as cnp

ctypedef Py_ssize_t intp_t  # like npy_intp and intp_t are defined in numpy

cdef void f(cnp.npy_intp* x):
    return x

cdef intp_t a = 0

f(&a)  # cython error: Cannot assign type 'intp_t *' to 'npy_intp *'
```
The function expects pointer to cnp.npy_intp, but we pass pointer to intp_t and it crashes although they are defined the same way.
It does work if we don't use pointers, and it does work if the function expects a pointer to a cnp.intp_t

I checked and it's fixed in the master branch of cython, but not in the 0.29.x branch.
We can however keep using the numpy types in the mean time where the problem occurs (in ``_binary_tree`` for instance)

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
I opened #25942 to finalize the removal of old defined types in utils._typedefs.

The next step will be to use the new defined types in utils._typedefs in cython modules that used to not import types from there but defined DTYPE_t, ITYPE_t, ... locally.

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
What's left to do? A short entry in the dev doc? Anything else?

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
There are still several files that use custom typedefs defined locally that should now use the types defined in `_typedefs`.
We also should have an entry in the dev docs indeed.

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
Some are useful. For HGBT, for instance, you can easily play around with the effect of different types only because it uses its own typedefs.

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
I noticed this when building a downstream application that leverages Cython code from scikit-learn. 

In https://github.com/scikit-learn/scikit-learn/pull/27352, I changed `cnp.npy_uint32 (i.e. UINT32_t)` to `unsigned int uint32_t` from _typedefs.pxd. However, I think this may not be valid on specifically Windows machines.

https://numpy.org/doc/stable/reference/arrays.scalars.html#numpy.uintc
https://stackoverflow.com/questions/76155091/np-uint32-np-uintc-on-windows

It seems that numpy's uint32 is an alias for either `uintc (aka unsigned int)` on Mac/Linux, which is fine for the new code, but is an alias for `uint (aka unsigned long)` on Windows. Lmk if my analysis is incorrect? I can propose a few solutions to fix this.

cc: @lorentzenchr and @jjerphan

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
`uint32_t` is `unsigned int` everywhere:
 - https://github.com/cython/cython/blob/master/Cython/Includes/numpy/__init__.pxd#L745
 - https://github.com/cython/cython/blob/master/Cython/Includes/numpy/__init__.pxd#L324

Have you observed problems, @adam2392?

Labels: RFC, cython
RFC Guideline for usage of Cython types

## Goal
Have a documented consensus on which types to use in Cython code.

### Types
We should distinguish between floating point numbers and integers. We may also split the use cases of integers: As data value and as index for pointers and memoryviews.

## Linked issues
#24153, https://github.com/scikit-learn/scikit-learn/pull/25555#discussion_r1100738562, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r926716129, https://github.com/scikit-learn/scikit-learn/pull/23865#discussion_r941346271

Also note the Cython bug with `const` types: https://github.com/cython/cython/issues/5230

COMMENT:
The above links no longer work with the latest version of Cython.

But I found:

https://github.com/cython/cython/blob/3.0.9/Cython/Includes/libc/stdint.pxd#L15

and on numpy's side:

https://github.com/numpy/numpy/blob/v1.26.4/numpy/__init__.pxd#L280

which are both consistent with aliasing `unsigned int` as `uint32_t` as we do in `_typedefs.pxd`.

Labels: RFC, cython