Talip7/scikit-learn-issues-multilabel huggingface dataset:
Maintenance releases for 1.1.x and 1.2.x with numpy < 2.0?

### Describe the workflow you want to enable

Having an environment file or requirement file with scikit-learn=1.1 or scikit-learn=1.2 will break, since neither supports numpy 2.0 but doesn't declare that.
Example:

```bash
$ conda create -n sklearn_numpy_test python=3.9
$ conda activate sklearn_numpy_test
$ pip install scikit-learn==1.1
$ python -c "import sklearn.linear_model"
```
> ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

### Describe your proposed solution

We could do patch release for those two releases that add a numpy < 2.0 requirement to the setup.py. This is not something we've done historically much, I think, but it's not ideal if old requirement files break. If someone pinned the patch release, adding a patch release with a fix won't help, unfortunately.

COMMENT:
I have had problems installing Scikit Learn 1.2.2, but I think I have found the solution in Linux and Windows.
To install it properly under the docker Image python:3.12.2-bookworm (Debian) I did the following:

pip install -U pip && pip install setuptools wheel "Cython<1" "numpy<2" "scipy<1.14" "joblib<1.4" "threadpool<3" && pip wheel --no-build-isolation scikit-learn==1.2.2

Which built the wheel successfuly in the current directory.

In my Windows instalation I was able to install it using the flag:
CFLAGS="-fno-strict-aliasing -Wno-error=incompatible-pointer-types" pip install scikit-learn==1.2.2

Note: I tried installing it into an Alpine image too, but had problem with binaries, but to wheel the library, the previous flag was necesary too.

Hope that can help you guys!

Labels: New Feature
Maintenance releases for 1.1.x and 1.2.x with numpy < 2.0?

### Describe the workflow you want to enable

Having an environment file or requirement file with scikit-learn=1.1 or scikit-learn=1.2 will break, since neither supports numpy 2.0 but doesn't declare that.
Example:

```bash
$ conda create -n sklearn_numpy_test python=3.9
$ conda activate sklearn_numpy_test
$ pip install scikit-learn==1.1
$ python -c "import sklearn.linear_model"
```
> ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

### Describe your proposed solution

We could do patch release for those two releases that add a numpy < 2.0 requirement to the setup.py. This is not something we've done historically much, I think, but it's not ideal if old requirement files break. If someone pinned the patch release, adding a patch release with a fix won't help, unfortunately.

COMMENT:
I think we reached a conclusion (to not do this). So I'll go ahead and close this. (as usual, re-open if you disagree or we change our mind)

Labels: New Feature
plot_tree fails with ValueError Invalid RGBA argument

### Describe the bug

When using `plot_tree` with `filled=True` (so the nodes are colored), one sometimes gets a `ValueError` such as
```
Invalid RGBA argument: '#cb 3-8d'
```

The same `plot_tree` will work fine if `filled=False`, and draw a decision tree. Below is an example. I attach two files with the dump of the DecisionTree and the list of columns used.

Thank you.

### Steps/Code to Reproduce

```
# BUG: fails with ValueError: Invalid RGBA argument: '#cb 4-8c'
import joblib
from sklearn.tree import plot_tree

dt = joblib.load('_br.dmp')
cols = joblib.load('_cols.dmp')
plot_tree(dt,
          feature_names=cols,
          class_names=['Reject', 'Accept'],
          filled=True, rounded=True,
          impurity=True,
          label='root',
          )
```


[_br.dmp](https://github.com/user-attachments/files/16512624/_br.dmp)
[_cols.dmp](https://github.com/user-attachments/files/16512625/_cols.dmp)


### Expected Results

No ValueError should be thrown and the plot should be drawn and correctly colored.

### Actual Results

ValueError Invalid RGBA argument: '#cb 3-8d'

### Versions

```shell
System:
    python: 3.11.4 (main, Jun  7 2023, 12:45:48) [GCC 11.3.0]
executable: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/bin/python
   machine: Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.5.1
          pip: 24.2
   setuptools: 65.5.0
        numpy: 1.26.4
        scipy: 1.12.0
       Cython: None
       pandas: 2.2.2
   matplotlib: 3.9.1
       joblib: 1.3.2
threadpoolctl: 3.3.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
What is `_br.dmp` and `_cols.dmp`? It would help if you can share a MVP with the necessary code using DecisionTreeClassifier in sklearn since loading pickled data arbitrarily is insecure, so one will generally not download and run the code you attached.

Labels: Bug, Needs Triage
plot_tree fails with ValueError Invalid RGBA argument

### Describe the bug

When using `plot_tree` with `filled=True` (so the nodes are colored), one sometimes gets a `ValueError` such as
```
Invalid RGBA argument: '#cb 3-8d'
```

The same `plot_tree` will work fine if `filled=False`, and draw a decision tree. Below is an example. I attach two files with the dump of the DecisionTree and the list of columns used.

Thank you.

### Steps/Code to Reproduce

```
# BUG: fails with ValueError: Invalid RGBA argument: '#cb 4-8c'
import joblib
from sklearn.tree import plot_tree

dt = joblib.load('_br.dmp')
cols = joblib.load('_cols.dmp')
plot_tree(dt,
          feature_names=cols,
          class_names=['Reject', 'Accept'],
          filled=True, rounded=True,
          impurity=True,
          label='root',
          )
```


[_br.dmp](https://github.com/user-attachments/files/16512624/_br.dmp)
[_cols.dmp](https://github.com/user-attachments/files/16512625/_cols.dmp)


### Expected Results

No ValueError should be thrown and the plot should be drawn and correctly colored.

### Actual Results

ValueError Invalid RGBA argument: '#cb 3-8d'

### Versions

```shell
System:
    python: 3.11.4 (main, Jun  7 2023, 12:45:48) [GCC 11.3.0]
executable: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/bin/python
   machine: Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.5.1
          pip: 24.2
   setuptools: 65.5.0
        numpy: 1.26.4
        scipy: 1.12.0
       Cython: None
       pandas: 2.2.2
   matplotlib: 3.9.1
       joblib: 1.3.2
threadpoolctl: 3.3.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
@adam2392 thanks, the `_br.dmp` is the fitted `DecisionTreeClassifier` and the `_cols.dmp` is the array of feature names. This is a specific example, in full detail, that reproduces the issue. I am not sure how else to share it. Is there a different format that is better liked?

For example, do you know if it is possible to persist a fitted `DecisionTreeClassifier` with [JSON](https://docs.python.org/3/library/json.html)? If that works for you, I will do it.

Labels: Bug, Needs Triage
plot_tree fails with ValueError Invalid RGBA argument

### Describe the bug

When using `plot_tree` with `filled=True` (so the nodes are colored), one sometimes gets a `ValueError` such as
```
Invalid RGBA argument: '#cb 3-8d'
```

The same `plot_tree` will work fine if `filled=False`, and draw a decision tree. Below is an example. I attach two files with the dump of the DecisionTree and the list of columns used.

Thank you.

### Steps/Code to Reproduce

```
# BUG: fails with ValueError: Invalid RGBA argument: '#cb 4-8c'
import joblib
from sklearn.tree import plot_tree

dt = joblib.load('_br.dmp')
cols = joblib.load('_cols.dmp')
plot_tree(dt,
          feature_names=cols,
          class_names=['Reject', 'Accept'],
          filled=True, rounded=True,
          impurity=True,
          label='root',
          )
```


[_br.dmp](https://github.com/user-attachments/files/16512624/_br.dmp)
[_cols.dmp](https://github.com/user-attachments/files/16512625/_cols.dmp)


### Expected Results

No ValueError should be thrown and the plot should be drawn and correctly colored.

### Actual Results

ValueError Invalid RGBA argument: '#cb 3-8d'

### Versions

```shell
System:
    python: 3.11.4 (main, Jun  7 2023, 12:45:48) [GCC 11.3.0]
executable: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/bin/python
   machine: Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.5.1
          pip: 24.2
   setuptools: 65.5.0
        numpy: 1.26.4
        scipy: 1.12.0
       Cython: None
       pandas: 2.2.2
   matplotlib: 3.9.1
       joblib: 1.3.2
threadpoolctl: 3.3.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
I meant is there a way to share the relevant code that also fit the DTC? So we can just run it end to end without relying on a pickled file?

Labels: Bug, Needs Triage
plot_tree fails with ValueError Invalid RGBA argument

### Describe the bug

When using `plot_tree` with `filled=True` (so the nodes are colored), one sometimes gets a `ValueError` such as
```
Invalid RGBA argument: '#cb 3-8d'
```

The same `plot_tree` will work fine if `filled=False`, and draw a decision tree. Below is an example. I attach two files with the dump of the DecisionTree and the list of columns used.

Thank you.

### Steps/Code to Reproduce

```
# BUG: fails with ValueError: Invalid RGBA argument: '#cb 4-8c'
import joblib
from sklearn.tree import plot_tree

dt = joblib.load('_br.dmp')
cols = joblib.load('_cols.dmp')
plot_tree(dt,
          feature_names=cols,
          class_names=['Reject', 'Accept'],
          filled=True, rounded=True,
          impurity=True,
          label='root',
          )
```


[_br.dmp](https://github.com/user-attachments/files/16512624/_br.dmp)
[_cols.dmp](https://github.com/user-attachments/files/16512625/_cols.dmp)


### Expected Results

No ValueError should be thrown and the plot should be drawn and correctly colored.

### Actual Results

ValueError Invalid RGBA argument: '#cb 3-8d'

### Versions

```shell
System:
    python: 3.11.4 (main, Jun  7 2023, 12:45:48) [GCC 11.3.0]
executable: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/bin/python
   machine: Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.5.1
          pip: 24.2
   setuptools: 65.5.0
        numpy: 1.26.4
        scipy: 1.12.0
       Cython: None
       pandas: 2.2.2
   matplotlib: 3.9.1
       joblib: 1.3.2
threadpoolctl: 3.3.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
@adam2392 thanks for the effort but sorry - I am stumped. It's part of a large program, with many thousands of lines of code and >100,000 rows of data. Dumping the `_br.dmp` and `_cols.dmp` was my way of getting around the complexity.

Is there any other serialization format that would be acceptable?

Labels: Bug, Needs Triage
plot_tree fails with ValueError Invalid RGBA argument

### Describe the bug

When using `plot_tree` with `filled=True` (so the nodes are colored), one sometimes gets a `ValueError` such as
```
Invalid RGBA argument: '#cb 3-8d'
```

The same `plot_tree` will work fine if `filled=False`, and draw a decision tree. Below is an example. I attach two files with the dump of the DecisionTree and the list of columns used.

Thank you.

### Steps/Code to Reproduce

```
# BUG: fails with ValueError: Invalid RGBA argument: '#cb 4-8c'
import joblib
from sklearn.tree import plot_tree

dt = joblib.load('_br.dmp')
cols = joblib.load('_cols.dmp')
plot_tree(dt,
          feature_names=cols,
          class_names=['Reject', 'Accept'],
          filled=True, rounded=True,
          impurity=True,
          label='root',
          )
```


[_br.dmp](https://github.com/user-attachments/files/16512624/_br.dmp)
[_cols.dmp](https://github.com/user-attachments/files/16512625/_cols.dmp)


### Expected Results

No ValueError should be thrown and the plot should be drawn and correctly colored.

### Actual Results

ValueError Invalid RGBA argument: '#cb 3-8d'

### Versions

```shell
System:
    python: 3.11.4 (main, Jun  7 2023, 12:45:48) [GCC 11.3.0]
executable: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/bin/python
   machine: Linux-5.15.153.1-microsoft-standard-WSL2-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.5.1
          pip: 24.2
   setuptools: 65.5.0
        numpy: 1.26.4
        scipy: 1.12.0
       Cython: None
       pandas: 2.2.2
   matplotlib: 3.9.1
       joblib: 1.3.2
threadpoolctl: 3.3.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /mnt/c/Workspace/py-pip-workspaces/pert.ai.main/.venv/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
@giladpn unfortunately we avoid downloading user provided binaries / serialized formats of any kind. Hope you understand.

Therefore a minimal reproducible would be necessary for us to be able to dig deeper and write tests when we find the fix. Closing for now, and happy to reopen if you find a way to provide a minimal reproducer.

Labels: Bug, Needs Triage
Add optional return of STD for kNeighboursRegressor

### Describe the workflow you want to enable

I would like to propose to add the option to get the standard deviation from the KNeighborsRegressor. The `.predict()` function already delivers the mean, as that's the way the target is calculated, so adding the standard deviation should not be a big deal.

### Describe your proposed solution

Similar to the `.predict(return_std=False)` Function of the `GaussianProcessRegressor`, a switch for the `.predict()` function of the `KNeighborsRegressor` could be implemented.
This can easily be implemented by e.g. changing
```
            y_pred = np.mean(_y[neigh_ind], axis=1)
```
to
```
            y_pred = [np.mean(_y[neigh_ind], axis=1), np.std(_y[neigh_ind], axis=1)]
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Can you give a few more details on your use case?

I think the question is whether it is worth doing this, adding tests, documentation and maintaining it all.

A workaround (not the prettiest) is to use `knn.kneighbors()` and a private attribute `_y`. So you could make a small class that uses `KNeghborsRegressor` and has a custom `predict`

Labels: Enhancement, module:neighbors
Add optional return of STD for kNeighboursRegressor

### Describe the workflow you want to enable

I would like to propose to add the option to get the standard deviation from the KNeighborsRegressor. The `.predict()` function already delivers the mean, as that's the way the target is calculated, so adding the standard deviation should not be a big deal.

### Describe your proposed solution

Similar to the `.predict(return_std=False)` Function of the `GaussianProcessRegressor`, a switch for the `.predict()` function of the `KNeighborsRegressor` could be implemented.
This can easily be implemented by e.g. changing
```
            y_pred = np.mean(_y[neigh_ind], axis=1)
```
to
```
            y_pred = [np.mean(_y[neigh_ind], axis=1), np.std(_y[neigh_ind], axis=1)]
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
My personal use cases are scenarios, where I need to have some indications about the confidence/"error bars" on predictions. This, for example, could be an active learning scenario, where this influences the choice of new data points which should be added to the training set.
I can do a "private" workaround (as you suggested), but I thought this could be interesting for more people than just me.

Labels: Enhancement, module:neighbors
Add optional return of STD for kNeighboursRegressor

### Describe the workflow you want to enable

I would like to propose to add the option to get the standard deviation from the KNeighborsRegressor. The `.predict()` function already delivers the mean, as that's the way the target is calculated, so adding the standard deviation should not be a big deal.

### Describe your proposed solution

Similar to the `.predict(return_std=False)` Function of the `GaussianProcessRegressor`, a switch for the `.predict()` function of the `KNeighborsRegressor` could be implemented.
This can easily be implemented by e.g. changing
```
            y_pred = np.mean(_y[neigh_ind], axis=1)
```
to
```
            y_pred = [np.mean(_y[neigh_ind], axis=1), np.std(_y[neigh_ind], axis=1)]
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I wouldn't mind a PR adding tihs. We have it in other estimators as well, where a `return_std` lets `.predict` return the `std` values as well. We can follow the same pattern here. However, one thing I'm a bit less comfortable with `std` here, is that the number of neighbors can be really sparse and therefore the `std` can be quite noisy.

Curious what @ogrisel thinks here.

Labels: Enhancement, module:neighbors
Add optional return of STD for kNeighboursRegressor

### Describe the workflow you want to enable

I would like to propose to add the option to get the standard deviation from the KNeighborsRegressor. The `.predict()` function already delivers the mean, as that's the way the target is calculated, so adding the standard deviation should not be a big deal.

### Describe your proposed solution

Similar to the `.predict(return_std=False)` Function of the `GaussianProcessRegressor`, a switch for the `.predict()` function of the `KNeighborsRegressor` could be implemented.
This can easily be implemented by e.g. changing
```
            y_pred = np.mean(_y[neigh_ind], axis=1)
```
to
```
            y_pred = [np.mean(_y[neigh_ind], axis=1), np.std(_y[neigh_ind], axis=1)]
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Also if you set `n_neighbors` to 1, calculating std will have no sense.

On top of that
> Regarding the Nearest Neighbors algorithms, if it is found that two neighbors, neighbor k+1 and k, have identical distances but different labels, the results will depend on the ordering of the training data.

Does it mean that std will be different with the same parameters and will be dependent on the sorting?

Labels: Enhancement, module:neighbors
Add optional return of STD for kNeighboursRegressor

### Describe the workflow you want to enable

I would like to propose to add the option to get the standard deviation from the KNeighborsRegressor. The `.predict()` function already delivers the mean, as that's the way the target is calculated, so adding the standard deviation should not be a big deal.

### Describe your proposed solution

Similar to the `.predict(return_std=False)` Function of the `GaussianProcessRegressor`, a switch for the `.predict()` function of the `KNeighborsRegressor` could be implemented.
This can easily be implemented by e.g. changing
```
            y_pred = np.mean(_y[neigh_ind], axis=1)
```
to
```
            y_pred = [np.mean(_y[neigh_ind], axis=1), np.std(_y[neigh_ind], axis=1)]
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
With respect to n_neighbors = 1, you are correct. But I guess when a person deliberately chooses `n_neighbors = 1`, that person will know the implications of calculation the std (and mean!) for the unknown label.

The ordering affects not only the std, but also the mean/the predicted label.

Labels: Enhancement, module:neighbors