Talip7/scikit-learn-issues-multilabel huggingface dataset:
Small discrepancy with not implemented MPS backend function

The discrepancy can be reproduced with the following snippet:

```python
# https://docs.scipy.org/doc/scipy/dev/api-dev/array_api.html
import os
os.environ["SCIPY_ARRAY_API"] = "1"
# os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"

import torch

import numpy as np
from sklearn.datasets import load_diabetes
from sklearn import config_context
from sklearn.pipeline import make_pipeline
from sklearn.decomposition import PCA
from sklearn.linear_model import Ridge
from sklearn.model_selection import cross_validate

X, y = load_diabetes(return_X_y=True)
X, y = X.astype(np.float32), y.astype(np.float32)

assert torch.backends.mps.is_available()
mps_device = torch.device("mps")
X = torch.from_numpy(X).to(device=mps_device)
y = torch.from_numpy(y).to(device=mps_device)

model = make_pipeline(PCA(svd_solver="full"), Ridge(solver="svd"))
with config_context(array_api_dispatch=True):
    cv_results = cross_validate(model, X, y)
cv_results
```

When using the following model:

```
model = make_pipeline(PCA(svd_solver="full"), Ridge(solver="svd"))
```

PyTorch will call `svd` solver and this is not available for MPS. In this case, it will fallback **automatically** on CPU and raise a warning.

However, for the following model:

```
model = make_pipeline(PCA(svd_solver="covariance_eigh"), Ridge(solver="svd"))
```

PyTorch will call `eigh` but this is also not supported for the MPS backend. However, in this case, scikit-learn is raising an error and request to explicitly set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK`.

This is slightly weird that we are to consistent on how to treat those similar cases. We should check what is the reason and decide if we prefer to raise a warning or an error.

COMMENT:
I don't think we should do anything in scikit-learn. This is a problem for PyTorch itself and we should refrain from trying to workaround it in scikit-learn otherwise we would have to increase source code complexity for a library specific concern.

As MPS support improves in PyTorch, this problem will hopefully disappear in a future PyTorch release.

Labels: Array API
Small discrepancy with not implemented MPS backend function

The discrepancy can be reproduced with the following snippet:

```python
# https://docs.scipy.org/doc/scipy/dev/api-dev/array_api.html
import os
os.environ["SCIPY_ARRAY_API"] = "1"
# os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"

import torch

import numpy as np
from sklearn.datasets import load_diabetes
from sklearn import config_context
from sklearn.pipeline import make_pipeline
from sklearn.decomposition import PCA
from sklearn.linear_model import Ridge
from sklearn.model_selection import cross_validate

X, y = load_diabetes(return_X_y=True)
X, y = X.astype(np.float32), y.astype(np.float32)

assert torch.backends.mps.is_available()
mps_device = torch.device("mps")
X = torch.from_numpy(X).to(device=mps_device)
y = torch.from_numpy(y).to(device=mps_device)

model = make_pipeline(PCA(svd_solver="full"), Ridge(solver="svd"))
with config_context(array_api_dispatch=True):
    cv_results = cross_validate(model, X, y)
cv_results
```

When using the following model:

```
model = make_pipeline(PCA(svd_solver="full"), Ridge(solver="svd"))
```

PyTorch will call `svd` solver and this is not available for MPS. In this case, it will fallback **automatically** on CPU and raise a warning.

However, for the following model:

```
model = make_pipeline(PCA(svd_solver="covariance_eigh"), Ridge(solver="svd"))
```

PyTorch will call `eigh` but this is also not supported for the MPS backend. However, in this case, scikit-learn is raising an error and request to explicitly set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK`.

This is slightly weird that we are to consistent on how to treat those similar cases. We should check what is the reason and decide if we prefer to raise a warning or an error.

COMMENT:
So I assume that we can close this issue.

Labels: Array API
possible bug in sklearn.utils.compute_class_weight

### Describe the bug

https://github.com/scikit-learn/scikit-learn/blob/b72e81af473c079ae95314efbca86557a836defa/sklearn/utils/class_weight.py#L85

In the 'if' statement on line 85 above, the condition 'n_weighted_classes != len(class_weight)' after 'and' may cause unexpected errors. In detail, if the total length of classes minus the length of unweighted_classes is equal to the length of class_weight, the program will not throw an exception even if there are abnormal values ​​in classes. 
As is shown in picture below.
![bug](https://github.com/user-attachments/assets/48760e8b-cb3b-43f5-8db8-bbacdf9f0c89)



### Steps/Code to Reproduce

```
from sklearn.utils import compute_class_weight
import numpy as np

class_weight = {1: 2, 2: 4, 3: 7}
y_label = np.array([1, 1, 1, 2, 3, 3])
classes = np.array([1, 2, 3, 99])  # the class 99 is an abnormal value
weight = compute_class_weight(class_weight, classes=classes, y=y_label)
print(weight)
```

### Expected Results


```
  File "E:\Anaconda3\envs\yolov5\lib\site-packages\sklearn\utils\class_weight.py", line 72, in compute_class_weight
    raise ValueError(
ValueError: The classes, [99], are not in class_weight

```

### Actual Results

```
[2. 4. 7. 1.]

Process finished with exit code 0
```

### Versions

```shell
System:
    python: 3.8.0 (default, Nov  6 2019, 16:00:02) [MSC v.1916 64 bit (AMD64)]
executable: E:\Anaconda3\envs\yolov5\python.exe
   machine: Windows-10-10.0.22621-SP0

Python dependencies:
      sklearn: 1.1.1
          pip: 22.1.2
   setuptools: 51.3.3
        numpy: 1.22.3
        scipy: 1.6.2
       Cython: 0.29.30
       pandas: 1.4.3
   matplotlib: 3.5.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: mkl
         prefix: mkl_rt
       filepath: E:\Anaconda3\envs\yolov5\Library\bin\mkl_rt.1.dll
        version: 2021.4-Product
threading_layer: intel
    num_threads: 6

       user_api: openmp
   internal_api: openmp
         prefix: vcomp
       filepath: E:\Anaconda3\envs\yolov5\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None
    num_threads: 6

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: E:\Anaconda3\envs\yolov5\Lib\site-packages\scipy\.libs\libopenblas.3HBPCJB5BPQGKWVZAVEBXNNJ2Q2G3TUP.gfortran-win_amd64.dll
        version: 0.3.9
threading_layer: pthreads
   architecture: Haswell
    num_threads: 6

       user_api: openmp
   internal_api: openmp
         prefix: libiomp
       filepath: E:\Anaconda3\envs\yolov5\Library\bin\libiomp5md.dll
        version: None
    num_threads: 6
None

Process finished with exit code 0
```

COMMENT:
Hi @HideOn1Bush,

there's a check that ``y`` doesn't contain classes that are not in ``classes`` but not the opposite. I think it's on purpose because it can happen that a subset of the target doesn't contain all classes, for instance in cross validation. 

@glemaitre might give more precisions.

Labels: Bug
possible bug in sklearn.utils.compute_class_weight

### Describe the bug

https://github.com/scikit-learn/scikit-learn/blob/b72e81af473c079ae95314efbca86557a836defa/sklearn/utils/class_weight.py#L85

In the 'if' statement on line 85 above, the condition 'n_weighted_classes != len(class_weight)' after 'and' may cause unexpected errors. In detail, if the total length of classes minus the length of unweighted_classes is equal to the length of class_weight, the program will not throw an exception even if there are abnormal values ​​in classes. 
As is shown in picture below.
![bug](https://github.com/user-attachments/assets/48760e8b-cb3b-43f5-8db8-bbacdf9f0c89)



### Steps/Code to Reproduce

```
from sklearn.utils import compute_class_weight
import numpy as np

class_weight = {1: 2, 2: 4, 3: 7}
y_label = np.array([1, 1, 1, 2, 3, 3])
classes = np.array([1, 2, 3, 99])  # the class 99 is an abnormal value
weight = compute_class_weight(class_weight, classes=classes, y=y_label)
print(weight)
```

### Expected Results


```
  File "E:\Anaconda3\envs\yolov5\lib\site-packages\sklearn\utils\class_weight.py", line 72, in compute_class_weight
    raise ValueError(
ValueError: The classes, [99], are not in class_weight

```

### Actual Results

```
[2. 4. 7. 1.]

Process finished with exit code 0
```

### Versions

```shell
System:
    python: 3.8.0 (default, Nov  6 2019, 16:00:02) [MSC v.1916 64 bit (AMD64)]
executable: E:\Anaconda3\envs\yolov5\python.exe
   machine: Windows-10-10.0.22621-SP0

Python dependencies:
      sklearn: 1.1.1
          pip: 22.1.2
   setuptools: 51.3.3
        numpy: 1.22.3
        scipy: 1.6.2
       Cython: 0.29.30
       pandas: 1.4.3
   matplotlib: 3.5.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: mkl
         prefix: mkl_rt
       filepath: E:\Anaconda3\envs\yolov5\Library\bin\mkl_rt.1.dll
        version: 2021.4-Product
threading_layer: intel
    num_threads: 6

       user_api: openmp
   internal_api: openmp
         prefix: vcomp
       filepath: E:\Anaconda3\envs\yolov5\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None
    num_threads: 6

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: E:\Anaconda3\envs\yolov5\Lib\site-packages\scipy\.libs\libopenblas.3HBPCJB5BPQGKWVZAVEBXNNJ2Q2G3TUP.gfortran-win_amd64.dll
        version: 0.3.9
threading_layer: pthreads
   architecture: Haswell
    num_threads: 6

       user_api: openmp
   internal_api: openmp
         prefix: libiomp
       filepath: E:\Anaconda3\envs\yolov5\Library\bin\libiomp5md.dll
        version: None
    num_threads: 6
None

Process finished with exit code 0
```

COMMENT:
Indeed, we need to be more lenient not only for cross-validation but also algorithm that can use bootstrap samples. This is related to the report here: https://github.com/scikit-learn/scikit-learn/issues/22413 with the implementation here https://github.com/scikit-learn/scikit-learn/pull/22595

Labels: Bug
⚠️ CI failed on Wheel builder (last failure: Jul 27, 2024) ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/10120613947)** (Jul 27, 2024)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://github.com/scikit-learn/scikit-learn/actions/runs/10128708839) on Jul 28, 2024

Labels: Bug
⚠️ CI failed on Wheel builder (last failure: Jul 27, 2024) ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/10120613947)** (Jul 27, 2024)

COMMENT:
For further reference, there was a aimilar bus error on July 27 for cp311-macosx_arm64 and cp312-macosx_arm64 see [build log](https://github.com/scikit-learn/scikit-learn/actions/runs/10120613947). On July 28 and July 29, all wheel builds passed.

Labels: Bug
Override precompute check in LassoCV

### Describe the workflow you want to enable

I am trying to use precompute=True for LassoCV. To save memory, I am passing in the inputs as float32's. However, I get an error that the Gram matrix precompute didn't match the true Gram matrix, where the error is some small epsilon like 1e-5 (see photo below). 

### Describe your proposed solution

It would be great to override the Gram check and allow for whatever was precomputed to be used. 

### Describe alternatives you've considered, if relevant

![image](https://github.com/user-attachments/assets/ba0a1c71-a73f-4a02-97f2-63a1dab827d3)


### Additional context

_No response_

COMMENT:
We could increase the relative tolerance `rtol` used for `float32` in `_check_precomputed_gram_matrix` from `1e-4` to `1e-3`.

We could also turn this exception into a warning instead.

Feel free to open a PR.

Labels: New Feature
Override precompute check in LassoCV

### Describe the workflow you want to enable

I am trying to use precompute=True for LassoCV. To save memory, I am passing in the inputs as float32's. However, I get an error that the Gram matrix precompute didn't match the true Gram matrix, where the error is some small epsilon like 1e-5 (see photo below). 

### Describe your proposed solution

It would be great to override the Gram check and allow for whatever was precomputed to be used. 

### Describe alternatives you've considered, if relevant

![image](https://github.com/user-attachments/assets/ba0a1c71-a73f-4a02-97f2-63a1dab827d3)


### Additional context

_No response_

COMMENT:
Hii, @ogrisel 
It's better to convert this exception into a warning. I will open a PR for this.

Labels: New Feature
Override precompute check in LassoCV

### Describe the workflow you want to enable

I am trying to use precompute=True for LassoCV. To save memory, I am passing in the inputs as float32's. However, I get an error that the Gram matrix precompute didn't match the true Gram matrix, where the error is some small epsilon like 1e-5 (see photo below). 

### Describe your proposed solution

It would be great to override the Gram check and allow for whatever was precomputed to be used. 

### Describe alternatives you've considered, if relevant

![image](https://github.com/user-attachments/assets/ba0a1c71-a73f-4a02-97f2-63a1dab827d3)


### Additional context

_No response_

COMMENT:
@Jaimin020 Are you still interested in opening a PR?

Labels: New Feature
Override precompute check in LassoCV

### Describe the workflow you want to enable

I am trying to use precompute=True for LassoCV. To save memory, I am passing in the inputs as float32's. However, I get an error that the Gram matrix precompute didn't match the true Gram matrix, where the error is some small epsilon like 1e-5 (see photo below). 

### Describe your proposed solution

It would be great to override the Gram check and allow for whatever was precomputed to be used. 

### Describe alternatives you've considered, if relevant

![image](https://github.com/user-attachments/assets/ba0a1c71-a73f-4a02-97f2-63a1dab827d3)


### Additional context

_No response_

COMMENT:
Hi @ogrisel ,

I am not sure if this has been fixed yet. Can you please confirm so I can go ahead?

Labels: New Feature
Override precompute check in LassoCV

### Describe the workflow you want to enable

I am trying to use precompute=True for LassoCV. To save memory, I am passing in the inputs as float32's. However, I get an error that the Gram matrix precompute didn't match the true Gram matrix, where the error is some small epsilon like 1e-5 (see photo below). 

### Describe your proposed solution

It would be great to override the Gram check and allow for whatever was precomputed to be used. 

### Describe alternatives you've considered, if relevant

![image](https://github.com/user-attachments/assets/ba0a1c71-a73f-4a02-97f2-63a1dab827d3)


### Additional context

_No response_

COMMENT:
Probably not. Please start by writing a minimal reproducer. If you can reproduce the problem, then you can adapt it as a non regression test for your subsequent PR.

Labels: New Feature
Override precompute check in LassoCV

### Describe the workflow you want to enable

I am trying to use precompute=True for LassoCV. To save memory, I am passing in the inputs as float32's. However, I get an error that the Gram matrix precompute didn't match the true Gram matrix, where the error is some small epsilon like 1e-5 (see photo below). 

### Describe your proposed solution

It would be great to override the Gram check and allow for whatever was precomputed to be used. 

### Describe alternatives you've considered, if relevant

![image](https://github.com/user-attachments/assets/ba0a1c71-a73f-4a02-97f2-63a1dab827d3)


### Additional context

_No response_

COMMENT:
Thanks for the input @ogrisel , will do!!

Labels: New Feature
Override precompute check in LassoCV

### Describe the workflow you want to enable

I am trying to use precompute=True for LassoCV. To save memory, I am passing in the inputs as float32's. However, I get an error that the Gram matrix precompute didn't match the true Gram matrix, where the error is some small epsilon like 1e-5 (see photo below). 

### Describe your proposed solution

It would be great to override the Gram check and allow for whatever was precomputed to be used. 

### Describe alternatives you've considered, if relevant

![image](https://github.com/user-attachments/assets/ba0a1c71-a73f-4a02-97f2-63a1dab827d3)


### Additional context

_No response_

COMMENT:
Hi @ogrisel ,

For the case when `fit_intercept=True`, the model ignores the user provided gram matrix so I am going with the case of `fit_intercept=False`.

Although I was unable to reproduce the exact type of error. I am highlighting a few scenarios that I went through. I have attached the jupyter file for your reference.

**Case 1:**
- Sample Dataset generated with 10 data points and 2 features
- precompute gram matrix provided
- fit_intercept=False

Error
```
ValueError: Gram matrix passed in via 'precompute' parameter did not pass validation when a single element was checked please check that it was computed properly. For element (1,1) we computed 4.250590801239014 but the user-supplied value was 4.762086868286133
```

**Case 2:**
- Sample Dataset generated with 10 data points and 2 features
- Input data mean centered
- precompute gram matrix provided
- fit_intercept=False

Error
```
ValueError: Gram matrix passed in via 'precompute' parameter did not pass validation when a single element was checked - please check that it was computed properly. For element (1,1) we computed 0.628188967704773 but the user-supplied value was 0.6334664225578308.
```

**Case 3:**
- Sample Dataset generated with 10 data points and 2 features
- Input data mean centered & scaled
- precompute gram matrix provided
- fit_intercept=False

Error
```
ValueError: Gram matrix passed in via 'precompute' parameter did not pass validation when a single element was checked - please check that it was computed properly. For element (1,1) we computed 9.9166898727417 but the user-supplied value was 10.0
```
[lasso_cv_scenarios.pdf](https://github.com/user-attachments/files/17596794/lasso_cv_scenarios.pdf)


I think reducing the `rtol` as you mentioned will handle the issue @kara-liu is facing but I am a little skeptical about turning the exception into a warning as you can see above that there cases where the the values are not within the tolerance level and significantly differ. Kindly advise.

> We could increase the relative tolerance `rtol` used for `float32` in `_check_precomputed_gram_matrix` from `1e-4` to `1e-3`.
> 
> We could also turn this exception into a warning instead.
> 
> Feel free to open a PR.

Labels: New Feature
Override precompute check in LassoCV

### Describe the workflow you want to enable

I am trying to use precompute=True for LassoCV. To save memory, I am passing in the inputs as float32's. However, I get an error that the Gram matrix precompute didn't match the true Gram matrix, where the error is some small epsilon like 1e-5 (see photo below). 

### Describe your proposed solution

It would be great to override the Gram check and allow for whatever was precomputed to be used. 

### Describe alternatives you've considered, if relevant

![image](https://github.com/user-attachments/assets/ba0a1c71-a73f-4a02-97f2-63a1dab827d3)


### Additional context

_No response_

COMMENT:
Hi @ogrisel ,

Kindly advise.

Labels: New Feature
Override precompute check in LassoCV

### Describe the workflow you want to enable

I am trying to use precompute=True for LassoCV. To save memory, I am passing in the inputs as float32's. However, I get an error that the Gram matrix precompute didn't match the true Gram matrix, where the error is some small epsilon like 1e-5 (see photo below). 

### Describe your proposed solution

It would be great to override the Gram check and allow for whatever was precomputed to be used. 

### Describe alternatives you've considered, if relevant

![image](https://github.com/user-attachments/assets/ba0a1c71-a73f-4a02-97f2-63a1dab827d3)


### Additional context

_No response_

COMMENT:
Hi @ogrisel ,

Kindly review.

Labels: New Feature
RFC Should cross-validation splitters validate that all classes are represented in each split?

This is a follow-up to the issue raised in https://github.com/scikit-learn/scikit-learn/issues/29554. However, I recall other issues raised for CV estimator in general.

So the context is the following: a CV estimator will use an internal cross-validation scheme. When we deal with a classifier, we don't have any safety mechanism in the CV to make sure that the classifier was at least trained on all classes. This could happen for two reasons on the top of the head: (i) the target is sorted and the training folds does not contain all classes and (ii) a class is potentially underrepresented and not selected.

In all cases, if the `fit` does not fail, we still obtain a broken estimator. If it breaks at `predict` at least this is not silently giving some wrong predictions but this is not a given. However, we don't provide a direct feedback to the user of what went wrong.

So I'm wondering if we should have a sort of mechanism in the CV strategies to ensure that at least all classes have been observed at `fit` time. I don't think that we should touch the estimators because we will repeat a lot of code and fundamentally the issue is raised because of the CV strategies.

NB: the same issue could happen with a simple classifier in a cross-validation to evaluate it. This is not necessarily a CV estimator.

COMMENT:
I think this is a possible issue in any classifier that inherently splits the data. E.g. in random forest, you can split the data, but in imbalanced cases, it is possible to get trees with a single class, which is useless. This makes me wonder why we don't stratify by default in the underlying CV/sample-splitting strategies.

E.g.
```Python
@pytest.mark.parametrize("Forest", FOREST_CLASSIFIERS.values())
def test_forest_sampling_imbalance(Forest):
    # Check that the forest can handle imbalanced classes with sampling
    # strategies.
    X, _ = datasets.make_classification(
        n_samples=1000, n_features=20, n_informative=2, n_redundant=2, n_classes=2
    )
    y = np.zeros(1000)
    y[10] = 1

    forest = Forest(
        n_estimators=10,
        random_state=0,
    )
    forest.fit(X, y)
    
    assert all(assert_array_equal(np.unique(y), np.unique(estimator.classes_)) for estimator in forest.estimators_)
```

Labels: RFC