Talip7/scikit-learn-issues-multilabel huggingface dataset:
Selecting Lasso via cross-validation data leakage

### Describe the issue linked to the documentation

```py
model = make_pipeline(StandardScaler(), LassoCV(cv=20)).fit(X, y)
```

Optimizing Alpha for lasso involves data leakage because standard scaling is applied on X and y in one go, instant for each fold.

It is used in the example Selecting Lasso via cross-validation
https://scikit-learn.org/stable/auto_examples/linear_model/plot_lasso_model_selection.html




### Suggest a potential alternative/fix

_No response_

COMMENT:
Hi,

Sorry for jumping in here, but this is an issue I have been revisiting a lot lately.

The problem with the current implementation of `LassoCV`  (and other *CV estimators like RidgeCV) is that it is quite restrictive since it is only appropriate when features in the input data are all in similar units. When features have different units (e.g. weight vs height), then you need to scale them so that some features (those with small units) do not get more penalized than others (those with large units). But you can't create a pipeline like @nielsbijl showed, because that would cause data leakage.  So the only alternative to do this properly would be through `GridSearchCV`, but this is less efficient compared to `LassoCV`.

How hard would it be to implement a scale step within `LassoCV`? In fact, common libraries in R (https://glmnet.stanford.edu/articles/glmnet.html) and MATLAB (https://www.mathworks.com/help/stats/lasso.html) implementing a similar function perform a scaling operation by default (it can be turned off though).

Labels: Documentation, module:linear_model
Selecting Lasso via cross-validation data leakage

### Describe the issue linked to the documentation

```py
model = make_pipeline(StandardScaler(), LassoCV(cv=20)).fit(X, y)
```

Optimizing Alpha for lasso involves data leakage because standard scaling is applied on X and y in one go, instant for each fold.

It is used in the example Selecting Lasso via cross-validation
https://scikit-learn.org/stable/auto_examples/linear_model/plot_lasso_model_selection.html




### Suggest a potential alternative/fix

_No response_

COMMENT:
at this point if you really want to do this I would suggest you use an older version of sklearn that had the "normalize" parameter for LassoCV.

Labels: Documentation, module:linear_model
Mean Absolute Percentage Error

### Describe the workflow you want to enable

Mean Absolute Percentage Error (MAPE): MAPE is a new evaluation metric for a regression problem. It is calculated as the mean absolute percentage error between the predicted and actual values. MAPE is a more robust metric than other metrics such as root mean squared error (RMSE) because it is not affected by outliers.

### Describe your proposed solution

The mean absolute percentage error (MAPE) is a measure of prediction accuracy of a forecasting method in statistics. It usually expresses the accuracy as a ratio defined by the formula:

Code snippet
MAPE = ∑ |(At − Ft)/At|
Use code with caution. [Learn more](https://bard.google.com/faq#coding)
where At is the actual value and Ft is the forecast value.

The MAPE has several advantages over other measures of accuracy, such as the root mean squared error (RMSE). First, the MAPE is not affected by outliers. Second, the MAPE is more intuitive to understand than the RMSE. Third, the MAPE is more sensitive to changes in the forecast values.

However, the MAPE also has some disadvantages. First, the MAPE can be very sensitive to small changes in the actual values. Second, the MAPE is not always a good measure of accuracy for forecasting methods that are not linear.

Here are some proposed solutions to the MAPE:

Use a robust measure of accuracy: The MAPE is not a robust measure of accuracy, meaning that it is sensitive to outliers. If there are outliers in the data, the MAPE may not be a good measure of accuracy. In this case, it is better to use a robust measure of accuracy, such as the median absolute error (MAE).
Use a linear forecasting method: The MAPE is not always a good measure of accuracy for forecasting methods that are not linear. For example, the MAPE is not a good measure of accuracy for exponential smoothing. In this case, it is better to use a linear forecasting method, such as a linear regression model.
Use a weighted MAPE: The MAPE can be weighted to give more importance to certain data points. This can be useful if some data points are more important than others.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
AI generated, spam.

Labels: New Feature, spam, Needs Triage
Mean Absolute Percentage Error

### Describe the workflow you want to enable

Mean Absolute Percentage Error (MAPE): MAPE is a new evaluation metric for a regression problem. It is calculated as the mean absolute percentage error between the predicted and actual values. MAPE is a more robust metric than other metrics such as root mean squared error (RMSE) because it is not affected by outliers.

### Describe your proposed solution

The mean absolute percentage error (MAPE) is a measure of prediction accuracy of a forecasting method in statistics. It usually expresses the accuracy as a ratio defined by the formula:

Code snippet
MAPE = ∑ |(At − Ft)/At|
Use code with caution. [Learn more](https://bard.google.com/faq#coding)
where At is the actual value and Ft is the forecast value.

The MAPE has several advantages over other measures of accuracy, such as the root mean squared error (RMSE). First, the MAPE is not affected by outliers. Second, the MAPE is more intuitive to understand than the RMSE. Third, the MAPE is more sensitive to changes in the forecast values.

However, the MAPE also has some disadvantages. First, the MAPE can be very sensitive to small changes in the actual values. Second, the MAPE is not always a good measure of accuracy for forecasting methods that are not linear.

Here are some proposed solutions to the MAPE:

Use a robust measure of accuracy: The MAPE is not a robust measure of accuracy, meaning that it is sensitive to outliers. If there are outliers in the data, the MAPE may not be a good measure of accuracy. In this case, it is better to use a robust measure of accuracy, such as the median absolute error (MAE).
Use a linear forecasting method: The MAPE is not always a good measure of accuracy for forecasting methods that are not linear. For example, the MAPE is not a good measure of accuracy for exponential smoothing. In this case, it is better to use a linear forecasting method, such as a linear regression model.
Use a weighted MAPE: The MAPE can be weighted to give more importance to certain data points. This can be useful if some data points are more important than others.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> AI generated, spam.





https://html-email.github.io/AJZKijfDElrK6pb1dcwSR/

Labels: New Feature, spam, Needs Triage
Add common tests for estimators that support the Array API

This issue is about agreeing on what to do about common tests for estimators that support the Array API.

There are two things we need to test for every estimator for which we add Array API support:
1. does it work with a selection of Array API implementations. I think testing with `numpy.array_api`, `pytorch` (CPU) as default and `pytoch` (GPU) and `cupy` when a GPU is available would be good and cover most bases.
2. do the results change when a Array API namespace is used compared to a vanilla Numpy array

There is a 2.5 point as well: different dtypes (`float64`, `float32`, `float16` (when supported)). Not sure what to do about that in terms of common test.

is there something else we should test in the common tests?

Implementation wise I think it would be good to add a estimator tag that we can use to mark estimators that support Array API. Something like `"array_api_support": True`?

Then I'd add one test that checks if the result of vanilla Numpy match the results of `numpy.array_api` (to cover (2)). And then maybe two more common tests. One to check that `numpy.array_api` and `pytorch` (CPU) work and another one for the GPU based tests in (1)? I'm wondering if having some code duplication between tests for (1) is worth it in terms of making the tests easier to read compared to trying to do it all via parametrisation.

Depending on how far #26243, #26315, #22554, #25956 are we should also clean up the tests that check this that were added in those PRs.

What do you think @ogrisel @thomasjpfan ?

COMMENT:
These measures sound helpful. I think, at least in the short-term, it should be okay to defer testing different dtypes until later just to keep things as simple as possible.

> Implementation wise I think it would be good to add a estimator tag that we can use to mark estimators that support Array API. Something like `"array_api_support": True`?

👍
> I'm wondering if having some code duplication between tests for (1) is worth it in terms of making the tests easier to read compared to trying to do it all via parametrisation.

Which code would be duplicated for readability here?

Also do we intend to allow for Array API compatibility in functions as well? If so should we consider common tests for them as well, or leave that for later?

Labels: Array API
Add common tests for estimators that support the Array API

This issue is about agreeing on what to do about common tests for estimators that support the Array API.

There are two things we need to test for every estimator for which we add Array API support:
1. does it work with a selection of Array API implementations. I think testing with `numpy.array_api`, `pytorch` (CPU) as default and `pytoch` (GPU) and `cupy` when a GPU is available would be good and cover most bases.
2. do the results change when a Array API namespace is used compared to a vanilla Numpy array

There is a 2.5 point as well: different dtypes (`float64`, `float32`, `float16` (when supported)). Not sure what to do about that in terms of common test.

is there something else we should test in the common tests?

Implementation wise I think it would be good to add a estimator tag that we can use to mark estimators that support Array API. Something like `"array_api_support": True`?

Then I'd add one test that checks if the result of vanilla Numpy match the results of `numpy.array_api` (to cover (2)). And then maybe two more common tests. One to check that `numpy.array_api` and `pytorch` (CPU) work and another one for the GPU based tests in (1)? I'm wondering if having some code duplication between tests for (1) is worth it in terms of making the tests easier to read compared to trying to do it all via parametrisation.

Depending on how far #26243, #26315, #22554, #25956 are we should also clean up the tests that check this that were added in those PRs.

What do you think @ogrisel @thomasjpfan ?

COMMENT:
> is there something else we should test in the common tests?

We could indeed try to do a common test but common tests typically run on estimators with default hyper-parameter values with very small datasets.

So we might need more estimator-specific tests to run such checks on particular hyper-parameter combinations or data with specific shapes. For this, a few reusable fixtures and/or private testing helpers might help factorize redundant boilerplate across estimators.

Labels: Array API
Add common tests for estimators that support the Array API

This issue is about agreeing on what to do about common tests for estimators that support the Array API.

There are two things we need to test for every estimator for which we add Array API support:
1. does it work with a selection of Array API implementations. I think testing with `numpy.array_api`, `pytorch` (CPU) as default and `pytoch` (GPU) and `cupy` when a GPU is available would be good and cover most bases.
2. do the results change when a Array API namespace is used compared to a vanilla Numpy array

There is a 2.5 point as well: different dtypes (`float64`, `float32`, `float16` (when supported)). Not sure what to do about that in terms of common test.

is there something else we should test in the common tests?

Implementation wise I think it would be good to add a estimator tag that we can use to mark estimators that support Array API. Something like `"array_api_support": True`?

Then I'd add one test that checks if the result of vanilla Numpy match the results of `numpy.array_api` (to cover (2)). And then maybe two more common tests. One to check that `numpy.array_api` and `pytorch` (CPU) work and another one for the GPU based tests in (1)? I'm wondering if having some code duplication between tests for (1) is worth it in terms of making the tests easier to read compared to trying to do it all via parametrisation.

Depending on how far #26243, #26315, #22554, #25956 are we should also clean up the tests that check this that were added in those PRs.

What do you think @ogrisel @thomasjpfan ?

COMMENT:
+1 for an estimator tag in any case. It could also be helpful to help us semi-automatize the maintenance of a list of estimators with Array API support in the doc.

Labels: Array API
Add common tests for estimators that support the Array API

This issue is about agreeing on what to do about common tests for estimators that support the Array API.

There are two things we need to test for every estimator for which we add Array API support:
1. does it work with a selection of Array API implementations. I think testing with `numpy.array_api`, `pytorch` (CPU) as default and `pytoch` (GPU) and `cupy` when a GPU is available would be good and cover most bases.
2. do the results change when a Array API namespace is used compared to a vanilla Numpy array

There is a 2.5 point as well: different dtypes (`float64`, `float32`, `float16` (when supported)). Not sure what to do about that in terms of common test.

is there something else we should test in the common tests?

Implementation wise I think it would be good to add a estimator tag that we can use to mark estimators that support Array API. Something like `"array_api_support": True`?

Then I'd add one test that checks if the result of vanilla Numpy match the results of `numpy.array_api` (to cover (2)). And then maybe two more common tests. One to check that `numpy.array_api` and `pytorch` (CPU) work and another one for the GPU based tests in (1)? I'm wondering if having some code duplication between tests for (1) is worth it in terms of making the tests easier to read compared to trying to do it all via parametrisation.

Depending on how far #26243, #26315, #22554, #25956 are we should also clean up the tests that check this that were added in those PRs.

What do you think @ogrisel @thomasjpfan ?

COMMENT:
Closing after # #26372 was merged. Let's open more focused issues as needed for further improvements.

Labels: Array API
Warning due to `lscpu` on MacOS on nightly builds

I created a conda environment containing the `scipy-dev` packages (to solve the deprecation warning shown in our CI). Once I built scikit-learn, I get the following error when importing:

```python
> ipython
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:12:31) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sklearn
/bin/sh: lscpu: command not found

In [2]: sklearn.__version__
Out[2]: '1.3.dev0'
```

@ogrisel @jeremiedbb Do you know if it is coming from `threadpoolctl` or `joblib`? Basically the `command not found` is raised because I am on MacOS

COMMENT:
Indeed this is from:

- https://github.com/joblib/loky/blob/a9dc2eb790cf8e50eb75b2aeb70cb5f243c093d2/loky/backend/context.py#L245

we should test if `lscpu` is available and raise a higher level warning (or a fallback that does not require `lscpu`?).

Edit: we can use [shutil.which](https://docs.python.org/3/library/shutil.html#shutil.which) to test if the `lscpu` program is in the PATH.

Labels: Bug
Warning due to `lscpu` on MacOS on nightly builds

I created a conda environment containing the `scipy-dev` packages (to solve the deprecation warning shown in our CI). Once I built scikit-learn, I get the following error when importing:

```python
> ipython
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:12:31) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sklearn
/bin/sh: lscpu: command not found

In [2]: sklearn.__version__
Out[2]: '1.3.dev0'
```

@ogrisel @jeremiedbb Do you know if it is coming from `threadpoolctl` or `joblib`? Basically the `command not found` is raised because I am on MacOS

COMMENT:
But this is weird since my platform should be `darwin` in this case. But at least I can check the code in `loky`. Thanks.

Labels: Bug
Warning due to `lscpu` on MacOS on nightly builds

I created a conda environment containing the `scipy-dev` packages (to solve the deprecation warning shown in our CI). Once I built scikit-learn, I get the following error when importing:

```python
> ipython
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:12:31) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sklearn
/bin/sh: lscpu: command not found

In [2]: sklearn.__version__
Out[2]: '1.3.dev0'
```

@ogrisel @jeremiedbb Do you know if it is coming from `threadpoolctl` or `joblib`? Basically the `command not found` is raised because I am on MacOS

COMMENT:
> But this is weird since my platform should be darwin in this case.

I missed that. Indeed it's weird.

Labels: Bug
Warning due to `lscpu` on MacOS on nightly builds

I created a conda environment containing the `scipy-dev` packages (to solve the deprecation warning shown in our CI). Once I built scikit-learn, I get the following error when importing:

```python
> ipython
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:12:31) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sklearn
/bin/sh: lscpu: command not found

In [2]: sklearn.__version__
Out[2]: '1.3.dev0'
```

@ogrisel @jeremiedbb Do you know if it is coming from `threadpoolctl` or `joblib`? Basically the `command not found` is raised because I am on MacOS

COMMENT:
> But this is weird since my platform should be darwin in this case

did you confirm that, just to be sure ?

Labels: Bug
Warning due to `lscpu` on MacOS on nightly builds

I created a conda environment containing the `scipy-dev` packages (to solve the deprecation warning shown in our CI). Once I built scikit-learn, I get the following error when importing:

```python
> ipython
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:12:31) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sklearn
/bin/sh: lscpu: command not found

In [2]: sklearn.__version__
Out[2]: '1.3.dev0'
```

@ogrisel @jeremiedbb Do you know if it is coming from `threadpoolctl` or `joblib`? Basically the `command not found` is raised because I am on MacOS

COMMENT:
> did you confirm that, just to be sure ?

I checked but the warning is not raised by `loky`.

Labels: Bug
Warning due to `lscpu` on MacOS on nightly builds

I created a conda environment containing the `scipy-dev` packages (to solve the deprecation warning shown in our CI). Once I built scikit-learn, I get the following error when importing:

```python
> ipython
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:12:31) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sklearn
/bin/sh: lscpu: command not found

In [2]: sklearn.__version__
Out[2]: '1.3.dev0'
```

@ogrisel @jeremiedbb Do you know if it is coming from `threadpoolctl` or `joblib`? Basically the `command not found` is raised because I am on MacOS

COMMENT:
facing same issue, mac m1.

Labels: Bug
Warning due to `lscpu` on MacOS on nightly builds

I created a conda environment containing the `scipy-dev` packages (to solve the deprecation warning shown in our CI). Once I built scikit-learn, I get the following error when importing:

```python
> ipython
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:12:31) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sklearn
/bin/sh: lscpu: command not found

In [2]: sklearn.__version__
Out[2]: '1.3.dev0'
```

@ogrisel @jeremiedbb Do you know if it is coming from `threadpoolctl` or `joblib`? Basically the `command not found` is raised because I am on MacOS

COMMENT:
any update on this?

Labels: Bug
Warning due to `lscpu` on MacOS on nightly builds

I created a conda environment containing the `scipy-dev` packages (to solve the deprecation warning shown in our CI). Once I built scikit-learn, I get the following error when importing:

```python
> ipython
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:12:31) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sklearn
/bin/sh: lscpu: command not found

In [2]: sklearn.__version__
Out[2]: '1.3.dev0'
```

@ogrisel @jeremiedbb Do you know if it is coming from `threadpoolctl` or `joblib`? Basically the `command not found` is raised because I am on MacOS

COMMENT:
@glemaitre can you check if it still happens ?

In any case this is not a scikit-learn issue, and doesn't come from joblib loky or threadpoolctl either. We should probably close and open an issue in the appropriate project when we manage to find the source

Labels: Bug
Warning due to `lscpu` on MacOS on nightly builds

I created a conda environment containing the `scipy-dev` packages (to solve the deprecation warning shown in our CI). Once I built scikit-learn, I get the following error when importing:

```python
> ipython
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:12:31) [Clang 14.0.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.13.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sklearn
/bin/sh: lscpu: command not found

In [2]: sklearn.__version__
Out[2]: '1.3.dev0'
```

@ogrisel @jeremiedbb Do you know if it is coming from `threadpoolctl` or `joblib`? Basically the `command not found` is raised because I am on MacOS

COMMENT:
It was fixed on its own :). Closing. We will never know the culprit.

Labels: Bug
⚠️ CI failed on linux_arm64_wheel ⚠️

**CI failed on [linux_arm64_wheel](https://cirrus-ci.com/build/6302053069225984)** (May 07, 2023)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://cirrus-ci.com/build/4827479530012672) on May 08, 2023

Labels: Needs Triage