Talip7/scikit-learn-issues-multilabel huggingface dataset:
`cross_validate` raises an exception when metadata routing is enabled

### Describe the bug

In the latest release (v1.6.0), `cross_validate` raises an exception when using it with metadata routing enabled. This is because `params` dict gets unpacked even if `None`, which is the default value. See this line:

https://github.com/scikit-learn/scikit-learn/blob/76ae0a539a0e87145c9f6fedcd7033494082fa17/sklearn/model_selection/_validation.py#L375

### Steps/Code to Reproduce

```python
import numpy as np
import sklearn
from sklearn.model_selection import cross_validate
from sklearn.model_selection.tests.test_validation import MockClassifier

sklearn.set_config(enable_metadata_routing=True)

X = np.ones((10, 2))
y = np.array([0, 0, 1, 1, 2, 2, 3, 3, 4, 4])

clf = MockClassifier()
cross_validate(clf, X, y)
```

### Expected Results

No exception being raised.

### Actual Results

```python-traceback
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<redacted>/lib/python3.10/site-packages/sklearn/utils/_param_validation.py", line 216, in wrapper
    return func(*args, **kwargs)
  File "<redacted>/lib/python3.10/site-packages/sklearn/model_selection/_validation.py", line 375, in cross_validate
    routed_params = process_routing(router, "fit", **params)
TypeError: sklearn.utils._metadata_requests.process_routing() argument after ** must be a mapping, not NoneType
```

### Versions

```shell
System:
    python: 3.10.12 (main, Nov  6 2024, 20:22:13) [GCC 11.4.0]
executable: <redacted>/bin/python
   machine: Linux-6.5.13netflix-g77293087f291-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.6.0
          pip: None
   setuptools: 75.6.0
        numpy: 1.26.4
        scipy: 1.14.1
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.9.3
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 4
         prefix: libopenblas
       filepath: /root/pycharm_projects/evaluations/.venv/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Cooperlake

       user_api: blas
   internal_api: openblas
    num_threads: 4
         prefix: libscipy_openblas
       filepath: /root/pycharm_projects/evaluations/.venv/lib/python3.10/site-packages/scipy.libs/libscipy_openblas-c128ec02.so
        version: 0.3.27.dev
threading_layer: pthreads
   architecture: Cooperlake

       user_api: openmp
   internal_api: openmp
    num_threads: 4
         prefix: libgomp
       filepath: <redacted>/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```
```

COMMENT:
We shouldn't error, that was an oversight, fixed here: https://github.com/scikit-learn/scikit-learn/pull/30451

Labels: Bug
DOC add FAQ link to scikit-learn course

### Describe the issue linked to the documentation

Given there are so many inquiries such as "How do I get started with scikit-learn?" let's add  a resource to the FAQ here:
https://scikit-learn.org/stable/faq.html

resource:
https://inria.github.io/scikit-learn-mooc/appendix/datasets_intro.html

### Suggest a potential alternative/fix

_No response_

COMMENT:
+1 for linking to the MOOC material, but I would rather link to the main page rather than the appendix dedicated to exploratory data analysis of the datasets used elsewhere in the MOOC.

https://inria.github.io/scikit-learn-mooc/index.html

Labels: Documentation
DOC add FAQ link to scikit-learn course

### Describe the issue linked to the documentation

Given there are so many inquiries such as "How do I get started with scikit-learn?" let's add  a resource to the FAQ here:
https://scikit-learn.org/stable/faq.html

resource:
https://inria.github.io/scikit-learn-mooc/appendix/datasets_intro.html

### Suggest a potential alternative/fix

_No response_

COMMENT:
I'd like to work on this issue, if it's available.

Labels: Documentation
Missing `inverse_transform` in `DictionaryLearning`and `SparseCoder`

### Describe the workflow you want to enable

The method is currently missing in those two classes which prevent doing a loop over all Linear decomposition methods when evaluation them for denoising for instance. 

### Describe your proposed solution

I propose to implement the method in the class `_BaseSparseCoding` from https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/decomposition/_dict_learning.py

I have an implementation with updated tests that I will propose as PR when the issue is created. Ping to @agramfort with whom I already discussed about this.

### Describe alternatives you've considered, if relevant

the documentation gives the following expale
```python 
X_hat = X_transformed @ dict_learner.components_ 
```
which is OK but requires the user to know about `components_` and to do that to all linear decomposition methods is tested in a loop  (the others have all implemented `inverse_transform`). Implementing the method closes a missing part of the API and would be better in my opinion.

### Additional context

_No response_

COMMENT:
+1 to add this method

Labels: New Feature
Example of binning of continous variables for chi2

### Describe the issue linked to the documentation

The [chi2](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.chi2.html) doesn't work on continuous variables. This issue has numerous discussions, e.g. [here](https://stats.stackexchange.com/questions/369945/feature-selection-using-chi-squared-for-continuous-features).

The Matlab counterpart command, [fscchi2](https://www.mathworks.com/help/stats/fscchi2.html), solves this issue by automatically binning data. I believe that the example of chi2 feature selection with pre-binning may be beneficial. 

### Suggest a potential alternative/fix

_No response_

COMMENT:
Hello ! 
I would like to work on this issue.

I am wondering whether it would be better to add an "automatic_binning" parameter to the chi2 function or to clarify the documentation by explicitly stating that continuous data must be pre-binned before using chi2.

In the second case, it might also be helpful to include an example demonstrating how to bin continuous data before applying chi2. What do you think would be the preferred approach?

Labels: Documentation
Example of binning of continous variables for chi2

### Describe the issue linked to the documentation

The [chi2](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.chi2.html) doesn't work on continuous variables. This issue has numerous discussions, e.g. [here](https://stats.stackexchange.com/questions/369945/feature-selection-using-chi-squared-for-continuous-features).

The Matlab counterpart command, [fscchi2](https://www.mathworks.com/help/stats/fscchi2.html), solves this issue by automatically binning data. I believe that the example of chi2 feature selection with pre-binning may be beneficial. 

### Suggest a potential alternative/fix

_No response_

COMMENT:
It is great to know that this issue is relevant.

As an immediate solution, some basic binning example is recommended.

In the long run, it is recommended that some binning technique has to be implemented. It is important to note, that the multivariate binning is non-trivial. For example, there are different binning strategies, e.g. [R binning for Chi2](https://cran.r-project.org/web/packages/discretization/discretization.pdf) and [handling infinite values](https://help.palisade.com/v8/en/@RISK/1-Define/3-Fit/Chi-Squared-Binning.htm). 

Personally, I am not proficient enough in statistics to propose and theoretically justify any particular binning strategy and its influence on the feature selection performance.

Labels: Documentation
Example of binning of continous variables for chi2

### Describe the issue linked to the documentation

The [chi2](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.chi2.html) doesn't work on continuous variables. This issue has numerous discussions, e.g. [here](https://stats.stackexchange.com/questions/369945/feature-selection-using-chi-squared-for-continuous-features).

The Matlab counterpart command, [fscchi2](https://www.mathworks.com/help/stats/fscchi2.html), solves this issue by automatically binning data. I believe that the example of chi2 feature selection with pre-binning may be beneficial. 

### Suggest a potential alternative/fix

_No response_

COMMENT:
Not very familiar with chi2 but it looks like from previous discussions that indeed using chi2 for continuous variable may not make sense. For example https://github.com/scikit-learn/scikit-learn/issues/17286 or https://github.com/scikit-learn/scikit-learn/issues/21455.

The [chi2 documentation](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.chi2.html) also mentions this:

> This score can be used to select the n_features features with the highest values for the test chi-squared statistic from X, which must contain only non-negative features such as booleans or frequencies (e.g., term counts in document classification)

@bykhov not sure what you are after:
- an improvement to the documentation. Where would you put it, in the `chi2` docstring, somewhere else? Is it enough to mention that `KBinsDiscretizer` can be used to do binning? Would you add some code to explain it better?
- a new feature that would do the binning automatically. Not a chi2 expert, but I would think this would be quite low in our list of priorities.

Labels: Documentation
Example of binning of continous variables for chi2

### Describe the issue linked to the documentation

The [chi2](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.chi2.html) doesn't work on continuous variables. This issue has numerous discussions, e.g. [here](https://stats.stackexchange.com/questions/369945/feature-selection-using-chi-squared-for-continuous-features).

The Matlab counterpart command, [fscchi2](https://www.mathworks.com/help/stats/fscchi2.html), solves this issue by automatically binning data. I believe that the example of chi2 feature selection with pre-binning may be beneficial. 

### Suggest a potential alternative/fix

_No response_

COMMENT:
@lesteve I believe that the proposed pull request by @hugoboulenger is excellent.

Labels: Documentation
Code Smells and Linting Errors in check-meson-openmp-dependencies.py

### Describe the workflow you want to enable

Using the Python Linter set to PEP 8 and Test Driven Development using the Sci-Kit Lean testing suite.

### Describe your proposed solution

I propose to reduce redundant code with helper functions, specifically with the has_openmp_flags function that iterates through the compiler and linker lists, repeating a few lines of code. By adding a helper that takes the lists as parameters, the code can be reduced. Change a few vague names (like file or message) to be more specific (like message_file and error_message), shorten long lines of code, and write missing docstrings.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

These issues were found as part of a code review for a school project.

COMMENT:
Closing based on https://github.com/scikit-learn/scikit-learn/pull/30423#pullrequestreview-2486580809

Labels: New Feature, Needs Triage
Identical branches in the conditional statement in "svm.cpp"

### Describe the bug

File svm/src/libsvm/svm.cpp, lines 1895-1903 contain the same statements. Is it correct?


### Steps/Code to Reproduce

		if(fabs(alpha[i]) > 0)
		{
			++nSV;
			if(prob->y[i] > 0)
			{
				if(fabs(alpha[i]) >= si.upper_bound[i])
					++nBSV;
			}
			else
			{
				if(fabs(alpha[i]) >= si.upper_bound[i])
					++nBSV;
			}
		}

### Expected Results

none

### Actual Results

none

### Versions

```shell
1.5.2
```

COMMENT:
Thanks for the report @ayv19. Looks like a bug indeed and it seems to have been fixed upstream, see https://github.com/cjlin1/libsvm/blob/35e55962f7f03ce425bada0e6b9db79193e947f8/svm.cpp#L1690-L1702
```
		if(fabs(alpha[i]) > 0)
		{
			++nSV;
			if(prob->y[i] > 0)
			{
				if(fabs(alpha[i]) >= si.upper_bound_p)
					++nBSV;
			}
			else
			{
				if(fabs(alpha[i]) >= si.upper_bound_n)
					++nBSV;
			}
```
This file hasn't been synchronized with upstream for a while. Fixing it would probably require many changes since the `si` struct itself has been refactored. I wonder if we should try to sync with upstream and see what happens. This is not a module of high priority for most scikit-learn maintainers though.

Labels: Bug
Identical branches in the conditional statement in "svm.cpp"

### Describe the bug

File svm/src/libsvm/svm.cpp, lines 1895-1903 contain the same statements. Is it correct?


### Steps/Code to Reproduce

		if(fabs(alpha[i]) > 0)
		{
			++nSV;
			if(prob->y[i] > 0)
			{
				if(fabs(alpha[i]) >= si.upper_bound[i])
					++nBSV;
			}
			else
			{
				if(fabs(alpha[i]) >= si.upper_bound[i])
					++nBSV;
			}
		}

### Expected Results

none

### Actual Results

none

### Versions

```shell
1.5.2
```

COMMENT:
As a time series forecasting user/dev, I frequently use SVR regressions.

There is probably a place here for a recurrent task for scikit-learn GA process. In order to keep Scikit-learn synchronized with external libraries (libsvm and other) as a show-stopper before releasing a new version. Especially when the code is copied AS IS in the source code.

By the way, the code snippet above does not seem too harmful. It computes two indicators (nSV and nBSV) only for logging purposes. But the fact that scikit-learn did not synchoronize with libsvm for a while is the real problem for me.

Labels: Bug
Make `param_grid` in `GridSearchCV` a callable with the `X` and `y` as the parameters

### Describe the workflow you want to enable

**CASE 1:**

I use a "pipeline" approach with `SelectKBest` and `RandomForestClassifier`, and I want to use `RandomForestClassifier.monotonic_cst` which is a number array now.

As `SelectKBest` chooses the arbitrary set of features each time, I don't have any ability to provide the proper monotonic flags in the pipeline.

**CASE 2:**

Similar, but with `SelectKBest` and `HistGradientBoostingClassifier`. The later allows to specify a map of monotonic rules in `HistGradientBoostingClassifier.monotonic_cst`. But if `SelectKBest` chooses to omit the fields then `HistGradientBoostingClassifier` fails saying the fields in the monotonic rules are missing in the `X`.

**CASE 3:**

The documentation to `HistGradientBoostingClassifier.min_samples_leaf` says:

> The minimum number of samples per leaf. For small datasets with less than a few hundred samples, it is recommended to lower this value since only very shallow trees would be built.

But in a fully automated functional "pipelined" trainer how would I know the dataset is small beforehand?

**CASE 4:**

I know I don't want to lose time on checking too small or to big learning rates if the set of features selected by `SelectKBest` has or has not some specific fields. Or the training set itself has or has not some features. 


### Describe your proposed solution

Right now the documentation says: "param_grid: dict or list of dictionaries"

scikit-learn needs to allow to specify a callback for `param_grid` as:

```
def param_grid_callback(X, y) -> dict or list of dictionaries
   ...
```

which will be called before `fit()`

If it is possible for the "CASE 1" I would be able to do that (considering the `X` is a Pandas dataframe):
``` 
def param_grid_callback(X, y):
    rules = {
        'feature_a': +1,
        'feature_b': -1,
    } 

    return [
        {
            `classifier__monotonic_cst`: [
                 None,
                 [rules.get(field, 0) for field in X.columns],
            ],
        },
    ]
```

For the "CASE 3" the solution will be:
``` 
def param_grid_callback(X, y):
    return [
        {
            `classifier__min_samples_leaf`: [
                 4 if len(X) < 100 else 200
            ],
        },
    ]
```


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
the proposition will solve this one https://github.com/scikit-learn/scikit-learn/issues/22189

Labels: New Feature, Needs Triage
`partial_fit` for `RobustScaler`

### Describe the workflow you want to enable

I would like to be able to use `partial_fit` with the `RobustScaler` preprocessing for streaming cases or when my data doesn't fit in memory.
As I understand from this paper https://sites.cs.ucsb.edu/~suri/psdir/ency.pdf, it would probably only be possible to compute the robustly estimated variance and mean up to some precision epsilon, that could probably become an attribute of the class.

Not sure whether this should be considered a new algorithm or not, let me know what you think.

### Describe your proposed solution

I haven't looked much into it but I think there were 2 approaches:
- use one of the solutions proposed in https://sites.cs.ucsb.edu/~suri/psdir/ency.pdf
- @amueller was suggesting in https://github.com/scikit-learn/scikit-learn/issues/5028#issuecomment-125981597 that binning could be an option. Maybe this is actually mentioned in the paper above

### Describe alternatives you've considered, if relevant

An alternative proposed in this [SO comment](https://stackoverflow.com/questions/57291876/robustscaler-partial-fit-similar-to-minmaxscaler-or-standardscaler/57292088#comment110923346_57291876) is to load the data column by column if it reduces the memory load.

However, that would be super impractical in my setting where I just cannot load all data into memory at once.

### Additional context

_No response_

COMMENT:
> it would probably only be possible to compute the robustly estimated variance and mean up to some precision epsilon

Nitpicking and to avoid any confusion: `RobustScaler` does not estimate the mean and variance but rather the median and IQR. Those can be very different values depending on the distribution.

Personally, I find the reservoir sampling approach interesting. Not that a `subsample` param similar to what we do in `KBinsDiscretizer` would make sense, irrespective of whether or not we implement `partial_fit`. It could later be reused to control the size of the reservoir sampler used for `partial_fit`.

In addition to the methods referenced in the paper linked above, there is also the t-digest out-of-core / online estimator: https://github.com/tdunning/t-digest (but I have no idea about what are the pros and cons).

Labels: New Feature, Needs Decision - Include Feature
`partial_fit` for `RobustScaler`

### Describe the workflow you want to enable

I would like to be able to use `partial_fit` with the `RobustScaler` preprocessing for streaming cases or when my data doesn't fit in memory.
As I understand from this paper https://sites.cs.ucsb.edu/~suri/psdir/ency.pdf, it would probably only be possible to compute the robustly estimated variance and mean up to some precision epsilon, that could probably become an attribute of the class.

Not sure whether this should be considered a new algorithm or not, let me know what you think.

### Describe your proposed solution

I haven't looked much into it but I think there were 2 approaches:
- use one of the solutions proposed in https://sites.cs.ucsb.edu/~suri/psdir/ency.pdf
- @amueller was suggesting in https://github.com/scikit-learn/scikit-learn/issues/5028#issuecomment-125981597 that binning could be an option. Maybe this is actually mentioned in the paper above

### Describe alternatives you've considered, if relevant

An alternative proposed in this [SO comment](https://stackoverflow.com/questions/57291876/robustscaler-partial-fit-similar-to-minmaxscaler-or-standardscaler/57292088#comment110923346_57291876) is to load the data column by column if it reduces the memory load.

However, that would be super impractical in my setting where I just cannot load all data into memory at once.

### Additional context

_No response_

COMMENT:
> Not sure whether this should be considered a new algorithm or not, let me know what you think.

We could introduce an extra `algorithm` or `quantile_method` constructor parameter for the `RobustScaler` class to select the strategy.

Note that in #29907, we are introducing the `quantile_method="averaged_inverted_cdf"` parameter as the future default method to be able to implement the expected weight/repetition semantics when fitting `KBinsDiscretizer` with `strategy="quantile"` and `sample_weight`.

We can problem reuse that parameter in `RobustScaler` (and also add support for `sample_weight`). This parameter could then be expanded to implement `quantile_method="t-digest"` or similar later, and some of those can be made to support `partial_fit` (with or without `subsample`).

Labels: New Feature, Needs Decision - Include Feature
`partial_fit` for `RobustScaler`

### Describe the workflow you want to enable

I would like to be able to use `partial_fit` with the `RobustScaler` preprocessing for streaming cases or when my data doesn't fit in memory.
As I understand from this paper https://sites.cs.ucsb.edu/~suri/psdir/ency.pdf, it would probably only be possible to compute the robustly estimated variance and mean up to some precision epsilon, that could probably become an attribute of the class.

Not sure whether this should be considered a new algorithm or not, let me know what you think.

### Describe your proposed solution

I haven't looked much into it but I think there were 2 approaches:
- use one of the solutions proposed in https://sites.cs.ucsb.edu/~suri/psdir/ency.pdf
- @amueller was suggesting in https://github.com/scikit-learn/scikit-learn/issues/5028#issuecomment-125981597 that binning could be an option. Maybe this is actually mentioned in the paper above

### Describe alternatives you've considered, if relevant

An alternative proposed in this [SO comment](https://stackoverflow.com/questions/57291876/robustscaler-partial-fit-similar-to-minmaxscaler-or-standardscaler/57292088#comment110923346_57291876) is to load the data column by column if it reduces the memory load.

However, that would be super impractical in my setting where I just cannot load all data into memory at once.

### Additional context

_No response_

COMMENT:
Thinking about it, the reservoir sampling approach might not be robust if the data is not i.i.d. or exchangeable, and might as a result lead to very different values that depend on the ordering of the data passed to successive `partial_fit` calls. Other methods are probably more robust to this problem.

Labels: New Feature, Needs Decision - Include Feature
Finding indexes with `np.where(condition)` or `np.asarray(condition).nonzero()`

Throughout the repo, we use `np.where(condition)` for getting indexes, for instance in [SelectorMixin.get_support()](https://github.com/scikit-learn/scikit-learn/blob/fba028b07ed2b4e52dd3719dad0d990837bde28c/sklearn/feature_selection/_base.py#L73), in [SimpleImputer.transform()](https://github.com/scikit-learn/scikit-learn/blob/fba028b07ed2b4e52dd3719dad0d990837bde28c/sklearn/impute/_base.py#L670) and in several of our examples ([example](https://github.com/scikit-learn/scikit-learn/blob/fba028b07ed2b4e52dd3719dad0d990837bde28c/examples/linear_model/plot_sgd_iris.py#L58)).

The numpy documentation [discourages](https://numpy.org/doc/2.1/reference/generated/numpy.where.html) the use of `np.where` with just passing a condition and recommends `np.asarray(condition).nonzero()` instead.

For cleanliness of code, should we adopt this recommendation, at least in the examples? Or are there good reasons why we do that?

COMMENT:
Maybe the note about the issue with subclasses is actually a reference to using `np.where` with a masked array: https://github.com/numpy/numpy/pull/11425#issuecomment-401296841.

Labels: good first issue