Talip7/scikit-learn-issues-multilabel huggingface dataset:
Allow for passing additional parameters to the estimator's `fit` method in `SequentialFeatureSelector`

### Describe the workflow you want to enable

I would like to be able to pass sample weights to the `fit` method of the estimator in `SequentialFeatureSelector`. (`SelectFromModel` has this feature as well.)

Looking at the code, it seems to me that [`sklearn.model_selection._validation._fit_and_score`](https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/model_selection/_validation.py#L662), which is where the actual scoring takes place, already supports sample weights in cross-validation (i.e., takes care of passing only the weights for samples that are actually part of the fold).

### Describe your proposed solution

Based on my current understanding, `SequentialFeatureSelector.fit()` would need to get an additional argument `fit_params`  (cf. [`SelectFromModel.fit()`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectFromModel.html#sklearn.feature_selection.SelectFromModel.fit)) that is passed on to the `cross_val_score` function called in `SequentialFeatureSelector._get_best_new_feature_score`. Everything else appears to be in place already.

if that's all it takes, I'm happy to prepare a PR. But I might very well be overlooking issues or unintended consequences.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> Just to be sure that I understand correctly: sample_weight would be considered during fitting but not during scoring, right?

Yes.

> If that's the case, I don't think there's much use for a sample_weight parameter right now. At the very least, it would be quite misleading for casual users that just look at the method's signature. I also cannot think of a case where fitting to weighted samples but scoring on unweighted samples is useful.

In that case, we can fix this after SLEP006 is implemented.

Labels: New Feature, module:feature_selection
Allow for passing additional parameters to the estimator's `fit` method in `SequentialFeatureSelector`

### Describe the workflow you want to enable

I would like to be able to pass sample weights to the `fit` method of the estimator in `SequentialFeatureSelector`. (`SelectFromModel` has this feature as well.)

Looking at the code, it seems to me that [`sklearn.model_selection._validation._fit_and_score`](https://github.com/scikit-learn/scikit-learn/blob/2e481f114169396660f0051eee1bcf6bcddfd556/sklearn/model_selection/_validation.py#L662), which is where the actual scoring takes place, already supports sample weights in cross-validation (i.e., takes care of passing only the weights for samples that are actually part of the fold).

### Describe your proposed solution

Based on my current understanding, `SequentialFeatureSelector.fit()` would need to get an additional argument `fit_params`  (cf. [`SelectFromModel.fit()`](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectFromModel.html#sklearn.feature_selection.SelectFromModel.fit)) that is passed on to the `cross_val_score` function called in `SequentialFeatureSelector._get_best_new_feature_score`. Everything else appears to be in place already.

if that's all it takes, I'm happy to prepare a PR. But I might very well be overlooking issues or unintended consequences.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Now fixed with metadata routing.

Labels: New Feature, module:feature_selection
sample_weight parameter in KBinsDiscretizer with kmeans strategy

### Discussed in https://github.com/scikit-learn/scikit-learn/discussions/25208

<div type='discussions-op-text'>

<sup>Originally posted by **glevv** December 19, 2022</sup>
Would adding `sample_weight` support when `strategy='kmeans'` make sense?</div>

Could be useful to give smaller `sample_weight` to outliers so they will have less influence on ssq in KMeans.

COMMENT:
I think it makes sense. It would mean that dispatch `sample_weight` to the `fit` of the `KMeans`.

Labels: New Feature
Handling NaNs in NMF

### Describe the workflow you want to enable

Motivation:
Sparse matrixes are very common in recommender systems problems. And matrix factorization approach is one of the most popular approaches for this task. But recommender systems often have sparse data with a big amount of missing values 
Problem:
In principle, non-negative matrix factorization can work with sparse matrices and optimize the solution based only on the present values. In the scikit-learn implementation, the validation doesn't allow the fit_transform method of NMF to accept sparse matrixes with NaN values

### Describe your proposed solution

- In this file https://github.com/scikit-learn/scikit-learn/blob/98cf537f5/sklearn/decomposition/_nmf.py
In the fit_transform method there is code:
  X = self._validate_data(
            X, accept_sparse=("csr", "csc"), dtype=[np.float64, np.float32]
        )
- The parameter **accept_sparse** with ("csr", "csc") default values will be passed to the check_array method in https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/utils/validation.py so validation accepts a sparse format
- But the check_array method has a parameter **force_all_finite** with a default value of True. And this parameter doesn't allow check_array to accept NaN values even when the sparse format ("csr", "csc") is allowed
- As a result, the method  def _assert_all_finite of the validation.py file throws the following error: ValueError("Input contains NaN")

Can you make it configurable to add force_all_finite='allow_nan' to the _validate_data method in the fit_transform method of NMF?

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Letting the matrix might not be sufficient because a sparse matrix needs to be handled with care and might require to use of specific methods of the `scipy.sparse`.

ping @jeremiedbb that knows better the internal implementation that can now if it would be more or less feasible?

Labels: New Feature, Needs Decision, module:decomposition
Handling NaNs in NMF

### Describe the workflow you want to enable

Motivation:
Sparse matrixes are very common in recommender systems problems. And matrix factorization approach is one of the most popular approaches for this task. But recommender systems often have sparse data with a big amount of missing values 
Problem:
In principle, non-negative matrix factorization can work with sparse matrices and optimize the solution based only on the present values. In the scikit-learn implementation, the validation doesn't allow the fit_transform method of NMF to accept sparse matrixes with NaN values

### Describe your proposed solution

- In this file https://github.com/scikit-learn/scikit-learn/blob/98cf537f5/sklearn/decomposition/_nmf.py
In the fit_transform method there is code:
  X = self._validate_data(
            X, accept_sparse=("csr", "csc"), dtype=[np.float64, np.float32]
        )
- The parameter **accept_sparse** with ("csr", "csc") default values will be passed to the check_array method in https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/utils/validation.py so validation accepts a sparse format
- But the check_array method has a parameter **force_all_finite** with a default value of True. And this parameter doesn't allow check_array to accept NaN values even when the sparse format ("csr", "csc") is allowed
- As a result, the method  def _assert_all_finite of the validation.py file throws the following error: ValueError("Input contains NaN")

Can you make it configurable to add force_all_finite='allow_nan' to the _validate_data method in the fit_transform method of NMF?

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
The issue seems to be about handling nan rather than supporting sparse matrices (which is already the case). Let me rename the issue. 

Regarding nan handling, there is this PR https://github.com/scikit-learn/scikit-learn/pull/8474. It would be interesting to revive this PR, but we need to take a look at the litterature to check if the PR implements a common approach.

Labels: New Feature, Needs Decision, module:decomposition
Handling NaNs in NMF

### Describe the workflow you want to enable

Motivation:
Sparse matrixes are very common in recommender systems problems. And matrix factorization approach is one of the most popular approaches for this task. But recommender systems often have sparse data with a big amount of missing values 
Problem:
In principle, non-negative matrix factorization can work with sparse matrices and optimize the solution based only on the present values. In the scikit-learn implementation, the validation doesn't allow the fit_transform method of NMF to accept sparse matrixes with NaN values

### Describe your proposed solution

- In this file https://github.com/scikit-learn/scikit-learn/blob/98cf537f5/sklearn/decomposition/_nmf.py
In the fit_transform method there is code:
  X = self._validate_data(
            X, accept_sparse=("csr", "csc"), dtype=[np.float64, np.float32]
        )
- The parameter **accept_sparse** with ("csr", "csc") default values will be passed to the check_array method in https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/utils/validation.py so validation accepts a sparse format
- But the check_array method has a parameter **force_all_finite** with a default value of True. And this parameter doesn't allow check_array to accept NaN values even when the sparse format ("csr", "csc") is allowed
- As a result, the method  def _assert_all_finite of the validation.py file throws the following error: ValueError("Input contains NaN")

Can you make it configurable to add force_all_finite='allow_nan' to the _validate_data method in the fit_transform method of NMF?

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
It makes sense, thanks for looking at this request. I agree, it is about handling Missing Values rather than sparse matrixes.
I see, there was work going on in PR [#8474](https://github.com/scikit-learn/scikit-learn/pull/8474)

Labels: New Feature, Needs Decision, module:decomposition
Handling NaNs in NMF

### Describe the workflow you want to enable

Motivation:
Sparse matrixes are very common in recommender systems problems. And matrix factorization approach is one of the most popular approaches for this task. But recommender systems often have sparse data with a big amount of missing values 
Problem:
In principle, non-negative matrix factorization can work with sparse matrices and optimize the solution based only on the present values. In the scikit-learn implementation, the validation doesn't allow the fit_transform method of NMF to accept sparse matrixes with NaN values

### Describe your proposed solution

- In this file https://github.com/scikit-learn/scikit-learn/blob/98cf537f5/sklearn/decomposition/_nmf.py
In the fit_transform method there is code:
  X = self._validate_data(
            X, accept_sparse=("csr", "csc"), dtype=[np.float64, np.float32]
        )
- The parameter **accept_sparse** with ("csr", "csc") default values will be passed to the check_array method in https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/utils/validation.py so validation accepts a sparse format
- But the check_array method has a parameter **force_all_finite** with a default value of True. And this parameter doesn't allow check_array to accept NaN values even when the sparse format ("csr", "csc") is allowed
- As a result, the method  def _assert_all_finite of the validation.py file throws the following error: ValueError("Input contains NaN")

Can you make it configurable to add force_all_finite='allow_nan' to the _validate_data method in the fit_transform method of NMF?

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I revived the PR adding support to missing values in NMF (https://github.com/scikit-learn/scikit-learn/pull/8474). Note that for now, the PR only adds support to missing values in dense datasets. Tell me if you have any question.

Labels: New Feature, Needs Decision, module:decomposition
Handling NaNs in NMF

### Describe the workflow you want to enable

Motivation:
Sparse matrixes are very common in recommender systems problems. And matrix factorization approach is one of the most popular approaches for this task. But recommender systems often have sparse data with a big amount of missing values 
Problem:
In principle, non-negative matrix factorization can work with sparse matrices and optimize the solution based only on the present values. In the scikit-learn implementation, the validation doesn't allow the fit_transform method of NMF to accept sparse matrixes with NaN values

### Describe your proposed solution

- In this file https://github.com/scikit-learn/scikit-learn/blob/98cf537f5/sklearn/decomposition/_nmf.py
In the fit_transform method there is code:
  X = self._validate_data(
            X, accept_sparse=("csr", "csc"), dtype=[np.float64, np.float32]
        )
- The parameter **accept_sparse** with ("csr", "csc") default values will be passed to the check_array method in https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/utils/validation.py so validation accepts a sparse format
- But the check_array method has a parameter **force_all_finite** with a default value of True. And this parameter doesn't allow check_array to accept NaN values even when the sparse format ("csr", "csc") is allowed
- As a result, the method  def _assert_all_finite of the validation.py file throws the following error: ValueError("Input contains NaN")

Can you make it configurable to add force_all_finite='allow_nan' to the _validate_data method in the fit_transform method of NMF?

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
That's wonderful! I am trying it locally. 
You added a test that the NMF imputation is better than SimpleImputer+NMF. That's why I was looking for it.

Please let me know if I can help with writing more tests or documentation.

Labels: New Feature, Needs Decision, module:decomposition
Handling NaNs in NMF

### Describe the workflow you want to enable

Motivation:
Sparse matrixes are very common in recommender systems problems. And matrix factorization approach is one of the most popular approaches for this task. But recommender systems often have sparse data with a big amount of missing values 
Problem:
In principle, non-negative matrix factorization can work with sparse matrices and optimize the solution based only on the present values. In the scikit-learn implementation, the validation doesn't allow the fit_transform method of NMF to accept sparse matrixes with NaN values

### Describe your proposed solution

- In this file https://github.com/scikit-learn/scikit-learn/blob/98cf537f5/sklearn/decomposition/_nmf.py
In the fit_transform method there is code:
  X = self._validate_data(
            X, accept_sparse=("csr", "csc"), dtype=[np.float64, np.float32]
        )
- The parameter **accept_sparse** with ("csr", "csc") default values will be passed to the check_array method in https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/utils/validation.py so validation accepts a sparse format
- But the check_array method has a parameter **force_all_finite** with a default value of True. And this parameter doesn't allow check_array to accept NaN values even when the sparse format ("csr", "csc") is allowed
- As a result, the method  def _assert_all_finite of the validation.py file throws the following error: ValueError("Input contains NaN")

Can you make it configurable to add force_all_finite='allow_nan' to the _validate_data method in the fit_transform method of NMF?

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Tested this functionality locally on data having a lot of NaNs. It worked perfectly.
Notes: 
1) NMF ‘cd’ solver doesn't handle missing values
2) Init method can not be 'nndsvd' / 'nndsvda', 'nndsvdar' in the case of data with missing values

Closing this feature request

![nmf_on_data_with_nan](https://user-images.githubusercontent.com/26848048/219826817-d4f59509-fc45-4a09-ab4d-6ef564f8ab9d.jpeg)

Labels: New Feature, Needs Decision, module:decomposition
Handling NaNs in NMF

### Describe the workflow you want to enable

Motivation:
Sparse matrixes are very common in recommender systems problems. And matrix factorization approach is one of the most popular approaches for this task. But recommender systems often have sparse data with a big amount of missing values 
Problem:
In principle, non-negative matrix factorization can work with sparse matrices and optimize the solution based only on the present values. In the scikit-learn implementation, the validation doesn't allow the fit_transform method of NMF to accept sparse matrixes with NaN values

### Describe your proposed solution

- In this file https://github.com/scikit-learn/scikit-learn/blob/98cf537f5/sklearn/decomposition/_nmf.py
In the fit_transform method there is code:
  X = self._validate_data(
            X, accept_sparse=("csr", "csc"), dtype=[np.float64, np.float32]
        )
- The parameter **accept_sparse** with ("csr", "csc") default values will be passed to the check_array method in https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/utils/validation.py so validation accepts a sparse format
- But the check_array method has a parameter **force_all_finite** with a default value of True. And this parameter doesn't allow check_array to accept NaN values even when the sparse format ("csr", "csc") is allowed
- As a result, the method  def _assert_all_finite of the validation.py file throws the following error: ValueError("Input contains NaN")

Can you make it configurable to add force_all_finite='allow_nan' to the _validate_data method in the fit_transform method of NMF?

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
It would be nice to add NMF to this page after merging https://scikit-learn.org/stable/modules/impute.html

Labels: New Feature, Needs Decision, module:decomposition
Mention that MLPRegressor can function as an autoencoder

### Describe the issue linked to the documentation

The MLPRegressor can function as an autoencoder by passing X as input and target (i.e. X == y).

```python
autoencoder = MLPRegressor(hidden_layer_sizes=(2,), activation="identity")
autoencoder.fit(X, X)
X_auto = autoencoder.predict(X)
```

I use PCA for dimensionality reduction a lot, but kept going to torch for autoencoders for comparison purposes. I thought adding an autoencoder to scikit-learn would be a good idea, but then it hit me that this is equivalent to an autoencoder. Yet this is not mentioned in the docs anywhere.

Relevant articles: 
Docs: https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html
Guide: https://scikit-learn.org/stable/modules/unsupervised_reduction.html#unsupervised-dimensionality-reduction

### Suggest a potential alternative/fix

I think mentioning this in the docs, or in the dimensionality reduction user guide could be a good idea. I'm happy to make a PR, but I'd like to get your input first.

COMMENT:
I would assume that the best is to make an example for the gallery. In the end, it comes to how to define your problem and really any specificity to the MLP model.

Labels: Documentation, module:neural_network
Mention that MLPRegressor can function as an autoencoder

### Describe the issue linked to the documentation

The MLPRegressor can function as an autoencoder by passing X as input and target (i.e. X == y).

```python
autoencoder = MLPRegressor(hidden_layer_sizes=(2,), activation="identity")
autoencoder.fit(X, X)
X_auto = autoencoder.predict(X)
```

I use PCA for dimensionality reduction a lot, but kept going to torch for autoencoders for comparison purposes. I thought adding an autoencoder to scikit-learn would be a good idea, but then it hit me that this is equivalent to an autoencoder. Yet this is not mentioned in the docs anywhere.

Relevant articles: 
Docs: https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html
Guide: https://scikit-learn.org/stable/modules/unsupervised_reduction.html#unsupervised-dimensionality-reduction

### Suggest a potential alternative/fix

I think mentioning this in the docs, or in the dimensionality reduction user guide could be a good idea. I'm happy to make a PR, but I'd like to get your input first.

COMMENT:
Hello, I am interested by this issue and I am new to open source contributions. Could you please assign me this issue ?

Labels: Documentation, module:neural_network
Mention that MLPRegressor can function as an autoencoder

### Describe the issue linked to the documentation

The MLPRegressor can function as an autoencoder by passing X as input and target (i.e. X == y).

```python
autoencoder = MLPRegressor(hidden_layer_sizes=(2,), activation="identity")
autoencoder.fit(X, X)
X_auto = autoencoder.predict(X)
```

I use PCA for dimensionality reduction a lot, but kept going to torch for autoencoders for comparison purposes. I thought adding an autoencoder to scikit-learn would be a good idea, but then it hit me that this is equivalent to an autoencoder. Yet this is not mentioned in the docs anywhere.

Relevant articles: 
Docs: https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html
Guide: https://scikit-learn.org/stable/modules/unsupervised_reduction.html#unsupervised-dimensionality-reduction

### Suggest a potential alternative/fix

I think mentioning this in the docs, or in the dimensionality reduction user guide could be a good idea. I'm happy to make a PR, but I'd like to get your input first.

COMMENT:
Thank you for your response! I am interested in opening a new PR that proposes an example if it is ok with you.

Labels: Documentation, module:neural_network
You should have an issue with your install. Could you reinstall scipy and scikit-learn.

You should have an issue with your install. Could you reinstall scipy and scikit-learn.

You can try the following

```
conda install numpy scipy joblib scikit-learn --force-reinstall
```

_Originally posted by @glemaitre in https://github.com/scikit-learn/scikit-learn/issues/16727#issuecomment-601633984_

COMMENT:
Not sure what it is about. Closing.

Labels: Needs Triage
Segmentation fault occurs when KernelPCA is loaded after torchvision

### Describe the bug

If KernelPCA is loaded after torchvision, then I get a segmentation fault. If I load them in the opposite order then I get no segmentaton fault.

### Steps/Code to Reproduce

```
import torchvision
from sklearn.decomposition import KernelPCA

import numpy as np

transformer = KernelPCA(n_components=7, kernel='rbf')
X_transformed = transformer.fit_transform(np.zeros([100,100]))
```

### Expected Results

No segmentation fault

### Actual Results

Segmentation fault (core dumped)

### Versions

```shell
System:
    python: 3.8.10 (default, Mar 15 2022, 12:22:08)  [GCC 9.4.0]
executable: /usr/bin/python3
   machine: Linux-5.4.0-77-generic-x86_64-with-glibc2.29

Python dependencies:
          pip: 22.1
   setuptools: 45.2.0
      sklearn: 0.24.2
        numpy: 1.22.4
        scipy: 1.4.1
       Cython: 0.29.23
       pandas: 1.4.3
   matplotlib: 3.5.2
       joblib: 1.0.1
threadpoolctl: 2.1.0

Built with OpenMP: True


torchvision version: 0.10.0+cu102
```

COMMENT:
@NicolasHug did you see something similar reported in the torchvision repository?

Labels: Bug, module:decomposition