Talip7/scikit-learn-issues-multilabel huggingface dataset:
Fallback value for NaN feature during classification

### Describe the workflow you want to enable

In code like this:

```python
probabilities = model.predict_proba(df)
```

where I need to predict classification probabilities from the features in the dataframe `df`, I could have NaNs. The way things are right now, the method will raise an exception and I would have to clean the dataframe myself. 

### Describe your proposed solution

I would like to have something like:

```python
probabilities = model.predict_proba(df, val_on_nan=-1, val_on_inf=2)
```

such that when the value is nan, the probability is -1 and on inf 2.

### Describe alternatives you've considered, if relevant

I have implemented this in my wrapper class:

https://github.com/acampove/dmu/blob/main/src/dmu/ml/cv_predict.py#L48
https://github.com/acampove/dmu/blob/main/src/dmu/ml/utilities.py#L17

where I patch the nans with zeros and then I replace the probabilities before the return:

https://github.com/acampove/dmu/blob/main/src/dmu/ml/cv_predict.py#L149

feel free to pick up whatever you need from my code.

### Additional context

_No response_

COMMENT:
> handling `nan`s and `inf`s is not trivial and it's not the case that one can simply replace the output of an estimator with a constant value. For instance, in `HistGradientBoosting*` estimators, we natively handle missing values and take them as _information_.
> 
> I'm glad the wrapper works for you, but in general, it's not a solution that we should be including in scikit-learn.
> 
> However, we're happy to improve supporting / handling `nan` in sklearn and there's been significant work in this front in the past years. Very happy to have your contributions on specific estimators as well :)

I think this depends on the usecase. In our case, we have around 0.01% of NaNs in our data. So they are mostly getting in the way of our analysis, we do not see them as information and even if they had some information, it's so little that it's not worth it for us to complicate things and we just drop them. From what I understand, most people out there do that with their NaNs anyway. 

In our case this wrapper just makes things simpler by enabling us to do the dropping in one place instead of implemented many times.

Labels: New Feature, Needs Triage
Fallback value for NaN feature during classification

### Describe the workflow you want to enable

In code like this:

```python
probabilities = model.predict_proba(df)
```

where I need to predict classification probabilities from the features in the dataframe `df`, I could have NaNs. The way things are right now, the method will raise an exception and I would have to clean the dataframe myself. 

### Describe your proposed solution

I would like to have something like:

```python
probabilities = model.predict_proba(df, val_on_nan=-1, val_on_inf=2)
```

such that when the value is nan, the probability is -1 and on inf 2.

### Describe alternatives you've considered, if relevant

I have implemented this in my wrapper class:

https://github.com/acampove/dmu/blob/main/src/dmu/ml/cv_predict.py#L48
https://github.com/acampove/dmu/blob/main/src/dmu/ml/utilities.py#L17

where I patch the nans with zeros and then I replace the probabilities before the return:

https://github.com/acampove/dmu/blob/main/src/dmu/ml/cv_predict.py#L149

feel free to pick up whatever you need from my code.

### Additional context

_No response_

COMMENT:
This sounds like a filtering step at the beginning of a pipeline, which would change the number of samples, which is not supported as of now in sklearn.

Labels: New Feature, Needs Triage
The Affinity Matrix Is NON-BINARY with`affinity="precomputed_nearest_neighbors"`

### Describe the issue linked to the documentation

## Issue Source:

https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/sklearn/cluster/_spectral.py#L452-L454

## Issue Description

The Affinity Matrix Is _non-binary_ with`affinity`=`"precomputed_nearest_neighbors"`. I.e., when a precomputed distance matrix is given as `x`, the affinity matrix from SpectralClustering.fit().affinity_matrix_ is NOT binary (as described in the document). It has 3 values: 0.0, 1.0, and 0.5.

## Reproducible Code Snippet
Generate a random distance ,a
```python
from sklearn.cluster import SpectralClustering
import numpy as np

## generate a random distance matrix --> symmetric
np.random.seed(0)
distmat=np.random.rand(200,200)
distmat=(np.triu(distmat,1)+np.triu(distmat,1).T)/2
print(f"Check asymmetric locations (if any):\t{np.where(distmat!=distmat.T)}")

## affinity matrix 
aff_mat=SpectralClustering(n_clusters=30,affinity='precomputed_nearest_neighbors',assign_labels='discretize', n_neighbors=50 ,n_jobs=-1).fit(distmat).affinity_matrix_.toarray()
print(f"Unique values (ought to be 'binary'):\t{np.unique(aff_mat)}")
```

## Machine & Version Info

```python
System:
    python: 3.8.3 (default, Jul  2 2020, 16:21:59)  [GCC 7.3.0]
executable: /opt/share/linux-rocky8-x86_64/gcc-12.2.0/anaconda3-2020.07-yv6vdwqiouaru27jxhpezh6t6mdpqf3e/bin/python
   machine: Linux-4.18.0-425.3.1.el8.x86_64-x86_64-with-glibc2.10

Python dependencies:
          pip: 20.1.1
   setuptools: 65.6.3
      sklearn: 0.23.1
        numpy: 1.22.3
        scipy: 1.5.0
       Cython: 0.29.21
       pandas: 1.4.2

Built with OpenMP: True
```

### Suggest a potential alternative/fix

Since the affinity matrix is calculated as  `(connectivity+connectivity.T)*0.5` [source_code](https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/sklearn/cluster/_spectral.py#L715C8-L720C74), and that the `connectivity` is calculated by `kneighbors_graph` [source_code](https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/sklearn/neighbors/_base.py#L998C5-L1002C1), it is intrinsically not symmetric -- i might be j's K nearest neighbor, while j could not be i's when `n_quirey `== `n_samples`.

COMMENT:
@TomDLT do you remember why this is done the way it is?

Labels: Documentation, Needs Investigation
DOC grammar issue in the governance page

### Describe the issue linked to the documentation

In the governance page at line 
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L70
"GitHub" is referred to as `github`
However, in the other references, such as at 
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L174
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L177
It is correctly written as `GitHub`. 

### Suggest a potential alternative/fix

To maintain consistency throughout the document, we should change `github` to `GitHub` at line no. 70 on governance page.

COMMENT:
@glemaitre, could you please review the issue and confirm the triage? Thank you!

Labels: Documentation
DOC grammar issue in the governance page

### Describe the issue linked to the documentation

In the governance page at line 
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L70
"GitHub" is referred to as `github`
However, in the other references, such as at 
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L174
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L177
It is correctly written as `GitHub`. 

### Suggest a potential alternative/fix

To maintain consistency throughout the document, we should change `github` to `GitHub` at line no. 70 on governance page.

COMMENT:
Sorry this is not my week of triage.

Labels: Documentation
DOC grammar issue in the governance page

### Describe the issue linked to the documentation

In the governance page at line 
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L70
"GitHub" is referred to as `github`
However, in the other references, such as at 
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L174
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L177
It is correctly written as `GitHub`. 

### Suggest a potential alternative/fix

To maintain consistency throughout the document, we should change `github` to `GitHub` at line no. 70 on governance page.

COMMENT:
Feel free to submit a fix @AdityaInnovates

Labels: Documentation
DOC grammar issue in the governance page

### Describe the issue linked to the documentation

In the governance page at line 
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L70
"GitHub" is referred to as `github`
However, in the other references, such as at 
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L174
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L177
It is correctly written as `GitHub`. 

### Suggest a potential alternative/fix

To maintain consistency throughout the document, we should change `github` to `GitHub` at line no. 70 on governance page.

COMMENT:
Can I take this issue as my first contribution to scikit-learn ?

Labels: Documentation
DOC grammar issue in the governance page

### Describe the issue linked to the documentation

In the governance page at line 
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L70
"GitHub" is referred to as `github`
However, in the other references, such as at 
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L174
https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L177
It is correctly written as `GitHub`. 

### Suggest a potential alternative/fix

To maintain consistency throughout the document, we should change `github` to `GitHub` at line no. 70 on governance page.

COMMENT:
Fixed in https://github.com/scikit-learn/scikit-learn/pull/30210

Labels: Documentation
DOC grammar issue in the governance page

### Describe the issue linked to the documentation

In the governance page at line: https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L161

there is a reference attached to "Enhancement proposals (SLEPs)." 
However, after compiling, it is displayed as "a Enhancement proposals (SLEPs)" which is grammatically incorrect.
Page at: https://scikit-learn.org/stable/governance.html

### Suggest a potential alternative/fix

Fix it by updating the line with 
```
an :ref:`slep`
```

COMMENT:
Feel free to submit a fix.

Labels: Documentation
DOC grammar issue in the governance page

### Describe the issue linked to the documentation

In the governance page at line: https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/doc/governance.rst?plain=1#L161

there is a reference attached to "Enhancement proposals (SLEPs)." 
However, after compiling, it is displayed as "a Enhancement proposals (SLEPs)" which is grammatically incorrect.
Page at: https://scikit-learn.org/stable/governance.html

### Suggest a potential alternative/fix

Fix it by updating the line with 
```
an :ref:`slep`
```

COMMENT:
I would like to take this simple documentation task as my first contribution if there are no objections.

Labels: Documentation
The best model and final model in RANSAC are not same.

### Describe the bug

The best model and final model in RANSAC are not same. Therefore, the final model inliers may not be same as the best model inliers.

In `_ransac.py`,  the following code snippet computes the final model using all inliers so the final model is not same as the best model computed using the selected samples before.

```python
        estimator.fit(X_inlier_best, y_inlier_best, **fit_params_best_idxs_subset)

        self.estimator_ = estimator
        self.inlier_mask_ = inlier_mask_best
```

### Steps/Code to Reproduce

Please debug the code using a custom loss function.  Probably, you would observe the difference for default loss functions as well.

### Expected Results

Different `estimator.coef_` and `estimator.intercept_` for the best and final estimators. Accordingly, `inlier_mask_best` are not same for the best estimator and the final estimator. However, the code uses the best estimator's `inlier_mask_best` for the final estimator.

### Actual Results

The best estimator:

```python
estimator.coef_
array([0.03249012], dtype=float32)
estimator.intercept_
-0.0016712397
```

The final estimator:

```python
estimator.coef_
array([0.03334882], dtype=float32)
estimator.intercept_
-0.0047605336
```

### Versions

```shell
System:
    python: 3.10.9 | packaged by conda-forge | (main, Feb  2 2023, 20:20:04) [GCC 11.3.0]
executable: /opt/conda/bin/python
   machine: Linux

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 75.1.0
        numpy: 1.26.4
        scipy: 1.13.0
       Cython: None
       pandas: 2.1.1
   matplotlib: 3.5.3
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /opt/conda/lib/libgomp.so.1.0.0
        version: None

       user_api: blas
   internal_api: mkl
    num_threads: 4
         prefix: libmkl_rt
       filepath: /opt/conda/lib/libmkl_rt.so.2
        version: 2023.1-Product
threading_layer: intel

       user_api: openmp
   internal_api: openmp
    num_threads: 4
         prefix: libomp
       filepath: /opt/conda/lib/libomp.so
        version: None

       user_api: blas
   internal_api: openblas


       num_threads: 8
       prefix: libopenblas
       filepath: /opt/conda/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-24bff013.3.26.dev.so
        version: 0.3.26.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /opt/conda/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
For me the final refit makes sense. We can probably amend the documentation to explain that `estimator_` is the estimator fitted on the inliers found after the iterative procedure.

Labels: Easy, Documentation
The best model and final model in RANSAC are not same.

### Describe the bug

The best model and final model in RANSAC are not same. Therefore, the final model inliers may not be same as the best model inliers.

In `_ransac.py`,  the following code snippet computes the final model using all inliers so the final model is not same as the best model computed using the selected samples before.

```python
        estimator.fit(X_inlier_best, y_inlier_best, **fit_params_best_idxs_subset)

        self.estimator_ = estimator
        self.inlier_mask_ = inlier_mask_best
```

### Steps/Code to Reproduce

Please debug the code using a custom loss function.  Probably, you would observe the difference for default loss functions as well.

### Expected Results

Different `estimator.coef_` and `estimator.intercept_` for the best and final estimators. Accordingly, `inlier_mask_best` are not same for the best estimator and the final estimator. However, the code uses the best estimator's `inlier_mask_best` for the final estimator.

### Actual Results

The best estimator:

```python
estimator.coef_
array([0.03249012], dtype=float32)
estimator.intercept_
-0.0016712397
```

The final estimator:

```python
estimator.coef_
array([0.03334882], dtype=float32)
estimator.intercept_
-0.0047605336
```

### Versions

```shell
System:
    python: 3.10.9 | packaged by conda-forge | (main, Feb  2 2023, 20:20:04) [GCC 11.3.0]
executable: /opt/conda/bin/python
   machine: Linux

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 75.1.0
        numpy: 1.26.4
        scipy: 1.13.0
       Cython: None
       pandas: 2.1.1
   matplotlib: 3.5.3
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /opt/conda/lib/libgomp.so.1.0.0
        version: None

       user_api: blas
   internal_api: mkl
    num_threads: 4
         prefix: libmkl_rt
       filepath: /opt/conda/lib/libmkl_rt.so.2
        version: 2023.1-Product
threading_layer: intel

       user_api: openmp
   internal_api: openmp
    num_threads: 4
         prefix: libomp
       filepath: /opt/conda/lib/libomp.so
        version: None

       user_api: blas
   internal_api: openblas


       num_threads: 8
       prefix: libopenblas
       filepath: /opt/conda/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-24bff013.3.26.dev.so
        version: 0.3.26.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /opt/conda/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
Ok. However, some of those inliers may not be validated by the loss function as the final model is not same as the best model.

Labels: Easy, Documentation
The best model and final model in RANSAC are not same.

### Describe the bug

The best model and final model in RANSAC are not same. Therefore, the final model inliers may not be same as the best model inliers.

In `_ransac.py`,  the following code snippet computes the final model using all inliers so the final model is not same as the best model computed using the selected samples before.

```python
        estimator.fit(X_inlier_best, y_inlier_best, **fit_params_best_idxs_subset)

        self.estimator_ = estimator
        self.inlier_mask_ = inlier_mask_best
```

### Steps/Code to Reproduce

Please debug the code using a custom loss function.  Probably, you would observe the difference for default loss functions as well.

### Expected Results

Different `estimator.coef_` and `estimator.intercept_` for the best and final estimators. Accordingly, `inlier_mask_best` are not same for the best estimator and the final estimator. However, the code uses the best estimator's `inlier_mask_best` for the final estimator.

### Actual Results

The best estimator:

```python
estimator.coef_
array([0.03249012], dtype=float32)
estimator.intercept_
-0.0016712397
```

The final estimator:

```python
estimator.coef_
array([0.03334882], dtype=float32)
estimator.intercept_
-0.0047605336
```

### Versions

```shell
System:
    python: 3.10.9 | packaged by conda-forge | (main, Feb  2 2023, 20:20:04) [GCC 11.3.0]
executable: /opt/conda/bin/python
   machine: Linux

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 75.1.0
        numpy: 1.26.4
        scipy: 1.13.0
       Cython: None
       pandas: 2.1.1
   matplotlib: 3.5.3
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /opt/conda/lib/libgomp.so.1.0.0
        version: None

       user_api: blas
   internal_api: mkl
    num_threads: 4
         prefix: libmkl_rt
       filepath: /opt/conda/lib/libmkl_rt.so.2
        version: 2023.1-Product
threading_layer: intel

       user_api: openmp
   internal_api: openmp
    num_threads: 4
         prefix: libomp
       filepath: /opt/conda/lib/libomp.so
        version: None

       user_api: blas
   internal_api: openblas


       num_threads: 8
       prefix: libopenblas
       filepath: /opt/conda/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-24bff013.3.26.dev.so
        version: 0.3.26.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /opt/conda/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
Yep but in practice, I don't think this is an issue. We don't report the loss function values so at least we don't expose any discrepancy.

Labels: Easy, Documentation
Refactor _check_partial_fit_first_call to separate validation from state modification

### Describe the workflow you want to enable

This change aims to improve the architectural design of `partial_fit` classes validation by separating the validation logic from state modification. This will make the code more maintainable and follow better the single responsibility principle.

Currently, `_check_partial_fit_first_call` both validates classes and modifies the classifier's state by setting `classes_`. This creates a hidden side effect that's not immediately obvious inside of the classifier.

### Describe your proposed solution

Split the current `_check_partial_fit_first_call` into a new validation function that only handles validation and returns the necessary information, letting the classifier handle its own state modification:
```python
def _validate_partial_fit_classes(clf, classes=None):
    """Validates classes parameter for partial_fit without modifying state.
    
    Returns:
        tuple: (is_first_call: bool, validated_classes: ndarray or None)
    """
    if getattr(clf, "classes_", None) is None:
        if classes is None:
            raise ValueError("classes must be passed on the first call to partial_fit.")
        return True, unique_labels(classes)
        
    if classes is not None:
        validated_classes = unique_labels(classes)
        if not np.array_equal(clf.classes_, validated_classes):
            raise ValueError(
                f"`classes={classes!r}` is not the same as on last call "
                f"to partial_fit, was: {clf.classes_!r}"
            )
    
    return False, None
```

This allows classifiers to use it like:

```python
def partial_fit(self, X, y, classes=None):
    is_first_call, validated_classes = _validate_partial_fit_classes(self, classes)
    
    if is_first_call:
        self.classes_ = validated_classes
        
    # Rest of partial_fit implementation...
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

This change:
- Is backwards compatible (no API changes)
- Improves code maintainability

The change would affect all classifiers that use partial_fit, but would not change their external behavior.

COMMENT:
We like to keep boilerplate code to a minimum if possible, and the current design allows us to do so by letting the function setting the `classes_` attribute. There are other validation functions (like `validate_data`) which mutate the estimator itself.

However, I'd like to know your motivation for the change. What is the usecase where the function as is, doesn't allow for what needs to be done?

Labels: Needs Info