Talip7/scikit-learn-issues-multilabel huggingface dataset:
Add TQDM progress bar to .fit

### Describe the workflow you want to enable

Add TQDM progress bar to .fit 

```
from sklearn.svm import SVC
svcmodel.fit(X_train,y_train)
```

### Describe your proposed solution

Add TQDM progress bar to .fit 

### Describe alternatives you've considered, if relevant

Add TQDM progress bar to .fit 

### Additional context

_No response_

COMMENT:
how to use progiter for svcmodel.fit(X_train,y_train) ?
@adrinjalali

Labels: New Feature, Needs Triage
Add TQDM progress bar to .fit

### Describe the workflow you want to enable

Add TQDM progress bar to .fit 

```
from sklearn.svm import SVC
svcmodel.fit(X_train,y_train)
```

### Describe your proposed solution

Add TQDM progress bar to .fit 

### Describe alternatives you've considered, if relevant

Add TQDM progress bar to .fit 

### Additional context

_No response_

COMMENT:
You can't. That would require you changing the libsvm's C++ code.

Labels: New Feature, Needs Triage
DOC broken image link in user guide due to removal of example

### Describe the issue linked to the documentation

The image at the bottom of Section 3.5.1 on https://scikit-learn.org/dev/modules/learning_curve.html is broken, which I believe is due to the removal of some example in #29936. We may want to rework or remove the last part.

### Suggest a potential alternative/fix

NA

COMMENT:
I believe the broken link was intended to reference the following example:

[https://scikit-learn.org/1.5/auto_examples/model_selection/plot_validation_curve.html](https://scikit-learn.org/1.5/auto_examples/model_selection/plot_validation_curve.html)

We can simply reintroduce it into the current example to maintain the logical flow.

Would it make sense to fix this by "reverting part of the example"? If not, I'd be happy to take on this issue.

CC: @glemaitre

Labels: Documentation
DOC broken image link in user guide due to removal of example

### Describe the issue linked to the documentation

The image at the bottom of Section 3.5.1 on https://scikit-learn.org/dev/modules/learning_curve.html is broken, which I believe is due to the removal of some example in #29936. We may want to rework or remove the last part.

### Suggest a potential alternative/fix

NA

COMMENT:
Yep I forgot to check the back reference to the example. I'll make a PR to solve the issue.

Labels: Documentation
⚠️ CI failed on linux_arm64_wheel (last failure: Oct 13, 2024) ⚠️

**CI failed on [linux_arm64_wheel](https://cirrus-ci.com/build/5764259953508352)** (Oct 13, 2024)

COMMENT:
Probably due to https://github.com/scikit-learn/scikit-learn/pull/29861, the same CI issue was seen the PR in https://github.com/scikit-learn/scikit-learn/pull/29861/commits/b44d4d9ef9bafa0988489f9e434b91f4aff8a1b0 and then it was merged running only the Github wheels CI ...

Labels: Needs Triage
⚠️ CI failed on linux_arm64_wheel (last failure: Oct 13, 2024) ⚠️

**CI failed on [linux_arm64_wheel](https://cirrus-ci.com/build/5764259953508352)** (Oct 13, 2024)

COMMENT:
By the way it would be good to figure out why 4 seemingly identical issues have been opened ... #30052 #30053 #30054 #30055

Labels: Needs Triage
DOC misleading version added info for `cv_results["n_features"]` in `RFECV`

### Describe the bug

I'm using the scikit-learn version 1.3.0. When I use 
`rfecv = RFECV(....)
        rfecv.fit(X, y)
        print(rfecv.cv_results_)`
that code gives me a traceback: `KeyError: 'n_features'`
I seen that key in the doc.
<img width="709" alt="image" src="https://github.com/user-attachments/assets/135b5a58-c7fa-4cbd-97c6-2778317457cd">



### Steps/Code to Reproduce

```python
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import StratifiedKFold
from sklearn.feature_selection import RFECV

X, y = load_iris(return_X_y=True)
model = RandomForestClassifier(random_state=42)
cv = StratifiedKFold(n_splits=5)

rfecv = RFECV(
    estimator=model,
    step=8,
    cv=cv,
    scoring='accuracy',
    n_jobs=-1,
    verbose=10
)
rfecv.fit(X, y)

print(rfecv.cv_results_["n_features"])
print(rfecv.n_features_)
```

### Expected Results

It should have the key.

### Actual Results

    print(rfecv.cv_results_["n_features"])
KeyError: 'n_features'

### Versions

```shell
1.3.0
```

COMMENT:
`n_features` has been added in 1.5: https://github.com/scikit-learn/scikit-learn/pull/28670

We could have a PR that move the `..versionadded` under the `cv_results_` add and a `..versionadded:: 1.5` specifically for the `n_features` key.

Labels: Documentation
DOC misleading version added info for `cv_results["n_features"]` in `RFECV`

### Describe the bug

I'm using the scikit-learn version 1.3.0. When I use 
`rfecv = RFECV(....)
        rfecv.fit(X, y)
        print(rfecv.cv_results_)`
that code gives me a traceback: `KeyError: 'n_features'`
I seen that key in the doc.
<img width="709" alt="image" src="https://github.com/user-attachments/assets/135b5a58-c7fa-4cbd-97c6-2778317457cd">



### Steps/Code to Reproduce

```python
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import StratifiedKFold
from sklearn.feature_selection import RFECV

X, y = load_iris(return_X_y=True)
model = RandomForestClassifier(random_state=42)
cv = StratifiedKFold(n_splits=5)

rfecv = RFECV(
    estimator=model,
    step=8,
    cv=cv,
    scoring='accuracy',
    n_jobs=-1,
    verbose=10
)
rfecv.fit(X, y)

print(rfecv.cv_results_["n_features"])
print(rfecv.n_features_)
```

### Expected Results

It should have the key.

### Actual Results

    print(rfecv.cv_results_["n_features"])
KeyError: 'n_features'

### Versions

```shell
1.3.0
```

COMMENT:
Thank you bro!

Labels: Documentation
Add partial_fit Functionality to LinearDiscriminantAnalysis Classifier

### Describe the workflow you want to enable

Currently, Scikit-learn's LinearDiscriminantAnalysis (LDA) classifier does not support incremental learning through the partial_fit method.   This poses challenges when processing large scale classification problems for which the full training set might not fit in memory. 

### Describe your proposed solution

Implementing partial_fit would allow users to train the LDA model incrementally, updating the model with batches of data as they become available.  This is consistent with the existing scikit-learn API, which currently supports this functionality for various other models such as the Naive Bayes classifiers.

### Describe alternatives you've considered, if relevant

It may prove to be difficult to support all possible solvers and functionality (e.g.., shrinkage).  As an alternative, an IncrementalLinearDiscriminantAnalysis classifier could be introduced that doesn't support all of the LInearDiscriminantAnalysis parameter options.

### Additional context

I've implemented a basic extension of the current LinearDiscriminantAnalysis classifier that accomplishes this, and am willing to do the development.

COMMENT:
@wcscr Do you have any resources on a reference implementation?

Do you implement the online version based on an incremental SVD?

Labels: New Feature
Implement the two-parameter Box-Cox transform variant

### Describe the workflow you want to enable

Currently, ony the single-parameter box-cox is implemented in sklearn.preprocessing.power_transform

The two parameter variant is defined as

![](https://wikimedia.org/api/rest_v1/media/math/render/svg/f0bcf29e7ad0c8261a9f15f4abd9468c9e73cbaf)

where both the parameters are to be fit from data via MLE

### Describe your proposed solution

Add the two-parameter variant as a new method to sklearn.preprocessing.power_transform

### Describe alternatives you've considered, if relevant

Of course, the default yeo-johnson transform can be used for negative data, but that is mathematically different 

### Additional context

wikipedia page: https://en.wikipedia.org/wiki/Power_transform

COMMENT:
Since we are wrapping the scipy.stats.boxcox function, I think that this feature needs to be implemented in SciPy instead of scikit-learn: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.boxcox.html

Labels: New Feature
OneVsRestClassifier cannot be used with TunedThresholdClassifierCV

https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/model_selection/_classification_threshold.py#L386

When predict is called on `OneVsRestClassifier`, it calls `predict_proba` on the underlying classifier.

If the underlying is a `TunedThresholdClassifierCV`, it redirects to the underlying estimator instead.

On the line referenced, I think that `OneVsRestClassifier` should check if the estimator is `TunedThresholdClassifierCV`, and if so use the `best_threshold_` instead of 0.5

COMMENT:
Yep, it seems that `OneVsRestClassifier` should treat `TunedThresholdClassifierCV` differently. Actually, all meta-estimators in the multiclass module should deal differently with it.

Labels: Bug, Needs Decision
OneVsRestClassifier cannot be used with TunedThresholdClassifierCV

https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/model_selection/_classification_threshold.py#L386

When predict is called on `OneVsRestClassifier`, it calls `predict_proba` on the underlying classifier.

If the underlying is a `TunedThresholdClassifierCV`, it redirects to the underlying estimator instead.

On the line referenced, I think that `OneVsRestClassifier` should check if the estimator is `TunedThresholdClassifierCV`, and if so use the `best_threshold_` instead of 0.5

COMMENT:
Can I work on this? @glemaitre

Labels: Bug, Needs Decision
OneVsRestClassifier cannot be used with TunedThresholdClassifierCV

https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/model_selection/_classification_threshold.py#L386

When predict is called on `OneVsRestClassifier`, it calls `predict_proba` on the underlying classifier.

If the underlying is a `TunedThresholdClassifierCV`, it redirects to the underlying estimator instead.

On the line referenced, I think that `OneVsRestClassifier` should check if the estimator is `TunedThresholdClassifierCV`, and if so use the `best_threshold_` instead of 0.5

COMMENT:
I think this might be blurring the abstraction layers and moves us to a direction where maintaining things might be quite dirty.

In this case, the core of the issue lies under such a line as this one:

https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/model_selection/_classification_threshold.py#L66

where we're making a _decision_ based on the output of a method from a sub-estimator (i.e. `predict_proba`), which should be done by using another method (i.e. `predict`), instead of trying to replicate what happens in the sub-estimator's `predict` outside that estimator.

Labels: Bug, Needs Decision
OneVsRestClassifier cannot be used with TunedThresholdClassifierCV

https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/model_selection/_classification_threshold.py#L386

When predict is called on `OneVsRestClassifier`, it calls `predict_proba` on the underlying classifier.

If the underlying is a `TunedThresholdClassifierCV`, it redirects to the underlying estimator instead.

On the line referenced, I think that `OneVsRestClassifier` should check if the estimator is `TunedThresholdClassifierCV`, and if so use the `best_threshold_` instead of 0.5

COMMENT:
Yes I considered that perspective too.

Labels: Bug, Needs Decision
OneVsRestClassifier cannot be used with TunedThresholdClassifierCV

https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/model_selection/_classification_threshold.py#L386

When predict is called on `OneVsRestClassifier`, it calls `predict_proba` on the underlying classifier.

If the underlying is a `TunedThresholdClassifierCV`, it redirects to the underlying estimator instead.

On the line referenced, I think that `OneVsRestClassifier` should check if the estimator is `TunedThresholdClassifierCV`, and if so use the `best_threshold_` instead of 0.5

COMMENT:
> I think this might be blurring the abstraction layers and moves us to a direction where maintaining things might be quite dirty.

Basically, what I said was not the solution but an assessment. I completely agree that we should not special case.

> where we're making a decision based on the output of a method from a sub-estimator (i.e. predict_proba), which should be done by using another method (i.e. predict), instead of trying to replicate what happens in the sub-estimator's predict outside that estimator.

Ouch, the only way that I see how to do that is to create a `_predict(X, cut_off_threshold=None)` method for each estimator such the `TunedThresholdClassifierCV` is calling it with a given `cut_off_threshold`.

Labels: Bug, Needs Decision
OneVsRestClassifier cannot be used with TunedThresholdClassifierCV

https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/model_selection/_classification_threshold.py#L386

When predict is called on `OneVsRestClassifier`, it calls `predict_proba` on the underlying classifier.

If the underlying is a `TunedThresholdClassifierCV`, it redirects to the underlying estimator instead.

On the line referenced, I think that `OneVsRestClassifier` should check if the estimator is `TunedThresholdClassifierCV`, and if so use the `best_threshold_` instead of 0.5

COMMENT:
I guess what I'm wondering is, at least in this case, why is this line even there?

>  return classes[map_thresholded_score_to_label[(y_score >= threshold).astype(int)]] 

doesn't calling `predict` do the same thing?

Labels: Bug, Needs Decision
OneVsRestClassifier cannot be used with TunedThresholdClassifierCV

https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/model_selection/_classification_threshold.py#L386

When predict is called on `OneVsRestClassifier`, it calls `predict_proba` on the underlying classifier.

If the underlying is a `TunedThresholdClassifierCV`, it redirects to the underlying estimator instead.

On the line referenced, I think that `OneVsRestClassifier` should check if the estimator is `TunedThresholdClassifierCV`, and if so use the `best_threshold_` instead of 0.5

COMMENT:
`predict` is calling this function. I'm not really sure what is the issue that you are pointing out here, actually. This private function is shared with the `FixedThresholdClassifier` as well and this it makes sense to me that it is not only predict because the `predict` from the `TunedThresholdClassifierCV` and `FixedThresholdClassifier` are different.

Labels: Bug, Needs Decision
OneVsRestClassifier cannot be used with TunedThresholdClassifierCV

https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/model_selection/_classification_threshold.py#L386

When predict is called on `OneVsRestClassifier`, it calls `predict_proba` on the underlying classifier.

If the underlying is a `TunedThresholdClassifierCV`, it redirects to the underlying estimator instead.

On the line referenced, I think that `OneVsRestClassifier` should check if the estimator is `TunedThresholdClassifierCV`, and if so use the `best_threshold_` instead of 0.5

COMMENT:
Looking a bit more at the code of `OneVsRestClassifier`:

https://github.com/scikit-learn/scikit-learn/blob/66bf6133508c1d6163bd2812b6771f878a8ead63/sklearn/multiclass.py#L497-L518

I'm not really sure why it does not call the `predict` function itself. The `_predict_binary` is actually what I would expect `predict` to do.

Labels: Bug, Needs Decision
OneVsRestClassifier cannot be used with TunedThresholdClassifierCV

https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/model_selection/_classification_threshold.py#L386

When predict is called on `OneVsRestClassifier`, it calls `predict_proba` on the underlying classifier.

If the underlying is a `TunedThresholdClassifierCV`, it redirects to the underlying estimator instead.

On the line referenced, I think that `OneVsRestClassifier` should check if the estimator is `TunedThresholdClassifierCV`, and if so use the `best_threshold_` instead of 0.5

COMMENT:
> I'm not really sure why it does not call the predict function itself. The _predict_binary is actually what I would expect predict to do.

Yes, that's what I meant.

Labels: Bug, Needs Decision
OneVsRestClassifier cannot be used with TunedThresholdClassifierCV

https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/model_selection/_classification_threshold.py#L386

When predict is called on `OneVsRestClassifier`, it calls `predict_proba` on the underlying classifier.

If the underlying is a `TunedThresholdClassifierCV`, it redirects to the underlying estimator instead.

On the line referenced, I think that `OneVsRestClassifier` should check if the estimator is `TunedThresholdClassifierCV`, and if so use the `best_threshold_` instead of 0.5

COMMENT:
> Yes, that's what I meant.

I see. I thought that you referred to the `predict` of the `*ThresholdClassifier*` :).
My brain was upside down :)

Labels: Bug, Needs Decision
OneVsRestClassifier cannot be used with TunedThresholdClassifierCV

https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/model_selection/_classification_threshold.py#L386

When predict is called on `OneVsRestClassifier`, it calls `predict_proba` on the underlying classifier.

If the underlying is a `TunedThresholdClassifierCV`, it redirects to the underlying estimator instead.

On the line referenced, I think that `OneVsRestClassifier` should check if the estimator is `TunedThresholdClassifierCV`, and if so use the `best_threshold_` instead of 0.5

COMMENT:
Hi guys, just taking another look at this, to me the problem lies in _where_ the responsibility of deciding decision boundaries is.

I think that OVR classifier should get the decision boundary/threshold from its underlying class instead of just arbitrariliy deciding 0.5.

0.5 is unique to the classifier. in `TTC` it is dynamic, in `FixedTC` it is set by the user, for all other classifieries 0.5. So I think some attribute like `_threshold` should be on the base classifier fixed at 0.5, then overwritten by the two Tuning classes.

The logic is that all classifiers have is the same `predict` logic:
1. Call predict_proba
2. Get this classifiers threshold boundary(s?)
3. Decide for each class, and return.

The root cause of this issue is 2.

When `OVR.predict` is called, it collects the probabilites from `TTC` (which just wraps the underlying classifier), and then it should also collect the thresholds from `TTC` too instead of deciding 0.5.

The `ClassifierMixin` should have a default `_threshold` of 0.5 to make everything else happy.

This is just one developers perspective. It's about "where does the threshold responsibility lie?" and to me, it is in the classifier itself, it's just that for all of them it is 0.5 except the tuning ones.


The code here
https://github.com/scikit-learn/scikit-learn/blob/6eeb16782ba417b2a00c436da0b83edadfcd7575/sklearn/multiclass.py#L118

Should instead return `estimator._threshold` instead of 0.5, and `ClassifierMixin` should have a default attribute of `_threshold = 0.5`. The tuning classes should overwrite this `_threshold` too when the finish fit/construct. I know it means there will be a `_best_threhold` with the same value but keep both and this should fix everything and I believe is backwards compatible.

Labels: Bug, Needs Decision
One-class SVM probabilistic output

### Describe the workflow you want to enable

LIBSVM introduced one-class probabilistic outputs last year in version 3.31.


### Describe your proposed solution

Add a `probability=True/False` argument to [OneClassSVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.OneClassSVM.html), similar to the [SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC) and [NuSCV](https://scikit-learn.org/stable/modules/generated/sklearn.svm.NuSVC.html#nusvc) 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

One-class probabilistic outputs are based on a density-based binning of decision values as described [here](https://www.csie.ntu.edu.tw/~cjlin/papers/oneclass_prob/oneclass_prob.pdf).

COMMENT:
This would be a challenge because our version of libsvm has diverged since many years. So I'm not sure that we can easily get the changes in.

Labels: New Feature
TfidfVectorizer does not preserve dtype for large size inputs

### Describe the bug

After fitting `TfidfVectorizer`, its `idf_` has `dtype` `np.float64` regardless of the provided `dtype` when the input data are large. The conversion from `np.float32` to `np.float64` happens [here](https://github.com/scikit-learn/scikit-learn/blob/d5082d32de2797f9594c9477f2810c743560a1f1/sklearn/feature_extraction/text.py#L1666).

Not propagating `dtype` to `TfidfTransformer` has been [discussed](https://github.com/scikit-learn/scikit-learn/pull/10443) in the past. Back then passing `dtype` to `np.log` or adding `.astype(dtype)` to `np.log` were not approved.

### Steps/Code to Reproduce

```python
from sklearn.feature_extraction.text import TfidfVectorizer
import pandas as pd
import numpy as np
import uuid

#check for 100 strings with output as np.float32, works fine
small_data=[str(uuid.uuid4()) for i in range(100)]
X = pd.Series(small_data)
vectorizer = TfidfVectorizer(dtype=np.float32)
vectorizer.fit(X)
print(vectorizer.idf_.dtype)

#check for 1000000 strings with output as np.float32
#the output of the following has dtype np.float64
large_data=[str(uuid.uuid4()) for i in range(1000000)]
X = pd.Series(large_data)
vectorizer = TfidfVectorizer(dtype=np.float32)
vectorizer.fit(X)
print(vectorizer.idf_.dtype)
```

### Expected Results

```python
float32
float32
```

### Actual Results

```python
float32
float64
```

### Versions

```shell
System:
    python: 3.10.13 (main, Feb  7 2024, 08:26:19) [Clang 15.0.0 (clang-1500.1.0.2.5)]
executable: .../bin/python
   machine: macOS-14.6.1-arm64-arm-64bit

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 65.5.0
        numpy: 1.23.5
        scipy: 1.14.1
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libopenblas
       filepath: .../lib/python3.10/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
        version: 0.3.20
threading_layer: pthreads
   architecture: armv8

       user_api: openmp
   internal_api: openmp
    num_threads: 12
         prefix: libomp
       filepath: .../lib/python3.10/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
```

COMMENT:
I could reproduce the behaviour by matching the exact version of the dependencies given by @chrispyl 

With NumPy >= 2, I get the proper behaviour thought. It is not ideal that we have a different behaviour depending on the version of NumPy.

I'll investigate a bit more.

Labels: Bug