Talip7/scikit-learn-issues-multilabel huggingface dataset:
Refactor _check_partial_fit_first_call to separate validation from state modification

### Describe the workflow you want to enable

This change aims to improve the architectural design of `partial_fit` classes validation by separating the validation logic from state modification. This will make the code more maintainable and follow better the single responsibility principle.

Currently, `_check_partial_fit_first_call` both validates classes and modifies the classifier's state by setting `classes_`. This creates a hidden side effect that's not immediately obvious inside of the classifier.

### Describe your proposed solution

Split the current `_check_partial_fit_first_call` into a new validation function that only handles validation and returns the necessary information, letting the classifier handle its own state modification:
```python
def _validate_partial_fit_classes(clf, classes=None):
    """Validates classes parameter for partial_fit without modifying state.
    
    Returns:
        tuple: (is_first_call: bool, validated_classes: ndarray or None)
    """
    if getattr(clf, "classes_", None) is None:
        if classes is None:
            raise ValueError("classes must be passed on the first call to partial_fit.")
        return True, unique_labels(classes)
        
    if classes is not None:
        validated_classes = unique_labels(classes)
        if not np.array_equal(clf.classes_, validated_classes):
            raise ValueError(
                f"`classes={classes!r}` is not the same as on last call "
                f"to partial_fit, was: {clf.classes_!r}"
            )
    
    return False, None
```

This allows classifiers to use it like:

```python
def partial_fit(self, X, y, classes=None):
    is_first_call, validated_classes = _validate_partial_fit_classes(self, classes)
    
    if is_first_call:
        self.classes_ = validated_classes
        
    # Rest of partial_fit implementation...
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

This change:
- Is backwards compatible (no API changes)
- Improves code maintainability

The change would affect all classifiers that use partial_fit, but would not change their external behavior.

COMMENT:
Closing due to lack of info.

Labels: Needs Info
Change forcing sequence in newton-cg solver of LogisticRegression

### Describe the workflow you want to enable

I'd like to have faster convergence of the `"newton-cg"` solver of `LogisticRegression` based on scientific publications with empirical studies as done in [A Study on Truncated Newton Methods for Linear Classification (2022)](https://doi.org/10.1109/TNNLS.2020.3045836) (free [pdf](https://www.csie.ntu.edu.tw/~cjlin/papers/tncg/tncg.pdf) version).

### Describe your proposed solution

It is about the inner stopping criterion in a truncated Newton solver, i.e. when should the inner solver for "hessian @ coefficients = -gradient" stop.

$eta = \eta$ is the forcing sequence.

#### Current stopping criterion
$residual ratio = \frac{\rVert res\lVert_1}{\rVert grad \lVert_1} \leq \eta$ with $res = residual = grad - hess @ coef$ and $\eta = \min([0.5, \sqrt{\rVert grad \lVert_1]})$ (this eta is called adaptive forcing sequence.

#### Proposed stopping criterion
As recommended by Chapter VII.
- Replace residual ratio with the quadratic approximation ratio $j\frac{Q_j - Q_{j-1}}{Q_j}$ and $Q_j = grad @ coef_j + \frac{1}{2} coef_j^T @ hessian @ coef_j$ and $j$ is the inner iteration number.
- Optionally replace L1-norm by L2-norm. For the quadratic ratio, this does not matter much.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
We had a discussion about a similar kind of suggestion (on a very different estimator) where the question is whether we should include such an improvement.

I think if empirically this makes the convergence faster, I wouldn't mind including it. However, I'm not sure what to think about whether to include it with a new constructor arg, or simply change the implementation.

cc @agramfort @ogrisel @antoinebaker

Labels: New Feature, Performance
Change forcing sequence in newton-cg solver of LogisticRegression

### Describe the workflow you want to enable

I'd like to have faster convergence of the `"newton-cg"` solver of `LogisticRegression` based on scientific publications with empirical studies as done in [A Study on Truncated Newton Methods for Linear Classification (2022)](https://doi.org/10.1109/TNNLS.2020.3045836) (free [pdf](https://www.csie.ntu.edu.tw/~cjlin/papers/tncg/tncg.pdf) version).

### Describe your proposed solution

It is about the inner stopping criterion in a truncated Newton solver, i.e. when should the inner solver for "hessian @ coefficients = -gradient" stop.

$eta = \eta$ is the forcing sequence.

#### Current stopping criterion
$residual ratio = \frac{\rVert res\lVert_1}{\rVert grad \lVert_1} \leq \eta$ with $res = residual = grad - hess @ coef$ and $\eta = \min([0.5, \sqrt{\rVert grad \lVert_1]})$ (this eta is called adaptive forcing sequence.

#### Proposed stopping criterion
As recommended by Chapter VII.
- Replace residual ratio with the quadratic approximation ratio $j\frac{Q_j - Q_{j-1}}{Q_j}$ and $Q_j = grad @ coef_j + \frac{1}{2} coef_j^T @ hessian @ coef_j$ and $j$ is the inner iteration number.
- Optionally replace L1-norm by L2-norm. For the quadratic ratio, this does not matter much.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
2 things:
- can you share empirical evidence of improvement eg using https://github.com/benchopt/benchmark_logreg_l2 ?
- if it's actually faster and if the results seem to change very lightly maybe we can just replace the inner default but if it's a bigger change for end users I would consider exposing the new one with solver newton-cg-ratio and saying that newton-cg is deprecated and in 2 releases we make newton-cg-ratio become newton-cg?

Labels: New Feature, Performance
Change forcing sequence in newton-cg solver of LogisticRegression

### Describe the workflow you want to enable

I'd like to have faster convergence of the `"newton-cg"` solver of `LogisticRegression` based on scientific publications with empirical studies as done in [A Study on Truncated Newton Methods for Linear Classification (2022)](https://doi.org/10.1109/TNNLS.2020.3045836) (free [pdf](https://www.csie.ntu.edu.tw/~cjlin/papers/tncg/tncg.pdf) version).

### Describe your proposed solution

It is about the inner stopping criterion in a truncated Newton solver, i.e. when should the inner solver for "hessian @ coefficients = -gradient" stop.

$eta = \eta$ is the forcing sequence.

#### Current stopping criterion
$residual ratio = \frac{\rVert res\lVert_1}{\rVert grad \lVert_1} \leq \eta$ with $res = residual = grad - hess @ coef$ and $\eta = \min([0.5, \sqrt{\rVert grad \lVert_1]})$ (this eta is called adaptive forcing sequence.

#### Proposed stopping criterion
As recommended by Chapter VII.
- Replace residual ratio with the quadratic approximation ratio $j\frac{Q_j - Q_{j-1}}{Q_j}$ and $Q_j = grad @ coef_j + \frac{1}{2} coef_j^T @ hessian @ coef_j$ and $j$ is the inner iteration number.
- Optionally replace L1-norm by L2-norm. For the quadratic ratio, this does not matter much.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I'd like to avoid exposing anything new.

Labels: New Feature, Performance
⚠️ CI failed on Wheel builder (last failure: Oct 27, 2024) ⚠️

**CI failed on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/11537349026)** (Oct 27, 2024)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://github.com/scikit-learn/scikit-learn/actions/runs/11546977899) on Oct 28, 2024

Labels: Needs Triage
average_precision_score not working as expected

### Describe the bug

When compute AP with average_precision_score, I get unexpected results. The y_scores (output from the models) are very low for positive samples, so my AP should be very low. Instead I get a perfect 1.0 AP score.

### Steps/Code to Reproduce

```python
from sklearn.metrics import average_precision_score

# labels
y_true = [1, 1, 1, 1, 1]  # 5 positives
# Predicted scores
y_scores = [0.1, 0.3, 0.1, 0.2, 0.1]  # Model's confidence in predictions

# Calculate average precision
average_precision = average_precision_score(y_true, y_scores)
print("AP score:", average_precision)
```

### Expected Results

An AP score close to 0.

### Actual Results

AP score: 1.0

### Versions

```shell
System:
    python: 3.9.19 (main, May  6 2024, 19:43:03)  [GCC 11.2.0]
executable: /bin/python
   machine: Linux-5.15.0-121-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 75.1.0
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: None
       pandas: 1.4.0
   matplotlib: 3.5.3
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scipy.libs/libopenblasp-r0-01191904.3.27.so
        version: 0.3.27
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
Precision is the ratio TP / (TP + FP). As all the true labels are positive, there will be no false positive (FP=0), and it seems reasonable that the precision is always 1.

Maybe try using other metrics to evaluate?

Labels: Bug
average_precision_score not working as expected

### Describe the bug

When compute AP with average_precision_score, I get unexpected results. The y_scores (output from the models) are very low for positive samples, so my AP should be very low. Instead I get a perfect 1.0 AP score.

### Steps/Code to Reproduce

```python
from sklearn.metrics import average_precision_score

# labels
y_true = [1, 1, 1, 1, 1]  # 5 positives
# Predicted scores
y_scores = [0.1, 0.3, 0.1, 0.2, 0.1]  # Model's confidence in predictions

# Calculate average precision
average_precision = average_precision_score(y_true, y_scores)
print("AP score:", average_precision)
```

### Expected Results

An AP score close to 0.

### Actual Results

AP score: 1.0

### Versions

```shell
System:
    python: 3.9.19 (main, May  6 2024, 19:43:03)  [GCC 11.2.0]
executable: /bin/python
   machine: Linux-5.15.0-121-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 75.1.0
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: None
       pandas: 1.4.0
   matplotlib: 3.5.3
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scipy.libs/libopenblasp-r0-01191904.3.27.so
        version: 0.3.27
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
Hello! I'd like to work on this issue if it's available. Would it make sense to add a parameter to control the inclusion of the final recall=1 point in the AP calculation to better align with other standards?

Labels: Bug
average_precision_score not working as expected

### Describe the bug

When compute AP with average_precision_score, I get unexpected results. The y_scores (output from the models) are very low for positive samples, so my AP should be very low. Instead I get a perfect 1.0 AP score.

### Steps/Code to Reproduce

```python
from sklearn.metrics import average_precision_score

# labels
y_true = [1, 1, 1, 1, 1]  # 5 positives
# Predicted scores
y_scores = [0.1, 0.3, 0.1, 0.2, 0.1]  # Model's confidence in predictions

# Calculate average precision
average_precision = average_precision_score(y_true, y_scores)
print("AP score:", average_precision)
```

### Expected Results

An AP score close to 0.

### Actual Results

AP score: 1.0

### Versions

```shell
System:
    python: 3.9.19 (main, May  6 2024, 19:43:03)  [GCC 11.2.0]
executable: /bin/python
   machine: Linux-5.15.0-121-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 75.1.0
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: None
       pandas: 1.4.0
   matplotlib: 3.5.3
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scipy.libs/libopenblasp-r0-01191904.3.27.so
        version: 0.3.27
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
I believe that would make a lot of sense yes!

Labels: Bug
average_precision_score not working as expected

### Describe the bug

When compute AP with average_precision_score, I get unexpected results. The y_scores (output from the models) are very low for positive samples, so my AP should be very low. Instead I get a perfect 1.0 AP score.

### Steps/Code to Reproduce

```python
from sklearn.metrics import average_precision_score

# labels
y_true = [1, 1, 1, 1, 1]  # 5 positives
# Predicted scores
y_scores = [0.1, 0.3, 0.1, 0.2, 0.1]  # Model's confidence in predictions

# Calculate average precision
average_precision = average_precision_score(y_true, y_scores)
print("AP score:", average_precision)
```

### Expected Results

An AP score close to 0.

### Actual Results

AP score: 1.0

### Versions

```shell
System:
    python: 3.9.19 (main, May  6 2024, 19:43:03)  [GCC 11.2.0]
executable: /bin/python
   machine: Linux-5.15.0-121-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 75.1.0
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: None
       pandas: 1.4.0
   matplotlib: 3.5.3
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scipy.libs/libopenblasp-r0-01191904.3.27.so
        version: 0.3.27
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
From you example, you will get the following PR curve:

```python
from sklearn.metric simport PrecisionRecallDisplay

PrecisionRecallDisplay.from_predictions(y_true, y_scores)
```

![image](https://github.com/user-attachments/assets/2b2d1ac2-f194-4001-8ce2-e807745ef142)

So for this case, it looks like the average precision is correct.

Labels: Bug
average_precision_score not working as expected

### Describe the bug

When compute AP with average_precision_score, I get unexpected results. The y_scores (output from the models) are very low for positive samples, so my AP should be very low. Instead I get a perfect 1.0 AP score.

### Steps/Code to Reproduce

```python
from sklearn.metrics import average_precision_score

# labels
y_true = [1, 1, 1, 1, 1]  # 5 positives
# Predicted scores
y_scores = [0.1, 0.3, 0.1, 0.2, 0.1]  # Model's confidence in predictions

# Calculate average precision
average_precision = average_precision_score(y_true, y_scores)
print("AP score:", average_precision)
```

### Expected Results

An AP score close to 0.

### Actual Results

AP score: 1.0

### Versions

```shell
System:
    python: 3.9.19 (main, May  6 2024, 19:43:03)  [GCC 11.2.0]
executable: /bin/python
   machine: Linux-5.15.0-121-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 75.1.0
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: None
       pandas: 1.4.0
   matplotlib: 3.5.3
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scipy.libs/libopenblasp-r0-01191904.3.27.so
        version: 0.3.27
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
On your other example, the last point that you have corresponds to the prevalence of the positive label estimated from the data.

This would be your chance level:
![image](https://github.com/user-attachments/assets/4d036401-956b-44bc-8c8c-3ae242cba71a)

This would be the score of a `DummyClassifier(strategy="constant", constant=1)`. It means that any predictor providing a score under this threshold do worse that a rule that does not learn any statistical association between the data and the target. So this is reasonable to me.

Labels: Bug
average_precision_score not working as expected

### Describe the bug

When compute AP with average_precision_score, I get unexpected results. The y_scores (output from the models) are very low for positive samples, so my AP should be very low. Instead I get a perfect 1.0 AP score.

### Steps/Code to Reproduce

```python
from sklearn.metrics import average_precision_score

# labels
y_true = [1, 1, 1, 1, 1]  # 5 positives
# Predicted scores
y_scores = [0.1, 0.3, 0.1, 0.2, 0.1]  # Model's confidence in predictions

# Calculate average precision
average_precision = average_precision_score(y_true, y_scores)
print("AP score:", average_precision)
```

### Expected Results

An AP score close to 0.

### Actual Results

AP score: 1.0

### Versions

```shell
System:
    python: 3.9.19 (main, May  6 2024, 19:43:03)  [GCC 11.2.0]
executable: /bin/python
   machine: Linux-5.15.0-121-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 75.1.0
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: None
       pandas: 1.4.0
   matplotlib: 3.5.3
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scipy.libs/libopenblasp-r0-01191904.3.27.so
        version: 0.3.27
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
ping @ArturoAmorQ to have a second opinion here.

Labels: Bug