Talip7/scikit-learn-issues-multilabel huggingface dataset:
average_precision_score not working as expected

### Describe the bug

When compute AP with average_precision_score, I get unexpected results. The y_scores (output from the models) are very low for positive samples, so my AP should be very low. Instead I get a perfect 1.0 AP score.

### Steps/Code to Reproduce

```python
from sklearn.metrics import average_precision_score

# labels
y_true = [1, 1, 1, 1, 1]  # 5 positives
# Predicted scores
y_scores = [0.1, 0.3, 0.1, 0.2, 0.1]  # Model's confidence in predictions

# Calculate average precision
average_precision = average_precision_score(y_true, y_scores)
print("AP score:", average_precision)
```

### Expected Results

An AP score close to 0.

### Actual Results

AP score: 1.0

### Versions

```shell
System:
    python: 3.9.19 (main, May  6 2024, 19:43:03)  [GCC 11.2.0]
executable: /bin/python
   machine: Linux-5.15.0-121-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 75.1.0
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: None
       pandas: 1.4.0
   matplotlib: 3.5.3
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scipy.libs/libopenblasp-r0-01191904.3.27.so
        version: 0.3.27
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
@pooja-singh702 probably you are doing it in good faith, but it feels like you are using ChatGPT or similar and pasting generic answer. I feel this is not particularly helpful in this case since the discussion has kind of reached a conclusion. 

As such I am going to hide your previous two comments. 

Please read our short section about our [Automated contribution policy](https://scikit-learn.org/dev/developers/contributing.html#automated-contributions-policy) where we decided that we can block users if we feel they show repeated offenses of our guidelines. See also https://github.com/scikit-learn/scikit-learn/pull/29287 for more details about the context.

Labels: Bug
average_precision_score not working as expected

### Describe the bug

When compute AP with average_precision_score, I get unexpected results. The y_scores (output from the models) are very low for positive samples, so my AP should be very low. Instead I get a perfect 1.0 AP score.

### Steps/Code to Reproduce

```python
from sklearn.metrics import average_precision_score

# labels
y_true = [1, 1, 1, 1, 1]  # 5 positives
# Predicted scores
y_scores = [0.1, 0.3, 0.1, 0.2, 0.1]  # Model's confidence in predictions

# Calculate average precision
average_precision = average_precision_score(y_true, y_scores)
print("AP score:", average_precision)
```

### Expected Results

An AP score close to 0.

### Actual Results

AP score: 1.0

### Versions

```shell
System:
    python: 3.9.19 (main, May  6 2024, 19:43:03)  [GCC 11.2.0]
executable: /bin/python
   machine: Linux-5.15.0-121-generic-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 75.1.0
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: None
       pandas: 1.4.0
   matplotlib: 3.5.3
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scipy.libs/libopenblasp-r0-01191904.3.27.so
        version: 0.3.27
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
       filepath: user/anaconda3/envs/perception39/lib/python3.9/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
Skimming the discussion I kind of agree with @virchan in https://github.com/scikit-learn/scikit-learn/issues/30147#issuecomment-2461431617 but if I miss something, feel free to reopen.

Labels: Bug
The input_tags.sparse flag is often incorrect

### Describe the bug

If I understood correctly the developer API for tags, `input_tags.sparse` tells us whether an estimator can accept sparse data or not. For many estimators it seems that `input_tags.sparse` is False but should be True.

### Steps/Code to Reproduce

```python
from sklearn.linear_model import LinearRegression
from sklearn.utils import get_tags

reg = LinearRegression()
tags = get_tags(reg)
tags.input_tags.sparse
```

### Expected Results

`True` as `LinearRegression` accepts sparse input data.

### Actual Results

`False`

### Versions

```shell
System:
    python: 3.12.5 | packaged by conda-forge | (main, Aug  8 2024, 18:32:50) [Clang 16.0.6 ]
executable: /Users/abaker/miniforge3/envs/sklearn-dev/bin/python
   machine: macOS-14.5-arm64-arm-64bit

Python dependencies:
      sklearn: 1.6.dev0
          pip: 24.2
   setuptools: 73.0.1
        numpy: 2.1.0
        scipy: 1.14.1
       Cython: 3.0.11
       pandas: 2.2.2
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /Users/abaker/miniforge3/envs/sklearn-dev/lib/libopenblas.0.dylib
        version: 0.3.27
threading_layer: openmp
   architecture: VORTEX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Users/abaker/miniforge3/envs/sklearn-dev/lib/libomp.dylib
        version: None
```

COMMENT:
The current estimator check `_check_estimator_sparse_container` does not rely on the presence of the tag: it just calls the estimator with sparse input and checks that the word "sparse" appears somewhere in the exception message if a `ValueError` is raised.

This is quite a low requirement. We could make this a stronger requirement based on the presence of the `input_tags.sparse` value (not raise any exception in this when `True` and check for exception with an informative error message when `False`).

Labels: Bug, Developer API
The input_tags.sparse flag is often incorrect

### Describe the bug

If I understood correctly the developer API for tags, `input_tags.sparse` tells us whether an estimator can accept sparse data or not. For many estimators it seems that `input_tags.sparse` is False but should be True.

### Steps/Code to Reproduce

```python
from sklearn.linear_model import LinearRegression
from sklearn.utils import get_tags

reg = LinearRegression()
tags = get_tags(reg)
tags.input_tags.sparse
```

### Expected Results

`True` as `LinearRegression` accepts sparse input data.

### Actual Results

`False`

### Versions

```shell
System:
    python: 3.12.5 | packaged by conda-forge | (main, Aug  8 2024, 18:32:50) [Clang 16.0.6 ]
executable: /Users/abaker/miniforge3/envs/sklearn-dev/bin/python
   machine: macOS-14.5-arm64-arm-64bit

Python dependencies:
      sklearn: 1.6.dev0
          pip: 24.2
   setuptools: 73.0.1
        numpy: 2.1.0
        scipy: 1.14.1
       Cython: 3.0.11
       pandas: 2.2.2
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /Users/abaker/miniforge3/envs/sklearn-dev/lib/libopenblas.0.dylib
        version: 0.3.27
threading_layer: openmp
   architecture: VORTEX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Users/abaker/miniforge3/envs/sklearn-dev/lib/libomp.dylib
        version: None
```

COMMENT:
I have submitted a pull request related to this topic. My pull request introduces a custom class, `SparseLinearRegressor`, which supports both sparse matrices and non-negative constraints on the coefficients.

The code may potentially address or complement the solution you're working on. If you have time, I’d be grateful if you could review it, as I believe it could offer valuable insights. I have also included a Jupyter notebook (`SparseLinearRegressor.ipynb`) and its Python conversion (`SparseLinearRegressor.py`) to showcase the functionality in different scenarios, such as sparse data handling.

PR Title: Implementation of a Custom Sparse Linear Regression Class with Sparse Support and Non-Negativity Constraints 
PR Number: #30146.

Labels: Bug, Developer API
The input_tags.sparse flag is often incorrect

### Describe the bug

If I understood correctly the developer API for tags, `input_tags.sparse` tells us whether an estimator can accept sparse data or not. For many estimators it seems that `input_tags.sparse` is False but should be True.

### Steps/Code to Reproduce

```python
from sklearn.linear_model import LinearRegression
from sklearn.utils import get_tags

reg = LinearRegression()
tags = get_tags(reg)
tags.input_tags.sparse
```

### Expected Results

`True` as `LinearRegression` accepts sparse input data.

### Actual Results

`False`

### Versions

```shell
System:
    python: 3.12.5 | packaged by conda-forge | (main, Aug  8 2024, 18:32:50) [Clang 16.0.6 ]
executable: /Users/abaker/miniforge3/envs/sklearn-dev/bin/python
   machine: macOS-14.5-arm64-arm-64bit

Python dependencies:
      sklearn: 1.6.dev0
          pip: 24.2
   setuptools: 73.0.1
        numpy: 2.1.0
        scipy: 1.14.1
       Cython: 3.0.11
       pandas: 2.2.2
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /Users/abaker/miniforge3/envs/sklearn-dev/lib/libopenblas.0.dylib
        version: 0.3.27
threading_layer: openmp
   architecture: VORTEX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Users/abaker/miniforge3/envs/sklearn-dev/lib/libomp.dylib
        version: None
```

COMMENT:
@Vishal-sys-code this is not really related to this issue. Let's keep the discussion focussed on the tag system of existing scikit-learn estimators and the developer API used to run common estimator checks on all estimators, rather than discussing of the inclusion of new estimators.

Labels: Bug, Developer API
Webpage typo

### Describe the issue linked to the documentation

In the first part of the [About Page](https://scikit-learn.org/stable/about.html), it says "Later that year, Matthieu Brucher **started work** on this project as part of his thesis."

### Suggest a potential alternative/fix

"Later that year, Matthieu Brucher **started working** on this project as part of his thesis."

COMMENT:
lol

Labels: Documentation
Webpage typo

### Describe the issue linked to the documentation

In the first part of the [About Page](https://scikit-learn.org/stable/about.html), it says "Later that year, Matthieu Brucher **started work** on this project as part of his thesis."

### Suggest a potential alternative/fix

"Later that year, Matthieu Brucher **started working** on this project as part of his thesis."

COMMENT:
Feel free to open a pull-request.

Labels: Documentation
Webpage typo

### Describe the issue linked to the documentation

In the first part of the [About Page](https://scikit-learn.org/stable/about.html), it says "Later that year, Matthieu Brucher **started work** on this project as part of his thesis."

### Suggest a potential alternative/fix

"Later that year, Matthieu Brucher **started working** on this project as part of his thesis."

COMMENT:
Please assign this to me

Labels: Documentation
LinearRegression on sparse matrices is not sample weight consistent

Part of #16298.

### Describe the bug

When using a sparse container like `csr_array` for `X`, `LinearRegression` even fails to give the same coefficients for unit or no sample weight, and more generally fails the `test_linear_regression_sample_weight_consitency` checks. In that setting, the underlying solver is `scipy.sparse.linalg.lsqr`.

### Steps/Code to Reproduce

```python
from sklearn.utils.fixes import csr_array
from sklearn.datasets import make_regression
from sklearn.linear_model import LinearRegression
from sklearn.utils._testing import assert_allclose

X, y = make_regression(100, 100, random_state=42)
X = csr_array(X)
reg = LinearRegression(fit_intercept=True)
reg.fit(X, y)
coef1 = reg.coef_
reg.fit(X, y, sample_weight=np.ones_like(y))
coef2 = reg.coef_
assert_allclose(coef1, coef2, rtol=1e-7, atol=1e-9)
```

### Expected Results

The `assert_allclose` should pass.

### Actual Results

```Python Traceback
AssertionError: 
Not equal to tolerance rtol=1e-07, atol=1e-09

Mismatched elements: 100 / 100 (100%)
Max absolute difference among violations: 0.00165048
Max relative difference among violations: 0.02621317
 ACTUAL: array([-2.450778e-01,  2.917985e+01,  1.678916e+00,  7.534454e+01,
        1.241587e+01,  1.076716e+00, -4.975206e-01, -9.262295e-01,
       -1.373931e+00, -1.624112e-01, -8.644422e-01, -5.986218e-01,...
 DESIRED: array([-2.452359e-01,  2.918078e+01,  1.678681e+00,  7.534410e+01,
        1.241459e+01,  1.076624e+00, -4.962305e-01, -9.257701e-01,
       -1.373862e+00, -1.622824e-01, -8.652183e-01, -5.981715e-01,...
```

The test also fails for `fit_intercept=False`. Note that this test and other sample weight consistency checks pass if we do not wrap `X` in a sparse container.

### Versions

```shell
System:
    python: 3.12.5 | packaged by conda-forge | (main, Aug  8 2024, 18:32:50) [Clang 16.0.6 ]
executable: /Users/abaker/miniforge3/envs/sklearn-dev/bin/python
   machine: macOS-14.5-arm64-arm-64bit

Python dependencies:
      sklearn: 1.6.dev0
          pip: 24.2
   setuptools: 73.0.1
        numpy: 2.1.0
        scipy: 1.14.1
       Cython: 3.0.11
       pandas: 2.2.2
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /Users/abaker/miniforge3/envs/sklearn-dev/lib/libopenblas.0.dylib
        version: 0.3.27
threading_layer: openmp
   architecture: VORTEX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Users/abaker/miniforge3/envs/sklearn-dev/lib/libomp.dylib
        version: None
```

EDIT: discovered while working on #30040 for the case of dense inputs.

COMMENT:
I think we should expose `tol` as in `Ridge`.

Labels: Bug
DOC Motivate preferably using conda-forge's distribution of scikit-learn

A lot of people use scikit-learn's python wheels uploaded on PyPI.

Wheels were not designed for scientific packages and this leads to a variety of problems for users who use them — for more information see [the limitations of PyPi](https://pypackaging-native.github.io/meta-topics/pypi_social_model/).

COMMENT:
I’d like to work on this issue by updating the documentation to recommend using conda-forge for installing scikit-learn, highlighting its advantages over PyPI wheels. Could I take this on ?

Labels: Documentation
DOC Motivate preferably using conda-forge's distribution of scikit-learn

A lot of people use scikit-learn's python wheels uploaded on PyPI.

Wheels were not designed for scientific packages and this leads to a variety of problems for users who use them — for more information see [the limitations of PyPi](https://pypackaging-native.github.io/meta-topics/pypi_social_model/).

COMMENT:
Sure, go for it @Maddouchhamza.

Labels: Documentation
DOC Motivate preferably using conda-forge's distribution of scikit-learn

A lot of people use scikit-learn's python wheels uploaded on PyPI.

Wheels were not designed for scientific packages and this leads to a variety of problems for users who use them — for more information see [the limitations of PyPi](https://pypackaging-native.github.io/meta-topics/pypi_social_model/).

COMMENT:
https://github.com/scikit-learn/scikit-learn/pull/32509 now has `conda` the default when selecting the OS, so I think we can close this one.

Labels: Documentation
DOC Motivate preferably using conda-forge's distribution of scikit-learn

A lot of people use scikit-learn's python wheels uploaded on PyPI.

Wheels were not designed for scientific packages and this leads to a variety of problems for users who use them — for more information see [the limitations of PyPi](https://pypackaging-native.github.io/meta-topics/pypi_social_model/).

COMMENT:
Thank you, @AnneBeyer.

Labels: Documentation
RISC-V

Can scikit-learn be installed and used normally on RISC-V architecture?

COMMENT:
I am not familiar with the topic but this certainly not a platform we test on. Looks like there a not too old discussion about the status of Python packaging on RISC-V [here](https://discuss.python.org/t/packaging-support-for-riscv64/58475) and searching for 5 minutes some people have provided some wheels for some packages including scikit-learn (somewhat old versions though), see https://github.com/KumaTea/riscv-wheels for more details for an example.

I am going to close this one, because I don't think we are planning on working on this in the foreseeable future.

Labels: Needs Triage
RISC-V

Can scikit-learn be installed and used normally on RISC-V architecture?

COMMENT:
In particular, we would need fast enough CI to run the test suite on a powerful enough RISC-V host. We could rely on emulation (via qemu) but I am afraid it might be very slow.

If someone can show that it is possible to run the scikit-learn test suite in less than 30 minutes on a public CI service, then we might reconsider adding official support (by producing official RISC-V wheels, assuming pypi.org accepts them by then, which is not the case at the time of writing).

Labels: Needs Triage
Add differential privacy noise injection to SGDRegressor with automatic calibration

### Describe the workflow you want to enable

Enable differential privacy in SGDRegressor by adding noise injection with:

1. Manual noise scale setting, or
2. Automatic noise calibration from desired privacy parameter ε

### Describe your proposed solution

Add parameters to SGDRegressor:
```python
def __init__(
    self,
    noise_scale=None,  # Manual override
    epsilon=None,      # Desired privacy parameter
    delta=1e-6,        # Secondary privacy parameter
    *args, **kwargs
):
    """
    Parameters
    ----------
    noise_scale : float, optional
        Standard deviation of Gaussian noise. If provided, overrides epsilon.
    epsilon : float, optional
        Privacy parameter. Ignored if noise_scale is provided.
        Requires clip_value to be set.
    delta : float, default=1e-6
        Secondary privacy parameter for (ε,δ)-DP.
    """
    if epsilon is not None:
        if self.clip_value is None:
            raise ValueError("epsilon requires clip_value")
        # Scale noise by sqrt(max_iter)
        self.noise_scale = (
            self.clip_value * 
            np.sqrt(2 * np.log(1.25/delta) * self.max_iter) / 
            epsilon
        )
```


### Research basis

Builds on:
- ["Deep Learning with Differential Privacy"](https://arxiv.org/abs/1607.00133) (Abadi et al 2016, [6000+ citations](https://scholar.google.com/scholar?cites=11431158613977668861&as_sdt=20005&sciodt=0,9&hl=en))
- ["Privacy-preserving logistic regression"](https://systems.cs.columbia.edu/private-systems-class/papers/Chaudhuri2009Privacy.pdf) (Chaudhuri & Monteleoni, 2009, [800+ citations](https://scholar.google.com/scholar?cites=7799634039900750565&as_sdt=20005&sciodt=0,9&hl=en))

Implemented in:
- TensorFlow Privacy
- Opacus (PyTorch)
- JAX Privacy


### Benefits

1. Enables both manual noise configuration and automatic calibration
2. Calculates correct noise scale for desired privacy level
3. Interfaces naturally with the clipping parameter from #30113

Would be happy to provide privacy-utility benchmarks across different epsilon values.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

The noise injection affects standard error calculations by adding variance s² to the error term, where s is noise_scale. The standard error formula becomes:

sqrt(diag((σ² + s²)(X'X)^(-1)))

This adjustment is straightforward to implement in statsmodels by modifying the existing standard error calculations.

COMMENT:
Oh this is a nice one.

In terms of inclusion, I very much like the proposal, it's well established, and easy to implement.

However, we don't really have any other privacy related feature in the rest of the library. So I wonder what kind of future proposals this one sets a precedence for. On the other hand, SGD is quite a major one, and adding this to it would enable many real life usecases.

I wonder what @lorentzenchr and @ogrisel  think maybe.

Labels: New Feature, Needs Decision