Talip7/scikit-learn-issues-multilabel huggingface dataset:
Error in `d2_log_loss_score` multiclass when one of the classes is missing in `y_true`.

### Describe the bug

Hello, I encountered an error with the `d2_log_loss_score` in the multiclass setting (i.e. when `y_pred` has shape (n, k) with k >= 3) when one of the classes is missing from the `y_true` labels, even when giving the labels through the `labels` argument. The error disappear when all the classes are present in `y_true`.

### Steps/Code to Reproduce

```python
from sklearn.metrics import d2_log_loss_score

y_true = [0, 1, 1]
y_pred = [[1, 0, 0], [1, 0, 0], [1, 0, 0]]
labels = [0, 1, 2]

d2_log_loss_score(y_true, y_pred, labels=labels)
```

### Expected Results

No error is thrown.

### Actual Results

```
Traceback (most recent call last):
  File "minimal.py", line 7, in <module>
    d2_log_loss_score(y_true, y_pred, labels=labels)
  File ".../python3.12/site-packages/sklearn/utils/_param_validation.py", line 216, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File ".../python3.12/site-packages/sklearn/metrics/_classification.py", line 3407, in d2_log_loss_score
    denominator = log_loss(
                  ^^^^^^^^^
  File ".../python3.12/site-packages/sklearn/utils/_param_validation.py", line 189, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File ".../python3.12/site-packages/sklearn/metrics/_classification.py", line 3023, in log_loss
    raise ValueError(
ValueError: The number of classes in labels is different from that in y_pred. Classes found in labels: [0 1 2]
```

### Versions

```shell
System:
    python: 3.12.8 | packaged by conda-forge | (main, Dec  5 2024, 14:19:53) [Clang 18.1.8 ]
executable: /Users/alexandreperez/dev/lib/miniforge3/envs/test/bin/python
   machine: macOS-15.2-arm64-arm-64bit

Python dependencies:
      sklearn: 1.6.1
          pip: 24.3.1
   setuptools: 75.8.0
        numpy: 2.2.2
        scipy: 1.15.1
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 14
         prefix: libomp
       filepath: .../miniforge3/envs/test/lib/python3.12/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
```

COMMENT:
Thanks for the report @aperezlebel. Are you interested in opening on a PR yourself?

Labels: Bug, Needs Investigation
Error in `d2_log_loss_score` multiclass when one of the classes is missing in `y_true`.

### Describe the bug

Hello, I encountered an error with the `d2_log_loss_score` in the multiclass setting (i.e. when `y_pred` has shape (n, k) with k >= 3) when one of the classes is missing from the `y_true` labels, even when giving the labels through the `labels` argument. The error disappear when all the classes are present in `y_true`.

### Steps/Code to Reproduce

```python
from sklearn.metrics import d2_log_loss_score

y_true = [0, 1, 1]
y_pred = [[1, 0, 0], [1, 0, 0], [1, 0, 0]]
labels = [0, 1, 2]

d2_log_loss_score(y_true, y_pred, labels=labels)
```

### Expected Results

No error is thrown.

### Actual Results

```
Traceback (most recent call last):
  File "minimal.py", line 7, in <module>
    d2_log_loss_score(y_true, y_pred, labels=labels)
  File ".../python3.12/site-packages/sklearn/utils/_param_validation.py", line 216, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File ".../python3.12/site-packages/sklearn/metrics/_classification.py", line 3407, in d2_log_loss_score
    denominator = log_loss(
                  ^^^^^^^^^
  File ".../python3.12/site-packages/sklearn/utils/_param_validation.py", line 189, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File ".../python3.12/site-packages/sklearn/metrics/_classification.py", line 3023, in log_loss
    raise ValueError(
ValueError: The number of classes in labels is different from that in y_pred. Classes found in labels: [0 1 2]
```

### Versions

```shell
System:
    python: 3.12.8 | packaged by conda-forge | (main, Dec  5 2024, 14:19:53) [Clang 18.1.8 ]
executable: /Users/alexandreperez/dev/lib/miniforge3/envs/test/bin/python
   machine: macOS-15.2-arm64-arm-64bit

Python dependencies:
      sklearn: 1.6.1
          pip: 24.3.1
   setuptools: 75.8.0
        numpy: 2.2.2
        scipy: 1.15.1
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 14
         prefix: libomp
       filepath: .../miniforge3/envs/test/lib/python3.12/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
```

COMMENT:
I would be, but I am afraid I won't have time in the near future.

Labels: Bug, Needs Investigation
Add sample_weight support to QuantileTransformer

### Describe the workflow you want to enable

Would be good to get sample_weight support for QuantileTransformer for dealing with sparse or imbalanced data, a la [#15601](https://github.com/scikit-learn/scikit-learn/issues/15601).  


```
scaler = QuantileTransformer(output_distribution="normal")

scaler.fit(X, sample_weight=w)

```
### Describe your proposed solution

As far as I know it would just require adding the weight argument to the quantiles_ computation in np.nanpercentile.

`KBinsDiscretizer `supports sample_weight and with strategy='quantile', encode='ordinal' this behavior can be achieved but it is much, much slower.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Searching in the issue tracker, looks like this feature is in scope if I understand correctly https://github.com/scikit-learn/scikit-learn/issues/20522#issuecomment-884233092.

cc @snath-xoc and @jeremiedbb for an informed opinion, since they have been working on sample weights support recently.

Labels: New Feature, Moderate
Add sample_weight support to QuantileTransformer

### Describe the workflow you want to enable

Would be good to get sample_weight support for QuantileTransformer for dealing with sparse or imbalanced data, a la [#15601](https://github.com/scikit-learn/scikit-learn/issues/15601).  


```
scaler = QuantileTransformer(output_distribution="normal")

scaler.fit(X, sample_weight=w)

```
### Describe your proposed solution

As far as I know it would just require adding the weight argument to the quantiles_ computation in np.nanpercentile.

`KBinsDiscretizer `supports sample_weight and with strategy='quantile', encode='ordinal' this behavior can be achieved but it is much, much slower.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@antipisa would you be interested in contributing a PR? There is already a common test named `check_sample_weight_equivalence_on_dense_data` that should be triggered as soon as you add the `sample_weight` kwarg to `fit`. Once this is done, the following command should pick it up:

```
pytest -k "check_sample_weight_equivalence and QuantileTransformer" sklearn/tests/test_common.py -v  -s 
```

You might want to adjust some non-default parameter values for that check in `PER_ESTIMATOR_CHECK_PARAMS` in `sklearn/utils/_test_common/instance_generator.py`.

Note that when subsampling is enabled, we need to follow a similar strategy as implemented in `KBinsDiscretizer`. At the time of writing, the code for `KBinsDiscretizer` in `main` is still half broken, but there is a fix that is almost ready to be merged: #29907. You can take a look at this PR for inspiration.

We are working on tooling to help test the case where the estimator's `fit` is stochastic (depends on `random_state` which is the case when subsampling is enabled). You can track the latest progress here: https://github.com/scikit-learn/scikit-learn/issues/16298#issuecomment-2631071604

Labels: New Feature, Moderate
Add sample_weight support to QuantileTransformer

### Describe the workflow you want to enable

Would be good to get sample_weight support for QuantileTransformer for dealing with sparse or imbalanced data, a la [#15601](https://github.com/scikit-learn/scikit-learn/issues/15601).  


```
scaler = QuantileTransformer(output_distribution="normal")

scaler.fit(X, sample_weight=w)

```
### Describe your proposed solution

As far as I know it would just require adding the weight argument to the quantiles_ computation in np.nanpercentile.

`KBinsDiscretizer `supports sample_weight and with strategy='quantile', encode='ordinal' this behavior can be achieved but it is much, much slower.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I will take this!

Labels: New Feature, Moderate
Add sample_weight support to QuantileTransformer

### Describe the workflow you want to enable

Would be good to get sample_weight support for QuantileTransformer for dealing with sparse or imbalanced data, a la [#15601](https://github.com/scikit-learn/scikit-learn/issues/15601).  


```
scaler = QuantileTransformer(output_distribution="normal")

scaler.fit(X, sample_weight=w)

```
### Describe your proposed solution

As far as I know it would just require adding the weight argument to the quantiles_ computation in np.nanpercentile.

`KBinsDiscretizer `supports sample_weight and with strategy='quantile', encode='ordinal' this behavior can be achieved but it is much, much slower.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
a quick update: the feature request is now being actively worked on in PR #32761, which builds on the previous effort in #31147 and incorporates the unified `_weighted_percentile` implementation suggested in the reviews

the progress looks promising, and it should bring full `sample_weight` support to `QuantileTransformer` once finalized.

Labels: New Feature, Moderate
CI Use explicit permissions for GHA workflows

CodeQL scanning is nudging us towards using explicit permission, see https://github.com/scikit-learn/scikit-learn/security/code-scanning?query=is%3Aopen+branch%3Amain+rule%3Aactions%2Fmissing-workflow-permissions

Once this is done we could in principle set the default workflow permissions to read as recommended by GitHub. [Settings](https://github.com/scikit-learn/scikit-learn/settings/actions)

![Image](https://github.com/user-attachments/assets/59b49d2d-d83d-45c7-9468-4216bfe52711)

~This is an excuse to try out sub-issues 😅~ Apparently you can not add PR as sub-issue oh well 😓 ?

COMMENT:
I would also suggest using [`zizmor`](https://woodruffw.github.io/zizmor/) to check all other (GitHub Actions only) workflows. It has [an `excessive-permissions` rule](https://woodruffw.github.io/zizmor/audits/#excessive-permissions) which should help identify all relevant cases. I'm happy to put together a PR if you want me to, @lesteve :)

Labels: Build / CI
CI Use explicit permissions for GHA workflows

CodeQL scanning is nudging us towards using explicit permission, see https://github.com/scikit-learn/scikit-learn/security/code-scanning?query=is%3Aopen+branch%3Amain+rule%3Aactions%2Fmissing-workflow-permissions

Once this is done we could in principle set the default workflow permissions to read as recommended by GitHub. [Settings](https://github.com/scikit-learn/scikit-learn/settings/actions)

![Image](https://github.com/user-attachments/assets/59b49d2d-d83d-45c7-9468-4216bfe52711)

~This is an excuse to try out sub-issues 😅~ Apparently you can not add PR as sub-issue oh well 😓 ?

COMMENT:
Interesting: do you know of repo using `zizmor` to check their GitHub actions? Would it be an additional workflow that runs `zizmor` in PRs? 

Do you know What `zizmor` provides on top of CodeQL? Something that is convenient with CodeQL is that alerts are private for maintainers.

Labels: Build / CI
CI Use explicit permissions for GHA workflows

CodeQL scanning is nudging us towards using explicit permission, see https://github.com/scikit-learn/scikit-learn/security/code-scanning?query=is%3Aopen+branch%3Amain+rule%3Aactions%2Fmissing-workflow-permissions

Once this is done we could in principle set the default workflow permissions to read as recommended by GitHub. [Settings](https://github.com/scikit-learn/scikit-learn/settings/actions)

![Image](https://github.com/user-attachments/assets/59b49d2d-d83d-45c7-9468-4216bfe52711)

~This is an excuse to try out sub-issues 😅~ Apparently you can not add PR as sub-issue oh well 😓 ?

COMMENT:
All the Github workflows have an explicit permissions now and I have switched the default to "Read repository contents and packages permissions". Let's close this one!

Labels: Build / CI
CI Use explicit permissions for GHA workflows

CodeQL scanning is nudging us towards using explicit permission, see https://github.com/scikit-learn/scikit-learn/security/code-scanning?query=is%3Aopen+branch%3Amain+rule%3Aactions%2Fmissing-workflow-permissions

Once this is done we could in principle set the default workflow permissions to read as recommended by GitHub. [Settings](https://github.com/scikit-learn/scikit-learn/settings/actions)

![Image](https://github.com/user-attachments/assets/59b49d2d-d83d-45c7-9468-4216bfe52711)

~This is an excuse to try out sub-issues 😅~ Apparently you can not add PR as sub-issue oh well 😓 ?

COMMENT:
> Interesting: do you know of repo using `zizmor` to check their GitHub actions? Would it be an additional workflow that runs `zizmor` in PRs?

Yes! `scikit-image` has started doing it: https://github.com/scikit-image/scikit-image/pull/7662. It would be an additional pre-commit hook that runs alongside the rest of the style checks (not sure how they are configured in `scikit-learn`, though).

> Do you know What `zizmor` provides on top of CodeQL? Something that is convenient with CodeQL is that alerts are private for maintainers.

As it can be configured as a pre-commit hook, we can ensure that changes that are improperly scoped in terms of security would not be pushed at all, rather than being privately alerted to the maintainers at a later time. One thing I like about it is that it can even check for sensitive external GitHub Actions and perform a few more checks as well, if you provide it a fine-grained PAT (in CI, it can use the `{{ secrets.GITHUB_TOKEN }}` one, I think).

I just tried it out with `uvx zizmor .github/workflows/ --pedantic` and it reports 92 findings: `92 findings (1 suppressed): 0 unknown, 4 informational, 54 low, 20 medium, 13 high`. If I remove the `--pedantic` option (which ignores the fact that actions are not pinned to hashes as we previously discussed), it suppresses 63 findings and still reports three instances with a high audit confidence and four with a medium level of confidence, which I think could be fixed.

Labels: Build / CI
CI Use explicit permissions for GHA workflows

CodeQL scanning is nudging us towards using explicit permission, see https://github.com/scikit-learn/scikit-learn/security/code-scanning?query=is%3Aopen+branch%3Amain+rule%3Aactions%2Fmissing-workflow-permissions

Once this is done we could in principle set the default workflow permissions to read as recommended by GitHub. [Settings](https://github.com/scikit-learn/scikit-learn/settings/actions)

![Image](https://github.com/user-attachments/assets/59b49d2d-d83d-45c7-9468-4216bfe52711)

~This is an excuse to try out sub-issues 😅~ Apparently you can not add PR as sub-issue oh well 😓 ?

COMMENT:
OK interesting, thanks! Not sure how I feel about a pre-commit hook that run zizmor ...

The worry with this kind of tool is false positives. If you notice issues that you feel are critical and unlikely to be false positives, feel free to report them privately by creating a [security advisory](https://github.com/scikit-learn/scikit-learn/security/advisories/new).

Labels: Build / CI
CI Use explicit permissions for GHA workflows

CodeQL scanning is nudging us towards using explicit permission, see https://github.com/scikit-learn/scikit-learn/security/code-scanning?query=is%3Aopen+branch%3Amain+rule%3Aactions%2Fmissing-workflow-permissions

Once this is done we could in principle set the default workflow permissions to read as recommended by GitHub. [Settings](https://github.com/scikit-learn/scikit-learn/settings/actions)

![Image](https://github.com/user-attachments/assets/59b49d2d-d83d-45c7-9468-4216bfe52711)

~This is an excuse to try out sub-issues 😅~ Apparently you can not add PR as sub-issue oh well 😓 ?

COMMENT:
Sure, I'll do that – thanks for the pointer, @lesteve!

Labels: Build / CI
Make scikit-learn OpenML more generic for the data download URL

According to https://github.com/orgs/openml/discussions/20#discussioncomment-11913122 our code hardcodes where to find the OpenML data.

I am not quite sure what needs to be done right now but maybe @PGijsbers has some suggestions (not urgent at all though, I am guessing you have bigger fish to fry right now 😉) or maybe @glemaitre .

COMMENT:
The `url` field in the dataset description should always point to a location where the arff file can be found, and the `parquet_url` field should always point to the parquet file. If that is not the case, that's an error with OpenML. If there are particular issues preventing you from using the links as provided, we'd be happy to help look for a solution.

Labels: Enhancement, module:datasets
Make scikit-learn OpenML more generic for the data download URL

According to https://github.com/orgs/openml/discussions/20#discussioncomment-11913122 our code hardcodes where to find the OpenML data.

I am not quite sure what needs to be done right now but maybe @PGijsbers has some suggestions (not urgent at all though, I am guessing you have bigger fish to fry right now 😉) or maybe @glemaitre .

COMMENT:
The production server is back. `fetch_openml` seems to work again. :tada:
Please let us know if you run into any other issues.

Labels: Enhancement, module:datasets
Make scikit-learn OpenML more generic for the data download URL

According to https://github.com/orgs/openml/discussions/20#discussioncomment-11913122 our code hardcodes where to find the OpenML data.

I am not quite sure what needs to be done right now but maybe @PGijsbers has some suggestions (not urgent at all though, I am guessing you have bigger fish to fry right now 😉) or maybe @glemaitre .

COMMENT:
The `parquet_url` links won't work at the moment. We need to update these links server-side.

Labels: Enhancement, module:datasets
FeatureHasher and HashingVectorizer does not expose requires_fit=False tag

While `FeatureHasher` and `HashingVectorizer` are stateless estimator (at least in their docstrings), they do not expose the `requires_fit` tag to `False` as other stateless estimator.

@adrinjalali Do you recall when changing the tags if there was a particular reason for those estimator to not behave the same way than others?

COMMENT:
My PR didn't change a tag in that regard: 

https://github.com/scikit-learn/scikit-learn/pull/29677/files#diff-2ce614636a32fc7ddace8d5f36e320e0b348012dff967ad17c7fed8ea40b5723R196

Note that `requires_fit` is tricky if the estimator does input / feature name / feature count validation.

Labels: Enhancement, good first issue
FeatureHasher and HashingVectorizer does not expose requires_fit=False tag

While `FeatureHasher` and `HashingVectorizer` are stateless estimator (at least in their docstrings), they do not expose the `requires_fit` tag to `False` as other stateless estimator.

@adrinjalali Do you recall when changing the tags if there was a particular reason for those estimator to not behave the same way than others?

COMMENT:
/take

Labels: Enhancement, good first issue
FeatureHasher and HashingVectorizer does not expose requires_fit=False tag

While `FeatureHasher` and `HashingVectorizer` are stateless estimator (at least in their docstrings), they do not expose the `requires_fit` tag to `False` as other stateless estimator.

@adrinjalali Do you recall when changing the tags if there was a particular reason for those estimator to not behave the same way than others?

COMMENT:
@glemaitre what condition would be require to set `requires_fit` tag to be False or is that the default behaviour?

Labels: Enhancement, good first issue
FeatureHasher and HashingVectorizer does not expose requires_fit=False tag

While `FeatureHasher` and `HashingVectorizer` are stateless estimator (at least in their docstrings), they do not expose the `requires_fit` tag to `False` as other stateless estimator.

@adrinjalali Do you recall when changing the tags if there was a particular reason for those estimator to not behave the same way than others?

COMMENT:
_Hi! I'm interested in working on this issue as my first contribution._

I **understand that `FeatureHasher` and `HashingVectorizer` are stateless estimators and should have the `requires_fit = False` tag set, just like other stateless estimators.**

I'd love to help fix this. Could you please **assign** this issue to me?

Thanks!

Labels: Enhancement, good first issue
FeatureHasher and HashingVectorizer does not expose requires_fit=False tag

While `FeatureHasher` and `HashingVectorizer` are stateless estimator (at least in their docstrings), they do not expose the `requires_fit` tag to `False` as other stateless estimator.

@adrinjalali Do you recall when changing the tags if there was a particular reason for those estimator to not behave the same way than others?

COMMENT:
We don't assign issues to anybody, and we also don't have a `/take` bot anymore. If you think you can handle the issue, please write how you think you can solve it, and then open a pull request.

Labels: Enhancement, good first issue
FeatureHasher and HashingVectorizer does not expose requires_fit=False tag

While `FeatureHasher` and `HashingVectorizer` are stateless estimator (at least in their docstrings), they do not expose the `requires_fit` tag to `False` as other stateless estimator.

@adrinjalali Do you recall when changing the tags if there was a particular reason for those estimator to not behave the same way than others?

COMMENT:
@adrinjalali Thank you for the clarification. I was reading through the online web documentation so I thought it was still in place. I mainly am new to open-source and want to find a way to contribute. 

Do you think it would be worth while if I opened an issue to update the online documentation to reflect that a bot is no longer used (Unless it states this that I didnt see)?

Labels: Enhancement, good first issue
FeatureHasher and HashingVectorizer does not expose requires_fit=False tag

While `FeatureHasher` and `HashingVectorizer` are stateless estimator (at least in their docstrings), they do not expose the `requires_fit` tag to `False` as other stateless estimator.

@adrinjalali Do you recall when changing the tags if there was a particular reason for those estimator to not behave the same way than others?

COMMENT:
Hi, I'd like to work on this issue.

My plan is to:
- Override the _get_tags() method in both FeatureHasher and HashingVectorizer to explicitly return "requires_fit": False.
- This will ensure the tag correctly reflects their stateless behavior, as neither class requires fitting before transforming.
- I'll also add unit tests to verify that this tag is correctly set.

I'll submit a PR shortly. Let me know if you have any suggestions or concerns. Thanks!

Labels: Enhancement, good first issue
FeatureHasher and HashingVectorizer does not expose requires_fit=False tag

While `FeatureHasher` and `HashingVectorizer` are stateless estimator (at least in their docstrings), they do not expose the `requires_fit` tag to `False` as other stateless estimator.

@adrinjalali Do you recall when changing the tags if there was a particular reason for those estimator to not behave the same way than others?

COMMENT:
Hello maintainers, I am new to OSS, but I used to use scikit-learn for my classes projects at OSU. Could you please let me know if I can work on this issue? Thank you and looking forward to hearing from you soon.

Labels: Enhancement, good first issue