Talip7/scikit-learn-issues-multilabel huggingface dataset:
Make `zero_division` parameter consistent in the different metric

This is an issue to report the step to actually take over the work of @marctorsoc in https://github.com/scikit-learn/scikit-learn/pull/23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the `zero_division` parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

- [x] Introduce the `zero_division` parameter to the `accuracy_score` function when `y_true` and `y_pred` are empty.
    - https://github.com/scikit-learn/scikit-learn/pull/29213
- [x] Introduce the `zero_division` parameter to the `class_likelihood_ratios` and remove `raise_warning`.
    - https://github.com/scikit-learn/scikit-learn/pull/31331
- [x] Introduce the `zero_division` parameter to the `cohen_kappa_score` function
  - https://github.com/scikit-learn/scikit-learn/pull/29210
- [x] Introduce the `zero_division` parameter to the `matthew_corr_coeff` function
  - #23183
  - #28509
- [ ] <del>Open a PR to make sure the empty input lead to `np.nan` in `classification_report` function.</del> `classification_report` should raise an error instead: see https://github.com/scikit-learn/scikit-learn/issues/29048#issuecomment-2857931743

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the `zero_division` parameter to the `precision_recall_curve` and `roc_curve` as well. This might add two additional items to the list above.

COMMENT:
Working on `class_likelihood_ratios` now.

Labels: Enhancement
Make `zero_division` parameter consistent in the different metric

This is an issue to report the step to actually take over the work of @marctorsoc in https://github.com/scikit-learn/scikit-learn/pull/23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the `zero_division` parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

- [x] Introduce the `zero_division` parameter to the `accuracy_score` function when `y_true` and `y_pred` are empty.
    - https://github.com/scikit-learn/scikit-learn/pull/29213
- [x] Introduce the `zero_division` parameter to the `class_likelihood_ratios` and remove `raise_warning`.
    - https://github.com/scikit-learn/scikit-learn/pull/31331
- [x] Introduce the `zero_division` parameter to the `cohen_kappa_score` function
  - https://github.com/scikit-learn/scikit-learn/pull/29210
- [x] Introduce the `zero_division` parameter to the `matthew_corr_coeff` function
  - #23183
  - #28509
- [ ] <del>Open a PR to make sure the empty input lead to `np.nan` in `classification_report` function.</del> `classification_report` should raise an error instead: see https://github.com/scikit-learn/scikit-learn/issues/29048#issuecomment-2857931743

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the `zero_division` parameter to the `precision_recall_curve` and `roc_curve` as well. This might add two additional items to the list above.

COMMENT:
Working on `classification_report`.

Labels: Enhancement
Make `zero_division` parameter consistent in the different metric

This is an issue to report the step to actually take over the work of @marctorsoc in https://github.com/scikit-learn/scikit-learn/pull/23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the `zero_division` parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

- [x] Introduce the `zero_division` parameter to the `accuracy_score` function when `y_true` and `y_pred` are empty.
    - https://github.com/scikit-learn/scikit-learn/pull/29213
- [x] Introduce the `zero_division` parameter to the `class_likelihood_ratios` and remove `raise_warning`.
    - https://github.com/scikit-learn/scikit-learn/pull/31331
- [x] Introduce the `zero_division` parameter to the `cohen_kappa_score` function
  - https://github.com/scikit-learn/scikit-learn/pull/29210
- [x] Introduce the `zero_division` parameter to the `matthew_corr_coeff` function
  - #23183
  - #28509
- [ ] <del>Open a PR to make sure the empty input lead to `np.nan` in `classification_report` function.</del> `classification_report` should raise an error instead: see https://github.com/scikit-learn/scikit-learn/issues/29048#issuecomment-2857931743

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the `zero_division` parameter to the `precision_recall_curve` and `roc_curve` as well. This might add two additional items to the list above.

COMMENT:
Isn't `Open a PR to make sure the empty input lead to np.nan in classification_report function.` already solved by #25531

Labels: Enhancement
Make `zero_division` parameter consistent in the different metric

This is an issue to report the step to actually take over the work of @marctorsoc in https://github.com/scikit-learn/scikit-learn/pull/23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the `zero_division` parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

- [x] Introduce the `zero_division` parameter to the `accuracy_score` function when `y_true` and `y_pred` are empty.
    - https://github.com/scikit-learn/scikit-learn/pull/29213
- [x] Introduce the `zero_division` parameter to the `class_likelihood_ratios` and remove `raise_warning`.
    - https://github.com/scikit-learn/scikit-learn/pull/31331
- [x] Introduce the `zero_division` parameter to the `cohen_kappa_score` function
  - https://github.com/scikit-learn/scikit-learn/pull/29210
- [x] Introduce the `zero_division` parameter to the `matthew_corr_coeff` function
  - #23183
  - #28509
- [ ] <del>Open a PR to make sure the empty input lead to `np.nan` in `classification_report` function.</del> `classification_report` should raise an error instead: see https://github.com/scikit-learn/scikit-learn/issues/29048#issuecomment-2857931743

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the `zero_division` parameter to the `precision_recall_curve` and `roc_curve` as well. This might add two additional items to the list above.

COMMENT:
@rritik772 It is only precision recall f1-score

Labels: Enhancement
Make `zero_division` parameter consistent in the different metric

This is an issue to report the step to actually take over the work of @marctorsoc in https://github.com/scikit-learn/scikit-learn/pull/23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the `zero_division` parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

- [x] Introduce the `zero_division` parameter to the `accuracy_score` function when `y_true` and `y_pred` are empty.
    - https://github.com/scikit-learn/scikit-learn/pull/29213
- [x] Introduce the `zero_division` parameter to the `class_likelihood_ratios` and remove `raise_warning`.
    - https://github.com/scikit-learn/scikit-learn/pull/31331
- [x] Introduce the `zero_division` parameter to the `cohen_kappa_score` function
  - https://github.com/scikit-learn/scikit-learn/pull/29210
- [x] Introduce the `zero_division` parameter to the `matthew_corr_coeff` function
  - #23183
  - #28509
- [ ] <del>Open a PR to make sure the empty input lead to `np.nan` in `classification_report` function.</del> `classification_report` should raise an error instead: see https://github.com/scikit-learn/scikit-learn/issues/29048#issuecomment-2857931743

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the `zero_division` parameter to the `precision_recall_curve` and `roc_curve` as well. This might add two additional items to the list above.

COMMENT:
If we start to introduce `replace_undefined_by`, it means that we should revert the `zero_division` introduced in 1.6 to not have additional deprecation.

Labels: Enhancement
Make `zero_division` parameter consistent in the different metric

This is an issue to report the step to actually take over the work of @marctorsoc in https://github.com/scikit-learn/scikit-learn/pull/23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the `zero_division` parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

- [x] Introduce the `zero_division` parameter to the `accuracy_score` function when `y_true` and `y_pred` are empty.
    - https://github.com/scikit-learn/scikit-learn/pull/29213
- [x] Introduce the `zero_division` parameter to the `class_likelihood_ratios` and remove `raise_warning`.
    - https://github.com/scikit-learn/scikit-learn/pull/31331
- [x] Introduce the `zero_division` parameter to the `cohen_kappa_score` function
  - https://github.com/scikit-learn/scikit-learn/pull/29210
- [x] Introduce the `zero_division` parameter to the `matthew_corr_coeff` function
  - #23183
  - #28509
- [ ] <del>Open a PR to make sure the empty input lead to `np.nan` in `classification_report` function.</del> `classification_report` should raise an error instead: see https://github.com/scikit-learn/scikit-learn/issues/29048#issuecomment-2857931743

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the `zero_division` parameter to the `precision_recall_curve` and `roc_curve` as well. This might add two additional items to the list above.

COMMENT:
When it comes to the release sensitive topics, we need to move fast. Otherwise we're not that fast at all. But we've wanted to push an RC for the release 2 weeks ago. And I also told you in a call about this anyway. It's not that you were not aware.

Labels: Enhancement
Make `zero_division` parameter consistent in the different metric

This is an issue to report the step to actually take over the work of @marctorsoc in https://github.com/scikit-learn/scikit-learn/pull/23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the `zero_division` parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

- [x] Introduce the `zero_division` parameter to the `accuracy_score` function when `y_true` and `y_pred` are empty.
    - https://github.com/scikit-learn/scikit-learn/pull/29213
- [x] Introduce the `zero_division` parameter to the `class_likelihood_ratios` and remove `raise_warning`.
    - https://github.com/scikit-learn/scikit-learn/pull/31331
- [x] Introduce the `zero_division` parameter to the `cohen_kappa_score` function
  - https://github.com/scikit-learn/scikit-learn/pull/29210
- [x] Introduce the `zero_division` parameter to the `matthew_corr_coeff` function
  - #23183
  - #28509
- [ ] <del>Open a PR to make sure the empty input lead to `np.nan` in `classification_report` function.</del> `classification_report` should raise an error instead: see https://github.com/scikit-learn/scikit-learn/issues/29048#issuecomment-2857931743

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the `zero_division` parameter to the `precision_recall_curve` and `roc_curve` as well. This might add two additional items to the list above.

COMMENT:
> But still, could you please move slower. This thing seems to have been discussed in a private call and implementation started in a matter of an afternoon. This issue had been presented to me as something I can do as a beginner and your speed in changing the direction here is overwhelming.

Here, the rush is only to avoid releasing the new feature. The private call was to figure out if we should indeed pause the implementation because we are shooting in our own foot because we overlooked some details. We also thought about the alternative implementation but then there is no rush to complete it for the release. And here, we should take time to answer any question and it might give rise to a better alternative.

Labels: Enhancement
Make `zero_division` parameter consistent in the different metric

This is an issue to report the step to actually take over the work of @marctorsoc in https://github.com/scikit-learn/scikit-learn/pull/23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the `zero_division` parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

- [x] Introduce the `zero_division` parameter to the `accuracy_score` function when `y_true` and `y_pred` are empty.
    - https://github.com/scikit-learn/scikit-learn/pull/29213
- [x] Introduce the `zero_division` parameter to the `class_likelihood_ratios` and remove `raise_warning`.
    - https://github.com/scikit-learn/scikit-learn/pull/31331
- [x] Introduce the `zero_division` parameter to the `cohen_kappa_score` function
  - https://github.com/scikit-learn/scikit-learn/pull/29210
- [x] Introduce the `zero_division` parameter to the `matthew_corr_coeff` function
  - #23183
  - #28509
- [ ] <del>Open a PR to make sure the empty input lead to `np.nan` in `classification_report` function.</del> `classification_report` should raise an error instead: see https://github.com/scikit-learn/scikit-learn/issues/29048#issuecomment-2857931743

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the `zero_division` parameter to the `precision_recall_curve` and `roc_curve` as well. This might add two additional items to the list above.

COMMENT:
After looking into this again, I would like to try to clarify a few specific things:

I am wondering what the default argument for `replace_undefined_by` would be? I would think:

-  `np.nan` for metrics that do not aggregate
- `"ignore"` for metrics that aggregate

Would others agree here?

Other values for `replace_undefined_by`  then are scores that are in the range of the metric (maybe not only 0 and 1 or floats; for instance `class_likelihood_ratios` can range to `np.inf`)

Apart from this we always raise a warning whenever there is a division by zero (indicating in the warning message which `replace_undefined_by` is set) and we document in the docstring, under which conditions an undefined value would occur.

We can also increase the `stacklevel` of the warning (currently 2) to higher, if we suspect that the warning might get lost otherwise?

Labels: Enhancement
Make `zero_division` parameter consistent in the different metric

This is an issue to report the step to actually take over the work of @marctorsoc in https://github.com/scikit-learn/scikit-learn/pull/23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the `zero_division` parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

- [x] Introduce the `zero_division` parameter to the `accuracy_score` function when `y_true` and `y_pred` are empty.
    - https://github.com/scikit-learn/scikit-learn/pull/29213
- [x] Introduce the `zero_division` parameter to the `class_likelihood_ratios` and remove `raise_warning`.
    - https://github.com/scikit-learn/scikit-learn/pull/31331
- [x] Introduce the `zero_division` parameter to the `cohen_kappa_score` function
  - https://github.com/scikit-learn/scikit-learn/pull/29210
- [x] Introduce the `zero_division` parameter to the `matthew_corr_coeff` function
  - #23183
  - #28509
- [ ] <del>Open a PR to make sure the empty input lead to `np.nan` in `classification_report` function.</del> `classification_report` should raise an error instead: see https://github.com/scikit-learn/scikit-learn/issues/29048#issuecomment-2857931743

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the `zero_division` parameter to the `precision_recall_curve` and `roc_curve` as well. This might add two additional items to the list above.

COMMENT:
That all sounds good to me @StefanieSenger . We just need to make sure, if needed, we go through deprecation for the change in behavior. That happens in cases where replace nan with 0 for instance, and in the future we'd just return nan.

Labels: Enhancement
Make `zero_division` parameter consistent in the different metric

This is an issue to report the step to actually take over the work of @marctorsoc in https://github.com/scikit-learn/scikit-learn/pull/23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the `zero_division` parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

- [x] Introduce the `zero_division` parameter to the `accuracy_score` function when `y_true` and `y_pred` are empty.
    - https://github.com/scikit-learn/scikit-learn/pull/29213
- [x] Introduce the `zero_division` parameter to the `class_likelihood_ratios` and remove `raise_warning`.
    - https://github.com/scikit-learn/scikit-learn/pull/31331
- [x] Introduce the `zero_division` parameter to the `cohen_kappa_score` function
  - https://github.com/scikit-learn/scikit-learn/pull/29210
- [x] Introduce the `zero_division` parameter to the `matthew_corr_coeff` function
  - #23183
  - #28509
- [ ] <del>Open a PR to make sure the empty input lead to `np.nan` in `classification_report` function.</del> `classification_report` should raise an error instead: see https://github.com/scikit-learn/scikit-learn/issues/29048#issuecomment-2857931743

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the `zero_division` parameter to the `precision_recall_curve` and `roc_curve` as well. This might add two additional items to the list above.

COMMENT:
> We just need to make sure, if needed, we go through deprecation for the change in behavior. That happens in cases where replace nan with 0 for instance, and in the future we'd just return nan.

That's confusing. I was thinking for metrics that already offer `zero_division`, we deprecate the whole param and introduce `replace_undefined_by` unrelated to whatever happened in `zero_division`. If so, then there is no need to deal with specific behaviours?

The message in the FutureWarning would indicate to `replace_undefined_by` and maybe it can also mention the new default.

Labels: Enhancement
Make `zero_division` parameter consistent in the different metric

This is an issue to report the step to actually take over the work of @marctorsoc in https://github.com/scikit-learn/scikit-learn/pull/23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the `zero_division` parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

- [x] Introduce the `zero_division` parameter to the `accuracy_score` function when `y_true` and `y_pred` are empty.
    - https://github.com/scikit-learn/scikit-learn/pull/29213
- [x] Introduce the `zero_division` parameter to the `class_likelihood_ratios` and remove `raise_warning`.
    - https://github.com/scikit-learn/scikit-learn/pull/31331
- [x] Introduce the `zero_division` parameter to the `cohen_kappa_score` function
  - https://github.com/scikit-learn/scikit-learn/pull/29210
- [x] Introduce the `zero_division` parameter to the `matthew_corr_coeff` function
  - #23183
  - #28509
- [ ] <del>Open a PR to make sure the empty input lead to `np.nan` in `classification_report` function.</del> `classification_report` should raise an error instead: see https://github.com/scikit-learn/scikit-learn/issues/29048#issuecomment-2857931743

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the `zero_division` parameter to the `precision_recall_curve` and `roc_curve` as well. This might add two additional items to the list above.

COMMENT:
Imagine this situation:

```py
jaccard_score(y_true, y_pred)
```

The user doesn't specify any values for either `zero_division` or `replace_undefined_by`. What's the behavior here? In most cases for metrics, we have been replacing zero division by zero by default.

Now after the change and after the deprecation, the same code (still not specifying any other arguments) will return `nan` if there's zero division. This is a change in behavior that we need to take care of.

So the deprecation of `zero_division` is different and separate from deprecating the old default behavior.

Labels: Enhancement
Make `zero_division` parameter consistent in the different metric

This is an issue to report the step to actually take over the work of @marctorsoc in https://github.com/scikit-learn/scikit-learn/pull/23183 and split the PR into smaller one to facilitate the review process.

The intend is to make the `zero_division` parameter consistent across different metrics in scikit-learn. In this regards, we have the following TODO list:

- [x] Introduce the `zero_division` parameter to the `accuracy_score` function when `y_true` and `y_pred` are empty.
    - https://github.com/scikit-learn/scikit-learn/pull/29213
- [x] Introduce the `zero_division` parameter to the `class_likelihood_ratios` and remove `raise_warning`.
    - https://github.com/scikit-learn/scikit-learn/pull/31331
- [x] Introduce the `zero_division` parameter to the `cohen_kappa_score` function
  - https://github.com/scikit-learn/scikit-learn/pull/29210
- [x] Introduce the `zero_division` parameter to the `matthew_corr_coeff` function
  - #23183
  - #28509
- [ ] <del>Open a PR to make sure the empty input lead to `np.nan` in `classification_report` function.</del> `classification_report` should raise an error instead: see https://github.com/scikit-learn/scikit-learn/issues/29048#issuecomment-2857931743

All those items have been addressed in #23183 and can be extracted in individual PRs. The changelog presenting the changes should acknowledge @marctorsoc.

In addition, we should investigate #27047 and check if we should add the `zero_division` parameter to the `precision_recall_curve` and `roc_curve` as well. This might add two additional items to the list above.

COMMENT:
I see. So this means `replace_undefined_by` cannot take over in these metrics that already have `zero_division` until `zero_division` is fully removed and users have seen its `FutureWarning` for 2 versions. But it can already be there, I think.

Labels: Enhancement