Talip7/scikit-learn-issues-multilabel huggingface dataset:
DecisionTreeClassifier having unexpected behaviour with 'min_weight_fraction_leaf=0.5'

### Describe the bug

When fitting DecisionTreeClassifier on a duplicated sample set (i.e. each sample repeated by two), the result is not the same as when fitting on the original sample set. This only happens for 'min_weight_fraction_leaf' specified as <0.5. This also effects ExtraTreesClassifier and ExtraTreeClassifier.

### Steps/Code to Reproduce

```python
from sklearn.tree import DecisionTreeClassifier
from scipy.stats import kstest
import numpy as np

rng = np.random.RandomState(0)
    
n_samples = 20
X = rng.rand(n_samples, n_samples * 2)
y = rng.randint(0, 3, size=n_samples)

X_repeated = np.repeat(X,2,axis=0)
y_repeated = np.repeat(y,2)

predictions = []
predictions_dup = []

## Fit estimator
for seed in range(100):
    est = DecisionTreeClassifier(random_state=seed, max_features=0.5, min_weight_fraction_leaf=0.5).fit(X,y)
    est_dup = DecisionTreeClassifier(random_state=seed, max_features=0.5, min_weight_fraction_leaf=0.5).fit(X_repeated,y_repeated)

    ##Get predictions
    predictions.append(est.predict_proba(X)[:,:-1])
    predictions_dup.append(est_dup.predict_proba(X)[:,:-1])

predictions = np.vstack(predictions)
predictions_dup = np.vstack(predictions_dup)

for pred, pred_dup in (predictions.T,predictions_dup.T):
    print(kstest(pred,pred_dup).pvalue)

```

### Expected Results

p-values are more than ˜0.05

### Actual Results

```
p-values = 2.0064970441275627e-69
```

### Versions

```shell
System:
    python: 3.12.4 | packaged by conda-forge | (main, Jun 17 2024, 10:13:44) [Clang 16.0.6 ]
executable: /Users/shrutinath/micromamba/envs/scikit-learn/bin/python
   machine: macOS-14.3-arm64-arm-64bit

Python dependencies:
      sklearn: 1.7.dev0
          pip: 24.0
   setuptools: 75.8.0
        numpy: 2.0.0
        scipy: 1.14.0
       Cython: 3.0.10
       pandas: 2.2.2
   matplotlib: 3.9.0
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
...
    num_threads: 8
         prefix: libomp
       filepath: /Users/shrutinath/micromamba/envs/scikit-learn/lib/libomp.dylib
        version: None
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...
```

COMMENT:
Hello! is this issue still open?

Labels: Bug
DecisionTreeClassifier having unexpected behaviour with 'min_weight_fraction_leaf=0.5'

### Describe the bug

When fitting DecisionTreeClassifier on a duplicated sample set (i.e. each sample repeated by two), the result is not the same as when fitting on the original sample set. This only happens for 'min_weight_fraction_leaf' specified as <0.5. This also effects ExtraTreesClassifier and ExtraTreeClassifier.

### Steps/Code to Reproduce

```python
from sklearn.tree import DecisionTreeClassifier
from scipy.stats import kstest
import numpy as np

rng = np.random.RandomState(0)
    
n_samples = 20
X = rng.rand(n_samples, n_samples * 2)
y = rng.randint(0, 3, size=n_samples)

X_repeated = np.repeat(X,2,axis=0)
y_repeated = np.repeat(y,2)

predictions = []
predictions_dup = []

## Fit estimator
for seed in range(100):
    est = DecisionTreeClassifier(random_state=seed, max_features=0.5, min_weight_fraction_leaf=0.5).fit(X,y)
    est_dup = DecisionTreeClassifier(random_state=seed, max_features=0.5, min_weight_fraction_leaf=0.5).fit(X_repeated,y_repeated)

    ##Get predictions
    predictions.append(est.predict_proba(X)[:,:-1])
    predictions_dup.append(est_dup.predict_proba(X)[:,:-1])

predictions = np.vstack(predictions)
predictions_dup = np.vstack(predictions_dup)

for pred, pred_dup in (predictions.T,predictions_dup.T):
    print(kstest(pred,pred_dup).pvalue)

```

### Expected Results

p-values are more than ˜0.05

### Actual Results

```
p-values = 2.0064970441275627e-69
```

### Versions

```shell
System:
    python: 3.12.4 | packaged by conda-forge | (main, Jun 17 2024, 10:13:44) [Clang 16.0.6 ]
executable: /Users/shrutinath/micromamba/envs/scikit-learn/bin/python
   machine: macOS-14.3-arm64-arm-64bit

Python dependencies:
      sklearn: 1.7.dev0
          pip: 24.0
   setuptools: 75.8.0
        numpy: 2.0.0
        scipy: 1.14.0
       Cython: 3.0.10
       pandas: 2.2.2
   matplotlib: 3.9.0
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
...
    num_threads: 8
         prefix: libomp
       filepath: /Users/shrutinath/micromamba/envs/scikit-learn/lib/libomp.dylib
        version: None
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...
```

COMMENT:
I think we can close this as https://github.com/scikit-learn/scikit-learn/issues/30917#issuecomment-2761139333 pointed out the mistake in the reproducer. After looking at the reproducer and the proposed change/fix I agree with @pedroL0pes. Thanks for putting in the effort to find this!

Labels: Bug
Typo in _k_means_lloyd.pyx

### Describe the issue linked to the documentation

I noticed that in the lloyd_iter_chunked_sparse function of _k_means_lloyd.pyx, there is a potential typo in the comment for handling an empty array. It reads (starting on line 280):
 "An empty array was passed, do nothing and return early (before
 attempting to compute n_chunks). This can typically happen when
 calling the prediction function of a bisecting k-means model with a
 large fraction of outiers."


### Suggest a potential alternative/fix

I'd like to propose editing the file to read "large fraction of outliers" instead of "large fraction of outiers". Let me know what you think!

COMMENT:
Hi, thanks @carlIkai for creating this issue. I have submitted a PR to fix this issue: [#30915](https://github.com/scikit-learn/scikit-learn/pull/30915). It corrects the typo "outiers" → "outliers" in the relevant K-Means functions. Let me know if any further changes are needed!

Labels: Documentation, Needs Triage
Wrong result in log_loss when labels and corresponding y_pred columns are not ordered

### Describe the bug

Log loss is not computed correctly when labels (and their corresponding columns in `y_pred`) are not in ascending (for numbers) / lexicographic (for strings) order. 

### Steps/Code to Reproduce

```
from sklearn.metrics import log_loss

y_true = ["dog", "cat"]

# labels are lexicographically ordered
labels = ["cat", "dog"]

y_pred = [
    [0, 1],  # -> predicts dog
    [1, 0],  # -> predicts cat
]

# loss is zero
print(log_loss(y_true, y_pred, labels=labels))

# labels are not ordered
labels = ["dog", "cat"]

y_pred = [
    [1, 0], # -> still predicts dog
    [0, 1], # -> still predicts cat
]

# loss should be zero again
print(log_loss(y_true, y_pred, labels=labels))
```

Labels beings strings is not the issue, swiping "dog" with 1 and "cat" with 0 reproduces the bug.

### Expected Results

Both log losses should be zero since in both cases `y_pred` predicts `y_true` with 100% probability

### Actual Results

```
>>> 2.2204460492503136e-16
>>> 36.04365338911715
```

First loss with ordered labels is zero. Second loss is 36.043

### Versions

```shell
System:
    python: 3.9.21 (main, Dec  4 2024, 08:53:33)  [GCC 11.4.0]
   machine: Linux-6.8.0-52-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.3.2
          pip: 24.3.1
   setuptools: 75.2.0
        numpy: 1.22.2
        scipy: 1.8.1
       Cython: 3.0.11
       pandas: 1.3.5
   matplotlib: 3.8.4
       joblib: 1.4.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libopenblas
        version: 0.3.18
threading_layer: pthreads
   architecture: Prescott

       user_api: openmp
   internal_api: openmp
    num_threads: 12
         prefix: libgomp
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libopenblas
        version: 0.3.17
threading_layer: pthreads
   architecture: Prescott
```

COMMENT:
Upon @OmarManzoor  clarification, docstring explicitly states that `y_pred` labels should be in alphabetical order, no matter what the order of `labels` is. Closing issue

Labels: Bug, Needs Triage
Improve `pos_label` switching for metrics

Supercedes #26758 

Switching `pos_label` for metrics, involves some manipulation for `predict_proba` (switch column you pass) and `decision_function` (for binary, multiply by -1) as you must pass the values for the positive class.

In discussions in #26758  we thought of two options:

* Add an example demonstrating what you need to do when switching `pos_label`
* Expose the (currently private) functions [`_process_decision_function`](https://github.com/scikit-learn/scikit-learn/blob/5eb676ac9afd4a5d90cdda198d174c2c8d2da226/sklearn/utils/_response.py#L76) and [`_process_predict_proba`](https://github.com/scikit-learn/scikit-learn/blob/5eb676ac9afd4a5d90cdda198d174c2c8d2da226/sklearn/utils/_response.py#L16)

This is a RFC to discuss if we prefer one, or both options.

cc @glemaitre and maybe @ogrisel ?

COMMENT:
I think that I would prefer an example that we could link for each pos_label docstring. 

I'm thinking that the functions will not be used enough. 

The example would be nice because we have space to be didactic and we have already so right place where it should be linked.

Labels: RFC
Improve `pos_label` switching for metrics

Supercedes #26758 

Switching `pos_label` for metrics, involves some manipulation for `predict_proba` (switch column you pass) and `decision_function` (for binary, multiply by -1) as you must pass the values for the positive class.

In discussions in #26758  we thought of two options:

* Add an example demonstrating what you need to do when switching `pos_label`
* Expose the (currently private) functions [`_process_decision_function`](https://github.com/scikit-learn/scikit-learn/blob/5eb676ac9afd4a5d90cdda198d174c2c8d2da226/sklearn/utils/_response.py#L76) and [`_process_predict_proba`](https://github.com/scikit-learn/scikit-learn/blob/5eb676ac9afd4a5d90cdda198d174c2c8d2da226/sklearn/utils/_response.py#L16)

This is a RFC to discuss if we prefer one, or both options.

cc @glemaitre and maybe @ogrisel ?

COMMENT:
Maybe @jeremiedbb has an opinion as well

Labels: RFC
Improve `pos_label` switching for metrics

Supercedes #26758 

Switching `pos_label` for metrics, involves some manipulation for `predict_proba` (switch column you pass) and `decision_function` (for binary, multiply by -1) as you must pass the values for the positive class.

In discussions in #26758  we thought of two options:

* Add an example demonstrating what you need to do when switching `pos_label`
* Expose the (currently private) functions [`_process_decision_function`](https://github.com/scikit-learn/scikit-learn/blob/5eb676ac9afd4a5d90cdda198d174c2c8d2da226/sklearn/utils/_response.py#L76) and [`_process_predict_proba`](https://github.com/scikit-learn/scikit-learn/blob/5eb676ac9afd4a5d90cdda198d174c2c8d2da226/sklearn/utils/_response.py#L16)

This is a RFC to discuss if we prefer one, or both options.

cc @glemaitre and maybe @ogrisel ?

COMMENT:
I also tend towards better documentation. But something that I don't understand is why these metrics (e.g. roc_curve) don't accept the raw result of `predict_proba` or `decision_function` and process it internally against `pos_label`. Wouldn't it be more intuitive from a user point of view ?

Labels: RFC
Improve `pos_label` switching for metrics

Supercedes #26758 

Switching `pos_label` for metrics, involves some manipulation for `predict_proba` (switch column you pass) and `decision_function` (for binary, multiply by -1) as you must pass the values for the positive class.

In discussions in #26758  we thought of two options:

* Add an example demonstrating what you need to do when switching `pos_label`
* Expose the (currently private) functions [`_process_decision_function`](https://github.com/scikit-learn/scikit-learn/blob/5eb676ac9afd4a5d90cdda198d174c2c8d2da226/sklearn/utils/_response.py#L76) and [`_process_predict_proba`](https://github.com/scikit-learn/scikit-learn/blob/5eb676ac9afd4a5d90cdda198d174c2c8d2da226/sklearn/utils/_response.py#L16)

This is a RFC to discuss if we prefer one, or both options.

cc @glemaitre and maybe @ogrisel ?

COMMENT:
For `decision_function` in the binary case (where output is 1D), how would you work out which class the values are for (I may be missing something)?

In the `predict_proba` case, the only reason I can think of is if the user is using it with values not created from a sklearn estimator and it's not 2d?

Labels: RFC
Improve `pos_label` switching for metrics

Supercedes #26758 

Switching `pos_label` for metrics, involves some manipulation for `predict_proba` (switch column you pass) and `decision_function` (for binary, multiply by -1) as you must pass the values for the positive class.

In discussions in #26758  we thought of two options:

* Add an example demonstrating what you need to do when switching `pos_label`
* Expose the (currently private) functions [`_process_decision_function`](https://github.com/scikit-learn/scikit-learn/blob/5eb676ac9afd4a5d90cdda198d174c2c8d2da226/sklearn/utils/_response.py#L76) and [`_process_predict_proba`](https://github.com/scikit-learn/scikit-learn/blob/5eb676ac9afd4a5d90cdda198d174c2c8d2da226/sklearn/utils/_response.py#L16)

This is a RFC to discuss if we prefer one, or both options.

cc @glemaitre and maybe @ogrisel ?

COMMENT:
> For decision_function in the binary case (where output is 1D), how would you work out which class the values are for (I may be missing something)?

Fair point. Estimators do that in a kind of implicit way. positive values correspond to `classes_[1]` while negative values correspond to `classes_[0]`. But here we don't have access to `classes_`. And we can't use `y_true` for that because it's not guaranteed to give the same order of classes (in weird settings but still).

Labels: RFC
DOC Update wikipedia article for scikit-learn

### Describe the issue linked to the documentation

The [wikipedia article on scikit-learn](https://en.wikipedia.org/wiki/Scikit-learn) covers its basic history, development, and features, but there are a few areas where additional details could enhance the content

Notice that this is not an issue for our internal documentation (and therefore cannot be fixed by a PR), but it is documentation nevertheless.

### Suggest a potential alternative/fix

We can potentially get inspired by other projects wikipedia articles, such as [XGBoost](https://en.wikipedia.org/wiki/XGBoost), but some axis that can be added to the article are:

- [x] The [wiki article for TensorFlow](https://en.wikipedia.org/wiki/TensorFlow) has an `Applications` section. We can do the same using our [Testimonials](https://scikit-learn.org/stable/testimonials/testimonials.html) as inspiration.

- [ ] Update the `Version history` section and add more recent developments and features as per the [release highlights](https://scikit-learn.org/stable/auto_examples/release_highlights/index.html).

- [x] Add an `Awards` section to mention e.g. [this Open Source Software Award](https://blog.scikit-learn.org/press/frenchaward/) or [this price for innovation](https://www.inria.fr/en/2019-inria-french-academy-sciences-dassault-systemes-innovation-prize-scikit-learn-success-story).

- [ ] Mention additional metrics, e.g. the [2021](https://www.kaggle.com/kaggle-survey-2021) and [2022](https://www.kaggle.com/kaggle-survey-2022) kaggle surveys question regarding which machine learning frameworks are used by data scientists on a regular basis?

The priority should be to update the English version, but not limited to it. In any case, this issue will be considered as solved once all the above points are addressed for said language. Because of that, if you have contributed to the article, please keep the community posted by commenting on this issue.

COMMENT:
One thing to keep in mind is that WIkipedia tends to frown upon people editing their own page. I am not up to date with the current rules/norms but we should keep that in mind.

Labels: Documentation
DOC Update wikipedia article for scikit-learn

### Describe the issue linked to the documentation

The [wikipedia article on scikit-learn](https://en.wikipedia.org/wiki/Scikit-learn) covers its basic history, development, and features, but there are a few areas where additional details could enhance the content

Notice that this is not an issue for our internal documentation (and therefore cannot be fixed by a PR), but it is documentation nevertheless.

### Suggest a potential alternative/fix

We can potentially get inspired by other projects wikipedia articles, such as [XGBoost](https://en.wikipedia.org/wiki/XGBoost), but some axis that can be added to the article are:

- [x] The [wiki article for TensorFlow](https://en.wikipedia.org/wiki/TensorFlow) has an `Applications` section. We can do the same using our [Testimonials](https://scikit-learn.org/stable/testimonials/testimonials.html) as inspiration.

- [ ] Update the `Version history` section and add more recent developments and features as per the [release highlights](https://scikit-learn.org/stable/auto_examples/release_highlights/index.html).

- [x] Add an `Awards` section to mention e.g. [this Open Source Software Award](https://blog.scikit-learn.org/press/frenchaward/) or [this price for innovation](https://www.inria.fr/en/2019-inria-french-academy-sciences-dassault-systemes-innovation-prize-scikit-learn-success-story).

- [ ] Mention additional metrics, e.g. the [2021](https://www.kaggle.com/kaggle-survey-2021) and [2022](https://www.kaggle.com/kaggle-survey-2022) kaggle surveys question regarding which machine learning frameworks are used by data scientists on a regular basis?

The priority should be to update the English version, but not limited to it. In any case, this issue will be considered as solved once all the above points are addressed for said language. Because of that, if you have contributed to the article, please keep the community posted by commenting on this issue.

COMMENT:
Since scikit-learn is a community-driven project, I am not sure what would the concept of the article being "ours" mean in this context.

Labels: Documentation
DOC Update wikipedia article for scikit-learn

### Describe the issue linked to the documentation

The [wikipedia article on scikit-learn](https://en.wikipedia.org/wiki/Scikit-learn) covers its basic history, development, and features, but there are a few areas where additional details could enhance the content

Notice that this is not an issue for our internal documentation (and therefore cannot be fixed by a PR), but it is documentation nevertheless.

### Suggest a potential alternative/fix

We can potentially get inspired by other projects wikipedia articles, such as [XGBoost](https://en.wikipedia.org/wiki/XGBoost), but some axis that can be added to the article are:

- [x] The [wiki article for TensorFlow](https://en.wikipedia.org/wiki/TensorFlow) has an `Applications` section. We can do the same using our [Testimonials](https://scikit-learn.org/stable/testimonials/testimonials.html) as inspiration.

- [ ] Update the `Version history` section and add more recent developments and features as per the [release highlights](https://scikit-learn.org/stable/auto_examples/release_highlights/index.html).

- [x] Add an `Awards` section to mention e.g. [this Open Source Software Award](https://blog.scikit-learn.org/press/frenchaward/) or [this price for innovation](https://www.inria.fr/en/2019-inria-french-academy-sciences-dassault-systemes-innovation-prize-scikit-learn-success-story).

- [ ] Mention additional metrics, e.g. the [2021](https://www.kaggle.com/kaggle-survey-2021) and [2022](https://www.kaggle.com/kaggle-survey-2022) kaggle surveys question regarding which machine learning frameworks are used by data scientists on a regular basis?

The priority should be to update the English version, but not limited to it. In any case, this issue will be considered as solved once all the above points are addressed for said language. Because of that, if you have contributed to the article, please keep the community posted by commenting on this issue.

COMMENT:
> Notice that this is not an issue for our internal documentation (and therefore cannot be fixed by a PR), but it is documentation nevertheless.

I would therefore argue that this issue should not be open here. I would prefer to have it moved in our discussion section.

Labels: Documentation
DOC Update wikipedia article for scikit-learn

### Describe the issue linked to the documentation

The [wikipedia article on scikit-learn](https://en.wikipedia.org/wiki/Scikit-learn) covers its basic history, development, and features, but there are a few areas where additional details could enhance the content

Notice that this is not an issue for our internal documentation (and therefore cannot be fixed by a PR), but it is documentation nevertheless.

### Suggest a potential alternative/fix

We can potentially get inspired by other projects wikipedia articles, such as [XGBoost](https://en.wikipedia.org/wiki/XGBoost), but some axis that can be added to the article are:

- [x] The [wiki article for TensorFlow](https://en.wikipedia.org/wiki/TensorFlow) has an `Applications` section. We can do the same using our [Testimonials](https://scikit-learn.org/stable/testimonials/testimonials.html) as inspiration.

- [ ] Update the `Version history` section and add more recent developments and features as per the [release highlights](https://scikit-learn.org/stable/auto_examples/release_highlights/index.html).

- [x] Add an `Awards` section to mention e.g. [this Open Source Software Award](https://blog.scikit-learn.org/press/frenchaward/) or [this price for innovation](https://www.inria.fr/en/2019-inria-french-academy-sciences-dassault-systemes-innovation-prize-scikit-learn-success-story).

- [ ] Mention additional metrics, e.g. the [2021](https://www.kaggle.com/kaggle-survey-2021) and [2022](https://www.kaggle.com/kaggle-survey-2022) kaggle surveys question regarding which machine learning frameworks are used by data scientists on a regular basis?

The priority should be to update the English version, but not limited to it. In any case, this issue will be considered as solved once all the above points are addressed for said language. Because of that, if you have contributed to the article, please keep the community posted by commenting on this issue.

COMMENT:
Editing discussion should probably happen here instead: https://en.wikipedia.org/wiki/Talk:Scikit-learn

Labels: Documentation