Talip7/scikit-learn-issues-multilabel huggingface dataset:
Add optional return of STD for kNeighboursRegressor

### Describe the workflow you want to enable

I would like to propose to add the option to get the standard deviation from the KNeighborsRegressor. The `.predict()` function already delivers the mean, as that's the way the target is calculated, so adding the standard deviation should not be a big deal.

### Describe your proposed solution

Similar to the `.predict(return_std=False)` Function of the `GaussianProcessRegressor`, a switch for the `.predict()` function of the `KNeighborsRegressor` could be implemented.
This can easily be implemented by e.g. changing
```
            y_pred = np.mean(_y[neigh_ind], axis=1)
```
to
```
            y_pred = [np.mean(_y[neigh_ind], axis=1), np.std(_y[neigh_ind], axis=1)]
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I would like to take this task and work on it. Will open a PR based on the discussion so far.

Labels: Enhancement, module:neighbors
Add optional return of STD for kNeighboursRegressor

### Describe the workflow you want to enable

I would like to propose to add the option to get the standard deviation from the KNeighborsRegressor. The `.predict()` function already delivers the mean, as that's the way the target is calculated, so adding the standard deviation should not be a big deal.

### Describe your proposed solution

Similar to the `.predict(return_std=False)` Function of the `GaussianProcessRegressor`, a switch for the `.predict()` function of the `KNeighborsRegressor` could be implemented.
This can easily be implemented by e.g. changing
```
            y_pred = np.mean(_y[neigh_ind], axis=1)
```
to
```
            y_pred = [np.mean(_y[neigh_ind], axis=1), np.std(_y[neigh_ind], axis=1)]
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@ankit-dhokariya we still don't have a conclusion on this issue regarding whether we'd like to have the feature or not.

Labels: Enhancement, module:neighbors
Add optional return of STD for kNeighboursRegressor

### Describe the workflow you want to enable

I would like to propose to add the option to get the standard deviation from the KNeighborsRegressor. The `.predict()` function already delivers the mean, as that's the way the target is calculated, so adding the standard deviation should not be a big deal.

### Describe your proposed solution

Similar to the `.predict(return_std=False)` Function of the `GaussianProcessRegressor`, a switch for the `.predict()` function of the `KNeighborsRegressor` could be implemented.
This can easily be implemented by e.g. changing
```
            y_pred = np.mean(_y[neigh_ind], axis=1)
```
to
```
            y_pred = [np.mean(_y[neigh_ind], axis=1), np.std(_y[neigh_ind], axis=1)]
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
My bad. Will keep this PR open until we have a conclusion. @adrinjalali can you add a label mentioning needs discussion. Thanks.

Labels: Enhancement, module:neighbors
Add optional return of STD for kNeighboursRegressor

### Describe the workflow you want to enable

I would like to propose to add the option to get the standard deviation from the KNeighborsRegressor. The `.predict()` function already delivers the mean, as that's the way the target is calculated, so adding the standard deviation should not be a big deal.

### Describe your proposed solution

Similar to the `.predict(return_std=False)` Function of the `GaussianProcessRegressor`, a switch for the `.predict()` function of the `KNeighborsRegressor` could be implemented.
This can easily be implemented by e.g. changing
```
            y_pred = np.mean(_y[neigh_ind], axis=1)
```
to
```
            y_pred = [np.mean(_y[neigh_ind], axis=1), np.std(_y[neigh_ind], axis=1)]
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I would love to see this feature, but 
- we 100% should add note about how ordering of the data would affect the results;
- we should handle the case when `n_neighbors = 1` or `n_neighbors = 2` (if we set `ddof=1`);
- we should handle edge cases, for example user set `n_neighbors = 5` and a sample has 5 neighbors with distances equal to 0. I'm not sure what to return in this case (returning std=0 would be fine by me, but I could be wrong);

P.S. Should we explain how `KNeighborsRegressor` would differ from `GaussianProcessRegressor` (GPR has `RationalQuadratic` and `RBF` kernels which use euclidean distance (a bit transformed), same as KNR by default)?

Labels: Enhancement, module:neighbors
mirrors-prettier pre-commit has been archived so maybe should be replaced

### Describe the bug

Noticed your [mirrors-prettier pre-commit](https://github.com/pre-commit/mirrors-prettier) has been archived. I was going to suggest you remove and/or look for alternative linters for the scss / js files.

### Steps/Code to Reproduce

Noticed this in the .pre-commit-config.yaml

```yaml
-   repo: https://github.com/pre-commit/mirrors-prettier
    rev: v2.7.1
    hooks:
    -   id: prettier
        files: ^doc/scss/|^doc/js/scripts/
        exclude: ^doc/js/scripts/vendor/
        types_or: ["scss", "javascript"]
```

### Expected Results

N/A

### Actual Results

N/A

### Versions

```shell
1.5.1
```

COMMENT:
Thanks for noticing this. The prettier hook is used only for the style sheets and some javascript stuff in the docs. I'm okay with removing it and just manually taking care of formatting, or it seems also easy to create one on our own.

Labels: Bug
mirrors-prettier pre-commit has been archived so maybe should be replaced

### Describe the bug

Noticed your [mirrors-prettier pre-commit](https://github.com/pre-commit/mirrors-prettier) has been archived. I was going to suggest you remove and/or look for alternative linters for the scss / js files.

### Steps/Code to Reproduce

Noticed this in the .pre-commit-config.yaml

```yaml
-   repo: https://github.com/pre-commit/mirrors-prettier
    rev: v2.7.1
    hooks:
    -   id: prettier
        files: ^doc/scss/|^doc/js/scripts/
        exclude: ^doc/js/scripts/vendor/
        types_or: ["scss", "javascript"]
```

### Expected Results

N/A

### Actual Results

N/A

### Versions

```shell
1.5.1
```

COMMENT:
If it helps we found someone had made a fork here:
https://github.com/rbubley/mirrors-prettier That apparently will continue
support and the apparent change in Prettier that caused issues with
pre-commit might not be as bad as originally made out.

On Fri, 23 Aug 2024, 3:40 am Yao Xiao, ***@***.***> wrote:

> Thanks for noticing this. The prettier hook is used only for the style
> sheets and some javascript stuff in the docs. I'm okay with removing it and
> just manually taking care of formatting, or it seems also easy to create
> one on our own.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/scikit-learn/scikit-learn/issues/29621#issuecomment-2305355124>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADAVYGYIHCVTP5KJR5FN5Q3ZSYSSNAVCNFSM6AAAAABL6TF5KKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBVGM2TKMJSGQ>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>

Labels: Bug
mirrors-prettier pre-commit has been archived so maybe should be replaced

### Describe the bug

Noticed your [mirrors-prettier pre-commit](https://github.com/pre-commit/mirrors-prettier) has been archived. I was going to suggest you remove and/or look for alternative linters for the scss / js files.

### Steps/Code to Reproduce

Noticed this in the .pre-commit-config.yaml

```yaml
-   repo: https://github.com/pre-commit/mirrors-prettier
    rev: v2.7.1
    hooks:
    -   id: prettier
        files: ^doc/scss/|^doc/js/scripts/
        exclude: ^doc/js/scripts/vendor/
        types_or: ["scss", "javascript"]
```

### Expected Results

N/A

### Actual Results

N/A

### Versions

```shell
1.5.1
```

COMMENT:
More info on this https://github.com/prettier/prettier/issues/15742#issuecomment-2136884297, oh well :sweat: 

Probaly using `rbubley/mirrors-prettier` is the pragmatic option until the dust settles on this one.

Labels: Bug
`base_estimator` in `Chain` classes while `estimator` is the convention in `Bagging` and `MultiOutput` classes?

### Describe the issue linked to the documentation

Currently most ensembling methods in `scikit-learn` such as [bagging methods](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingClassifier.html), and independent multioutput classes ([`MultiOutputClassifier`](https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html) and [`MultiOutputRegressor`](https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputRegressor.html)) use the `estimator` parameter during instantiation, but `ClassifierChain` and `ClassifierRegressor` use the `base_estimator` parameter, which I think is the old convention? Or is there any other reason to continue to use `base_estimator` for `ClassifierChain` and `RegressorChain`? Even the stacking methods use `estimators` the plural term as a matter of consistence.

### Suggest a potential alternative/fix

Perhaps, the class definitions and documentations for `ClassifierChain` and `RegressorChain` needs to be updated?

COMMENT:
I think it might just be a leftover/historical thing. I think it would make sense to change it (together with deprecation warnings, etc).

Labels: API, help wanted
`base_estimator` in `Chain` classes while `estimator` is the convention in `Bagging` and `MultiOutput` classes?

### Describe the issue linked to the documentation

Currently most ensembling methods in `scikit-learn` such as [bagging methods](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingClassifier.html), and independent multioutput classes ([`MultiOutputClassifier`](https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html) and [`MultiOutputRegressor`](https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputRegressor.html)) use the `estimator` parameter during instantiation, but `ClassifierChain` and `ClassifierRegressor` use the `base_estimator` parameter, which I think is the old convention? Or is there any other reason to continue to use `base_estimator` for `ClassifierChain` and `RegressorChain`? Even the stacking methods use `estimators` the plural term as a matter of consistence.

### Suggest a potential alternative/fix

Perhaps, the class definitions and documentations for `ClassifierChain` and `RegressorChain` needs to be updated?

COMMENT:
+1 for making those consistent as well.

Labels: API, help wanted
`base_estimator` in `Chain` classes while `estimator` is the convention in `Bagging` and `MultiOutput` classes?

### Describe the issue linked to the documentation

Currently most ensembling methods in `scikit-learn` such as [bagging methods](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingClassifier.html), and independent multioutput classes ([`MultiOutputClassifier`](https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html) and [`MultiOutputRegressor`](https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputRegressor.html)) use the `estimator` parameter during instantiation, but `ClassifierChain` and `ClassifierRegressor` use the `base_estimator` parameter, which I think is the old convention? Or is there any other reason to continue to use `base_estimator` for `ClassifierChain` and `RegressorChain`? Even the stacking methods use `estimators` the plural term as a matter of consistence.

### Suggest a potential alternative/fix

Perhaps, the class definitions and documentations for `ClassifierChain` and `RegressorChain` needs to be updated?

COMMENT:
This issue can be solved by looking at the process carry on in the following PR: https://github.com/scikit-learn/scikit-learn/pull/23819 that enforce the consistency for the classes in the ensemble module.

Labels: API, help wanted
`base_estimator` in `Chain` classes while `estimator` is the convention in `Bagging` and `MultiOutput` classes?

### Describe the issue linked to the documentation

Currently most ensembling methods in `scikit-learn` such as [bagging methods](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingClassifier.html), and independent multioutput classes ([`MultiOutputClassifier`](https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html) and [`MultiOutputRegressor`](https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputRegressor.html)) use the `estimator` parameter during instantiation, but `ClassifierChain` and `ClassifierRegressor` use the `base_estimator` parameter, which I think is the old convention? Or is there any other reason to continue to use `base_estimator` for `ClassifierChain` and `RegressorChain`? Even the stacking methods use `estimators` the plural term as a matter of consistence.

### Suggest a potential alternative/fix

Perhaps, the class definitions and documentations for `ClassifierChain` and `RegressorChain` needs to be updated?

COMMENT:
I am trying to work on this issue.. Let's see what it brings to the table.

Labels: API, help wanted
Student-t Mixture Model

### Describe the workflow you want to enable

Gaussian mixtures are extremely useful, but many datasets are noisy enough that a GMM fit can be challenging. In these cases, adding a degree of freedom by using a t distribution instead of a normal distribution can make fitting significantly simpler and easier.

### Describe your proposed solution

In 2000, Peel & McLachlan published [Robust mixture modelling using the t distribution](https://people.smp.uq.edu.au/GeoffMcLachlan/pm_sc00.pdf). This paper is the basis of [an unmaintained GitHub repo](https://github.com/omritomer/student_mixture) containing an implementation that utilizes the existing scikit-learn `BaseMixture` class infrastructure but is somewhat out of date. With some minor changes, that code in this repo has been extremely helpful with fitting some particularly noisy datasets that I need to routinely handle. It seems like this the sort of thing that would also be useful to others, especially since it fits wells into the current scikit-learn design.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

I'd be happy to develop a PR to integrate the `student_mixture` code into the scikit-learn codebase, but I thought it would be a good idea to get feedback before starting that work.

COMMENT:
Thanks for opening the issue first to start a discussion!

Adding a new estimator to scikit-learn is amongst the hardest things known to humans ;) Often the reasons for this is a mixture of maintainer time to review, getting consensus, reworking existing code to full-fill all of scikit-learn's requirements, considerations around maintenance effort, etc.

This means [scikit-learn-contrib](https://github.com/scikit-learn-contrib/scikit-learn-contrib/blob/master/README.md) is something to consider. Especially as there is already a repository that implements all of what you need. It might be easier to ask the author if they'd like to move their project to the scikit-learn-contrib organisation (Or if they are fine with someone else doing it), do the work to refurbish the project and be off to the races. The hurdle for inclusion is much lower, and if the package gets lots of users the step to including it in scikit-learn is smaller.

Do you know how Student's t for GMMs ranks according to https://scikit-learn.org/stable/faq.html#what-are-the-inclusion-criteria-for-new-algorithms? This is a good/obvious question to figure out the answer to as it will come up in discussions for inclusion.

Is there someone amongst the @scikit-learn/core-devs who knows about GMMs, Student t's and wants to champion (or veto?) inclusion?

Labels: New Feature
Student-t Mixture Model

### Describe the workflow you want to enable

Gaussian mixtures are extremely useful, but many datasets are noisy enough that a GMM fit can be challenging. In these cases, adding a degree of freedom by using a t distribution instead of a normal distribution can make fitting significantly simpler and easier.

### Describe your proposed solution

In 2000, Peel & McLachlan published [Robust mixture modelling using the t distribution](https://people.smp.uq.edu.au/GeoffMcLachlan/pm_sc00.pdf). This paper is the basis of [an unmaintained GitHub repo](https://github.com/omritomer/student_mixture) containing an implementation that utilizes the existing scikit-learn `BaseMixture` class infrastructure but is somewhat out of date. With some minor changes, that code in this repo has been extremely helpful with fitting some particularly noisy datasets that I need to routinely handle. It seems like this the sort of thing that would also be useful to others, especially since it fits wells into the current scikit-learn design.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

I'd be happy to develop a PR to integrate the `student_mixture` code into the scikit-learn codebase, but I thought it would be a good idea to get feedback before starting that work.

COMMENT:
I would stick to the inclusion criteria to make a decision — there is a variety of neat mixture models using the generalized normal distribution (which includes the Student-t distribution), but I do not know if there are that used in practice.

Labels: New Feature
Student-t Mixture Model

### Describe the workflow you want to enable

Gaussian mixtures are extremely useful, but many datasets are noisy enough that a GMM fit can be challenging. In these cases, adding a degree of freedom by using a t distribution instead of a normal distribution can make fitting significantly simpler and easier.

### Describe your proposed solution

In 2000, Peel & McLachlan published [Robust mixture modelling using the t distribution](https://people.smp.uq.edu.au/GeoffMcLachlan/pm_sc00.pdf). This paper is the basis of [an unmaintained GitHub repo](https://github.com/omritomer/student_mixture) containing an implementation that utilizes the existing scikit-learn `BaseMixture` class infrastructure but is somewhat out of date. With some minor changes, that code in this repo has been extremely helpful with fitting some particularly noisy datasets that I need to routinely handle. It seems like this the sort of thing that would also be useful to others, especially since it fits wells into the current scikit-learn design.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

I'd be happy to develop a PR to integrate the `student_mixture` code into the scikit-learn codebase, but I thought it would be a good idea to get feedback before starting that work.

COMMENT:
From the text of the inclusion criteria, it sounds like there are 2 measurable criteria and 1 qualitative criteria:

1. 3+ years since publication
2. 200+ citations
3. "wide use and usefulness"

Criteria 1 seems straightforward; the current year is 2024, the year of publication is 2000, 2024 - 2000 = 24, and 24 > 5, therefore Criteria 1 is met. Criteria 2 seems slightly less straightforward since it depends on accurate indexing performed by external parties, but accepting that caveat, [Google Scholar indicates 1,167 citations to the cited article](https://scholar.google.com/scholar?cites=3889995576520047328&as_sdt=2005&sciodt=0,5&hl=en), and 1167 > 200, therefore Criteria 2 is met. Perhaps Web of Science indicates a smaller number of citations, if someone who has access to WoS could check? Criteria 3 does not immediately suggest to me how I might determine whether or not it is met except that, as @betatim suggests, its use and usefulness are obvious to someone on the core team.

Labels: New Feature
Student-t Mixture Model

### Describe the workflow you want to enable

Gaussian mixtures are extremely useful, but many datasets are noisy enough that a GMM fit can be challenging. In these cases, adding a degree of freedom by using a t distribution instead of a normal distribution can make fitting significantly simpler and easier.

### Describe your proposed solution

In 2000, Peel & McLachlan published [Robust mixture modelling using the t distribution](https://people.smp.uq.edu.au/GeoffMcLachlan/pm_sc00.pdf). This paper is the basis of [an unmaintained GitHub repo](https://github.com/omritomer/student_mixture) containing an implementation that utilizes the existing scikit-learn `BaseMixture` class infrastructure but is somewhat out of date. With some minor changes, that code in this repo has been extremely helpful with fitting some particularly noisy datasets that I need to routinely handle. It seems like this the sort of thing that would also be useful to others, especially since it fits wells into the current scikit-learn design.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

I'd be happy to develop a PR to integrate the `student_mixture` code into the scikit-learn codebase, but I thought it would be a good idea to get feedback before starting that work.

COMMENT:
No word from the original developer & lack of interest here. Fortunately, it's easy enough to maintain my own hack fork. Thanks for all your hard work! ✌️

Labels: New Feature
⚠️ CI failed on Wheel builder (last failure: Aug 08, 2024) ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/10295577740)** (Aug 08, 2024)

COMMENT:
There was a similar bus error reported in #29567, also when calling numpy 2.0.1's `_unique1d` under a cross-validation loop on macOS arm64.

And @lesteve linked to similar failures under the following comment:

- https://github.com/scikit-learn/scikit-learn/issues/29567#issuecomment-2255044431

All of them involve `_unique1d`, numpy 2.0.1 on macOS arm64.

Labels: Bug, Build / CI
⚠️ CI failed on Wheel builder (last failure: Aug 08, 2024) ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/10295577740)** (Aug 08, 2024)

COMMENT:
Build wheel for `cp311-macosx_arm64-` also failed last night with the same `_unique1d` crash.

Labels: Bug, Build / CI
⚠️ CI failed on Wheel builder (last failure: Aug 08, 2024) ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/10295577740)** (Aug 08, 2024)

COMMENT:
The bus errors on macOS arm64 seem to have been successfully fixed by https://github.com/numpy/numpy/pull/27126. They will disappear once numpy 2.1.0 (or maybe 2.0.2?) is released.

Labels: Bug, Build / CI
⚠️ CI failed on Wheel builder (last failure: Aug 08, 2024) ⚠️

**CI is still failing on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/10295577740)** (Aug 08, 2024)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://github.com/scikit-learn/scikit-learn/actions/runs/10313295232) on Aug 09, 2024

Labels: Bug, Build / CI