Talip7/scikit-learn-issues-multilabel huggingface dataset:
Progress bar for BaseSearchCV and its inheritors to track iterative progress more conveniently

### Describe the workflow you want to enable

When `verbose>0`, the fitting is shown as a progress bar including the average time/pace and last fit attempted time. 

### Describe your proposed solution

Implement `tqdm`-like or `keras`-like (familiar to epoch tracking) progress bar such that the feature can be enabled through an argument of `verbose`. This proposal is under the assumption that the iteration is accessible like `for` loop. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

I've been using GridSearchCV and brute-forcing parameters (regarding algorithms, domain-knowledge agnostic) to find the best model parameters while leaving the machine unattended (disconnect from the cloud). But sometimes, the kernel might freeze without any indication. Even with `verbose` tracking, it's hard to skim through the log, especially among hundreds of fits to deduce that the process has been interrupted. 
The issue expanded from a simple inquiry of #6021 .

COMMENT:
Closing since this is a fairly common request being worked on [here](https://github.com/scikit-learn/scikit-learn/pull/22000) and is a duplicate of several prior issues (e.g. https://github.com/scikit-learn/scikit-learn/issues/7574, https://github.com/scikit-learn/scikit-learn/issues/78).

Labels: New Feature, Needs Triage
Add plotting AUC plotting tools to "plot_roc" example

### Describe the issue linked to the documentation

The following example:
https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html
does not use sklearn.metrics.plot_roc_curve or any of the related plotting tools.

### Suggest a potential alternative/fix

We should either:
- Reference the corresponding functionality on this example
- [preferably]use this functionality on this example, which might require improving it, if writing the corresponding example is clunky with our plotting utilities.

COMMENT:
Cc  @ArturoAmorQ : this might be good for you to work on.

Labels: Documentation, module:metrics
Add plotting AUC plotting tools to "plot_roc" example

### Describe the issue linked to the documentation

The following example:
https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html
does not use sklearn.metrics.plot_roc_curve or any of the related plotting tools.

### Suggest a potential alternative/fix

We should either:
- Reference the corresponding functionality on this example
- [preferably]use this functionality on this example, which might require improving it, if writing the corresponding example is clunky with our plotting utilities.

COMMENT:
I am currently working on such example in #24200. There I do implement the [`sklearn.metrics.RocCurveDisplay`](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.RocCurveDisplay.html#sklearn.metrics.RocCurveDisplay.from_predictions) for the visualizations.

Labels: Documentation, module:metrics
Add plotting AUC plotting tools to "plot_roc" example

### Describe the issue linked to the documentation

The following example:
https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html
does not use sklearn.metrics.plot_roc_curve or any of the related plotting tools.

### Suggest a potential alternative/fix

We should either:
- Reference the corresponding functionality on this example
- [preferably]use this functionality on this example, which might require improving it, if writing the corresponding example is clunky with our plotting utilities.

COMMENT:
Ok. Are you making sure that you extend the existing example, rather then creating a new one?

We have too many examples. 

On Aug 28, 2022, 17:04, at 17:04, Arturo Amor ***@***.***> wrote:
>I am currently working on such example in #24200. There I do implement
>the
>[`sklearn.metrics.RocCurveDisplay`](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.RocCurveDisplay.html#sklearn.metrics.RocCurveDisplay.from_predictions)
>for the visualizations.
>
>-- 
>Reply to this email directly or view it on GitHub:
>https://github.com/scikit-learn/scikit-learn/issues/24288#issuecomment-1229480502
>You are receiving this because you authored the thread.
>
>Message ID:
***@***.***>

Labels: Documentation, module:metrics
Add plotting AUC plotting tools to "plot_roc" example

### Describe the issue linked to the documentation

The following example:
https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html
does not use sklearn.metrics.plot_roc_curve or any of the related plotting tools.

### Suggest a potential alternative/fix

We should either:
- Reference the corresponding functionality on this example
- [preferably]use this functionality on this example, which might require improving it, if writing the corresponding example is clunky with our plotting utilities.

COMMENT:
> Are you making sure that you extend the existing example, rather then creating a new one?

Yes.

Labels: Documentation, module:metrics
Add plotting AUC plotting tools to "plot_roc" example

### Describe the issue linked to the documentation

The following example:
https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html
does not use sklearn.metrics.plot_roc_curve or any of the related plotting tools.

### Suggest a potential alternative/fix

We should either:
- Reference the corresponding functionality on this example
- [preferably]use this functionality on this example, which might require improving it, if writing the corresponding example is clunky with our plotting utilities.

COMMENT:
Yey!!! You're on top of things, it's awesome 👌 

On Aug 28, 2022, 17:50, at 17:50, Arturo Amor ***@***.***> wrote:
>> Are you making sure that you extend the existing example, rather then
>creating a new one?
>
>Yes.
>
>-- 
>Reply to this email directly or view it on GitHub:
>https://github.com/scikit-learn/scikit-learn/issues/24288#issuecomment-1229491310
>You are receiving this because you authored the thread.
>
>Message ID:
***@***.***>

Labels: Documentation, module:metrics
small typo on webpage

### Describe the issue linked to the documentation

There is a typo in https://scikit-learn.org/stable/modules/clustering.html#bisect-k-means

_Please notice the bold font below._

> [BisectingKMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.BisectingKMeans.html#sklearn.cluster.BisectingKMeans) is more efficient than [KMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html#sklearn.cluster.KMeans) when **the number the number** of clusters is large since it only works on a subset of the data at each bisection while [KMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html#sklearn.cluster.KMeans) always works on the entire dataset.

### Suggest a potential alternative/fix

_No response_

COMMENT:
Thank you for opening this issue. Do you want to open a pull requests to fix the typo?

Labels: Documentation, module:cluster
small typo on webpage

### Describe the issue linked to the documentation

There is a typo in https://scikit-learn.org/stable/modules/clustering.html#bisect-k-means

_Please notice the bold font below._

> [BisectingKMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.BisectingKMeans.html#sklearn.cluster.BisectingKMeans) is more efficient than [KMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html#sklearn.cluster.KMeans) when **the number the number** of clusters is large since it only works on a subset of the data at each bisection while [KMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html#sklearn.cluster.KMeans) always works on the entire dataset.

### Suggest a potential alternative/fix

_No response_

COMMENT:
@thomasjpfan 
Sure. Could you please provide me with the link that takes me to the corresponding file? 

I found `https://scikit-learn.org/stable/_sources/modules/clustering.rst.txt` by clicking on "show this page source" on the bottom-right of the page. However, I cannot find it on GitHub.

Labels: Documentation, module:cluster
small typo on webpage

### Describe the issue linked to the documentation

There is a typo in https://scikit-learn.org/stable/modules/clustering.html#bisect-k-means

_Please notice the bold font below._

> [BisectingKMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.BisectingKMeans.html#sklearn.cluster.BisectingKMeans) is more efficient than [KMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html#sklearn.cluster.KMeans) when **the number the number** of clusters is large since it only works on a subset of the data at each bisection while [KMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html#sklearn.cluster.KMeans) always works on the entire dataset.

### Suggest a potential alternative/fix

_No response_

COMMENT:
Here is the file and the lines with the typo:

https://github.com/scikit-learn/scikit-learn/blob/c01fad4fb47c8963c365491880aa4a54e9f03482/doc/modules/clustering.rst?plain=1#L782-L783

Labels: Documentation, module:cluster
⚠️ CI failed on macOS.pylatest_conda_forge_mkl ⚠️

**CI failed on [macOS.pylatest_conda_forge_mkl](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=45966&view=logs&j=97641769-79fb-5590-9088-a30ce9b850b9)** (Aug 26, 2022)
Unable to find junit file. Please see link for details.

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=46030&view=logs&j=97641769-79fb-5590-9088-a30ce9b850b9) on Aug 29, 2022

Labels: Build / CI
Rename OneHotEncoder option sparse to sparse_output

### Task
Introduce new parameter `sparse_output` in `OneHotEncoder` and deprecate the then old `sparse` parameter.

### Background
Several estimators have an option to return sparse output.
- `RandomTreesEmbedding(sparse_output=True)`
- `LabelBinarizer(sparse_output=True)`
- `MultiLabelBinarizer(sparse_output=True)`

`OneHotEncoder(sparse=True)` seems to be the only one to deviate from `sparse_output`.

COMMENT:
@ogrisel @thomasjpfan As you suggested in https://github.com/scikit-learn/scikit-learn/pull/24145#discussion_r940333030.

Labels: module:preprocessing
Rename OneHotEncoder option sparse to sparse_output

### Task
Introduce new parameter `sparse_output` in `OneHotEncoder` and deprecate the then old `sparse` parameter.

### Background
Several estimators have an option to return sparse output.
- `RandomTreesEmbedding(sparse_output=True)`
- `LabelBinarizer(sparse_output=True)`
- `MultiLabelBinarizer(sparse_output=True)`

`OneHotEncoder(sparse=True)` seems to be the only one to deviate from `sparse_output`.

COMMENT:
Hello, is anyone working on this? I would like to help with this.

Labels: module:preprocessing
Rename OneHotEncoder option sparse to sparse_output

### Task
Introduce new parameter `sparse_output` in `OneHotEncoder` and deprecate the then old `sparse` parameter.

### Background
Several estimators have an option to return sparse output.
- `RandomTreesEmbedding(sparse_output=True)`
- `LabelBinarizer(sparse_output=True)`
- `MultiLabelBinarizer(sparse_output=True)`

`OneHotEncoder(sparse=True)` seems to be the only one to deviate from `sparse_output`.

COMMENT:
@rusdes Go on. We would welcome a PR. Let us know if you need direction or help.

Labels: module:preprocessing
Rename OneHotEncoder option sparse to sparse_output

### Task
Introduce new parameter `sparse_output` in `OneHotEncoder` and deprecate the then old `sparse` parameter.

### Background
Several estimators have an option to return sparse output.
- `RandomTreesEmbedding(sparse_output=True)`
- `LabelBinarizer(sparse_output=True)`
- `MultiLabelBinarizer(sparse_output=True)`

`OneHotEncoder(sparse=True)` seems to be the only one to deviate from `sparse_output`.

COMMENT:
@lorentzenchr I'm working on this. I was confused about what warning message to display.
```'sparse' was renamed to sparse_output in version 1.x and will be removed in 1.y.```

What version numbers should I mention?

Also, what changes should I make in `test_encoders.py`? Should I change `sparse` to `sparse_output` in all tests or add a new test that checks for the deprecation warning? Sorry, I'm a little new to this and am trying to learn :)

Labels: module:preprocessing
Rename OneHotEncoder option sparse to sparse_output

### Task
Introduce new parameter `sparse_output` in `OneHotEncoder` and deprecate the then old `sparse` parameter.

### Background
Several estimators have an option to return sparse output.
- `RandomTreesEmbedding(sparse_output=True)`
- `LabelBinarizer(sparse_output=True)`
- `MultiLabelBinarizer(sparse_output=True)`

`OneHotEncoder(sparse=True)` seems to be the only one to deviate from `sparse_output`.

COMMENT:
Have a look at https://scikit-learn.org/stable/developers/contributing.html#deprecation and also search in the code base for other deprecations that are already in place.

We should aim to deprecate with version 1.2.0 and removing with 1.4.0. Let's see if we make it into the next release.

Labels: module:preprocessing
Rename OneHotEncoder option sparse to sparse_output

### Task
Introduce new parameter `sparse_output` in `OneHotEncoder` and deprecate the then old `sparse` parameter.

### Background
Several estimators have an option to return sparse output.
- `RandomTreesEmbedding(sparse_output=True)`
- `LabelBinarizer(sparse_output=True)`
- `MultiLabelBinarizer(sparse_output=True)`

`OneHotEncoder(sparse=True)` seems to be the only one to deviate from `sparse_output`.

COMMENT:
@lorentzenchr Shouldn't it be `OneHotEncoder` instead of `OrdinalEncoder`? This comment https://github.com/scikit-learn/scikit-learn/pull/24145#discussion_r946532252 talks about making changes to `OneHotEncoder`.

I've gone ahead and added `sparse_output` to `OneHotEncoder`. Let me know if I missed anything.
Thanks!

Labels: module:preprocessing
Rename OneHotEncoder option sparse to sparse_output

### Task
Introduce new parameter `sparse_output` in `OneHotEncoder` and deprecate the then old `sparse` parameter.

### Background
Several estimators have an option to return sparse output.
- `RandomTreesEmbedding(sparse_output=True)`
- `LabelBinarizer(sparse_output=True)`
- `MultiLabelBinarizer(sparse_output=True)`

`OneHotEncoder(sparse=True)` seems to be the only one to deviate from `sparse_output`.

COMMENT:
@rusdes Thanks, I corrected title and description.

Labels: module:preprocessing
Problem with ransac predictions.

I think I have encountered a bug in sklearn.linear_model.RANSACRegressor.

I was using it to simplify OpenCV contours, on the occupancy grid map. Unfortunately, RANSAC was pretty bad at detecting almost straight points perpendicular to the x-axis. I have checked it on a simpler training sample, but still, it's doing the same thing (my RANSAC settings were much more complex than what you can see here).

My code:
```python
ransac = RANSACRegressor()


X,y = np.array([0,5,5,5,5]).reshape(-1,1),np.array([0.5,0,1,2,3]).reshape(-1,1)
ransac.fit(X,y)

inliers = ransac.inlier_mask_

xpred = X[inliers].reshape(-1,1)
ypred = ransac.predict(xpred).reshape(-1,1)

plt.scatter(X,y, color='blue')
plt.scatter(xpred,ypred,color='orange')
plt.show()
```

Is it my fault or some flaw in the source code?

COMMENT:
Moving this in discussion for the moment.

Labels: Needs Triage
MAINT Simplify some `KMeans` routines' signatures

Looking at the [scikit-learn's `_kmeans_single_lloyd`'s routines](https://github.com/scikit-learn/scikit-learn/blob/7bdd426704fc49316b3273fb33de7f6931c22692/sklearn/cluster/_k_means_lloyd.pyx), some of their signatures can be simplified.

For instance:
 - `x_squared_norms` must be passed (and hence computed before-hand as done here) for the routines to be usable (as it's typed as a memoryview but is `None` if not specified as a numpy array). However, it is never used in the routines. This was first observed in https://github.com/soda-inria/sklearn-numba-dpex/pull/2#discussion_r953852999.
 - potentially more to come.

cc @jeremiedbb

COMMENT:
Hello, is anyone working on this?

Labels: Easy, cython
MAINT Simplify some `KMeans` routines' signatures

Looking at the [scikit-learn's `_kmeans_single_lloyd`'s routines](https://github.com/scikit-learn/scikit-learn/blob/7bdd426704fc49316b3273fb33de7f6931c22692/sklearn/cluster/_k_means_lloyd.pyx), some of their signatures can be simplified.

For instance:
 - `x_squared_norms` must be passed (and hence computed before-hand as done here) for the routines to be usable (as it's typed as a memoryview but is `None` if not specified as a numpy array). However, it is never used in the routines. This was first observed in https://github.com/soda-inria/sklearn-numba-dpex/pull/2#discussion_r953852999.
 - potentially more to come.

cc @jeremiedbb

COMMENT:
Hi @rusdes,

@Vincent-Maladiere or @fcharras might. Please do indicate if it is the case here.

This issue description is rather brief, and there might be several points to treat. In that case, each of you should be able to solve one of them independently. I might update the PR description with more points and will let you know.

Labels: Easy, cython
How can I perform multiple transformations of columns with some columns being same across the transformations

For my usecase, I wanted to perform target encoding for some columns (say c1, c2,c3) and I also want to perform imputation for a column (c4) and I now wanted to perform standardscalar (once the previous target encoding and imputation are performed) for all these and more columns (c1,c2,c3,c4,c5,c6,c7,c8). 

Note I also have more columns that I need to perform, say  OHE, but those columns are disjoint from the above c1...c8. 

I am not sure how to use column transformer and pipeline, such that when I perform different transformations with some of the columns being same among the transformations (as in the above example). Is there a way to do this as part of the pipeline. Thank you.

COMMENT:
Moving this into a discussion

Labels: Needs Triage
Random Forest Neighbors

### Describe the workflow you want to enable

Random Regression forests allow for an interpretation where the values at a new point  `x`  is a weighted linear combination of the values observed in the dataset 
<img width="188" alt="image" src="https://user-images.githubusercontent.com/61052993/186520968-88e824ed-77c1-4bf8-b7be-9885c288778e.png">
I think it would be helpful if one could obtain these weights for using the sklearn random forest for defining (for example) local neighbourhoods.

### Describe your proposed solution

I think there should be a method which would allow one to access these neighbourhoods.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

If there is any interest I could look deeper into this issue and work to get a PR on it.

COMMENT:
Hey there @velezbeltran! Scikit-learn has some strict [inclusion criteria](https://scikit-learn.org/stable/faq.html#what-are-the-inclusion-criteria-for-new-algorithms) for new features. To that end, do you have any resources/papers you could link to which use/justify this technique?

Labels: New Feature
Random Forest Neighbors

### Describe the workflow you want to enable

Random Regression forests allow for an interpretation where the values at a new point  `x`  is a weighted linear combination of the values observed in the dataset 
<img width="188" alt="image" src="https://user-images.githubusercontent.com/61052993/186520968-88e824ed-77c1-4bf8-b7be-9885c288778e.png">
I think it would be helpful if one could obtain these weights for using the sklearn random forest for defining (for example) local neighbourhoods.

### Describe your proposed solution

I think there should be a method which would allow one to access these neighbourhoods.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

If there is any interest I could look deeper into this issue and work to get a PR on it.

COMMENT:
Sincere apologies, I completely missed that there had been an answer and thought it had been ignored. 

Yes this is used quite a bit. Take a look for example at this one
https://arxiv.org/pdf/1807.11408.pdf

Or more importantly this paper with over 1000 citations.
https://arxiv.org/pdf/1610.01271.pdf
What I refer to is what they call `alpha` in the paper above. 

It can also be used for explainability: See what neighbors contributed to a prediction and look at the weights to see what the model is "thinking."

Essentially, what I am thinking of is something akin to this.
https://grf-labs.github.io/grf/reference/get_forest_weights.html
This is what I was using but I thought it would be nice if these could be pulled using sci-kit learn. 

Again sincere apologies about missing this @Micky774

Labels: New Feature
Random Forest Neighbors

### Describe the workflow you want to enable

Random Regression forests allow for an interpretation where the values at a new point  `x`  is a weighted linear combination of the values observed in the dataset 
<img width="188" alt="image" src="https://user-images.githubusercontent.com/61052993/186520968-88e824ed-77c1-4bf8-b7be-9885c288778e.png">
I think it would be helpful if one could obtain these weights for using the sklearn random forest for defining (for example) local neighbourhoods.

### Describe your proposed solution

I think there should be a method which would allow one to access these neighbourhoods.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

If there is any interest I could look deeper into this issue and work to get a PR on it.

COMMENT:
No worries, things get lost in the shuffle a lot! 

I don't think it's unreasonable to expose the weights themselves, but I'm not super familiar with the current trees/forests API.

Wondering if perhaps @ogrisel @glemaitre @jjerphan have any thoughts on this.

Labels: New Feature
Random Forest Neighbors

### Describe the workflow you want to enable

Random Regression forests allow for an interpretation where the values at a new point  `x`  is a weighted linear combination of the values observed in the dataset 
<img width="188" alt="image" src="https://user-images.githubusercontent.com/61052993/186520968-88e824ed-77c1-4bf8-b7be-9885c288778e.png">
I think it would be helpful if one could obtain these weights for using the sklearn random forest for defining (for example) local neighbourhoods.

### Describe your proposed solution

I think there should be a method which would allow one to access these neighbourhoods.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

If there is any interest I could look deeper into this issue and work to get a PR on it.

COMMENT:
Great. If people think it would be helpful I'd be happy to make give a PR a shot.

Labels: New Feature
Random Forest Neighbors

### Describe the workflow you want to enable

Random Regression forests allow for an interpretation where the values at a new point  `x`  is a weighted linear combination of the values observed in the dataset 
<img width="188" alt="image" src="https://user-images.githubusercontent.com/61052993/186520968-88e824ed-77c1-4bf8-b7be-9885c288778e.png">
I think it would be helpful if one could obtain these weights for using the sklearn random forest for defining (for example) local neighbourhoods.

### Describe your proposed solution

I think there should be a method which would allow one to access these neighbourhoods.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

If there is any interest I could look deeper into this issue and work to get a PR on it.

COMMENT:
Hi @velezbeltran,

Thank you for the proposal. We need sometime to study it and know if some methods are worth integrating in scikit-learn.

In the meantime, you can have a read at #24000 #24577 which aim at making tree-based models' extensions easier within scikit-learn but also within third party packages.

Labels: New Feature