Talip7/scikit-learn-issues-multilabel huggingface dataset:
Spherical K-means support (unit norm centroids and input)

### Describe the workflow you want to enable

Hi,
I was wondering if there is—or has been—any initiative to support cosine similarity in the KMeans implementation (i.e., spherical KMeans). I find the algorithm quite useful and would be happy to propose an implementation. The addition should be relatively straightforward.

### Describe your proposed solution

Enable the use of cosine similarity with KMeans or implement a separate SphericalKMeans class.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> Hey I'd like to take up this [@Radu1999](https://github.com/Radu1999)

No, it's ok, I was planning to implement it once I confirm there is interest for it.

Labels: New Feature, Needs Decision - Include Feature
Spherical K-means support (unit norm centroids and input)

### Describe the workflow you want to enable

Hi,
I was wondering if there is—or has been—any initiative to support cosine similarity in the KMeans implementation (i.e., spherical KMeans). I find the algorithm quite useful and would be happy to propose an implementation. The addition should be relatively straightforward.

### Describe your proposed solution

Enable the use of cosine similarity with KMeans or implement a separate SphericalKMeans class.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Fairs go for it then

Labels: New Feature, Needs Decision - Include Feature
Spherical K-means support (unit norm centroids and input)

### Describe the workflow you want to enable

Hi,
I was wondering if there is—or has been—any initiative to support cosine similarity in the KMeans implementation (i.e., spherical KMeans). I find the algorithm quite useful and would be happy to propose an implementation. The addition should be relatively straightforward.

### Describe your proposed solution

Enable the use of cosine similarity with KMeans or implement a separate SphericalKMeans class.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@scikit-learn/core-devs any interest in this?

@Radu1999 to help evaluate this, could you provide some references and context that helps answer the questions from https://scikit-learn.org/stable/faq.html#what-are-the-inclusion-criteria-for-new-algorithms

Labels: New Feature, Needs Decision - Include Feature
Spherical K-means support (unit norm centroids and input)

### Describe the workflow you want to enable

Hi,
I was wondering if there is—or has been—any initiative to support cosine similarity in the KMeans implementation (i.e., spherical KMeans). I find the algorithm quite useful and would be happy to propose an implementation. The addition should be relatively straightforward.

### Describe your proposed solution

Enable the use of cosine similarity with KMeans or implement a separate SphericalKMeans class.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
+1

Labels: New Feature, Needs Decision - Include Feature
Spherical K-means support (unit norm centroids and input)

### Describe the workflow you want to enable

Hi,
I was wondering if there is—or has been—any initiative to support cosine similarity in the KMeans implementation (i.e., spherical KMeans). I find the algorithm quite useful and would be happy to propose an implementation. The addition should be relatively straightforward.

### Describe your proposed solution

Enable the use of cosine similarity with KMeans or implement a separate SphericalKMeans class.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> @Radu1999 to help evaluate this, could you provide some references and context that helps answer the questions from https://scikit-learn.org/stable/faq.html#what-are-the-inclusion-criteria-for-new-algorithms

@betatim [the paper](https://www.jstatsoft.org/index.php/jss/article/view/v050i10/633) has more than 200+ citations and it is published in 2012..
IMO it excels at clustering normalized, directional data (like text), where vector direction matters more than magnitude.

Labels: New Feature, Needs Decision - Include Feature
Spherical K-means support (unit norm centroids and input)

### Describe the workflow you want to enable

Hi,
I was wondering if there is—or has been—any initiative to support cosine similarity in the KMeans implementation (i.e., spherical KMeans). I find the algorithm quite useful and would be happy to propose an implementation. The addition should be relatively straightforward.

### Describe your proposed solution

Enable the use of cosine similarity with KMeans or implement a separate SphericalKMeans class.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@betatim I'll like to take this!

Labels: New Feature, Needs Decision - Include Feature
Spherical K-means support (unit norm centroids and input)

### Describe the workflow you want to enable

Hi,
I was wondering if there is—or has been—any initiative to support cosine similarity in the KMeans implementation (i.e., spherical KMeans). I find the algorithm quite useful and would be happy to propose an implementation. The addition should be relatively straightforward.

### Describe your proposed solution

Enable the use of cosine similarity with KMeans or implement a separate SphericalKMeans class.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> @scikit-learn/core-devs any interest in this?

I'm not sure that I would consider this as a priority

Labels: New Feature, Needs Decision - Include Feature
Spherical K-means support (unit norm centroids and input)

### Describe the workflow you want to enable

Hi,
I was wondering if there is—or has been—any initiative to support cosine similarity in the KMeans implementation (i.e., spherical KMeans). I find the algorithm quite useful and would be happy to propose an implementation. The addition should be relatively straightforward.

### Describe your proposed solution

Enable the use of cosine similarity with KMeans or implement a separate SphericalKMeans class.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I'd say with a small maintainable implementation, I'd be happy to have it.

Labels: New Feature, Needs Decision - Include Feature
Spherical K-means support (unit norm centroids and input)

### Describe the workflow you want to enable

Hi,
I was wondering if there is—or has been—any initiative to support cosine similarity in the KMeans implementation (i.e., spherical KMeans). I find the algorithm quite useful and would be happy to propose an implementation. The addition should be relatively straightforward.

### Describe your proposed solution

Enable the use of cosine similarity with KMeans or implement a separate SphericalKMeans class.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I'd suggest adding configurable distance metric with 'euclidean' by default into existing Kmeans, rather than implementing a separate class. Just like here: https://spark.apache.org/docs/latest/api/python/_modules/pyspark/ml/clustering.html#KMeans

Labels: New Feature, Needs Decision - Include Feature
⚠️ CI failed on Wheel builder (last failure: May 28, 2025) ⚠️

**CI failed on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/15291085639)** (May 28, 2025)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://github.com/scikit-learn/scikit-learn/actions/runs/15315855640) on May 29, 2025

Labels: Needs Triage
Folder/Directory descriptions not present

### Describe the issue linked to the documentation

I was navigating through the codebase, trying to find source code for some algorithms. I noticed that there are no descriptions of files present within a folder, which would actually make it easier to navigate through the codebase. We can have a small readme file within folders which would describe what is present in that folder. 

### Suggest a potential alternative/fix

_No response_

COMMENT:
Is it open? Can I take this PR?

Labels: Documentation
Folder/Directory descriptions not present

### Describe the issue linked to the documentation

I was navigating through the codebase, trying to find source code for some algorithms. I noticed that there are no descriptions of files present within a folder, which would actually make it easier to navigate through the codebase. We can have a small readme file within folders which would describe what is present in that folder. 

### Suggest a potential alternative/fix

_No response_

COMMENT:
This is a great point.  I’ve also found that navigating large codebases like scikit-learn can be challenging without brief descriptions or READMEs in subdirectories. Even a single sentence explaining the intent of each folder (e.g., algorithm families, utilities, tests) could go a long way for new contributors or those exploring the internals for the first time.

It might be helpful to prioritize documentation for the most commonly accessed directories first (like sklearn/linear_model/, sklearn/ensemble/, etc.), and even include guidance for contributors who want to help write these descriptions.

Definitely a solid suggestion that aligns well with improving dev onboarding and overall accessibility.

Labels: Documentation
Folder/Directory descriptions not present

### Describe the issue linked to the documentation

I was navigating through the codebase, trying to find source code for some algorithms. I noticed that there are no descriptions of files present within a folder, which would actually make it easier to navigate through the codebase. We can have a small readme file within folders which would describe what is present in that folder. 

### Suggest a potential alternative/fix

_No response_

COMMENT:
Yeah exactly, that was the whole point. I am ready to work on this issue, if someone assigns me this

Labels: Documentation
Folder/Directory descriptions not present

### Describe the issue linked to the documentation

I was navigating through the codebase, trying to find source code for some algorithms. I noticed that there are no descriptions of files present within a folder, which would actually make it easier to navigate through the codebase. We can have a small readme file within folders which would describe what is present in that folder. 

### Suggest a potential alternative/fix

_No response_

COMMENT:
The intent behind this sounds great, had similar experience starting up but since I work with scikit it was comparatively easier to navigate.

That said, adding and maintaining separate `README` files across many subdirectories might be a lot of overhead for relatively low impact, especially as the structure evolves. (IMHO)

It would also require a ton of maintenance I guess with every iteration.

Would it make more sense to:
- Add brief docstrings
- Or maybe create a single `CODEBASE_OVERVIEW.md` file at the root level that explains the role of each major submodule like `linear_model`, `ensemble`, `neighbors`, etc. from a "for contributors" lens?

What do you think?

Labels: Documentation
Folder/Directory descriptions not present

### Describe the issue linked to the documentation

I was navigating through the codebase, trying to find source code for some algorithms. I noticed that there are no descriptions of files present within a folder, which would actually make it easier to navigate through the codebase. We can have a small readme file within folders which would describe what is present in that folder. 

### Suggest a potential alternative/fix

_No response_

COMMENT:
Any kind of description about what files in which folders contain what, is doomed to be outdated really fast since we have quite a lot of contributions to the repo merged daily.

However, I don't mind a generic overview of where things are in the `contributing.rst` file. The issue is most people don't read that file and many questions are already answered there.

Note that this is NOT a good first issue. Needs to be written by someone familiar with the codebase.

Maybe @StefanieSenger wants to have a look.

Labels: Documentation
Folder/Directory descriptions not present

### Describe the issue linked to the documentation

I was navigating through the codebase, trying to find source code for some algorithms. I noticed that there are no descriptions of files present within a folder, which would actually make it easier to navigate through the codebase. We can have a small readme file within folders which would describe what is present in that folder. 

### Suggest a potential alternative/fix

_No response_

COMMENT:
There are docstrings at the top of all the files, including the `__init__.py` which give a more generic overview of the directory. Combined with the fact that generally the naming of folders/submodules is pretty good I think we have enough existing documentation.

Labels: Documentation
Folder/Directory descriptions not present

### Describe the issue linked to the documentation

I was navigating through the codebase, trying to find source code for some algorithms. I noticed that there are no descriptions of files present within a folder, which would actually make it easier to navigate through the codebase. We can have a small readme file within folders which would describe what is present in that folder. 

### Suggest a potential alternative/fix

_No response_

COMMENT:
For the record, something I always repeat in our sprints, is along the lines of this answer:


A generic overview of the files could be something like this:

There are a few main categories of files in the repository:

- Documentation: the documentation is generated from 3 different souces:
    - `doc/` which includes all user guides, which are written in ReST (`.rst`) format.
        `sphinx` generates HTML files from these files.
    - `examples/` which includes all examples. This are written as a Python (`.py`)
        file, but all comments inside them are in ReST format. `sphincs-gallery`
        generates `.rst` files from these, which are then processed by `sphinx` to
        generate HTML files.
    - The API docs are generated from the docstrings written inside the codebase itself.

- Implementation: the main part of the codebase is under the `sklearn/` directory.
    Most submodules have their own folders there. The best way to find the implementaiton
    of a class is to look for `class ClassName` in the codebase.

- Tests: each module has its own set of tests located under a `tests` folder in the same
    directory as the module. We use `pytest` to write and run the tests.

- CI: there are different folders in the repository such as `.github` or `.circleci`
    which define our CI pipelines on different platforms. We use CircleCI to generate
    our documentation and Azure and Github actions to run our tests.

- Benchmarks: `asv_benchmarks/` includes a set of benchmarks, which are not touched 
    very often and can be used to benchmark certain changes in the codebase.

Labels: Documentation
Regression error characteristic curve

### Describe the workflow you want to enable

Add more fine-grained diagnostic, similar to ROC or Precision-Recall curves, to regression problems. It appears that this library has a lot of excellent tools for classification, and I believe it would benefit from some additional tools for regression.

### Describe your proposed solution

Compute Regression Error Characteristic (REC) [1] curve - for each error threshold the percentage of samples whose error is below that threshold. This is essentially the CDF of the regression errors. Its function is similar to that of ROC curves - allows comparing performance profiles of regressors beyond just one summary statistic, such as RMSE or MAE.

I already implement a pull-request:
https://github.com/scikit-learn/scikit-learn/pull/31380

Screenshot from the merge request:

![Image](https://github.com/user-attachments/assets/1974e8e7-03da-47c7-adb5-5c75eb24d61e)

If you believe this feature is useful, please help me with reviewing and merging it.

### Describe alternatives you've considered, if relevant

Regression Receiver Operating Characteristic (RROC) curves, proposed [2], which plot over-prediction vs under-prediction, are a different form of diagnostic curves for regression. They may also be useful, but I think we should begin from somewhere, and I belive it's better to begin from REC, both because the paper has more citations, and because it turned out to be very useful for me at work, and I believe it can be similarly useful to other scientists.

### Additional context

**References**
---

[1]: Bi, J. and Bennett, K.P., 2003. Regression error characteristic curves. In Proceedings of the 20th international conference on machine learning (ICML-03) (pp. 43-50).
[2]: Hernández-Orallo, J., 2013. ROC curves for regression. Pattern Recognition, 46(12), pp.3395-3411.

COMMENT:
This is a fantastic idea. thank you for bringing REC curves into the conversation!

Having more nuanced diagnostics for regression models has been a long-standing gap in the standard ML workflow. The analogy to ROC/PR curves is spot-on: one summary metric (e.g., MAE/RMSE) can’t always capture performance tradeoffs, especially across varying error tolerances. The cumulative nature of the REC curve gives a much clearer picture of model robustness in practical contexts.

Also appreciate the inclusion of reference. Bi & Bennett (2003) is a solid foundation. And your submitted PR (#31380) looks very promising from a quick glance. I’d love to see this feature included in future versions of scikit-learn. It could really benefit both academic research and applied ML workflows.

Looking forward to seeing how this progresses! Great work.

Labels: New Feature, Needs Triage
Regression error characteristic curve

### Describe the workflow you want to enable

Add more fine-grained diagnostic, similar to ROC or Precision-Recall curves, to regression problems. It appears that this library has a lot of excellent tools for classification, and I believe it would benefit from some additional tools for regression.

### Describe your proposed solution

Compute Regression Error Characteristic (REC) [1] curve - for each error threshold the percentage of samples whose error is below that threshold. This is essentially the CDF of the regression errors. Its function is similar to that of ROC curves - allows comparing performance profiles of regressors beyond just one summary statistic, such as RMSE or MAE.

I already implement a pull-request:
https://github.com/scikit-learn/scikit-learn/pull/31380

Screenshot from the merge request:

![Image](https://github.com/user-attachments/assets/1974e8e7-03da-47c7-adb5-5c75eb24d61e)

If you believe this feature is useful, please help me with reviewing and merging it.

### Describe alternatives you've considered, if relevant

Regression Receiver Operating Characteristic (RROC) curves, proposed [2], which plot over-prediction vs under-prediction, are a different form of diagnostic curves for regression. They may also be useful, but I think we should begin from somewhere, and I belive it's better to begin from REC, both because the paper has more citations, and because it turned out to be very useful for me at work, and I believe it can be similarly useful to other scientists.

### Additional context

**References**
---

[1]: Bi, J. and Bennett, K.P., 2003. Regression error characteristic curves. In Proceedings of the 20th international conference on machine learning (ICML-03) (pp. 43-50).
[2]: Hernández-Orallo, J., 2013. ROC curves for regression. Pattern Recognition, 46(12), pp.3395-3411.

COMMENT:
@alexshtf Thanks for opening this issue.

I am -1 on this feature for 2 reasons:
- The proposed functionality is very easy to implement:
  ```python
   import matplotlib.pyplot as plt
   from scipy.stats import ecdf
   errors = my_error_function(y_obs, y_pred)
   cdf_errors = ecdf(errors)
   ax = plt.subplot()
   cdf_errors.cdf.plot(ax)
   plt.show()
   ```
- I am not convinced about the insight this generates. A much better diagnostic tool for regression in the CAP curve, see #10003.

Therefore, I am closing this issue. But still, feel free to continue discussion.

Labels: New Feature, Needs Triage
Regression error characteristic curve

### Describe the workflow you want to enable

Add more fine-grained diagnostic, similar to ROC or Precision-Recall curves, to regression problems. It appears that this library has a lot of excellent tools for classification, and I believe it would benefit from some additional tools for regression.

### Describe your proposed solution

Compute Regression Error Characteristic (REC) [1] curve - for each error threshold the percentage of samples whose error is below that threshold. This is essentially the CDF of the regression errors. Its function is similar to that of ROC curves - allows comparing performance profiles of regressors beyond just one summary statistic, such as RMSE or MAE.

I already implement a pull-request:
https://github.com/scikit-learn/scikit-learn/pull/31380

Screenshot from the merge request:

![Image](https://github.com/user-attachments/assets/1974e8e7-03da-47c7-adb5-5c75eb24d61e)

If you believe this feature is useful, please help me with reviewing and merging it.

### Describe alternatives you've considered, if relevant

Regression Receiver Operating Characteristic (RROC) curves, proposed [2], which plot over-prediction vs under-prediction, are a different form of diagnostic curves for regression. They may also be useful, but I think we should begin from somewhere, and I belive it's better to begin from REC, both because the paper has more citations, and because it turned out to be very useful for me at work, and I believe it can be similarly useful to other scientists.

### Additional context

**References**
---

[1]: Bi, J. and Bennett, K.P., 2003. Regression error characteristic curves. In Proceedings of the 20th international conference on machine learning (ICML-03) (pp. 43-50).
[2]: Hernández-Orallo, J., 2013. ROC curves for regression. Pattern Recognition, 46(12), pp.3395-3411.

COMMENT:
@lorentzenchr I believe ease of implementation is not that important - otherwise, why does scikit-learn have `PredictionErrorDisplay` visualization?

So I believe the only remaining issue are the insights. Regressors are used for many downstream tasks, not just "let's predict a value and show it to the user", i.e,. computing bids for ad auctions. In general, more accurate bids generate better revenue, but simple summary metrics don't always help you understand why a certain predictors behaves as it does. So I thought it might be useful and valuable for other people as well.

It appears you disagree on the diagnostic value, but it appears I should first dig deeper into CAP curves and see if they could provide similar value for the same use cases.

Labels: New Feature, Needs Triage