Talip7/scikit-learn-issues-multilabel huggingface dataset:
PERF `PairwiseDistancesReductions` subsequent work

`PairwiseDistancesReductions` have been introduced as a hierarchy of Cython classes to implement back-ends of some scikit-learn algorithms.

Initial work was listed in #22587.

:bulb: See [this presentation](https://docs.google.com/presentation/d/1RwX_P9lnsb9_YRZ0cA88l3VoEYhiMndQYoKLLF_0Dv0/edit?usp=sharing) for a better understanding of the design of `PairwiseDistancesReductions`.

:bulb: **Before working on improving performance, one must profile the current execution of algorithms to see if there are any substantial benefits.**

Subsequent possible work includes:

 - [ ] #26983 
 - [ ] #25097
 - [ ] Support `"precomputed"` distances
 - Implement back-end for other `Estimators` or `Transformers`, in particular, introduce a specialised back-end for which would remove the costly sequential portion around the current call to `kneighbors` or `radius_neighbors` for (non-complete list):
 	- [ ] computing the MST from the data matrix (`mst_from_data_matrix`) see https://github.com/scikit-learn/scikit-learn/pull/25656#pullrequestreview-1326267314
 	- [ ] computing the MST linkage ([`mst_linkage_core`](https://github.com/scikit-learn/scikit-learn/blob/2ab1d81e167512719b6272d4ddf286bba5d3d232/sklearn/cluster/_hierarchical_fast.pyx#L428-L506))
 	- [x] `LocalOutlierFactor` ([no further optimization possible](https://github.com/scikit-learn/scikit-learn/pull/26316))
 	- [ ] `KNeighbors*.predict*`
 	   - [x] #24076 by @Micky774
 	   - [ ] #28219 by @OmarManzoor
 	   - [ ] Implement multi-output support (WIP [here](https://github.com/Micky774/scikit-learn/pull/3))
 	   - [ ] Support `"distance"` weighting
 	- [ ] `RadiusNeighbors*.predict*`
 	   - [x] #26828
 	   - [ ] Implement Euclidean specialization
 	   - [ ] Implement multi-output support
 	   - [ ] Support `"distance"` weighting
 	- some part of `KMeans`
 - [ ] Force the use of `PairwiseDistancesReductions` for F-contiguous arrays
   - use a scikit-learn configuration entry to accept and convert F-contiguous arrays to C-contiguous arrays
 - [ ] Release the GIL before calling `parallel_on_{X|Y}` instead of releasing it in those methods
    - :bulb: I once have tried and I don't think this is possible.
 - [ ] Integrate `mimalloc` to have proper memory allocation for multi-threaded implementations:
    - :bulb: Changing the default implementation of `malloc(1)` might come with unexpected changes and might break the whole ecosystem. Also maintaining it might be costly.
 - [ ] Tests improvements ideas:
   - https://github.com/scikit-learn/scikit-learn/pull/27281#issuecomment-1704334881
   - parametrize old tests for public API backed by `PairwiseDistancesReductions`?
   - systematically test public API on combinations of sparse and dense datasets?
 - [ ] Extend this design for a new back-end for [`pairwise_kernels`](https://github.com/scikit-learn/scikit-learn/blob/7d1c31892d34e6456c467c3d538ecb0ec4e01c35/sklearn/metrics/pairwise.py#L2116)-based interfaces

COMMENT:
@OmarManzoor: you also might be interested in items of this issue. 🙂

Labels: Performance, cython, Meta-issue
PERF `PairwiseDistancesReductions` subsequent work

`PairwiseDistancesReductions` have been introduced as a hierarchy of Cython classes to implement back-ends of some scikit-learn algorithms.

Initial work was listed in #22587.

:bulb: See [this presentation](https://docs.google.com/presentation/d/1RwX_P9lnsb9_YRZ0cA88l3VoEYhiMndQYoKLLF_0Dv0/edit?usp=sharing) for a better understanding of the design of `PairwiseDistancesReductions`.

:bulb: **Before working on improving performance, one must profile the current execution of algorithms to see if there are any substantial benefits.**

Subsequent possible work includes:

 - [ ] #26983 
 - [ ] #25097
 - [ ] Support `"precomputed"` distances
 - Implement back-end for other `Estimators` or `Transformers`, in particular, introduce a specialised back-end for which would remove the costly sequential portion around the current call to `kneighbors` or `radius_neighbors` for (non-complete list):
 	- [ ] computing the MST from the data matrix (`mst_from_data_matrix`) see https://github.com/scikit-learn/scikit-learn/pull/25656#pullrequestreview-1326267314
 	- [ ] computing the MST linkage ([`mst_linkage_core`](https://github.com/scikit-learn/scikit-learn/blob/2ab1d81e167512719b6272d4ddf286bba5d3d232/sklearn/cluster/_hierarchical_fast.pyx#L428-L506))
 	- [x] `LocalOutlierFactor` ([no further optimization possible](https://github.com/scikit-learn/scikit-learn/pull/26316))
 	- [ ] `KNeighbors*.predict*`
 	   - [x] #24076 by @Micky774
 	   - [ ] #28219 by @OmarManzoor
 	   - [ ] Implement multi-output support (WIP [here](https://github.com/Micky774/scikit-learn/pull/3))
 	   - [ ] Support `"distance"` weighting
 	- [ ] `RadiusNeighbors*.predict*`
 	   - [x] #26828
 	   - [ ] Implement Euclidean specialization
 	   - [ ] Implement multi-output support
 	   - [ ] Support `"distance"` weighting
 	- some part of `KMeans`
 - [ ] Force the use of `PairwiseDistancesReductions` for F-contiguous arrays
   - use a scikit-learn configuration entry to accept and convert F-contiguous arrays to C-contiguous arrays
 - [ ] Release the GIL before calling `parallel_on_{X|Y}` instead of releasing it in those methods
    - :bulb: I once have tried and I don't think this is possible.
 - [ ] Integrate `mimalloc` to have proper memory allocation for multi-threaded implementations:
    - :bulb: Changing the default implementation of `malloc(1)` might come with unexpected changes and might break the whole ecosystem. Also maintaining it might be costly.
 - [ ] Tests improvements ideas:
   - https://github.com/scikit-learn/scikit-learn/pull/27281#issuecomment-1704334881
   - parametrize old tests for public API backed by `PairwiseDistancesReductions`?
   - systematically test public API on combinations of sparse and dense datasets?
 - [ ] Extend this design for a new back-end for [`pairwise_kernels`](https://github.com/scikit-learn/scikit-learn/blob/7d1c31892d34e6456c467c3d538ecb0ec4e01c35/sklearn/metrics/pairwise.py#L2116)-based interfaces

COMMENT:
> @OmarManzoor: you also might be interested in items of this issue. 🙂

Nice. Thanks!

Labels: Performance, cython, Meta-issue
PERF `PairwiseDistancesReductions` subsequent work

`PairwiseDistancesReductions` have been introduced as a hierarchy of Cython classes to implement back-ends of some scikit-learn algorithms.

Initial work was listed in #22587.

:bulb: See [this presentation](https://docs.google.com/presentation/d/1RwX_P9lnsb9_YRZ0cA88l3VoEYhiMndQYoKLLF_0Dv0/edit?usp=sharing) for a better understanding of the design of `PairwiseDistancesReductions`.

:bulb: **Before working on improving performance, one must profile the current execution of algorithms to see if there are any substantial benefits.**

Subsequent possible work includes:

 - [ ] #26983 
 - [ ] #25097
 - [ ] Support `"precomputed"` distances
 - Implement back-end for other `Estimators` or `Transformers`, in particular, introduce a specialised back-end for which would remove the costly sequential portion around the current call to `kneighbors` or `radius_neighbors` for (non-complete list):
 	- [ ] computing the MST from the data matrix (`mst_from_data_matrix`) see https://github.com/scikit-learn/scikit-learn/pull/25656#pullrequestreview-1326267314
 	- [ ] computing the MST linkage ([`mst_linkage_core`](https://github.com/scikit-learn/scikit-learn/blob/2ab1d81e167512719b6272d4ddf286bba5d3d232/sklearn/cluster/_hierarchical_fast.pyx#L428-L506))
 	- [x] `LocalOutlierFactor` ([no further optimization possible](https://github.com/scikit-learn/scikit-learn/pull/26316))
 	- [ ] `KNeighbors*.predict*`
 	   - [x] #24076 by @Micky774
 	   - [ ] #28219 by @OmarManzoor
 	   - [ ] Implement multi-output support (WIP [here](https://github.com/Micky774/scikit-learn/pull/3))
 	   - [ ] Support `"distance"` weighting
 	- [ ] `RadiusNeighbors*.predict*`
 	   - [x] #26828
 	   - [ ] Implement Euclidean specialization
 	   - [ ] Implement multi-output support
 	   - [ ] Support `"distance"` weighting
 	- some part of `KMeans`
 - [ ] Force the use of `PairwiseDistancesReductions` for F-contiguous arrays
   - use a scikit-learn configuration entry to accept and convert F-contiguous arrays to C-contiguous arrays
 - [ ] Release the GIL before calling `parallel_on_{X|Y}` instead of releasing it in those methods
    - :bulb: I once have tried and I don't think this is possible.
 - [ ] Integrate `mimalloc` to have proper memory allocation for multi-threaded implementations:
    - :bulb: Changing the default implementation of `malloc(1)` might come with unexpected changes and might break the whole ecosystem. Also maintaining it might be costly.
 - [ ] Tests improvements ideas:
   - https://github.com/scikit-learn/scikit-learn/pull/27281#issuecomment-1704334881
   - parametrize old tests for public API backed by `PairwiseDistancesReductions`?
   - systematically test public API on combinations of sparse and dense datasets?
 - [ ] Extend this design for a new back-end for [`pairwise_kernels`](https://github.com/scikit-learn/scikit-learn/blob/7d1c31892d34e6456c467c3d538ecb0ec4e01c35/sklearn/metrics/pairwise.py#L2116)-based interfaces

COMMENT:
> * [ ]  Support `"precomputed"` distances

Looking into this...

Labels: Performance, cython, Meta-issue
PERF `PairwiseDistancesReductions` subsequent work

`PairwiseDistancesReductions` have been introduced as a hierarchy of Cython classes to implement back-ends of some scikit-learn algorithms.

Initial work was listed in #22587.

:bulb: See [this presentation](https://docs.google.com/presentation/d/1RwX_P9lnsb9_YRZ0cA88l3VoEYhiMndQYoKLLF_0Dv0/edit?usp=sharing) for a better understanding of the design of `PairwiseDistancesReductions`.

:bulb: **Before working on improving performance, one must profile the current execution of algorithms to see if there are any substantial benefits.**

Subsequent possible work includes:

 - [ ] #26983 
 - [ ] #25097
 - [ ] Support `"precomputed"` distances
 - Implement back-end for other `Estimators` or `Transformers`, in particular, introduce a specialised back-end for which would remove the costly sequential portion around the current call to `kneighbors` or `radius_neighbors` for (non-complete list):
 	- [ ] computing the MST from the data matrix (`mst_from_data_matrix`) see https://github.com/scikit-learn/scikit-learn/pull/25656#pullrequestreview-1326267314
 	- [ ] computing the MST linkage ([`mst_linkage_core`](https://github.com/scikit-learn/scikit-learn/blob/2ab1d81e167512719b6272d4ddf286bba5d3d232/sklearn/cluster/_hierarchical_fast.pyx#L428-L506))
 	- [x] `LocalOutlierFactor` ([no further optimization possible](https://github.com/scikit-learn/scikit-learn/pull/26316))
 	- [ ] `KNeighbors*.predict*`
 	   - [x] #24076 by @Micky774
 	   - [ ] #28219 by @OmarManzoor
 	   - [ ] Implement multi-output support (WIP [here](https://github.com/Micky774/scikit-learn/pull/3))
 	   - [ ] Support `"distance"` weighting
 	- [ ] `RadiusNeighbors*.predict*`
 	   - [x] #26828
 	   - [ ] Implement Euclidean specialization
 	   - [ ] Implement multi-output support
 	   - [ ] Support `"distance"` weighting
 	- some part of `KMeans`
 - [ ] Force the use of `PairwiseDistancesReductions` for F-contiguous arrays
   - use a scikit-learn configuration entry to accept and convert F-contiguous arrays to C-contiguous arrays
 - [ ] Release the GIL before calling `parallel_on_{X|Y}` instead of releasing it in those methods
    - :bulb: I once have tried and I don't think this is possible.
 - [ ] Integrate `mimalloc` to have proper memory allocation for multi-threaded implementations:
    - :bulb: Changing the default implementation of `malloc(1)` might come with unexpected changes and might break the whole ecosystem. Also maintaining it might be costly.
 - [ ] Tests improvements ideas:
   - https://github.com/scikit-learn/scikit-learn/pull/27281#issuecomment-1704334881
   - parametrize old tests for public API backed by `PairwiseDistancesReductions`?
   - systematically test public API on combinations of sparse and dense datasets?
 - [ ] Extend this design for a new back-end for [`pairwise_kernels`](https://github.com/scikit-learn/scikit-learn/blob/7d1c31892d34e6456c467c3d538ecb0ec4e01c35/sklearn/metrics/pairwise.py#L2116)-based interfaces

COMMENT:
You are welcome.

Labels: Performance, cython, Meta-issue
PERF `PairwiseDistancesReductions` subsequent work

`PairwiseDistancesReductions` have been introduced as a hierarchy of Cython classes to implement back-ends of some scikit-learn algorithms.

Initial work was listed in #22587.

:bulb: See [this presentation](https://docs.google.com/presentation/d/1RwX_P9lnsb9_YRZ0cA88l3VoEYhiMndQYoKLLF_0Dv0/edit?usp=sharing) for a better understanding of the design of `PairwiseDistancesReductions`.

:bulb: **Before working on improving performance, one must profile the current execution of algorithms to see if there are any substantial benefits.**

Subsequent possible work includes:

 - [ ] #26983 
 - [ ] #25097
 - [ ] Support `"precomputed"` distances
 - Implement back-end for other `Estimators` or `Transformers`, in particular, introduce a specialised back-end for which would remove the costly sequential portion around the current call to `kneighbors` or `radius_neighbors` for (non-complete list):
 	- [ ] computing the MST from the data matrix (`mst_from_data_matrix`) see https://github.com/scikit-learn/scikit-learn/pull/25656#pullrequestreview-1326267314
 	- [ ] computing the MST linkage ([`mst_linkage_core`](https://github.com/scikit-learn/scikit-learn/blob/2ab1d81e167512719b6272d4ddf286bba5d3d232/sklearn/cluster/_hierarchical_fast.pyx#L428-L506))
 	- [x] `LocalOutlierFactor` ([no further optimization possible](https://github.com/scikit-learn/scikit-learn/pull/26316))
 	- [ ] `KNeighbors*.predict*`
 	   - [x] #24076 by @Micky774
 	   - [ ] #28219 by @OmarManzoor
 	   - [ ] Implement multi-output support (WIP [here](https://github.com/Micky774/scikit-learn/pull/3))
 	   - [ ] Support `"distance"` weighting
 	- [ ] `RadiusNeighbors*.predict*`
 	   - [x] #26828
 	   - [ ] Implement Euclidean specialization
 	   - [ ] Implement multi-output support
 	   - [ ] Support `"distance"` weighting
 	- some part of `KMeans`
 - [ ] Force the use of `PairwiseDistancesReductions` for F-contiguous arrays
   - use a scikit-learn configuration entry to accept and convert F-contiguous arrays to C-contiguous arrays
 - [ ] Release the GIL before calling `parallel_on_{X|Y}` instead of releasing it in those methods
    - :bulb: I once have tried and I don't think this is possible.
 - [ ] Integrate `mimalloc` to have proper memory allocation for multi-threaded implementations:
    - :bulb: Changing the default implementation of `malloc(1)` might come with unexpected changes and might break the whole ecosystem. Also maintaining it might be costly.
 - [ ] Tests improvements ideas:
   - https://github.com/scikit-learn/scikit-learn/pull/27281#issuecomment-1704334881
   - parametrize old tests for public API backed by `PairwiseDistancesReductions`?
   - systematically test public API on combinations of sparse and dense datasets?
 - [ ] Extend this design for a new back-end for [`pairwise_kernels`](https://github.com/scikit-learn/scikit-learn/blob/7d1c31892d34e6456c467c3d538ecb0ec4e01c35/sklearn/metrics/pairwise.py#L2116)-based interfaces

COMMENT:
> You are welcome.

This looks interesting but I might need time to understand it properly.

Labels: Performance, cython, Meta-issue
PERF `PairwiseDistancesReductions` subsequent work

`PairwiseDistancesReductions` have been introduced as a hierarchy of Cython classes to implement back-ends of some scikit-learn algorithms.

Initial work was listed in #22587.

:bulb: See [this presentation](https://docs.google.com/presentation/d/1RwX_P9lnsb9_YRZ0cA88l3VoEYhiMndQYoKLLF_0Dv0/edit?usp=sharing) for a better understanding of the design of `PairwiseDistancesReductions`.

:bulb: **Before working on improving performance, one must profile the current execution of algorithms to see if there are any substantial benefits.**

Subsequent possible work includes:

 - [ ] #26983 
 - [ ] #25097
 - [ ] Support `"precomputed"` distances
 - Implement back-end for other `Estimators` or `Transformers`, in particular, introduce a specialised back-end for which would remove the costly sequential portion around the current call to `kneighbors` or `radius_neighbors` for (non-complete list):
 	- [ ] computing the MST from the data matrix (`mst_from_data_matrix`) see https://github.com/scikit-learn/scikit-learn/pull/25656#pullrequestreview-1326267314
 	- [ ] computing the MST linkage ([`mst_linkage_core`](https://github.com/scikit-learn/scikit-learn/blob/2ab1d81e167512719b6272d4ddf286bba5d3d232/sklearn/cluster/_hierarchical_fast.pyx#L428-L506))
 	- [x] `LocalOutlierFactor` ([no further optimization possible](https://github.com/scikit-learn/scikit-learn/pull/26316))
 	- [ ] `KNeighbors*.predict*`
 	   - [x] #24076 by @Micky774
 	   - [ ] #28219 by @OmarManzoor
 	   - [ ] Implement multi-output support (WIP [here](https://github.com/Micky774/scikit-learn/pull/3))
 	   - [ ] Support `"distance"` weighting
 	- [ ] `RadiusNeighbors*.predict*`
 	   - [x] #26828
 	   - [ ] Implement Euclidean specialization
 	   - [ ] Implement multi-output support
 	   - [ ] Support `"distance"` weighting
 	- some part of `KMeans`
 - [ ] Force the use of `PairwiseDistancesReductions` for F-contiguous arrays
   - use a scikit-learn configuration entry to accept and convert F-contiguous arrays to C-contiguous arrays
 - [ ] Release the GIL before calling `parallel_on_{X|Y}` instead of releasing it in those methods
    - :bulb: I once have tried and I don't think this is possible.
 - [ ] Integrate `mimalloc` to have proper memory allocation for multi-threaded implementations:
    - :bulb: Changing the default implementation of `malloc(1)` might come with unexpected changes and might break the whole ecosystem. Also maintaining it might be costly.
 - [ ] Tests improvements ideas:
   - https://github.com/scikit-learn/scikit-learn/pull/27281#issuecomment-1704334881
   - parametrize old tests for public API backed by `PairwiseDistancesReductions`?
   - systematically test public API on combinations of sparse and dense datasets?
 - [ ] Extend this design for a new back-end for [`pairwise_kernels`](https://github.com/scikit-learn/scikit-learn/blob/7d1c31892d34e6456c467c3d538ecb0ec4e01c35/sklearn/metrics/pairwise.py#L2116)-based interfaces

COMMENT:
Yes, let's probably finish https://github.com/scikit-learn/scikit-learn/pull/25914 first.

Labels: Performance, cython, Meta-issue
PERF `PairwiseDistancesReductions` subsequent work

`PairwiseDistancesReductions` have been introduced as a hierarchy of Cython classes to implement back-ends of some scikit-learn algorithms.

Initial work was listed in #22587.

:bulb: See [this presentation](https://docs.google.com/presentation/d/1RwX_P9lnsb9_YRZ0cA88l3VoEYhiMndQYoKLLF_0Dv0/edit?usp=sharing) for a better understanding of the design of `PairwiseDistancesReductions`.

:bulb: **Before working on improving performance, one must profile the current execution of algorithms to see if there are any substantial benefits.**

Subsequent possible work includes:

 - [ ] #26983 
 - [ ] #25097
 - [ ] Support `"precomputed"` distances
 - Implement back-end for other `Estimators` or `Transformers`, in particular, introduce a specialised back-end for which would remove the costly sequential portion around the current call to `kneighbors` or `radius_neighbors` for (non-complete list):
 	- [ ] computing the MST from the data matrix (`mst_from_data_matrix`) see https://github.com/scikit-learn/scikit-learn/pull/25656#pullrequestreview-1326267314
 	- [ ] computing the MST linkage ([`mst_linkage_core`](https://github.com/scikit-learn/scikit-learn/blob/2ab1d81e167512719b6272d4ddf286bba5d3d232/sklearn/cluster/_hierarchical_fast.pyx#L428-L506))
 	- [x] `LocalOutlierFactor` ([no further optimization possible](https://github.com/scikit-learn/scikit-learn/pull/26316))
 	- [ ] `KNeighbors*.predict*`
 	   - [x] #24076 by @Micky774
 	   - [ ] #28219 by @OmarManzoor
 	   - [ ] Implement multi-output support (WIP [here](https://github.com/Micky774/scikit-learn/pull/3))
 	   - [ ] Support `"distance"` weighting
 	- [ ] `RadiusNeighbors*.predict*`
 	   - [x] #26828
 	   - [ ] Implement Euclidean specialization
 	   - [ ] Implement multi-output support
 	   - [ ] Support `"distance"` weighting
 	- some part of `KMeans`
 - [ ] Force the use of `PairwiseDistancesReductions` for F-contiguous arrays
   - use a scikit-learn configuration entry to accept and convert F-contiguous arrays to C-contiguous arrays
 - [ ] Release the GIL before calling `parallel_on_{X|Y}` instead of releasing it in those methods
    - :bulb: I once have tried and I don't think this is possible.
 - [ ] Integrate `mimalloc` to have proper memory allocation for multi-threaded implementations:
    - :bulb: Changing the default implementation of `malloc(1)` might come with unexpected changes and might break the whole ecosystem. Also maintaining it might be costly.
 - [ ] Tests improvements ideas:
   - https://github.com/scikit-learn/scikit-learn/pull/27281#issuecomment-1704334881
   - parametrize old tests for public API backed by `PairwiseDistancesReductions`?
   - systematically test public API on combinations of sparse and dense datasets?
 - [ ] Extend this design for a new back-end for [`pairwise_kernels`](https://github.com/scikit-learn/scikit-learn/blob/7d1c31892d34e6456c467c3d538ecb0ec4e01c35/sklearn/metrics/pairwise.py#L2116)-based interfaces

COMMENT:
I would run some profiling ahead of implementations, yes.

Labels: Performance, cython, Meta-issue