PERF `PairwiseDistancesReductions` subsequent work

`PairwiseDistancesReductions` have been introduced as a hierarchy of Cython classes to implement back-ends of some scikit-learn algorithms.

Initial work was listed in #22587.

:bulb: See [this presentation](https://docs.google.com/presentation/d/1RwX_P9lnsb9_YRZ0cA88l3VoEYhiMndQYoKLLF_0Dv0/edit?usp=sharing) for a better understanding of the design of `PairwiseDistancesReductions`.

:bulb: **Before working on improving performance, one must profile the current execution of algorithms to see if there are any substantial benefits.**

Subsequent possible work includes:

 - [ ] #26983 
 - [ ] #25097
 - [ ] Support `"precomputed"` distances
 - Implement back-end for other `Estimators` or `Transformers`, in particular, introduce a specialised back-end for which would remove the costly sequential portion around the current call to `kneighbors` or `radius_neighbors` for (non-complete list):
 	- [ ] computing the MST from the data matrix (`mst_from_data_matrix`) see https://github.com/scikit-learn/scikit-learn/pull/25656#pullrequestreview-1326267314
 	- [ ] computing the MST linkage ([`mst_linkage_core`](https://github.com/scikit-learn/scikit-learn/blob/2ab1d81e167512719b6272d4ddf286bba5d3d232/sklearn/cluster/_hierarchical_fast.pyx#L428-L506))
 	- [x] `LocalOutlierFactor` ([no further optimization possible](https://github.com/scikit-learn/scikit-learn/pull/26316))
 	- [ ] `KNeighbors*.predict*`
 	   - [x] #24076 by @Micky774
 	   - [ ] #28219 by @OmarManzoor
 	   - [ ] Implement multi-output support (WIP [here](https://github.com/Micky774/scikit-learn/pull/3))
 	   - [ ] Support `"distance"` weighting
 	- [ ] `RadiusNeighbors*.predict*`
 	   - [x] #26828
 	   - [ ] Implement Euclidean specialization
 	   - [ ] Implement multi-output support
 	   - [ ] Support `"distance"` weighting
 	- some part of `KMeans`
 - [ ] Force the use of `PairwiseDistancesReductions` for F-contiguous arrays
   - use a scikit-learn configuration entry to accept and convert F-contiguous arrays to C-contiguous arrays
 - [ ] Release the GIL before calling `parallel_on_{X|Y}` instead of releasing it in those methods
    - :bulb: I once have tried and I don't think this is possible.
 - [ ] Integrate `mimalloc` to have proper memory allocation for multi-threaded implementations:
    - :bulb: Changing the default implementation of `malloc(1)` might come with unexpected changes and might break the whole ecosystem. Also maintaining it might be costly.
 - [ ] Tests improvements ideas:
   - https://github.com/scikit-learn/scikit-learn/pull/27281#issuecomment-1704334881
   - parametrize old tests for public API backed by `PairwiseDistancesReductions`?
   - systematically test public API on combinations of sparse and dense datasets?
 - [ ] Extend this design for a new back-end for [`pairwise_kernels`](https://github.com/scikit-learn/scikit-learn/blob/7d1c31892d34e6456c467c3d538ecb0ec4e01c35/sklearn/metrics/pairwise.py#L2116)-based interfaces

COMMENT:
> > I have actually set up scikit learn on a windows laptop. So I can't get this to work properly. This is the script I am using
> > ```python
> > from sklearn.model_selection import train_test_split
> > from sklearn.neighbors._classification import RadiusNeighborsClassifier
> > from sklearn.datasets import make_classification
> > 
> > 
> > n_features = 10
> > n_samples = 500000
> > n_classes = 100
> > 
> > X, y = make_classification(
> >     n_samples=n_samples,
> >     n_features=n_features,
> >     n_redundant=0,
> >     n_informative=10,
> >     n_classes=n_classes,
> >     random_state=0,
> > )
> > 
> > 
> > X_train, X_test, y_train, _ = train_test_split(X, y, random_state=0)
> > clf = RadiusNeighborsClassifier(radius=10, algorithm="brute")
> > clf.fit(X_train, y_train)
> > proba = clf.predict_proba(X_test)
> > ```
> > 
> > 
> >     
> >       
> >     
> > 
> >       
> >     
> > 
> >     
> >   
> > And this is the command I am trying to run py-spy with
> > ```
> > Invoke-Command -ScriptBlock {cmd /c start /affinity 1 py-spy record --native -o py-spy.profile -f speedscope -- python run_radius_neighbor
> > s.py}
> > ```
> > 
> > 
> >     
> >       
> >     
> > 
> >       
> >     
> > 
> >     
> >   
> > Do you have an idea whether this looks correct? I keep getting errors of the form below where the seconds behind keeps on increasing
> > ```
> > py-spy> 89.90s behind in sampling, results may be inaccurate. Try reducing the sampling rate
> > ```
> 
> A couple notes:
> 
> 1. I also use a windows machine for scikit-learn development, but I **highly** reccommend using the Windows Subsystems Linux ([WSL2](https://learn.microsoft.com/en-us/windows/wsl/install)). It gives you a full and proper linux environment that you can use with ease. Windows tends to be an afterthought for many useful tools and utilities, and lacks streamlined support for important development utilities (as you have experienced).
> 2. Try changing `n_samples` to something much smaller. Start with, maybe, `n_samples=20_000`. Using `500_000` is way too much I think.

Thank you for the clarification.

Labels: Performance, cython, Meta-issue