Talip7/scikit-learn-issues-multilabel huggingface dataset:
Introduce SIMD intrinsics for `_dist_metrics.pyx`

# Context
Pairwise distance computation is an essential part of many estimators in scikit-learn, and can take up a significant portion of run time in certain workflows. I believe that we may achieve significant performance gains in several (perhaps most) distance metric implementations by leveraging SIMD intrinsics.

# Proof of Concept

I built a quick proof of concept just to see what kinds of performance gains we could observe with a potentially-naive implementation of SIMD intrinsics. I chose to optimize the `ManhattanDistance.dist` function. This implementation uses intrinsics found in `SSE{1,2,3}`. To ensure that the instructions are supported, it checks for the presence of the `SSE3`  instruction set (`SSE3` implies `SSE{1,2}`) and provides the optimized implementation if so. Otherwise it provides a dummy implementation just to appease Cython, and the main function falls back to the current implementation on `main`. Note that on most modern hardware, support for `SSE3` is a reasonable expectation (indeed numpy assumes it is always present when optimization is enabled). For the specific implementation referred to here, please take a look at this PR: https://github.com/Micky774/scikit-learn/pull/11

Note that the full benefit of the intrinsics are gained when compiling with `-march="native"`, however the benefit is still significant when compiling with `-march="nocona"`, as is often default (e.g when following the scikit-learn development instructions on linux).

# Benchmarks
The following benchmarks were produced by this gist: https://gist.github.com/Micky774/567a5fa199c05d90c4c08625b077840e

### **Summary: The SIMD implementations are ~2x faster than the current implementation for `float32` and 1.5x faster for `float64`.**

<details>
<summary>Plots</summary>

![f2b1f1e8-59b0-4ec5-b91c-fe1d19abd9ec](https://user-images.githubusercontent.com/34613774/228374474-99ee13b3-228d-4c53-9dcc-3a8f8b639a9f.png)
</details>

# Discussion
I haven't looked too deeply into this yet, as first I wanted to see whether there was interest in the venture. I would love to hear what the other maintainers' thoughts are regarding exploring this route in a bit more detail. Obviously SIMD implementations will bring with them added complexity, but the performance gains are pretty compelling. In my opinion, the tradeoff is worth it.

CC: @scikit-learn/core-devs

COMMENT:
IIRC, the (implicit?) consensus on the mailing list was to first assess the value of efficient non-euclidean distance metrics computations for end-users.

Labels: Enhancement, Needs Decision, cython
Customizing the order of values in `classes_` of LabelEncoder gives weird results

### Describe the bug

I want to customize `LabelEncoder` to encode 3 and 1 into 0 and 1, respectively. I manually set the `classes_` as `np.array([3, 1])`, but it doesn't work. I checked the source code and found that `classes_` seems the only attribute in `LabelEncoder`. It is confusing why controlling `classes_` can't make it work.

### Steps/Code to Reproduce

```
le = LabelEncoder()
le.fit([3, 1])
print(le.classes_)
le.classes_ = np.array([3,1])
print(le.classes_)
print(le.transform([1]))
print(le.transform([3]))
```

### Expected Results

```
array([1, 3])
array([3, 1])
array([1])
array([0])
```

### Actual Results

```
array([1, 3])
array([3, 1])
array([0])
array([2])
```

### Versions

```shell
System:
    python: 3.8.16 (default, Jan 17 2023, 23:13:24)  [GCC 11.2.0]
executable: /home/ubuntu/anaconda3/envs/ag-dev-2/bin/python
   machine: Linux-5.4.0-1084-aws-x86_64-with-glibc2.17

Python dependencies:
      sklearn: 1.1.1
          pip: 22.3.1
   setuptools: 65.6.3
        numpy: 1.23.5
        scipy: 1.10.0
       Cython: 0.29.33
       pandas: 1.5.3
   matplotlib: 3.6.3
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/ubuntu/anaconda3/envs/ag-dev-2/lib/python3.8/site-packages/torch/lib/libgomp-a34b3233.so.1
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /home/ubuntu/anaconda3/envs/ag-dev-2/lib/python3.8/site-packages/numpy.libs/libopenblas64_p-r0-742d56dc.3.20.so
        version: 0.3.20
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 16

       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/ubuntu/anaconda3/envs/ag-dev-2/lib/python3.8/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 16

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /home/ubuntu/anaconda3/envs/ag-dev-2/lib/python3.8/site-packages/scipy.libs/libopenblasp-r0-41284840.3.18.so
        version: 0.3.18
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 16
```

COMMENT:
We don't support reordering `classes_`. The reason that it fails it is that we expect `classes_` to be ordered and take profit from it by calling `np.searchsorted` which is then much more efficient that mapping an unordered array.

Labels: Bug, Needs Triage
Add documentation about re-building extensions to "Installing the development version of scikit-learn"

### Describe the issue linked to the documentation

Quite regularly when contributing to scikit-learn I realise that I need to re-build the Cython extensions. I realise this when I run pytest and get an import error. When I started contributing, I didn't know about this and I would have been rather lost when I faced this error the first time if @adrinjalali didn't tell me how to fix this. As many people will face the same issue (especially people from underrepresented groups in tech because they're often career changers), a note about re-building the extensions should be put in the documentation. The pandas team for example now recommends to their contributors to do this after each fetch and merge from `upstream/main`: https://pandas.pydata.org/docs/dev/development/contributing_environment.html#step-3-build-and-install-pandas

### Suggest a potential alternative/fix

I think the best place to make a note about this is here in the "Note" box under point 7: https://scikit-learn.org/dev/developers/advanced_installation.html#building-from-source
Similar to the pandas documentation, it could be recommended here to run `make in` after each fetch and merge (or pull) from `upstream/main`. Additionally, it could be recommended to run `make clean` in case people run into unexpected errors after pulling from `upstream/main`.

COMMENT:
Indeed, it could be great to have this inside the documentation. We cannot recommend `make in` because it will only work in Linux and MacOS because this is a `MakeFile`. However, we can provide the full command which is at the top of my mind `python setup.py build_ext -i`

Labels: Documentation
Add documentation about re-building extensions to "Installing the development version of scikit-learn"

### Describe the issue linked to the documentation

Quite regularly when contributing to scikit-learn I realise that I need to re-build the Cython extensions. I realise this when I run pytest and get an import error. When I started contributing, I didn't know about this and I would have been rather lost when I faced this error the first time if @adrinjalali didn't tell me how to fix this. As many people will face the same issue (especially people from underrepresented groups in tech because they're often career changers), a note about re-building the extensions should be put in the documentation. The pandas team for example now recommends to their contributors to do this after each fetch and merge from `upstream/main`: https://pandas.pydata.org/docs/dev/development/contributing_environment.html#step-3-build-and-install-pandas

### Suggest a potential alternative/fix

I think the best place to make a note about this is here in the "Note" box under point 7: https://scikit-learn.org/dev/developers/advanced_installation.html#building-from-source
Similar to the pandas documentation, it could be recommended here to run `make in` after each fetch and merge (or pull) from `upstream/main`. Additionally, it could be recommended to run `make clean` in case people run into unexpected errors after pulling from `upstream/main`.

COMMENT:
we can (and I think should) still tell people about `main in` if they're on a posix system. In the last sprint we didn't manage to setup windows users' systems anyway. Ended up using WSL, which would also have `make`.

Since this is mostly encountered when users run `pytest`, I wonder if we can add a test or a plugin or something, which would check if the user needs to recompile something, and if yes, give them a nice message instead of an import error.

@BenjaminBossan would know more about `pytest` stuff than I do.

Labels: Documentation
Add documentation about re-building extensions to "Installing the development version of scikit-learn"

### Describe the issue linked to the documentation

Quite regularly when contributing to scikit-learn I realise that I need to re-build the Cython extensions. I realise this when I run pytest and get an import error. When I started contributing, I didn't know about this and I would have been rather lost when I faced this error the first time if @adrinjalali didn't tell me how to fix this. As many people will face the same issue (especially people from underrepresented groups in tech because they're often career changers), a note about re-building the extensions should be put in the documentation. The pandas team for example now recommends to their contributors to do this after each fetch and merge from `upstream/main`: https://pandas.pydata.org/docs/dev/development/contributing_environment.html#step-3-build-and-install-pandas

### Suggest a potential alternative/fix

I think the best place to make a note about this is here in the "Note" box under point 7: https://scikit-learn.org/dev/developers/advanced_installation.html#building-from-source
Similar to the pandas documentation, it could be recommended here to run `make in` after each fetch and merge (or pull) from `upstream/main`. Additionally, it could be recommended to run `make clean` in case people run into unexpected errors after pulling from `upstream/main`.

COMMENT:
Hi there, I'm interested in contributing to this project by adding the documentation about re-building extensions to "Installing the development version of scikit-learn". I think it's an important piece of information for new contributors and I'd like to help make it easier for them to get started. Let me know if I can be of any assistance!
Aryan Mishra

Labels: Documentation
Add documentation about re-building extensions to "Installing the development version of scikit-learn"

### Describe the issue linked to the documentation

Quite regularly when contributing to scikit-learn I realise that I need to re-build the Cython extensions. I realise this when I run pytest and get an import error. When I started contributing, I didn't know about this and I would have been rather lost when I faced this error the first time if @adrinjalali didn't tell me how to fix this. As many people will face the same issue (especially people from underrepresented groups in tech because they're often career changers), a note about re-building the extensions should be put in the documentation. The pandas team for example now recommends to their contributors to do this after each fetch and merge from `upstream/main`: https://pandas.pydata.org/docs/dev/development/contributing_environment.html#step-3-build-and-install-pandas

### Suggest a potential alternative/fix

I think the best place to make a note about this is here in the "Note" box under point 7: https://scikit-learn.org/dev/developers/advanced_installation.html#building-from-source
Similar to the pandas documentation, it could be recommended here to run `make in` after each fetch and merge (or pull) from `upstream/main`. Additionally, it could be recommended to run `make clean` in case people run into unexpected errors after pulling from `upstream/main`.

COMMENT:
@Aryan-Mishra24 thanks, but we haven't decided how to proceed here yet I think.

Labels: Documentation
Add documentation about re-building extensions to "Installing the development version of scikit-learn"

### Describe the issue linked to the documentation

Quite regularly when contributing to scikit-learn I realise that I need to re-build the Cython extensions. I realise this when I run pytest and get an import error. When I started contributing, I didn't know about this and I would have been rather lost when I faced this error the first time if @adrinjalali didn't tell me how to fix this. As many people will face the same issue (especially people from underrepresented groups in tech because they're often career changers), a note about re-building the extensions should be put in the documentation. The pandas team for example now recommends to their contributors to do this after each fetch and merge from `upstream/main`: https://pandas.pydata.org/docs/dev/development/contributing_environment.html#step-3-build-and-install-pandas

### Suggest a potential alternative/fix

I think the best place to make a note about this is here in the "Note" box under point 7: https://scikit-learn.org/dev/developers/advanced_installation.html#building-from-source
Similar to the pandas documentation, it could be recommended here to run `make in` after each fetch and merge (or pull) from `upstream/main`. Additionally, it could be recommended to run `make clean` in case people run into unexpected errors after pulling from `upstream/main`.

COMMENT:
A test or plug-in for telling users to recompile when this is needed is a good idea. However, I think it would also be helpful to have this information included in the documentation. 

@Aryan-Mishra24 thanks for your interest in contributing to this issue! As @adrinjalali said, a decision about how to proceed still needs to be made. If you would like to already start contributing to scikit-learn I can recommend issue #24862.

Labels: Documentation
Add documentation about re-building extensions to "Installing the development version of scikit-learn"

### Describe the issue linked to the documentation

Quite regularly when contributing to scikit-learn I realise that I need to re-build the Cython extensions. I realise this when I run pytest and get an import error. When I started contributing, I didn't know about this and I would have been rather lost when I faced this error the first time if @adrinjalali didn't tell me how to fix this. As many people will face the same issue (especially people from underrepresented groups in tech because they're often career changers), a note about re-building the extensions should be put in the documentation. The pandas team for example now recommends to their contributors to do this after each fetch and merge from `upstream/main`: https://pandas.pydata.org/docs/dev/development/contributing_environment.html#step-3-build-and-install-pandas

### Suggest a potential alternative/fix

I think the best place to make a note about this is here in the "Note" box under point 7: https://scikit-learn.org/dev/developers/advanced_installation.html#building-from-source
Similar to the pandas documentation, it could be recommended here to run `make in` after each fetch and merge (or pull) from `upstream/main`. Additionally, it could be recommended to run `make clean` in case people run into unexpected errors after pulling from `upstream/main`.

COMMENT:
Thank you for your response. I completely agree that having information about recompiling included in the documentation would be beneficial for users.

Thank you for recommending issue #24862. I will definitely take a look at it and see if I can contribute to scikit-learn in any way. Please let me know if there are any specific areas where I could be of help.

Thank you again for your time and assistance!

Labels: Documentation
Add documentation about re-building extensions to "Installing the development version of scikit-learn"

### Describe the issue linked to the documentation

Quite regularly when contributing to scikit-learn I realise that I need to re-build the Cython extensions. I realise this when I run pytest and get an import error. When I started contributing, I didn't know about this and I would have been rather lost when I faced this error the first time if @adrinjalali didn't tell me how to fix this. As many people will face the same issue (especially people from underrepresented groups in tech because they're often career changers), a note about re-building the extensions should be put in the documentation. The pandas team for example now recommends to their contributors to do this after each fetch and merge from `upstream/main`: https://pandas.pydata.org/docs/dev/development/contributing_environment.html#step-3-build-and-install-pandas

### Suggest a potential alternative/fix

I think the best place to make a note about this is here in the "Note" box under point 7: https://scikit-learn.org/dev/developers/advanced_installation.html#building-from-source
Similar to the pandas documentation, it could be recommended here to run `make in` after each fetch and merge (or pull) from `upstream/main`. Additionally, it could be recommended to run `make clean` in case people run into unexpected errors after pulling from `upstream/main`.

COMMENT:
How about the following text for the "note box" Maren referred to in the top post:

---

You will have to run the `pip install --no-build-isolation --editable .` command every time the source code of a Cython file is updated (ending in `.pyx` or `.pxd`). This can happen when you edit them or when you use a git command. Use the `--no-build-isolation` flag to avoid compiling the whole project each time, only the files that have been modified.

---

I left it as "via a git command" because it can happen when you change branches (because you work on more than one PR), or by fetching updates from the upstream repo, etc. I think it would be better to be more concrete, but I have no good proposal that isn't a multi-sentence thing.

Labels: Documentation
Add documentation about re-building extensions to "Installing the development version of scikit-learn"

### Describe the issue linked to the documentation

Quite regularly when contributing to scikit-learn I realise that I need to re-build the Cython extensions. I realise this when I run pytest and get an import error. When I started contributing, I didn't know about this and I would have been rather lost when I faced this error the first time if @adrinjalali didn't tell me how to fix this. As many people will face the same issue (especially people from underrepresented groups in tech because they're often career changers), a note about re-building the extensions should be put in the documentation. The pandas team for example now recommends to their contributors to do this after each fetch and merge from `upstream/main`: https://pandas.pydata.org/docs/dev/development/contributing_environment.html#step-3-build-and-install-pandas

### Suggest a potential alternative/fix

I think the best place to make a note about this is here in the "Note" box under point 7: https://scikit-learn.org/dev/developers/advanced_installation.html#building-from-source
Similar to the pandas documentation, it could be recommended here to run `make in` after each fetch and merge (or pull) from `upstream/main`. Additionally, it could be recommended to run `make clean` in case people run into unexpected errors after pulling from `upstream/main`.

COMMENT:
Another idea which came up when talking to @BenjaminBossan was that in any case, when doing an `import`, if compiled files are older than the source files, it's an issue and a build should most probably happen. This is only relevant when the package is installed in `editable` mode though (some suggestions for checking that here: https://stackoverflow.com/questions/43348746/how-to-detect-if-module-is-installed-in-editable-mode)

This is independent of the suggestion to edit the contributing docs though, that's a good addition anyway, but note that the command should be:
```
pip install -v --no-use-pep517 --no-build-isolation -e .
```

Labels: Documentation
Add documentation about re-building extensions to "Installing the development version of scikit-learn"

### Describe the issue linked to the documentation

Quite regularly when contributing to scikit-learn I realise that I need to re-build the Cython extensions. I realise this when I run pytest and get an import error. When I started contributing, I didn't know about this and I would have been rather lost when I faced this error the first time if @adrinjalali didn't tell me how to fix this. As many people will face the same issue (especially people from underrepresented groups in tech because they're often career changers), a note about re-building the extensions should be put in the documentation. The pandas team for example now recommends to their contributors to do this after each fetch and merge from `upstream/main`: https://pandas.pydata.org/docs/dev/development/contributing_environment.html#step-3-build-and-install-pandas

### Suggest a potential alternative/fix

I think the best place to make a note about this is here in the "Note" box under point 7: https://scikit-learn.org/dev/developers/advanced_installation.html#building-from-source
Similar to the pandas documentation, it could be recommended here to run `make in` after each fetch and merge (or pull) from `upstream/main`. Additionally, it could be recommended to run `make clean` in case people run into unexpected errors after pulling from `upstream/main`.

COMMENT:
I made a PR for this issue (see above) in which I updated the documentation to include the `--no-use-pep517` flag as proposed by @adrinjalali and I also integrated @betatim's suggestion in a slightly modified way. Have a look at it and let me know what you think. Regarding the command `make in`, there's more detailed documentation on the editable mode [here](https://scikit-learn.org/dev/developers/advanced_installation.html#editable-mode) which is linked [here](https://scikit-learn.org/dev/developers/advanced_installation.html#building-from-source) under point 5. So this is already covered in the documentation and I think the way it is currently documented is fine. 

I'm still not super happy with the documentation because I think it's still not clear enough for new contributors that when they run into import errors from pytest, they likely need to do a recompile. I was thinking that that kind of information could maybe be added to the `scikit-learn-inria-fondation/EuroSciPy22` repo [here](https://github.com/scikit-learn-inria-fondation/EuroSciPy22/blob/master/4.test.md) under a new section called "Debugging" because this repo is generally used for community sprints and contains a lot more helpful information for beginners.

Labels: Documentation