Talip7/scikit-learn-issues-multilabel huggingface dataset:
Sphinx version information in "Building the documentation" section needs reevaluation

### Describe the issue linked to the documentation

At the bottom of the ["Building the documentation" section](https://github.com/scikit-learn/scikit-learn/blob/main/doc/developers/contributing.rst#building-the-documentation) there is a warning about the best performing `sphinx` version, which leads to [this file](https://github.com/search?q=repo%3Ascikit-learn%2Fscikit-learn+sphinx+path%3Abuild_tools%2Fcircle%2Fdoc_environment.yml&type=code), supposedly mirroring the configuration on `CircleCI`.

In the file, the suggested version is `sphinx=6.0.0`. However, `sphinx=7.0.0` is the minimum necessary to make the current package combination work. If the version is reverted back to `sphinx=6.0.0`, the following error will appear: `sphinx-prompt 1.8.0 requires Sphinx<8.0.0,>=7.0.0, but you have sphinx 6.0.0 which is incompatible`. 

### Suggest a potential alternative/fix

Replace `sphinx=6.0.0` with `sphinx=7.0.0`.

COMMENT:
And by the way we pin sphinx to 6.0 because there was an issue with sphinx search with more recent versions at one point https://github.com/scikit-learn/scikit-learn/pull/25504, not sure whether the issue is still here.

Also some issues have been reported with sphinx 7, some of them have been fixed e.g. https://github.com/scikit-learn/scikit-learn/pull/26627 but there may be others left ...

Labels: Documentation
Sphinx version information in "Building the documentation" section needs reevaluation

### Describe the issue linked to the documentation

At the bottom of the ["Building the documentation" section](https://github.com/scikit-learn/scikit-learn/blob/main/doc/developers/contributing.rst#building-the-documentation) there is a warning about the best performing `sphinx` version, which leads to [this file](https://github.com/search?q=repo%3Ascikit-learn%2Fscikit-learn+sphinx+path%3Abuild_tools%2Fcircle%2Fdoc_environment.yml&type=code), supposedly mirroring the configuration on `CircleCI`.

In the file, the suggested version is `sphinx=6.0.0`. However, `sphinx=7.0.0` is the minimum necessary to make the current package combination work. If the version is reverted back to `sphinx=6.0.0`, the following error will appear: `sphinx-prompt 1.8.0 requires Sphinx<8.0.0,>=7.0.0, but you have sphinx 6.0.0 which is incompatible`. 

### Suggest a potential alternative/fix

Replace `sphinx=6.0.0` with `sphinx=7.0.0`.

COMMENT:
Thanks for the idea Loic about how a user can end up in this situation. Maybe we can improve our documentation guide? It sounds like this might be easier than upgrading to sphinx 7?

Labels: Documentation
Sphinx version information in "Building the documentation" section needs reevaluation

### Describe the issue linked to the documentation

At the bottom of the ["Building the documentation" section](https://github.com/scikit-learn/scikit-learn/blob/main/doc/developers/contributing.rst#building-the-documentation) there is a warning about the best performing `sphinx` version, which leads to [this file](https://github.com/search?q=repo%3Ascikit-learn%2Fscikit-learn+sphinx+path%3Abuild_tools%2Fcircle%2Fdoc_environment.yml&type=code), supposedly mirroring the configuration on `CircleCI`.

In the file, the suggested version is `sphinx=6.0.0`. However, `sphinx=7.0.0` is the minimum necessary to make the current package combination work. If the version is reverted back to `sphinx=6.0.0`, the following error will appear: `sphinx-prompt 1.8.0 requires Sphinx<8.0.0,>=7.0.0, but you have sphinx 6.0.0 which is incompatible`. 

### Suggest a potential alternative/fix

Replace `sphinx=6.0.0` with `sphinx=7.0.0`.

COMMENT:
I think it's worth testing if we can remove the pinning to sphinx 6. That would be more sustainable in the long term.

Labels: Documentation
Sphinx version information in "Building the documentation" section needs reevaluation

### Describe the issue linked to the documentation

At the bottom of the ["Building the documentation" section](https://github.com/scikit-learn/scikit-learn/blob/main/doc/developers/contributing.rst#building-the-documentation) there is a warning about the best performing `sphinx` version, which leads to [this file](https://github.com/search?q=repo%3Ascikit-learn%2Fscikit-learn+sphinx+path%3Abuild_tools%2Fcircle%2Fdoc_environment.yml&type=code), supposedly mirroring the configuration on `CircleCI`.

In the file, the suggested version is `sphinx=6.0.0`. However, `sphinx=7.0.0` is the minimum necessary to make the current package combination work. If the version is reverted back to `sphinx=6.0.0`, the following error will appear: `sphinx-prompt 1.8.0 requires Sphinx<8.0.0,>=7.0.0, but you have sphinx 6.0.0 which is incompatible`. 

### Suggest a potential alternative/fix

Replace `sphinx=6.0.0` with `sphinx=7.0.0`.

COMMENT:
> If you are looking to contribute to scikit-learn, the pragmatic solution is to downgrade sphinx-prompt: `pip install sphinx=6.0 sphinx-prompt`.
> 
> I am not sure this is that easy to document because this kind of thing tend to change a lot with time ... we could link to the [lock file](https://github.com/scikit-learn/scikit-learn/blob/main/build_tools/circle/doc_linux-64_conda.lock) we use in the CI since it has all the versions which we use for the CI but this is quite verbose.
> 
> This is easy to get into such a situation with pip:
> 
> ```
> # gets sphinx 7.2.6 sphinx-prompt 1.8 (latest releases) 
> pip install sphinx sphinx-prompt
> ```
> 
> Oh wait the doc says I need sphinx 6.0:
> 
> ```
> pip install sphinx==6.0
> ```
> 
> At the end of the output you get some kind of warnings that you have incompatible packages but pip still installs them:
> 
> ```
> ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
> sphinx-prompt 1.8.0 requires Sphinx<8.0.0,>=7.0.0, but you have sphinx 6.0.0 which is incompatible.
> Successfully installed sphinx-6.0.0
> ```

I can confirm that it was a similar process. I am working on adding a new example in the documentation, I was reading the building docs and saw the note, changed the version I had. I have since then reverted it to the newer version and I am working without issues on adding the example, but I thought maybe this is worthwhile looking into as it might be confusing for others. Maybe it would be enough just to remove/reprahse that part of the building documentation(the "warning") while this is being worked on, or in the process of eventually updating?

Labels: Documentation
Sphinx version information in "Building the documentation" section needs reevaluation

### Describe the issue linked to the documentation

At the bottom of the ["Building the documentation" section](https://github.com/scikit-learn/scikit-learn/blob/main/doc/developers/contributing.rst#building-the-documentation) there is a warning about the best performing `sphinx` version, which leads to [this file](https://github.com/search?q=repo%3Ascikit-learn%2Fscikit-learn+sphinx+path%3Abuild_tools%2Fcircle%2Fdoc_environment.yml&type=code), supposedly mirroring the configuration on `CircleCI`.

In the file, the suggested version is `sphinx=6.0.0`. However, `sphinx=7.0.0` is the minimum necessary to make the current package combination work. If the version is reverted back to `sphinx=6.0.0`, the following error will appear: `sphinx-prompt 1.8.0 requires Sphinx<8.0.0,>=7.0.0, but you have sphinx 6.0.0 which is incompatible`. 

### Suggest a potential alternative/fix

Replace `sphinx=6.0.0` with `sphinx=7.0.0`.

COMMENT:
> I think it's worth testing if we can remove the pinning to sphinx 6. That would be more sustainable in the long term.

Locally it seems to work fine with the latest Sphinx version. I have opened https://github.com/scikit-learn/scikit-learn/pull/27656 to see if there are issues.

Labels: Documentation
Sphinx version information in "Building the documentation" section needs reevaluation

### Describe the issue linked to the documentation

At the bottom of the ["Building the documentation" section](https://github.com/scikit-learn/scikit-learn/blob/main/doc/developers/contributing.rst#building-the-documentation) there is a warning about the best performing `sphinx` version, which leads to [this file](https://github.com/search?q=repo%3Ascikit-learn%2Fscikit-learn+sphinx+path%3Abuild_tools%2Fcircle%2Fdoc_environment.yml&type=code), supposedly mirroring the configuration on `CircleCI`.

In the file, the suggested version is `sphinx=6.0.0`. However, `sphinx=7.0.0` is the minimum necessary to make the current package combination work. If the version is reverted back to `sphinx=6.0.0`, the following error will appear: `sphinx-prompt 1.8.0 requires Sphinx<8.0.0,>=7.0.0, but you have sphinx 6.0.0 which is incompatible`. 

### Suggest a potential alternative/fix

Replace `sphinx=6.0.0` with `sphinx=7.0.0`.

COMMENT:
Now that #27656 has been merged the sphinx version isn't pinned anymore. Do you think this is good enough to close this issue or should we make some additional changes to the instructions (especially from the viewpoint of a new contributor)?

Labels: Documentation
Sphinx version information in "Building the documentation" section needs reevaluation

### Describe the issue linked to the documentation

At the bottom of the ["Building the documentation" section](https://github.com/scikit-learn/scikit-learn/blob/main/doc/developers/contributing.rst#building-the-documentation) there is a warning about the best performing `sphinx` version, which leads to [this file](https://github.com/search?q=repo%3Ascikit-learn%2Fscikit-learn+sphinx+path%3Abuild_tools%2Fcircle%2Fdoc_environment.yml&type=code), supposedly mirroring the configuration on `CircleCI`.

In the file, the suggested version is `sphinx=6.0.0`. However, `sphinx=7.0.0` is the minimum necessary to make the current package combination work. If the version is reverted back to `sphinx=6.0.0`, the following error will appear: `sphinx-prompt 1.8.0 requires Sphinx<8.0.0,>=7.0.0, but you have sphinx 6.0.0 which is incompatible`. 

### Suggest a potential alternative/fix

Replace `sphinx=6.0.0` with `sphinx=7.0.0`.

COMMENT:
> Now that #27656 has been merged the sphinx version isn't pinned anymore. Do you think this is good enough to close this issue or should we make some additional changes to the instructions (especially from the viewpoint of a new contributor)?

This looks great to me, and not confusing any more. The issue that I had yesterday has no way happening like this.

Labels: Documentation
Sphinx version information in "Building the documentation" section needs reevaluation

### Describe the issue linked to the documentation

At the bottom of the ["Building the documentation" section](https://github.com/scikit-learn/scikit-learn/blob/main/doc/developers/contributing.rst#building-the-documentation) there is a warning about the best performing `sphinx` version, which leads to [this file](https://github.com/search?q=repo%3Ascikit-learn%2Fscikit-learn+sphinx+path%3Abuild_tools%2Fcircle%2Fdoc_environment.yml&type=code), supposedly mirroring the configuration on `CircleCI`.

In the file, the suggested version is `sphinx=6.0.0`. However, `sphinx=7.0.0` is the minimum necessary to make the current package combination work. If the version is reverted back to `sphinx=6.0.0`, the following error will appear: `sphinx-prompt 1.8.0 requires Sphinx<8.0.0,>=7.0.0, but you have sphinx 6.0.0 which is incompatible`. 

### Suggest a potential alternative/fix

Replace `sphinx=6.0.0` with `sphinx=7.0.0`.

COMMENT:
OK let's close this one then, thanks for your feed-back @TamaraAtanasoska!

Labels: Documentation
Please provide option to set unknown_values during test time to same as encoded min_frequency  in OrdinalEncoder(Infrequent categories)

### Describe the workflow you want to enable

It seems that OneHotEncoder has a parameter for setting` handle_unknown='infrequent_if_exist'` but the same is missing in OrdinalEncoder . Currently `unknown_value` and the value encoded by setting the parameter `min_frequency` seems to be different. There is always workaround to figure out the encoded value on `min_frequency` and pass the same to `unknown_values` but I think having something similar to OneHotEncoder's parameter `handle_unknown='infrequent_if_exist'` seems intuitive as we would want to treat unseen values as infrequent ones. Not sure if this feature already exists and I'm missing it somehow. 

### Describe your proposed solution

Implement parameter option similar to OneHotEncoder's parameter `handle_unknown='infrequent_if_exist'`  where unknown (unseen values during training) get similar encoding as happened for infrequent_categories during training. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Sounds reasonable, if there's a PR for it, happy to review.

Labels: New Feature, help wanted, Hard
Please provide option to set unknown_values during test time to same as encoded min_frequency  in OrdinalEncoder(Infrequent categories)

### Describe the workflow you want to enable

It seems that OneHotEncoder has a parameter for setting` handle_unknown='infrequent_if_exist'` but the same is missing in OrdinalEncoder . Currently `unknown_value` and the value encoded by setting the parameter `min_frequency` seems to be different. There is always workaround to figure out the encoded value on `min_frequency` and pass the same to `unknown_values` but I think having something similar to OneHotEncoder's parameter `handle_unknown='infrequent_if_exist'` seems intuitive as we would want to treat unseen values as infrequent ones. Not sure if this feature already exists and I'm missing it somehow. 

### Describe your proposed solution

Implement parameter option similar to OneHotEncoder's parameter `handle_unknown='infrequent_if_exist'`  where unknown (unseen values during training) get similar encoding as happened for infrequent_categories during training. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
If you eliminate the distinction between infrequent categories and the missing category, there is also no need for the `infrequent_categories_` attribute, which can require a lot of storage on a large dataset where many of the infrequent values are unique. Having an option to not store those categories at all within the encoder's state would make it much smaller when pickled. This is a rather important change for being able to use OrdinalEncoder with confidence on high-cardinality columns (where _some_ of the values may still be frequent) without fearing it will massively blow up the size of the final artifacts. The whole point of the new `max_categories` field seems to be to enable this kind of use on high-cardinality fields (if we use it solely for low-cardinality columns, why do we even need to limit the number of categories?)

Labels: New Feature, help wanted, Hard
Please provide option to set unknown_values during test time to same as encoded min_frequency  in OrdinalEncoder(Infrequent categories)

### Describe the workflow you want to enable

It seems that OneHotEncoder has a parameter for setting` handle_unknown='infrequent_if_exist'` but the same is missing in OrdinalEncoder . Currently `unknown_value` and the value encoded by setting the parameter `min_frequency` seems to be different. There is always workaround to figure out the encoded value on `min_frequency` and pass the same to `unknown_values` but I think having something similar to OneHotEncoder's parameter `handle_unknown='infrequent_if_exist'` seems intuitive as we would want to treat unseen values as infrequent ones. Not sure if this feature already exists and I'm missing it somehow. 

### Describe your proposed solution

Implement parameter option similar to OneHotEncoder's parameter `handle_unknown='infrequent_if_exist'`  where unknown (unseen values during training) get similar encoding as happened for infrequent_categories during training. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
How should `infrequent_if_exist` communicate with the current "nan semantics" in the `OrdinalEncoder`?

Currently `OrdinalEncoder` treats `np.nan` differently than `OneHotEncoder`:

|scenario|OneHotEncoder|OrdinalEncoder|
|-----------|---------------------|---------------------|
|np.nan is frequent|own category|np.nan|
|np.nan is infrequent|infrequent category|np.nan|
|np.nan is unknown|infrequent category| ??|

One would imagine that it should be mapped to `np.nan`, but in the situation we have `handle_unknown='encoded_value'` and `unknown_value='value'` it gets mapped to `value`.

Labels: New Feature, help wanted, Hard
DOC link benchmark results site

### Describe the issue linked to the documentation

Mention https://scikit-learn.org/scikit-learn-benchmarks somewhere in our docs. I could only find it in the Readme.

### Suggest a potential alternative/fix

_No response_

COMMENT:
It is already mentioned in https://scikit-learn.org/dev/developers/contributing.html#monitoring-performance.

@lorentzenchr do you have a more specific idea from where to link to them in addition to this page?

Labels: Documentation
DOC link benchmark results site

### Describe the issue linked to the documentation

Mention https://scikit-learn.org/scikit-learn-benchmarks somewhere in our docs. I could only find it in the Readme.

### Suggest a potential alternative/fix

_No response_

COMMENT:
I was not able to find that link. It appears as one of the last hits when searching for „benchmark“ under „contributing“. That’s not where I would look.

Labels: Documentation
DOC link benchmark results site

### Describe the issue linked to the documentation

Mention https://scikit-learn.org/scikit-learn-benchmarks somewhere in our docs. I could only find it in the Readme.

### Suggest a potential alternative/fix

_No response_

COMMENT:
I think it makes sense to have it in the contributing guide. If we want it to pop up in the documentation, we would need to have a top section with "Benchmark" in it. 

IMO, the issue is more related to our search engine that is really limited. In a perfect world, I would like to:

- have better recommendations when searching
- display in a better way the results (top match maybe)

It would not be super beneficial in the long term and not only for the benchmarks.

Maybe the first thing to reconsider here is to have the summary displayed:

https://github.com/scikit-learn/scikit-learn/blob/38d499b50752339795f077f40e7846c0c86b0026/doc/conf.py#L311-L312

Labels: Documentation
euclidean_distances with float64 x,y and float32 xx and yy

### Describe the bug

When running `euclidean_distances` I think it is possible to get to [this](https://github.com/scikit-learn/scikit-learn/blob/d99b728b3a7952b2111cf5e0cb5d14f92c6f3a80/sklearn/metrics/pairwise.py#L380) line of code with `XX` being `None`. This will happen when the input `X` and `Y` are `float64` but `X_norm_squared` and `Y_norm_squared` are `float32`.

### Steps/Code to Reproduce

```python
import numpy as np
from sklearn.metrics.pairwise import euclidean_distances

x = np.random.randint(0, 10, size=(100, 5)).astype(np.float64)
y = np.random.randint(0, 10, size=(100, 5)).astype(np.float64)

xx = np.einsum("ij,ij->i", x, x)
yy = np.einsum("ij,ij->i", y, y)

euclidean_distances(x, y, Y_norm_squared=yy.astype(np.float32), X_norm_squared=xx.astype(np.float32))
```

### Expected Results

No error is thrown

### Actual Results

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ageorgiou/projects/test/venv/lib/python3.10/site-packages/sklearn/metrics/pairwise.py", line 338, in euclidean_distances
    return _euclidean_distances(X, Y, X_norm_squared, Y_norm_squared, squared)
  File "/home/ageorgiou/projects/test/venv/lib/python3.10/site-packages/sklearn/metrics/pairwise.py", line 380, in _euclidean_distances
    distances += XX
numpy.core._exceptions._UFuncOutputCastingError: Cannot cast ufunc 'add' output from dtype('O') to dtype('float64') with casting rule 'same_kind'
```

### Versions

```shell
System:
    python: 3.10.6 (main, Oct 30 2022, 13:35:37) [GCC 12.2.0]
executable: /home/ageorgiou/projects/test/venv/bin/python
   machine: Linux-6.2.12-1-MANJARO-x86_64-with-glibc2.38

Python dependencies:
      sklearn: 1.3.1
          pip: 22.2.1
   setuptools: 63.2.0
        numpy: 1.26.1
        scipy: 1.11.3
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/ageorgiou/projects/test/venv/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /home/ageorgiou/projects/test/venv/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/ageorgiou/projects/test/venv/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Haswell

```
```

COMMENT:
Thanks for reporting this. I can reproduce it. Maybe @Micky774 or @jjerphan can have a look at this?

Labels: Bug