Talip7/scikit-learn-issues-multilabel huggingface dataset:
Slightly weird installation buttons maybe due to pydata-sphinx-theme 1.5.3?

### Describe the issue linked to the documentation

I recently browsed the dev website curious to see if I could spot differences after https://github.com/scikit-learn/scikit-learn/pull/29134 was merged.

Maybe it's just me, but I found that the install boxes where you select pip vs conda and potentially your OS is a bit confusing, the nesting of the two boxes looks visually complicated.
https://scikit-learn.org/dev/install.html#installing-the-latest-release
![image](https://github.com/scikit-learn/scikit-learn/assets/1680079/25735f3f-427b-45df-b526-09f2d34a9fc2)

Compare this to the stable website which I find a bit clearer maybe:
https://scikit-learn.org/stable/install.html#installing-the-latest-release
![image](https://github.com/scikit-learn/scikit-learn/assets/1680079/d585150b-919e-450b-940e-e44896fb5106)

Maybe @Charlie-XIAO you have some suggestions on how to improve it with some CSS magic?

### Suggest a potential alternative/fix

The PyTorch similar info for example seems a lot clearer to me than ours (both stable and dev website) https://pytorch.org/get-started/locally/#start-locally:
![image](https://github.com/scikit-learn/scikit-learn/assets/1680079/359c203f-8c94-42a7-a458-2f5298c48568)

The scikit-learn 1.4 seemed clearer as well:
https://scikit-learn.org/1.4/install.html#installing-the-latest-release
![image](https://github.com/scikit-learn/scikit-learn/assets/1680079/63339001-c402-43f4-8a84-a87b984ff325)

COMMENT:
> Also I think having the OS first and then afterwards pip / conda makes more sense to me. There is the main question (OS) and then the more subtle variation (conda vs pip). I am guessing, the order was reversed compared to 1.4.2 in part because I because this is implemented as a nested table  (but maybe I am wrong) and also because for conda this is the same thing for all the OSes.

Yes it's nested tabs. I agree to put OSes at the first level. This would mean repeating the conda thing three times but I don't think it's big deal.

> I get the argument about pure rst which is more maintainable than js + HTML. Maybe one compromise would be some CSS magic to make tabs to appear more like individual buttons and remove the lines around the table. Sorry I know only how to do the latter so here is a screenshot if that helps:

Should be doable without too much CSS, only at the risk of being broken when we update pydata theme again (but that's not big deal either). TBH I don't like this accessibility improvement either (and the one for the tables, especially in dark theme). But well for a web theme they have different considerations.

> Then there is the question whether rst + CSS magic is more maintainable than js + HTML, maybe not ... and I still prefer OS before conda/pip ...

AFAIC RST + CSS is definitely more maintainable, at least for this case.

Labels: Documentation, Low Priority
Slightly weird installation buttons maybe due to pydata-sphinx-theme 1.5.3?

### Describe the issue linked to the documentation

I recently browsed the dev website curious to see if I could spot differences after https://github.com/scikit-learn/scikit-learn/pull/29134 was merged.

Maybe it's just me, but I found that the install boxes where you select pip vs conda and potentially your OS is a bit confusing, the nesting of the two boxes looks visually complicated.
https://scikit-learn.org/dev/install.html#installing-the-latest-release
![image](https://github.com/scikit-learn/scikit-learn/assets/1680079/25735f3f-427b-45df-b526-09f2d34a9fc2)

Compare this to the stable website which I find a bit clearer maybe:
https://scikit-learn.org/stable/install.html#installing-the-latest-release
![image](https://github.com/scikit-learn/scikit-learn/assets/1680079/d585150b-919e-450b-940e-e44896fb5106)

Maybe @Charlie-XIAO you have some suggestions on how to improve it with some CSS magic?

### Suggest a potential alternative/fix

The PyTorch similar info for example seems a lot clearer to me than ours (both stable and dev website) https://pytorch.org/get-started/locally/#start-locally:
![image](https://github.com/scikit-learn/scikit-learn/assets/1680079/359c203f-8c94-42a7-a458-2f5298c48568)

The scikit-learn 1.4 seemed clearer as well:
https://scikit-learn.org/1.4/install.html#installing-the-latest-release
![image](https://github.com/scikit-learn/scikit-learn/assets/1680079/63339001-c402-43f4-8a84-a87b984ff325)

COMMENT:
#29160, something like this:

| Light | Dark |
| :---: | :--: |
| ![image](https://github.com/scikit-learn/scikit-learn/assets/108576690/d8ef6313-eb88-4b41-941c-66184a5292f4) | ![image](https://github.com/scikit-learn/scikit-learn/assets/108576690/0aac2a90-1ea8-468b-a52e-6e88a92e59e1) |

Labels: Documentation, Low Priority
Facing issue while installing abcpy

### Describe the bug

While installing abcpy package it has dependency package for scikit-learn which is already installed. But while it is installing it is searching for deprecated package name which is 'sklearn'. Error is showing like this:

![image](https://github.com/scikit-learn/scikit-learn/assets/59678268/57bdebd7-8935-437d-9ed0-ada9f8574e27)


### Steps/Code to Reproduce

No code

### Expected Results

I need abcpy package to be installed

### Actual Results

Package installing

### Versions

```shell
System:
    python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
executable: /home/mohammed/raghava/raghava/bin/python
   machine: Linux-5.15.0-105-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.5.0
          pip: 22.0.2
   setuptools: 59.6.0
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: None
       pandas: 2.2.2
   matplotlib: 3.9.0
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 40
         prefix: libopenblas
       filepath: /home/mohammed/raghava/raghava/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 40
         prefix: libopenblas
       filepath: /home/mohammed/raghava/raghava/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-01191904.3.27.so
        version: 0.3.27
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 40
         prefix: libgomp
       filepath: /home/mohammed/raghava/raghava/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
This is not a scikit-learn problem. This is an issue with `abcpy` requirement file. You can interact with the project on this issue: https://github.com/eth-cscs/abcpy/issues/111

To solve the problem, they should make a release since they already made the fix. Otherwise, you can install from source using their instructions.

Labels: Bug, Needs Triage
Facing issue while installing abcpy

### Describe the bug

While installing abcpy package it has dependency package for scikit-learn which is already installed. But while it is installing it is searching for deprecated package name which is 'sklearn'. Error is showing like this:

![image](https://github.com/scikit-learn/scikit-learn/assets/59678268/57bdebd7-8935-437d-9ed0-ada9f8574e27)


### Steps/Code to Reproduce

No code

### Expected Results

I need abcpy package to be installed

### Actual Results

Package installing

### Versions

```shell
System:
    python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
executable: /home/mohammed/raghava/raghava/bin/python
   machine: Linux-5.15.0-105-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.5.0
          pip: 22.0.2
   setuptools: 59.6.0
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: None
       pandas: 2.2.2
   matplotlib: 3.9.0
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 40
         prefix: libopenblas
       filepath: /home/mohammed/raghava/raghava/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 40
         prefix: libopenblas
       filepath: /home/mohammed/raghava/raghava/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-01191904.3.27.so
        version: 0.3.27
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 40
         prefix: libgomp
       filepath: /home/mohammed/raghava/raghava/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
Hey @glemaitre thanks for the reply. If possible can you share the link to get the instructions to install it?

Labels: Bug, Needs Triage
Facing issue while installing abcpy

### Describe the bug

While installing abcpy package it has dependency package for scikit-learn which is already installed. But while it is installing it is searching for deprecated package name which is 'sklearn'. Error is showing like this:

![image](https://github.com/scikit-learn/scikit-learn/assets/59678268/57bdebd7-8935-437d-9ed0-ada9f8574e27)


### Steps/Code to Reproduce

No code

### Expected Results

I need abcpy package to be installed

### Actual Results

Package installing

### Versions

```shell
System:
    python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
executable: /home/mohammed/raghava/raghava/bin/python
   machine: Linux-5.15.0-105-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.5.0
          pip: 22.0.2
   setuptools: 59.6.0
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: None
       pandas: 2.2.2
   matplotlib: 3.9.0
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 40
         prefix: libopenblas
       filepath: /home/mohammed/raghava/raghava/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 40
         prefix: libopenblas
       filepath: /home/mohammed/raghava/raghava/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-01191904.3.27.so
        version: 0.3.27
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 40
         prefix: libgomp
       filepath: /home/mohammed/raghava/raghava/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
No sorry. I don't know the project. You can probably find it on their GitHub repository (and documentation).

Labels: Bug, Needs Triage
V1.5 randomly hangs on import

### Describe the bug

Since upgrading to version 1.5.0 we are seeing issues where the process stalls on import. Through a core dump, it appears to be locking up [here](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/__init__.py#L154). We run hundreds of processes in parallel and it will hang on only a handful (less than 10).

### Steps/Code to Reproduce

```
import sklearn
```

### Expected Results

it imports

### Actual Results

process hangs indefinitely.

### Versions

```shell
System:
    python: 3.11.3 (main, May 15 2023, 15:45:52) [GCC 11.2.0]
executable: /home/jcoder/git/neo/neo/pyenv/bin/python
   machine: Linux-6.1.87-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.0
          pip: 24.0
   setuptools: 70.0.0
        numpy: 1.24.4
        scipy: 1.11.1
       Cython: 0.29.35
       pandas: 2.0.2
   matplotlib: 3.7.2
       joblib: 1.3.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /opt/anaconda/2023/envs/pandas2/lib/libopenblasp-r0.3.23.so
        version: 0.3.23
threading_layer: pthreads
   architecture: Zen
    num_threads: 64

       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/~~~~~~/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 64
```

COMMENT:
Difficult to reproduce here.

> We run hundreds of processes in parallel and it will hang on only a handful (less than 10).

Could you provide more details with some code or pseudo-code. Do you make the import in the different processes?

Labels: Bug, Needs Reproducible Code
V1.5 randomly hangs on import

### Describe the bug

Since upgrading to version 1.5.0 we are seeing issues where the process stalls on import. Through a core dump, it appears to be locking up [here](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/__init__.py#L154). We run hundreds of processes in parallel and it will hang on only a handful (less than 10).

### Steps/Code to Reproduce

```
import sklearn
```

### Expected Results

it imports

### Actual Results

process hangs indefinitely.

### Versions

```shell
System:
    python: 3.11.3 (main, May 15 2023, 15:45:52) [GCC 11.2.0]
executable: /home/jcoder/git/neo/neo/pyenv/bin/python
   machine: Linux-6.1.87-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.0
          pip: 24.0
   setuptools: 70.0.0
        numpy: 1.24.4
        scipy: 1.11.1
       Cython: 0.29.35
       pandas: 2.0.2
   matplotlib: 3.7.2
       joblib: 1.3.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /opt/anaconda/2023/envs/pandas2/lib/libopenblasp-r0.3.23.so
        version: 0.3.23
threading_layer: pthreads
   architecture: Zen
    num_threads: 64

       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/~~~~~~/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 64
```

COMMENT:
I agree with @glemaitre, it would be really useful if you could try some of these things (in rough order of time necessary to do it):
- upgrade threadpoolctl to the latest release 3.5.0  just in case, or even the threadpoolctl development version from `main` (for example via `pip install git+https://github.com/joblib/threadpoolctl`)
- provide more information how you start your processes, are they all started roughly at the same time?
- provide more information on the number of cores you have on your machine, the OpenBLAS info says `num_threads: 64` but maybe if you run hundred of processes you may have more than this?
- try to reproduce without scikit-learn i.e. try to create a `threadpoolctl.ThreadPoolController()` does it still happen? In case it does not happen, what if you do `import numpy` before creating the `ThreadpoolController()`, then add `scipy.linalg`, then add `from sklearn.utils._show_versions import show_versions`
- in an ideal world you would provide a stand-alone snippet that reproduces for you. This would be a first step in order for us to try to reproduce, which may not be that easy but let's see ...

cc @jeremiedbb because he is the `threadpoolctl` expert :wink: 

I quickly looked at reported hangs in threadpoolctl and the only thing I could find was https://github.com/joblib/threadpoolctl/issues/176, not sure at all this is relevant, since this kind of symptoms can have different root causes.

Labels: Bug, Needs Reproducible Code
V1.5 randomly hangs on import

### Describe the bug

Since upgrading to version 1.5.0 we are seeing issues where the process stalls on import. Through a core dump, it appears to be locking up [here](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/__init__.py#L154). We run hundreds of processes in parallel and it will hang on only a handful (less than 10).

### Steps/Code to Reproduce

```
import sklearn
```

### Expected Results

it imports

### Actual Results

process hangs indefinitely.

### Versions

```shell
System:
    python: 3.11.3 (main, May 15 2023, 15:45:52) [GCC 11.2.0]
executable: /home/jcoder/git/neo/neo/pyenv/bin/python
   machine: Linux-6.1.87-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.0
          pip: 24.0
   setuptools: 70.0.0
        numpy: 1.24.4
        scipy: 1.11.1
       Cython: 0.29.35
       pandas: 2.0.2
   matplotlib: 3.7.2
       joblib: 1.3.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /opt/anaconda/2023/envs/pandas2/lib/libopenblasp-r0.3.23.so
        version: 0.3.23
threading_layer: pthreads
   architecture: Zen
    num_threads: 64

       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/~~~~~~/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 64
```

COMMENT:
Thanks for the additional info! So it looks like this hangs when loading a shared library either OpenBLAS or OpenMP. I don't have a good explanation off the top of my head why this could happen.

> The filepath in this case seems to be /opt/anaconda/2023/envs/pandas2/lib/libopenblasp-r0.3.23.so, however in another core dump, it showed /home/~~~~~~/pyenv/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0 as the filepath it was loading.

Just to make sure I follow, you mean `self._name` in `self._handle = _dlopen(self._name, mode)` (last line of the stack-trace) right?

Also can you give a more details about how you create your environment?

It seems very weird that OpenBLAS library is from `/opt/anaconda/2023/envs/pandas2` so a `conda` environment and that OpenMP is from a completely different place namely `/home/~~~/pyenv` so I am guessing a `pyenv` environment ...

Maybe try to create a new environment from scratch for using a single tool (for example `conda`) and see whether the problem still persists?

Just out of curiosity, can you give some details about why you use `~~~` in paths, is it to hide your username for privacy reasons? Is it because you are on a HPC center and usernames are sometimes considered sensitive information? As a maintainer, I see this from time to time and it always feels a bit frustrating because I am never quite sure what people remove and sometimes this may be useful information. If you only remove your username, can I suggest using something like `/home/<username>/pyenv` so that at least it is more explicit :pray:?

Labels: Bug, Needs Reproducible Code
V1.5 randomly hangs on import

### Describe the bug

Since upgrading to version 1.5.0 we are seeing issues where the process stalls on import. Through a core dump, it appears to be locking up [here](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/__init__.py#L154). We run hundreds of processes in parallel and it will hang on only a handful (less than 10).

### Steps/Code to Reproduce

```
import sklearn
```

### Expected Results

it imports

### Actual Results

process hangs indefinitely.

### Versions

```shell
System:
    python: 3.11.3 (main, May 15 2023, 15:45:52) [GCC 11.2.0]
executable: /home/jcoder/git/neo/neo/pyenv/bin/python
   machine: Linux-6.1.87-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.0
          pip: 24.0
   setuptools: 70.0.0
        numpy: 1.24.4
        scipy: 1.11.1
       Cython: 0.29.35
       pandas: 2.0.2
   matplotlib: 3.7.2
       joblib: 1.3.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /opt/anaconda/2023/envs/pandas2/lib/libopenblasp-r0.3.23.so
        version: 0.3.23
threading_layer: pthreads
   architecture: Zen
    num_threads: 64

       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/~~~~~~/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 64
```

COMMENT:
Hmmm if the problem goes away without New Relic, I would encourage you to follow up with New Relic in case you have access to their support. If that is indeed the issue, it looks like all the New Relic users trying to use scikit-learn 1.5 may have this problem since this happens on import and New Relic may have some incentive to fix the issue.

Of course, if you manage to reproduce without New Relic, we would try to take a look, but this likely a significant amount of work.

This is quite low-level and out of my area of expertise, so I don't have good suggestions on what the root cause could be. It looks like maybe trying to load shared libraries in two different threads may be an issue?

About `gdb` the first thing I do is `bt` which will show you a back-trace. Searching a bit I found this [SO answer](https://stackoverflow.com/a/3491304) that seems to indicate you can find the two threads where the dead-lock occurs and get the stack-trace of each thread.

More context: the reason this happens in scikit-learn 1.5 and not scikit-learn 1.4 is likely https://github.com/scikit-learn/scikit-learn/pull/28838 that creates a `ThreadPoolController` in `sklearn/__init__.py` (import time) whereas before the loaded shared libraries would be inspected later (usage time), i.e. when threadpoolctl is used with a `threadpool_limits` or similar construct.

Labels: Bug, Needs Reproducible Code
V1.5 randomly hangs on import

### Describe the bug

Since upgrading to version 1.5.0 we are seeing issues where the process stalls on import. Through a core dump, it appears to be locking up [here](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/__init__.py#L154). We run hundreds of processes in parallel and it will hang on only a handful (less than 10).

### Steps/Code to Reproduce

```
import sklearn
```

### Expected Results

it imports

### Actual Results

process hangs indefinitely.

### Versions

```shell
System:
    python: 3.11.3 (main, May 15 2023, 15:45:52) [GCC 11.2.0]
executable: /home/jcoder/git/neo/neo/pyenv/bin/python
   machine: Linux-6.1.87-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.5.0
          pip: 24.0
   setuptools: 70.0.0
        numpy: 1.24.4
        scipy: 1.11.1
       Cython: 0.29.35
       pandas: 2.0.2
   matplotlib: 3.7.2
       joblib: 1.3.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /opt/anaconda/2023/envs/pandas2/lib/libopenblasp-r0.3.23.so
        version: 0.3.23
threading_layer: pthreads
   architecture: Zen
    num_threads: 64

       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/~~~~~~/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 64
```

COMMENT:
> One other thing to mention is that the New Relic python agent has threads running

It could be that they create subprocesses using the `fork` method which is known to be unsafe and broken with OpenMP (https://stackoverflow.com/questions/49049388/understanding-openmp-shortcomings-regarding-fork). If the issue is there with New Relic and disappears without it, that's where I'd look.

Labels: Bug, Needs Reproducible Code