Talip7/scikit-learn-issues-multilabel huggingface dataset:
BisectingKMeans floating point exception fatal error on OSX

### Describe the bug

Hi, thanks for this fantastic library 🚀 🎉 

I believe I found a bug. In the BisectingKMeans clustering, the `predict` function cannot be used when the to-be-predicted-data is on a different numerical scale than the fitted data. Other clustering methods like e.g., KMeans dont have this problem.

In the example below, I fitted the BisectingKMeans on random data and then multiplied the data that should be predicted by 50. It causes a floating point exception error which is pretty bad because python silently exits.

### Steps/Code to Reproduce

```py
from sklearn.cluster import BisectingKMeans, KMeans
import numpy as np
x = np.random.rand(3000, 10)

bisect_means = BisectingKMeans(n_clusters=10).fit(x)

labels = bisect_means.predict(50*np.random.rand(100, 10))
print(labels)
```

### Expected Results

A list of predicted class labels

### Actual Results

Python silently exits with:
```
[1]    45074 floating point exception  python foo.py
```
In Jupyter/ipython the kernel simply dies.

### Versions

```shell
System:
    python: 3.9.16 (main, Mar  8 2023, 04:29:44)  [Clang 14.0.6 ]
executable: /Users/jab/miniconda3/envs/qondot/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.3.0
          pip: 23.0.1
   setuptools: 67.8.0
        numpy: 1.21.6
        scipy: 1.10.1
       Cython: 0.29.35
       pandas: 1.5.3
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/numpy/.dylibs/libopenblas.0.dylib
        version: 0.3.17
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 4
```

COMMENT:
In particular it looks like an integer division by 0.

Labels: Bug, Needs Triage
BisectingKMeans floating point exception fatal error on OSX

### Describe the bug

Hi, thanks for this fantastic library 🚀 🎉 

I believe I found a bug. In the BisectingKMeans clustering, the `predict` function cannot be used when the to-be-predicted-data is on a different numerical scale than the fitted data. Other clustering methods like e.g., KMeans dont have this problem.

In the example below, I fitted the BisectingKMeans on random data and then multiplied the data that should be predicted by 50. It causes a floating point exception error which is pretty bad because python silently exits.

### Steps/Code to Reproduce

```py
from sklearn.cluster import BisectingKMeans, KMeans
import numpy as np
x = np.random.rand(3000, 10)

bisect_means = BisectingKMeans(n_clusters=10).fit(x)

labels = bisect_means.predict(50*np.random.rand(100, 10))
print(labels)
```

### Expected Results

A list of predicted class labels

### Actual Results

Python silently exits with:
```
[1]    45074 floating point exception  python foo.py
```
In Jupyter/ipython the kernel simply dies.

### Versions

```shell
System:
    python: 3.9.16 (main, Mar  8 2023, 04:29:44)  [Clang 14.0.6 ]
executable: /Users/jab/miniconda3/envs/qondot/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.3.0
          pip: 23.0.1
   setuptools: 67.8.0
        numpy: 1.21.6
        scipy: 1.10.1
       Cython: 0.29.35
       pandas: 1.5.3
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/numpy/.dylibs/libopenblas.0.dylib
        version: 0.3.17
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 4
```

COMMENT:
Your test suite does not happen to run on a `macos-latest` (or similar) instance that uses an intel mac?

Labels: Bug, Needs Triage
BisectingKMeans floating point exception fatal error on OSX

### Describe the bug

Hi, thanks for this fantastic library 🚀 🎉 

I believe I found a bug. In the BisectingKMeans clustering, the `predict` function cannot be used when the to-be-predicted-data is on a different numerical scale than the fitted data. Other clustering methods like e.g., KMeans dont have this problem.

In the example below, I fitted the BisectingKMeans on random data and then multiplied the data that should be predicted by 50. It causes a floating point exception error which is pretty bad because python silently exits.

### Steps/Code to Reproduce

```py
from sklearn.cluster import BisectingKMeans, KMeans
import numpy as np
x = np.random.rand(3000, 10)

bisect_means = BisectingKMeans(n_clusters=10).fit(x)

labels = bisect_means.predict(50*np.random.rand(100, 10))
print(labels)
```

### Expected Results

A list of predicted class labels

### Actual Results

Python silently exits with:
```
[1]    45074 floating point exception  python foo.py
```
In Jupyter/ipython the kernel simply dies.

### Versions

```shell
System:
    python: 3.9.16 (main, Mar  8 2023, 04:29:44)  [Clang 14.0.6 ]
executable: /Users/jab/miniconda3/envs/qondot/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.3.0
          pip: 23.0.1
   setuptools: 67.8.0
        numpy: 1.21.6
        scipy: 1.10.1
       Cython: 0.29.35
       pandas: 1.5.3
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/numpy/.dylibs/libopenblas.0.dylib
        version: 0.3.17
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 4
```

COMMENT:
Note, even if scikit-learn was not build with specific Cython config, it might be possible to tell lldb to break on the symbol names found in the backtrace (e.g. `__pyx_fuse_1__pyx_pw_7sklearn_7cluster_14_k_means_lloyd_7lloyd_iter_chunked_dense` in this case). However then I am not sure how to inspect the values of local variables generated by the Cython code.

Labels: Bug, Needs Triage
BisectingKMeans floating point exception fatal error on OSX

### Describe the bug

Hi, thanks for this fantastic library 🚀 🎉 

I believe I found a bug. In the BisectingKMeans clustering, the `predict` function cannot be used when the to-be-predicted-data is on a different numerical scale than the fitted data. Other clustering methods like e.g., KMeans dont have this problem.

In the example below, I fitted the BisectingKMeans on random data and then multiplied the data that should be predicted by 50. It causes a floating point exception error which is pretty bad because python silently exits.

### Steps/Code to Reproduce

```py
from sklearn.cluster import BisectingKMeans, KMeans
import numpy as np
x = np.random.rand(3000, 10)

bisect_means = BisectingKMeans(n_clusters=10).fit(x)

labels = bisect_means.predict(50*np.random.rand(100, 10))
print(labels)
```

### Expected Results

A list of predicted class labels

### Actual Results

Python silently exits with:
```
[1]    45074 floating point exception  python foo.py
```
In Jupyter/ipython the kernel simply dies.

### Versions

```shell
System:
    python: 3.9.16 (main, Mar  8 2023, 04:29:44)  [Clang 14.0.6 ]
executable: /Users/jab/miniconda3/envs/qondot/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.3.0
          pip: 23.0.1
   setuptools: 67.8.0
        numpy: 1.21.6
        scipy: 1.10.1
       Cython: 0.29.35
       pandas: 1.5.3
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/numpy/.dylibs/libopenblas.0.dylib
        version: 0.3.17
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 4
```

COMMENT:
Maybe the variable names found in the generated `sklearn/cluster/_k_means_lloyd.c` file when building scikit-learn from source would be reachable via lldb?

Labels: Bug, Needs Triage
BisectingKMeans floating point exception fatal error on OSX

### Describe the bug

Hi, thanks for this fantastic library 🚀 🎉 

I believe I found a bug. In the BisectingKMeans clustering, the `predict` function cannot be used when the to-be-predicted-data is on a different numerical scale than the fitted data. Other clustering methods like e.g., KMeans dont have this problem.

In the example below, I fitted the BisectingKMeans on random data and then multiplied the data that should be predicted by 50. It causes a floating point exception error which is pretty bad because python silently exits.

### Steps/Code to Reproduce

```py
from sklearn.cluster import BisectingKMeans, KMeans
import numpy as np
x = np.random.rand(3000, 10)

bisect_means = BisectingKMeans(n_clusters=10).fit(x)

labels = bisect_means.predict(50*np.random.rand(100, 10))
print(labels)
```

### Expected Results

A list of predicted class labels

### Actual Results

Python silently exits with:
```
[1]    45074 floating point exception  python foo.py
```
In Jupyter/ipython the kernel simply dies.

### Versions

```shell
System:
    python: 3.9.16 (main, Mar  8 2023, 04:29:44)  [Clang 14.0.6 ]
executable: /Users/jab/miniconda3/envs/qondot/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.3.0
          pip: 23.0.1
   setuptools: 67.8.0
        numpy: 1.21.6
        scipy: 1.10.1
       Cython: 0.29.35
       pandas: 1.5.3
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/numpy/.dylibs/libopenblas.0.dylib
        version: 0.3.17
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 4
```

COMMENT:
Thanks! It also persists when setting the OMP threads to 1.
I'm not familiar with debugging C or Cython code. I can try setting debug statements manually but the error originates from a binary (`_k_means_lloyd.cpython-311-darwin.so*`) and I dont know how to debug that

Labels: Bug, Needs Triage
BisectingKMeans floating point exception fatal error on OSX

### Describe the bug

Hi, thanks for this fantastic library 🚀 🎉 

I believe I found a bug. In the BisectingKMeans clustering, the `predict` function cannot be used when the to-be-predicted-data is on a different numerical scale than the fitted data. Other clustering methods like e.g., KMeans dont have this problem.

In the example below, I fitted the BisectingKMeans on random data and then multiplied the data that should be predicted by 50. It causes a floating point exception error which is pretty bad because python silently exits.

### Steps/Code to Reproduce

```py
from sklearn.cluster import BisectingKMeans, KMeans
import numpy as np
x = np.random.rand(3000, 10)

bisect_means = BisectingKMeans(n_clusters=10).fit(x)

labels = bisect_means.predict(50*np.random.rand(100, 10))
print(labels)
```

### Expected Results

A list of predicted class labels

### Actual Results

Python silently exits with:
```
[1]    45074 floating point exception  python foo.py
```
In Jupyter/ipython the kernel simply dies.

### Versions

```shell
System:
    python: 3.9.16 (main, Mar  8 2023, 04:29:44)  [Clang 14.0.6 ]
executable: /Users/jab/miniconda3/envs/qondot/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.3.0
          pip: 23.0.1
   setuptools: 67.8.0
        numpy: 1.21.6
        scipy: 1.10.1
       Cython: 0.29.35
       pandas: 1.5.3
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/numpy/.dylibs/libopenblas.0.dylib
        version: 0.3.17
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 4
```

COMMENT:
That's great news!! 🚀 Very satisfying you can finally reproduce it.

Interestingly, when I install from source, then add **any** print statement to `_k_means_lloyd.pyx`, then re-cythonize via `cythonize -i sklearn/cluster/_k_means_lloyd.pyx` even running it via python (`python debug_bkmeans.py`) gives me the same log trace you are observing now with `lldb`

Labels: Bug, Needs Triage
BisectingKMeans floating point exception fatal error on OSX

### Describe the bug

Hi, thanks for this fantastic library 🚀 🎉 

I believe I found a bug. In the BisectingKMeans clustering, the `predict` function cannot be used when the to-be-predicted-data is on a different numerical scale than the fitted data. Other clustering methods like e.g., KMeans dont have this problem.

In the example below, I fitted the BisectingKMeans on random data and then multiplied the data that should be predicted by 50. It causes a floating point exception error which is pretty bad because python silently exits.

### Steps/Code to Reproduce

```py
from sklearn.cluster import BisectingKMeans, KMeans
import numpy as np
x = np.random.rand(3000, 10)

bisect_means = BisectingKMeans(n_clusters=10).fit(x)

labels = bisect_means.predict(50*np.random.rand(100, 10))
print(labels)
```

### Expected Results

A list of predicted class labels

### Actual Results

Python silently exits with:
```
[1]    45074 floating point exception  python foo.py
```
In Jupyter/ipython the kernel simply dies.

### Versions

```shell
System:
    python: 3.9.16 (main, Mar  8 2023, 04:29:44)  [Clang 14.0.6 ]
executable: /Users/jab/miniconda3/envs/qondot/bin/python
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.3.0
          pip: 23.0.1
   setuptools: 67.8.0
        numpy: 1.21.6
        scipy: 1.10.1
       Cython: 0.29.35
       pandas: 1.5.3
   matplotlib: 3.7.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/numpy/.dylibs/libopenblas.0.dylib
        version: 0.3.17
threading_layer: pthreads
   architecture: Haswell
    num_threads: 4

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/jab/miniconda3/envs/qondot/lib/python3.9/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: SkylakeX
    num_threads: 4
```

COMMENT:
Actually, I think I understand how I could get the exception above. this might not be related to lldb as I first thought but more because I was running `cython -a sklearn/cluster/_k_means_lloyd.pyx` manually instead of the Cython config of our `setup.py` which can turn `ZeroDivisionError` into lower level crashes.

Still, I don't understand why the macOS Intel C/C++ compiler is the only one to actually generate code that crashes on integer division by zero...

Labels: Bug, Needs Triage
⚠️ CI failed on Linux.pylatest_pip_openblas_pandas ⚠️

**CI failed on [Linux.pylatest_pip_openblas_pandas](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=57986&view=logs&j=78a0bf4f-79e5-5387-94ec-13e67d216d6e)** (Aug 17, 2023)
- test_pairwise_distances_argkmin[45-float32-parallel_on_X-cityblock-0-500]
- test_pairwise_distances_argkmin[45-float32-parallel_on_Y-cityblock-0-500]

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=58166&view=logs&j=78a0bf4f-79e5-5387-94ec-13e67d216d6e) on Aug 22, 2023

Labels: Needs Triage
⚠️ CI failed on Linux.pylatest_pip_openblas_pandas ⚠️

**CI failed on [Linux.pylatest_pip_openblas_pandas](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=57986&view=logs&j=78a0bf4f-79e5-5387-94ec-13e67d216d6e)** (Aug 17, 2023)
- test_pairwise_distances_argkmin[45-float32-parallel_on_X-cityblock-0-500]
- test_pairwise_distances_argkmin[45-float32-parallel_on_Y-cityblock-0-500]

COMMENT:
Closing as duplicate of #27126 which seems related. Let's reopen if it isn't.

Labels: Needs Triage
Numpy 2.0 release thoughts

Some notes from the EuroSciPy maintainer track about Numpy 2.0

You can track the progress Numpy is making towards 2.0 and what changes are coming in https://github.com/numpy/numpy/issues/24300.

Some things we should consider doing:

* [ ] pin to numpy<2 in the next releases to reduce the number of people who will install a version of scikit-learn that is incompatible with the numpy 2 version
* [ ] add some code in scikit-learn that detects that numpy 2 is installed and output a good error message for users
   * Olivier: do we really need this? Maybe the upper-bound pinning is enough.
* numpy 2 might come out at the end of 2023 or early 2024
* [ ] modify existing conda-forge packages to include a numpy<2 constraint
  * unclear if we can do this on PyPi as well
  * https://github.com/conda-forge/scikit-learn-feedstock/issues/227
* [x] once numpy 2.0 has been released we should build against that instead of "oldest-supported-numpy"
  * already do this now in the nightlies?
  * make sure that this works while still having 1.x.quite-old as the minimum required numpy
  * We no longer build against `oldest-supported-numpy` for Python 3.9 thanks to #27735 that was merged in `main` before the release of scikit-learn 1.4.

cc @ogrisel

COMMENT:
well, last we talked about putting upper bounds, we decided not to 😁  (xref: https://github.com/scikit-learn/scikit-learn/issues/26869)

Labels: Packaging
Numpy 2.0 release thoughts

Some notes from the EuroSciPy maintainer track about Numpy 2.0

You can track the progress Numpy is making towards 2.0 and what changes are coming in https://github.com/numpy/numpy/issues/24300.

Some things we should consider doing:

* [ ] pin to numpy<2 in the next releases to reduce the number of people who will install a version of scikit-learn that is incompatible with the numpy 2 version
* [ ] add some code in scikit-learn that detects that numpy 2 is installed and output a good error message for users
   * Olivier: do we really need this? Maybe the upper-bound pinning is enough.
* numpy 2 might come out at the end of 2023 or early 2024
* [ ] modify existing conda-forge packages to include a numpy<2 constraint
  * unclear if we can do this on PyPi as well
  * https://github.com/conda-forge/scikit-learn-feedstock/issues/227
* [x] once numpy 2.0 has been released we should build against that instead of "oldest-supported-numpy"
  * already do this now in the nightlies?
  * make sure that this works while still having 1.x.quite-old as the minimum required numpy
  * We no longer build against `oldest-supported-numpy` for Python 3.9 thanks to #27735 that was merged in `main` before the release of scikit-learn 1.4.

cc @ogrisel

COMMENT:
Yeah, I'd still vote against upper limits though. It feels like this is a once in a life time situation. (I hope this comment will age well ;) ).

A thing that will make this tricky is that a lot of the changes for Numpy 2.0 aren't yet merged. This makes it hard to find out how much breakage there will be. I still hope that maybe older versions of scikit-learn can "just work" with Numpy 2.0 if we rebuild the wheels against Numpy 2 as part of a patch release? But yeah.

Labels: Packaging