Talip7/scikit-learn-issues-multilabel huggingface dataset:
Different HDBSCAN clusters from scikit-learn and scikit-learn-contrib packages

### Describe the bug

The `HDBSCAN()` functions provided by [scikit-learn-contrib/hdbscan](https://github.com/scikit-learn-contrib/hdbscan) and this package can give different clustering results, e.g. when using the **`cluster_selection_epsilon`** parameter.

### Steps/Code to Reproduce

```python
# run this with only one uncommented, then run again with the other:
from hdbscan import HDBSCAN
# from sklearn.cluster import HDBSCAN

import numpy as np
from sklearn.datasets import make_blobs

data, _ = make_blobs(1000, centers=30, random_state=3)

clusterer = HDBSCAN(min_cluster_size=10, cluster_selection_epsilon=1.3)
cluster_labels = clusterer.fit_predict(data)

print(np.unique(cluster_labels))
```

### Expected Results

`hdbscan.HDBSCAN` output:

```python
[-1  0  1  2]
```

### Actual Results

`sklearn.cluster.HDBSCAN` output:

```python
[-1  0  1]
```

### Versions

```shell
System:
    python: 3.12.0 | packaged by conda-forge | (main, Oct  3 2023, 08:43:22) [GCC 12.3.0]
executable: /home/tom/miniforge3/envs/datasci/bin/python
   machine: Linux-4.4.0-19041-Microsoft-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.3.2
          pip: 23.3.1
   setuptools: 68.2.2
        numpy: 1.26.0
        scipy: 1.11.3
       Cython: 3.0.5
       pandas: 2.1.1
   matplotlib: 3.8.0
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/tom/miniforge3/envs/datasci/lib/libopenblasp-r0.3.24.so
        version: 0.3.24
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /home/tom/miniforge3/envs/datasci/lib/libgomp.so.1.0.0
        version: None
```

COMMENT:
I can reproduce.

Labels: Bug, help wanted
Different HDBSCAN clusters from scikit-learn and scikit-learn-contrib packages

### Describe the bug

The `HDBSCAN()` functions provided by [scikit-learn-contrib/hdbscan](https://github.com/scikit-learn-contrib/hdbscan) and this package can give different clustering results, e.g. when using the **`cluster_selection_epsilon`** parameter.

### Steps/Code to Reproduce

```python
# run this with only one uncommented, then run again with the other:
from hdbscan import HDBSCAN
# from sklearn.cluster import HDBSCAN

import numpy as np
from sklearn.datasets import make_blobs

data, _ = make_blobs(1000, centers=30, random_state=3)

clusterer = HDBSCAN(min_cluster_size=10, cluster_selection_epsilon=1.3)
cluster_labels = clusterer.fit_predict(data)

print(np.unique(cluster_labels))
```

### Expected Results

`hdbscan.HDBSCAN` output:

```python
[-1  0  1  2]
```

### Actual Results

`sklearn.cluster.HDBSCAN` output:

```python
[-1  0  1]
```

### Versions

```shell
System:
    python: 3.12.0 | packaged by conda-forge | (main, Oct  3 2023, 08:43:22) [GCC 12.3.0]
executable: /home/tom/miniforge3/envs/datasci/bin/python
   machine: Linux-4.4.0-19041-Microsoft-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.3.2
          pip: 23.3.1
   setuptools: 68.2.2
        numpy: 1.26.0
        scipy: 1.11.3
       Cython: 3.0.5
       pandas: 2.1.1
   matplotlib: 3.8.0
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/tom/miniforge3/envs/datasci/lib/libopenblasp-r0.3.24.so
        version: 0.3.24
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /home/tom/miniforge3/envs/datasci/lib/libgomp.so.1.0.0
        version: None
```

COMMENT:
We have a discrepancy when computing the distance to the nearest neighbors where we account for 1 neighbors less:

https://github.com/scikit-learn/scikit-learn/blob/1bcbc00da6a859f57cc5d44cf12dbbf42ac3140b/sklearn/cluster/_hdbscan/hdbscan.py#L341

while the original code is:

```python
    core_distances = tree.query(
        X, k=min_samples + 1, dualtree=True, breadth_first=True
    )[0][:, -1].copy(order="C")
```

My intuition to add 1 neighbor is that the closest neighbor at `fit` is the sample it self so, if you want to get the 5-th nearest neighbors, you need to query 6.

@Micky774 Would you have time to confirm that.

Labels: Bug, help wanted
Different HDBSCAN clusters from scikit-learn and scikit-learn-contrib packages

### Describe the bug

The `HDBSCAN()` functions provided by [scikit-learn-contrib/hdbscan](https://github.com/scikit-learn-contrib/hdbscan) and this package can give different clustering results, e.g. when using the **`cluster_selection_epsilon`** parameter.

### Steps/Code to Reproduce

```python
# run this with only one uncommented, then run again with the other:
from hdbscan import HDBSCAN
# from sklearn.cluster import HDBSCAN

import numpy as np
from sklearn.datasets import make_blobs

data, _ = make_blobs(1000, centers=30, random_state=3)

clusterer = HDBSCAN(min_cluster_size=10, cluster_selection_epsilon=1.3)
cluster_labels = clusterer.fit_predict(data)

print(np.unique(cluster_labels))
```

### Expected Results

`hdbscan.HDBSCAN` output:

```python
[-1  0  1  2]
```

### Actual Results

`sklearn.cluster.HDBSCAN` output:

```python
[-1  0  1]
```

### Versions

```shell
System:
    python: 3.12.0 | packaged by conda-forge | (main, Oct  3 2023, 08:43:22) [GCC 12.3.0]
executable: /home/tom/miniforge3/envs/datasci/bin/python
   machine: Linux-4.4.0-19041-Microsoft-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.3.2
          pip: 23.3.1
   setuptools: 68.2.2
        numpy: 1.26.0
        scipy: 1.11.3
       Cython: 3.0.5
       pandas: 2.1.1
   matplotlib: 3.8.0
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/tom/miniforge3/envs/datasci/lib/libopenblasp-r0.3.24.so
        version: 0.3.24
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /home/tom/miniforge3/envs/datasci/lib/libgomp.so.1.0.0
        version: None
```

COMMENT:
I confirm that adding one neighbors will provide the same results as the original implementation.

Labels: Bug, help wanted
Different HDBSCAN clusters from scikit-learn and scikit-learn-contrib packages

### Describe the bug

The `HDBSCAN()` functions provided by [scikit-learn-contrib/hdbscan](https://github.com/scikit-learn-contrib/hdbscan) and this package can give different clustering results, e.g. when using the **`cluster_selection_epsilon`** parameter.

### Steps/Code to Reproduce

```python
# run this with only one uncommented, then run again with the other:
from hdbscan import HDBSCAN
# from sklearn.cluster import HDBSCAN

import numpy as np
from sklearn.datasets import make_blobs

data, _ = make_blobs(1000, centers=30, random_state=3)

clusterer = HDBSCAN(min_cluster_size=10, cluster_selection_epsilon=1.3)
cluster_labels = clusterer.fit_predict(data)

print(np.unique(cluster_labels))
```

### Expected Results

`hdbscan.HDBSCAN` output:

```python
[-1  0  1  2]
```

### Actual Results

`sklearn.cluster.HDBSCAN` output:

```python
[-1  0  1]
```

### Versions

```shell
System:
    python: 3.12.0 | packaged by conda-forge | (main, Oct  3 2023, 08:43:22) [GCC 12.3.0]
executable: /home/tom/miniforge3/envs/datasci/bin/python
   machine: Linux-4.4.0-19041-Microsoft-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.3.2
          pip: 23.3.1
   setuptools: 68.2.2
        numpy: 1.26.0
        scipy: 1.11.3
       Cython: 3.0.5
       pandas: 2.1.1
   matplotlib: 3.8.0
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/tom/miniforge3/envs/datasci/lib/libopenblasp-r0.3.24.so
        version: 0.3.24
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /home/tom/miniforge3/envs/datasci/lib/libgomp.so.1.0.0
        version: None
```

COMMENT:
One thing that I am not sure is that for the mutual reachibility, it seems that we are also working with 1 neighbor less. I am wondering if we actually did this implementation on purpose.

Edit: this actually something that we expect as documented in the user guide:

> the distance to its min_samples th-nearest neighbor, counting itself.

So I don't know if we should consider as a bug and just better document the difference between the 2 implementations and advice for `min_cluster_size++` to get the same results.

Labels: Bug, help wanted
Different HDBSCAN clusters from scikit-learn and scikit-learn-contrib packages

### Describe the bug

The `HDBSCAN()` functions provided by [scikit-learn-contrib/hdbscan](https://github.com/scikit-learn-contrib/hdbscan) and this package can give different clustering results, e.g. when using the **`cluster_selection_epsilon`** parameter.

### Steps/Code to Reproduce

```python
# run this with only one uncommented, then run again with the other:
from hdbscan import HDBSCAN
# from sklearn.cluster import HDBSCAN

import numpy as np
from sklearn.datasets import make_blobs

data, _ = make_blobs(1000, centers=30, random_state=3)

clusterer = HDBSCAN(min_cluster_size=10, cluster_selection_epsilon=1.3)
cluster_labels = clusterer.fit_predict(data)

print(np.unique(cluster_labels))
```

### Expected Results

`hdbscan.HDBSCAN` output:

```python
[-1  0  1  2]
```

### Actual Results

`sklearn.cluster.HDBSCAN` output:

```python
[-1  0  1]
```

### Versions

```shell
System:
    python: 3.12.0 | packaged by conda-forge | (main, Oct  3 2023, 08:43:22) [GCC 12.3.0]
executable: /home/tom/miniforge3/envs/datasci/bin/python
   machine: Linux-4.4.0-19041-Microsoft-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.3.2
          pip: 23.3.1
   setuptools: 68.2.2
        numpy: 1.26.0
        scipy: 1.11.3
       Cython: 3.0.5
       pandas: 2.1.1
   matplotlib: 3.8.0
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/tom/miniforge3/envs/datasci/lib/libopenblasp-r0.3.24.so
        version: 0.3.24
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /home/tom/miniforge3/envs/datasci/lib/libgomp.so.1.0.0
        version: None
```

COMMENT:
I think adding some documentation to be aware of this fact is the best option. If I recall correctly this ``min_samples`` choice was made to match the DBSCAN implementation here (or, at least, it should). In the question of whether you count yourself as a neighbor the ``hdbscan`` package made a call a long time ago and then got stuck with that for compatibility going forward.

Labels: Bug, help wanted
Different HDBSCAN clusters from scikit-learn and scikit-learn-contrib packages

### Describe the bug

The `HDBSCAN()` functions provided by [scikit-learn-contrib/hdbscan](https://github.com/scikit-learn-contrib/hdbscan) and this package can give different clustering results, e.g. when using the **`cluster_selection_epsilon`** parameter.

### Steps/Code to Reproduce

```python
# run this with only one uncommented, then run again with the other:
from hdbscan import HDBSCAN
# from sklearn.cluster import HDBSCAN

import numpy as np
from sklearn.datasets import make_blobs

data, _ = make_blobs(1000, centers=30, random_state=3)

clusterer = HDBSCAN(min_cluster_size=10, cluster_selection_epsilon=1.3)
cluster_labels = clusterer.fit_predict(data)

print(np.unique(cluster_labels))
```

### Expected Results

`hdbscan.HDBSCAN` output:

```python
[-1  0  1  2]
```

### Actual Results

`sklearn.cluster.HDBSCAN` output:

```python
[-1  0  1]
```

### Versions

```shell
System:
    python: 3.12.0 | packaged by conda-forge | (main, Oct  3 2023, 08:43:22) [GCC 12.3.0]
executable: /home/tom/miniforge3/envs/datasci/bin/python
   machine: Linux-4.4.0-19041-Microsoft-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.3.2
          pip: 23.3.1
   setuptools: 68.2.2
        numpy: 1.26.0
        scipy: 1.11.3
       Cython: 3.0.5
       pandas: 2.1.1
   matplotlib: 3.8.0
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/tom/miniforge3/envs/datasci/lib/libopenblasp-r0.3.24.so
        version: 0.3.24
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /home/tom/miniforge3/envs/datasci/lib/libgomp.so.1.0.0
        version: None
```

COMMENT:
You're right, it's not just when using the `cluster_selection_epsilon` parameter as I had assumed. Removing that parameter from my example above also gives different answers:

```python
[-1  0  1  2  3]  # hdbscan.HDBSCAN
[-1  0  1  2  3  4  5]  # sklearn.cluster.HDBSCAN
```

and changing the `sklearn.cluster.HDBSCAN` version to `HDBSCAN(min_cluster_size=11)` gives the expected result:

```python
# using min_cluster_size=11
[-1  0  1  2  3]  # sklearn.cluster.HDBSCAN
```

Thanks for getting to the bottom of this.

Labels: Bug, help wanted
Different HDBSCAN clusters from scikit-learn and scikit-learn-contrib packages

### Describe the bug

The `HDBSCAN()` functions provided by [scikit-learn-contrib/hdbscan](https://github.com/scikit-learn-contrib/hdbscan) and this package can give different clustering results, e.g. when using the **`cluster_selection_epsilon`** parameter.

### Steps/Code to Reproduce

```python
# run this with only one uncommented, then run again with the other:
from hdbscan import HDBSCAN
# from sklearn.cluster import HDBSCAN

import numpy as np
from sklearn.datasets import make_blobs

data, _ = make_blobs(1000, centers=30, random_state=3)

clusterer = HDBSCAN(min_cluster_size=10, cluster_selection_epsilon=1.3)
cluster_labels = clusterer.fit_predict(data)

print(np.unique(cluster_labels))
```

### Expected Results

`hdbscan.HDBSCAN` output:

```python
[-1  0  1  2]
```

### Actual Results

`sklearn.cluster.HDBSCAN` output:

```python
[-1  0  1]
```

### Versions

```shell
System:
    python: 3.12.0 | packaged by conda-forge | (main, Oct  3 2023, 08:43:22) [GCC 12.3.0]
executable: /home/tom/miniforge3/envs/datasci/bin/python
   machine: Linux-4.4.0-19041-Microsoft-x86_64-with-glibc2.31

Python dependencies:
      sklearn: 1.3.2
          pip: 23.3.1
   setuptools: 68.2.2
        numpy: 1.26.0
        scipy: 1.11.3
       Cython: 3.0.5
       pandas: 2.1.1
   matplotlib: 3.8.0
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /home/tom/miniforge3/envs/datasci/lib/libopenblasp-r0.3.24.so
        version: 0.3.24
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /home/tom/miniforge3/envs/datasci/lib/libgomp.so.1.0.0
        version: None
```

COMMENT:
Happy to submit a PR for the small edit to the docs if that's the call, I'd like the opportunity to contribute!

Labels: Bug, help wanted
Issue with MeanShift?

### Describe the bug

Since I updated to python 3.11, Meanshift through me an error....


### Steps/Code to Reproduce

```python
from sklearn.cluster import MeanShift
import numpy as np
X = np.array([[1, 1], [2, 1], [1, 0],
              [4, 7], [3, 5], [3, 6]])
clustering = MeanShift(bandwidth=2).fit(X)
clustering.labels_
clustering.predict([[0, 0], [5, 5]])
clustering
```

### Expected Results

No error

### Actual Results

```pytb
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[9], [line 5](vscode-notebook-cell:?execution_count=9&line=5)
      [2](vscode-notebook-cell:?execution_count=9&line=2) import numpy as np
      [3](vscode-notebook-cell:?execution_count=9&line=3) X = np.array([[1, 1], [2, 1], [1, 0],
      [4](vscode-notebook-cell:?execution_count=9&line=4)               [4, 7], [3, 5], [3, 6]])
----> [5](vscode-notebook-cell:?execution_count=9&line=5) clustering = MeanShift(bandwidth=2).fit(X)
      [6](vscode-notebook-cell:?execution_count=9&line=6) clustering.labels_
      [7](vscode-notebook-cell:?execution_count=9&line=7) clustering.predict([[0, 0], [5, 5]])

File [~/miniconda3/lib/python3.11/site-packages/sklearn/base.py:1152](https://file+.vscode-resource.vscode-cdn.net/Users/slacour/LP/exogravity-obs/~/miniconda3/lib/python3.11/site-packages/sklearn/base.py:1152), in _fit_context.<locals>.decorator.<locals>.wrapper(estimator, *args, **kwargs)
   1145     estimator._validate_params()
   1147 with config_context(
   1148     skip_parameter_validation=(
   1149         prefer_skip_nested_validation or global_skip_validation
   1150     )
   1151 ):
-> 1152     return fit_method(estimator, *args, **kwargs)

File [~/miniconda3/lib/python3.11/site-packages/sklearn/cluster/_mean_shift.py:516](https://file+.vscode-resource.vscode-cdn.net/Users/slacour/LP/exogravity-obs/~/miniconda3/lib/python3.11/site-packages/sklearn/cluster/_mean_shift.py:516), in MeanShift.fit(self, X, y)
    514 for i, center in enumerate(sorted_centers):
    515     if unique[i]:
--> 516         neighbor_idxs = nbrs.radius_neighbors([center], return_distance=False)[
    517             0
    518         ]
...
--> 646 config = get_config().split()
    647 if config[0] == b"OpenBLAS":
    648     return config[1].decode("utf-8")

AttributeError: 'NoneType' object has no attribute 'split'
```

### Versions

```shell
System:
    python: 3.11.5 (main, Sep 11 2023, 08:17:37) [Clang 14.0.6 ]
executable: /Users/slacour/miniconda3/bin/python
   machine: macOS-14.0-arm64-arm-64bit

Python dependencies:
      sklearn: 1.3.2
          pip: 23.2.1
   setuptools: 68.0.0
        numpy: 1.24.4
        scipy: 1.11.1
       Cython: 3.0.5
       pandas: 2.1.3
   matplotlib: 3.8.1
       joblib: 1.2.0
threadpoolctl: 2.2.0

Built with OpenMP: True
```

COMMENT:
You only need to update `threadpoolctl`.

Labels: Bug, Needs Triage
Is it a good idea to have different definitions of cluster radius for BIRCH?

### Describe the workflow you want to enable

I want to enable different definitions of cluster radius, i.e., `threshold`, for `BIRCH` clustering algorithm implemented in scikit-learn. The cluster radius, i.e., `threshold` is arguably one of the most important parameters for `BIRCH` to decide how to cluster data, and it is now defined as the root mean squared value of Euclidean distances between data points and centroid in each cluster. (see [codes here](https://github.com/scikit-learn/scikit-learn/blob/3f89022fa04d293152f1d32fbc2a5bdaaf2df364/sklearn/cluster/_birch.py#L332-L358))

I believe this is not the optimal definition, and it is not reasonable in the below aspect.

A data point could be merged to its closest cluster when the distance of that data point to the cluster is much larger than the desired cluster radius, i.e., `threshold`.  This can easily happen when the cluster already have a lot of data points in it, leading to the fact that adding a new data point wouldn't really increase the as-defined radius of this cluster, even if the new data point is far away. (See the very simple example in snapshot below)![Screenshot 2023-11-20 at 10 41 22 PM](https://github.com/scikit-learn/scikit-learn/assets/54908836/11a69e39-c77e-4e3a-89e9-92dfbbf0b4a6)

The above unreasonable behavior is because of the definition of cluster radius, i.e., `threshold`, which I think should have more definitions to choose from or change to a more reasonable one.

### Describe your proposed solution

The cluster radius could be defined as the maximum allowed Euclidean distance from centroid to each data point in the cluster. This definition will solve the problem described above by making each cluster a shell-like shape with a "radius", instead of an arbitrary shape containing data points with distance to centroid larger than threshold. 

I think there could be more than one way to define this threshold, and I'm interested to know if there is any suggestions of easy-to-implement definitions. 

**I'm willing to submit a PR to change the definition of cluster radius or to allow different options. Before that, I would like to hear about the comments from scikit-learn maintainers. Thank you!**

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I cannot think of an appropriate solution for the problem I described here, so I'm closing this issue as of now.

Labels: New Feature, Needs Triage