HDBSCAN's `max_cluster_size` parameter has no effect

### Describe the bug

I am trying to apply HDBSCAN to a dataset in order to find clusters with a certain maximum size (e.g. 5), but the max_cluster_size parameter is not working (i.e. the result contains clusters bigger than 5).
As an example, I will generate the same dataset of the tutorial "[Demo of HDBSCAN clustering algorithm](https://scikit-learn.org/stable/auto_examples/cluster/plot_hdbscan.html#sphx-glr-auto-examples-cluster-plot-hdbscan-py)" I You can verify this by looking at the cluster size distribution plot at the end of the following code.

### Steps/Code to Reproduce

```python
import matplotlib.pyplot as plt
import numpy as np

from sklearn.cluster import DBSCAN, HDBSCAN
from sklearn.datasets import make_blobs

def plot(X, labels, probabilities=None, parameters=None, ground_truth=False, ax=None):
    if ax is None:
        _, ax = plt.subplots(figsize=(10, 4))
    labels = labels if labels is not None else np.ones(X.shape[0])
    probabilities = probabilities if probabilities is not None else np.ones(X.shape[0])
    # Black removed and is used for noise instead.
    unique_labels = set(labels)
    colors = [plt.cm.Spectral(each) for each in np.linspace(0, 1, len(unique_labels))]
    # The probability of a point belonging to its labeled cluster determines
    # the size of its marker
    proba_map = {idx: probabilities[idx] for idx in range(len(labels))}
    for k, col in zip(unique_labels, colors):
        if k == -1:
            # Black used for noise.
            col = [0, 0, 0, 1]

        class_index = np.where(labels == k)[0]
        for ci in class_index:
            ax.plot(
                X[ci, 0],
                X[ci, 1],
                "x" if k == -1 else "o",
                markerfacecolor=tuple(col),
                markeredgecolor="k",
                markersize=4 if k == -1 else 1 + 5 * proba_map[ci],
            )
    n_clusters_ = len(set(labels)) - (1 if -1 in labels else 0)
    preamble = "True" if ground_truth else "Estimated"
    title = f"{preamble} number of clusters: {n_clusters_}"
    if parameters is not None:
        parameters_str = ", ".join(f"{k}={v}" for k, v in parameters.items())
        title += f" | {parameters_str}"
    ax.set_title(title)
    plt.tight_layout()

centers = [[1, 1], [-1, -1], [1.5, -1.5]]
X, labels_true = make_blobs(
    n_samples=750, centers=centers, cluster_std=[0.4, 0.1, 0.75], random_state=0
)

param = {"cut_distance": 0.1}
hdb = HDBSCAN(min_cluster_size=2, max_cluster_size=5, cluster_selection_method='eom')
#help(HDBSCAN)
hdb.fit(X)
labels = hdb.dbscan_clustering(**param)
plot(X, labels, hdb.probabilities_)
plt.show()

# Cluster's size distribution
import pandas as pd
labels_series = pd.Series(labels)
labels_series = labels_series[labels_series != -1]
labels_counts = labels_series.value_counts()
plt.figure()
plt.hist(labels_counts, bins=len(labels_counts))
plt.ylabel('Counts')
plt.xlabel('Size')
plt.show()
```

### Expected Results

I expected only clusters with sizes within 2 and 5.

### Actual Results

There are clusters bigger than 5 (e.g. 6, 7, 12, 13, 14, 18, 20, 121 and 250).

### Versions

```shell
System:
    python: 3.9.18 | packaged by conda-forge | (main, Aug 30 2023, 03:49:32)  [GCC 12.3.0]
executable: /home/wecapstor1/caph/mppi146h/software/miniforge3/envs/fermipy/bin/python
   machine: Linux-4.18.0-477.13.1.el8_8.x86_64-x86_64-with-glibc2.28

Python dependencies:
      sklearn: 1.3.2
          pip: 23.3
   setuptools: 68.2.2
        numpy: 1.26.0
        scipy: 1.11.3
       Cython: None
       pandas: 2.1.1
   matplotlib: 3.8.0
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 48
         prefix: libopenblas
       filepath: /home/wecapstor1/caph/mppi146h/software/miniforge3/envs/fermipy/lib/libopenblasp-r0.3.24.so
        version: 0.3.24
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 48
         prefix: libgomp
       filepath: /home/wecapstor1/caph/mppi146h/software/miniforge3/envs/fermipy/lib/python3.9/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
So how should I change my code exactly? Thank you.

Best regards,
Giovanni
________________________________
From: Meekail Zain ***@***.***>
Sent: Thursday, November 16, 2023 9:22:05 PM
To: scikit-learn/scikit-learn ***@***.***>
Cc: giovannicozzolongo ***@***.***>; Author ***@***.***>
Subject: Re: [scikit-learn/scikit-learn] HDBSCAN's `max_cluster_size` parameter has no effect (Issue #27776)


Yes, as @glemaitre<https://github.com/glemaitre> mentioned the HDBSCAN.dbscan_clustering is a separate clustering strategy which is used to produce DBSCAN-like (flat uniform density) clusters, as opposed the hierarchical clusters normally produced by HDBSCAN (accessible via HDBSCAN.labels_).

Everything here is expected behavior from what I can see.

As an aside, note that the HDBSCAN.dbscan_clustering method recycles the core-distance MST computation done during HDBSCAN.fit, so it is significantly faster than DBSCAN in many instances, while producing approximately equivalent results. It can be useful for inspecting what a uniform density assertion would look like for different data (e.g. after producing a hierarchical clustering with HDBSCAN itself) but is not essential.

If you have any further questions, feel free to ask them here, but I will be closing the issue itself. Thanks all :)

—
Reply to this email directly, view it on GitHub<https://github.com/scikit-learn/scikit-learn/issues/27776#issuecomment-1815256026>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AS3NUKRL7MVPS4PMZQPVNKLYEZYW3AVCNFSM6AAAAAA7JINTHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVGI2TMMBSGY>.
You are receiving this because you authored the thread.Message ID: ***@***.***>

Labels: Bug