New clustering metrics

### Describe the workflow you want to enable

Scikit-learn defines three popular metrics for evaluating clustering performance when there are no ground-truth cluster labels: [sklearn.metrics.silhouette_score](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.silhouette_score.html), [sklearn.metrics.calinski_harabasz_score](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.calinski_harabasz_score.html) and [sklearn.metrics.davies_bouldin_score](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.davies_bouldin_score.html). But there are lots of others, and it's previously been [discussed](https://github.com/scikit-learn/scikit-learn/discussions/21164) whether to integrate more into scikit-learn.

### Describe your proposed solution

I've implemented four relatively popular ones, using the same interface and code style as in https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/metrics/cluster/_unsupervised.py. Would there be interest in integrating these into scikit-learn?

```python
import numpy as np
from itertools import combinations
from sklearn.utils import check_X_y
from sklearn.metrics.cluster._unsupervised import check_number_of_labels
from sklearn.preprocessing import LabelEncoder


def log_ss_ratio(X, labels):
    X, labels = check_X_y(X, labels)
    le = LabelEncoder()
    labels = le.fit_transform(labels)

    n_samples, _ = X.shape
    n_labels = len(le.classes_)

    check_number_of_labels(n_labels, n_samples)

    extra_disp, intra_disp = 0.0, 0.0
    mean = X.mean(axis=0)
    for k in range(n_labels):
        cluster_k = X[labels == k]
        mean_k = cluster_k.mean(axis=0)
        extra_disp += len(cluster_k) * ((mean_k - mean) ** 2).sum()
        intra_disp += ((cluster_k - mean_k) ** 2).sum()

    return np.log(extra_disp / intra_disp)


def ball_hall(X, labels):
    X, labels = check_X_y(X, labels)
    le = LabelEncoder()
    labels = le.fit_transform(labels)

    n_samples, _ = X.shape
    n_labels = len(le.classes_)

    check_number_of_labels(n_labels, n_samples)

    sum_mean_dispersions = 0

    for k in range(n_labels):
        cluster_k = X[labels == k]
        mean_k = cluster_k.mean(axis=0)
        intra_disp_cluster = ((cluster_k - mean_k) ** 2).sum()
        sum_mean_dispersions += intra_disp_cluster / len(cluster_k)

    return sum_mean_dispersions / n_labels


def banfeld_raftery(X, labels):
    X, labels = check_X_y(X, labels)
    le = LabelEncoder()
    labels = le.fit_transform(labels)

    n_samples, _ = X.shape
    n_labels = len(le.classes_)

    check_number_of_labels(n_labels, n_samples)

    br_index = 0
    for k in range(n_labels):
        cluster_k = X[labels == k]
        mean_k = cluster_k.mean(axis=0)
        intra_disp_cluster = ((cluster_k - mean_k) ** 2).sum()
        br_index += len(cluster_k) * \
                    np.log(intra_disp_cluster / len(cluster_k))

    return br_index


def ray_turi(X, labels):
    X, labels = check_X_y(X, labels)
    le = LabelEncoder()
    labels = le.fit_transform(labels)

    n_samples, _ = X.shape
    n_labels = len(le.classes_)

    check_number_of_labels(n_labels, n_samples)

    intra_disp = 0.0

    cluster_means = []
    for k in range(n_labels):
        cluster_k = X[labels == k]
        mean_k = cluster_k.mean(axis=0)
        intra_disp += ((cluster_k - mean_k) ** 2).sum()
        cluster_means.append(mean_k)

    min_cluster_mean_diff = min(
        ((mean_i - mean_j) ** 2).sum()
        for mean_i, mean_j in combinations(cluster_means, 2))

    return intra_disp / (min_cluster_mean_diff * n_samples)
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
So I went on a bit of a deep dive through a few implementations of unsupervised cluster evaluation metrics:

- clusterCrit (https://rdrr.io/cran/clusterCrit/f/inst/doc/clusterCrit.pdf)
- Clusters-Features (https://github.com/Simon-Bertrand/Clusters-Features)
- cclust (https://www.rdocumentation.org/packages/cclust/versions/0.6-26/topics/clustIndex)
- NbClust (https://hal.science/hal-01126138/document)
- evaluclusters (https://www.mathworks.com/help/stats/evalclusters.html)

and found 37 metrics. I looked at their number of Google hits and their number of citations. Keep in mind that some papers are cited for things other than the clustering metric. I've bolded the ones that particularly stand out on both popularity metrics.

1. `"Baker-Hubert gamma index" clustering`: 3,370 Google hits (3,330 for `"Hubert index" clustering`, 3,360 for `"Hubert gamma index" clustering`), [379 citations](https://scholar.google.com/scholar?cites=15348286302309694518)
2. `"Ball-Hall index" clustering`: 904 Google hits (15,400 for `"Ball index" clustering`), [1,681 citations](https://scholar.google.com/scholar?cites=14408990917969310556)
3. `"Banfield-Raftery index" clustering`: 10 Google hits (but 463 for the misspelling `"Banfeld-Raftery index" clustering`), [3,185 citations](https://scholar.google.com/scholar?cites=18020478635454549936)
4. `"Beale index" clustering`: 1,940 Google hits, [444 citations](https://scholar.google.com/scholar?cites=203759068857086091)
5. `"C index" clustering`: 224,000 Google hits (may include false positives talking about AUROC, which is also a "C index", but `"C index" clustering Hubert Levin` gets 8,510 results), [486 citations](https://scholar.google.com/scholar?cites=10733806999993347905)
6. **`"Calinski-Harabasz index" clustering`: 24,200 Google hits (already in scikit-learn), [8,244 citations](https://scholar.google.com/scholar?cites=17038427928303078953)**
7. `"Cubic Clustering Criterion" clustering`: 11,400 Google hits, [604 citations](https://scholar.google.com/scholar?cites=13569346661065183741)
8. **`"D index" clustering`: 45,100 Google hits, [3,033 citations](https://scholar.google.com/scholar?cites=10039818796919680418)**
9. **`"Davies-Bouldin index" clustering`: 72,000 Google hits (already in scikit-learn), [9,078 citations](https://scholar.google.com/scholar?cites=11992317260814516272)**
10. `"Det Ratio index" clustering`: 1,180 Google hits, [654 citations](https://scholar.google.com/scholar?cites=9459893636282660101) - this paper also introduced the Log Det Ratio and Scott-Symons indices
11. `"Duda index" clustering`: 294 Google hits, [26,694 citations](https://scholar.google.com/scholar?cites=8346018495684415412) - this paper also introduced the pseudo t2 index
12. **`"Dunn index" clustering`: 51,900 Google hits, [3,097 citations](https://scholar.google.com/scholar?cites=6061737653495083855)**
13. `"Frey index" clustering`: 557 Google hits, ?? citations (paper doesn't seem to be available on the internet)
14. `"Friedman index" clustering`: 869 Google hits (74 for `"Trace WiB index" clustering`), [943 citations](https://scholar.google.com/scholar?cites=500476383924244999) - this paper also introduced the Rubin index
15. **`"Gap index" clustering`: 11,300 Google hits, [6,673 citations](https://scholar.google.com/scholar?cites=12551704574370620859)**
16. `"Generalized Dunn index" clustering`: 2,640 Google hits, [1,487 citations](https://scholar.google.com/scholar?cites=8323637445052435102)
17. `"G plus index" clustering`: 548 Google hits, [226 citations](https://scholar.google.com/scholar?cites=14836392976447969654) - this paper also introduced the Tau index
18. `"Hartigan index" clustering`: 2,390 Google hits (10 for "Log SS Ratio index"), [47 citations](https://scholar.google.com/scholar?cites=16039191443986173748)
19. `"KL index" clustering`: 1,620 Google hits (1,650 for `"Krzanowski-Lai index" clustering`), [899 citations](https://scholar.google.com/scholar?cites=12017686831308160193)
20. `"Log Det Ratio index" clustering`: 68 Google hits, [654 citations](https://scholar.google.com/scholar?cites=9459893636282660101) - this paper also introduced the Det Ratio and Scott-Symons indices
21. `"Marriot index" clustering`: 344 Google hits (1,260 for `"Marriott index" clustering`, 10 for `"Ksq DetW index" clustering`), [370 citations](https://scholar.google.com/scholar?cites=11377514914623101104)
22. `"McClain-Rao index" clustering`: 238 Google hits (377 for `"McClain index" clustering`), [176 citations](https://scholar.google.com/scholar?cites=17812421010414755917)
23. `"PBM index" clustering`: 3,730 Google hits, [956 citations](https://scholar.google.com/scholar?cites=11655871862450295604)
24. `"Point biserial index" clustering`: 1,710 Google hits, [575 citations](https://scholar.google.com/scholar?cites=14274723426945092878)
25. `"Pseudo t2 index" clustering`: 135 Google hits, [26,694 citations](https://scholar.google.com/scholar?cites=8346018495684415412) - this paper also introduced the Duda index
26. `"Ratkowsky-Lance index" clustering`: 174 Google hits (116 for `"Ratkowsky index" clustering`), [171 citations](https://scholar.google.com/scholar?cites=3682668732682499631)
27. `"Ray-Turi index" clustering`: 729 Google hits, [932 citations](https://scholar.google.com/scholar?cites=167414037530955163)
28. `"Rubin index" clustering`: 653 Google hits, [943 citations](https://scholar.google.com/scholar?cites=500476383924244999) - this paper also introduced the Friedman index
29. `"Scott-Symons index" clustering`: 1,230 Google hits (4,690 for `"Scott index" clustering`), [654 citations](https://scholar.google.com/scholar?cites=9459893636282660101) - this paper also introduced the Det Ratio and Log Det Ratio indices
30. `"SD index" clustering`: 12,400 Google hits (6,470 for `"SDindex" clustering`), [466 citations](https://scholar.google.com/scholar?cites=9489367793537799452)
31. `"S Dbw index" clustering`: 1,280 Google hits (954 for `"S_Dbw index" clustering`), [610 citations](https://scholar.google.com/scholar?cites=7938287841551072760)
32. **`"Silhouette index" clustering`: 51,900 Google hits (68,000 for `"Silhouette score" clustering`; already in scikit-learn), [18,202 citations](https://scholar.google.com/scholar?cites=16489762663333339453)**
33. `"Tau index" clustering`: 3,530 Google hits, [226 citations](https://scholar.google.com/scholar?cites=14836392976447969654) - this paper also introduced the G plus index
34. `"Trace W index" clustering`: 97 Google hits (170 for `"TraceW index" clustering`), [785 citations](https://scholar.google.com/scholar?cites=1154177259440894019)
35. `"trcovw index" clustering`: 157 Google hits, [5,119 citations](https://scholar.google.com/scholar?cites=10225087707023129378) - this may not be the earliest citation
36. `"Wemmert-Gancarski index" clustering`: 170 Google hits, [52 citations](https://scholar.google.com/scholar?cites=1651458708738915177)
37. **`"Xie-Beni index" clustering`: 9,000 Google hits, [4,380 citations](https://scholar.google.com/scholar?cites=3788068846610361775)**

Besides the three in scikit-learn (Silhouette, Davies-Bouldin and Calinski-Harabasz), the ones that stand out are the D index, Dunn index, Gap index, and Xie-Beni index. However, the D index citation is a book that seems to be mostly cited for things other than the index. 

To sum up, the most popular unsupervised clustering metrics missing from Scikit-learn seem to be the Dunn index, Gap index, and Xie-Beni index. However, maybe the even bigger take-away here is that there are _a lot_ of widely used and widely cited metrics, and it might be worth supporting a more comprehensive set than just Dunn, Gap and Xie-Beni.

Labels: New Feature, Needs Decision - Include Feature