Package: kmedoids
k-Medoids Clustering in Python with FasterPAMThis python package implements k-medoids clustering with PAM and variants of clustering by direct optimization of the (Medoid) Silhouette.
It can be used with arbitrary dissimilarites, as it requires a dissimilarity matrix as input.This software package has been introduced in JOSS:Erich Schubert and Lars LenssenFast k-medoids Clustering in Rust and PythonJournal of Open Source Software 7(75), 4183https://doi.org/10.21105/joss.04183(open access)For further details on the implemented algorithm FasterPAM, see:Erich Schubert, Peter J. RousseeuwFast and Eager k-Medoids Clustering:O(k) Runtime Improvement of the PAM, CLARA, and CLARANS AlgorithmsInformation Systems (101), 2021, 101804https://doi.org/10.1016/j.is.2021.101804(open access)an earlier (slower, and now obsolete) version was published as:Erich Schubert, Peter J. Rousseeuw:Faster k-Medoids Clustering: Improving the PAM, CLARA, and CLARANS AlgorithmsIn: 12th International Conference on Similarity Search and Applications (SISAP 2019), 171-187.https://doi.org/10.1007/978-3-030-32047-8_16Preprint:https://arxiv.org/abs/1810.05691This is a port of the original Java code fromELKIto Rust.
TheRust versionis then wrapped for use with Python.For further details on medoid Silhouette clustering with automatic cluster number selection (FasterMSC, DynMSC), see:Lars Lenssen, Erich Schubert:Medoid silhouette clustering with automatic cluster number selectionInformation Systems (120), 2024, 102290https://doi.org/10.1016/j.is.2023.102290Preprint:https://arxiv.org/abs/2309.03751the basic FasterMSC method was first published as:Lars Lenssen, Erich Schubert:Clustering by Direct Optimization of the Medoid SilhouetteIn: 15th International Conference on Similarity Search and Applications (SISAP 2022)https://doi.org/10.1007/978-3-031-17849-8_15If you use this code in scientific work, please cite above papers. Thank you.DocumentationFull python documentation is included, and available onpython-kmedoids.readthedocs.ioInstallationInstallation with pip or condaPre-built packages for many Linux, Windows, and OSX systems are available
inPyPIandconda-forgecan be installed withpip install kmedoidsrespectivelyconda install -c conda-forge kmedoids.On uncommon architectures, you may need to firstinstall Cargo(i.e., the Rust programming language) first, and a subsequentpip install kmedoidswill try to compile the package for your CPU architecture and operating system.Compilation from sourceYou need to have Python 3 installed.Unless you already have Rust,install Rust/Cargo.Installation usesmaturinfor compiling and installing the Rust extension.
Maturin is best used within a Python virtual environment:# activate your desired virtual environment first, then:pipinstallmaturin
gitclonehttps://github.com/kno10/python-kmedoids.gitcdpython-kmedoids# build and install the package:maturindevelop--releaseIntegration test to validate the installation.pipinstallnumpy
python-munittestdiscovertestsThis procedure uses the latest git version fromhttps://github.com/kno10/rust-kmedoids.
If you want to use local modifications to the Rust code, you need to provide the source folder of the Rust module inCargo.tomlby setting thepath=option of thekmedoidsdependency.ExampleGiven a distance matrixdistmatrix, cluster intok = 5clusters:importkmedoidsc=kmedoids.fasterpam(distmatrix,5)print("Loss is:",c.loss)Using the sklearn-compatible APINote that KMedoids defaults to the"precomputed"metric, expecting a pairwise distance matrix.
If you have sklearn installed, you can also usemetric="euclidean"and other distances supported by sklearn.importkmedoidskm=kmedoids.KMedoids(5,method='fasterpam')c=km.fit(distmatrix)print("Loss is:",c.inertia_)MNIST (10k samples)importkmedoids,numpy,timefromsklearn.datasetsimportfetch_openmlfromsklearn.metrics.pairwiseimporteuclidean_distancesX,_=fetch_openml('mnist_784',version=1,return_X_y=True,as_frame=False)X=X[:10000]diss=euclidean_distances(X)start=time.time()fp=kmedoids.fasterpam(diss,100)print("FasterPAM took:%.2fms"%((time.time()-start)*1000))print("Loss with FasterPAM:",fp.loss)start=time.time()pam=kmedoids.pam(diss,100)print("PAM took:%.2fms"%((time.time()-start)*1000))print("Loss with PAM:",pam.loss)Choose the optimal number of clustersThis package includes DynMSC, an algorithm that optimizes the Medoid Silhouette,
and chooses the "optimal" number of clusters in a range of 2..kmax.
Beware that if you allow a too large kmax, the optimum result will likely have many
one-elemental clusters. A too high kmax may mask more desirable results, hence it
is recommended that you choose only 2-3 times the number of clusters you expect as maximum.importkmedoids,numpyfromsklearn.datasetsimportfetch_openmlfromsklearn.metrics.pairwiseimporteuclidean_distancesX,_=fetch_openml('mnist_784',version=1,return_X_y=True,as_frame=False)X=X[:10000]diss=euclidean_distances(X)dm=kmedoids.dynmsc(diss,100)print("Optimal number of clusters according to the Medoid Silhouette:",dm.bestk)print("Medoid Silhouette over range of k:",dm.losses)print("Range of k:",dm.rangek)Memory RequirementsBecause the algorithms require a distance matrix as input, you need O(N²) memory to use these implementations. With single precision, this matrix needs 4·N² bytes, so a typical laptop with 8 GB of RAM could handle data sets of over 40.000 instances, but if your computation of the distance matrix incurs copying the matrix, only 30.000 or less may be feasible.The majority of run time usually is the distance matrix computation, so it is recommended you only compute it once, then experiment with different algorithm settings. Avoid recomputing it repeatedly.For larger data sets, it is recommended to only cluster a representative sample of the data. Usually, this will still yield sufficient result quality.Implemented AlgorithmsFasterPAM(Schubert and Rousseeuw, 2020, 2021)FastPAM1 (Schubert and Rousseeuw, 2019, 2021)PAM (Kaufman and Rousseeuw, 1987) with BUILD and SWAPAlternating optimization (k-means-style algorithm)Silhouette index for evaluation (Rousseeuw, 1987)FasterMSC(Lenssen and Schubert, 2022)FastMSC (Lenssen and Schubert, 2022)DynMSC(Lenssen and Schubert, 2023)PAMSIL (Van der Laan and Pollard, 2003)PAMMEDSIL (Van der Laan and Pollard, 2003)Medoid Silhouette index for evaluation (Van der Laan and Pollard, 2003)Note that the k-means-like algorithm for k-medoids tends to find much worse solutions.Contributing topython-kmedoidsThird-party contributions are welcome. Please usepull requeststo submit patches.Reporting issuesPlease report errors as anissuewithin the repository's issue tracker.Support requestsIf you need help, please submit anissuewithin the repository's issue tracker.License: GPL-3 or laterThis program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.You should have received a copy of the GNU General Public License
along with this program.  If not, seehttps://www.gnu.org/licenses/.