Package: genieclust
genieclustPackage for R and PythonGenie: Fast and Robust Hierarchical Clustering with Noise Point DetectionGenie finds meaningful clusters quickly – even on large data sets.A comprehensive tutorial, benchmarks, and a reference manual is available
athttps://genieclust.gagolewski.com/.When usinggenieclustin research publications, please
cite (Gagolewski, 2021) and (Gagolewski, Bartoszuk, Cena, 2016)
as specified below. Thank you.AboutA faster and more powerful version ofGenie– a robust and outlier
resistant clustering algorithm (see Gagolewski, Bartoszuk, Cena, 2016),
originally included in the R packagegenie.The idea behind Genie is beautifully simple. First, make each individual
point the only member of its own cluster. Then, keep merging pairs
of the closest clusters, one after another. However, toprevent
the formation of clusters of highly imbalanced sizesa point group of
thesmallestsize will sometimes be matched with its nearest neighbour.Genie's appealing simplicity goes hand in hand with its usability;
itoften outperforms other clustering approachessuch as K-means, BIRCH, or average, Ward, and complete linkage
onbenchmark data.
Of course, there is no, nor will there ever be, a single best
universal clustering approach for every kind of problem, but Genie
is definitely worth a try!Thanks to its being based on minimal spanning trees of the pairwise distance
graphs, Genie is alsovery fast– determining the whole cluster hierarchy
for datasets of millions of points can be completed within minutes. Therefore,
it is nicely suited for solvingextreme clustering tasks(large datasets
with any number of clusters to detect) for data (also sparse) that fit into
memory. Thanks to the use ofnmslib(if available), sparse or string inputs are also supported.It also allows clustering with respect to mutual reachability distances
so that it can act as anoise point detectoror a
robustified version ofHDBSCAN*(see Campello et al., 2013)
that is able to detect a predefined
number of clusters and hence it doesn't dependent on theDBSCAN's somewhat
difficult-to-setepsparameter.The package also features an implementation of economic inequality indices
(the Gini, Bonferroni index), external cluster validity measures
(e.g., the normalised clustering accuracy and partition similarity scores
such as the adjusted Rand, Fowlkes-Mallows, adjusted mutual information,
and the pair sets index), and internal cluster validity indices
(e.g., the Calinski-Harabasz, Davies-Bouldin, Ball-Hall, Silhouette,
and generalised Dunn indices).Author and ContributorsAuthor and Maintainer:Marek GagolewskiContributors:Maciej Bartoszuk,Anna Cena(R packagesgenieandCVI),Peter M. Larsen(rectangular_lsap).Examples, Tutorials, and DocumentationR's interface is compatible withstats::hclust(), but there is more.X<-...# some datah<-gclust(X)plot(h)# plot cluster dendrogramcutree(h,k=2)# or genie(X, k=2)To learn more about R, check out Marek's open-access (free!) textbookDeep R Programming.The Python language version ofgenieclusthas a familiarscikit-learn-like look-and-feel:importgenieclustX=...# some datag=genieclust.Genie(n_clusters=2)labels=g.fit_predict(X)Tutorials and the package documentation are availablehere.To learn more about Python, check out Marek's recent open-access (free!) textbookMinimalist Data Wrangling in Python.How to InstallPython VersionTo install viapip(seePyPI):pip3installgenieclustThe package requires Python 3.7+ together withcythonas well asnumpy,scipy,matplotlib, andscikit-learn.
Optional dependencies:nmslibandmlpack.R VersionTo install the most recent release, call:install.packages("genieclust")See the package entry onCRAN.OtherThe core functionality is implemented in the form of a header-only
C++ library. It can thus be easily adapted for use in
other environments.Any contributions are welcome (e.g., Julia, Matlab, ...).LicenseCopyright (C) 2018–2023 Marek Gagolewskihttps://www.gagolewski.comThis program is free software: you can redistribute it and/or modify it
under the terms of the GNU Affero General Public License Version 3, 19
November 2007, published by the Free Software Foundation.This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero
General Public License Version 3 for more details. You should have
received a copy of the License along with this program. If not, see
(https://www.gnu.org/licenses/).The filesrc/c_scipy_rectangular_lsap.his adapted from thescipyproject (https://scipy.org/scipylib/), source:/scipy/optimize/rectangular_lsap/rectangular_lsap.cpp.
Author: Peter M. Larsen. Distributed under the BSD-3-Clause license.The implementation of internal cluster validity measures
were adapted from our previous project (Gagolewski, Bartoszuk, Cena, 2021);
seeoptim_cvi.
Originally distributed under the GNU Affero General Public License Version 3.ReferencesGagolewski M., genieclust: Fast and robust hierarchical clustering,SoftwareX15, 2021, 100722.DOI: 10.1016/j.softx.2021.100722.https://genieclust.gagolewski.com.Gagolewski M., Bartoszuk M., Cena A., Genie: A new, fast, and
outlier-resistant hierarchical clustering algorithm,Information
Sciences363, 2016, 8–23.DOI: 10.1016/j.ins.2016.05.003.Gagolewski M., Bartoszuk M., Cena A., Are cluster validity measures (in)valid?,Information Sciences581, 2021, 620–636.DOI: 10.1016/j.ins.2021.10.004.Gagolewski M., Cena A., Bartoszuk M., Brzozowski L.,
Clustering with minimum spanning trees: How good can it be?, 2023,
under review (preprint),DOI: 10.48550/arXiv.2303.05679.Gagolewski M., Normalised clustering accuracy: An asymmetric external
cluster validity measure, 2023, under review (preprint),DOI: 10.48550/arXiv.2209.02935.Gagolewski M., A framework for benchmarking clustering algorithms,SoftwareX20, 2022, 101270.DOI: 10.1016/j.softx.2022.101270.https://clustering-benchmarks.gagolewski.com.Campello R.J.G.B., Moulavi D., Sander J.,
Density-based clustering based on hierarchical density estimates,Lecture Notes in Computer Science7819, 2013, 160–172.DOI: 10.1007/978-3-642-37456-2_14.Mueller A., Nowozin S., Lampert C.H., Information theoretic clustering
using minimum spanning trees,DAGM-OAGM, 2012.Rezaei M., Fränti P., Set matching measures for external cluster validity,IEEE Transactions on Knowledge and Data Engineering28(8), 2016,
2173–2186DOI: 10.1109/TKDE.2016.2551240.See the package'shomepagefor more
references.