Package: divik
divikPython implementation of Divisive iK-means (DiviK) algorithm.Tools within this packageClustering at your command line with fit-clustersSet of algorithm implementations for unsupervised analysesClusteringDiviK - hands-free clustering method with built-in feature selectionK-Means with Dunn method for selecting the number of clustersK-Means with GAP index for selecting the number of clustersModular K-Means implementation with custom distance metrics and initializationsFeature extractionPCA with knee-based components selectionLocally Adjusted RBF Spectral EmbeddingFeature selectionEXIMSGaussian Mixture Model based data-driven feature selectionHigh Abundance And Variance Selector - allows you to select highly variant features above noise level, based on GMM-decompositionOutlier based SelectorOutlier Abundance And Variance Selector - allows you to select highly variant features above noise level, based on outlier detectionPercentage based Selector - allows you to select highly variant features above noise level with your predefined thresholds for eachSamplingStratifiedSampler - generates samples of fixed number of rows from given datasetUniformPCASampler - generates samples of random observations within boundaries of an original dataset, and preserving the rotation of the dataUniformSampler - generates samples of random observations within boundaries of an original datasetInstallationDockerThe recommended way to use this software is throughDocker. This is the most convenient way, if you want
to usedivikapplication.To install latest stable version use:dockerpullgmrukwa/divikPython packagePrerequisites for installation of base package:Python 3.7 / 3.8 / 3.9compiler capable of compiling the native C code and OpenMP supportInstallation of OpenMP for Ubuntu / DebianYou should have it already installed with GCC compiler, but if somehow
not, try the following:sudoapt-getinstalllibgomp1Installation of OpenMP for MacOpenMP is available as part of LLVM. You may need to install it with conda:condainstall-cconda-forge"compilers>=1.0.4,!=1.1.0"llvm-openmpInstallation of dependencied on MacYou may see messages that some dependencies are invalid for the platform.
It is aknown bug,
witha workaround.Use:SYSTEM_VERSION_COMPAT=0pipinstalldivikDiviK InstallationHaving prerequisites installed, one can install latest base version of the
package:pipinstalldivikIf you want to have compatibility withgin-config, you can install
necessary extras with:pipinstalldivik[gin]Note:Remember about\before[and]inzshshell.You can install all extras with:pipinstalldivik[all]High-Volume Data ConsiderationsIf you are using DiviK to run the analysis that could fail to fit RAM of your
computer, consider disabling the default parallelism and switch todask. It's easy to achieve through configuration:set all parameters namedn_jobsto1;set all parameters namedallow_dasktoTrue.Note:Never setn_jobs>1andallow_dask=Trueat the same time, the
computations will freeze due to howmultiprocessinganddaskhandle
parallelism.Known IssuesSegmentation FaultIt can happen if the hegamred_nativepackage (part ofdivikpackage) was
compiled with different numpy ABI than scikit-learn. This could happen if you
used different set of compilers than the developers of the scikit-learn
package.In such a case, a handler is defined to display the stack trace. If the trace
comes from_matlab_legacy.py, the most probably this is the issue.To resolve the issue, consider following the installation instructions once
again. The exact versions get updated to avoid the issue.ContributingContribution guide will be developed soon.Format the code with:isort-m3--fgw3--tc.
black-tpy36.ReferencesThis software is part of contribution made byData Mining Group of Silesian
University of Technology, rest of which is
publishedhere.Mrukwa, G. and Polanska, J., 2020. DiviK: Divisive intelligent K-means for
hands-free unsupervised clustering in biological big data.arXiv preprint
arXiv:2009.10706.