Package: densmap-learn
densMAPThis software package contains an implementation of density-preserving data visualization tooldensMAP, which augments theUMAPalgorithm (based on v0.3.10).
Some of the following instructions are adapted from the UMAP repository.InstallationdensMAP shares the same dependencies as UMAP, including:numpyscipyscikit-learnnumba==0.48.0Our code currently does not support the latest version of numba (0.49.0).Install OptionsPyPI installation of densMAP can be performed as:pipinstalldensmap-learnFor a manual install, first download this package:wgethttps://github.com/hhcho/densvis/archive/master.zip
unzipdensvis-master.zip
rmdensvis-master.zipcddensvis-master/densmap/Install the requirements:sudopipinstall-rrequirements.txtorcondainstallscikit-learnnumbaFinally, install the package:pythonsetup.pyinstallUsageLike UMAP, the densMAP package inherits from sklearn classes, and thus drops in neatly
next to other sklearn transformers with an identical calling API.importdensmapfromsklearn.datasetsimportfetch_openmlfromsklearn.utilsimportresampledigits=fetch_openml(name='mnist_784')subsample,subsample_labels=resample(digits.data,digits.target,n_samples=7000,stratify=digits.target,random_state=1)embedding,ro,re=densmap.densMAP().fit_transform(subsample)Input argumentsThere are a number of parameters that can be set for the densMAP class; the
major ones inherited from UMAP are:n_neighbors: This determines the number of neighboring points used in
local approximations of manifold structure. Larger values will result in
more global structure being preserved at the loss of detailed local
structure. In general this parameter should often be in the range 5 to
50; we set a default of 30.min_dist: This controls how tightly the embedding is allowed compress
points together. Larger values ensure embedded points are more evenly
distributed, while smaller values allow the algorithm to optimise more
accurately with regard to local structure. Sensible values are in the
range 0.001 to 0.5, with 0.1 being a reasonable default.metric: This determines the choice of metric used to measure distance
in the input space. A wide variety of metrics are already coded, and a user
defined function can be passed as long as it has been JITd by numba.The additional parameters specific to densMAP are:dens_frac: This determines the fraction of iterations that will include
the density-preservation term of the gradient (float, between 0 and 1); default 0.3.dens_lambda: This determines the weight of the density-preservation
objective. See the original paper for the effect this parameter has when changed (float, non-negative); default 2.0.final_dens: When this flag isTrue, the code returns, in addition to the embedding,
the local radii for the original dataset and for the embedding. IfFalse, only the embedding
is returned (bool); defaultTrue.Other parameters that can be set include:ndim: Dimensions of the embedding (int); default 2.n_epochs: Number of epochs to run the algorithm (int); default 750.var_shift: Regularization term added to the variance of embedding local radius for stability (float, non-negative); default 0.1.Output argumentsIffinal_densisTrue, returns(embedding, ro, re), where:embedding: a (number of data points)-by-ndimsnumpy array containing the embedding coordinatesro: a numpy array of length (number of data points) that contains the log local radius of
the input datare: a numpy array of length (number of data points) that contains the log local radius
of the embedded dataIffinal_densisFalse, returns justembedding.An example of making use of these options:embedding,ro,re=densmap.densMAP(n_neighbors=25,n_epochs=500,dens_frac=0.3,dens_lambda=0.5).fit_transform(data)R wrapperWe use thereticulatelibrary to provide compatibility with R as well with the
scriptdensmap.R. Sincereticulateruns Python code with an R wrapper, to use this
library you must have Python3 installed. The script will automatically install thedensmap-learnpackage viapipif it is not installed.From then, within your R script, you can runsource("densmap.R")# Assume `data` is an R dataframe, needs to be converted to a matrixout<-densMAP(as.matrix(data))The R functiondensMAPtakes the same optional arguments listed in Input Arguments section
above with the same names and default values. So you can, for example, run:out<-densMAP(as.matrix(data),n_neighbors=25,n_epochs=500,dens_frac=0.3,dens_lambda=0.5)Iffinal_densisTRUEthenout[[1]]will contain the embedding,out[[2]]will be the
log original local radii, andout[[3]]the log embedding local radii.Iffinal_densisFALSEthenoutwill be the embeddings itself.From the command lineWe also provide the filedensmap.pywhich allows you to run densMAP from the terminal,
specifying the major options from above. Simply run:pythondensmap.py-i[--inputINPUT]-o[--outnameOUTNAME]-f[--dens_fracDENS_FRAC]-l[--dens_lambdaDENS_LAMBDA]-s[--var_shiftVAR_SHIFT]-d[--ndimNDIM]-n[--n-epochsN-EPOCHS]-k[--n-neiN-NEIGHBORS][--final_dens/--no_final_densFINAL_DENS]where within the square braces are the long-form flag and the capitalized text corresponds to
the parameters above. For example:python densmap.py -i data.txt -o out -f .3 -k 25andpython densmap.py --input data.txt --outname out --dens_frac .3 --n-nei 25both run densMAP on input filedata.txtto produce output filesout_emb.txtandout_dens.txt, usingdens_frac=0.3andn_neighbors=25.
The input file is parsed using numpy’sloadtxtfunction if it is a.txtfile; another option is to provide a.pklfile.
We assume that the first dimension (row index) iterates over the data instances, and the second dimension (column index) iterates over the features.The output files include:out_emb.txta TSV file containing the embedding coordinatesof the data, andout_dens.txta (number of data points)-by-2 TSV file containing in the first column the log local radii in the original data and in the second column the log local radii in the embedding.Example data and scriptWe have included the filetrial_densmap.pywhich allows you to run an example straight out of
the box.Run:pythontrial_densmap.pyThe code will load a dataset that contains a mixture of six Gaussian point clouds
with increasing variance and
will run both densMAP and UMAP on the dataset with default parameters and plot the embeddings
(if you havematplotlibinstalled),
and alignment of the local radius in each case. It will also save the embeddings in{umap,densmap}_trial_emb.txt,
the local radii in{umap,densmap}_trial_dens.txt, and the plot indensmap_trial_fig.png.The plot will look like:ReferencesOur densMAP algorithm is described in:Ashwin Narayan, Bonnie Berger*, and Hyunghoon Cho*. "Density-Preserving Data Visualization Unveils Dynamic Patterns of Single-Cell Transcriptomic Variability",bioRxiv, 2020.Original UMAP algorithm is described in:Leland McInnes, John Healy, and James Melville. "UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction",arXiv, 1802.03426, 2018.LicenseThis package is licensed under the MIT license.Contact for questionsAshwin Narayan,ashwinn@mit.eduHoon Cho,hhcho@broadinstitute.orgAdditionally, some questions regarding the UMAP-specific aspects of this software may be answered by browsing the UMAP documentation atRead the Docs, whichincludes an FAQ.