Package: trimap
TriMap is a dimensionality reduction method that uses triplet constraints
to form a low-dimensional embedding of a set of points. The triplet constraints
are of the form “pointiis closer to pointjthan pointk”. The triplets are
sampled from the high-dimensional representation of the points and a weighting
scheme is used to reflect the importance of each triplet.TriMap provides a significantly better global view of the data than the
other dimensionality reduction methods such t-SNE, LargeVis, and UMAP. The global
structure includes relative distances of the clusters, multiple scales in
the data, and the existence of possible outliers. We define a global score to quantify the quality of an embedding in reflecting the global structure of the data.CIFAR-10 dataset (test set) passed through a CNN (n = 10,000, d = 1024): Notice the semantic structure unveiled by TriMap.The following implementation is in Python. Further details and more experimental results are available in thepaper. See theexample colabfor some analysis.News![Mar 16, 2022] An example colab using TriMapJAX implementationis now available athttps://github.com/eamid/examples/blob/master/TriMap.ipynb. We analyze the results on S-curve, MNIST, Fashion MNIST, etc. using t-SNE, UMAP, TriMap, and PCA.[Feb 17, 2022] A JAX implementation is now available athttps://github.com/google-research/google-research/tree/master/trimap. More updates are coming soon!How to use TriMapTriMap has a transformer API similar to other sklearn libraries. To use
TriMap with the default parameters, simply do:importtrimapfromsklearn.datasetsimportload_digitsdigits=load_digits()embedding=trimap.TRIMAP().fit_transform(digits.data)To find the embedding using a precomputed pairwise distance matrix D, pass D as input and set use_dist_matrix to True:embedding=trimap.TRIMAP(use_dist_matrix=True).fit_transform(D)You can also pass the precomputed k-nearest neighbors and their corresponding distances as a tuple (knn_nbrs, knn_distances). Note that the rows must be in order, starting from point 0 to n-1. This feature also requires X to compute the embeddingembedding=trimap.TRIMAP(knn_tuple=(knn_nbrs,knn_distances)).fit_transform(X)To calculate the global score, do:gs=trimap.TRIMAP(verbose=False).global_score(digits.data,embedding)print("global score%2.2f"%gs)ParametersThe list of parameters is given blow:n_dims: Number of dimensions of the embedding (default = 2)n_inliers: Number of nearest neighbors for forming the nearest neighbor triplets (default = 12).n_outliers: Number of outliers for forming the nearest neighbor triplets (default = 4).n_random: Number of random triplets per point (default = 3).distance: Distance measure (‘euclidean’ (default), ‘manhattan’, ‘angular’ (or ‘cosine’), ‘hamming’)weight_temp: Temperature of the logarithm applied to the weights. Larger temperatures generate more compact embeddings. weight_temp=0. corresponds to no transformation (default=0.5).weight_adj(deprecated): The value of gamma for the log-transformation (default = 500.0).lr: Learning rate (default = 0.1).n_iters: Number of iterations (default = 400).The other parameters include:knn_tuple: Use the precomputed nearest-neighbors information in form of a tuple (knn_nbrs, knn_distances) (default = None)use_dist_matrix: Use the precomputed pairwise distance matrix (default = False)apply_pca: Reduce the number of dimensions of the data to 100 if necessary before applying the nearest-neighbor search (default = True).opt_method: Optimization method {‘sd’ (steepest descent), ‘momentum’ (GD with momentum), ‘dbd’ (delta-bar-delta, default)}.verbose: Print the progress report (default = False).return_seq: Store the intermediate results and return the results in a tensor (default = False).An example of adjusting these parameters:importtrimapfromsklearn.datasetsimportload_digitsdigits=load_digits()embedding=trimap.TRIMAP(n_inliers=20,n_outliers=10,n_random=10).fit_transform(digits.data)The nearest-neighbor calculation is performed usingANNOY.ExamplesThe following are some of the results on real-world datasets. The values of nearest-neighbor accuracy and global score are shown as a pair (NN, GS) on top of each figure. For more results, please refer to ourpaper.USPS Handwritten Digits (n = 11,000, d = 256)20 News Groups (n = 18,846, d = 100)Tabula Muris (n = 53,760, d = 23,433)MNIST Handwritten Digits (n = 70,000, d = 784)Fashion MNIST (n = 70,000, d = 784)TV News (n = 129,685, d = 100)Runtime of t-SNE, LargeVis, UMAP, and TriMap in the hh:mm:ss format on a single machine with 2.6 GHz Intel Core i5 CPU and 16 GB of memory is given in the following table. We limit the runtime of each method to 12 hours. Also, UMAP runs out of memory on datasets larger than ~4M points.InstallingRequirements:numpyscikit-learnnumbaannoyInstalling annoyIf you are having trouble with installingannoyon macOS using the command:pip3installannoyyou can alternatively try:pip3installgit+https://github.com/sutao/annoy.git@masterInstall OptionsIf you have all the requirements installed, you can use pip:sudopipinstalltrimapPlease regularly check for updates and make sure you are using the most recent version. If you have TriMap installed and would like to upgrade to the newer version, you can use the command:sudopipinstall--upgrade--force-reinstalltrimapAn alternative is to install the dependencies manually using anaconda and using pip
to install TriMap:condainstallnumpycondainstallscikit-learncondainstallnumbacondainstallannoypipinstalltrimapFor a manual install get this package:wgethttps://github.com/eamid/trimap/archive/master.zipunzipmaster.ziprmmaster.zipcdtrimap-masterInstall the requirementssudopipinstall-rrequirements.txtorcondainstallscikit-learnnumbaannoyInstall the packagepythonsetup.pyinstallSupport and ContributionThis implementation is still a work in progress. Any comments/suggestions/bug-reports
are highly appreciated. Please feel free contact me at:eamid@ucsc.edu. If you would
like to contribute to the code, pleasefork the projectand send me a pull request.CitationIf you use TriMap in your publications, please cite our current reference on arXiv:@article{2019TRIMAP,
     author = {{Amid}, Ehsan and {Warmuth}, Manfred K.},
     title = "{TriMap: Large-scale Dimensionality Reduction Using Triplets}",
     journal = {arXiv preprint arXiv:1910.00204},
     archivePrefix = "arXiv",
     eprint = {1910.00204},
     year = 2019,
}LicensePlease see the LICENSE file.