W iz M ap: Scalable Interactive Visualization for Exploring Large Machine Learning Embeddings (2023)

Abstract: Machine learning models often learn latent embedding representations that capture the domain semantics of their training data. These embedding representations are valuable for interpreting trained models, building new models, and analyzing new datasets. However, interpreting and using embeddings can be challenging due to their opaqueness, high dimensionality, and the large size of modern datasets. To tackle these challenges, we present WizMap, an interactive visualization tool to help researchers and practitioners easily explore large embeddings. With a novel multi-resolution embedding summarization method and a familiar map-like interaction design, WizMap enables users to navigate and interpret embedding spaces with ease. Leveraging modern web technologies such as WebGL and Web Workers, WizMap scales to millions of embedding points directly in users’ web browsers and computational notebooks without the need for dedicated backend servers. WizMap is open-source and available at the following public demo link: https://poloclub.github.io/wizmap .

## introduction modern machine learning (ml) models learn highdimensional embedding representations to capture the domain semantics and relationships in the training data @xcite . ml researchers and domain experts are increasingly using expressive embedding representations to interpret trained models @xcite , develop models for new domains @xcite and modalities @xcite , as well as analyze and synthesize new datasets @xcite . however, it can be difficult to interpret and use embeddings in practice, as these high-dimensional representations are often opaque, complex, and can contain unpredictable structures @xcite . furthermore, analysts face scalability challenges as large datasets can require them to study millions of embeddings holistically @xcite . to tackle these challenges, researchers have proposed several interactive visualization tools to help users explore embedding spaces (e.g., @xcite . these tools often visualize embeddings in a low-dimensional scatter plot where users can browse, filter, and compare embedding points. however, for large datasets, it is taxing or even implausible to inspect embedded data point by point to make sense of the global structure of an embedding space. alternatively, recent research explores using contour plots to summarize embeddings @xcite . although contour abstractions enable users to obtain an overview of the embedding space and compare multiple embeddings through superposition, a user study reveals that contour plots restrict users' exploration of an embedding's local structures, where users would prefer to have more visual context @xcite . to bridge this critical gap between two visualization approaches and provide users with a holistic view, we design and develop wizmap (fig. 1 ). our work makes the following major contributions: ## background and related work researchers can extract a data point's embeddings by collecting its corresponding layer activations in neural networks trained for specific tasks such as classification and generation @xcite . additionally, researchers have developed task-agnostic models, such as word2vec @xcite , elmo @xcite , and clip @xcite that generate transferable embeddings directly. these embeddings have been shown to outperform task-specific, stateof-the-art models in downstream tasks @xcite . ## multi-scale embedding summarization researchers have highlighted users' desire for embedding visualizations to provide visual contexts and embedding summaries to facilitate exploration of various regions within the embedding space @xcite . however, generating embedding summaries is challenging for two reasons. first, efficiently summarizing millions of data points in larger datasets can be a formidable task. second, selecting the embedding regions to summarize is difficult, as users possess varying interests in regions of different sizes and levels of granularity. to tackle this challenge, we propose a novel method to automatically generate multiresolution embedding summaries at scale. ## user interface leveraging pre-computed multi-resolution embedding summarization ( § 3), wizmap tightly integrates three interface components (fig. 1a-c ). ## usage scenarios we present two hypothetical scenarios, each with real embedding data, to demonstrate how wizmap can help ml researchers and domain experts easily explore embeddings and gain a better understanding of ml model behaviors and dataset patterns. ## future work and conclusion wizmap integrates a novel quadtree-based embedding summarization technique that enables users to easily explore and interpret large embeddings across different levels of granularity. our usage scenarios showcase our tool's potential for providing ml researchers and domain experts with a holistic view of their embeddings. reflecting on our design and development of wizmap, we acknowledge its limitations and distill future research directions that could further assist users in interpreting and applying embeddings for downstream tasks.