Dodrio: Exploring Transformer Models with Interactive Visualization (2021)

Abstract: Why do large pre-trained transformer-based models perform so well across a wide variety of NLP tasks? Recent research suggests the key may lie in multi-headed attention mechanism’s ability to learn and represent linguistic information. Understanding how these models represent both syntactic and semantic knowledge is vital to investigate why they succeed and fail, what they have learned, and how they can improve. We present Dodrio, an open-source interactive visualization tool to help NLP researchers and practitioners analyze attention mechanisms in transformer-based models with linguistic knowledge. Dodrio tightly integrates an overview that summarizes the roles of different attention heads, and detailed views that help users compare attention weights with the syntactic structure and semantic information in the input text. To facilitate the visual comparison of attention weights and linguistic knowledge, Dodrio applies different graph visualization techniques to represent attention weights scalable to longer input text. Case studies highlight how Dodrio provides insights into understanding the attention mechanism in transformer-based models. Dodrio is available at https://poloclub.github.io/dodrio/ .

## introduction the rise of transformer-based models have brought dramatic performance improvements across many nlp tasks @xcite . in particular, bert @xcite has demonstrated that transformer-based models pre-trained on largescale corpora can be effectively fine-tuned for a wide variety of downstream tasks, such as sentiment analysis, question answering, and text summarization. however, how these language models generalize text representations learned from an unsupervised training process to downstream sentence understanding tasks remains unclear. there is a growing research body in interpreting transformerbased models, as understanding what these models have learned and why they succeed and fail is vital for nlp researchers to develop better models, and critical for decision makers to trust these models. the current approach on interpreting transformer-based models focuses on probing and attention weight analysis @xcite . there is an active discussion on whether attention weights are explanations @xcite , but more recent work has shown that they do provide insights on what the models have learned @xcite . in particular, research has shown that transformerbased models have learned to represent semantic knowledge and lexical structure in text @xcite . furthermore, interaction visualization systems have shown great potential in explaining complex deep learning models @xcite . some visualization tools have been developed for transformer-based models @xcite @xcite . however, these systems usually focus on visualizing and analyzing attention weights, instead of visually connecting them to linguistic knowledge that is crucial to investigate why transformer-based models work so well across different tasks @xcite . to address this research challenge, we present dodrio (figure 1 ), an interactive visualization tool to help nlp researchers and practitioners analyze and compare attention mechanisms with linguistic knowledge. for a demo video of do-drio, visit https://youtu.be/qb-t9j7utge . in this work, our primary contributions are: ## background attention heads are comprised of weights incurred from words when calculating the next representation of the current word @xcite , which are known as attention weights. easily interpretable, using attention to understand model predictions across domains is a very popular research area @xcite . in nlp, there has been a growing body of research on attention used as a tool for interpretability across many language tasks @xcite @xcite . existing visualization systems and techniques do not visually connect attention mechanisms to linguistic knowledge @xcite , we propose novel visualization approaches that foster exploration across semantically and syntactically significant attention heads in complex model architectures. for example, for every attention head in the 144 heads of bert, the entry a i,j in the attention map a, represents the attention weight from token i to token j. with 144 × number of tokens × number of tokens attention weights in bert for each input instance, it is challenging to systematically analyze these attention weights without abstraction and linguistic context. dodrio aims to address this challenge by applying novel interactive visualization techniques. ## discussion dodrio aims to help nlp researchers and practitioners to explore attention mechanisms in transformer-based models with linguistic knowledge. with overview + detail, linking + brushing, graph visualization techniques, dodrio enables the users to investigate attention weights with different levels of abstraction in a context with both semantic and syntactic information. through use cases, we demonstrate that dodrio not only helps users validate existing research results regarding the connections between attention weights with linguistic information, but also inspires the users to form hypothesis regarding the behavior and roles of attention heads across different models. we acknowledge that there is an active discussion on whether attention weights can help people interpret transformer-based models @xcite and whether the attentions can be directly linked to the corresponding tokens in interpretation tasks @xcite . our work joins the growing research body in nlp interpretability and human-centered nlp, highlighting novel visualization designs that can be generalized to other interactive nlp systems. despite the increasing popularity of applying human-computer interaction techniques to help people from various fields interact with complex nlp systems, little work have been done to evaluate how effective these tools are @xcite . to fill this research gap, we plan to run a user study to evaluate the usability and usefulness of dodrio. ## conclusion we present dodrio, an interactive visualization system that fosters the exploration of the attention mechanism in transformer-based models with linguistic knowledge. through analysis from the model to the attention head level, users can explore how attention differs across a complex, state-of-theart architecture over any instance within a dataset. our tool runs in modern web browsers and is opensourced, broadening the public's access to modern ai techniques. we hope our work will inspire further research in understanding attention mechanisms and development of visualization tools that help people interact with complex nlp models.