T3-Vis: visual analytic for Training and fine-Tuning Transformers in NLP (2021)

Abstract: Transformers are the dominant architecture in NLP, but their training and fine-tuning is still very challenging. In this paper, we present the design and implementation of a visual analytic framework for assisting researchers in such process, by providing them with valuable insights about the model’s intrinsic properties and behaviours. Our framework offers an intuitive overview that allows the user to explore different facets of the model (e.g., hidden states, attention) through interactive visualization, and allows a suite of built-in algorithms that compute the importance of model components and different parts of the input sequence. Case studies and feedback from a user focus group indicate that the framework is useful, and suggest several improvements. Our framework is available at: https://github.com/raymondzmc/T3-Vis .

## introduction approaches through neural networks have made significant progress in the field of nlp, with transformer models @xcite rapidly becoming the dominant architecture due to their efficient parallel training and ability to effectivelymodel long sequences. following the release of bert @xcite along with other transformer-based models pretrained on large corpora @xcite @xcite , the most successful strategy on many nlp leaderboards has been to directly fine-tune such models on the downstream tasks (e.g., summarization, classification). however, despite the strong empirical performance of this strategy, understanding and interpreting the training and fine-tuning processes remains a critical and challenging step for researchers due to the inherent black-box nature of neural models @xcite @xcite . generally speaking, a large number of visual analytics tools have been shown to effectively support the analysis and interpretation of deep learning models @xcite . for instance, to remedy the black-box nature of neural network hidden states, previous work has used scatterplots to visualize high dimensional vectors through projection techniques @xcite , with aken et al. (2020) visualizing the differences of token representations from different layers of bert @xcite . similarly, despite some limitations regarding the explanatory capabilities of the attention mechanism @xcite , the visualization of its weights has also been shown to be beneficial in discovering learnt features @xcite , with promising recent work focusing on transformers @xcite . besides the works on exploring what has been learnt in the pretrained models, there are also several visualization tools developed to show saliency scores generated by gradient-based @xcite @xcite or perturbation-based interpretation methods @xcite , which can help with visualizing the relative importance of individual tokens in the input with respect to a target prediction @xcite @xcite . however, only a few studies have instead focused on visualizing the overall training dynamics, where support is critical for identifying mislabeled examples or failure cases @xcite @xcite in essence, the framework we propose in this paper, namely t 3 -vis, synergistically integrates some of the interactive visualizations mentioned above to support developers in the challenging task of training and fine-tuning transformers. this is in contrast with other similar recent visual tools ( following the well-established nested model for visualization design @xcite , we first perform an extensive requirement analysis, from which we derive user tasks and data abstractions to guide the design of visual encoding and interaction techniques. more specifically, the resulting t 3 -vis framework provides an intuitive overview that allows users to explore different facets of the model (e.g., hidden states, attention, training dynamics) through interactive visualization. our contributions are as follows: ## visualization design the design of our t 3 -vis is based on the nested model for infovis design @xcite . ## conclusion in this paper, we presented t 3 -vis, a visual analytic framework designed to help researchers better understand training and fine-tuning processes of transformer-based models. our visual interface provides faceted visualization of a transformer model and allows exploring data across multiple granularities, while enabling users to dynamically interact with the model. additionally, our implementation and design allows flexible customization to support a diverse range of tasks and workflows. our focus group and case studies demonstrated the effectiveness of our interface by assisting the researchers in interpreting the models' behaviour and identifying potential directions to improve task performances. for future work, we will continue to improve our framework through the iterative process of exploring further usage scenarios and collecting feedback from users. we will extend our framework to provide a more advanced visualization for custom transformers. for example, we may want to support the visualization of models with more complex connections (e.g. parallel attention layers) or an advanced attention mechanism (e.g. sparse attention).