SEAL : Interactive Tool for Systematic Error Analysis and Labeling (2022)

Abstract: With the advent of Transformers, large language models (LLMs) have saturated well-known NLP benchmarks and leaderboards with high aggregate performance. However, many times these models systematically fail on tail data or rare groups not obvious in aggregate evaluation. Identifying such problematic data groups is even more challenging when there are no explicit labels (e.g., ethnicity, gender, etc.) and further compounded for NLP datasets due to the lack of visual features to characterize failure modes (e.g., Asian males, animals indoors, waterbirds on land etc.). This paper introduces an interactive Systematic Error Analysis and Labeling (SEAL) tool that uses a two-step approach to first identify high-error slices of data and then, in the second step, introduce methods to give human-understandable semantics to those underperforming slices. We explore a variety of methods for coming up with coherent semantics for the error groups using language models for semantic labeling and a text-to-image model for generating visual features.SEAL is available at https://huggingface.co/spaces/nazneen/seal .

## introduction machine learning systems that seemingly perform well on average can still make systematic errors on important subsets of data. examples include such systems performing poorly for marginalized groups in chatbots (stuart-ulin, 2018), recruiting tools @xcite , cloud products (kayser-bril, 2020), ad targeting @xcite , credit services @xcite , and image cropping @xcite . discovering and labeling systematic errors in ml systems is an open research problem that would enable building robust models that generalize across subpopulations of data. uncovering underperforming groups of data of a ml system is not straightforward. firstly, the highdimensional space of the representations learned by the deep learning models makes it difficult to identify such groups of systematic errors. secondly, it is difficult to extract and label the hidden semantic information in such groups with high errors without a human-in-the-loop setup. identifying systematic model failures requires practitioners to think creatively about model evaluation @xcite @xcite @xcite . however, current approaches are mostly limited to examining and manipulating model mispredictions. the onus of identifying what group or subset of data to evaluate still falls on the practitioner, making it inefficient and prone to oversight. recent works on finegrained error analysis, such as domino @xcite and @xcite provide solutions to this problem but focus on image datasets which are easier to visualize. error analysis for text data is less explored and more challenging. it also highlights the need to provide semantic summaries of text, which we tackle in seal. for example, nlp models could apart from the dataset and model, the user can select the loss quantile that want to examine for systematic errors, if they want seal to group those errors using kmeans++ with the number of clusters, and how many data points they want to visualize at a time in the visual component of the interface downsampled proportional to the group size (we use altair for plotting that supports a maximum of 5000 data points to be visualized at once). our desiderata is a tool that summarizes failures of a model on textual data in a concise, coherent and human intepretable way. systematic error analysis and labeling (seal) is an interactive tool to 1. identify candidate groups of data with high systematic errors and 2. generate semantic labels for those groups. for 1, we use k-means++ on subset of evaluation data with highest loss. semantic labeling uses llms (like gpt3) in zero-shot setting for identifying concepts or topics common to examples in the candidate group. we also explored using a text-to-image model to generate visual features for high error clusters using the dalle-mini @xcite . semantic descriptions (via labeling or visual features) of such systematic model errors not only enable practitioners to better understand the failure modes of their model during evaluation but also gives actionable insight to fix them via some form of model patching or data augmentation. ## seal we present systematic error analysis and labeling (seal), an interactive visualization tool that provides rich data point comparison for text classification systems, enabling fine-grained understanding of model performance on data groups as shown in figure 2 . it comes pre-loaded with model outputs for most downloaded huggingface (hf) models and datasets, as well as scripts for loading data for any dataset provided by the datasets api and extracting embeddings of any hf-compatible model. foot_0 ## results and case study in this section, we discuss some results using the seal pipeline and walk through a case study for an interactive analysis with the tool. ## mathematical robustness of seal in this section, we provide theoretical guarantees for the stability of semantic labels generated by the seal pipeline. more specifically, our stability theorem states that a small perturbation of the input of our seal pipeline would only cause a small bounded difference of the semantic labels. an implication of our theoretical results is that, even if two users are using different versions of an evaluation set (e.g., a different split, or a smaller subset), seal would generate similar semantic labels. more formally, we ask: how does a small change in the input dataset {(x i , y i )} n @xmath0, k denotes the number of explanations, m k ≜ (w k , s k , a k ) encodes the kth explanation message, where w k , s k , and a k represent the sentence vector, the number of data points explained by this mes- ## conclusion in this work we introduced seal, an interactive visualization tool for discovering systematic errors and labeling them. through case studies we showed how seal can efficiently identify the systematic failures of state-of-the-art sentiment classification models on well known datasets. we released a set of pre-computed model outputs to enable easy, out-of-the-box use especially for noncoding audience such as domain experts. we hope this work will positively contribute to the ongoing efforts in building tools for systematic error analysis and model debugging.