C ross C heck: Rapid, Reproducible, and Interpretable Model Evaluation (2021)

Abstract: Evaluation beyond aggregate performance metrics, e.g. F1-score, is crucial to both establish an appropriate level of trust in machine learning models and identify avenues for future model improvements. In this paper we demonstrate CrossCheck, an interactive capability for rapid cross-model comparison and reproducible error analysis. We describe the tool, discuss design and implementation details, and present three NLP use cases – named entity recognition, reading comprehension, and clickbait detection that show the benefits of using the tool for model evaluation. CrossCheck enables users to make informed decisions when choosing between multiple models, identify when the models are correct and for which examples, investigate whether the models are making the same mistakes as humans, evaluate models’ generalizability and highlight models’ limitations, strengths and weaknesses. Furthermore, CrossCheck is implemented as a Jupyter widget, which allows for rapid and convenient integration into existing model development workflows.

## crosscheck crosscheck is embedded in a jupyter foot_3 notebook and input is a single mixed-type table, i.e. a pandas dataframe foot_4 , allowing for tight integration with data scientists' workflows (see figure 1a ). below we outline the features of crosscheck in detail. crosscheck's main view (see figure 1b ) extends the confusion matrix visualization technique by replacing each cell in the matrix with a histogramwe call this view the histogram heatmap. each cell shows the distribution of a third variable conditioned on the values of the corresponding row and column variables. every bar represents a subset of instances, i.e., rows in the input table, and encodes the relative size of that group. this view also contains a legend showing the bins or categories for this third variable (see figure 1c ). the histograms in each cell in crosscheck are drawn horizontally to encourage comparison across cells in the vertical direction. crosscheck supports three normalization schemes (see figure 1d ), i.e., setting the limit of the x-axis in each cell: (1) normalizing by the maximum count within the entire matrix, (2) within each column, or (3) within each cell. we hide certain axes and adjust the padding between cells to emphasize the selected normalization. figure 2 illustrates how these different normalization options appear in crosscheck. by design, there is no equivalent row normalization option, but the matrix can be transposed (see figure 1g ) for an equivalent effect. any variables not directly compared in the histogram heatmap are visualized on the left side of the widget as histograms (see figure 1e ). these histograms also allow the user to filter data when it is rendered in the main view by clicking on the bar(s) corresponding to the data they want to keep. users can click on any bar in the histogram heatmap to view those instances in a sidebar where they can annotate noteworthy findings. enabling "notes only" (see figure 1f ) shows only instances that have been annotated in the histogram heatmap, revealing what has been annotated in the context of the current variable groupings. ## use cases and evaluation in this section, we highlight how crosscheck can be used in core nlp tasks such as named entity recognition (ner) and reading comprehension (rc) or practical applications of nlp such as clickbait detection (cb). we present an overview of the datasets used for each task below: ## design and implementation we designed crosscheck following a usercentered design methodology. this is a continuous, iterative process where we identify needs and goals, implement prototypes, and solicit feedback from our users to incorporate in the tool. our users were data scientists, specifically nlp researchers and practitioners, tasked with the aforementioned model evaluation challenge. we identified crosscheck's goals as allowing the user to: understand how instance attributes relate to model errors; provide convenient access to raw instance data; integrate into a data scientists workflow; and reveal and understand disagreement across models, and support core nlp tasks and applications. ## discussion crosscheck is designed to quickly and easily explore many combinations of characteristics of models (e.g., parameter settings, network architectures) as well as datasets used for training or evaluation. it also provides users the ability to efficiently compare and explore model behavior in specific situations and generalizability of models across datasets or domains. crosscheck can also generalize to support model comparison, e.g. when ground truth is absent, by visualizing model agreement. crosscheck enables users to perform error analysis in an efficient, concise, and reproducible manner due to its effective integration into data scientists' workflows. the tool can be used to evaluate across models trained on image, video, tabular data, or combinations of data types with interactive exploration of specific instances on demand. a limitation of the tool is that adding new use cases might require end users to write custom javascript code to visualize instances in the details sidebar beyond what is currently implemented. future work includes expanding to include additional generic components to cover more core nlp or ml tasks. ## conclusions we have presented crosscheck, a new interactive capability that enables rapid model evaluation and error analysis. there are several key benefits to performing evaluation and analyses using cross-check, especially compared to i.e., adhoc or manual approaches because crosscheck is generalizable across text, images, video, tabular, or combinations of multiple data types, can be integrated directly into existing workflows for rapid and highly reproducible error analysis during and after model development, users can interactively explore errors conditioning on different model/data features, and users can view specific instances of inputs that cause model errors or other interesting behavior within the tool itself.