LM diff: A Visual Diff Tool to Compare Language Models (2021)

Abstract: While different language models are ubiquitous in NLP, it is hard to contrast their outputs and identify which contexts one can handle better than the other. To address this question, we introduce LMdiff, a tool that visually compares probability distributions of two models that differ, e.g., through finetuning, distillation, or simply training with different parameter sizes. LMdiff allows the generation of hypotheses about model behavior by investigating text instances token by token and further assists in choosing these interesting text instances by identifying the most interesting phrases from large corpora. We showcase the applicability of LMdiff for hypothesis generation across multiple case studies. A demo is available at http://lmdiff.net .

## introduction interactive tools play an important role when analyzing language models and other machine learning models in natural language processing (nlp) as they enable the qualitative examination of examples and help assemble anecdotal evidence that a model exhibits a particular behavior in certain contexts. this anecdotal evidence informs hypotheses that are then rigorously studied (e.g., @xcite @xcite . many such tools exist, for example to inspect attention mechanisms @xcite , explain translations through nearest neighbors @xcite , investigate neuron values @xcite , and many more that focus on the outputs of models (e.g., @xcite . there also exist multiple frameworks that aggregate methods employed in the initial tools to enable others to extend or combine them @xcite @xcite . however, notably absent from the range of available tools are those that aim to compare distributions produced by different models. while comparisons according to performance numbers are common practice in benchmarks @xcite @xcite , there exists only rudimentary support in existing tools for inspecting how model outputs compare for specific tasks or documents. yet, this problem motivates many current studies, including questions about how models handle gendered words, whether domain transfer is easy between models, what happens during finetuning, where differences lie between models of different sizes, or how multilingual and monolingual models differ. to fill this gap, we introduce lmdiff: an interactive tool for comparing language models by qualitatively comparing per-token likelihoods. our design provides a global and a local view: in the global step, we operate on an entire corpus of texts, provide aggregate statistics across thousands of data points, and help users identify the most interesting examples. an interesting example can then be further analyzed in the local view. finegrained information about the model outputs for the chosen example is visualized, including the probability of each token and the difference in rank within each model's distribution. similar to other visual tools, lmdiff helps form hypotheses that can then be tested through rigorous statistical analyses. across six case studies, we demonstrate how it enables an effective exploration of model differences and motivates future research. a deployed version of lmdiff with six corpora and nine models is available at http://lmdiff.net/ and its code is released at https://github.com/ hendrikstrobelt/lmdiff (apache 2.0 license) with support to easily add additional models, corpora, or evaluation metrics. ## methods lmdiff compares two models m {1,2} by analyzing their probability distributions at the position of each token x1:n in a specific text. a correct token's probability distribution p m j (x @xmath0:i-1 ) is easily influenced the scaling factor β in the function @xmath1. for this reason, we also include the correct token's rank in p m j (x i |x 1:i-1 ). from the probabilities and ranks, we derive eight measures of global difference (comparison over a corpus) and eight measures of local difference (comparison over an example). the global measures are the (1) difference in rank of each token, (2) the difference in rank after clamping a rank to a maximum of 50, (3) the difference in probability of each token, and (4) the number of different tokens within the top-10 predicted tokens. 1 for each measure, we allow filtering by its average or maximum in a sequence. ## supported data and models the deployed version of lmdiff currently supports six datasets and nine models, detailed in ## case studies as discussed above, this tool aims to generate hypotheses by discovering anecdotal evidence for certain model behavior. it will not be able to give definite proofs for discovered hypotheses, which should instead be explored more in-depth in followup studies. as such, in this section, we provide examples of new kinds of questions that lmd-iff helps investigate and explore further questions inspired by past findings. ## system description all comparisons in lmdiff begin with three provided arguments: a dataset containing the interesting phases to analyze, and two comparable transformer models. lmdiff wraps huggingface transformers @xcite and can use any of their pretrained autoregressive or masked language models from their model hub 3 or a local directory. two models are comparable if they use the same tokenization scheme and vocabulary. this is required such that a phrase passed to either of them will have an identical encoding, with special tokens added in the same locations. lmdiff then does the work of recording each model's predictions across the dataset into an anal-ysiscache. each token in each phrase of the dataset is analyzed for its "rank" and "probability". we define a token's rank as the affinity of the lm to predict the token relative to all other tokens, where a rank of 1 indicates it is the most favorable token, and the probability is computed from a direct softmax of the token's logit. other useful information is also stored, such as the top-10 tokens (and their probabilities) that would have been predicted in that token's spot. this information can 3 https://huggingface.co/models then be compared to other caches and explored in the visual interface. the interface can also be used independently of cache files to compare models on individual inputs. the modular design separating datasets, models, and their caches makes it easy to compare the differences between many different models on distinct datasets. once a cache has been made of a (model, dataset_d) pair, it can be compared to any other cache of a (comparable_model, dataset_d) pair within seconds. more information is provided in appendix b. adding models and datasets it is easy to load additional models and datasets. first, ensure that the model can be loaded through the huggingface automodelwithlmhead and autotokenizer function from_pretrained(...) which supports loading from a local directory. the following script prepares two models and a dataset for comparison: the output configuration directory out can be passed directly to the lmdiff server and interface which will automatically load the new data: the interface works equally well to compare two models on individual examples without a preprocessed cache: