Snoopy: An Online Interface for Exploring the Effect of Pretraining Term Frequencies on Few-Shot LM Performance (2022)

Abstract: Current evaluation schemes for large language models often fail to consider the impact of the overlap between pretraining corpus and test data on model performance statistics. Snoopy is an online interface that allows researchers to study this impact in few-shot learning settings. Our demo provides term frequency statistics for the Pile, which is an 800 GB corpus, accompanied by the precomputed performance of EleutherAI/GPT models on more than 20 NLP benchmarks, including numerical, commonsense reasoning, natural language understanding, and question-answering tasks. Snoopy allows a user to interactively align specific terms in test instances with their frequency in the Pile, enabling exploratory analysis of how term frequency is related to the accuracy of the models, which are hard to discover through automated means. A user can look at correlations over various model sizes and numbers of in-context examples and visualize the result across multiple (potentially aggregated) datasets. Using Snoopy, we show that a researcher can quickly replicate prior analyses for numerical tasks, while simultaneously allowing for much more expansive exploration that was previously challenging. Snoopy is available at https://nlp.ics.uci.edu/snoopy .

## introduction large language models have achieved impressive few-shot performance on various nlp benchmarks with in-context learning @xcite @xcite . this improvement is primarily driven by increasing the scale of the models and the pretraining data @xcite . by leveraging diverse data sources such as github and arxiv, these models have demonstrated the ability to perform complicated tasks such as quantitative reasoning @xcite and writing computer programs @xcite . however, the current evaluation schemes for these language models often underestimate the possibility of data leakage between the evaluation data and the pretraining data. various studies have demonstrated the capacity of large language models to memorize the pretraining data @xcite , as well as the impact of pretraining term frequency on reasoning performance @xcite . these observations highlight the importance of measuring the impact of pretraining data in evaluating large language models. a critical barrier to performing research related to pretraining data statistics is the cost of analyzing the large corpus of pretraining data. since the size of these corpora is usually large (e.g., pile is 800gb), analyses involving the pretraining data can be time-consuming and expensive. furthermore, evaluating large language models such as gpt-j-6b is also expensive-even inference queries require high-memory gpus-which further impedes analysis of the capabilities and limitations of large language models. to facilitate research in understanding the relationship between the pretraining corpus and model behavior, we introduce snoopy, an online platform that assists researchers in studying the impact of pretraining term frequencies on language model performance on downstream tasks. snoopy includes unigram and low-order co-occurrence statistics of terms in the pile dataset (the pretraining data for all of the eleutherai/gpt models). it uses these counts to show the correlation between the model's few-shot performance on instances and the frequency of instance terms in the pretraining data (illustrated in figure 1 ). our web app supports this analysis on more than 20 nlp benchmarks (mostly from the lm-evaluation-harness @xcite ) including, numerical and commonsense reasoning, natural language understanding, and question answering tasks. in addition, the user can highlight desired terms on the plots, explore individual in-figure 1 : using snoopy to study the effect of term frequencies on gpt-j-6b's 2-shot accuracy on multiplication. each point represents a term (numbers in this case), with x-axis the frequency of the term in pretraining corpus and y-zxis the average performance on the instances that include that term (for 2-shot multiplication using gpt-j-6b). snoopy demonstrates a strong correlation between the accuracy of a number and its frequency in pretraining data. users can select terms to highlight i.e. 11, 12, 24, 23 here. ## snoopy architecture in this section, we describe the architecture behind snoopy (as illustrated in figure 2 ) snoopy precomputes term counts from pretraining data and instance-level performance statistics on evaluation datasets, and allows users to create performance vs. frequency plots dynamically. in the following, we describe each of these components. ## snoopy capabilities as mentioned in section 1, snoopy supports a subset of tasks from lm-evaluation-harness benchmark @xcite ) in addition to all numerical reasoning tasks from @xcite . it provides a simple and performant interface that allows researchers to compare results across various experimental settings with visualizations of the precomputed results in a user-friendly manner. the plots are generated using plotly.js, foot_1 which enables easy download, zoom in-and-out, and re-scaling of the plots. the following is a brief description of snoopy's functionalities on numerical reasoning and other language understanding tasks. ## case study in this section, we present a case study of using snoopy. here, we want to study the effect of termfrequencies on gpt-j-6b model accuracy in 2-shot in-context learning setting. we are going to perform this study on three different datasets of sentiment analysis (sst), question answering (trivi-aqa), and a reading comprehension task (wnli). step 1: we want to investigate whether the gpt-j-6b model accuracy on instances is affected by the unigram term frequencies on the mentioned datasets. first, we need to specify the model, dataset, and the number of shots we want to focus on. for this case, we want to observe the impact of term frequencies on gpt-j-6b models with 2 shot on sst, triviaqa, and wnli tasks. we do this using the drop-down menus shown in figure 4 . upon this selection, snoopy generates the accuracy v.s frequency plots for all these three datasets. step 2: now, we want to observe if the model performance is different on instances with certain terms of "he", "she", "man", and "woman". we use the "add terms" option to add these specific terms as shown in figure 6 ; instances from the sst dataset containing the term "he" have much higher average performance than those with "she", which is not the case for wnli and triviaqa datasets. step 3: in this step, we want to study the average accuracy of gpt-j-6b on instances containing these terms over all three datasets. by choosing the comparison option (presented in figure 7 ), we see that gpt-j-6b model performance on instances that contain the term "he" in comparison to instances with the term "she" on the three datasets. we observe that the model has better performance on sst instances contaning the term "he" in comparison to the instances with the term "she". this is not the case for wnli and triviaqa datasets. step 4: figure 8 provide an example for snoopy's instance visualization feature. using this feature, snoopy provides a random selection of instances from each dataset. this option helps the user get familiar with instance queries from each dataset and observe the model performance on each instance. moreover, the user can select the highlight frequent words option. this option color codes the terms on the instances based on their frequency in the pretraining dataset, as shown in figure 8 . step 5: now we want to visualize the average performance of gpt-j-6b vs. the count of cooccurrences of terms on the x-axis as a measure of frequency for these three datasets. to do so, we select the option of co-occurrence instead of the unigram from the top bar as shown in figure 9 . 2021 ) have studied the pretraining data of large language models. they provide documentation for the c4 corpus which has been used as a part of pretraining datasets such as pile @xcite . many works have illustrated language model capabilities to memorize parts of the pretraining data @xcite . recently, some works has measured the model's memorization of pretraining data through controlled experiments on fact retrieval @xcite , classification tasks @xcite , and text generation @xcite . all this research emphasizes the importance of studying the pretraining data statistics and considering the pretraining data in interpreting the model evaluation performances. evaluation frameworks for lms since the emergence of large language models, many works have provided a unified and easy to use framework for evaluating them @xcite @xcite . our demo, snoopy, can augment these frameworks by associating pretraining data statistics to the evaluation scheme. ## conclusions in this paper, we presented snoopy, a tool that enables researchers to study the impact of pretraining term frequencies on a model's few-shot performance without requiring expensive computing resources. we illustrated how snoopy could be used to create performance vs. frequency plots, aggregate statistics over multiple datasets, and several other functionalities for further investigating pretraining data statistics. we hope that this tool makes it easier for researchers to study the effect of term frequencies on language model performance.