Frequency Explains the Inverse Correlation of Large Language Models’ Size, Training Data Amount, and Surprisal’s Fit to Reading Times (2024)

Abstract: Recent studies have shown that as Transformer-based language models become larger and are trained on very large amounts of data, the fit of their surprisal estimates to naturalistic human reading times degrades. The current work presents a series of analyses showing that word frequency is a key explanatory factor underlying these two trends. First, residual errors from four language model families on four corpora show that the inverse correlation between model size and fit to reading times is the strongest on the subset of least frequent words, which is driven by excessively accurate predictions of larger model variants. Additionally, training dynamics reveal that during later training steps, all model variants learn to predict rare words and that larger model variants do so more accurately, which explains the detrimental effect of both training data amount and model size on fit to reading times. Finally, a feature attribution analysis demonstrates that larger model variants are able to accurately predict rare words based on both an effectively longer context window size as well as stronger local associations compared to smaller model variants. Taken together, these results indicate that Transformer-based language models’ surprisal estimates diverge from human-like expectations due to the superhumanly complex associations they learn for predicting rare words.

## introduction the predictability of linguistic material in its context has been shown to be an important factor of real-time processing difficulty @xcite , with a large body of empirical work showing surprisal @xcite to be a strong predictor of relevant behavioral and neural measures @xcite @xcite . therefore, a core research goal of expectation-based theories of sentence processing has been to characterize the latent probability distribution of the human comprehender. language models (lms) that define a conditional probability distribution are helpful for exploring these questions, as they can be trained to embody different predictive processes and yield concrete surprisal estimates that can be evaluated against measures of processing difficulty. recent work using surprisal estimates from transformer-based lms has revealed a strong inverse correlation between the size of lms and the fit of their surprisal estimates to naturalistic human reading times, where larger models yield surprisal estimates that are less predictive of reading times @xcite @xcite . large amounts of training data have also been shown to play a detrimental role, with fit to reading times starting to degrade after lms see about two billion tokens @xcite . this robust inverse correlation is meaningful, as it shows that increasingly larger lms are less appropriate as models of human cognition, and that human sentence processing is not driven by the predictions lms make with more model parameters and training data. while open-class words like nouns and adjectives have been identified as driving the adverse effect of model size @xcite , how model size and the training data interact during lm training to give rise to such systematic divergence from human-like expectations remains unclear. studies on the scaling behavior of large lms have recently shown that larger models learn examples faster by increasing their probabilities to a greater extent given the same amount of exposure @xcite . however, during early training stages, models of all sizes exhibit similar next-token predictions by learning to accurately predict frequent function words @xcite . this suggests that the difference in surprisal estimates as a function of model size will be modulated by frequency and will increase as models see larger amounts of training data. based on these observations, this work presents a series of analyses showing that word frequency is a key explanatory factor of the inverse correlation between model size, training data amount, and surprisal's fit to reading times. first, residual errors from four lm families on four corpora show that the inverse correlation between model size and fit to reading times is the strongest on the subset of least frequent words, which is driven by excessively accurate predictions of larger model variants. moreover, training dynamics reveal that all model variants learn to predict rare words during later training steps and larger model variants do so more accurately, which explains the detrimental effect of both training data amount and model size on fit to reading times. finally, a feature attribution analysis demonstrates that larger model variants predict rare words more accurately compared to smaller model variants based on both an effectively longer context window and stronger local associations. these results provide evidence that transformer-based lms' surprisal estimates diverge from human-like expectations due to the superhumanly complex associations they learn for predicting rare words. ## experiment 1: effect of frequency on strength of inverse correlation the first experiment examines the influence of word frequency on the strength of the inverse correlation between model size and fit to reading times by evaluating surprisal estimates from four lm families on four corpora of naturalistic reading times collected through both self-paced reading and eye-tracking paradigms. ## experiment 2: effect of training data amount on fit to reading times the previous experiment showed that the excessively accurate predictions of larger variants on the subset of least frequent words strongly drive the inverse correlation between model size and fit to human reading times. the second experiment examines the training dynamics of pythia model variants to study the influence of training data amount and model size on the ability of lms to predict rare words, as well as the resulting fit of their surprisal estimates to human reading times. ## follow-up analysis: what enables larger models to predict rare words? the previous experiment showed that as lm variants are trained on large amounts of data, all variants learn to predict rare words during later training steps. however, larger variants are able to do so more accurately, thereby ultimately resulting in surprisal estimates that are poorer predictors of human reading times. the goal of this follow-up analysis is to provide a mechanistic explanation underlying the behavior of larger lm variants identified through the main experiments, by elucidating how the increase in model size enables lms to make more accurate predictions of rare words. although the predictions of transformer-based language models are opaque and difficult to interpret, one possible explanation for this phenomenon is that the larger variants have a longer 'effective' context window size than smaller variants. in other words, although all transformers theoretically have veridical access to the context, the smaller variants may not be able to learn associations to early material in the context due to their limited capacity. this follow-up analysis gauges the contribution of early and recent context tokens using a feature attribution method to examine what enables larger model variants to predict rare words more accurately than their smaller counterparts. ## discussion this work presents word frequency as a unified explanation for the degradation in fit of surprisal estimates to naturalistic human reading times as a function of lm size and training data amount observed in recent studies. first, evaluation of model variants from four lm families on four corpora of reading times shows that the inverse correlation between model size and fit to reading times is the strongest on the subset of least frequent words. this is due to the larger model variants' disproportionately accurate predictions on this subset, where the target reading times are generally longer. these findings are consistent with those from the analyses of @xcite , who found that more severe underpredictions of reading times of openclass words like nouns and adjectives most strongly drive the trend between model size and regression model fit. word frequency provides a more general and parsimonious account of the trend on the entire corpus, as well as a complementary view of the phenomenon. the training dynamics of lms also show a strong interaction between frequency and model size through the course of training. more specifically, model variants of all sizes learn to accurately predict frequent tokens and also show little difference in surprisal values during early training steps. however, as they continue to see large amounts of data, they start learning to predict rare tokens accurately. it is at this later point in training where the difference between model variants begins to manifest, with larger variants learning to make more accurate predictions of these rare tokens. these trends are consistent with prior work on the scaling behavior of large lms @xcite , as well as observations that neural lms first approximate unigram and then bigram probabilities during early training @xcite . residual errors from regression models containing surprisal estimates at intermediate points during lm training show that learning to predict rare tokens initially improves fit to reading times by primarily improving the prediction of reading times of these rare words. nonetheless, as the model variants continue learning to predict rare tokens, the errors on these reading times begin to increase. since larger model variants learn to predict rare tokens more accurately, the regression models containing their surprisal estimates also exhibit a steeper increase in errors. this illustrates the detrimental effect of training data amount and model size on fit to reading times, and also explains @xcite observation of the peak in fit to reading times at around two billion training tokens. the follow-up feature attribution analysis that ablates the contribution of early context tokens suggests that larger model variants utilize both an effectively longer context window and stronger local associations to predict rare tokens more accurately than their smaller counterparts. limiting the number of tokens in the context window weakens these associations for predicting rare words, which is most likely the reason why this improves the fit of lm surprisal to reading times, as demonstrated by @xcite . taken together, these results indicate that both model size and large amounts of training data allow transformer-based lms to learn superhumanly complex associations for predicting rare words, which in turn adversely affects fit to human reading times. in other words, surprisal from model variants that are smaller and trained on less data yield a better fit to naturalistic reading times because they implicitly capture word frequency. this has important implications for research into whether frequency effects are dissociable from predictability effects in naturalistic reading (e.g. @xcite @xcite . one possible interpretation of the current results is that they provide support for a strong and dissociable frequency effect, as the subset of rare words is where lm surprisal estimates diverge most from naturalistic reading times as a whole. however, they may also indicate that the excessive number of parameters and training data result in surprisal estimates that have washed out frequency effects which could have been explained by predictability. an interesting direction for future work is extending the current analyses to data collected in other languages (e.g. @xcite @xcite . based on the training dynamics of lms observed in this work, to the extent that they are of sufficient sizes and are trained on large amounts of data, the explanation based on word frequency is expected to robustly generalize to data from other languages. ## conclusion this work proposes word frequency as an explanation for the inverse correlation observed between transformer-based lms' size, training data amount, and surprisal's fit to human reading times. four lm families on four corpora show the strongest inverse correlation between model size and fit to reading times on the least frequent words, which is driven by the more accurate predictions of the larger variants. training dynamics reveal that all variants learn to predict rare words with large amounts of data and larger variants do so more accurately, which explains the detrimental effect of both model size and training data amount. these results indicate that the superhumanly complex associations for predicting rare words make transformer-based lms' surprisal estimates diverge from human-like expectations.