Temperature-scaling surprisal estimates improve fit to human reading times – but does it do so for the “right reasons”? (2024)

Abstract: A wide body of evidence shows that human language processing difficulty is predicted by the information-theoretic measure surprisal, a word’s negative log probability in context. However, it is still unclear how to best estimate these probabilities needed for predicting human processing difficulty – while a long-standing belief held that models with lower perplexity would provide more accurate estimates of word predictability, and therefore lead to better reading time predictions, recent work has shown that for very large models, psycholinguistic predictive power decreases. One reason could be that language models might be more confident of their predictions than humans, because they have had exposure to several magnitudes more data. In this paper, we test what effect temperature-scaling of large language model (LLM) predictions has on surprisal estimates and their predictive power of reading times of English texts. Firstly, we show that calibration of large language models typically improves with model size, i.e. poorer calibration cannot account for poorer fit to reading times. Secondly, we find that temperature-scaling probabilities lead to a systematically better fit to reading times (up to 89% improvement in delta log likelihood), across several reading time corpora. Finally, we show that this improvement in fit is chiefly driven by words that are composed of multiple subword tokens.

## introduction 032 in psycholinguistics, a key finding is that words 033 with higher surprisal (= negative log probability 034 of the word in context) require more time for pro-035 cessing @xcite . numerous stud-036 ies provided experimental evidence supporting this 037 theory, demonstrating that surprisal is a powerful 038 predictive measure of processing complexity (e.g., 039 @xcite , 040 1 code available at: https://github.com/ tongliu-github/temperaturescaling4rts . 2023; @xcite , and that the relationship between surprisal and reading times (rts) seems to be linear @xcite @xcite . however, prior work implicitly made the assumption that human predictability estimates would be similar to the actual probability of a word occurring in a given context, and that therefore, surprisal values estimated from models that achieve lower perplexities should also approximate human processing difficulty better @xcite . recent research has however found that this is not true -surprisal values from very large llms provide in fact a very poor fit to reading times. @xcite hypothesize that this might be due to llms being "too confident" in their estimates of rare named entities compared to humans, thanks to their manifold larger exposure to data and greater memory capacity compared to humans. furthermore, work on nlp applications like question answering has reported that probability estimates from pretrained language models are often overconfident, i.e. they are higher than the ground truth probability @xcite . these findings hence beg the question whether current llms are well-calibrated with respect to "objective" word occurrence probabilities. relatedly, we ask whether llm probability estimates are overconfident compared to human estimates (as observed in reading times). one approach to address calibration problems is to use temperature scaling, as done e.g., in vision tasks @xcite . temperature-scaling with a temperature @xmath0. temperature-scaling hence incorporates uncertainty into the probability estimates from llms. we note that the idea to work with flattened dis-1 9598 tributions instead of the original probability distributions from llms is also related to contextual rényi entropy as discussed by @xcite , as well as the super/sub-linear surprisal effect by @xcite ; @xcite . however, rather than merely adjust the power of surprisal in super/sub-logarithmic patterns or the power of probability in rényi entropy, our work represents a distinct branch of study (i.e., probability calibration) in machine learning: shaping the probability distribution itself through shaping the logits before softmax. we also discuss the motivation for why a slightly flattened distribution may be more suitable, and whether this change in distribution is applied when calculating surprisal vs. when calculating entropy. our experimental results show that scaling probabilities can largely improve the fit to reading times in all 12 settings (3 corpora × 4 neural lms). our contributions are summarized as follows: (1) we propose temperature-scaled surprisal, where surprisal is calculated from temperaturescaled probabilities. (2) we demonstrate that temperature-scaling with temperature t≈2.5 improves predictability of human reading times of english texts compared to @xmath1. (3) we identify linguistic phenomena that correlate with the benefit of temperature-scaled surprisal by analyzing residual errors from regression models. ## predictive power for reading times in psycholinguistics, rts on a word are believed to correlate with its processing difficulty. rts can be gathered using different paradigms, including eyetracking while reading text on a screen @xcite , self-paced reading @xcite and the maze task @xcite . the most common procedure for predicting words' rt is first to select a set of predictor variables thought to impact rts @xmath2, ..., v (d) ] ⊤ ∈ r d , which include, e.g., the length of a word w t , |w t |, the frequency of a word freq(w t ). let f ϕ : r @xmath3: rt(w t |@xmath4, given the previous context @xmath5. the performance of f ϕ is quantified by its log-likelihood, with a higher loglikelihood indicating a better psychometric predictive power for human @xcite . besides the word length |w t | and word frequency 132 freq(w t ), a word's surprisal (i.e., its negative log-133 probability in context) @xcite 134 has been shown to be predictive of rts @xcite @xcite 136 @xcite . ## conclusion this paper studies the prediction of human rts from the perspective of probability distribution. we make the following contributions: (1) we demonstrate that the prediction of rts can be significantly improved via temperature scaling of llm probability estimates. (2) we demonstrate that the primary benefit of temperature-scaled surprisal is driven by words composed of several subword tokens. these words also tend to be rarer / long open-class words. future work should investigate the interaction of subword tokenization and temperature scaling, as well as the issue of tokenization in the analysis of eye-tracking data. 8 limitations 625 in this work, the identification of the optimal t for 626 temperature-scaled surprisal is manually tuned. fu-627 ture research could develop an automated method 628 to determine this optimal value, e.g., from specific 629 characteristics of llms or corpora. additionally, 630 a question may be asked whether the possible non-631 linear relationship between surprisal and reading 632 times @xcite could 633 influence the temperature-scaled surprisal's superi-634 ority over original surprisal. investigating the effec-635 tiveness of temperature-scaled surprisal using gen-636 eralized additive models, a branch of models that 637 assume less about the linearity than linear mixed ef-638 fect models employed here, would be an extension.