Temperature-scaling surprisal estimates improve fit to human reading times – but does it do so for the “right reasons”? (2024)

Abstract: A wide body of evidence shows that human 001 language processing difficulty is predicted by 002 the information-theoretic measure surprisal, 003 a word’s negative log probability in context. 004 However, it is still unclear how to best estimate 005 these probabilities needed for predicting human 006 processing difficulty – while a long-standing 007 belief held that models with lower perplexity 008 would provide more accurate estimates of word 009 predictability, and therefore lead to better read010 ing time predictions, recent work has shown 011 that for very large models, psycholinguistic 012 predictive power decreases. One reason could 013 be that language models might be more confi014 dent of their predictions than humans, because 015 they have had exposure to several magnitudes 016 more data. In this paper, we test what effect 017 temperature-scaling of large language model 018 (LLM) predictions has on surprisal estimates 019 and their predictive power of reading times of 020 English texts. Firstly, we show that calibration 021 of large language models typically improves 022 with model size, i.e. poorer calibration cannot 023 account for poorer fit to reading times. Sec024 ondly, we find that temperature-scaling prob025 abilities lead to a systematically better fit to 026 reading times (up to 89% improvement in delta 027 log likelihood), across several reading time cor028 pora. Finally, we show that this improvement in 029 fit is chiefly driven by words that are composed 030 of multiple subword tokens.1 031

text: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9598–9619 August 11-16, 2024 ©2024 Association for Computational Linguistics
A wide body of evidence shows that human001 language processing difficulty is predicted by002 the information-theoretic measure surprisal,003 a word’s negative log probability in context.004 However, it is still unclear how to best estimate005 these probabilities needed for predicting human006 processing difficulty – while a long-standing007 belief held that models with lower perplexity008 would provide more accurate estimates of word009 predictability, and therefore lead to better read-010 ing time predictions, recent work has shown011 that for very large models, psycholinguistic012 predictive power decreases. One reason could013 be that language models might be more confi-014 dent of their predictions than humans, because015 they have had exposure to several magnitudes016 more data. In this paper, we test what effect017 temperature-scaling of large language model018 (LLM) predictions has on surprisal estimates019 and their predictive power of reading times of020 English texts. Firstly, we show that calibration021 of large language models typically improves022 with model size, i.e. poorer calibration cannot023 account for poorer fit to reading times. Sec-024 ondly, we find that temperature-scaling prob-025 abilities lead to a systematically better fit to026 reading times (up to 89% improvement in delta027 log likelihood), across several reading time cor-028 pora. Finally, we show that this improvement in029 fit is chiefly driven by words that are composed030 of multiple subword tokens.1031
1 Introduction032
In psycholinguistics, a key finding is that words033 with higher surprisal (= negative log probability034 of the word in context) require more time for pro-035 cessing (Hale, 2001; Levy, 2008). Numerous stud-036 ies provided experimental evidence supporting this037 theory, demonstrating that surprisal is a powerful038 predictive measure of processing complexity (e.g.,039 Demberg and Keller, 2008; Wilcox et al., 2020,040
1Code available at: https://github.com/ TongLiu-github/TemperatureScaling4RTs.
2023; Shain et al., 2022), and that the relationship 041 between surprisal and reading times (RTs) seems 042 to be linear (Smith and Levy, 2013; Wilcox et al., 043 2020; Shain et al., 2022). 044
However, prior work implicitly made the as- 045 sumption that human predictability estimates 046 would be similar to the actual probability of a word 047 occurring in a given context, and that therefore, sur- 048 prisal values estimated from models that achieve 049 lower perplexities should also approximate human 050 processing difficulty better (Goodkind and Bick- 051 nell, 2018; Merkx and Frank, 2021). 052
Recent research has however found that this is 053 not true – surprisal values from very large LLMs 054 provide in fact a very poor fit to reading times. Oh 055 and Schuler (2023b) hypothesize that this might 056 be due to LLMs being “too confident” in their esti- 057 mates of rare named entities compared to humans, 058 thanks to their manifold larger exposure to data 059 and greater memory capacity compared to humans. 060 Furthermore, work on NLP applications like ques- 061 tion answering has reported that probability esti- 062 mates from pretrained language models are often 063 overconfident, i.e. they are higher than the ground 064 truth probability (Si et al., 2022; Kumar, 2022). 065 These findings hence beg the question whether 066 current LLMs are well-calibrated with respect to 067 “objective” word occurrence probabilities. Relat- 068 edly, we ask whether LLM probability estimates 069 are overconfident compared to human estimates (as 070 observed in reading times). 071
One approach to address calibration problems is 072 to use temperature scaling, as done e.g., in vision 073 tasks (Guo et al., 2017; Hendrycks et al., 2019). 074 Temperature-scaling with a temperature T > 1 075 has the effect that the probability distribution is 076 flattened such that it becomes more similar to a 077 uniform distribution. Temperature-scaling hence 078 incorporates uncertainty into the probability esti- 079 mates from LLMs. 080
We note that the idea to work with flattened dis- 081
1 9598
tributions instead of the original probability dis-082 tributions from LLMs is also related to contex-083 tual Rényi Entropy as discussed by Pimentel et al.084 (2023), as well as the super/sub-linear surprisal ef-085 fect by Shain et al. (2022); Hoover et al. (2023).086 However, rather than merely adjust the power of087 surprisal in super/sub-logarithmic patterns or the088 power of probability in Rényi entropy, our work089 represents a distinct branch of study (i.e., probabil-090 ity calibration) in machine learning: shaping the091 probability distribution itself through shaping the092 logits before softmax. We also discuss the motiva-093 tion for why a slightly flattened distribution may be094 more suitable, and whether this change in distribu-095 tion is applied when calculating surprisal vs. when096 calculating entropy.097
Our experimental results show that scaling prob-098 abilities can largely improve the fit to reading099 times in all 12 settings (3 corpora × 4 neural100 LMs). Our contributions are summarized as fol-101 lows: (1) We propose temperature-scaled surprisal,102 where surprisal is calculated from temperature-103 scaled probabilities. (2) We demonstrate that104 temperature-scaling with temperature T≈2.5 im-105 proves predictability of human reading times of106 English texts compared to T=1. (3) We identify lin-107 guistic phenomena that correlate with the benefit of108 temperature-scaled surprisal by analyzing residual109 errors from regression models.110
2 Predictive Power for Reading Times111
In psycholinguistics, RTs on a word are believed to112 correlate with its processing difficulty. RTs can be113 gathered using different paradigms, including eye-114 tracking while reading text on a screen (Rayner,115 1998), self-paced reading (Aaronson and Scarbor-116 ough, 1976; Mitchell and Green, 1978) and the117 Maze task (Forster et al., 2009).118
The most common procedure for predicting119 words’ RT is first to select a set of predictor vari-120 ables thought to impact RTs v = [v(1), ..., v(d)]⊤ ∈121 Rd, which include, e.g., the length of a word122 wt, |wt|, the frequency of a word freq(wt). Let123 fϕ : Rd → R be a regression model parametrized124 by ϕ used to fit these predictors for the prediction125 of human RTs rt: rt(wt|w<t) ∼ fϕ(v), given the126 previous context w<t. The performance of fϕ is127 quantified by its log-likelihood, with a higher log-128 likelihood indicating a better psychometric predic-129 tive power for human RTs (Frank and Bod, 2011;130 Fossum and Levy, 2012).131
Besides the word length |wt| and word frequency 132 freq(wt), a word’s surprisal (i.e., its negative log- 133 probability in context) (Hale, 2001; Levy, 2008) 134 has been shown to be predictive of RTs (Demberg 135 and Keller, 2008; Goodkind and Bicknell, 2018; 136 Wilcox et al., 2020; Shain et al., 2022). 137
3 Methods 138
In this section, we delve into key aspects of 139 information-theoretic measures in language com- 140 prehension. We start with surprisal, a method con- 141 necting processing difficulty to word predictabil- 142 ity. As word predictability is empirically estimated 143 by LLMs, we introduce the notion of calibration 144 errors, metrics quantifying how good the estima- 145 tion of word predictability is. Further, we lay out 146 temperature-scaled surprisal, and the relation be- 147 tween varying temperature vs. varying α in contex- 148 tual Rényi entropy. 149
3.1 Surprisal 150 Starting from Shannon (1948), the information con- 151 veyed by a word wt has been quantified as the 152 negative log probability of the word wt given its 153 previous context w<t. In Surprisal Theory (Hale, 154 2001; Levy, 2008), this quantity is called surprisal 155 s(wt) and proposed to be predictive of the word’s 156 processing difficulty, typically quantified as its RT. 157 Surprisal values are typically estimated from lan- 158 guage models p̂(wt|w<t). 159
s(wt) = −log2 p(wt|w<t), (1) 160
3.2 Calibration error 161 Definitions Let D = {(xi, yi)}Ni be a data set 162 where xi ∈ X is an sample (i.e., context) and 163 yi ∈ K = [K] is a category label. Let gθ and 164 ẑi = gθ(xi) denote a language model parametrized 165 by θ and the output logit vector of sample i, respec- 166 tively. The predicted class label ŷi for sample i is 167 given by ŷi = argmaxk∈K g(xi)k and confidence 168 for sample i is given by p̂i = maxk∈K g(xi)k. A 169 model is perfectly calibrated when the confidence 170 p̂ is equal to the frequency of correctness, i.e., 171 P(ŷi = yi|p̂i = p) = p holding for all p ∈ [0, 1] 172 and any sample i. Any difference between the left 173 and right sides of the above equation indicates there 174 exists a calibration error. 175
Expected calibration error (ECE) (Guo et al., 176 2017) ECE is the most popular calibration met- 177 ric, which empirically approximates the calibration 178
2
error by discretizing the probability interval into a179 fixed number of bins (Bm with m ∈ {1, 2, ...,M}),180 and measures the gaps of averaged confidence and181 averaged accuracy in each bin Bm.182
ECE = 1
N
M∑
m=1
| ∑
i∈Bm p̂i−
∑
i∈Bm 1[ŷi = yi]|, (2)183
where 1 is the indicator function. However, it does184 not necessarily measure the actual-word probabil-185 ity, which is the probability required for calculating186 surprisal in Eq. 1. It focuses only on the top-label187 probability for a given sample.188
Classwise-ECE (CECE) (Kumar et al., 2019;189 Kull et al., 2019) In comparison, CECE mea-190 sures probabilities of all classes. For each bin and191 every class k, it assesses the difference between the192 average confidence of samples for class k and the193 actual proportion of class k. If assuming all classes194 weigh equally, we have:195
CECE
= 1
NK
K∑
k=1
M∑
m=1
| ∑
i∈Bm p̂i,k −
∑
i∈Bm 1[k = yi]|,
(3)
196
where p̂i,k is the predicted probability of sample i197 for class k.198
Human-likeness calibration error (HCE) We199 define the HCE as the Kullback-Leibler divergence200 (KL divergence) between predicted probability p̂201 from a neural LM and actual probability p∗ of hu-202 man language model.203
HCE = DKL(p̂||p∗). (4)204
Empirically, since p∗ is not directly observable, we205 approximate it by the estimates of a temperature-206 scaled model that best fits human reading times207 (as discussed later). We denote the approximated208 HCE using such a method as HCETS.209
3.3 Temperature-scaled surprisal210 Temperature scaling (Guo et al., 2017) is a widely-211 used method to improve model calibration. Given212 the output logit vector ẑi for sample i, a single213 scalar T > 0 is applied to rescale ẑi before the214 softmax activation:215
q̂i = max k σSM ( ẑi T )(k), (5)216
where q̂i is the calibrated confidence for sample 217 i, and σSM is the softmax function. Scaling by 218 a scalar T does not alter the ranking; hence, the 219 predicted label ŷi remains unchanged. As T > 1, it 220 “softens” the probability distribution (i.e., makes the 221 distribution more uniform), increasing uncertainty 222 and entropy of the probability distribution, while 223 T < 1 peaks the distribution. The parameter T in 224 research on calibration is optimized by minimizing 225 the negative log-likelihood on the validation set. In 226 our experiments of fit to human RTs, we manually 227 tune this temperature with T > 1. 228
Temperature scaling has been successfully ap- 229 plied in several applications: In knowledge distilla- 230 tion (Hinton et al., 2015), temperature scaling (with 231 T > 1) is used to “soften” the knowledge (i.e., 232 probability distribution) provided by the teacher 233 model; in text generation, temperature is used to 234 shape the probability distribution to ease certain 235 aspects of the problems of top-k sampling (e.g., 236 choosing an appropriate k value across varying 237 contexts) (Ficler and Goldberg, 2017; Fan et al., 238 2018). Temperature tuning inherently shifts the 239 model’s output in the generation’s quality/diversity 240 spectrum (Caccia et al., 2018), with higher tem- 241 perature decreasing the quality of generation while 242 improving its diversity. This also aligns with our 243 consideration of a possibility that human proba- 244 bility distributions might be flatter than the ones 245 learned by language models and thus increasing the 246 predictive diversity of surprisal provided by LLMs 247 could potentially yield more human-like distribu- 248 tions. 249
Given Eq. 5, temperature-scaled surprisal is: 250
sT (wt, T ) = −log2(σSM (ẑwt/T )(k ∗)), (6) 251
where ẑwt and k ∗ = ywt denote the logit vector and 252 the actual word wt class, respectively. For given 253 t ∈ (0,∞), we simply denote sT (wt, T = t) as 254 sT |T=t. A temperature T with its best performance 255 of final fit to RTs is denoted as T ∗. 256
The extent to which a word’s surprisal is affected 257 by temperature scaling depends on the distribution 258 and thus correlates with the entropy at word wt. 259 Consider an example of two five-class probability 260 distributions pi = [0.8, 0.05, 0.05, 0.05, 0.05] 261 and pj = [0.8, 0.2, 0, 0, 0], for which the word 262 indicated by the first position in the probabil- 263 ity vector has identical surprisal in both pi 264 and pj . Notably, pi is more uniform and pj 265 is more peaked, resulting in distinct entropy 266
3
characteristics: H(wi|w<i) > H(wj |w<j),267 where the entropy defined as the expectation268 of surprisal of current word wt over vocab-269 ulary, H(wt|w<t) = Ew′∼p(·|w<t)[s(w ′ )] =270
−∑w′∈W p(w ′ |w<t) log2 p(w
′ |w<t), where271 W = W ∪ {EOS} denotes the set of vocabulary272 W with EOS token. Fig. 1 illustrates a greater273 increase in surprisal for a word with a more274 uniform distribution than with a more peaked275 distribution.276
This figure also anecdotally shows that the effect277 of applying temperature scaling with T > 1 is278 similar to the effect of setting α < 1 in Rényi279 entropy. We will discuss the relationship between280 these parameters in more detail in Appendix A.281
4 Experimental setup282
4.1 Datasets283
We conduct analyses on two self-paced reading284 corpora, the Natural Stories Corpus (Futrell et al.,285 2018) and the Brown Corpus (Smith and Levy,286 2013), as well as on the Dundee Corpus (Kennedy287 et al., 2003), which contains the eye-movement288 record; our analyses in this paper focus on first-289 pass times2 from the Dundee corpus. We follow290 previous work with respect to the preprocessing291 steps for each corpus (Kuribayashi et al., 2022;292 Shain et al., 2022). Appendix C includes details293 about the preprocessing steps of each corpus.294
2First pass times are calculated as the sum of all fixation durations from first entering to first leaving the word during the first pass, i.e., only those cases are counted where no words further advanced in the text have been fixated.
4.2 Language Models 295
Recent observations showed that surprisal provided 296 by LLMs with more parameters and lower perplex- 297 ity is less predictive of self-paced reading times 298 and eye-gaze durations (Shain et al., 2022; Oh 299 and Schuler, 2023b); across different experiments, 300 GPT-2 (Radford et al., 2019) surprisals were found 301 to predict human RTs the best. Therefore, we take 302 four variants of pretrained GPT-2 (small, medium, 303 large, xl) as our language models in all experiments. 304 Following prior work, we obtain the surprisal for 305 words composed of more than one subword by sum- 306 ming up the surprisal estimates of the subwords. 307
4.3 Metrics and evaluation 308
We measure the predictive power of surprisal es- 309 timates from different language models, which is 310 denoted as the log-likelihood difference per data 311 point between a linear mixed-effects (LME) re- 312 gression model using lme4 package (Bates et al., 313 2015) with a predictor of surprisal estimates (target 314 model) and a model without surprisal (base model), 315 following Goodkind and Bicknell (2018); Wilcox 316 et al. (2020). More specifically, the metric of delta 317 log-likelihood is defined as: 318
∆llh = llh(fϕ(v tgt))− llh(fϕ(vbase)), (7) 319
where vtgt is target predictor variables that in- 320 clude baseline predictor variables as well as pre- 321 dictor variables of our interest, such as surprisal 322 or temperature-scaled surprisal. vbase is base pre- 323 dictor variables only including baseline predictor 324 variables. The greater the value of ∆llh, the more 325 valuable the additional surprisal estimates are for 326 predicting human reading times. 327
For the calibration error evaluation, we set the 328 number of bins M to 15 for both ECE and CECE, 329 aligning with prior literature, such as works by Guo 330 et al. (2017); Kumar et al. (2019); Rahimi et al. 331 (2020b), to ensure consistency in addressing prob- 332 lems where comparable probability ranges are rele- 333 vant. The calibration metrics (ECE and CECE) are 334 evaluated separately on each of the reading time 335 corpus D. For simplicity, our calibration evaluation 336 is conducted at the token level. Given that many 337 words have extremely low probabilities and thus 338 are often grouped into a single bin, we also evalu- 339 ate the calibration error under the log probability 340 binning scheme. For further descriptions regarding 341 the metrics and evaluation, see Appendix D. 342
4
5 Results343
5.1 Calibration of LLMs344
Table 1 shows ECE and CECE in log binning345 scheme for GPT-2 models of different sizes. LLMs346 are in general well calibrated on language mod-347 eling. Besides, LLM calibration improves with348 scale. Larger LMs are better calibrated. This349 conclusion is consistent with calibration investiga-350 tion evaluated in BIG-bench multiple-choice tasks351 in Srivastava et al. (2023) as well as in several tasks352 including language modelling in Zhu et al. (2023).353
5.2 Main result: temperature-scaled surprisal354 improves human reading time prediction355
We evaluate the predictive power of temperature-356 scaled surprisal. We scale T in the range of [1, 10]357 and measure ∆llh, see Fig. 2. First, a confirma-358 tory observation regarding the relationship between359 model size and predictive power: At T = 1, GPT-2360 small exhibits the best predictive performance, and361 as the model size increases, ∆llh declines, which is362 consistent with previous studies (Shain et al., 2022;363 Oh et al., 2022; Oh and Schuler, 2023b). Secondly,364 scaling the surprisal with T > 1 can signifi-365 cantly improve the predictive power across all366 corpora and LLMs. With optimal T ∗, on Dundee,367 Natural Stories, and Brown, the ∆llh improvement368 is 23-43%, 60-89%, and 14-24%, respectively. We369 assess statistical significance of GPT-2 small in Ap-370 pendix H, where we report a result of p < 0.001371 on three corpora. We also observe a consistent372 pattern: when increasing T , ∆llh first rises then373 declines; the optimal value T ∗ falls within the374 range of (2, 3) (around 2.5) across all models375 and corpora in our setting. At T ∗, even though the376 impact of model size on final performance is not377 fully recovered, the disparity diminishes. Smaller378 models continue to outperform, but the extent of379 model sizes influencing performance is reduced.380 Finally, larger LMs typically have a larger381 human-likeness calibration error, shown in Ta-382 ble 1. Larger LMs also require a higher value of T383 to reach their best performance and have a greater384 increase by temperature-scaled surprisal.385
5.3 Calibration error vs. RT prediction error386
Table 2 shows ECE and CECE in both equally-387 spaced and log binning schemes when T equals 1388 and T ∗ on three corpora. Probability distribution389 shaped by an optimal T ∗ learnt for fit to human390
RTs drastically hurts the model calibration regard- 391 ing these two metrics. ECE and CECE with T ∗ are 392 more than 10 times worse than values with T = 1. 393 This discrepancy can be attributed to the differ- 394 ent minima of deviations in LM human RT pre- 395 diction and expected calibration error. The former 396 is minimized towards words where LMs surprisal 397 significantly deviates from human processing diffi- 398 culty, while the latter is typically minimized with 399 respect to the negative log-likelihood on a hold-out 400 dataset (Guo et al., 2017; Rahimi et al., 2020a). 401
6 Linguistic analysis 402
Next we want to gain insight into what words ben- 403 efit the most from temperature scaling. To this end, 404 we analyze residuals from fitting LME regression 405 models, identifying data points where scaling the 406 temperature parameter notably enhances the fit of 407 human RTs. Specifically, we quantify the improve- 408 ment in fit by comparing the mean squared error 409 (MSE) before and after adjusting the temperature 410
5
to its optimal value as follows:411
∆MSE(F ) = MSET=1(xF )−MSET=T ∗(xF ), (8) 412
where MSET=T ′ (xF ) is the MSE calculated by all413 the data xF under the linguistic factor F . The414 difference ∆MSE(F ) thus quantifies the impact415 of scaling relative to the linguistic factor F . A416 higher ∆MSE(F ) signifies a greater influence of417 temperature-scaled surprisal of factor F . To ensure418 sufficient data in each subset, we only consider sub-419 sets including more than 1% of the data in each420 corpus.421
6.1 Influence of low probability words422
Given that temperature scaling enhances human423 likeness by shaping the probability distribution,424 it is natural to think about investigating whether425 there exists an inherent relationship between the426 distribution of probability and ∆MSE. Specifically,427 one might ask questions like if samples with low428 probability gain more from temperature scaling or429 the other way around. We find that high surprisal430 words benefit more from temperature scaling than431 low surprisal words, across all corpora, see Fig. 3.432
6.2 Influence of word types 433
We investigate the effects of word-level properties, 434 which include: 435
Named entities. Research has substantiated 436 that named entities (NEs) require increased read- 437 ing time for humans since during the processing 438 of such words (Damasio et al., 2004; Wang et al., 439 2013). Oh and Schuler (2023b) showed that NEs 440 are among the top two significant factors contribut- 441 ing to the discrepancies of large and small LMs 442 across all corpus-by-LM combinations. There- 443 fore, we were wondering whether the effect of 444 temperature-scaling might be driven by NE. To test 445 this, we automatically tagged NEs using a BERT 446 base model (Devlin et al., 2019) fined-tuned for 447 NER3. 448
Part-of-speech tags. Similarly, previous re- 449 search has argued that the poor fit of large LMs 450 is primarily due to assigning too low surprisal es- 451 timates to open-class words like nouns and adjec- 452 tives (Oh and Schuler, 2023b). We POS-tagged 453 the corpora using the NLTK toolkit (Bird et al., 454 2009) with the default Penn Treebank Tag set. In 455 the following, we mainly focus on the four classes 456 of open-class tags, as well as a subset of the whole 457 closed-class tags (CC). 458
3Link: https://huggingface.co/dslim/bert-base-NER
6
Results. The result, as shown in Table 3, shows459 primary factors responsible for the benefit of us-460 ing sT (wt, T ) for each corpus-by-LM combination.461 The top three influential subsets for each corpus462 are underlined. Among all datasets and models,463 named entities perform to be the most benefi-464 cial word-level attribute. In contrast, closed-class465 words profit the least from temperature scaling.466 Performance trends are consistent across different467 model variants on the same corpus.468
We also measured empirically how often tem-469 perature scaling increased vs. decreased the sur-470 prisal estimate of a word. Our results show that for471 ca. 90% of words, surprisal estimates are increased472 through temperature scaling across all word classes.473 For the subset of named entities, a slightly smaller474 percentage exhibits increased surprisal estimates.475 For a full analysis across different corpora and mod-476 els, see Table 5 in Appendix B.477
We further investigate the benefit of temperature-478 scaled surprisal (quantified by ∆MSE) given the479 subset of words whose probability decreases (or480 increases). The results are in Table 4. On Dundee,481 the main gain arises from the reduction of large482 probabilities via temperature scaling. Conversely,483 for Natural Stories, the primary benefit comes more484 strongly from words with originally very low prob-485 ability, which become more probable. For Brown,486 the effects are evenly split. These findings align487 with our theoretical intuition that temperature488 scaling enhances the fit performance by mak-489 ing probabilities more smooth, which means not490 only making high probabilities lower but also mak-491 ing very low probabilities higher and close to 1/K,492 since a very low probability also means the model493 is confident in the incorrectness of certain classes.494
Considering effects on named entities more495 specifically, we find that on Natural Stories and496 Brown, the benefit of temperature scaling can497 mostly be attributed to reducing the probability498 estimates of highly predictable entities, while on499 Dundee the beneficial effect mostly arises from in-500 creasing probabilities of named entities. We spec-501 ulate that this could be due to the types of most502 frequent named entities that occur in the different503 text sorts, and present a more detailed analysis of504 this aspect in Appendix B.505
6.3 Influence of multiple-token words506
A fact that is often ignored (but see Nair and Resnik,507 2023) is that modern LLMs use subword tokeniza-508
tion. This means that long words may consist of 509 several tokens. In this case, the probability of the 510 complete word is calculated by multiplying the 511 probabilities of the subword tokens (and the word’s 512 surprisal is correspondingly calculated by adding 513 the surprisals of the subwords). While this may 514 often not matter, whether a word is tokenized into 515 a single subword or several subwords can make a 516 remarkable difference when applying temperature 517 scaling: imagine a long / difficult word which has 518 a low probability (and correspondingly a high sur- 519 prisal). If this word were to be represented as a 520 single subword token, temperature scaling might 521 have the effect that the probability of this word 522 gets increased during temperature scaling, and its 523 surprisal estimate is hence decreased at T > 1. 524
If, on the other hand, the same word were to 525 be composed of two subword tokens, one or both 526 of the subword tokens can be expected to have a 527 higher probability (than a hypothetical single sub- 528 word token), and it is possible that during tempera- 529 ture scaling, the probabilities of the subword tokens 530 would each be decreased at T > 1, such that the 531 sum of the surprisals of the subword tokens would 532 be much higher, compared to the word’s surprisal 533 estimate at T = 1. 534
To summarize, whether the surprisal of a certain 535 word would increase or decrease after temperature 536 scaling could depend on whether that word happens 537 to be included in the subword token vocabulary or 538
7
not.4 In order to quantify to what extent subword539 tokenization affects surprisal estimates, we con-540 ducted several analyses.541 Fig. 4 shows ∆llh under various conditions:542 scaling all words (consistent with experiments in543 Section 5.2) vs. taking into the analysis only the544 subset of single-token words and multiple-token545 words. The comparison between the full, dotted,546 and dashed lines highlights that the benefit of547 temperature-scaled surprisal comes primarily548 from the scaling of multiple-token words.549 Next, it is interesting to consider for what550 percentage of multiple-token words temperature-551 scaling increases the surprisal. We find that the sur-552 prisal of more than 90% of multiple-token words in-553 creases, and the ratio is higher than across single-554 token words by ca. 6% on Dundee and Brown, see555 Table 12 in Appendix L for more details.556
7 Discussion557
Our experiments demonstrate that choosing a tem-558 perature around 2.5 improves the fit to human read-559 ing times. Furthermore, we find that this effect is560 chiefly driven by an improved fit for words which561 consist of several subword tokens.5 Named entities562 and other open class words tend to have a larger563 tendency to contain several subword tokens, which564 can explain why temperature scaling is particularly565 effective for these words.566 So what does all of this mean for surprisal esti-567 mates from LLMs and reading time prediction?568 Firstly, following the argumentation of Oh and569 Schuler (2023b), it is possible that indeed the effect570 is driven by humans failing to accurately estimate571 the probability of rare words (rare words being572 the ones that are split up into several subwords),573 because they do not reach sufficient language ex-574
4Distributions of surprisal for single vs. multiple token words before and after temperature scaling are provided in Fig. 8 in Appendix L.
5Appendix K shows that subword tokenization has larger explanatory power than word class.
perience or because human language models do 575 not track these probabilities well. In this case, 576 temperature-scaling rare words to which the LLM 577 assigns a too high probability (and hence a low 578 surprisal) would be a good strategy to counteract 579 the discrepancy between humans and LLMs. From 580 LLMs’ perspective, recalling the observation from 581 Section 5.3 that larger LLMs that yield poorer fits 582 to RTs are actually better calibrated, hence the mas- 583 sive training dataset might be at the cause of driving 584 these models away from the human-like predictive 585 processing, aligning with Oh and Schuler (2023a). 586
Secondly, it is likely that the beneficial effect of 587 temperature scaling is an artifact of subword tok- 588 enization, and that this effect would diminish if all 589 words were composed of only a single subword 590 token (cf. our explanation in Section 6.3). That 591 is, temperature scaling would not be beneficial be- 592 cause of the reasons that motivated this research 593 originally, but only because it is a way of assign- 594 ing higher surprisal to words consisting of several 595 subword tokens. In order to test this hypothesis, 596 one would have to re-train a GPT-2 model using a 597 vocabulary that at least includes all words that are 598 contained in the reading time corpora, and then re- 599 running the analysis to check whether a beneficial 600 effect of temperature scaling can still be found. 601
Finally, it is also possible that the splitting of a 602 word into subwords coincides with the reader fixat- 603 ing a word several times, and that these added fixa- 604 tions lead to an overestimate in RTs compared to 605 the actual surprisal experienced by a human reader. 606 Future work could investigate this hypothesis by 607 analysing RTs on subwords instead of aggregated 608 words (with the caveat that subword tokens may 609 not be cognitively plausible units). 610
8 Conclusion 611
This paper studies the prediction of human RTs 612 from the perspective of probability distribution. We 613 make the following contributions: (1) We demon- 614 strate that the prediction of RTs can be significantly 615 improved via temperature scaling of LLM probabil- 616 ity estimates. (2) We demonstrate that the primary 617 benefit of temperature-scaled surprisal is driven by 618 words composed of several subword tokens. These 619 words also tend to be rarer / long open-class words. 620 Future work should investigate the interaction of 621 subword tokenization and temperature scaling, as 622 well as the issue of tokenization in the analysis of 623 eye-tracking data. 624
8
Limitations625
In this work, the identification of the optimal T for626 temperature-scaled surprisal is manually tuned. Fu-627 ture research could develop an automated method628 to determine this optimal value, e.g., from specific629 characteristics of LLMs or corpora. Additionally,630 a question may be asked whether the possible non-631 linear relationship between surprisal and reading632 times (Shain et al., 2022; Hoover et al., 2023) could633 influence the temperature-scaled surprisal’s superi-634 ority over original surprisal. Investigating the effec-635 tiveness of temperature-scaled surprisal using gen-636 eralized additive models, a branch of models that637 assume less about the linearity than linear mixed ef-638 fect models employed here, would be an extension.639 Finally, exploring effects of temperature-scaled sur-640 prisal on different measures of fixation duration641 could be considered in future work.642
Ethical Considerations643
The datasets and packages we used are all publicly644 available and have no privacy issues.645
Acknowledgements646
The authors thank Xudong Hong and Dongqi Pu647 for useful discussions and comments.648
References649 Doris Aaronson and Hollis S Scarborough. 1976. Per-650 formance theories for sentence coding: Some quanti-651 tative evidence. Journal of Experimental Psychology:652 Human perception and performance, 2(1):56.653
Bernhard Angele, Elizabeth R Schotter, Timothy J Slat-654 tery, Tara L Tenenbaum, Klinton Bicknell, and Keith655 Rayner. 2015. Do successor effects in reading reflect656 lexical parafoveal processing? evidence from corpus-657 based and experimental eye movement data. Journal658 of Memory and Language, 79:76–96.659
Christoph Aurnhammer and Stefan L Frank. 2019. Eval-660 uating information-theoretic measures of word pre-661 diction in naturalistic sentence reading. Neuropsy-662 chologia, 134:107198.663
Douglas Bates, Martin Mächler, Ben Bolker, and Steve664 Walker. 2015. Fitting linear mixed-effects models665 using lme4. Journal of Statistical Software, 67:1–48.666
Steven Bird, Ewan Klein, and Edward Loper. 2009. Nat-667 ural language processing with Python: analyzing text668 with the natural language toolkit. " O’Reilly Media,669 Inc.".670
Massimo Caccia, Lucas Caccia, William Fedus, Hugo 671 Larochelle, Joelle Pineau, and Laurent Charlin. 672 2018. Language gans falling short. arXiv preprint 673 arXiv:1811.02549. 674
Hanna Damasio, Daniel Tranel, Thomas Grabowski, 675 Ralph Adolphs, and Antonio Damasio. 2004. Neural 676 systems behind word and concept retrieval. Cogni- 677 tion, 92(1-2):179–229. 678
Vera Demberg and Frank Keller. 2008. Data from eye- 679 tracking corpora as evidence for theories of syntactic 680 processing complexity. Cognition, 109(2):193–210. 681
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and 682 Kristina Toutanova. 2019. BERT: Pre-training of 683 deep bidirectional transformers for language under- 684 standing. In Proceedings of the 2019 Conference of 685 the North American Chapter of the Association for 686 Computational Linguistics: Human Language Tech- 687 nologies, Volume 1 (Long and Short Papers), pages 688 4171–4186, Minneapolis, Minnesota. Association for 689 Computational Linguistics. 690
Angela Fan, Mike Lewis, and Yann Dauphin. 2018. 691 Hierarchical neural story generation. In Proceedings 692 of the 56th Annual Meeting of the Association for 693 Computational Linguistics (Volume 1: Long Papers), 694 pages 889–898, Melbourne, Australia. Association 695 for Computational Linguistics. 696
Jessica Ficler and Yoav Goldberg. 2017. Controlling 697 linguistic style aspects in neural language generation. 698 In Proceedings of the Workshop on Stylistic Variation, 699 pages 94–104, Copenhagen, Denmark. Association 700 for Computational Linguistics. 701
Kenneth I Forster, Christine Guerrera, and Lisa Elliot. 702 2009. The maze task: Measuring forced incremental 703 sentence processing time. Behavior research meth- 704 ods, 41:163–171. 705
Victoria Fossum and Roger Levy. 2012. Sequential vs. 706 hierarchical syntactic models of human incremen- 707 tal sentence processing. In Proceedings of the 3rd 708 Workshop on Cognitive Modeling and Computational 709 Linguistics (CMCL 2012), pages 61–69, Montréal, 710 Canada. Association for Computational Linguistics. 711
Stefan L Frank and Rens Bod. 2011. Insensitivity of the 712 human sentence-processing system to hierarchical 713 structure. Psychological science, 22(6):829–834. 714
Richard Futrell, Edward Gibson, Harry J Tily, Idan 715 Blank, Anastasia Vishnevetsky, Steven Piantadosi, 716 and Evelina Fedorenko. 2018. The natural stories 717 corpus. In Proceedings of the Eleventh International 718 Conference on Language Resources and Evaluation 719 (LREC 2018). 720
Adam Goodkind and Klinton Bicknell. 2018. Predictive 721 power of word surprisal for reading times is a linear 722 function of language model quality. In Proceedings 723 of the 8th Workshop on Cognitive Modeling and Com- 724 putational Linguistics (CMCL 2018), pages 10–18, 725 Salt Lake City, Utah. Association for Computational 726 Linguistics. 727
9
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Wein-728 berger. 2017. On calibration of modern neural net-729 works. In International conference on machine learn-730 ing, pages 1321–1330. PMLR.731
John Hale. 2001. A probabilistic earley parser as a732 psycholinguistic model. In Second meeting of the733 north american chapter of the association for com-734 putational linguistics.735
John Hale. 2003. The information conveyed by words736 in sentences. Journal of psycholinguistic research,737 32:101–123.738
John Hale. 2006. Uncertainty about the rest of the739 sentence. Cognitive science, 30(4):643–672.740
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. 2019.741 Using pre-training can improve model robustness and742 uncertainty. In International conference on machine743 learning, pages 2712–2721. PMLR.744
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015.745 Distilling the knowledge in a neural network. arXiv746 preprint arXiv:1503.02531.747
Jacob Louis Hoover, Morgan Sonderegger, Steven T748 Piantadosi, and Timothy J O’Donnell. 2023. The749 plausibility of sampling as an algorithmic theory of750 sentence processing. Open Mind, 7:350–391.751
Alan Kennedy, Robin Hill, and Joël Pynte. 2003. The752 dundee corpus. In Proceedings of the 12th European753 Conference on Eye Movement.754
Meelis Kull, Miquel Perello Nieto, Markus Kängsepp,755 Telmo Silva Filho, Hao Song, and Peter Flach.756 2019. Beyond temperature scaling: Obtaining well-757 calibrated multi-class probabilities with dirichlet cal-758 ibration. Advances in neural information processing759 systems, 32.760
Ananya Kumar, Percy S Liang, and Tengyu Ma. 2019.761 Verified uncertainty calibration. Advances in Neural762 Information Processing Systems, 32.763
Sawan Kumar. 2022. Answer-level calibration for free-764 form multiple choice question answering. In Pro-765 ceedings of the 60th Annual Meeting of the Associa-766 tion for Computational Linguistics (Volume 1: Long767 Papers), pages 665–679, Dublin, Ireland. Association768 for Computational Linguistics.769
Tatsuki Kuribayashi, Yohei Oseki, Ana Brassard, and770 Kentaro Inui. 2022. Context limitations make neural771 language models more human-like. In Proceedings772 of the 2022 Conference on Empirical Methods in773 Natural Language Processing, pages 10421–10436,774 Abu Dhabi, United Arab Emirates. Association for775 Computational Linguistics.776
Roger Levy. 2008. Expectation-based syntactic compre-777 hension. Cognition, 106(3):1126–1177.778
Tal Linzen and T Florian Jaeger. 2014. Investigating the 779 role of entropy in sentence processing. In Proceed- 780 ings of the fifth workshop on cognitive modeling and 781 computational linguistics, pages 10–18. 782
Danny Merkx and Stefan L. Frank. 2021. Human sen- 783 tence processing: Recurrence or attention? In Pro- 784 ceedings of the Workshop on Cognitive Modeling 785 and Computational Linguistics, pages 12–22, Online. 786 Association for Computational Linguistics. 787
Don C Mitchell and David W Green. 1978. The ef- 788 fects of context and content on immediate processing 789 in reading. The quarterly journal of experimental 790 psychology, 30(4):609–636. 791
Sathvik Nair and Philip Resnik. 2023. Words, subwords, 792 and morphemes: What really matters in the surprisal- 793 reading time relationship? In Findings of the As- 794 sociation for Computational Linguistics: EMNLP 795 2023, pages 11251–11260, Singapore. Association 796 for Computational Linguistics. 797
Byung-Doh Oh, Christian Clark, and William Schuler. 798 2022. Comparison of structural parsers and neural 799 language models as surprisal estimators. Frontiers in 800 Artificial Intelligence, 5:777963. 801
Byung-Doh Oh and William Schuler. 2023a. 802 Transformer-based language model surprisal 803 predicts human reading times best with about two 804 billion training tokens. In Findings of the Associa- 805 tion for Computational Linguistics: EMNLP 2023, 806 Singapore, December 6-10, 2023, pages 1915–1921. 807 Association for Computational Linguistics. 808
Byung-Doh Oh and William Schuler. 2023b. Why 809 does surprisal from larger transformer-based lan- 810 guage models provide a poorer fit to human reading 811 times? Transactions of the Association for Computa- 812 tional Linguistics, 11:336–350. 813
Tiago Pimentel, Clara Meister, Ethan G. Wilcox, Roger 814 Levy, and Ryan Cotterell. 2023. On the effect of 815 anticipation on reading times. Transactions of the 816 Association for Computational Linguistics. 817
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, 818 Dario Amodei, Ilya Sutskever, et al. 2019. Language 819 models are unsupervised multitask learners. OpenAI 820 blog, 1(8):9. 821
Amir Rahimi, Kartik Gupta, Thalaiyasingam Ajan- 822 than, Thomas Mensink, Cristian Sminchisescu, and 823 Richard Hartley. 2020a. Post-hoc calibration of neu- 824 ral networks. arXiv preprint arXiv:2006.12807, 2. 825
Amir Rahimi, Amirreza Shaban, Ching-An Cheng, 826 Richard Hartley, and Byron Boots. 2020b. Intra 827 order-preserving functions for calibration of multi- 828 class neural networks. Advances in Neural Informa- 829 tion Processing Systems, 33:13456–13467. 830
Keith Rayner. 1998. Eye movements in reading and 831 information processing: 20 years of research. Psy- 832 chological bulletin, 124(3):372. 833
10
David Reeb and Michael M Wolf. 2015. Tight bound on834 relative entropy by entropy difference. IEEE Trans-835 actions on Information Theory, 61(3):1458–1473.836
Alfréd Rényi. 1961. On measures of entropy and infor-837 mation. In Proceedings of the Fourth Berkeley Sym-838 posium on Mathematical Statistics and Probability,839 Volume 1: Contributions to the Theory of Statistics,840 volume 4, pages 547–562. University of California841 Press.842
Cory Shain, Clara Meister, Tiago Pimentel, Ryan Cot-843 terell, and Roger Levy. 2022. Large-scale evidence844 for logarithmic effects of word predictability on read-845 ing time.846
Claude Elwood Shannon. 1948. A mathematical theory847 of communication. The Bell system technical journal,848 27(3):379–423.849
Chenglei Si, Chen Zhao, Sewon Min, and Jordan Boyd-850 Graber. 2022. Re-examining calibration: The case851 of question answering. In Findings of the Associa-852 tion for Computational Linguistics: EMNLP 2022,853 pages 2814–2829, Abu Dhabi, United Arab Emirates.854 Association for Computational Linguistics.855
Nathaniel J Smith and Roger Levy. 2013. The effect856 of word predictability on reading time is logarithmic.857 Cognition, 128(3):302–319.858
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao,859 Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch,860 Adam R Brown, Adam Santoro, Aditya Gupta, Adrià861 Garriga-Alonso, et al. 2023. Beyond the imitation862 game: Quantifying and extrapolating the capabili-863 ties of language models. Transactions on Machine864 Learning Research.865
Marten van Schijndel and Tal Linzen. 2019. Can en-866 tropy explain successor surprisal effects in reading?867 In Proceedings of the Society for Computation in868 Linguistics (SCiL) 2019, pages 1–7.869
Lin Wang, Zude Zhu, Marcel Bastiaansen, Peter Ha-870 goort, and Yufang Yang. 2013. Recognizing the emo-871 tional valence of names: An erp study. Brain and872 Language, 125(1):118–127.873
Ethan Gotlieb Wilcox, Jon Gauthier, Jennifer Hu, Peng874 Qian, and Roger P. Levy. 2020. On the predictive875 power of neural language models for human real-876 time comprehension behavior. In Proceedings of877 the 42nd Annual Meeting of the Cognitive Science878 Society, page 1707–1713.879
Ethan Gotlieb Wilcox, Tiago Pimentel, Clara Meister,880 Ryan Cotterell, and Roger P Levy. 2023. Testing881 the predictions of surprisal theory in 11 languages.882 Transactions of the Association for Computational883 Linguistics.884
Chiwei Zhu, Benfeng Xu, Quan Wang, Yongdong885 Zhang, and Zhendong Mao. 2023. On the calibra-886 tion of large language models and alignment. In887
Findings of the Association for Computational Lin- 888 guistics: EMNLP 2023, pages 9778–9795, Singapore. 889 Association for Computational Linguistics. 890
11

heading: A Connection to Contextual Rényi891
text: Entropy892
While a lot of work has investigated the effect of893 next word entropy on reading times (Hale, 2003,894 2006; Linzen and Jaeger, 2014; Angele et al., 2015;895 van Schijndel and Linzen, 2019; Aurnhammer and896 Frank, 2019; Pimentel et al., 2023), we will here fo-897 cus on contextual Rényi entropy (the entropy of the898 probability distribution at the current time stamp,899 which is parameterized by α), as proposed in Pi-900 mentel et al. (2023) to represent human anticipatory901 reading process. Pimentel et al. (2023) find that902 Rényi entropy with an optimal α∗ in the range of903 (0, 1) (around 1/2) obtains the best performance904 in reading time prediction (compared to Shannon905 Entropy (α = 1) or compared to unscaled surprisal906 estimates).907 Mathematically, Contextual Rényi en-908 tropy (Rényi, 1961) is defined as:909
Hα(wt | w<t)
= lim β→α
1 1− β log2 ∑
w∈W (p(w|w<t))β.
(9)
910
For given α ′ ∈ (0,∞), we simply denote Hα(wt |911 w<t)|α=α′ as Hα|α=α′ .912 Theorem 1 (Monotonicity of sT (wt, T ) and913 Hα(wt | w<t)). Given any probability distribution914 p with actual-word probability pwt > 1/K, where915 K is the number of classes, temperature-scaled sur-916 prisal sT (wt, T ) is strictly monotonically increas-917 ing in ∆T ∈ [1,∞], Rényi entropy Hα(wt | w<t)918 is strictly monotonically decreasing in ∆α ∈ [0, 1],919 especially,920
sT |T=1 < sT |T=T ∗ < lim T→∞ sT (wt, T ) (10)921 Hα|α=1 < Hα|α=1/2 < Hα|α=0, (11)922 where T ∗ is the optimal T of fit to RTs in the range923 of ∆T .924
Proof. Eq. (10) can be easily verified by con-925 sidering the monotonicity of temperature-scaled926 softmax output σSM (ẑwt/T ). The second part of927 Eq. (11) can be rewritten as:928
Hα|α=1/2 = 2 log2 ∑
w∈W
√ p(w|w<t) (12)929
< 2 log2
√ K ∑
w∈W p(w|w<t) (13)930
= − log2(1/K) = Hα|α=0, (14)931
where for the step from Eq. (12) to Eq. (13) we 932 use AM-QM inequality and K is the number of 933 classes in tokenizer. The first part of Eq. (11) can 934 be rewritten as: 935
Hα|α=1/2 = 2 log2 ∑
w∈W
√ p(w|w<t) (15) 936
> 2 log2
√√√√ ∏
w∈W (
1
p(w|w<t) )p(w|w<t)
(16)
937
= ∑
w∈W p(w|w<t) log2 p(w|w<t) = Hα|α=1,
(17)
938
where from Eq. (15) to Eq. (16) we use AM-GM 939 inequality. 940
Theorem 2 Rényi entropy with α = 0 is equiva- 941 lent to temperature-scaled surprisal with T → ∞. 942
Hα(wt | w<t)|α=0 = lim T→∞ sT (wt, T ). (18) 943
Proof. By plugging in α = 0, Contextual Rényi 944 entropy recovers to be the entropy that readers 945 concentrate on the count of potential words with 946 nonzero probabilities, which is defined in Eq. (5) 947 in Pimentel et al. (2023). As T → ∞, temperature- 948 scaled surprisal converges to the surprisal induced 949 by random guessing. Given the assumtion that 950 p(w|w<t) > 0 for each word w ∈ W , LHS be- 951 comes: 952
LHS = −log2(1/K), (19) 953
where K is the number of classes. As T → ∞, 954 RHS becomes: 955
RHS = − lim T→∞
log2 ezwt/T∑
w∈W e zw/T
(20) 956
= −log2(1/K) (21) 957
Theorem 3 For K ≥ 2, the expectation of the 958 L1 norm between Rényi entropy with α = 1 and 959 temperature-scaled surprisal with T = 1 has an 960 upper bound. 961
E[|sT |T=1 −Hα|α=1|] < √ 1
4 log2(K − 1) + 1
(22)
962
12
Proof. With Jensen’s inequality, we have:963
E[|sT |T=1 −Hα|α=1|] (23)964 ≤ √ E[(sT |T=1 −Hα|α=1)2] (24)965
= √ E[(− log2 pwt − ∑
w∈W p(w)(− log2 p(w)))2]
(25)
966
= √ Var[sT |T=1] (26)967
<
√ 1
4 log2(K − 1) + 1, (27)968
where Var[·] denotes the variance. The last inequal-969 ity is shown by Lemma 4, completing the proof of970 this theorem.971
Lemma 4 (Maximum variance of the surprisal).972 (See Theorem 8 and Lemma 15 in (Reeb and Wolf,973 2015)). Let ρ = diag(p1, p2, ..., pd) be a state on a974 d-dimensional system. Let − log pi be the surprisal975 of the output i in this system. Define Nd to be:976
Nd := 1
4 log2(d− 1) + 1. (28)977
For d ≥ 2, the variance of surprisal has a tight978 upper bound:979
varρ(− log ρ) < Nd (29)980
Theorem 2 claims the equivalence of temperature-981 scaled surprisal sT (wt, T ) and Rényi entropy Hα982 when T → ∞ and α = 0. Theorem 3, on the983 other side, gives an upper bound when T = 1984 and α = 1. Intuitively, when T ∈ (1,∞), sT985 can be considered as a softened version of sT |T=1.986 Similarly, when α ∈ (0, 1), Hα can be consid-987 ered as a softened version of Hα|α=1. Math-988 ematically, Theorem 1 provides the monotonic-989 ity of both functions within their respective do-990 mains. Hypothetically, given the above conditions,991 when tuning both functions with the aim of a bet-992 ter fit to RTs, sT |T=T ∗ and Hα|α=1/2 might be993 close. Empirically, Fig. 5 illustrates the relationship994 between averaged Rényi entropy Hα|α={0,1/2,1}995 and sT |T={1,T ∗,∞} on probabilities on three cor-996 pora. Notably, Hα|α=1/2 and sT |T=T ∗ are closely997 aligned, especially when compared with other en-998 tropy and surprisal data points. This empirical ev-999 idence partly verifies Theorem 2, Theorem 3 and1000 our hypothesis.1001
Figure 5: A comparison of averaged temperaturescaled surprisal sT |T={1,T∗,∞} and Rényi entropy Hα|α={0,1/2,1}.
B Further analysis in Section 6.2 1002
We observe that larger LMs exhibit an increased 1003 ∆MSE by utilizing temperature-scaled surprisal, 1004 as shown in the average column (Avg.) of Table 3. 1005 Specifically, on Dundee, the top 2 models achiev- 1006 ing the largest improvement through temperature 1007 scaling are GPT-2 medium and xl, while GPT-2 1008 large and xl have the most benefit on Natural Sto- 1009 ries and Brown. This result is consistent with previ- 1010 ously observed ∆llh improvement (∆llh+) across 1011 the corpus-by-LM reported in Table 1, suggest- 1012 ing a correlation between model likelihood and 1013 MSEs of the regression models. We do not ob- 1014 serve a mismatch between them, as posited by Oh 1015 and Schuler (2023b) that LME models achieve sim- 1016 ilar MSEs irrespective of obvious differences in 1017 model likelihood. 1018
Regarding the effect of the change (increase or 1019 decrease) of actual-word probability on the final fit 1020 to RTs, we first analyzed the ratio of probabilities 1021 decreasing (or increasing) for all words, as well 1022 as for subsets with specific word-level properties, 1023 choosing named entities as the representative, as 1024 shown in Table 5. We observed that probabilities 1025 of the majority of words (around 80-90%) de- 1026 crease by temperature scaling. Compared with 1027 the average across all word types (as indicated in 1028 the ’Avg.’ column), named entities exhibit a lower 1029 ratio of probability reduction. Larger LMs tend to 1030 have a higher ratio, especially the ratio for named 1031 entities, likely because smaller models may lack 1032 the specific knowledge of less common terms, such 1033 as named entities. 1034
Recalling one of the results in Section 6.2 that 1035 the main advantage of temperature-scaled surprisal 1036 arises from reduction of large probabilities on 1037 Dundee and the amplification of small probabilities 1038
13
on Natural Stories. However, for named entities,1039 the story is converse on Dundee vs. on Natural Sto-1040 ries and Brown, where for the latter two corpora,1041 the advantage is primarily due to reducing the prob-1042 abilities of highly predictable entities. We shed1043 light to the possible reason of such a discrepancy in1044 Fig. 6, which displays the top 15 frequent words for1045 GPT-2 small on three corpora. Notably, Natural1046 Stories and Brown show a marked lack of words1047 with increased probabilities (blue bins) compared1048 to Dundee. This lack weakens the overall impact1049 of rising probabilities (quantified by ∆MSE(pwt↑)).1050 Specifically, on Brown, only 4 out of 15 top fre-1051 quent words have the part of increased probabilities1052 (blue bins), correlating with the largest discrep-1053 ancy in ∆MSE between probabilities that decrease1054 (329.7) and those that increase (-170.6) in Table 4.1055
Avg. Named entities Corpus GPT2 pwt ↓ |res| ↓ pwt ↓ |res| ↓
Dundee s 88.0 51.8 78.1 52.3 m 89.6 52.5 80.1 54.1 l 90.2 52.3 80.1 53.5 xl 91.4 52.4 82.7 54.3 s 93.8 55.0 85.3 51.8 Natural m 94.7 55.2 89.1 53.2 Stories l 93.5 55.7 89.1 53.4
xl 92.1 55.5 88.2 52.8
Brown s 91.8 51.5 87.3 50.9 m 93.2 51.5 86.1 50.9 l 93.3 51.8 88.6 52.1 xl 93.5 51.7 87.8 53.3
Table 5: The ratio of probability of predicted word pwt getting smaller and the absolute value of residuals |res| getting smaller for GPT-2 models on three corpora.
C Preprocessing steps1056
On Dundee ET corpus (Kennedy et al., 2003),1057 we use the first-pass gaze duration. Following1058 prior work (Kuribayashi et al., 2022), we remove1059 words containing numbers or punctuation, words1060 that are either the first or the last one in a line,1061 as well as words whose previous words contain1062 numbers or punctuation. On Natural Stories SPR1063 corpus (Futrell et al., 2018), following Shain et al.1064 (2022), we remove words if the RT is less than1065 100ms or greater than 3,000ms, if the words are in1066 the first or last position of each story, if participants1067 answered less than 5 out of 8 comprehension ques-1068 tions correctly, if words contain numbers or punc-1069
tuation, and if words whose previous words con- 1070 taining numbers or punctuation. On Brown SPR 1071 corpus (Smith and Levy, 2013), following Shain 1072 et al. (2022), we remove words if the RT is less 1073 than 100ms or greater than 3,000ms and if words 1074 contain numbers or punctuation. 1075
D Further descriptions on metrics and 1076 evaluation 1077
We evaluate calibration error (% ECE and % 1078 CECE) in both equally-spaced and log binning 1079 schemes. In equally-spaced binning scheme, 1080 the samples are grouped into M ∈ N equally- 1081 spaced interval bins based on their confidences 1082 p̂i. Conversely, the log binning scheme operates 1083 under an empirical upper limit for − log2 p̂i, de- 1084 noted as max(− log2 p̂). Table 6 shows ranges 1085 of p̂ and − log2 p̂ for GPT2s on three corpora. 1086 For this scheme, we establish M ∈ N log- 1087 equally-spaced interval bins within the range of 1088 (0, max(− log2 p̂)]. 1089
We investigate scaling T ∈ [1, 10], considering 1090 both densely and sparsely distributed points. The 1091 values examined are detailed as follows: [1.0, 1.1, 1092 ..., 1.9] for dense intervals, [2.0, 2.25, ..., 3.25] for 1093 moderately spaced intervals, and [3.5, 4.0, ..., 10.0] 1094 for sparse intervals. 1095
Following Kuribayashi et al. (2022), reading 1096 times of a base model are modelled by the fol- 1097 lowing formula: 1098
rt ∼ freq ∗ length + freq_prev_1 ∗ length_prev_1 + (1|article) + (1|subj_id)
(30)
1099
A target model additionally includes surprisal esti- 1100 mates of current words and previous words: 1101
rt ∼ surprisal + surprisal_prev_1 + surprisal_prev_2 + freq ∗ length + freq_prev_1 ∗ length_prev_1 + (1|article) + (1|subj_id).
(31)
1102
On Dundee corpus, both models also include fea- 1103 tures of [screenN, lineN, segmentN]. We also per- 1104 form experiments with both models without inter- 1105 actions among predictors in Appendix I. 1106
E Exploring further effectiveness of 1107 temperature-scaled surprisal over basic 1108 predictors 1109
In this section, we explore the question of whether 1110 the benefit of temperature-scaled surprisal holds 1111
14
15
only for regression models already containing other1112 predictors such as length and frequency. We con-1113 duct experiments similar to those detailed in Sec-1114 tion 5.2 while setting base predictor variables vbase1115 to 0 and target predictor variables vtgt to only1116 temperature-scaled surprisal sT (wt, T ) in Eq. 7.1117 Fig. 7 shows that excluding base predictors de-1118 crease but not totally impact the effectiveness of1119 temperature-scaled surprisal.1120
F Calibration error for single-token and1121 multiple-token words1122
In Table 7, we demonstrate the calibration error1123 (%ECE) for single-token and multiple-token words1124 for GPT-2 small. Calibration evaluation is con-1125 ducted at the token level as before. Results indicate1126 that multiple-token words show larger calibra-1127 tion errors than single-token words.1128
G Probability distribution before and1129 after temperature scaling1130
Fig. 8 shows actual-word probability distribution1131 before and after temperature scaling for GPT-21132 small on three corpora. Multiple-token words1133 tend to have smaller probabilities than single-1134 token words, both before and after temperature1135 scaling.1136
H Significant test of temperature-scaled1137 surprisal1138
We report the statistical significance based on se-1139 lecting the most representative model, GPT2s, on1140 three corpora in Table 8. Models with temperature-1141 scaled surprisal lead to statistically significant posi-1142 tive ∆llh (p < 0.001).1143
I Analysis on correlations among 1144 predictors 1145
We investigate the question of whether the benefit 1146 of temperature-scaled surprisal is primarily due to 1147 the interactions and correlations among predictors. 1148 We first run experiments with the original target 1149 LME model as in Eq. 31 (denoted as model 1), a 1150 model that has no interactions between frequency 1151 and length as in Eq. 32 (denoted as model 2) and a 1152 third model that has no interactions and addition- 1153 ally includes random slopes for subject as in Eq. 33 1154 (denoted as model 3). 1155
rt ∼ surprisal + surprisal_prev_1 + surprisal_prev_2 + freq + length + freq_prev_1 + length_prev_1
+ (1|article) + (1|subj_id). (32)
1156
rt ∼ surprisal + surprisal_prev_1 + surprisal_prev_2 + freq + length + freq_prev_1 + length_prev_1
+ (1|article) + (surprisal|subj_id). (33)
1157
The results are in Table 9. Removing the in- 1158 teractions among predictors or additionally in- 1159 cluding random slopes does not influence the 1160 effectiveness of temperature-scaled surprisal. 1161
Furthermore, we also investigated the correla- 1162 tions among predictors by examining the correla- 1163 tion matrix for GPT2 small on three corpora (model 1164 1). Table 9, 10 and 11 indicate that temperature- 1165 scaled surprisal does not exhibit a stronger cor- 1166 relation with the other predictors in comparison 1167 to the original surprisal, as shown in the surprisal 1168 column (’surp’), which excludes the concern that 1169 the primary benefits are simply due to correlations 1170 between the baseline predictor and temperature- 1171 scaled surprisal. 1172
16
ECEsingle ECEmultiple
17
18
19
J Influence of multiple-token words1173 vs. model size1174
Table 10 shows the increase of ∆llh of temperature-1175 scaled surprisal by only taking into the analysis1176 the subset of multiple-token words. The benefit of1177 temperature-scaled surprisal being primarily from1178 the scaling of multiple-token words still holds for1179 larger LLMs. For larger LLMs, the influence of1180 multiple-token words is also larger.1181
K Influence of word-level attributes 1182 vs. influence of multiple-token words 1183
We explore which of these two factors has a 1184 stronger effect on the benefit of temperature-scaled 1185 surprisal, word-level attributes in Section 6.2 or 1186 multiple-token words in Section 6.3. For word 1187 types, we select named entities as the representa- 1188 tive attribute since they perform to be the most 1189 beneficial ones as discussed in Section 6.2. For 1190 multiple-token words, we select all multiple-token 1191 words with more-than-one tokens. In order to fairly 1192 compare the influence, we normalize ∆MSE of 1193 each category under the linguistic factor F with 1194 the ratio of that category words among the total 1195 words: ∆̄MSE(F ) = ∆MSE(F ) ·ratio(F ). Table 11 1196 shows that multiple-token words drive the much 1197 stronger averaged benefit of temperature-scaled 1198 surprisal, compared with the averaged benefit of 1199 named entities. 1200
L Other results in Section 6 1201
20
21
22

heading: Future Work
text: 606 Future work could investigate this hypothesis by 607 analysing RTs on subwords instead of aggregated 608 words (with the caveat that subword tokens may 609 not be cognitively plausible units)