The Effect of Surprisal on Reading Times in Information Seeking and Repeated Reading (2024)

Abstract: The effect of surprisal on processing difficulty has been a central topic of investigation in psycholinguistics. Here, we use eyetracking data to examine three language processing regimes that are common in daily life but have not been addressed with respect to this question: information seeking, repeated processing, and the combination of the two. Using standard regime-agnostic surprisal estimates we find that the prediction of surprisal theory regarding the presence of a linear effect of surprisal on processing times, extends to these regimes. However, when using surprisal estimates from regime-specific contexts that match the contexts and tasks given to humans, we find that in information seeking, such estimates do not improve the predictive power of processing times compared to standard surprisals. Further, regime-specific contexts yield near zero surprisal estimates with no predictive power for processing times in repeated reading. These findings point to misalignments of task and memory representations between humans and current language models, and question the extent to which such models can be used for estimating cognitively relevant quantities. We further discuss theoretical challenges posed by these results.

## introduction a key question in psycholinguistics concerns the cognitive processes that underlie the real-time integration of new linguistic material with previously processed linguistic context. a central framework for examining this question is surprisal theory @xcite . this theory ties word processing cost to the word's surprisal, and predicts a linear relation between surprisal and processing difficulty. due to its theoretical implications (see @xcite for an extended discussion), multiple studies have tested this prediction empirically with different behavioral methodologies (e.g. eyetracking and self paced reading), corpora (among others, dundee @xcite , natural stories @xcite , @xcite and celer @xcite ), language models, and languages @xcite @xcite @xcite @xcite @xcite . all these studies found significant surprisal effects on processing times. with the exception of @xcite and @xcite who obtained evidence for superlinear effects, these studies found a linear relation between surprisal and processing times. however, thus far this relation has been examined only in one reading regime, which can be referred to as ordinary reading. this regime presupposes that the comprehender did not have prior, or at least recent, exposure to the linguistic material. it further assumes that they have no specific goals beyond general comprehension of this material. these assumptions do not hold in many daily situations, where language comprehenders often have specific goals with respect to the linguistic input, process the same input multiple times, or both. this limits the generality of the conclusions that can be drawn from prior studies. in this work, we examine the effect of surprisal on reading times in english l1 in three common, but understudied language processing regimes: (1) information seeking, (2) repeated processing, and (3) the combination of the two. prior work on information seeking @xcite and repeated reading @xcite @xcite has shown substantial differences in eye movement patterns in these regimes compared to ordinary reading, and the extent to which the predictions of surprisal theory hold in these regimes is currently unknown. we analyze and compare the functional form and predictive power of two types of contexts, standard regime-agnostic contexts that capture the general predictability of a word, and regime-specific contexts which include the task in information seeking and a prior appearance of the linguistic content in repeated reading. we examine two main hypotheses stemming from surprisal theory. (1) the presence and functional form of surprisal effects for standard surprisal estimates should extend nonordinary reading regimes. (2) surprisal estimates from regime-specific contexts should yield higher predictive power for processing times in the respective regimes compared to regime-agnostic contexts, due to a more accurate representation of the context and the processing goals, which should lead to better alignment with subjective word probabilities. our main results are the following: ## related work the first studies to empirically examine the relation between surprisal and reading times were @xcite . they used broad coverage eyetracking and self-paced reading data for english, and found evidence for a linear relation. following this work, several studies obtained similar results using additional corpora, languages and different methodologies for curve fitting and testing linearity, including goodkind and bicknell (2018), wilcox et al. (2020), shain et al. (2024b) and wilcox et al. (2023). hoover et al. (2023) and xu et al. (2023) obtained evidence for superlinearity. brothers and kuperberg (2021) found a linear relation in word probability using a controlled self-paced reading experiment and cloze estimates of word probabilities. re-analysis of this data with language model probabilities resulted in a linear relation in surprisal @xcite . our study continues this line of work and extends it to different reading regimes. both information seeking and repeated reading have received limited attention in psycholinguistics. work that examined information seeking @xcite found substantial differences in eye movement patterns compared to ordinary reading. the differences were shown to be driven by the division to taskrelevant and task-irrelevant information. different eye movement behavior was also found in repeated reading, where among others, shorter reading times and longer saccades were observed @xcite . while the presence and magnitude of surprisal effects in information seeking and repeated reading was previously established @xcite , their functional form and predictive power are yet to be determined. multiple studies have pointed out divergences between surprisal estimates and human next word expectations @xcite @xcite , as well as an inverse relationship between the quality of recent language models (as measured by perplexity) and their fit to reading times @xcite . closest to our work is @xcite , who found that in a repeated reading cloze task, language models have substantially higher next word prediction accuracy compared to humans. they further identified "induction heads", which are attention heads that recognize repeated token sequences and increase the probability of the previously observed continuation @xcite , as a core contributor to this behavior in language models. our findings for repeated reading are in line with these results. ## data we use onestop, an extended version of the dataset by @xcite , with eye movements from 360 english l1 readers, recorded with an eyelink 1000+ eyetracker (sr research). the ex-periment was conducted under an institutional irb protocol, and all the participants provided written consent before participating in the study. the textual materials are taken from onestopqa @xcite and comprise 30 articles from the guardian with 4-7 paragraphs (162 paragraphs in total). each paragraph in onestopqa is accompanied by three reading comprehension questions. the textual span in the paragraph which contains the essential information for answering the question correctly, called the critical span, is manually annotated in each paragraph for each question. an experimental trial consists of reading a single paragraph on a page, followed by answering one reading comprehension question on a new page without the ability to go back to the paragraph. ordinary reading vs information seeking 180 participants are in an ordinary reading regime in which they see the question only after having read the paragraph. the remaining 180 participants are in an information-seeking regime in which the question (but not the answers) is presented prior to reading the paragraph. first vs repeated reading each participant reads 10 articles in a random presentation order, followed by two articles that are presented for a second time with identical text but with a different question for each paragraph. the article in position 11 is a repeated presentation of the article in position 10. the article in position 12 is a repeated presentation of one of the articles in positions 1-9. thus, onestop contains both consecutive and nonconsecutive repeated reading at the article level. 2 onestop has 2,532,799 data points (i.e. word tokens over which eyetracking data was collected). we exclude words that were not fixated, words with a total reading time greater than 3,000 ms, words that start or end a paragraph, words with punctuation, and surprisal values greater than 20 bits. after these filtering steps, we remain with 1,157,609 data points: 541,875 in ordinary reading, 474,674 in first reading information seeking, 82,357 in repeated ordinary reading, and 58,703 in repeated reading information seeking. ## methodology we examine four different reading regimes that take advantage of the experimental manipulations in onestop and reflect different types of interac-tions with the text. the first is ordinary reading during the first presentation of the text. this regime corresponds to the standard experimental setup in reading studies. additionally, new to this work, we examine information seeking during first reading, and both ordinary reading and information seeking during repeated text presentation. we estimate the functional form of the relation between surprisal and reading times using generalized additive models (gams, @xcite , which can fit non-linear relations between predictors and responses. we predict word reading times from surprisal and two control variables that were shown to be predictive of reading times above and beyond surprisal: word frequency and word length @xcite . to account for spillover effects @xcite , our models also include the surprisal, frequency and length of the previous word. following prior work (e.g. @xcite ) our primary reading time measure is first pass gaze duration; the time from first entering a word to first leaving it during first pass reading. this measure is associated with the processing difficulty of a word given left-only context and is thus especially suitable for benchmarking against surprisal. in the appendix, we examine additional measures: gaze duration and total fixation duration. for completeness, we also provide results for first pass first fixation duration and first fixation duration, which tend to have small surprisal effects and are associated with lexical processing @xcite . definitions of all the measures are in section a in the appendix. surprisal, defined as -log p(w i |@xmath0, where w i is the current word and @xmath1, is estimated using a language model (see section 4.3). the language models we use provide a distribution over sub-words (tokens). we therefore sum the sub-word probabilities to obtain the word's probability. frequency is defined as -log p(w i ), using word counts from wordfreq @xcite . word length is measured in number of characters. we define three models of interest: foot_3 ## surprisal from standard context in our first set of analyses, we follow prior work on ordinary first reading, as well as information seeking and repeated reading @xcite , and use standard, reading regime-agnostic surprisal estimates, which are obtained by conditioning the model on the prior textual material in the paragraph. ## surprisal from regime-specific context thus far, we used surprisal estimates based on the textual context in the paragraph. however, this context does not fully capture the reading task conditioning in the human data. human participants in the first reading -information seeking regime receive a question prior to reading the paragraph. in repeated ordinary reading they have already read that paragraph. in repeated reading during information seeking they have previously read the paragraph and received a question prior to both the first and the second reading of the paragraph. these manipulations can alter linguistic expectations and were previously shown to influence reading times @xcite @xcite . furthermore, human participants receive explicit instructions regarding the different trial components in the reading experiment. in the remainder of this work, we compare our results using standard surprisal estimates to surprisal estimates based on context types that more closely match the textual contexts and instructions presented to humans in each of the reading regimes. our analyses focus on the following questions regarding the three regimes that are not ordinary first reading. (1) do the linear surprisal effects persist under regime-conditioned surprisal estimates? (2) regime standard context regime context description prompting + regime context prompt text first reading ordinary reading p p the preceding words in the paragraph. prompt1 + p prompt1: "you will now read a paragraph." first reading information seeking p q + p the question followed by the preceding words in the paragraph. prompt1 + q + p prompt1: "you will now be given a question about a paragraph followed by the paragraph. you will need to answer the question." repeated reading ordinary reading p p + p the entire paragraph followed by the preceding words in the same paragraph. prompt1 + p + prompt2 + p prompt1: "you will now read a paragraph." prompt2: "you will now read the same paragraph again." repeated reading information seeking p q' + p + q + p the question for the first reading, followed by the paragraph, the question for the second reading and the preceding words in the same paragraph. prompt1 + q' + p + prompt2 + q + p prompt1: "you will now be given a question about a paragraph followed by the paragraph. you will need to answer the question." prompt2: "you will now read the same paragraph again with a different question before the paragraph. you will need to answer the question." to address these questions, in addition to the standard context used in section 5, we examine three regime-specific contexts that correspond to each of the three reading regimes that involve information seeking and repeated reading. to further enhance the similarity to the experimental setup in the human data, we also examine a variant of the regime contexts in which the model additionally receives prompts that emulate the reading instructions received by human participants. the prompts convey the same content provided in the instructions to human participants in the eyetrack-ing experiment, but are not a verbatim copy, as the original instructions further contain details relevant only for the eyetracking experiment, such as the text triggering targets and button presses associated with each part of the trial. the regime-specific contexts and prompts are presented in we note that although these contexts include the essential components of each reading regime, they do not fully match the eyetracking experiment as they do not include intervening textual material between first and second presentations of a paragraph. this is because the context window of our models is too small to include the text of a full experimental session. to partially address this limitation, in table a2 in the appendix we present a prompting scheme for article-level analysis for articles 10 and 11. we use this scheme with the pythia-70m model, for which we employ a sliding window mechanism with an overlap size that ensures that each paragraph's first appearance is fully included in the context window of its repeated appearance. ## discussion and conclusion surprisal theory predicts a linear relationship between surprisal and word processing times. this prediction found support in studies with ordinary reading, but was not previously examined in information seeking and repeated reading. we find evidence that with standard surprisal estimates, the prediction of surprisal theory for a linear effect of surprisal on reading times holds in these regimes. we further find that the effect size and predictive power of standard surprisal estimates diminish in information seeking and repeated reading. our attempt to improve language model predictive power with regime-specific contexts yields two primary findings. first, we observe that regimespecific surprisal estimates in first reading -information seeking do not improve the fit to human reading times. a more severe case of estimation collapse is observed in repeated reading, where we find near zero surprisal estimates with no predictive power for reading times, likely due to in-context memorization. these findings highlight two different types of misalignment between language models and humans. information seeking demonstrates a misalignment in the representation of task information. repeated reading suggests very different memory and retrieval abilities in humans and current language models. these misalignments question not only the suitability of current language models as cognitive models of human language processing, but also the psycholinguistic relevance of quantities extracted from such models. we entertain two possible explanations for the discrepancies in the real-time processing and memory mechanisms of humans and language models. the first explanation is that this mismatch stems from architectural and/or training aspects of current language models. if this is indeed the case, they can be potentially alleviated or even completely resolved with architectural or training procedure changes to said models; it is well possible that future architectures will better capture task relevant information, or handle repeated text in ways that are more commensurate with human processing. the second explanation poses a challenge to language processing theory, and in particular to the view of surprisal as a "causal bottleneck" for observed behavior @xcite . according to this view, whatever the underlying linguistic processing mechanisms and representations may be, their effect on processing times is mediated through surprisal. although better representation of the context should yield better estimates of subjective surprisals and thus better reflect processing times, we do not observe this in practice. one could alternatively argue that factors that come into play in non-ordinary processing regimes and affect reading times either cannot or should not be encoded in surprisals. surprisal theory accounts only for processing difficulty, while reading times may reflect additional factors of cognitive state, which do not directly speak to processing difficulty (e.g. one may skim through portions of the text because they are less relevant for the comprehender goals, not because they are easier to process). future empirical and theoretical work is required to make further progress on these questions.