Beyond the Average Reader: the Reader Embedding Approach (2025)

Abstract: Focus of this work is the prediction of reading times as the task is customarily dealt with in literature: that is, by collecting eye-tracking data that are averaged and employed to train learning models. We start by observing that systems trained on average values are ill-suited for the prediction of the reading times for specific subjects, as they fail to account for individual variability and accurately analyze the reading gestures of specific reader groups, or to target specific user needs. To overcome such limitation, that is to predict the reading times for a specific subject, we propose a novel approach based on creating an embedding to compactly describe her/his fixations. Embeddings are used to individuate readers that share same or similar reading behavior from a reference corpus. Models are then trained on values averaged over this subset of similar readers. Experimental results indicate that the proposed approach consistently outperforms its corresponding variants, in which predictions of reading times for specific readers are based on data from all subjects rather than from the most similar ones.

## introduction eye-tracking reading data are acknowledged as an important resource for both natural language processing and psycholinguistics. the ability to model gaze features is crucial to advance our understanding of language processing @xcite . eye-tracking data typically collect information on where readers look at in the form of timestamped fixations. these offer a glimpse on the path of attention deployed by readers providing valuable insights into what is the focus of the attention of the reader through time. eye-tracking data mostly include measures to record the duration of the first fixation(s) on words, the number of fixations on a given word, and the overall time spent, going back and forth through the reading process, on a specific word. such measures have been fruitfully employed to investigate linguistic processing mechanisms, such as, e.g., frequency and predictability effects on reading. based on the analysis of reading times (rts), we know that it typically takes longer to read infrequent words than frequent ones @xcite , and also context effects have been proven: in left-to-right languages such as english, words that can hardly be predicted based on their left context will also require longer rts than those that are predictable @xcite . analyzing how text documents are read is relevant from a scientific viewpoint, and also has many applications. precious insights can be gained about aspects as diverse as lexical access, semantic integration, individual differences including disorders and deficits @xcite , and investigating the reading task may shed light on overall words predictability @xcite , on the incrementality hypothesis @xcite , on the assessment of competing theories, such as dependency locality theory vs. surprisal @xcite , on the interplay between reader's expertise and the features of the text documents @xcite , and many other intriguing areas at the intersection of computational linguistics and cognitive science. additionally, many sorts of applications may be drawn that benefit from this kind of research. being able to predict rts may impact on applications such as the assessment of linguistic complexity and text readability @xcite @xcite , the design of text simplification strategies @xcite and readers profiling @xcite , the organization of personalized reading experiences, learning assessments, e.g., in l2 acquisition @xcite , in problem solving skills acquisition @xcite . eye-tracking data can be complemented with the many analytical tools developed for language analysis to predict reading times and readers' behavior. different approaches have been proposed, that are based on either boosting methods relying on tree-based algorithms, or on the use of neural approaches casting rts prediction to a regression task, solved through the fine-tuning of transformerbased language models @xcite . other approaches, mainly inspired by probabilistic accounts of language processing, provided evidence that reading times are correlated to the conditional word probabilities estimated by language models @xcite . in this view, unpredictability would act as as a major source of processing difficulty in language comprehension: increasing effort would produce a slower reader response (corresponding, e.g., to higher gaze duration) as a consequence of the increased distance intervening between the cognitive resource allocation and the actually encountered input @xcite . however, readers may be differently affected by such sources of difficulty, and taking into account such factors is crucial to predict the rts of specific readers, which would be beneficial to analyze the behavior of specific reader groups or to target specific user needs. one major limitation of the current approaches is that, to the best of our knowledge, all reported work relies on average measures, that can be extracted from reference corpora collecting data on readers fixations. for example, if the total reading time -that is, the overall duration of eye fixations received by each word-was considered, existing resources typically provide an average value across all readers. in point of fact, models that are trained on reading times obtained by averaging values over the subjects are not able to accurately account for individual differences in reading behavior. provided that this approach is in general justified by the inter-subject consistency of the recorded data, it may overlook the fact that different readers adopt different reading strategies, and that the same text can be read in different ways by different readers. if such models are employed to predict rts, while they will exhibit a good fit with average rts, they will mostly obtain lower accuracy in predicting the rts featuring specific readers. the present study addresses this issue, and is concerned with improving rts predictions for specific readers. the following novelties are introduced: i) a vectorial representation for describing different behavioral (reading) gestures and aptitudes; ii) the idea of predicting a specific subject reading times based on the rts of similar readers; additionally, we explore the output of models based on widely varied assumptions and features, and prove that few readers' data are actually needed to build a vectorial user profile ensuring consistent improvements in the prediction of reading times. in all cases, we compare and contrast experimental results obtained by testing on averaged reading time values (as done in literature) vs. the actual values characterizing the reading performance of specific subjects. we created an array of experiments to test different models and architectures: the proposed approach always outperforms average-based variants in predicting reading times for specific readers. also remarkably, this approach only requires a fraction of data to overcome the traditional averagingbased variants, thereby allowing for an efficient, resource-saving, and effective way to predict reading times. ## related work reading involves two main eye movements, fixations and saccades. fixations are brief stops (ranging from 50 to 1500 ms) that typically occur at each word; sometimes even more stops are needed, also depending on words length and on lexical and syntactic complexity. saccades are fast (from 10 to 100 ms) movements between each two fixations, and are used to reposition the point of focus. in general, individual words are fixated differently: e.g., a pioneering work by @xcite reports that 85% content words and 35% function words get fixations. among the main variables that impact on eye movements, one must additionally consider i) words length: shorter (2-3 letter) words are skipped 75% of the time, while longer (8 letters or more) words are fixated almost always @xcite ; and ii) syntactic and conceptual difficulty of the considered text @xcite . several measures have been proposed to analyze text reading and processing times. while the total reading time (trt) is supposed to grasp the time taken by the overall semantic integration @xcite , two partial and finer-grained measures have been also proposed: the duration of the first fixation (ffd) that allows estimating the cost underlying lexical access @xcite , and the number of fixations (nf), which is deemed to report about words integration in the context of what has been read so far @xcite . most work focused on the processes underlying lexical access and semantic integration falls into two broad approaches to model context. in the first case we have models concerned with the semantic relatedness between words and their context: in this setting, reading times are predicted based on the similarity between embeddings describing words and their context. works adopting the second approach mostly rely on a probabilistic framework whereby words are predicted based on their (left) context. in this view, word predictability should be intended as a function of the probability of a word given the context, and the probability of that word may work, in turn, as a main predictor of reading times: in essence, the less likely the emission of a word, the higher the surprisal associated to that word, and the longer the time it requires for readers to process it. such probabilistic device has also proven useful in distinguishing the linguistic productions of individuals with dementia from those produced by healthy controls @xcite @xcite . in the last few years neural language models gained a central role in analyzing reading as well, since they are able to acquire conditional probability distributions over the lexicon that are also predictive of human processing times. while word length and frequency are widely acknowledged as predictors for determining lexical access, different sorts of language models have been recently compared to analyze and explain syntactic and semantic factors @xcite : n-gram models have been found to succeed in capturing shortrange lexical access, while models based on recurrent neural networks show better fit in predicting the next-word. other studies found that surprisal scores are strong predictors of reading times and eye fixations obtained through eye-tracking @xcite , along with a substantial linear relationship between models' next-word prediction accuracy and their ability in predicting reading times @xcite @xcite . a different line of research hypothesizes that the reading process may also be anticipatory @xcite : in this view, readers predict upcoming words and, based on their expectations, allocate time to process them. this anticipatory predictability effect is quantified as words' contextual entropy @xcite , and has been found to be predictive of reading times as well @xcite . the interaction between gaze and subjective hatefulness rating has been studied by @xcite , who observed that the annotator's gaze provides predictors of their subjective hatefulness rating. specifically, the trt and the nf correlate with annotators' subjective hate ratings and improve predictions of text-only hate speech models. the study by @xcite evaluates the predictive power of surprisal and entropy at the individual level by incorporating information on individuals' cognitive capacities and allowing them to modulate the magnitude of surprisal and entropy effects. the findings indicate that including cognitive capacities increases the predictive power of surprisal and entropy on reading times, and that high performance in the psychometric tests is associated with lower sensitivity to predictability effects. ## readers profiling eye-tracking features capture different aspects of the reading process, such as fixation duration and saccade length @xcite . in the conventional approach, multiple participants read the same texts, and the values of each feature are averaged across readers @xcite producing a representation that approximates the behavior of an average reader. for example, in the provo corpus (luke and christianson, 2018), the mean values (complemented by standard deviations) are: 198.14 (107.13) ms for trt, 0.95 (0.47) for nf, and 139.80 (52.00) ms for ffd. averaged features are assumed to provide a generalizable representation of reading behavior and are utilized for both training and evaluating models that predict reading times. however, no studies have assessed the performance of these models in predicting the behavior of specific readers. in section 4.4 an evaluation on this task is presented, where performances of the common approach based on averaging rts is compared to the results of three reader profiling methods: in this newly proposed approach, the values used to train the model are obtained using only readers who are similar to the target reader, rather than all readers data. this procedure focuses on an individually carved representation, as the reading times predictions for each reader are based on a subset of readers exhibiting similar behavior. moreover, this approach is applicable in real-world scenarios where the profiling (along with the selection of the most similar readers) of a specific reader is computed on the basis of a limited set of texts. a reader embedding is created using the first fixation duration (ffd) and the number of fixations (nf) for all tokens in the selected texts. 2 the procedure employed to compute the new readers embeddings involves sampling their fixations while reading a text, which was also read by other subjects in a reference corpus. given a text composed of n lexical items, for each item therein both ffd and nf data are collected, which results in a reader embedding with a size of 2n elements. cosine similarity is employed as the distance function to compare our embeddings: in particular, a cosine similarity matrix is constructed (figure 1 ), where each cell (i,j) represents the cosine similarity score between the embeddings of reader i and reader j. different selection strategies were considered to individuate the closest readers from the dataset: top k similar readers, above threshold, and ffd-nf plot. top k similar readers the top k most similar readers of a reader i are determined from the cosine similarity matrix by selecting the k highest values in row i, excluding the cell (i,i), which corresponds to the reader itself. above threshold a given reader i is associated only with the readers j whose cosine similarity value (cell (i,j)) in the cosine similarity matrix is above a predetermined threshold, thereby picking a variable number of readers ensuring at least some degree of similarity. ffd -nf plot for each reader, the average ffd and nf on the selected texts are computed, and the results are plotted as points in a ffd-nf plot. four classes are defined based on mean ffd and nf values, with readers within the same class considered as similar readers. in this setting, referring to figure 2 , we identify four classes: class 1 includes readers featured by ffd above average and nf below average; class 2 composed by readers that exhibit both ffd and nf above average; class 3 with subjects whose ffd and nf are both below average; and class 4, with subjects whose ffd is below average and nf above average. ## experiments our hypothesis is that the prediction of individual readers reading times may be improved by selecting few readers deemed as similar based on their ffds and nfs. to test such hypothesis we devised an array of experiments in which we compared different approaches to select the most similar readers. additionally, we tested different models relying on diverse learning and representational rationales. all results reported hereafter are referred to the prediction of the ffd measure. ## conclusions we started by identifying a key issue: the rts predictions delivered through systems trained on average values (as is customarily done in literature) are not suited for the prediction of specific subjects. models trained on average values will mostly learn to predict average values, disregarding (since they were not exposed to them) elements related to individual variability. being able to predict how a specific user behaves while reading a text excerpt may be beneficial for many different purposes, such as to improve text accessibility (e.g., for specific groups, either based on age or suffering from the same disorder or disease), to stimulate learning and engagement with text, and more in general to improve the overall reading experience in a personalized way. we introduced a novel approach to improve rt predictions on specific readers. first, it builds on a vectorial representation whereby subjects are represented through a collection of fixation data (ffd and nf information is embedded to build such vectors) reporting about how they read texts: this allows creating a shared representational space where readers can be placed, directly compared, and selected based on similarity accounts. 6 in the same spirit as for word embeddings, such vectors can be thought of as points over a multidimensional euclidean space, where distance acts like a proxy for similarity. secondly, few similar readers are identified, and for each such subject a small fraction of all available data (10 texts, along with their rts) are then employed to select the most similar readers to a specific reader. once these profiles have been created, and the reader under consideration has been equipped with a set of readers with analogous reading behavior, various models have been trained that ensure up to 10% improvement (averaged over three measures, accl, acct, accs) in the accuracy of rts predictions. we reported the results obtained in four experiments testing different models and architectures: the proposed approach always outperforms averagebased variants in predicting reading times for specific readers, providing evidence that the approach itself is beneficial, regardless of the specific learning algorithm used. also remarkably, this approach only requires a limited quantity of data to overcome the traditional averaging-based variants, thereby allowing for an efficient, resource-saving, and effective way to predict reading times.