How Well Can AI Models Generate Human Eye Movements During Reading? (2025)

Abstract: Eye movement analysis has become an essential tool for studying cognitive processes in reading, serving both psycholinguistic research and natural language processing applications aimed at enhancing language model performance. However, the scarcity of eye-tracking data and its limited generalizability constrain data-driven approaches. Synthetic scanpath generation offers a potential solution to these limitations. While recent advances in scanpath generation show promise, current literature lacks systematic evaluation frameworks that comprehensively assess models’ ability to reproduce natural reading gaze patterns. Existing studies often focus on isolated metrics rather than holistic evaluation of cognitive plausibility. This study presents a systematic evaluation of contemporary scanpath generation models, assessing their capacity to replicate natural reading behavior through comprehensive scanpath analysis. We demonstrate that while synthetic scanpath models successfully reproduce basic gaze patterns, significant limitations persist in capturing part-of-speech dependent gaze features and reading behaviors. Our cross-dataset comparison reveals performance degradation in three key areas: generalization across text genres, processing of long sentences, and reproduction of psycholinguistic effects. These findings underscore the need for more robust evaluation protocols and model architectures that better account for psycholinguistic complexity. Through detailed analysis of fixation sequences, durations, and reading patterns, we identify concrete pathways for developing more cognitively plausible scanpath generation models.

## introduction eye movements during reading reflect readers' attention @xcite , processing difficulty, and information integration @xcite . thus, eye-tracking data provides a rich source of insights into human language processing. models derived from gaze data not only shed light on attention and comprehension but also have practical applications in readability estimation @xcite , educational technology @xcite , and cognitively plausible nlp @xcite . however, the utility of such data is constrained by its limited availability. synthetic data generation has emerged as a critical solution across domains, particularly for enhancing deep learning models in data-scarce scenarios. recently, eye-tracking models for reading have gained traction in machine learning research. studies suggest that cognitive models like e-z reader @xcite , which simulate gaze patterns during reading, can improve language models in standard nlp tasks @xcite . modern approaches follow two key paradigms: predicting aggregated eye-tracking features (e.g., fixation durations) @xcite @xcite ; generating scanpaths-temporal sequences of word fixations with durations @xcite @xcite . for instance, @xcite used a gaze feature prediction model @xcite to train a reward model by concatenating predicted eye-tracking features with contextual embeddings. evaluations on the oasst1 and helpsteer2 datasets showed significant accuracy improvements over baselines. by generating scanpaths, these models can additionally compute reading-related gaze features, thereby increasing their utility. scanpaths enable modeling of gaze phenomena such as refixations (repeated word fixations) and regressive saccades (backward eye movements). the latter has drawn increasing attention, as it not only enhances the performance of established models like e-z reader @xcite and swift @xcite but also shows promise for downstream nlp applica-tions. despite progress, existing studies lack a comprehensive analysis of generated scanpaths and standardized evaluation metrics. for example @xcite proposed eyettention, evaluated using normalized levenshtein distance (nld) @xcite . however, nld ignores fixation durations, lacks spatial sensitivity, and has limited interpretability. eyettention has been applied to improve nlp task performance on the glue benchmark @xcite by reordering text to mimic natural reading patterns @xcite @xcite . @xcite introduced scantextgan, employing both nld and multimatch @xcite . yet, kümmerer and bethge (2021) demonstrated that multi-match can favor incorrect models over ground truth. scantextgan's integration of predicted scanpaths (via lstm and multi-head attention) improved performance on glue, sentiment analysis, and sarcasm detection @xcite . @xcite developed scandl later extended to scandl 2.0 @xcite , using two separate models for fixation sequences and durations. they use scasim (von der malsburg and vasishth, 2011), a metric penalizing spatial/temporal deviations between fixations. while scasim addresses nld's limitations, their reproducibility analysis excluded fixation durations, and no comparison was made against randomly generated scanpaths for scasim or gaze features. this work synthesizes prior research on scanpath generation models and addresses their limitations. our contributions are: 1) a unified evaluation framework for scanpath generation models, covering critical gaze properties. 2) quantitative benchmarking of publicly available models using this framework. 3) analysis of scanpath generation models weaknesses to guide future improvements. ## methodology the core task involves predicting a complete scanpath representation @xmath0, ..., s n ⟩, where each point s i consists of both fixation positions @xmath1, ..., f n ⟩ and corresponding durations @xmath2, ..., d n ⟩, given an input sentence @xmath3, ..., w m ⟩. here, each fixation position f i corresponds to the index j (where 1 ≤ @xmath4. contemporary models demonstrate the capability to generate diverse scanpaths for identical text inputs, effectively simulating individual differences in reading patterns among human subjects. our analysis focuses on publicly available implementations of three existing approaches. the e-z reader model foot_0 implements a cognitive architecture that incorporates multiple psycholinguistic variables including lexical frequency, word predictability, and integration time parameters. this framework provides a comprehensive computational account of the interaction between perceptual, cognitive, and oculomotor processes during reading, explicitly modeling the mechanisms underlying saccade programming and execution that produce characteristic eye movement patterns. eyettention foot_1 adopts a probabilistic approach to predict subsequent fixation locations through the conditional distribution p (f i |w, s 1 , ..., s i-1 ), where the model considers both the textual input w and the preceding scanpath segment ⟨s 1 , ..., s i-1 ⟩ that includes landing position information. during inference, the model utilizes only the fixation position component of this history. the model architecture employs parallel processing streams: a word-sequence encoder leveraging bert embeddings @xcite with word-level aggregation, enhanced through bidirectional lstm processing and supplemented with explicit word length features; a fixation-sequence encoder implemented as a unidirectional lstm that processes concatenated representations of fixation word embeddings, normalized duration values, and within-word landing positions. these parallel representations are integrated through a cross-attention mechanism, with final predictions generated by a relu-activated fully-connected decoder network. the model produces scanpaths through iterative sampling from a probability distribution over possible saccade targets, including both progressive (forward) and regressive (backward) movements within the range -m + 1, ..., m (where m denotes maximum sentence length), plus an additional end-of-scanpath marker class, resulting in a 2m + 1-dimensional output space. training optimizes the mean negative log-likelihood objective. scandl 2.0 foot_2 introduces a modular architecture comprising two specialized components: the scandl module implements a discrete diffusion sequence-to-sequence model for sequence genera3 experiments ## conclusions this study systematically evaluates contemporary approaches to scanpath generation and comprehensively compares their capabilities and limitations against authentic human gaze patterns. our analysis of two distinct eye-tracking datasets reveals several important patterns that advance our understanding of current modeling paradigms. the scandl model for fixation sequence generation combined with the fixation duration module proves to be the most robust among evaluated models, demonstrating consistent performance across multiple evalu-ation metrics while maintaining reasonable proximity to the human baseline. however, even this model shows notable deficiencies in reproducing certain aspects of natural gaze behavior, particularly when evaluated on a new dataset containing longer sentences of different domains. the primary limitation is insufficiently accurate reproduction of gaze features, especially in correlation metrics. the model also fails to fully capture part-of-speechdependent variations in gaze patterns, particularly for re-reading time. while it performs well in assessing psycholinguistic predictors for within-dataset evaluation, its performance degrades in cross-dataset evaluation. eyettention represents an alternative approach that achieves competitive results. although it matches scandl on several key metrics, it underperforms in overall evaluation. when evaluated with the fixation duration module, eyettention shows deterioration in gaze latency-based features compared to scandl. this outcome highlights the importance of fixation sequence quality for the fixation duration module's performance. the e-z reader model, representing more traditional cognitive modeling approaches, demonstrates an interesting performance dichotomy. it performs similarly to scandl in within-dataset evaluation of fixation sequences regarding similarity, word skipping, and fixation counts, and outperforms scandl in cross-dataset evaluation. however, e-z reader shows significant difficulties with more complex gaze phenomena like regressions and fixation duration modeling. initially, the e-z reader model accepts parameters derived empirically, which complicates its application for generating synthetic data. consequently, the use of averaged and simulated parameters inevitably leads to a degradation in the quality of the generated gaze sequences. this pattern suggests that while symbolic cognitive models retain value for certain theoretical applications, they may require substantial enhancement to compete with data-driven approaches in practical implementations. comparative dataset analysis yields particularly valuable insights. the increased performance variability observed in the zuco dataset, with its more diverse text domains and longer sentences, underscores a critical challenge in gaze modeling -the need for systems capable of generalizing across different text types. this finding has important implications for practical applications, suggesting that future models will need to incorporate more diverse text domains. the persistent gap between model performance and human baselines across both datasets, particularly in correlation metrics, points to fundamental limitations in how current architectures represent the cognitive processes underlying reading. scandl was chosen as the reference model since it achieves the strongest overall performance. the results show that scandl significantly outperforms other models and random baselines on most metrics. however, some metrics highlight weaknesses of the model: for example, gaze feature metrics aggregated at the sentence level are significantly worse than those of other models. compared to the human baseline, scandl generally performs significantly worse, indicating the need for further modifications of scanpath generation models. several promising directions for improving scanpath generation systems emerge from these results. integrating multi-task learning objectives could help bridge the gap between gaze prediction and higher-level language understanding. incorporating psycholinguistic and other features may enhance models' ability to capture nuances of reading behavior. developing more comprehensive evaluation protocols, particularly those assessing models' capacity to reproduce known psycholinguistic phenomena across text domains, could drive significant improvements in model architectures and training approaches.