Decoding Open-Ended Information Seeking Goals from Eye Movements in Reading
Authors: Cfir Avraham Hadar
Omer Shubi
Yoav Meiri
Amit Heshes
Yevgeni Berzak
Venue: ICLR 2026 Poster (2026)
Keywords: Eye Movements in Reading
Multimodal Large Language Models
Information Seeking
Cognitive State Decoding
Primary area: applications to neuroscience & cognitive science
TL;DR: This paper introduces the task of decoding open-ended reading goals from eye movements in reading, and develops several models for this task.

Abstract: When reading, we often have specific information that interests us in a text. For example, you might be reading this paper because you are curious about LLMs for eye movements in reading, the experimental design, or perhaps you wonder ``This sounds like science fiction. Does it actually work?''. More broadly, in daily life, people approach texts with any number of text-specific goals that guide their reading behavior. In this work, we ask whether open-ended reading goals can be automatically decoded solely from eye movements in reading. To address this question, we introduce goal decoding tasks and evaluation frameworks using large-scale eye tracking for reading data in English with hundreds of text-specific information seeking tasks and auxiliary annotations of task-critical information. We develop and compare several discriminative and generative multimodal text and eye movements LLMs for these tasks. Our experiments show considerable success on selecting the correct goal among several options, and even progress towards free-form textual reconstruction of the precise goal formulation. We further tie model performance to cognitively interpretable aspects of human gaze behavior. These results open the door for further scientific investigation of goal driven reading, as well as the development of educational and assistive technologies that will rely on real-time decoding of reader goals from their eye movements.

Published as a conference paper at ICLR 2026
DECODING OPEN-ENDED INFORMATION SEEKING GOALS FROM EYE MOVEMENTS IN READING
Cfir Avraham Hadar, Omer Shubi; Yoav Meiri, Amit Heshes, Yevgeni Berzak Faculty of Data and Decision Sciences, Technion Israel Institute of Technology, Haifa, Israel [kfi r-hadar, shubá, mei ri. yoav, ami heshes campus techni on acc....... Berzak@techninn ac.
ABSTRACT
When reading, we often have specific information that interests us in text. For example, you might be reading this paper because you are curious about LLMs for eye movements in reading, the experimental design, or perhaps you wonder "This sounds like science fiction. Does it actually work?". More broadly, daily life, people approach texts with any number of text-specific goals that guide their reading behavior. In this work, we ask whether open-ended reading goals can be automatically decoded solely from eye movements in reading. To address this question, we introduce goal decoding tasks and evaluation frameworks using large-scale eye tracking for reading data in English with hundreds of text-specific information seeking tasks and auxiliary annotations of task-critical information. We develop and compare several discriminative and generative multimodal text and eye movements LLMs for these tasks. Our experiments show considerable success on selecting the correct goal among several options, and even progress to- wards free-form textual reconstruction of the precise goal formulation. We further tie model performance to cognitively interpretable aspects human gaze behav- ior. These results open the door for further scientific investigation of goal driven reading, as well as the development of educational and assistive technologies that will rely real-time decoding of reader goals from their eye movements.
INTRODUCTION
Eye movements in reading are a key methodology for studying the cognitive basis of reading and language processing. When we read, our eyes move over the text in saccadic manner, with pro- longed periods of time in which the gaze is stable, called fixations, and fast transitions between fixations called saccades (Schotter Dillon, 2025). This trajectory contains rich behavioral traces of the ways in which readers interact with texts and process language in real time (Just & Carpenter, 1980; Rayner, 1998). Understanding the nature and strength of the relation between eye movements, the text, and online language processing has been major research avenue in the psychology of read- ing the past few decades, and in recent years has also been gaining increasing interest in NLP and machine learning (Barrett & Hollenstein, 2020; Reich et al., 2025).
In this work, we investigate a reading scenario that prevalent in daily life but remains understudied seeking specific information in a text. This scenario deviates from common implicit assumption in psycholinguistic, according to which the comprehender's goal is constant across communicative contexts: general understanding of the linguistic input. In practice, readers approach texts with variety of goals and information seeking needs. Each such goal can have profound impact on the cognitive processes of language comprehension and the corresponding reading behavior (Radach & Kennedy, 2004; Kaakinen & Hyönä, 2010; Hahn & Keller, 2023; Shubi & Berzak, 2023).
Our study focuses on the following question: can arbitrary, text-specific information goals be auto- matically decoded from eye movements in reading? Specifically, given a single participant reading single passage with a question in mind that they would like answer via the text, can this question
"Equal contribution.
Code available http:://gggiaaaa.coom/ccccchahhe Goal-Decoding.

---

Published as a conference paper at ICLR 2026
be decoded from their eye movements over the passage? This task has not been addressed in prior research and has both scientific and practical importance. Scientifically, it allows obtaining insights into the strength and nature of the relation between task conditioning and reading behavior, and more broadly, the extent to which eye movements contain information on the cognitive state of the reader. Practically, it opens the door for applications in education, content personalization, and text accessibility, which will rely on detecting information seeking behavior, its specific purpose, and the extent which this behavior effective.
Our key contributions are the following:
We present a new cognitive state decoding challenge from eye movements in reading: de- coding arbitrary information seeking goals over the content of specific texts. We instantiate this challenge as goal selection task given several possible goals, and as an even more challenging goal reconstruction task where the iiformatioonseekingg goal has to be generated.
Tasks:
Models: For the goal selection task, we introduce two strong baseline models and adjust two state- of-the- art discriminative models of text and eye movements (Haller RNN and RoBERTEye). For the goal reconstruction task, we develop and finetune two generative LLMs, with textual representations of eye movements (DalEye-L.lama, DalEye-GPTT. We further use textual eye movement representa- tions to address this task using a frontier L.MM (Gemini) in zero-shot and few-shot regimes.
We perform systematic evaluations of goal selection accuracy at two levels of task difficulty and utilize several generation quality measures for the goal reconstruction task. For both tasks, we examine model generalization across new texts and new readers. Finally, we tie model performance to cognitively interpretable properties of human gaze behavior during goal decoding.
Evaluations:
EYE TRACKING DATA
We use OneStop (Berzak et al., 2025a), a public eye tracking dataset collected with an EyeLink 1000 Plus eye tracker. It comprises 360 adult native English speakers reading Guardian articles from the OneStopQA multiple choice reading comprehension dataset (Vajjala & Lučié, 2018; Berzak et al., 2020). In each experimental trial, participant reads a paragraph on screen and, the following screen, answers a multiple-choic reading comprehension question, without the ability to return to the paragraph. Each paragraph can appear in its original or human- simaa--impliiied form. 180 participants read in an information seeking condition, where they receive the question (but not the answers) before reading the paragraph, and therefore have specific reading goal for the upcoming paragraph. The remaining 180 participants do not receive the question ahead of time, and thus have to be ready to answer any subsequent question. We use the information seeking part of OneStop (Berzak al., 2025b), which is uniquely suited for our study.
For each paragraph, a participant receives one of three possible questions over the paragraph. Each such question has manually annotated critical span in the paragraph, which contains the infor- mation that is essential for answering the question. The questions tend to be more inferential than extractive: typically, the critical span does not contain the answer in verbatim form, but rather the essential information from which it can be inferred (Berzak et al. (2020), and see Appendix A for an empirical analysis which supports this property). Two of the questions share the same critical span, while the third question has a distinct critical span. This design supports a question selection task with two tiers of difficulty, distinguishing between questions with non-overlapping critical spans and the more challenging variant of distinguishing between questions over the same critical span. Figure presents an example of paragraph, its three questions, and their critical span annotations.
Questions
Paragraph
critical span c₁
Angele Erdimet never knew her grandfsther. He died in 1946, six years before she was born, Dul, Tuesday April 28h, 2014, she desenbed the extraord nary moment when recaived message in bottlo, 101 years throw the Battle See. bottle ls possibly the word'' oldest message in a bottlo It was preserted to Erdmern by the museum thas is now und sodiibling in Germanty.
Rkee Who threw the bottle into the Ballio Bee?
Rhae Where the botle now/
4sry How did Angela ind out pout the bottl??
critical span Co
Example of OneStop paragraph, its three questions, and each question's critical span: the paragraph segment essential for answering the question. See details notation in Section 3.
Figure 1:
2

---

Published as a conference paper at ICLR 2026
OneStop Information Seeking has 486 questions over 162 paragraphs, with 20 participants answer- ing each question. The mean question length is 10 words. The mean paragraph length is 109 words (5.8 sentences), of which the critical span is 32 words average. Overall, there 1,055,429 paragraph word tokens for which eye tracking data was collected.
3 PROBLEM FORMULATION
The general problem we address is using eye movements over a text (in our setting, text is always paragraph) to decode which question a participant seeks to answer using that text. Each text T has three text specific questions, Qr fg 92.c2: 93.ca J. The critical information for answering question Ω1.:: is located in the text in span 01, while for both questions (2,ca and gs.cz it in span cg- In each experiment trial, a participant P is presented with one question, qP E Qr, before reading the text. The recording of the participant's eye movements while reading text denoted by Ep,r- We propose the following two task variants.
Goal Selection In the goal selection task, a classifier has to select which question from Qr the participant received, from their eye movement recording over the text:
(T, EP,T, QT) - qr E QT
Goal Reconstruction In the goal reconstruction task, a generative model has to generate the question that the participant received, from their eye movement recording over the text:
(T, EYT))
MODELS
We introduce discriminative and generative models, presented schematically in Figure 2.
Chosen Question dp Qr
No Or
Scores
Genersted Queetion
Discriminative Modell
Generative Madiel g
q1 Q2 Q3
Cordidate
Text Y'
Partictpant's P Eye Movements over the Text Egy
Text 7
Paticciant's P Eye Movemanta over the Text
Questrons Or
(a) Discriminative models (Section 4.1)
(b) Generative models (Section 4.2)
Two model types for decoding the question presented to participant before reading the text, from their eye movements over the text. (a) Discriminative models that score candidate questions and are evaluated question selection accuracy. (b) Generative models that reconstruct the question presented to the reader, and are evaluated on question reconstruction quality.
Figure 2:
4.1 DISCRIMINATIVE MODELS
READING-TIME INFORMED EMBEDDING SIMILARITY MODELS
We introduce two baseline models which build on the following observations from prior literature: (i) readers tend to spend more time on question-relevant than on question-irrede inrelevant portions of the text (Kaakinen et al., 2002; Malmaud et al., 2020; Hahn & Keller, 2023; Shubi & Berzak, 2023), and text segments tend to have higher semantic similarity to the question compared (ii) irrelevant segments (Mitra et al., 2016). to question-irrelevant
Question Similarity to RT-Weighted Passage (Figure 3a) For a participant reading a text of N words, compute text embedding, emb(T), weighted average of its contextualized word

---

Published as a conference paper at ICLR 2026
(mr)
Score
sierlo
stitle
stmip
Scoree
RT (m)))
JET (MNN)
emb(ww))
emb(ww))
embh(My)) amb(T)
(0L2(M)
cmsbtin))
smii(ay))
emb
(a) Question Similarity to RT- RT-Weighted Passage
(b) RT Similarity to Question- Seosionn-Word Similarities
Figure 3: Reading-Time Time Informed Embedding Similarity models for the question selection task.
embeddings, where the weight of each word is the fraction of time spent reading it. Formally: emb(T) = - (T;))) emb(w,), (w;). The embeddings for word uy emb(u;). are extracted from RoBERTa (Liu et al., 2019), and (T(w)) ) is the word's Total Fixation Duration (i.e., sum of all the fixation durations) normalized by the text's total reading time. For each candidate question gj. we use its [CLS] token embedding as emb(g;). and compute its cosine similarity with emb(T). We then predict the question with the highest similarity: arg (1.... cos(emb(T), (mmmmb)))
RT Similarity Question-Word Similarities (Figure 3b) An alternative to the model above, compares the vector of word reading times for the text to the vector of word-question similari- ties. Specifically, for a given candidate question g E Qr and each text word wi, we first compute sim(gj, w;) cos coosemm(g)) emb(w:)). where as before, emb(w:) is a contextual word embedding from RoBERTa, and emb(g)) (qj) is the question [CLS] embedding, yielding a length-N vector of simi- larities sim (z. ? [sim(gj, W1), sim(qj, W2). .... sim(g;. W,)]] We select the question maximizing the cosine similarity between this vector and RT),). the length-N vector of speed-nowd norealzorm Total Fixation Durations for the text words: arg max j cos (sim((g)) T), RT(T)).
"TATE-OF-THE-ARE NEURAL MODELS
We adapt two state-of-the-art encoder models for eye movements and text. Building on the approach of Radford al. (2019) for encoding multiple-choice QA items, we several copies of each text, where instead of combining each copy with a candidate answer, we combine it with a candidate question, and further provide the participant's eye movements over the text (I' EPT, q E Qr). The models assign a probability to each triplet and select the highest probability question.
Haller RNN (Haller et al., 2022) An RNN model that represents each fixation on the text using eye movement features concatenated with the corresponding RoBERTa-bens based word embedding. We append an embedding of the question to the input of the model. The model scores each candidate question. Additional details on the model, including diagram of the modified architecture and the used eye movement features, are presented Appendix B.I.
Fixations (Shubi et al., 2024) A transformer- model which integrates fixation- level eye movement features into RoBERTa. An eye movements feature vector for each fixation is first projected to the dimension of the word embeddings. These fixation vectors are then aligned with their corresponding words via the addition of position embeddings, and finally concatenated with the word sequence. We adapt this model to incorporate candidate question embeddings and modify the classification layer to score questions. Appendix B.2 provides further details, including model diagram and the eye movement features that the model uses.
RoBERTEye-Fixaat
4.2 GENERATIVE MODELS
We experiment with two approaches for generating the question based on eye movements over the text. The first approach (DalEye-Llama, DalEye-GPT) is fine-tuning, where the task, the text, and the eye movements are provided to the model in the prompt in textual form. The second approach uses the same prompts for zero-shot and few-shot generation from frontier LLM (Gemini).
DalEya-Llama We provide the task, the text, and the eye movement trajectory a participant the text Llama 3.1 (Grattafiori et al., 2024) in textual form. The model is fine-tuned to
4

---

Published as a conference paper at ICLR 2026
reconstruct the true question given this input using autoregressiiv next-token prediction objective. We use a seation-basee eye movements representation, where each fixation is represented as tuple containing the index of the fixated word, the word itself, the fixation duration, and the direction of the next saccade. This encoding preserves both the spatial locations and the temporal order of the fixations. This input is preceded by the text and task description instructing the model generate the question that was presented to the reader. An example of a prompt is shown in Appendix C.I.
DalEye-GPT A similar approach to DalBye-Llama, with identical prompts, where instead we fine- tune commercial LLM, GPT-4o-mini (Hurst et al., 2024).
Gemini We evaluate frontier commercial LLM,, Peeiini---Pr--Peeiiie (Google DeepMind, 2025) in zero-shot and few-shot settings. In the zero-shot setting, the model receives the same textual input as DalEye- Llama and DalEye-GPT. In the few-shot setting, the prompt further includes 10 randomly drawn trial examples with ground truth questions (see Appendix C.1.2 for details).
In Appendix C.2 we further provide details on DalEye-LLaVA, an alternative, embedding based approach to finetuning based on the LLaVA-One Vision language- -viionn model architecture (Li et al., 2025). Although this model mostly did not perform well on our task, we provide its description and evaluations in the Appendix reference point for future embedding based models.
EXPERIMENTAL SETUP AND EVALUATION
5.1
DATA SPLITS
Following prior work on predictive modeling with OneStop (e.g. Shubi et al., 2024), we use a 10- fold volss-aaiittionn procedure where each of the 10 data splits has training, validation, and test set. The OneStop eye tracking data is not i..d.; each participant reads multiple paragraphs, and each paragraph read by multiple participants. To account for this structured nature of the data, the validation set and the test set of each data split are each partitioned into three disjoint parts that capture three different levels of model generalization, ordered by task difficulty:
New Participant: Training eye tracking data is available for the text (with each of the three possible questions) but not for the participant.
New Text: Training eye tracking data is available for the participant but not for the text.
New Text& Participant: Neither the participant nor the text was in the training data.
Each data split includes 64% of the trials in the training set, 17% in the validation set, and 19% in the test set - divided into 9% New Participant, 9% New Text, and 1% New Text Participant. When aggregated across the 10 splits, 90% of the trials appear in both New Participant and New Text evaluation regimes, and 10% appear in the New Text & Participant regime. We ensure that in each split into training, validation, and test sets, there is balanced distribution of question types, ie. T1,e (2,02> 43.c2 for each text in each part of the split. Hyperparameter optimization and model selection are performed separately for each split. We assume that the evaluation regime with spect to novelty texts and participants at test time is unknown, and therefore model selection is performed using the entire validation set within each split. Further details on the hyperparameter search space, training, and evaluation procedures are provided in Appendix D.
5.2 GOAL SELECTION
As described Section 4, the discriminative models assign probability to a triplet (T, EyT, q E Qr), and select the candidate question with the highest probability. We report three-way selection accuracy All, as well as accuracy for two specific cases of interest:
Different critical spans This evaluation tests the ability to distinguish between questions with disjoint critical spans. We pose the selection problem as a binary decision between Q1,c; and (92.0,, ). }. Random choice accuracy in this evaluation is approximately 55%, rather than 50%, due to Monty Hall-type setup arising from grouping two questions under one critical span (see Appendix E).
5

---

Published as a conference paper at ICLR 2026
Same eritical span This is more challenging evaluation for differentiating between the two ques- tions with the same critical span C2: 92,c, versus 93,e2- Note that here we disregard the question whose critical span is C1. The corresponding chance accuracy is 50%.
5.3 GOAL RECONSTRUCTION
The generative models are evaluated on the quality of the reconstructed questions. As with other generation tasks, evaluating reconstruction quality is major challenge. Here, we use several heuris- tic evaluations for both the surface form and the semantic content of the generated questions.
Question word accuracy We test whether the generated question word matches the question word of the ground truth question. The possible question words are What, When, Where, Who, Whom, Which, Whose, Why, and How. An additional category is Other, reserved for all other words starting a question (7% of the questions).
UIUC question category accuracy We check whether the generated question has the same semantic category as the ground truth question according to the question-type taxonomy of Li & Roth (2002). We use GPT-4o (Hurst et al., 2024) categorize the questions into the main taxonomy categories: entity, human, numeric, and location, and the subcategories of 'description': manner, reasoning, definition, and description. Appendix F.l includes further details on the classification process and on the agreement of GPT-4o with manual human annotations.
BLEU The BLEU surface similarity score (Papineni al., 2002) between the generated question and the true question.
BERTScore (Zhang et al., 2020) cosine similarity score between the generated question and the true question based on contextual embeddings extracted from RoBERTa.
QA accuracy downstream evaluation with the following logic: the closer the generated question is to the true question, the easier it should be to choose the correct answer for the true question given the reconstructed question. We consider question as a valid reconstruction of the true question and only if, given this question, the text, and the four answers for the true question, the model selects the correct answer. We implement this evaluation using RoBERTa fine-tuned for multiple choice QA on RACE (Lai et al., 2017), the 86% of OneStopQA questions that RoBERTa answers correctly.
BASELINES
To facilitate model performance interpretation, we introduce the following baseline questions which are composed without eye movement information.
Human incorrect questions The two additional human-composed questions for each text, one with different and one with the same critical span as the true question (Section 2). These questions are
guaranteed to fulfill two key criteria: they are well-formed and can answered from the passage.
LLM arbitrary questions An arbitrary question for each passage generated with the most recent frontier LLM, Gemini-3-Pro-Preee Pree-reee using the prompt presented in Appendix F.2.
LLM questions from text-only question decoding models Questions from Llama 3.1 and GPT- 4o-mini finetuned to decode the question in using the same experimental setup as DalEye-Llama and DalEye-GPT. but without eye movements data.
The questions generated from the LLM baselines enable quantifying the contribution of eye move- ment information in each of our generative models, above and beyond textual heuristics. This in- cludes effects of possible model exposure to the textual materials of OneStopQA during pretraining.
RESULTS
6.1 GOAL SELECTION
Table presents the aggregated test accuracy across the 10 data splits for the three-way All goal selection, and a breakdown into questions with Different Spans, and the Same Span evaluations. A division of the results into the New Participant, New Text, and New Text & Participant regimes is

---

Published as a conference paper at ICLR 2026
Table I: Goal selection test accuracy aggregated over 10 data splits, with 95% confidence inter- vals. A majority choice is identical to chance because the data is balanced across questions. As the data is not i.i.d. (each participant reads multiple paragraphs, and each paragraph is read by multi- ple participants), we test for differences between models using a linear mixed effects model with random intercepts and slopes for participants and paragraphs: is is_correcl ~ model + (model parlicipant) + (model paragraph). Significant gains over chance performance are marked with ** p < 0.05, *#*' < 0.01, and ***** p < 0.001. The best performing model in each evaluation is marked in bold. Significant drops compared to the best model are marked with '+++' p 0.001.
Model
All
Different Spans
Same Span
Chance Majority Bascline
33.0 + 0.4+++
55.3 0.4+++
49.9 = 0.4+++
Question Similarity to WT-Weighted Passage
33.4 + 0.3.++
55.1 + 3.7
|
50.0 = 0.4+++
RT Similarity to Question- Word Similarities
34.2 + 0.42,+
54.8 + 1.4,++
51.1 = 0.5
Haller RNN (Haller et al., 2022)
41.8 + 0.6214
65.6 + 0.5144
52.1 = 0.5
RoBERTEye-Fixation (Shubi et al., 2024)
49.3 + 0.3
70.9 + 0.5*
57.3 + 0.5
reported in Tables 5 to 7 in Appendix G.1.1. Validation set results are reported in Tables 8 to 11 in Appendix G.1.1. A secondary evaluation of the open generative models DalEye- -lamm and DalEye- LLaVA on the question selection task is reported in Appendix G.1.2.
Both Reading-Time Informed Embedding Similarity baseline models perform at chance level across all three evaluation regimes, with the exception of RT Similarity to Question Word Similarities in the All condition, where performance is marginally above chance. A possible reason for this outcome lies in the inherent noise in the distribution of reading times of single participant over single paragraph, which deems heuristic methods that rely only on this distribution to be ineffective. Haller RNN and RoBERTEye-Fixatnn Fixations the other hand perform well above chance.
RoBERTEye- Fiixttions achieves the highest accuracy across all three evaluation regimes. In Table 5 of Appendix 1.1.1, we observe that this performance advantage is consistent across the New Partic- ipant, New Text, and New Text & Participant evaluations. Thus, the model is able to generalize not only new participants but importantly also new texts. Notably, in addition achieving 70.9% accuracy on the Different Spans evaluation, it is also the only model that is well above chance in the Same Span regime with 57.3% accuracy. While these results confirm that distinguishing be- tween questions that ask about the same portion of the text is indeed much more challenging than distinguishing between questions over different parts of the text, the above-chance performance of RoBERTEye- Fixations on the Same Span evaluation speaks to the fine-grained nature of the infor- mation that can extracted about the reader's information seeking goals from their eye movements alone. Appendix 1.1.3 reports feature ablation analysis of RoBERTEye- Fixations, which sug- gests that the model relies much more heavily on the ordering of word embeddings according to the sequence of fixations than any additional features of each individual fixation.
LINKING MODEL PERFORMANCE TO COGNITIVE INDICATORS OF INFORMATION SEEKING
To tie automated goal decoding to cognitive aspects of gaze behavior during information seeking, we the top performing model in the
present a model performance analysis of RoBERTTEye-Fixtton goal selection task, based on the framework proposed by Shubi al. (2025). The analysis examines how features of the experimental trial that are pertinent to cognitive theory of information seeking (e.g. Kaakinen al. (2002); Hahn & Keller (2023); Shubi & Berzak (2023)) and are not provided to the model, affect model performance, Specifically, it uses the following linear mixed effects model which accounts for the non i.i.d structure of the data: Plis_ ((is_coorect) ~ feat; + + feat, + (1 participant) + (1 paragraph), where P(is_ correct) is the probability assigned by RoBERTEye- Fixations to the correct question given an input of a paragraph and eye movements. We use the 10 trial features from Shubi et al. (2025) which capture aspects of the participant's interaction with the textual item, and the item itself, and further include the lexical overlap between the question and Figure 4 depicts the resulting model coefficients.
critical span, quantified with
7

---

Published as a conference paper at ICLR 2026
Reading Time Before CS
Textual item Participant
Reading Time In CS
Reading Time After CS
***
Paragraph Position
Answered Correctly
Paragraph Length
Paragraph Difficulty Level
Critical Span Start Location
Critical Span Length
Question Difficulty
Question Critical Span Lexical Overlap
-0.05
0.00
0.05
0.10
0.15
Coefficient
Figure 4: Coefficients from a mixed-effects model which predicts the probability assigned by RoBERTEyy- to the correct question from 11 features of the experimental trial. Predictors are z-normalized, to make the coefficient magnitudes comparable. Coefficient statistical significance after 1x Bonferroni correction is marked with p < 0.05, *#** 0.01, and *$*** p 0.001.
Participant features over the textual item Prior studies in psycholinguistics have observed shorter reading times on goal irrelevant compared to goal relevant information, which was linked to ratio- nal processing during information seeking (Kaakinen al., 2002; Hahn & Keller, 2023; Shubi & Berzak, 2023). Intriguingly, we find that the extent to which this holds affects model performance a highly interpretable manner: longer reading times (per word Total Fixation Duration) before and after the critical span hurt model performance (p < 10 and p < 10-71 respectively), while longer reading times within the critical span improve it (p < 10-276). In other words, the more goal directed and resource efficient the reader is, the easier it is to identify their goal automatically. The paragraph's position in the experiment (1-54) is also positively correlated with model performance (p < 10 10-3), likely due to participants' adaptation to the task over time (e.g. Wells et al., 2009; Chromý & Tomaschek, 2024). Better reading comprehension is similarly positively associated with model performance (p 0.01), in line with work linking better time division between goal relevant and irrelevant information and reading comprehension performance (Shubi & Berzak, 2023).
Textual item features Longer paragraphs associated with improved model performance (p 10-9), possibly as they allow better separation between question relevant and question irrelevant material, as well as between critical spans of different questions. Longer critical spans, which po- tentially support more robustness in the extraction of critical span related eye movements, are also positively correlated with model performance (p < 10-5). Interestingly, this result complements an opposite trend for models that are trained to distinguish between information seeking and reading for comprehension without receiving the question ahead of time (Shubi al., 2025), where longer critical spans make it harder for models to distinguish these two reading regimes. Similarly to the results in Shubi al. (2025), paragraph difficulty level (original or simplified), and critical span lo- cation do not have significant effect on the model's performance. Finally, we do not find evidence for an influence of the lexical overlap of the critical span with the question. Overall, these findings provide converging evidence and expand prior observations from the psycholinguistic literature information seeking using the developed model as an analytical tool.
6.2 GOAL RECONSTRUCTION
Figure 5 presents the goal reconstruction evaluations in the New Participant and the New Text evalu- ation regimes. Tables and 15 in Appendix G.2 present these results numerically, and also include the New Text & Participant regime, which yields similar results to the New Text regime. In the New Participant evaluation, Lalmaa, DalEye-GPT and Gemini few-shot generated questions outperform all the baseline questions on all five evaluation measures (p 0.001 in all cases)?, cept for Gemini few-shot in the Question Category evaluation. These performance gains suggest that these models generalize relatively well to new participants when the text and possible ques- tions appeared in the training set. Gemini few-shot is the top performer in this regime, with the exception of Question Category where DalEye- performs best. Notably, the text-only version
Perfforancc differences tested using: measure model (1 participant) paragraph).
8

---

Published as a conference paper at ICLR 2026
New Participant
New Texxt
Question Word accuracy
Question Calegory accuracy
BLEU
BERTScore
QA aacuraccy
0.8
0.6-
0.8
0.05
0.6
0.6
0.4
0.65
0.4
0.4
0.2
0.45
0.2
0.2
0.2
0.0
0.0
0.0
0.0
0.25
In Human
Text-orly Uaaa 3.1
Text-anly GPT-4o-miri
Arbitrary Gemini
(different critical span)
DalEye-Uaama
DolEye-GPT
Gemini zero-shot
[same critical span)
Incorrect Human
few-shot
Figure 5: Goal reconstruction from eye movements evaluations of the finetuned models DalEye- Llama and DalEye-GPPP GPree as well as zero-shot and few-shot Gemini for (1) the identity of the gener- ated question word, (2) UTCC semantic category of the question, (3) BLEU score, (4) BERTScore, (5) downstream QA accuracy based on the answer selection of multiple-choice QA model. The models are benchmarked against five baselines that do not include eye movements. Two baselines human-composed questions for different and the same critical span. Three additional base- lines are LLM-generated questions: an arbitrary question about the text generated with Gemini 3, and questions from text-only variants of DalEye- Llama and DalEye- GPP finetuned on the question decoding task using only the text. Results are aggregated over 10 data splits. Presented are intercept coefficients with 95% confidence intervals from linear mixed effects models with random intercepts for participants and paragraphs: measure ~ 1 + (1 participant) + (1 paragraph) for eye movement models and measure + (1 paragraph) for the baselines. The highest score in each combination of evaluation measure and evaluation regime is marked with *.
of DalEye-Llamen Llama 3.1 finetuned for question reconstruction without eye movements, is the strongest baseline for this evaluation regime across all five evaluation measures.
The performance of DalEye-Llama, DPPE and Gemini few-shot drops considerably in the New Text regime, in absolute terms and relative to the baselines, suggesting weaker generalization to new texts and questions. However, even in this challenging evaluation regime, all three models consistently outperform their corresponding text-only or arbitrary counterparts, except for DalEye- Llama on the Question Word and Question Category evaluation. This indicates that eye movements valuable for the generation task even when the textual item is novel. The best performing model differs by evaluation measure: DalEye-Llama on Question Word, DalEye-GPT on Question Cate- gory, and Gemini few-shot QA accuracy. The Incorrect Human (same critical span) questions achieve the best results on BLEU and BERTScore, likely due to lexical overlap and semantic prox- imity the correct question, stemming from sharing the same critical span. The performance of Gemini zero-shot is similar across both regimes, because similarly to the human baselines, this model is not finetuned, and it tends to perform poorly in both regimes.
Table 16 in Appendix G.2 presents examples of questions generated by DalEye-Llama, DalEye- GPT, and Gemini. Table 18 in Appendix G.2 presents experimental results for DalEye-Llama using a differently worded task prompt, which yields similar results, as well as two alternative textual representations of the eye movement sequence, one based on word-level features and another com- bining word- and fixation-level features. The alternative representations yield inferior results to those presented above.
RELATED WORK
Analyses of Goal Based Reading While most prior work psycholinguisties focuses on reading for comprehension (Rayner et al., 2012), goal or task based reading has been acknowledged as key frontier in this area (Radach & Kennedy, 2004). Several studies found differences eye movements between reading for comprehension and tasks that define reading manner: skimming, speed read- ing and proofreading (Just et al., 1982; Kaakinen & Hyönä, 2010; Schotter et al., 2014; Strukelj & Nichorster, 2018; Chen et al., 2023). Additional studies found differences between reading for com- prehension and tasks of target word search (Rayner & Raney, 1996; Rayner & Fischer, 1996), word verification (Radach et al., 2008), and topic search (White et al., 2015). Prior work also analyzed

---

Published as a conference paper at ICLR 2026
eye movements during linguistic annotation tasks (Tomanek et al., 2010; Tokunaga et al., 2017)
Differences in reading patterns were also found when readers adopt different perspectives on a text (Kaakinen et al., 2002; Kaakinen & Hyönä, 2008), or given different learning goals (Rothkopf & Billington, 1979). Overall, these works consistently found systematic influence of the reading task on reading patterns. We build most directly Kaakinen et al. (2015), Malmaud et al. (2020), Hahn & Keller (2023), and Shubi & Berzak (2023), who compared eye movements in reading for com- prehension and open-ended information seeking. These studies showed task conditioning effects on reading patterns, especially as a function of the task-relevance of the textual information. Similarly to these studies, we formulate the information seeking task as a reading comprehension question.
Decoding Cognitive State from Eye Movements Multimodal modeling of eye movements and text has been a growing research area recent years (Reich et al., 2025), with applications ranging from improving NLP (e.g. Klerke et al., 2016; Mishra et al., 2016; Barrett & Hollenstein, 2020; Sood et al., 2020; Khurana et al., 2023; Deng et al., 2024; López-Cardone et al., 2025) to prediction of eye movements (e.g. Hollenstein et al., 2021a; Bolliger et al., 2023; Deng et al., 2023; Chen et al., 2024). Within this line of research, several studies focused on cognitive state decoding: prediction of language-relin related aspects of the reader and their cognitive state from eye movements. These include studies on prediction of the reader's ' linguistic background (Berzak et al., 2017: Reich et al., 2022; Skerath et al., 2023), language proficiency (Berzak al., 2018), reading comprehension (Ahn al., 2020; Reich et al., 2022; Shubi et al., 2024), subjective readability (Reich et al., 2022) and perceived relevance (Bhattacharya al., 2020a;b).
Decoding Reading Goals Most notably, two studies addressed decoding of the reader's goals, Hol- lenstein al. (2021b) classified reading for comprehension versus annotation of semantic relations, Shubi et al. (2025) classified reading for comprehension versus information seeking. Importantly, in both studies, the decoding tasks are procedural, where the goal is distinguish between a small set of pre-defined manners of reading that are not specific to particular text. In contrast, our study addresses related but conceptually and practically different task, which is semantic; instead of a small number of categories that can apply to any text, we have hundreds of text-specific goals that can be arbitrary in nature. See further discussion on the procedural versus semantic distinction in Shubi & Berzak (2023). Furthermore, these and other studies cognitive state decoding typically use discriminative, encoder-basee
models that are limited to classification tasks. Here, we also de- velop generative, decoder- based language models. We envision that such models will have broad relevance for future eye movement conditioned text generation tasks.
8 SUMMARY AND DISCUSSION
We introduce new challenge of both scientific and practical value: decoding arbitrary information seeking goals from eye movements in reading. We tackle this challenge in two formulations, goal se- lection and goal reconstruction, using number of evaluation frameworks and modeling approaches. The best performing model on the selection task is able to predict the correct question with a con- siderable degree of success, even when the candidate reading goals pertain to the same information in the text. The performance of the model is driven by cognitively interpretable factors of human information seeking. Our results further suggest that although the reconstruction task extremely challenging, meaningful progress can also be made in this open-ended regime. Overall, we find that effective modeling can extract highly valuable information regarding specific information seeking goals at the challenging granularity level of a single paragraph.
Automatic decoding of information seeking goals can pave the way for new applications with posi- tive societal value. In the future, media outlets and providers of municipal and governmental services could better understand the information needs of users who access their websites and render informa- tion in a manner that better meets these needs. Automatic identification of information seeking goals can also be used for real-time assistance to special populations, when accessing critical information on the web. Future e-learning systems could assess students' progress information-ssekingg tasks and monitor their information- seeiinn skills over time. Finally, one can imagine interactive reading interfaces that include real-time text personalization, such simplification and suggestion of addi- tional relevant information, according to the reader's information seeking goals. The present study is first step in enabling such applications.
10

---

Published as a conference paper at ICLR 2026
ETHICAL CONSIDERATIONS
The OneStop Eye Movements dataset used in this work was collected by Berzak et al. (2025a). The study was conducted under institutional IRB protocol, and all the participants provided written consent before participation. The data is anonymized. Analyses and predictive modeling of task- based reading are among the primary use cases for which the data was collected.
While the current work is scientific proof of concept and is not aimed at a particular user-facing application, has consider potential cases and risks for the presented task. In particular, it is important to note that given the current level of accuracy of goal decoding, educational and assistive technologies that rely on this task may be unreliable and introduce biases for various individuals and groups, such as L2 readers and groups with reading and cognitive impairments. Additional data collection and analyses are needed to assess such biases.
Prior work has demonstrated that eye movements can be used for user identification (e.g. Bednarik al., 2005; Jäger al., 2020). We do not perform user identification this study. We further emphasize that future reading goal decoding technologies must be used in real-world applications only with explicit consent from potential users have their eye movements collected and analyzed for this purpose.
10 ACKNOWLEDGMENTS
This work was supported by ISF grant 1499/22.
11 REPRODUCIBILITY STATEMENT
We describe the model training and selection procedure, the evaluation protocol, the hyperpa- rameter search space for each model, and the hardware and software specifications in Section and appendix D. The OneStop dataset (Berzak et al., 2025a) used in the experiments and de- scribed in Section 2 is publicly available https: http//offf///p///// /ttp/p//////../// The code for the paper, which implements all the models, experimental procedures, and analyses, is available at: https://gihubb.co//lalcaaba Eaee--Edded-dd Goal-Decodingg
REFERENCES
Seoyoung Ahn, Conor Kelton, Aruna Balasubramanian, and Greg Zelinsky. Towards Predicting Reading Comprehension From Gaze Behavior. In ACM Symposium on Eye Tracking Research and Applications, pp. 1-5, 2020. doi: 10.1145//379155.33391333
Maria Barrett and Nora Hollenstein. Sequence labelling and sequence classification with gaze: Novel uses of eye-tracking data for Natural Language Processing. Language and Linguistics Compass, 14(1 :):e1 2020. doi: 1a1111//nn3.1233
Douglas Bates, Phillip Alday, Dave Kleinschmidt, PhD José Bayoán Santiago Calderón, Likan Zhan, Andreas Noack, Milan Bouchet- Valat, Alex Arslan, Tony Kelman, Antoine Baldas- sari, Benedikt Ehinger, Daniel Karrasch, Elliot Saba, Jacob Quinn, Michael Hatherly, Morten Piibeleht, Patrick Kofod Mogensen, Simon Babayan, and Yakir Luc Gagnon. JuliaS- tats/MiiedMMddel.jj.. v4.7.0, 2022. URL hittps:/zzenddo zenodo org http:///orrd/////////// 6925652.
Roman Bednarik, Tomi Kinnunen, Andrei Mihaila, and Pasi Fränti. Eye-movements biometric. In Image Analysis: 14th Scandinavian Conference, pp. 780-789, 2005.
Yevgeni Berzak, Chie Nakamura, Suzanne Flynn, and Boris Katz. Predicting Native Language from Gaze. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 541-551, 2017. doi: 10. 1653/v1//17-110001
Yevgeni Berzak, Boris Katz, and Roger Levy. Assessing Language Proficiency from Eye Move- ments in Reading. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1986-1996, 2018. doi: 10. 1
11

---

Published as a conference paper at ICLR 2026
Yevgeni Berzak, Jonathan Malmaud, and Roger Levy. STARC: Structured Annotations for Reading Comprehension. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics., 2020.
Yevgeni Berzak, Jonathan Malmaud, Omer Shubi, Yoav Meiri, Ella Lion, and Roger Levy. OneStop: A 360-participant English eye-tracking dataset with different reading regimes. Scientific Data, 2025a. doi: 1411038/41159--25-666666
Yevgeni Berzak, Jonathan Malmaud, Omer Shubi, Yoav Meiri, Ella Lion, and Roger Levy. OneStop Information Seeking. doi: 10. (0.17655/SF::0//PGXXX 2025b.
Nilavra Bhattacharya, Somnath Rakshit, and Jacek Gwizdka. Towards real-time webpage relevance prediction using convex hull based eye-tracking features. In ACM Symposium on Eye Tracking Research and Applications, 2020a. doi: 10. 10.1145/3379155.3399130
Nilavra Bhattacharya, Somnath Rakshit, Jacek Gwizdka, and Paul Kogut. Relevance prediction from eye-movements using semi-interpretable convolutional neural networks. In Proceedings of the 2020 Conference Human Information Interaction and Retrieval, pp. 223-233 2020b. doi: 10. 00.1145/33434133337996/
Lena Bolliger, David R Reich, Patrick Haller, Deborah Jakobi, Paul Prasse, and Lena Jäger. ScanDL: A diffusion model for generating synthetic scanpaths on texts. In Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pp. 15513-15538, 2023. doi:
Xianyu Chen, Ming Jiang, and Qi Zhao. Gazexplain: Learning predict natural language explana- tions of visual scanpaths. In European Conference on Computer Vision, pp. 314-333 2024.
Xiuge Chen, Namrata Srivastava, Rajiv Jain, Jennifer Healey, and Tilman Dingler. Characteristics of deep and skim reading on smartphones vs. desktop: A comparative study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1-14, 2023.
Jan Chromý and Fabian Tomaschek. Learning or boredom? task adaptation effects in sentence processing experiments. Open Mind, 8:1447-1468, 2024.
Shuwen Deng, David Reich, Paul Prasse, Patrick Haller, Tobias Scheffer, and Lena Jäger. Eyettention: An attention-based dual-sequence model for predicting human scanpaths during reading. In Proceedings of the ACM on Human Computt Interaction, pp. 1-24, 2023. doi: 10.1145/359
Shuwen Deng, Paul Prasse, David R Reich, Tobias Scheffer, and Lena A Jäger. Fine-tuning pre- trained language models with gaze supervision. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pp. 217-224, 2024.
William Falcon and The PyTorch Lightning team. Pytorch lightning, 2024.
Google DeepMind. Gemini 3 pro model card. https: wow.heape..oogeppppeo.oo...l...e com/ deepmind Soeepinnd-eedia///ee 1eroodee-Cards//ee Satmiini--Prr---ee Pro-Mooeell-- Marde-oaaeedddaeeeeeeee pdf, 2025. Model card published November 2025.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.2178. 2024.
Michael Hahn and Frank Keller. Modeling task effects in human reading with neural network-based attention. Cognition, 230: 105289, 2023.
Patrick Haller, Andreas Säuberli, Sarah Kiener, Jinger Pan, Ming Yan, and Lena A Jäger. Eye- tracking based classification of Mandarin Chinese readers with and without dyslexia using neural sequence models. In Proceedings the Workshop on Text Simplification, Accessibility, and Read- ability (TSAR-2022), pp. 111-118, 2022, doi: 10. 10.18653/v122022..sar.
Daniel Han, Michael Han, and Unsloth team. unslothai / /uns//nsllt
Unsloth, 2023. URL http: http//gitttt ggtghbtthhu/gg/g/gg com/
12

---

Published as a conference paper at ICLR 2026
Nora Hollenstein, Federico Pirovano, Ce Zhang, Lena A Jäger, and Lisa Beinborn. Multilingual language models predict human reading behavior. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 106-123, 2021a. doi: 10.18653/v1/2011 .naacl-main. 10.
Nora Hollenstein, Marius Tröndle, Martyna Plomecka, Samuel Kiegeland, Yilmazcan Özyurt, Lena Jäger, and Nicolas Langer. Reading Task Classification Using EEG and Eye-Tracking Data. arXiv:21 10 [es], 2021b.
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022. URL https: www.reviewwe....mom net forum?
id-nZeVKeeFYE9.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024.
Lena A Jäger, Silvia Makowski, Paul Prasse, Sascha Liehr, Maximilian Seidler, and Tobias Scheffer. Deep eyedentification: Biometric identification using micro-movements of the eye. In Machine Learning and Knowledge Discovery in Databases: European Conference, pp. 299-314, 2020.
Marcel Just and Patricia Carpenter. A theory of reading: From eye fixations to comprehension. Psychological Review, $7:329-354 1980. ISSN 1939-1471. doi: 10.1037/0033-295X. 87.9..29.
Marcel A Just, Patricia A Carpenter, and MEJ Masson. What eye fixations tell us about speed reading and skimming. Eye-lab Technical Report Carnegie-Mellon University, Pittsburgh, 1982.
Johanna K Kaakinen and Jukka Hyönä. Perspective-driven text comprehension. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 21(3):11-333, 2008.
Johanna K Kaakinen and Jukka Hyönä. Task Effects Eye Movements During Reading. Journal of Experimental Psychology. Learning, Memory, and Cognition, 36:1561-6, 2010. doi: 10.1037/ a0020693.
Johanna K Kaakinen, Jukka Hyönä, and Janice M Keenan. Perspective effects on online text pro- cessing. Discourse Processes, 159-):159-173, 2002.
Johanna K Kaakinen, Annika Lehtola, and Satu Paattilammi. The influence of reading task on children's 's eye movements during reading. Journal of Cognitive Psychology, 27(5):640-656, 2015.
Varun Khurana, Yaman Kumar, Nora Hollenstein, Rajesh Kumar, and Balaji Krishnamurthy, Syn- thesizing human gaze feedback for improved NLP performance. Proceedings of the 17th Con- ference of the European Chapter of the Association for Computational Linguistics, pp. 1895- 1908, 2023. doi: 10.18653/v 1/202/..212222211111 main.
Sigrid Klerke, Yoav Goldberg, and Anders Søgaard. Improving sentence compression by learning to predict gaze. In Proceedings of the 2016 Conference of the North American Chapter of the As- sociation for Computational Linguistics: Human Language Technologies, pp. 1528-1533, 2016. doi: 10. 10.18653/1//N16-11177
Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. RACE: Large-scale ReAding comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 785-794, 2017. doi: 10.18653/1/D117-10822
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. LLaVA-onevision. Easy visual task trans- fer. Transactions on Machine Learning Research, 2025. URL https: / /oppnreviee net./ forum? (ilidkkkkkkVVV
Xin Li and Dan Roth. Learning question classifiers. In COLING 2002: The 19th International Conference on Computational Linguistics, 2002.
13

---

Published as a conference paper at ICLR 2026
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 84, 2004.
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A Robustly Optimized BERT Pre- training Approach. arXiv:1907.11692 [es], 2019.
Ángela López-Cardona, Carlos Segura, Alexandros Karatzoglou, Sergi Abadal, and Ioannis Ara- pakis. Seeing eye AI: Human alignment via gaze-based response rewards for large language models. In ICLR, 2025.
Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization. In International Con- ference Learning Representations, 2018. URL https: http:////ppeeeeeeeeeeeeeeeee net forum? Ld-Bkg6RicqY7.
Jonathan Malmaud, Roger Levy, and Yevgeni Berzak. Bridging Information-Sseekng Human Gaze and Machine Reading Comprehension. In Proceedings of the 24th Conference on Computational Natural Language Learning, pp. 142-152, 2020. doi: 1/26:///1/222.ooo0--....
Abhijit Mishra, Diptesh Kanojia, Seema Nagar, Kuntal Dey, and Pushpak Bhattacharyya. Leverag- ing cognitive features for sentiment analysis. In Proceedings of the 20th SIGNLL Conference Computational Natural Language Learning, pp. 156-166, 2016.
Bhaskar Mitra, Eric Nalisnick, Nick Craswell, and Rich Caruana. A dual embedding space model for document ranking. arXiv preprint arXiv: 137, 2016.
Marius Mosbach, Maksym Andriushchenko, and Dietrich Klakow. On the stability of fine- tuning BERT: Misconceptions, explanations, and strong baselines. In International Confer- Learning Representations, 2021. URL https:h//penreeeeeeeeeeee nzpLRnVAyah.
net hettfform?id-
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. BLEU: method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Ralph Radach and Alan Kennedy. Theoretical perspectives on eye movements in reading: Past controversies, current issues, and agenda for future research. European journal cognitive psychology, 16(1-2):3-26, 2004.
Ralph Radach, Lynn Huestegge. and Ronan Reilly. The role of global top-down factors in local eye-movement control in reading. Psychological research, 72(6))675-688, 2008.
Alec Radford, Karthik Narasimhan, Tim Salimans. and Ilya Sutskever. Improving Language Understanding by Generative Pre-Training 2019. URL https: openai....p....... com/
senearch-coverss anguage aeeeeuunuuervvv sed/ weed///nggugeeenddresttttoe paper.pdff
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-2392. 2016.
Keith Rayner. Psychological Bulletin Eye Movements Reading and Information Processing: 20 Years of Research. Psychological bulletin, 124(3):372, 1998.
Keith Rayner and Martin H Fischer. Mindless reading revisited: Eye movements during reading and scanning different. Perception psychophysics, 58(5):734-747, 1996.
Keith Rayner and Gary E Raney. Eye movement control in reading and visual search: Effects of word frequency. Psychonomic Bulletin & Review, (12):245-248, 1996.
Keith Rayner, Alexander Pollatsek, Jane Ashby, and Charles Clifton Jr. Psychology of reading. Psychology Press, 2012.
14

---

Published as a conference paper at ICLR 2026
David R Reich, Paul Prasse, Chiara Tschirner, Patrick Haller, Frank Goldhammer, and Lena A Jäger. Inferring Native and Non-Native Human Reading Comprehension and Subjective Text Difficulty from Scanpaths in Reading. In 2022 Symposhum Eye Tracking Research and Applications, pp- 1-8, 2022. doi: 10.1 145/351 .3529639.
David R Reich, Omer Shubi, Lena A Jäger, and Yevgeni Berzak. Eye tracking and NLP. In Pro- ceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 5: Tutorial Abstracts), pp. 2-2, 2025. doi: 10. 118653//120025..ccttuuuom0
Emstt Z Rothkopf and MJ Billington. Goal-guided learning from text: inferring descriptive pro- cessing model from inspection times and eye movements. Journal of educational psychology, 71 (3)3310, 1979.
Elizabeth R Schotter and Brian Dillon. A beginner's ' guide eye tracking for psycholinguistic studies of reading. Behavior Research Methods, 57(2))68, 2025.
Elizabeth R Schotter, Klinton Bicknell, Ian Howard, Roger Levy, and Keith Rayner. Task effects reveal cognitive flexibility responding to frequency and predictability: Evidence from eye move- ments in reading and proofreading. Cognition, 131(1):1-27, 2014.
Omer Shubi and Yevgeni Berzak. Eye movements in information-seekingg reading. In Proceedings of the Annual Meeting of the Cognitive Science Society, 2023.
Omer Shubi, Yoav Meiri, Cfir A Hadar, and Yevgeni Berzak. Fine-grained prediction of reading comprehension from eye movements. In The 2024 Conference on Empirical Methods in Natural Language Processing, 2024.
Omer Shubi, Cfir A Hadar, and Yevgeni Berzak. Decoding reading goals from eye movements. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, pp- 5616-5637, 2025, doi:
1/663////222.acl-oonn
Lina Skerath, Paulina Toborek, Anita Zielińska, Maria Barrett, and Rob Van Der Goot. Native language prediction from gaze: reproducibility study. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), pp- 152-159, 2023.
Ekta Sood, Simon Tannert, Philipp Müller, and Andreas Bulling. Improving natural language pro- cessing tasks with human gaze-guided neural attention. Advances in Neural Information Process- ing Systems, 33:6327-6344 2020.
Alexander Strukelj and Diederick C Niehorster. One page of text: Eye movements during regular and thorough reading, skimming, and spell checking. Journal of Eye Movement Research, 11(1), 2018.
Takenobu Tokunaga, Hitoshi Nishikawa, and Tomoya Iwakura. An eye-tracking study of named entity annotation. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017, pp. 758-764, 2017. doi: 0001661197799944552228
6_097.
Katrin Tomanek, Udo Hahn, Steffen Lohmann, and Jürgen Ziegler. cognitive cost model of anno- tations based on eye-tracking data. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 1158-1167, 2010.
Sowmya Vajjala and Ivana Lučić. OneStopEnglish corpus: A new corpus for automatic readability assessment and text simplification. In Proceedings of the Thirteenth Workshop Innovative Use of NLP for Building Educational Applications, pp. 297-304, 2018. doi: 10. 121188633/v /W18-0533
Justine B Wells, Morten H Christiansen, David S Race, Daniel J Acheson, and Maryellen C Mac- Donald. Experience and sentence processing: Statistical learning and relative clause comprehen- sion. Cognitive psychology, 58(2):250-271 2009.
Sarah J White. Kayleigh L Warrington, Victoria A McGowan, and Kevin B Paterson. Eye move- ments during reading and topic scanning: Effects of word frequency. Journal of Experimental Psychology: Human Perception and Performance, (1(:):333, 2015.
15

---

Published as a conference paper at ICLR 2026
An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. 2024.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evalu- ating text generation with bert. In International Conference on Learning Representations, 2020. URL https: httpp::///ppeeeveeet net/ forum? SceSEE.
16

---

Published as a conference paper at ICLR 2026
APPENDIX
A N-GRAM OVERLAP BETWEEN QUESTIONS AND TEXTS
To assess the extent which the question answering task in OneStop (Berzak al., 2025a) could be solved through simple lexical matching, we computed the n-gram overlap between each target question and its corresponding passage in OneStopQA (Berzak et al., 2020). We measured this over- lap using the proportion of matching unigrams (ROUGE-1*) bigrams (ROUGE-2), and unigrams restricted to content words, with respect to the number of n-grams in the question (precision), in the text (recall), and the harmonic mean of the two (F)). We further provide a breakdown of the question overlap with the passage into the words within and outside the question's 's critical portion of the text. We compared this overlap with the corresponding overlap in SQuAD (Rajpurkar et al., 2016), an extractive QA dataset in which answers appear verbatim in the text. The results can be seen at table 2.
Table 2: Average overlap results for OneStop and SQuAD, comparing questions to different parts of the context. Values in brackets refer to the analysis using only content words from both the question and the context. P (precision), R (recall), and Pi refer overlap proportion with respect question length, context length, and their harmonic mean, respectively.
Dataset
Text Part
ROUGE-1
ROUGE-2
P
R
F
P
R
Fi
SQuAD
Paragraph
0.527 (0.488)
0.071 (0.052)
0.122 (0.091)
0.208
0.021
0.037
Paragraph
0.463 (0.376)
0.059 (0.040)
0.104 (0.071)
0.127
0.012
0.022
OneStop
CS
0.338 (0.285)
0.126 (0.097)
0.176 (0.
0.100
0.033
0.048
out of CS
0.283 (0.153)
0.050 (0.024)
0.084 (0.040)
0.034
0.005
0.008
We found that the unigram overlap between OneStop questions and passages (46%) is lower than the corresponding overlap in SQuAD (53%). We similarly find smaller overlap OneStop for content words (38% OneStop, 49% SQuAD), and for bigrams (12% OneStop, 20% SQuAD). We note that these comparisons are adequate with the ROUGE measures as the OneStop and SQuAD datasets have similar passage lengths on average (1 10 OneStop. 120 SQuAD) and question lengths (10 words on average in both). We further found that the n-gram overlap of the questions in OneStop with the critical span text is relatively low (33% for unigrams, 10% bigrams), and only slightly higher than with the text outside of the critical span (28% unigrams, 3% bigrams). This further suggests that the relevant text does not differ drastically from non-relevant text terms of substring overlap with the question. Overall, we find that the overlap in OneStopQA is relatively low, both in absolute terms and compared to SQuAD, providing empirical support for the inferential nature of the information seeking tasks in OneStop.
B DISCRIMINATIVE MODELS
The figures and descriptions below, depicting model architectures, show the processing of a single candidate question. As mentioned in Section 4, each candidate question is processed independently.
B.1 HALLER RNN
Given a text (a paragraph in our case) T and candidate question q E Qr. the model suggested by Haller et al. (2022) and adapted by us performs the following steps:
Word Embeddings: L.IM- contextualized word embeddings, for T and g, resulting in three subsequences: (1) ZICLS)). (2) [Z ... Lg,. and (3) (Zr.,..... Zr._.
We use ROUGE metrics (Lin, 2004), implemented by the rouge python package, version 1.0.1.
17

---

Published as a conference paper at ICLR 2026
2. Fixation Sequence: Paragraph-only token-level embeddings sum- sum-pooled to word-level embeddings, and ordered by the fixation sequence, including repetitions of words accord- ing the number of fixations that word. Then, each embedding in the sequence is concatenated with the respective fixation features. The fixation-level features are:
Horizontal position of the fixation
Total gaze duration (sum of all fixations on the word; repeated for each fixation on that word)
Duration of first fixation
Duration of outgoing saccade
Horizontal distance of outgoing saccade
Vertical distance of outgoing saccade
Total distance of outgoing saccade
Duration of incoming saccade
Horizontal distance of incoming saccade
Vertical distance of incoming saccade
3. Projection: ŽICLS) and [Zn, Z,,,] are projected to higher dimension to the
mension of the concatenated embeddings from the previous step.
4. RNN Input Sequence: The processed embeddings then concatenated after the other to create the RNN input sequence.
5. Classifier: Finally, a classifier layer scores each candidate question.
Figure 6a depicts the architecture adapted to our task.
B.2 ROBERTEYE--FIXTIOOS
Formally, given a paragraph T and a candidate question q E Qr, the model constructs two parallel representations:
1. Textual Representation (Zw): The input follows the format Zw
[CLS; T; SEP: SEP).
2. Fixation-Level Eye Movement Representation (Zær): The eye movement sequence
defined as ZE, = [Z Ze,ml I. The fixation representation is computed as:
ZE FC(EF;) + Embpo + Emb
Here, E,, captures fixation properties (e.g., duration, position) for fixation fi on word W/, The fully connected layer FC projects this feature vector into the word embedding space, Embpos(w:) (w;) is the positional embedding w; used to map each fixation to corresponding word, and Embeye is learnable embedding marking the presence of eye movement information. The fixation-level features are described in Table 3.
3. Fusion and Prediction:
SEP The combined sequence [Z SEP g; Zw] is processed by the
transformer encoder, where SEP g is a separator token initialized the same as SEP. The final CLS token representation passed through two fully connected layers score each candidate question.
Figure 6b depicts the adapted architecture.
18

---

Published as a conference paper at ICLR 2026
Table 3: Eye movement and word property features used by RoBERTEye-Fixations and DalEye- LLaVA. See Berzak et al. (2025a) for further details.
Feature Name
Deseription
Word-Level Eye Movement Features
JA DWELL_ TIME
The sum of the duration across all fixations that fell in the current interest area spent on the current interest area.
DWELL, TIME_
Percentage
JA FIXATION %
Percentage of all fixations in trial falling in the current interest area.
FIXATION
Total number lixations falling
JA REGRESSION IN COUNT
Number of times interest area was entered from a higher IA_ID.
REGRESSION OUT COLL_COUNT
Number
interest
exited lower IA_ID.
JA RUN_ COUNT
Number of times the Interest Arca was entered and left (runs). first-pass fixation.
FIRSTIST_BEE FIX_ PROGRESSIVE
Checks whether the first fixation
JA FIRST FIXATION DURATION
Duration of the first tixation event that was within the current interest area
FIRST FIXATION. VISITED LEDDDAAACUUUNS
Number distinct interest
visited before the first fixation the current area.
JA FIRST RUN DWELL, TIME
Dwell time of the first run.
FIRST_ LUN_TIXATION COUNT
first of the current IA SKIP 1 if the area received no fixation in first-pass reading. interest until moving higher IA_ID. Exits from current TA to lower IDs before fixating a higher ID).
Number
fixations trial falling
IA SKIP
REGRESSION PATH DURATION
Total fixation duration
REGRESSION OUTTOCCCCO COUNT
SELECTIVE REGRESSION
fixations refixations interest urea until eyes move to a higher ID.
NUTHLDURATION
LAST FIXATION DURATION
event was Dwell time of the last
Duration the
current interest area.
LA RUN DWELL_
LASLSARUN_FNIII R___RUUXNTIOON COUNT
(sequence) within current interest area. Y coordinate of the top of the interest arca.
Number fixations during
TA_
IA
coordinate of left-most part
interest area.
IA FIRST FIX, PROGRESSIVE
Whether the first fixation in the interest area is progressive (left to right, first pass).
nonnalized, Word_
Position paragraph of word interest area, normalized from zero to one. Binary indicator whether the word was fixated on.
total, skip
Puragraph-level Features
FARACRAPH_RT
Reading
entire paragraph.
Fixation-leve Fixation Features
CURRENT FIX INDEX
The position of the current fixation in the trial.
CURRENT __X_____P DURATION
Duration of the current lixation.
CURRENT FIX PUPII
Average pupil size during the current fixation.
CURRENT_FID FIX_
coordinate of the current lixation.
CURRENT FIX_Y
Y coontinate of the current fixation.
Fixation-level Saccade Features
NEXT NEXT_FIX SIN_FFIXANNGLE ANGLE, PREVIOUS FIXLENGLE NEXT/PREVIOUS DISTANCE
Angle between the borizontal and the line from the current the next/presious fixation.
Distance between current fixation nextipresious fixation.
NEXT SAC AMPLITUDE
Amplitude of the following saceade degrees of visual angle.
NEXT SAC, ANGLE
Angle hetween horizontal plane and direction the next saceade.
NEXT SAC AVG. VELOCITY
Average velocity the next saccade.
NEXT SAC_ DURATION
Duration the next saceade in milliseconds.
NEXT SAC PEAK VELOCITY
Peak values of velocity (in visual degrees per second) of the next saccade.
NEXT INTEREST AREA AREA_INDEX
Index interest area
next fixation
Word Properties
spC2 surprisal
Difticulty predicted by GPT-2 model
wordfreg frequency
Frequency word language usage
word Jength line
Number of characters in the word
Indicates
cnd of line
Indicates word is at end of line the word content word (noun, verb, etc.)
content word
left dopendents__comet count
Count of dependents left of word
right, dependents count
Count of dependents right of word
distance_ to bead
Syntactic distance to head word
19

---

Published as a conference paper at ICLR 2026
Scorey
Clessitiar
RNN
Concalt
Scorey
Aej FF
FF
Classifier
Matrix
Matrix
Concet
LM
Weights
FFy. FFf
Orderiby flaations
Concst
7
Word Zn
Porticipant's P Movements over the Text Epyr
Zyros
Zey Aur
Eye Projection
Word Embeddding
q
q
Candidate Question
Participant's Movements over the Text Fpyr
Candidate Question
(a) Haller RNN
(b) RoBERTEye-ixxtions
Figure 6: Visualization of the Haller RNN and RoBERTEye-Fixatonn architectures. LM stands for a language model, RNN for recurrent neural network. FF,, stands for the fixation features and wy. for the word corresponding to the i-th fixation.
20

---

Published as a conference paper at ICLR 2026
C GENERATIVE MODELS
C.I PROMPTS FOR DALEYE-LLAMA, DALEYE-GPT AND GEMINI
In Appendix C.I.I, we present the task prompt and eye-movement input representations used for question generation with DalEye-Llama, DalEye-GPy and Gemini. Variants of these prompts and inputs are described in Appendix G.2.3. The few-shot examples selection method described in Appendix C.1.2.
C.I.1 INPUT FORMAT
***
You will be given data from an eye-tracking for reading experiment which participants first read a question about a paragraph, then read the paragraph, and finally answered the question.
Input: You will be provided with paragraph and eye movements of single participant over that paragraph.
Output: Your task is to generate the question that was presented to the participant prior to reading the paragraph.
Eye Movements Representation:
You will receive the eye movements data for the paragraph formatted as (FORMAT)
Instructions:
Output only the original question presented to the reader, matching it as best as you can. DO NOT include any additional commentary explanation.
[PARAGRAPH]
(SCANPATH) т**
[PARAGRAPH] the textual paragraph, for example: "The quick brown fox jumps..."
(SCANPATH) is the eye movements data which depends on the (FORMAT) depicted below.
C.1 GEMINI FEW-SHOT INPUT
In the few-shot setting, we prepend 10 sampled input-output examples, each pairing eye move- ment trajectory with its corresponding paragraph and question. Examples are selected match the evaluation regime while preventing information leakage: for New Text, the examples come from the same participant but different articles; for New Participant, from the same paragraph but different participants; and for New Participant & Text, from different participants and different articles. The examples precede the test instance in the prompt as shown below.
(SYSTEM MESSAGE)
<Example>
<INPUT>
Paragraph:
(PARAGRAPI_EXMMPLEE EXAMPLE)
Fixations data:
(SCANPATH EXAMPLEX
</INPOT>
21

---

Published as a conference paper at ICLR 2026
<OUTPUT> (EXAMPLE TRUE QUESTION)
</OUTPUT>
</ </Exmmll>>
# X 10 examples
Paragraph:
(PARAGRAPH_TEST)
Fixati exxations_daaal
(SCANPATH_TEST
WISSTEMMMESAAGE is the same input prompt in section C.I.1 (excluding the paragraph and scanpath data).
The four parameters with name format [PARACRAPPISCAAPPRLLEEEAPIIIIIIP matted as the (PARAGRAPH) and (SCANPATH) section C.I.1 for each few-shot example and the test instance, correspondingly. (EXAMPLE TRUE, QUESTION] is the corresponding question for each example.
are for-
C.2 DALEYE-LLAVA
This model is based on the LLaVA-One Vision language-vision model architecture (Li al., 2025). The model input consists of task prompt, the text, and eye movement feature embeddings for the text words. We keep Qwen 2-0.5B (Yang et al., 2024) as the language model backbone, and replace the vision encoder with novel trainable eye movement encoder, which encodes eye movement features using fully connected and convolutional layers. The eye movement embeddings are posi- tionally aligned with the text words. The model is fine-tuned using autoregressive next-token prediction objective for the correct question. Additional details, including a model diagram and eye movement features, provided below. The model input is structured comprising three components: (1) task description, (2) the paragraph text, and (3) the participant's eye movement features, positionally aligned with the paragraph words. The prompt concludes with an <eos> token. The entire sequence is tokenized, and training uses teacher forcing, with cross- entropy loss applied the ground-trui question tokens.
instruction-styy prompt
The forward architecture mirrors LLaVA's design, with three main stages, as depicted in Figure 7:
Text encoding:
The task prompt and paragraph are tokenized and embedded by the back- bone language model.
2. Fixation encoding:
Eye movement features (defined in Table 3) are processed by a ded- icated fixation encoder. The encoder first applies a two-layer MLP with LeakyReLU ac- tivation and dropout over the feature dimension (Linear - LeakyReLU Linear >> LeakyReLUU) and then applies two temporal 1D convolution layers (kernel size 3, stride 1, padding 1), with LeakyReLU after each convolution and dropout between the two convolutions. This yields a sequence of fixation embeddings.
Dropout +
3. Fusion:
Fixation embeddings replace the <i <image> placeholder token in the input se- quence. Word tokens are assigned positional IDs according to their order in the paragraph, while each fixation embedding inherits the positional ID of the corresponding fixated word.
22

---

Published as a conference paper at ICLR 2026
LLM
Zuly m. Zw.. FByy
FT. la
Zig on Zem Zap on Zote
FB FF
Prompt
MLP Conv
Fixation Encoder
FFy FP
T Task Prompt
Text
Participant's P Eye Movements over the Text Eg.r
Figure 7: DalEye-LLaVA architecture. Text tokens and eye movement features fused via fixation encoder and rotary positional embeddings before being passed to the LLM.
23

---

Published as a conference paper at ICLR 2026
D MODEL TRAINING AND HYPERPARAMETERS
We use a stringent evaluation protocol, in which paragraphs are assigned to training, validation, and test sets at the article level, such that all the paragraphs from the same article appear in the same portion of each data split.
D.1 DISCRIMINATIVE MODELS
Since the models we use were developed for different tasks and datasets, we conducted hyperpa- rameter search for each model. The search space for each model is described below. In all cases, it includes the optimal parameters reported in the work that introduced the model, extended to provide a fair comparison between models.
For all neural models, we train with learning rates of {0.00001, 0.00003, 0.0001, 0.0002), following Shubi et al. (2024). Additionally, for all models that make use of word embeddings, we include both frozen and unfrozen language model variants in the search space.
For Haller RNN, we search over LSTM hidden sizes of {10, 40, 70, 140), using one layer and dropout rate of 0.1. The model is trained with and without freezing language model parameters.
For RoBERTEye-Fixation we search over dropout rates of (0.1, 0.3, 0.5) for the eye movement projection layer. The model is trained with and without freezing language model parameters.
We train deep-learning-bas models a maximum of 40 epochs, with early stopping after epochs if no improvement in the validation error is observed. A single training epoch took roughly minutes for EEBERTEyy-Fixationn 10 minutes for Haller RNN, and minutes for DalEye-Llama and DalEye-LLLaV, Each individual Mosbach et al. (2021); Shubi et al. (2024), we use the Adam optimizer (Loshchilov & Hutter, 2018) with batch size of 16. RoBERTEyy-Fixation uses linear warm-up ratio of 0.06 and weight decay of 0.1. We standardize each eye movement feature using statistics computed on the training set, zero mean unit variance.
capped at 24 hours. Following Liu et al. (2019);
Both reading-ting time informed embedding similarity baseline models use word embeddings from the RoBERTa-Large (Liu al., 2019) language model.
D.2 GENERATIVE MODELS
DalEye-Llama This model is fine-tuned using Unsloth with the Sitta-llama-311-80 backbone loaded in 4-bit precision. Fine-tuning uses Low-Rank Adaptation (LoRA)(Hu et al., 2022) with all base model parameters frozen. We train only the LoRA adapters, configured with rank r 16, scaling factor a = 16, and RS-LoRA regularization. LoRA is applied to the transformer projec- tion modules: proj, k_ k_proj, v_proj, up up_proj, down_projj, o_proj. and gate_proj. The model is fine-tuned for 2 epochs with a linear scheduler and warm-up of 10 steps, batch size of 1, gradient accumulation over steps, AdamW-Sha optimizer with learning rate 1 10-4 and weight decay of 0.01.
DalEye-GPT We fine-tune go1-4o-minn--2024-07-1 using the OpenAT API for one epoch, with an automatically selected batch size of 4 and leariin-rrat multiplier of 1.8.
DalEye-LLaVA The LLaVA backbone remains frozen, and training employs LoRA with dropout rate of 0.1, RS-LORA, and rank r 16. The hyperparameter search includes learning rates of x 10-5.3 x 1 5, x 10 10-4, fixation embedding hidden sizes of 512, 1024, and early stopping after 7 epochs without validation improvement.
The discriminative models, DalEye-Llama, and DalEye-LLaVA were trained using the PyTorch Lightning (Falcon & team, 2024) library on NVIDIA A100-40GB and L40S-48GB GPUs.
We utilize the Hugging Face implementation of LLaVA-One Vision,,,,,,, specifically LLaVA LLfLL- --..sionnn-uee22----------- For GPT-4o we use version gp1-40-2024-08-06 For Gemini 3
24

---

Published as a conference paper at ICLR 2026
Srnnintt--pro-pevviie version last updated on November 2025. We Unsloth (version 2025.9.7) (Han et al., 2023) to fine-tune the Llama model. For the QA model we use LIAMF-USPhroberta large-finetuned-race.
There are roughly 355M trainable parameters for RoBERTEye-Fixation reduced to 3M when the RoBERTa backbone is frozen. Haller RNN consists of 357M trainable parameters, or 2.4M when the backbone is frozen. DalEye-LLaVA has 505M parameters, out of which roughly 12M were unfrozen. DalEye-Llama has 8.1B parameters, out of which roughly 42M were unfrozen.
Statistical testing is done with the Julia MixedModels package (Bates et al., 2022).
25

---

Published as a conference paper at ICLR 2026
E MONTY HALL-TYPE SETUP
Table illustrates the structure of the question selection task and the resulting chance accuracies under the three evaluation regimes. the three-way All setting (a), the model selects among three candidate questions, yielding a chance accuracy of 3/9 33.3%. In the Different critical spans evaluation (b), success relaxed to selecting either of the two questions associated with the critical span when it represents the correct span. Here, the probability of correct random choice in- creases to 5/9 55.5%, analogous to a Monty Hall-type setup. Finally, in the Same critical span evaluation (c), the task reduces to a balanced binary choice between the two questions sharing resulting chance accuracy 2/4 50%.
Table 4: Overview of (a) the way question selection task and the breakdown this task into (b) questions with different critical spans and (c) questions with the same critical span. Chance accuracy under the three evaluation regimes: (a) 3/9 (33.3%), (b) 5/9 (55.5%). (c) 2/ (50%).
(a) All
(b) Different critical spans
(c) Same critical span
Predicted
91.cr
92,03
93,02
91,c
92.02
93,c3
f1,cr
Q2,0z
93.02
True
91,e1
X
X
X
42,02
V
/
93,0g
X
X
/
X
/
X
v
26

---

Published as a conference paper at ICLR 2026
F QUESTION RECONSTRUCTION
F.1 QUESTION ANNOTATION INTO UIUC CATEGORIES
This section provides additional information regarding the question annotation into UTUC categories (Li & Roth, 2002). Specifically, in Appendix F.I.1 we include the full prompt given the model, in Appendix F.1.2 examples of questions and their corresponding annotations, and in Appendix F.1.3 agreement with human annotators.
F.1.1 FULL PROMPT
You are an expert at classifying questions into the standard UIUC *main question categories**.
The main categories include:
ABBR: Abbreviation
For example:
What does the abbreviation AIDS stand for?
what is the abbreviation for micro?
What is the abbreviation of the company name 'General Motors'?
What does G.M.T. stand for?
ENTITY: Entity
For example:
What kind animal is Babar?
what killed Bob Marley?
What is fear of weakness?
Where does your hair grow the fastest?
DESCRIPTION: Description
For example:
What do Mormons believe?
What is the history of skateboarding?
what is the difference between generator and alternator?
Where do rocks come from?
MANNER: Manner
For example:
How do get another city's newspaper?
How do you solve "Rubik's Cube"?
How do you look up criminal records the Internet?
How do you find oxidation numbers?
REASON: Reason
For example:
What is the purpose of car bra?
what makes tornado turn?
What causes the redness in your cheeks when you blush?
Why do horseshoes bring luck?
DEFINITION: Definition
For example:
What is a dental root canal?
What is the contents of proposition 98?
Hazmat stands for what?
What does the name Billie mean?
HUMAN: People groups
27

---

Published as a conference paper at ICLR 2026
For example:
Who invented baseball?
What stereo manufacturer is 'Slightly ahead of its time/?
who played the original Charlie's Angels?
what company's logo is 'w in circle?
LOCATION: Geographic locations
For example:
Where is the highest point in Japan?
what European city do Nicois live in?
what is Answers com 's address?
what U.S. state borders Illinois to the north?
NUMERIC: Numbers, quantities, and dates For example:
What is the temperature for baking Peachy oat Muffins?
How many colleges are in Wyoming?
What is the average temperature in the Arctic?
what is the speed of light?
Return an array of tuples the format:
[ (C, "HUMAN"), (1, "DESCRIPTION") (2, "NUMERIC"), (3, "DESCRIPTION"), ]
If unsure, choose the closest matching category.
.u*
F.1.2 CATEGORIZATION EXAMPLES
For each UIUC question category we present an example question sourced from OneStopQA (Berzak et al., 2020):
REASON Why does Myslajek mention Russia, Lithuania and Belarus?
NUMERIC Approximately how many taxi drivers are there in the UK?
MANNER How does Myslajek react to what he sees between the two paw prints?
LOCATION Where was wolf-hunting banned in 1995?
ENTITY Which of the following will featured at Pestival 2013?
HUMAN Who threw the bottle into the Baltic Sea?
DESCRIPTION What does Angella think of the state of the sea today?
F.1.3 AGREEMENT WITH HUMAN ANNOTATORS
To evaluate the quality of the GPT-4o question category classifications, we randomly sampled 100 questions. A human annotator (one of the paper's authors) manually labeled them with a UIUC category. We find an 86% agreement (0.794 Cohen's 's kappa) between the human annotations and the model classifications.
F.2 ARBITRARY AND TEXT-ONLY QUESTION GENERATION
Below is the prompt used for generating questions using the Text-only Llama 3.1, Text-only GPT- 4o-mini, Arbitrary Gemini and Text-only -LaVA-OneVision
Task Description:
Your task is to generate question reader had in mind before reading a given paragraph. The input data is the paragraph itself in standard textual format.
28

---

Published as a conference paper at ICLR 2026
Input Format:
You will receive paragraph in plain text format:
[Paragraph text]
Expected Output:
Generate the exact original question provided to the reader, accurately inferred from the paragraph content. Identify key themes, concepts, or statements in the paragraph that strongly indicate the question that initially motivated the reading.
Instructions:
Your output must precisely match the original question presented to the reader.
Focus specifically on central concepts, themes, or statements within the paragraph.
Produce only the exact original question as your output.
29

---

Published as a conference paper at ICLR 2026
G ADDITIONAL RESULTS
G.1 GOAL SELECTION
G.1.1 TEST AND VALIDATION RESULTS ACROSS SUBTASKS AND EVALUATION REGIMES
The tables below present a breakdown the test and validation results by the All Spans, Differ- ent Spans and Same Spans tasks, and by New Text, New Participant and New Text & Participant evaluation regimes aggregated across 10 cross-validation splits.
Model performance is compared to the Majority class baseline using linear mixed effects model. In R notation: is sccorrect ~ model + (model participant) + (model paragraph). Significant gains over this baseline superscript. The best performing model in each evaluation regime is marked in bold. Significant drops compared the best model are marked in subscript with '+' p 0.05, '++' p 0.01 and "+++' < 0.001.
marked with p < 0.05, **** p 0.01 and "***" p 0.001 in
Table 5: Test accuracy results for the All Spans task by evaluation regime.
Model
New Participant
New Text
New Text & Participant
Chance Majority Baseline
33.2 0.5+++
32.7 + 0.6+++
33.3 0.9++
Question Similarity RT- Weighted Passage
33.3 0.4+++
33.4 0.4+++
34.9 1.4++
RT Similarity Question-Wont Similarities
34.3 0.6+++
34.1 = 0.6
34.2 + 1.2+++
Haller RNN (Haller et al., 2022)
43.3 = 1.1777
40.4 + 0.5;17
40.8 + 1.477"
RoBERTEEye-ixation (Shubi et al., 2024)
49.9 + 0.7***
49.0 + 0.8***
47.8 + 1.5***
DalBye-LLavA
33.2 0.3+++
33.6 0.2+++
34.6 1.1++
DalEye-T.lama
43.1 + 1.6:44
35.4 + 0.4
39.3 + 1.0,4+
Table 6: Test accuracy results for the Different Spans task variation by evaluation regime.
New Participant Different Spans
New Text Different Spans
New Text & Participant Different Spans
Model
Chance Majority Baseline
55.7 = 0.5+++
55.2 0.7+++
53.5 1.5+++
Question Similarity to RT- reighigt Passage
54.9 = 3.7
55.1 3.7
57.8 3.6,++
RT Similarity Question-Wonl Similaritics
54.8 1.4++
54.8 + 1.4+++
55.1 + 1.7+++
Haller RNN (Haller et al., 2022)
66.2 = 0.9717
64.9 + 0.6447
66.4 + 1.2
RoBERTEye-Fixation (Shubi et al., 2024)
70.7 + 0.7**
71.2 + 0.9
69.1 + 1.1
DalEye-LLaVA
55.7 0.5+++
58.1 1.2441
58.7 1.8
MalEye-Llama
65.7 1.4
57.4 1.0,++
59.9 + 0.9
30

---

Published as a conference paper at ICLR 2026
Table 7: Test accuracy results for the Same Spans task variation by evaluation regime.
New Participant Same Span
New Text Same Span
New Text & Participant Same Span
Model
Chance Majority Baseline
50.2 0.6
49.4 + 0.5+++
51.4 1.8
Question Similarity to RT KT-Weighted Passage
49.8 + 0.8+++
50.3 + 0.5+++
49.7 + 2.3++
RT Similarity to Question-Wart Word Similarities
51.0 + 0.5+++
51.0 + 0.7+++
53.1 + 1.4
Haller RNN (Haller al., 2022)
52.6 1.0.++
51.7 0.54.+
50.3 2.0+
RoBERTEyy-Fixation (Shubi et al., 2024)
58.2 1: 0.7***
56.5 1: 0.8***
56.7 +L 1.6
DalBye-LLavA
49.0 = 0.4+++
49.6 + 0.4+++
51.5 + 1.6+
DalEye-Llama
54.2 +: 0.7:47
50.4 + 0.3
55.0 + 1.5
Table 8: Validation accuracy results for each of the tasks.
Model
All Spans
Diff Span
Same Span
Chance Majority Baseline
33.1 + 0.3+++
55.3 + 0.4
50.1 + 0.3
Question Similarity RT- RT-Weighted Passage
33.0 + 0.4++
54.6 + 3.8+++
50.0 0.5+++
RT Similarity to Question-Want Word Similarities
34.1 1: 0.3+++
54.3 1: 1.3+++
51.1 +: 0.5+++
Haller RNN (Haller et al., 2022)
44.2 + 0.3474
66.4 + 0.417
53.3 + 0.411:
RoBERTEyy-Fixations (Shubi al., 2024)
50.9 + 0.3
71.5 + 0.4
58.5 0.5*
DalEye-LLaVA
34.8 1 0.347
56.8 1 0.74++
50.8 + 0.4+++
DalEye-Llama
38.4 1: 0.4
60.5 1: 0.6
51.7 +: 0.4
Table 9: Validation accuracy results for the All Spans task by evaluation regime.
Model
New Participant
New Text
New Text & Participant
Chance Majority Baseline
33.0 + 0.6+++
33.3 0.5+++
31.7 + 1.2++
Question Similarity to RT-Weithted Passage
32.6 + 0.3
33.2 + 0.7+++
34.7 1.6
RT Similarity Question- Word Similarities
34.2 1 0.6+++
34.1 + 0.6+++
34.3 + 1.3++
Haller RNN (Haller al., 2022)
45.7 + 0.8117
42.9 + 0.3777
42.4 + 1.177
RoBERTEyy-Fixations (Shubi et al., 2024)
51.3 1 0.7***
50.6 1. 0.8
50.7 1 1.5
DalEye-LLaVA
34.8 + 0.6
34.3 + 0.3+++
38.5 1.3
Dalaye-Llama
1 0.7:1:
35.8 1 0.4
39.1 1.
31

---

Published as a conference paper at ICLR 2026
Table 10: Validation accuracy results for the Different Spans task variation by evaluation regime.
New Participant Different Spans
New Text Different Spans
New Text & Participant Different Spans
Model
Chance Majority Baseline
55.4 0.6+++
55.8 0.6++
53.2 0.9+++
Question Similarity RT-Weighted Passage
54.4 3.9++
54.6 3.8+++
57.3 +: 3.8+++
RT Similarity to Question- Similarities
54.4 1.2+++
54.3 + 1.4+++
53.8 + 2.3+++
Haller RNN (Haller et al., 2022)
67.8 = 1.0244
65.1 0.8444
65.3 0.72**
RoBERTEyy-Fixations (Shubi et al., 2024)
71.6 0.7
71.6 0.6
70.4 1,9
DalEye-LLavA
56.4 0.8+++
56.6 + 1.1++
60.8 + 1.9,44
DalEye-Llama
62.6 = 0.9747
58.5 + 1.11++
61.2 + 0.9
Table 11: Validation accuracy results for the Same Spans task variation by evaluation regime.
New Participant Same Span
New Text Same Span
New Text & Participant Same Span
Model
Chance Majority Baseline
50.0 1 0.6
50.1 1 0.4.++
50.7 1 2.0++
Question Similarity RT- WTighighted Passage
50.6 = 0.8+++
49.3 - 0.6+++
49.9 + 2.2++
RT Similarity to Question-Want Word Similarities
51.0 1 0.6
50.9 1. 0.6
53.0 1: 1.5+
Haller RNN (Haller et al., 2022)
53.4 + 0.744+
53.2 + 0.4
52.2 1.7+
RoBERTEye-Fixation (Shuhi et al., 2024)
59.7 + 0.5***
57.3 + 0.9
58.6 + 1.9"*
DalVy
50.7 + 0.9+++
51.0 0.3
49.8 + 1.5++
Dalaye-L.lama
52.5 + 0.6
50.9 + 0.4
53.2 1.2+
32

---

Published as a conference paper at ICLR 2026
G.1.2 EVALUATION OF GENERATIVE MODELS ON GOAL SELECTION
For the generative models, goal selection is performed by computing the model-assigned log- likelihood of each candidate question given the input and selecting the question with the highest log-likelihodd. As GPT-4o-ming and Gemini 3 do not allow extracting input token logits, we do not evaluate them on the question selection task.
Table 12: Goal selection test accuracy aggregated over 10 cross-validation splits, with 95% confi- dence intervals. We test for differences between models using a linear mixed effects model with random intercepts and slopes for participants and paragraphs: is correct model + (model parlicipant) + (model paragraph). Model accuracy is compared to the chance performance of the discriminative models. Significant gains over this baseline are marked with **** p < 0.01, and ***** p 0.001. The best performing model in each evaluation regime is marked in bold.
Model Type
Model
All
Different Spans
Same Span
Discriminative
Chance / Majority Baseline
33.0 +: 0.4
55.3 0.4
49.9 +: 0.4
DalEye-LLaVA
33,5 1 0.3
57.0 0.7
49.4 = 0.4
Generative
DalEye-Llama
39.2 0.9
61.4 0.9
52.5 + 0.4
The results are presented in Table 12. DalEye-Llama outperforms DalEye-LLa VA, where the latter is at chance level except for the Different Spans condition. The advantage of the discriminative models over the generative models not surprising, as differently from the discriminative models, the generative models are not explicitly trained on the question selection task.
G.1.3 ROBERTEYL-FIXATUN FEATURE ABLATION
To quantify the contribution of different eye-movement and textual features to the best perform- ing discriminative model, RoBERTEye-Fixatione we conducted a feature ablation study. Here, we repeat the main experiment using the same training procedure and hyperparameter search space, ablating each of three feature groups: fixation-level eye movement features (e.g. current fixation du- ration, next saccade direction), word-level eye movement features (reading measures aggregated for each word, c.g. sum of all fixation durations on the word), and linguistic features of the words (c.g. word frequency, surprisal). This allows us evaluate the contributions of two different eye move- ment representations (fixation-level and word-level) and the importance of enabling interactions of eye movement features with linguistic word characteristics. We further divide the fixation-level features into two groups: fixations and saccades, and evaluate the contribution of each group. Fur- thermore, we also ablate the total paragraph reading time feature. To complement this analysis, we also evaluate two additional representations: one with only linguistic features and another that in- cludes linguistic features and total paragraph reading time feature, a measure that doesn't 't require an eyetracker. Table 3 presents the features of each feature group.
The feature ablation analysis results are reported in Table 13. We find that removing any of the feature groups reduces performance numerically across all evaluation settings except for ablating the text features or the saccade features in the Same Span regime. In the Different Spans evaluation regime, ablating any feature group, except for the word-level eye movement features, results in statistically significant drop in performance. Notably, the drop in performance when removing either total paragraph reading time or text features indicates that the model exploits information beyond eye movement features to differentiate between questions tied to different critical spans. By contrast, in the Same Span evaluation regime, as candidate questions refer to the same critical span, finer-grained information than global reading times or text properties, such as fixation durations, is necessary. The largest and only statistically significant drop in performance is observed when both fixation-level feature groups are ablated jointly, with most of the effect driven by the non-saccade fixation features. Overall, the relatively small effect sizes across ablations and the similar performance of the ablations to the linguistic-featte suggest redundancy among feature groups, and that a substantial portion of the signal is already possibly captured by the fixation sequence via the alignment of fixations to word embeddings.
and linguistic features plus paragraph reading time representations
33

---

Published as a conference paper at ICLR 2026
Table 13: Feature Ablation Results for RoBERTEye-Fixation The best performing model in each evaluation regime is marked in bold. Significant drops compared to the best model are marked with '+' (p 0.05) and '++' (p < 0.01) in subscript.
All
Different Spans
Same Span
RoBERTEye-Fixations
49.2 + 0.4
71.0
57.1 + 0.6
Fixation-Level Features
48.2
7.30
55.9 = 0.8
Fixation Features
48.4 0.5
70.1 0.6
56.4 0.6
Saccade Features
4.6
6..8
57.2 + 0.7
-Word-Level Features
4.6
7..3
56.7 0.7
Linguistic Text Features
48.6
0.88
0.7
Paragraph Reading Time
4..
69.5
5.79
Only Linguistic Text Features
48.6
69.9
5..6
Only Linguistic Text Features Paragraph Reading Time
+: 0.55+
69.7 0.5
5..2
G.2 GOAL RECONSTRUCTION
G.2.1 NUMERICAL RESULTS WITH STATISTICAL TESTS
Tables 14 and 15 present the goal reconstruction evaluations of Figure in numerical format. Pre- sented are question reconstruction evaluations of DalEye-Llama, DalEye-GPP and Gemini (zero- shot and few-shot) for (1) the identity of the generated question word, (2) UIUC semantic category of the question, (3) BLEU score, (4) BERTScore, (5) downstream QA accuracy based on the answer selection of a multiple-choice QA model. We additionally provide evaluations for aalEyeLLLaVA, The models are benchmarked against six baselines that do not include eye movements. Two baselines human-composed questions for different and the are LLM-generated questions: an arbitrary question about the text generated with Gemini 3, and questions from text-only variants of DalEye-Llama, DaIEye-GPT and DalEye-LLaVa VA, finetuned on the question decoding task using only the text.
critical span. Four additional baselines
34

---

Published as a conference paper at ICLR 2026
Table 14: Goal reconstruction evaluations on the Question Word accuracy, Question Category accuracy and BLEU measures. Presented are intercept coefficients with 95% confidence inter- vals from a linear mixed effects model with random intercepts for participants and paragraphs: measure ~ 1 + (1 participant) + (1 paragraph) for eye movement models and the paragraph) the baselines. The highest score in each combination of evaluation measure and evaluation regime is marked in bold. Model comparisons were conducted using measure ~ model + (1 participant) + (1 paragraph). Significant drops compared to the best model are marked in superseript with '*' < 0.05, ***" p 0.01, and '##*' < 0.001. Significant improvements over corresponding baselines are marked subscript with '+" < 0.05,
measure 1
"++' p < 0.01, and '+++' p < 0.001.
New Participant
New Item
New Participant & Item
Question Word accuracy
Incorrect Human (different critical span)
0.416 + 0.022*
0.407 + 0.022"
0.403 0.024** 0.401 + 0.021 ...
Incorrect Human (same critical span)
0.370 + 0.019
0.370 + 0.019
Text-only Llama 3.1 DalEye-Llama
0.591 + 0.014"
0.540 + 0.018
0.340 + 0.020
0.674 + 0.013:71
0.545 + 0.017
0.548 + 0.023
Text-only GPT-4o-mini DalEye-GPT
0.561 + 0.014*t
0.450 + 0.014*th
0.479 + 0.020
0.617 0.014;77
0.495 + 0.015:47
0.496 + 0.023
Arbitrary Gemini 3
0.318 + 0.013"
0.327 + 0.013"
0.328 + 0.019
Gemini zero-shot
0.382 + 0.012:44
0.378 0.013:14
0.393 + 0.022:51
Gemini few-shot
0.713 1 0.012.++
0.395 - 0.013
0.410 1 0.022:
Text-only LLaVA VneVision Dall
0.538 + 0.017
0.499 + 0.019
0.498 + 0.021
0.558 + 0.018444
0.548 + 0.019,++
0.548 + 0.024,++
Question Category accuracy
Incorrect Human (different critical span)
0.410 0.021
0.410 + 0.021
0.423 0.024" 0.357 + 0.021 ***
Incorrect Human (same critical span)
0.349 + 0.019*
0.349 + 0.019
Text-only Llama DalBye-Llama
0.535 + 0.014
0.454 + 0.017
0.453 + 0.021 0.428 + 0.023
0.558 + 0.014,++
0.450 + 0.017
Text-only GPT-4o-mini DaIEye-GPT
0.458 + 0.013***
0.430 + 0.014
0.429 + 0.020**
0.550 1 0.013+++
0.488 1 0.015
0.454 0.022
Arbitrary Gemini
0.345 + 0.013***
0.367 + 0.014"
0.355 + 0.019*^^
Gemini zero-shot
0.327 + 0.012
0.335 - 0.012
0.323 + 0.021 -.y
Gemini few-shot
0.526 + 0.014;74
+ 0 0.013:54
0.439 + 0.022
Text-only LLaVA OneVisson DalEye-LLaVA
0.408 + 0.013***
0.424 + 0.018
0.412 + 0.021 0.460 + 0.024..4
0.445 + 0.015
- 0.466 0.019
BLEU
Incorrect Human (different critical span)
0.219 + 0.006
0.219 + 0.006
0,219 + 0.007 0.300 + 0.009
Incorrect Human (same critical span)
0.298 + 0.
0.298 + 0.009
Text-only Llama DalEye-Llama
0.454 + 0.
0.236 + 0.004
0.238 0.005 T.
0.556 + 0.008:15
0.253 + 0.005:11
0.244 + 0.000
Text-only GPT-4o-mini DalEye-GPT
0.452 + 0.006***
0.240 + 0.004
0.237 + 0.006
0.514 + 0.008
0.257 + 0.00
0.256 + 0.007:5!
Arbitrary Gemini
0.191 + 0.004""*
0.194 + 0.005
0.184 + 0.006
Gemini zero-shot
0.246 + 0.005:2:
0.247 + 0.005
0.242 + 0.009224
Gemini few-shot
0.644 + 0.009...
0.272 4 0.005.::
0.268 1 0.008:11
Text-only LLaVA VneVision
0.304 + 0.
0.210 + 00
0.207 +: 0.005
DalBye-LLaVA
0.265 + 0.005
0.231 + 0.006:11
0.231 + 0.007:11
35

---

Published as a conference paper at ICLR 2026
Table 15: Goal reconstruction evaluations on the BERTScore and QA accuracy measures. Pre- sented are intercept coefficients with 95% confidence intervals from linear mixed effects model with random intercepts for participants and paragraphs: measure + (1 participant) paragraph) for eye movement models and the measure + paragraph) for the baselines. The highest score in each combination of evaluation measure and evaluation regime is marked in bold. Model comparisons were conducted using measure model + (1 participant) + (1 paragraph). Sigmificant drops compared to the best model are marked in superscript with *** p 0.05, "**" p < 0.01, and "ake* p 0.001. Significant improvements over corresponding base- lines are marked in subscript with '+' < 0.05, '++' < 0.01, and '+++' < 0.001.
New Participant
New Item
New Participant Item
BERTScore
Incorrect Human (different critical span)
0.604 + 0.004""
0.603 + 0.004"M
0.602 + 0.004
Incorrect Human (same critical span)
0.651 + 0.004"
0.651 0.004
0.651 + 0.004
Text-only Llama 3.1
0.723 + 0.003***
0.617 + 0.003
0.618 + 0.003
DalEye-Llama -lama
0.776 + 0.004:7:
0.631 + 0.003:1:
0.626 + 0.004
Text-only GPT-4o-mini Dallye-GPT
0.722 + 0*
0.619 + 0.003
0,618 : 0.004
0.755 + 0.004777
0.630 + 0.003717
0.627 + 0.004:7
Arbitrary Gemini
0.611 + 0.
0.612 + 0.003
0.605 + 0.003
Gemini zero-shot
0.628 + 0.003;74
0.629 + 0.003:14
0.627 + 0.005::4
Gemini few-shol
0.818 + 0.005.++
0.642 1 0.003:14
0.644 + 0.005::
Text-only LLaVA OneVision DalEye-LLaVA
0.647 + 0.003
0.595 + 0.003
0.591 + 0
0.638 + 0*
0.618 : 0.003:14
0.617 + 0.004::1
QA accuracy
Incorrect Human (different critical span)
0.495 + 0.018
0.490 + 0.017
0.490 + 0.020 0.686 + 0.022
Incorrect Human (same critical span)
0.677 + 0.
0.677 + 0.021
Text-only Llama 3.1 DalEye-Llama
0.684 + 0.012
0.609 + 0.014
0.616 + 0.020
0.763 + 0.011
0.648 + 0.01
0.644 + 0.023;"
Text-only GPT-4o-mini DaIEye-GPT
0.679 + 0.012***
0.619 + 0.013**>
0.608 + 0.020***
0.738 + 0.01
0.658 + 0.013:11
0.644 + 0.023
Arbitrary Gemini
0.635 + 0.015*tA
0.636 + 0.015
0.614 + 0.020
Gemini zero-shot
0,674 + 0.013;77
0.664 + 0.013
0.690 1 0.022
Gemini few-shot
0.804 + 0.010...
0.683 + 0.012...
0.671 + 0.023...
Text-only LLaVA VneVision DalEye-LLaVA
0.632 + 0.012
0.636 + 0
0.612 + 0.01g
0.623 + 0.013
0.610 0.016
0.579 4 0.021
36

---

Published as a conference paper at ICLR 2026
G.2.2 EXAMPLE GENERATIONS
Example generations for DalEye-Llama DaIEye-GPT, Gemini zero-shot, and Gemini few-shot presented in Table 16. The complete set of model outputs for all three evaluation regimes, includ- ing the baselines, together with the corresponding annotations, released in the project's GitHub cettrratedgqeetttteeeee quest iones.ioo..... csv.
repository under generated
37

---

Published as a conference paper at ICLR 2026
Table 16: An example showing a paragraph, the corresponding ground truth question, and, for DalEye-Llama, Gemini (zero-shot and few-shot), and DalEye-GP four generated questions from the New Text Regime and four from the New Participant Regime. Each generated question reflects a different outcome from the QA model, as indicated by the answer choice (A-D) selected by the model. "(no example)" indicates that the model did not generate a question for the corresponding answer choice. The possible answers, structured according to the STARC annotation guidelines (Berzak et al., 2020), to the ground truth question were: A) (correct answer) A large number of cultural destinations. B) (miscomprehhensin of the critical span) Beaches constantly full of locals and tourists. C) (incorrect, related to a different span) 3,000 luxury hotels. D) (no support in the passage) Highly-regarded restaurants.
Paragraph: Lemmos has wild beaches, where you can swim and sumbathe almost alone, a small nightlife scene and many cultural sites. Lemnos is the eighth largest island in Greece it will have to pay the first round of increases autumnn 2015. But Lemnos far less wealthy than many smaller islands. It has just over 3, 3,000 beds for visitors Rhodex, for example, has tens of thousands of beds. "We have been suffering economically in recent years and now we will suffer more." said Lemnos Mayor, Dimitris Marinakis.
True Question: What is among the primary attractions of Lemnos?
Model
New Participant Generated Questions
New Text Generated Questions
(A) What is among the primary attractions of Lemnos?
(A) What is one thing note about the eponymous village?
What will happen spring 2013?
(B) Who is quoted as saying that hotels benefit from the increase the number of tourists in the area?
DalBye-Llama
(C) Why does Lemnos have to pay the first round of tax increases?
(C) What does the quote convey about the current economic situation in Greece?
(D) What is one reason why the wealthy tend to spend more time on the islands?
(D) What is true of the other Greek islands?
(A) What can you do on the wild beaches of Lemnos?
(A) What does Lemnos have?
Gemini Zero-Shot
(B) Where can you swim and sunbathe al- most alone?
(B) What kind of nightlife does Lemnos have?
(C) Why is Lemnos in line for the first round of increases?
(C) When will Lemnos have to pay the first round of tax increases?
(D) (no example)
(D) Are the beaches on Lemnos crowded in the summer?
(A) What is among the primary attractions of Lemnos?
(A) What does Lemnos have dozens of?
Gemini Few-Shot
B (no example)
B (no example)
(C) (no example)
(C) Why is Lemnos in line for the first round of increases?
(D) Why does Lemnos have to pay the first round of tax increases?
(D) Why will Lemnos have pay the first round of tax increases?
(A) What is among the primary attractions of Lemnos?
(A) What does Lemnos have dozens of?
B (no example)
What is true of Rhodes?
DalEye-GPT
(C) Why does Lemnos have to pay the first round of tax increases?
(C) What does Dimitris Marinakis say about the current economic situation on Lemnos?
(D) Why does Lemnos have pay the first
(D) What is one thing Lemnos' councilors say they will do?
round of tax increases?
38

---

Published as a conference paper at ICLR 2026
G.2.3 DALEYE-LLAMA PROMPT AND EYE MOVEMENT REPRESENTATION VARIATIONS
We experiment with three input representations that capture different granularities of eye movement information: (i) Fixation-level, (ii) Word-level, and (iii) a combination of both fixation- and word- level information, detailed below:
Fixations:
Fixation Duration + Next saccade direction
list of fixation-level features: [fixated word index, fixated word, fixation duration in ma, direction of next saccade (backward to earlier word within word forward to later word)
Format:
Example: [[4, "fox", 220, backward), ....
2. Words: Total Fixation Duration Incoming/Outgoing Backward/Forward Saccades
Format:
a list of word-level features: [word index, word, total fixation duration in ms, incoming forward saccades (from earlier word), incoming backward saccades (from later word), outgoing forward saccades (to later word), outçoing backward saccades (to earlier word)
Example: [[4, "fox", 320, 2, 1, 3, 0), ...]
3. Words+Fixations
two lists of word-level and fixation-level [fixated word index, fixated word, fixation duration in ms, direction of next saccade (backward / within / forward) [word index, word, total fixation duration in ms, incoming forward saccades, incoming backward saccades, outgoing forward saccades, outgoing backward saccades]
Format:
features:
Example: [[4, "fox", 220, backward), ...] [[4, "fox", 320, 2, 1, 3, 0], ...]
We further experiment with alternative prompt that de-emphasizes the experimental setup and places more focus on the generation task itself:
Task Description:
Your task is to generate the exact original question reader had in mind before reading a given paragraph. The input data is (a/two) time series composed of and lixation-level)
features.
Input Format: You will receive the paragraph and eye movement data formatted as (FORMAT)
Expected Output: Generate the exact original question provided to the reader, accurately inferred from the fixation patterns.
Instructions:
Your output must precisely match the original question presented to the reader. Produce only the exact original question your output.
(PARAGRAPH)
(SCANPATH)
39

---

Published as a conference paper at ICLR 2026
In Table 17 present the question selection results for DalEye-Llama across the two proposed task prompts and the three input representations. Across all three task variations, Main task prompt with fixation-level input yields the best accuracy. Both Alternative task prompt and the word- level/combined inputs underperform this setting, where the choice of representation matters more than the task prompt. In Table 18 present goal reconstruction results that similarly suggest that the alternative prompt and eye movements representations for DalEye-Llama weaken its perfor- mance in most cases.
Table 17: Test accuracy results for each of the tasks and for each of the input variations.
Task Prompt
Representation
All Spans
Diff Span
Same Span
Chance Majority Baseline
None
33.0 + 0.4+++
55.3 + 0.4++
49.9 + 0.4,++
Fixation-level
39.2 0
0.9
52.5 0.4
Main task prompt
Word-level
33.9 = 0.54++
55.5 + 0.6
50.4 + 0.3++
Both fixation- and word-level
34.7 + 0.6
56.2 + 0.8
50.5 + 0.4,+
Fixation-level
37.2 + 0.611
60.3 1 0.6
51.0 + 0.3,
Alternative task prompt
Word-level
33.4 - 0.2
55.0 + 0.6
50.2 + 0.2,++
Both fixation- and word-level
34.4 + 0.7
55.5 + 1.0,e+
50.5 + 0.5,,
40

---

Published as a conference paper at ICLR 2026
Table 18: Test evaluations of the DalEye-Llama model with the fixation-level, the word-level, and combined word- and fixation-level input representations, on two task prompt variants.
New Participant
New Item
New Participant & Item
Question Word accuracy
Fixation-level
0.674 0.013
0.545 + 0.017
0.548 = 0.023
Main task prompt
Word-level
0.185 + 0.019
0.128 + 0.014
0.125 = -.0016
Both fixation- and word-level
0.191 0.020
0.130 0.014
0.134 0.016
Fixation-level
0.635 0.014
0.534 1 0.017
0.347 1 0.023
Alternatvve task prompt
Word-level
0. + 0.018
0.129 0.014
0.133 0.016
Both fixation- and word-level
0.134 = 0.017
0,091 0.011
0.099 = 0.015
Question Category accuracy
Fixation-level
0.558 0.014
0.450 0.017
0.428 0.023
Main task prompt
Word-level
0.448 0.015
0.432 1 0.015
0.438 0.022
Both fixation- and word-level
0.286 0.015
0.257 + 0.015
0.259 0.020
Fixation-level
0.550 0.015
0.484
0.473 0.024
Alternative task prompt
Word-level
0.537 0.018
0.538 0.019
0.529 0.024
Both fixation- and word-level
0.515 0.018
0.507 0.018
0.498 0.024
BLEU
Fixation-level
0.556 = 0.008
0.253 + 0.005
0,244 = 0.006
Main task prompt
Word-level
0.140 0.014
0.089
1 0.005
0.085 1 0.006
Both fixation- and word-level
0.146 0.015
0.005 0.088
0.085 0.006
Fixation-level
0.493 0.009
0.264
0.004
0.262 0. 007
Alternatvve task prompt
Word-level
0.126 = 0.013
0.077
0.005
0.073 = 0.006
Both fixation- and word-level
0.096 + 0.012
0,065 0.004
0.063 0.005
BERTScore
Fixation-level
0.776 0.004
0.631
0.003
0.626 0.004
Main task prompt
Word-level
0.429 0.012
0.410
0.007
0.408 0.009
Both fixation- and word-level
0.426 0.013
0.406
0.007
0.402 = 0.010
Fixation-level
0.746 0.005
0.633
0.003
0.632 0.004
Alternative task prompt
Word-level
U.410 0.013
0.388
0.007
0.384 0.009
Both fixation- and word-level
0.409 0.010
0.393 0.005
0.390 0.006
QA accuracy
Fixation-level
0.763 = 0.011
0.648 + 0.014
0.644 = 0.023
Main task prompt
Word-level
0.600 0.014
0.569 1 0.015
0.557 0.024
Both fixation- and word-level
0.619 0.014
0.595 + 0.015
0.582 = 0.024
Fixation-level
0.717 + 0.012
0.646 + 0.014
0.670 + 0.022
Alterattvv task prompt
Word-level
0.577 + 0.015
0.560 0.015
0.537 = 0.023
Both fixation- and word-level
0.565 1 0.016
0.541 1 1.015
0.538 0.024
41


Reviews:
[{"review_id": "MXrzQgbYbR", "reviewer": "Reviewer_rzyv", "summary": "The paper introduces a cognitive-state decoding problem of recovering information-seeking goals (text questions) from a reader’s eye movements while reading a piece of text. The authors propose different task formulations for recovering the stimulus – (1) question selection and (2) question generation. They develop discriminative and generative models for the above tasks, that recover the question given the text and eye movement data. Results shown on the OneStop eye-tracking data showing discriminative models (RoBERTEye-Fixations) perform better than random while generative models (DalEye-Llama) shows promising performance on less challenging cases.", "questions": "NA", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The dataset (OneStop) and problem setup is well suited to the objective of recovering reading goals. The evaluation regimes which include splitting data by new participant and new text is well conceived and creation of two tiers of difficulty are useful in comparing model performance in challenging settings.\n\nThe authors experiment with different types of baselines – heuristics, discriminative models based on adaptions of prior work and generative LLM models (DalEye-LLaVA, DalEye-Llama).", "weaknesses": "1. I would like to see what types of gaze features (eg: fixation durations, word revisits) are more useful for recovering the information seeking goals. Stronger experiments are required to investigate the feature attributions by gradually phasing out these features one by one from the eye movements data to train the models.\n\n2. It is not clear in the paper if the question and the text span containing the corresponding answer have significant substring overlap. If so, the problem becomes more trivial where users just have to look for specific strings from the question in the text. A more realistic and challenging case to evaluate would be if the user would have to understand and infer the meaning from the passage if the language is phrase differently.\n\n3. The authors should more comprehensively discuss about relevant literature in the field of eye movements conditioned on information seeking goals, for instance\n- Synthesizing Human Gaze Feedback for Improved NLP Performance (EACL 2023) which generates gaze patterns conditioned on the reader’s intent / task\n- GazeXplain: Learning to Predict Natural Language Explanations of Visual Scanpaths (ECCV 2024) which predicts scanpaths during performing visual question answering tasks or when instructed to search for a particular object in the image\n\n\nQuestions / Suggestions for Improvement:\n- Show n-gram overlap between target question and corresponding text span and correlate with model correctness.\n- More thorough experiments to analyse eye feature importances", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cA7VKQejYz", "reviewer": "Reviewer_4c7d", "summary": "This paper presents a novel and important investigation into decoding open-ended, text-specific information goals from eye movements during reading. The work is rigorous, introducing a clear task framework (goal selection and reconstruction) and systematically evaluating a range of discriminative and generative models. The experiments are comprehensive, using a large-scale, high-quality dataset (OneStop) and thoughtfully evaluating model generalization. While the generative task remains highly challenging, the strong performance of the best discriminative model, especially on the difficult \"same critical span\" task, provides compelling evidence that eye movements contain fine-grained information about a reader's goal. This work opens promising new directions for both scientific inquiry and practical applications.", "questions": "1.In the RoBERTEye-Fixations model, which achieved the best results, did you perform any ablation studies to understand which specific eye-movement features (e.g., fixation duration, saccade amplitude, regressions) were most critical for its performance, particularly on the challenging \"Same Span\" task?\n\n2.For the generative DalEye-Llama model, performance drops most significantly in the \"New Text\" regime. Do you attribute this primarily to the model's difficulty in comprehending the new text content itself, or in associating the novel eye-movement patterns on that text with the question generation process?\n\n3.The paper mentions promising applications in education and assistive technology. Given the current accuracy levels (e.g., ~49% for 3-way selection), what do you see as the minimum performance threshold for such systems to be reliably deployable in real-world scenarios? What are the key next steps to bridge this gap?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1.This is the first work to systematically address the decoding of arbitrary, text-specific information goals from eye movements. It moves beyond previous work that classified pre-defined procedural reading tasks (e.g., reading vs. skim-reading) to a more challenging and practically relevant semantic decoding task. The contribution is significant and opens a new research direction.\n\n2.The experimental design is exemplary. The data splits are carefully constructed to evaluate generalization to \"New Participants,\" \"New Texts,\" and \"New Text & Participant,\" providing a clear and honest assessment of model robustness. The subdivision of the selection task into \"Different\" and \"Same\" critical spans offers nuanced insight into task difficulty.\n\n3 The paper provides a thorough benchmark, covering simple heuristics, adapted state-of-the-art discriminative models, and pioneering generative LLM-based approaches. This offers a valuable overview of the landscape for this new task.\n\n4. The evaluation methodology is a strength. Beyond selection accuracy, the authors propose a robust set of metrics for the generative task, including question word/category agreement, BERTScore, and a creative downstream QA accuracy metric, which convincingly demonstrates the utility of the generated questions.\n\n5. The paper is accompanied by a code repository and includes extensive details in the main text and appendices regarding model architectures, hyperparameters, and training procedures, ensuring the work is easily reproducible.", "weaknesses": "1. As the results show, the generative task is exceptionally difficult, and model performance, especially on new texts, is still limited. The generated questions are not yet on par with human-composed ones.\n\n2.The paper successfully demonstrates that goals can be decoded, but offers less insight into how or why the models make their decisions from a cognitive perspective. The models remain somewhat black-box.\n\n\n3. While generalization is tested, the significant performance drop in the \"New Text & Participant\" regime indicates that model robustness is still limited to the distribution of the training data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eh3ZfzCWb1", "reviewer": "Reviewer_H8io", "summary": "This paper tackles the task of decoding open-ended reading goals from eye movements. Using the OneStop dataset, the authors propose two tasks-target selection and target reconstruction—and design multimodal models combining gaze and text features. Results show that gaze–text integration slightly improves performance, indicating that eye movements encode semantic goal information. This paper provides a new research paradigm for the combination of eye movement and language models, but further improvement is needed in terms of interpretability, cross-text generalization, and comparison with more powerful baselines. Overall, the paper is interesting, but its current form cannot totally support all authors' claims.", "questions": "1. What causes DalEye-Llama's performance to drop dramatically on new text? Is it due to limitations in feature transfer or model overfitting?\n2. The paper mentions \"inherent noise\" in eye tracking data but does not discuss mitigation strategies. Have you tested noise reduction or fixation filtering to improve model stability?\n3. The generation model is only compared to GPT-4o in zero-shot mode. Could results with a fine-tuned GPT-4o (using the same text and gaze inputs) clarify advantages in efficiency or robustness?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The authors claim that they are the first study to decode open-ended, text-specific reading goals from eye movements, framed as dual tasks of selection and reconstruction.\n- Integrating text and gaze features markedly improves target selection; RoBERTEye-Fixations achieves 49.3% accuracy vs. 33% baseline.\n- DalEye-Llama attains 76.3% QA accuracy on unseen participants (vs. 68.1% for human distractors), validating the gaze–goal correspondence.", "weaknesses": "1. Lacks cognitive interpretation of gaze behavior and its link to goal decoding.\n2. Sharp performance drop on unseen texts (Kappa 0.478 → 0.069) unexplained.\n3. No comparison with fine-tuned multimodal LLMs (e.g., GPT-4o, LLaVA-1.5).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)