L’Impact de la complexité textuelle sur le comportement de lecture : une analyse oculométrique et de la surprise des textes français (2025)

Abstract: L’Impact de la complexité textuelle sur le comportement de lecture : une analyse oculométrique et de la surprise des textes français Cette étude examine comment la complexité du texte affecte les processus de lecture à travers différents types de textes en combinant la méthodologie d’oculométrie avec l’analyse de la surprise. Nous avons créé un corpus en français avec des textes généraux, cliniques et médicaux, dans leurs versions originales et simplifiées, annotés avec des mesures oculométriques complètes provenant de 23 participants. La modélisation linéaire à effets mixtes révèle que la surprise prédit significativement les temps de lecture pour tous les types de textes, les textes médicaux montrant une sensibilité accrue aux mots inattendus. De façon importante, la simplification a des effets différentiels selon le type de texte : bien qu’elle ne réduit pas significativement les temps de lecture pour les textes cliniques, elle diminue considérablement les temps de lecture pour les textes médicaux. De plus, la simplification atténue l’effet de la surprise spécifiquement dans les textes médicaux, réduisant le coût cognitif associé au traitement des mots inattendus.

## dataset and experimental setup for this study, we constructed a novel dataset of french texts that spans both general and medical domains. the dataset was built by leveraging excerpts from two corpora, clear @xcite and cas @xcite , which include a variety of source materials such as wikipedia articles 1 , literature reviews, drug leeflets, and clinical cases. our dataset is composed of three main categories of texts : the texts are divided into two complementary sets a and b. if a given text appears in its original form in one set, its simplified version is included in the other set. thus, each participant reads only one version of any text, preventing potential bias from familiarity with both versions of one text. texts are presented to participants paragraph by paragraph to accommodate the presentation screen. to maintain participant attention, some screens include comprehension questions. french sentence english translation original : les hémocultures ont permis d'isoler un staphylococcus aureus. original : blood cultures made it possible to isolate a staphylococcus aureus. simplified : les hémocultures (analyses des bactéries éventuelles dans le sang) ont montré la présence de la bactérie staphylococcus aureus. simplified : blood cultures (tests for possible bacteria in the blood) showed the presence of the staphylococcus aureus bacterium. ## methodology our methodology relies on the notion of surprisal, an information-theoretic measure that quantifies how unexpected a word is given its preceding context @xcite . mathematically, the surprisal of a word w given its context c is defined as : surprisal(w | c) =log 2 p (w | c), where p (w | c) is the conditional probability of w based on the context c. this can also be understood as the logarithm of the ratio between the prefix probabilities computed before and after a word is processed. higher surprisal values indicate that a word is less predictable, which is hypothesized to correlate with increased processing difficulty during reading. to estimate these probabilities for french texts, we employ a pre-trained french gpt-2 model gpt-fr-cased-base foot_0 . this model was specifically trained on a large and diverse corpus of french text (1,017b parameters), making it well-suited for capturing french linguistic patterns and probabilities. as an autoregressive language model, it naturally produces the conditional probabilities needed for surprisal calculations. although newer language models are available, gpt-2 remains a standard choice for surprisal estimation in psycholinguistic research due to its computational tractability and established validation against human reading data in prior studies. this model uses subword tokenization (e.g., byte-pair encoding) to decompose words into smaller units. even if this tokenization is not linguistically grounded, it enables to handle rare or morphologically complex words more effectively. then, we merge surprisal values computed for subtokens to get a single surprisal value per word. specifically, if a word is split into subtokens t i , t i+1 , . . . , t j , we sum the surprisal values for those subtokens to produce a single word-level surprisal. this approach sums surprisal values (measured in bits aslog 2 (p )), not probabilities themselves, which ensures mathematical validity since surprisal values are additive by definition. these word-level surprisal measures are aligned with eye-tracking data to explore how surprisal predictability affects reading behaviour observed with eye-tracking. for instance, we examine whether words with higher surprisal (i.e., lower predicted probability) are associated with eye-tracking indicators suggesting greater processing difficulty, like longer fixation durations or increased regression counts. in subsequent statistical analyses, correlations using spearman methods @xcite are computed between surprisal values and eye-tracking measures, while controlling for factors such as text version (original vs. simplified) and text genre differences. we also employ lmems (linear mixed effects models) to analyze how surprisal, text type, and simplification influence reading times while controlling for participant-specific variation. lmems are statistical models that account for both fixed effects (e.g., experimental manipulations) and random effects (e.g., individual variability) simultaneously. specifically, we model reading measures (e.g., first-pass duration, total fixation times) as a function of surprisal (centered), text type (medical vs. clinical), and version (original vs. simplified), including all possible interactions as fixed effects. ## results and their discussion we present the results across three lines : global descriptive statistics, statistical correlation analysis between surprisal and eye-tracking measures, and lmem analysis. descriptive statistics. tables 3 and 4 in the appendix present the descriptive statistics for six eye-tracking measures across different text types (clinical, general, and medical) in both original and simplified versions. overall, the data reveals several notable patterns. (1) there are evident processing differences between original and simplified texts : in most cases, simplified versions show reduced reading times across multiple measures compared to their original counterparts, suggesting improved readability, as has been observed in a previous study with experimental data comparable to ours @xcite . (2) text type variations are also observed, with clinical texts generally exhibiting longer fixation durations and more fixations than general texts, indicating higher processing demands. ## conclusion and future work this study examined how text simplification affects reading processes across different text types through a combination of eye-tracking methodology and surprisal analysis, which quantifies how unexpected a word is given its preceding context. we created a corpus of french general, clinical, and medical texts in both original and simplified versions, annotated with comprehensive eye-tracking measurements from 23 participants. this corpus will be made publicly available, providing a resource for researchers investigating reading processes, and domain-specific language processing. our analysis across multiple eye-tracking measures revealed that surprisal significantly predicts reading behaviour in all text types, with medical texts showing heightened sensitivity to unexpected words. we found that simplification benefits are not uniform across text types. while simplification had minimal effects on clinical texts, it substantially reduced reading times for medical texts and mitigated the impact of surprisal specifically in these texts. this differential effect was consistent across multiple eye-tracking measures, particularly in those capturing overall reading behaviour rather than just initial word recognition. we plan to expand the corpus by increasing both the number of annotated texts and the number of participants. particularly important will be the inclusion of pathology speech students as an additional participant group, which will allow to examine how domain expertise modulates the effects of surprisal and simplification. this comparison could reveal whether experts process specialized terminology differently and whether simplification benefits vary with expertise level. we note that simplified texts in our corpus tend to be longer and contain a higher proportion of content words, which could influence surprisal estimates. we plan to apply post-hoc pos-based filtering of function versus content words in future work to isolate this effect. we also intend to employ multiple language models for surprisal calculation to ensure robustness and to investigate whether different models capture different aspects of predictability in specialized texts. furthermore, future analyses will explore additional psycholinguistic phenomena (such as spillover effects @xcite , entropy @xcite , perplexity @xcite , etc.). these more detailed analyses may reveal subtler aspects of how predictability influences reading across different text types and simplification versions.