An Analysis of Reader Engagement in Literary Fiction through Eye Tracking and Linguistic Features (2023)

Abstract: Capturing readers’ engagement in fiction is a challenging but important aspect of narrative understanding. In this study, we collected 23 readers’ reactions to 2 short stories through eye tracking, sentence-level annotations, and an overall engagement scale survey. We analyzed the significance of various qualities of the text in predicting how engaging a reader is likely to find it. As enjoyment of fiction is highly contextual, we also investigated individual differences in our data. Furthering our understanding of what captivates readers in fiction will help better inform models used in creative narrative generation and collaborative writing tools.

## introduction the question of reader engagement in fiction has been studied in the psychology field for decades, with some of the foundational theoretical work from @xcite on transportation theory paving the way for more recent theoretical frameworks and experimental setups, notably the work by melanie c. @xcite and @xcite . however, as @xcite emphasized in his article on the neurocognitive science of literary reading, the samples normally collected are small and not enough to compensate for individual differences in reading patterns due to reader context and other situational factors. in order to help close the experimental gap, one contribution of this study is to provide a data set of reader reactions to natural stories, which jacobs refers to as "hot" experimental research. this data, along with the extraction of linguistic features, allows us to test theories around reader engagement and discover which textual qualities have the most impact. in our study, we have the following research questions: with respect to rq1, our findings indicated that negatively-valenced, concrete sentences had higher dwell times. no relationship was found between the highlights and dwell times. this may be due to the fact that the highlighting data is sparse. for rq2, we found that features such as valence, sentiment, and emotion were significant across readers, although the reader context accounted for much of the variance in highlighting annotations. regarding rq3, there was a high amount of variance between readers for dwell time. however, once dwell times were individually scaled, we could see some consistency in their patterns, particularly when looking only at highly engaged readers. for future studies, a modified highlighting exercise in which participants must select a category for each sentence -including none -could result in less sparse annotation data. a more complete annotation of the story text would allow us to explore the connection between dwell time and different modes of engagement. as new methods are created for representing complex features of stories, such as character relationships and story tension, data sets like ours can be used to find more meaningful relationships between the story text and how engaging it is. ## related work in his model for the neurocognitive poetics of literary reading, @xcite proposed two modes of reading: one fast track -"immersion" and one slow -"aesthetic trajectory". the former is proposed to be brought on by things like familiarity, suspense, sympathy, and vicarious hope; whereas the latter is a more reflective and connected mode brought on by aesthetic appreciation, more complex emotion, and unfamiliar situations. we used this framework to inform what variables we expected to have an impact on dwell time. @xcite conducted a series of studies to narrow down the salient aspects of reader engagement and created a general media engagement scale. the aspects they defined are narrative understanding, attentional focus, emotional engagement, and narrative presence, and the scale they created include questions related to those aspects. we adapted this scale for written narrative to gauge overall interest in the stories used in our study. in addition, in order to obtain more granular information, we used these aspects to design an annotation task that would provide sentence-level feedback. using visualizations and linear mixed effect models, we explored textual features that had an impact on engagement and dwell time across readers. there have been several other eye tracking as well as fmri studies in the area of reader engagement (a few are shown in ## methods participant study design the study asked 31 english speakers (17 female, 11 male, 3 other, average age: 26) to read two short stories by anton chekhov foot_1 while their eyes were tracked, and then answer an engagement scale survey: • i found my mind wandering while reading the story. (-) • i could vividly imagine the scenes in the story. (+) • at points, i had a hard time making sense of what was going on in the story (-) after reading through both stories, they completed a highlighting exercise where they highlighted areas according to the following categories: eye-tracking data due to calibration issues, 8 samples were discarded, leaving 23 (13 female, 8 male, 2 other, average age: 28, std.: 10). see ## results other studies have shown that valence and arousal play an important role in predicting interest in a story @xcite and @xcite emphasized the importance of affective processes in his framework. in order to determine the importance of these values for our data, we used linear mixed model analysis. using lme4 @xcite and lmertest (kuznetsova 4 booknlp et al., 2017), we fit predictions of the proportion of the sentence highlighted and dwell time, with random effects of participant (@xmath0. variables were tested for collinearity using the variance inflation factor (vif) method outlined by @xcite , and no variables exceeded the recommended threshold of 3. observations and fixed effects are on a [0, 1] scale. see appendix b for exact model definitions. ## conclusion by collecting reader feedback and eye tracking data on literary fiction, we were able to support findings of other studies that emphasized the importance of affective language for reader immersion. although we found no direct relationship between dwell times and highlighted text, the dwell time model and the highlight model shared some predictors, such as valence and arousal. one possibility to explore for future studies would be to look at whether this overlap is related to two different modes of engagement -one that leads to higher dwell times and one that leads to lower dwell times. however, as mentioned, this exploration would require a more complete annotation. this could be achieved by selecting more engaging stories and modifying the highlighting exercise to require readers to annotate each sentence with a category or select none. further analysis on our data set could be done by extracting more complex features. this would expand the analysis beyond the lexical level would allow us to find more interesting relationships.