Team R ead M e at CMCL 2021 Shared Task: Predicting Human Reading Patterns by Traditional Oculomotor Control Models and Machine Learning (2021)

Abstract: This system description paper describes our participation in CMCL 2021 shared task on predicting human reading patterns. Our focus in this study is making use of well-known,traditional oculomotor control models and machine learning systems. We present experiments with a traditional oculomotor control model (the EZ Reader) and two machine learning models (a linear regression model and a re-current network model), as well as combining the two different models. In all experiments we test effects of features well-known in the literature for predicting reading patterns, such as frequency, word length and predictability. Our experiments support the earlier findings that such features are useful when combined. Furthermore, we show that although machine learning models perform better in comparison to traditional models, combination of both gives a consistent improvement for predicting multiple eye tracking variables during reading.

## introduction oculomotor control in reading has been a wellestablished domain in eye tracking research. from the perspective of perceptual and cognitive mechanisms that drive eye movement control, the characteristics of the visual stimuli is better controlled in reading research than visual scene stimuli. several computational models have been developed for the past two decades, which aimed at modeling the relationship between a set of independent variables, such as word characteristics and dependent variables, such as fixation duration and location @xcite . based on the theoretical and experimental research in reading, the leading independent variables include the frequency of a word in daily use, the length of the word and its sentential predictability. the term sentential predictability (or word predictability) is used to define predictability score which is the probability of guessing a word from the sequence of previous words of the sentence @xcite . the dependent variables include numerous metrics, including fixation duration metrics such as first fixation duration (ffd) and total gaze time on a word, as well as location and numerosity metrics such as the location of a fixation on a word and gaze-regressions. a major caveat of the computational models that have been developed since the past two decades is that they weakly address linguistic concepts beyond the level of the fixated word, with a few exceptions, such as the spillover effects related to the preview of a next word n+1 during the current fixation on word n @xcite . these models are also limited in their recognition of syntactic, semantic and discourse characteristics of the text due to their complexity, despite they are indispensable aspects of reading for understanding. machine learning (ml) models of oculomotor control address some of those limitations by presenting high predictive power. however, the holistic approach of the learning models has drawbacks in terms of the explainability of the model underpinnings. in this study, we present experiments with a traditional model of oculomotor control in reading, namely the ez reader @xcite , two ml models (a regression model and a recurrent network model), and their combination. we present an evaluation of the results by focusing on the model inputs that reveal relatively higher accuracy. accordingly, the aim of the present paper is to investigate the effectiveness of both types of models and their combinations on predicting human reading behavior as set up by the cmcl 2021 shared task @xcite . the task is defined as predicting five eye-tracking features, namely number of fixations (nfix), first fixation duration (ffd), total reading time (trt), go-past time (gpt), and fixation proportion (fixprop). the eye-tracking data of the zurich congitive language processing corpus (zuco 1.0 and zuco 2.0) were used @xcite , @xcite . details of these variables and further information on the data set can be found in the task description paper @xcite . ## methodology we created our models and identified the input features following the findings in research on oculomotor control in reading. the previous studies have shown that word length, frequency and sentential predictability are well known parameters that influence eye movement patterns in reading @xcite . there exist further parameters that influence eye movement characteristics. for instance, the location of a word in the sentence has been proposed as a predictor on first fixation duration @xcite . therefore, we used those additional parameters to improve the accuracy of the learning models, as well as running a traditions oculomotor control model (viz., the ez reader) with its required parameter set. below we present a description of the models that have been employed in the present study. ## results we participated in the cmcl 2021 shared task with two submissions, one with the linear model described in section 2.1.2, and the other with the lstm model (section 2.1.3). ## conclusion in this paper, we analyzed a linear baseline model and an lstm model that employed the outputs of a traditional model as its inputs. we built models with input features in isolation, and their combination. the evaluation of the mean absolute errors (mae) supported a major finding in reading research: the oculomotor processes in reading are influenced by multiple factors. temporal and spatial aspects of eye movement control are determined by linguistic factors as well as low-level nonlinguistic factors @xcite . the models that employ their combinations return higher accuracy. our findings also indicate that besides the frequently used features in the literature, the ez-sfd (single frequency duration outputs of the ez reader model) may contribute to the accuracy of the learning based models. nevertheless, given the high variability of the machine learning model outputs a systematic investigation is necessary that address several operational assumptions in the present study. in particular, future research should improve statistical analysis for comparing the model outputs. it should also address the influence of the location of a word in a sentence, besides its interaction with the duration measures. last but not the least, future research on developing ml models of oculomotor control in reading should focus on the relationship between the aspects of the ml model design and basic findings in reading research. the gcmw (gaze contingent moving window) paradigm and the boundary paradigm @xcite are some examples of those findings that could be used for oculomotor control modeling. figure 6: mae scores for trt