Eye-tracking based classification of M andarin C hinese readers with and without dyslexia using neural sequence models (2022)

Abstract: Eye movements are known to reflect cognitive processes in reading, and psychological reading research has shown that eye gaze patterns differ between readers with and without dyslexia. In recent years, researchers have attempted to classify readers with dyslexia based on their eye movements using Support Vector Machines (SVMs). However, these approaches (i) are based on highly aggregated features averaged over all words read by a participant, thus disregarding the sequential nature of the eye movements, and (ii) do not consider the linguistic stimulus and its interaction with the reader’s eye movements. In the present work, we propose two simple sequence models that process eye movements on the entire stimulus without the need of aggregating features across the sentence. Additionally, we incorporate the linguistic stimulus into the model in two ways—contextualized word embeddings and manually extracted linguistic features. The models are evaluated on a Mandarin Chinese dataset containing eye movements from children with and without dyslexia. Our results show that (i) even for a logographic script such as Chinese, sequence models are able to classify dyslexia on eye gaze sequences, reaching state-of-the-art performance, and (ii) incorporating the linguistic stimulus does not help to improve classification performance.

## introduction reading effortlessly constitutes a key skill in modern society. individuals suffering from developmental dyslexia are characterized by specific and persistent reading problems. global prevalence estimates range from 3 to 7% @xcite . previous research has consistently shown that early diagnosis and intervention is key to mitigate the resulting long-term consequences @xcite . psychological and clinical research on eye movement patterns has revealed that individuals with dyslexia exhibit gaze patterns that differ significantly from the patterns observed in individuals without dyslexia @xcite . in particular, scanpaths of individuals with dyslexia are characterized by longer fixation durations, more fixations, decreased saccade durations and a higher proportion of regressions. in recent years, increasing effort has been spent on utilizing these findings and applying supervised classification methods such as svms and random forests on eye movement data (see @xcite for an overview) to infer the presence or absence of dyslexia. there are several reasons why automatized approaches for assistance in dyslexia detection are desirable. currently, paper-pencil diagnostic tools are conducted by trained speech therapists. these tools are time-intensive and are typically only considered after a suspected case has been reported by observant educational staff, leaving many cases overlooked. eye-movement-based diagnostic tools have the potential to be deployed in schools in a relatively inexpensive manner and as part of a standard procedure aimed at early and comprehensive detection of dyslexia; making an important contribution to educational equity. although the aforementioned approaches provide promising results, they suffer from specific drawbacks: (i) the model input consists of eye movement features, aggregated for each subject over the presented stimulus material (text), thus disregarding the sequential nature of the eye movements; (ii) both the linguistic stimulus and its interaction with the reader's eye movements are not considered. for classification purposes, this does not pose a problem per se. however, it does not allow us to investigate questions such as: which words (or, more specifically, what linguistic properties of the stimulus) are particularly informative to discriminate between individuals with and without dyslexia? in the present work, we propose two neural sequence models, depicted in figure 1 , that process the eye movements on the entire stimulus without the necessity of feature aggregation over the sentence. to incorporate the linguistic stimulus into the model, we use pre-trained contextualized word embeddings. we evaluate our model on an eyetracking-while-reading dataset from children with and without dyslexia reading mandarin chinese sentences by @xcite . ## problem setting we investigate the two closely related tasks of classifying (i) whether a given eye gaze sequence on one sentence is from a reader with or without dyslexia and (ii) whether a given eye gaze sequence on a set of sentences is from a reader with or without dyslexia. formally, our training data can be represented as a set @xmath0, y 1 ), . . . , (w n m , y n )}, where w @xmath1. . . w ijk ⟩ is a sequence of reading measure vectors 2 for each word @xmath2. . . k j obtained from subject i reading sentence j, where n is the number of participants, m is the number of stimulus sentences read by each of the participants and k j the number of words in a given sentence j. each reading measure vector consists of r reading measures, i.e., w @xmath3. . . r ijkr ). the binary target label y i denotes whether participant i is a reader with or without dyslexia. for (i), our goal is to train a binary classifier g θ such that the performance of a binary model can be characterized by a false-positive and a true-positive rate. by altering the decision threshold δ, a receiver operator characteristic (roc) curve can be derived, with the area under the curve providing an aggregated measure for all possible values of δ. ## experiments data. we employ eye-tracking-while-reading data from 62 mandarin chinese children (33 with dyslexia) provided by @xcite . participants were instructed to read 60 sentences out loud while their eye movements were recorded. 40 sentences were selected from fifth grade textbooks and 20 additional control sentences were extracted from the beijing sentence corpus @xcite . the dyslexia label had been assigned when a child scored at least 1.5 standard deviations below their corresponding age mean in standard character recognition test @xcite . ## discussion our proposed neural sequence models reach stateof-the-art performance on solving the task of detecting dyslexia from eye gaze sequences, for the first time investigated for a logographic script such as chinese. our results suggest that for our dataset, (i) neural architectures processing eye-movement sequences along the sentence have no advantage over the parsimonious svm-baseline where features are aggregated over the sentence, and (ii) enabling the interaction between stimulus input and eye movements does not improve classification performance. however, after having shown that our approach is able to reach sota performance, we aim to exploit its properties to investigate the informativeness of particular sentences, words, and other linguistic sub-units for dyslexia detection in the future. furthermore, for all investigated models, the overall performance appears to be driven by a small subset of individuals who presumably exhibit less typical reading behavior among their group and were more difficult to classify. given that dyslexia is a spectrum disorder-not binary as it is often perceived-it is to be expected that individuals that are not located at the two extremes (clearly dyslexic or clearly not dyslexic) are more difficult to classify in a binary environment. our study was able to show that an svm-based approach, previously applied to alphabetic languages such as swedish and spanish, also works well on a logographic script such as chinese. in future work, we would like to test our approach on alphabetic language data sets. this is particularly interesting given the fact that young chinese readers are faced with different challenges, e.g., the absence of orthographic word boundaries, therefore requiring word segmentation, and the much larger number of characters required to be memorized.