L ang R esearch L ab_ NC at CMCL 2021 Shared Task: Predicting Gaze Behaviour Using Linguistic Features and Tree Regressors (2021)

Abstract: Analysis of gaze data behaviour has gained momentum in recent years for different NLP applications. The present paper aims at modelling gaze data behaviour of tokens in the context of a sentence. We have experimented with various Machine Learning Regression Algorithms on a feature space comprising the linguistic features of the target tokens for prediction of five Eye-Tracking features. CatBoost Regressor performed the best and achieved fourth position in terms of MAE based accuracy measurement for the ZuCo Dataset.

## introduction eye-tracking data or gaze data compiles millisecond-accurate records about where humans look while reading. this yields valuable insights into the psycho-linguistic and cognitive aspects of various tasks requiring human intelligence. eye-tracking data has been successfully employed for various downstream nlp tasks, such as part of speech tagging @xcite , named entity recognition @xcite , sentiment analysis @xcite ), text simplification @xcite , and sequence classification @xcite among others. development of systems for automatic prediction of gaze behaviour has become an important topic of research in recent years. for example, @xcite and @xcite used bi-lstm and cnn, respectively for learning different gaze features. in the present work, eye-tracking features for words/tokens of given sentences are learned using tree regressors trained on a feature space comprising the linguistic properties of the target tokens. the proposed feature engineering scheme aims at encoding shallow lexical features, possible familiarity with the readers, interactions of a target token with other words in its context, and statistical language model features. ## task setup the shared task is designed to predict five eye-tracking features namely, number of fixations (nf), first fixation duration (ffd), total reading time (tr), go-past time (gp) and, fixation proportion (fxp). zuco eye-tracking dataset is used for the present task @xcite @xcite . the dataset contains three subsets corresponding to train, trial and test which contains 700, 100, and 191 sentences, respectively. their respective token counts are 13765, 1971, and 3554. each input token is uniquely represented by a @xmath0, wid >, where sid is the sentence_id and wid is the word_id. mean absolute error (mae) is used for evaluation. ## feature engineering for the above-mentioned task, linguistic features for a given input token are extracted in order to encode the lexical, syntactic, and contextual properties of the input token. additionally, familiarity of the input token and its collocation with surrounding words is also modelled as explained below. ## description of algorithms experiments were conducted using the following machine learning regression algorithms: since five target eye-tracking metrics had to be predicted, multioutput (mo) and regressor chain (rc) algorithms were deployed using sklearn. ## experimental details the input tokens containing only punctuations were removed. the eye-tracking feature for token '&' is assigned a fixed value 8 . for all other punctuation tokens, the assigned eye-tracking feature value is 0. spacy 9 is used for pos tagging, lemmatization, dependency parsing and ner. stopword feature, corpus features and frequency features as described in section 3.2 were extracted after lower casing and lemmatizing the input token. for rc the order is tuned between the 120 possibilities and the max_depth denoted as d, is tuned between 1 to 15. for nn the number of intermediate dense layers is tuned between 1 to 4, the layer dimension is tuned between {10, 25, 50, 100, 150, 200, 250, 300, 500} and dropouts is tuned randomly between 0 to 1. relu activation function is used in the intermediate dense layers, batch size is set to 32, learning rate is set to 0.005, and mae is minimized using adam optimizer @xcite . ## results the individual mae for the five predicted features along with overall mae for various regression techniques are reported in ## analysis system predictions are presented in to analyze the importance of each feature, the corresponding feature is eliminated and the cb+mo model is trained on the reduced feature space. it was observed that elimination of each individual feature increased the error and thus, each feature plays an important role in the overall performance of the system. the mae on the trial set corresponding to individual features are reported in ## conclusion and future work automatic prediction of gaze features without human intervention is important for scalability of these features for tasks involving large datasets. the shared task aims at prediction of five eye-tracking features for each token of a given sentence. in the present work, a set of linguistic features focused on representing the shallow lexical characteristics of the token, rarity of the token, and interaction and collocation of the target token with its context are extracted. cb+mo regressor trained on the above feature space secured fourth rank on the shared task. error analysis indicates that there is high variation of eye-tracking features for the same words in different contexts. however, the proposed system does not capture this variation. in future we would like to incorporate more features in order to represent the context of the target token more effectively.