Revising with a Backward Glance: Regressions and Skips during Reading as Cognitive Signals for Revision Policies in Incremental Processing (2023)

Abstract: In NLP, incremental processors produce output in instalments, based on incoming prefixes of the linguistic input. Some tokens trigger revisions, causing edits to the output hypothesis, but little is known about why models revise when they revise. A policy that detects the time steps where revisions should happen can improve efficiency. Still, retrieving a suitable signal to train a revision policy is an open problem, since it is not naturally available in datasets. In this work, we investigate the appropriateness of regressions and skips in human reading eye-tracking data as signals to inform revision policies in incremental sequence labelling. Using generalised mixed-effects models, we find that the probability of regressions and skips by humans can potentially serve as useful predictors for revisions in BiLSTMs and Transformer models, with consistent results for various languages.

## motivation currently on-trend models like bi-lstms @xcite and transformers @xcite operate in a non-incremental fashion, relying on the availability of complete input sentences or texts to deliver output. one workaround to employ non-incremental encoders in real-time is applying a restart-incremental interface @xcite , enabling outputs to be revised as a by-product of recomputations, as explored by @xcite and @xcite . although possible, it forces recomputation from scratch at every new piece of input, which increases the computational load and can become infeasible for long sequences @xcite . on the other hand, inherently incremental models like rnns have the disadvantage of not being able to recover from mistakes via revisions (at least their prototypical versions). the sweet spot would be a model that can detect the need to revise. initiatives in this direction are hear @xcite , which has a module that predicts the need to restart, and tapir @xcite , which integrates an rnn with a transformer-revisor, predicting whether to recompute or to just extend the current output. a difficulty encountered in the latter is how to obtain a ground-truth signal for the revision policy. they derived silver labels from the outputs of another transformer, which is possibly too model-specific and its linguistic motivation is not explored. hear compares partial outputs to the non-incremental gold standard which, however, does not encode locally valid hypotheses (which only future input will rule out) and does not accommodate the fact that the gold standard may differ from its final output, thus penalising the incremental metrics with the model's non-incremental deficits @xcite . we usually do not have corpora containing annotation for the incremental hypotheses for input prefixes by humans, only the annotated gold labels for the final output. but there is vast literature using human reading data as a supervision signal in nlp tasks @xcite . inspired by that, we ask ourselves whether a model's revisions coincide with human regressions in eye-tracking reading data. a positive answer would mean that human reading data could help modelling a dedicated policy for revisions (as opposed to naive recomputations or restarts), and would serve as a cognitively motivated yardstick to judge a models' revisions. among all revisions, some are effective, i.e. they edit the prefix into a better state, with respect to a gold standard or to the final output @xcite . identifying them can contribute to reducing undesired revisions, which cause instability without bringing the advantage of improvement in output quality. therefore, if human reading behaviour can help perform only effective revisions, the signal is even more useful for incremental processing. ## related literature during reading, humans fixate the gaze on some words and make saccades that can be progressive or regressive with respect to the order of the words in the text, so that scanpaths and various measures regarding gaze position, direction and duration can be extracted with eye-tracking devices @xcite , a technique that is becoming more accessible at scale @xcite . research based on eye-tracking reading data often rely on the eye-mind hypothesis, which assumes that the eye remains fixated on a word as long as it is being processed @xcite . various research fields rely on the temporal and spatial dimensions of human reading data. we identify at least three (non-mutually exclusive) uses. a consolidated line of research involves studying human cognition and verifying linguistic theories of sentence processing (e.g. @xcite and @xcite ). another field is occupied with understanding to what extent com-putational models like artificial neural networks resemble human cognition in how they process language, for example by estimating their psychometric predictive power @xcite . a relationship commonly investigated is the surprisal of language models versus human reading time @xcite @xcite . nlp has been incorporating eye-tracking data in recent years @xcite , with the emerging use of human reading data both as input to enhance nlp models (see @xcite and @xcite for recent surveys) and as a means for their interpretability @xcite . in this work, the phenomenon of interest is regressions, i.e. eye movements that move backwards in the text and can be shorter or longer-range @xcite . they are a common topic in psycholinguistics research @xcite and various hypotheses account for their role, such as comprehension or word identification difficulties, low-level visuomotor processes, rereading, memory cues and tools for language processing (see @xcite , @xcite and booth and weger (2013) for comprehensive discussions and references). relevant measures are at which word a regression initiates, at which word it lands, regression path duration (how long the reader remains in past text before progressing to unseen text), and how many regressions are initiated for each word. we can also differentiate between firstpass and subsequent regressions. regressions in nlp reading data has been used as a source of psycholinguistic information for various nlp tasks. when it comes to regressions, @xcite used eye-movements to predict syntactic categories, an idea further explored in @xcite , who augmented postaggers with various gaze features, among which was the number of regressions originating from a word. @xcite used the number of regressions from and to a word as features to predict grammatical functions. the number of total regressions per word was also used as a feature by @xcite for sarcasm understandability prediction. regression duration, i.e. the total time spent on a word after the first pass over it, was a useful feature for sentence compression proposed by @xcite . regressions during coreference resolution annotation were investigated by @xcite , who used it to propose a heuristic for pruning candidates in a coreference resolution model. in @xcite , the total duration of regressions from a word was used as a context feature in named-entity recognition. we draw inspiration from the work by @xcite , who hypothesised that backward saccades are involved in online syntactic analysis, in which case regressions should coincide, at least partially, with the edges of the relations computed by a dependency parser. they found a significant effect of the number of left-hand side dependency relations on the number of backward saccades. while the authors were interested at predicting human regressions from a model instantiating a parsing theory, we are conversely interested in using human regressions as a signal to train an nlp model. 3 ## method to perform the analysis, we use binomial generalised linear mixed models (glmm) with a logit link function to predict model revisions. similar to the approach by @xcite , for each combination of dataset and nlp model/task, we fit two glmms: the baseline model (1) only includes the token position variable as a fixed effect and texts as random effects. since a model's revisions may vary depending on the word's position in the text, we add token position as a baseline predictor and include texts to account for any variability due to different types of texts. we fit model (2) with the same structure, adding the predictors of regression probability and skipping probability as fixed effects. the binary dependent variable is a token's revise/not-revise label. we use likelihood ratio tests (lrt) between the null and the full models to evaluate the goodness of fit. lrts are used to compare a baseline model to (…) that night there was scarcely a square inch of earth that was not illuminated by aurora. an example of our data structure for a portion of a text in the provo corpus, processed by a restartincremental transformer predicting dependency relations. each token is annotated with the reading variable for each subject (eyes: regressed, 0: not regressed, -: skipped) and the model's decision (r: revised, /: not revised). a more complex one with more predictors and decide if certain predictors should be included, consequently selecting the model that fits the data better. to infer statistical significance, we obtain p-values using the χ 2 distribution. we do not intend to make claims about why regressions occur. for our purposes, we take at face value that they did occur in the eye-tracking experiments (and when). we are interested in words at which regressions are initiated when they are first read, knowing that, for some reason, the reader went to past input before continuing (as a consequence, we also analyse words that are not fixated in the first pass). still, the hypothesis that regressions occur due to reanalysis, when humans encounter garden path sentences @xcite , is at our favour, since revisions represent updates in the current model's interpretation caused by input seen for the first time. ## data in this section, we explain the data structure constructed for the analysis. we then introduce the eye-tracking corpora and the models selected for this study, and discuss how we extract the incremental outputs from non-incremental, pre-trained sequence labelling models. 4 procedure our method requires knowing, for each token w in a text, what was the behaviour of the model while performing sequence labelling and of the humans while reading the text. more specifically, we need to know whether the model revised its hypothesis upon processing w and whether humans skipped w, fixated it but moved forward, or fixated it and regressed. we thus construct an annotation mapping tokens to human and model data as illustrated in figure 2 . the texts come from the eye-tracking corpora, from which we also extract the human skips or regressions. the revisions are retrieved by feeding the same texts to the nlp models, prefix by prefix in a restart-incremental fashion, and checking if labels change at each time step. ## results we summarise the full glmm results in we found that average human gaze patterns, namely the estimated word's regression and skip probability, were significant predictors of revisions. this was a consistent result across all eye-tracking corpora, for the bilstm and the transformer, both for dependency parsing and pos-tagging. on the one hand, human regressions were often positively related to revisions, so that words with a higher regression probability were more likely to be revised by models (meco-l2-transformer-pos was the only exception where regression probability negatively affected revisions). conversely, a word's skip probability decreased the probability of it triggering a revision in most cases (with the exceptions of potec and provo-transformer-pos and nicenboim-bilstm-pos). these relationships are illustrated in figure 4 . the magnitude of the regression coefficient did not follow a general pattern for the tasks, but the skip coefficient was more often larger for the task of predicting the head than for the dependency relation, which was usually larger than for pos-tagging (exceptions to this is rastros-transformer and meco-l1-bilstm). in a further analysis, we repeated the same procedure to predict only the effective revisions and observed the same trend in regression and skip coefficients when predicting effective revisions, in terms of direction and significance, in all experiments. however, the magnitude of the coefficients differed, sometimes being larger in one or the other, which does not allow us to draw general conclusions at this point. the coefficient of token position was, in most cases, smaller in the model that predicts effective revisions. similarly, in many models the magnitude of the coefficient of skips was larger for models predicting effective revisions. to assess the fit of the model to the data in more detail, we evaluated its predictions by running permutation tests with the null hypothesis that the probabilities assigned to (effective) revisions and to not-revisions are randomly sampled from the same distribution. besides, we computed the area under the roc curve in each model. as we can see in we have gathered evidence that there is a relationship between nlp restart-incremental models' revisions and human gaze behaviour in reading, which manifests as the probability of revision at a given token being partially predictable from it being often skipped or triggering regressions, when token position and text are accounted for. interestingly, the overall findings hold for bilstm and transformers, even though their encoding mechanisms are different, and also for all five languages, despite the eye-tracking data having been collected from different text genres and the readers having performed different tasks (or no additional task beyond reading for comprehension, as in provo). for this conclusion, we did not rely on any assumptions for the connection between human regressions and incremental models' revisions beyond the analogy of what we factually know: when seeing text areas for the first time, humans made decisions to skip or fixate, and possibly to revisit past text, and at some words, models "decided" to revisit past decisions. some exceptions to the general trend in predicting model revisions occurred in pos-tagging, for which relatively fewer revisions occur (see the model has to wait until the root is processed to make good predictions. there may also be a difference in processing, since the humans could regress to previous sentences in the text, whereas the nlp models depend on their internal tokenisation and sentence boundary detection. this suggests that eye-tracking measures can be transformed into a useful signal to inform the decision of when to revise in mixed restart-incremental processors, especially when the model's task entails more syntactic tasks with frequent revisions to the input. still, preliminary investigation of the revision probabilities predicted by the model did not yield a straightforward threshold for binary classification, despite the difference in means being statistically significant. this invites a more detailed extrinsic evaluation, by incorporating the human predictors into a revision controller like tapir @xcite , and assessing the revisions with the evaluation methods discussed by @xcite . one approach is to train an incremental sequence labelling model whose revision policy relies on eye-tracking data as part of the input and comparing its performance against a model without it. since skips had a negative effect, it may also be possible to use other variables that relate to the probability of a token being skipped, like pos-tags or word frequency and length, as additional input, which are cheaper to obtain. the analysis should also be done with larger datasets and other models and tasks. the usefulness of our findings presupposes the availability of eye-tracking measures during inference on truly unseen data, which is an open problem because such signal is not always available in real time. one possibility is to use pretrained eyetracking models to predict regressions and skips, as in approaches discussed in the literature @xcite . down the road, a revision policy should not only detect times to revise, but times to revise effectively, since wrong revisions make the partial outputs less reliable for downstream processors. our experiments showed that regressions and skips are also good predictors for effective revisions. identifying ways to filter this more specific signal demands further investigation. an immediate next step is to evaluate the predictions of each model in unseen data for all revisions and for effective revisions. ## conclusion let us conclude with a backward glance to our contribution. we have addressed the open question of whether pre-trained sequence labelling models, when employed incrementally, perform revisions in a similar fashion as humans skip words or make regressive eye movements while reading. we have found a significant effect in all the experiments, supporting the use of human reading data as a cognitive signal to inform revision policies. this is a valuable finding: bilstms and transformers are bidirectional, trained on full sequences, but if we make them process linguistic input incrementally, their revisions can be partially predicted by human reading behaviour. this is also a step forward towards understanding why these models change hypotheses at some tokens, when only partial prefixes are available. besides advancing the research on eye-trackingaugmented nlp, this study also opens the door to exploring other cognitive perspectives with restartincremental nlp models. we see a potential to go the other direction and investigate to what extent a "mixed incrementality" model (architectures relying on an incremental processor with occasional restarts) would capture the patterns of human gaze in reading, and hence function as a model of that. in this case, revisions would serve as predictors of human regressions, with control variables like word frequency, surprisal and word length. other possibility for future work is to investigate whether other measures, like number of fixations or regressions to a token, are related to the edits per label.