Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension (2020)

Abstract: While neural networks with attention mechanisms have achieved superior performance on many natural language processing tasks, it remains unclear to which extent learned attention resembles human visual attention. In this paper, we propose a new method that leverages eye-tracking data to investigate the relationship between human visual attention and neural attention in machine reading comprehension. To this end, we introduce a novel 23 participant eye tracking dataset - MQA-RC, in which participants read movie plots and answered pre-defined questions. We compare state of the art networks based on long short-term memory (LSTM), convolutional neural models (CNN) and XLNet Transformer architectures. We find that higher similarity to human attention and performance significantly correlates to the LSTM and CNN models. However, we show this relationship does not hold true for the XLNet models – despite the fact that the XLNet performs best on this challenging task. Our results suggest that different architectures seem to learn rather different neural attention strategies and similarity of neural to human attention does not guarantee best performance.

## introduction due to the high ambiguity of natural language, humans have to detect the most salient information in a given text and allocate a higher level of attention to specific regions to successfully process and comprehend it @xcite @xcite . eye tracking studies have been extensively used in various reading comprehension tasks to capture and investigate these attentive strategies @xcite and have, as such, helped to interpret cognitive processes and behaviors during reading. attention mechanisms in neural networks have been inspired by human visual attention @xcite . similar to humans, they allow networks to focus and allocate more weight to different parts of the input sequence @xcite @xcite @xcite . as such, neural attention can be viewed as a model of visual saliency that makes predictions over the elements in the network's input -whether a region in an image or a word in a sentence @xcite . attention mechanisms have recently gained significant popularity and have boosted performance in natural language processing tasks and computer vision @xcite @xcite @xcite . although attention mechanisms can significantly improve performance for different nlp tasks, performance degrades when models are exposed to inherent properties of natural language, such as semantic ambiguity, inferring information, or out of domain data @xcite . these findings encourage work towards enhancing network's generalizability, deterring reliance on the closed-world assumption @xcite . in machine reading comprehension (mrc), it has been proposed that the more similar systems are to human behavior, the more suitable they become for such a task @xcite @xcite . as a result, much recent work aims to build machines which read and understand text, mimicking specific aspects of human behavior @xcite @xcite laszlo orders the house band to defiantly play "la marseillaise". when the band looks to rick, he nods his head. cnn laszlo orders the house band to defiantly play "la marseillaise". when the band looks to rick, he nods his head. lstm laszlo orders the house band to defiantly play "la marseillaise". when the band looks to rick, he nods his head. xlnet laszlo orders the house band to defiantly play "la marseillaise". when the band looks to rick, he nods his head. human 0.01 0.02 0.03 0.04 0.05 figure 2 : an exemplary scan path shows a reading pattern. the red circle corresponds to the location of the current fixation. its size is proportional to the duration of the fixation. @xcite . to that end, by employing selfattention, researchers attempt to enhance comprehension by building models which better capture deep contextual and salient information @xcite @xcite @xcite . as neural attention allows us to "peek" inside neural networks, it can help us to better understand how models make predictions (see figure 1 ). similarly, human visual attention (which is captured by physiological data such as eye tracking), allows us to quantify the relative importance of items within the visual field when reading texts (see figure 2 ). in this work, we propose a novel method that leverages human eye tracking data to investigate the relationship between neural performance and human attention strategies. concretely, by interpreting and comparing the relationship between neural attention distributions of three state of the art mrc models to human visual attention, our research for the first time addresses the following questions: (i) what is the correlation between a particular network behavior and the human visual attention? (ii) is the emulation of the human attention system the reason why neural models with attention mechanisms achieve state of the art results on machine reading comprehension tasks? to answer these questions, we first extend the movieqa dataset @xcite with eye tracking data. in addition, we present a novel vi-sualization tool to qualitatively compare the differences in attentive behaviors between neural models and humans by showing their patterns over time in a split screen mode. second, as widely suggested in the cognitive science literature, we quantify human attention in terms of the word-level gaze duration recorded in our eye tracking dataset @xcite @xcite @xcite . third, we interpret the relationship between human attention and three state of the art systems based on cnn, lstm, and xlnet @xcite using kullback-leibler divergence @xcite . by doing so, we are able to compare, evaluate and better understand neural attention distributions on text across these attention models. to the best of our knowledge, we are the first to propose a systematic approach for comparing neural attention to human gaze data in machine reading comprehension. the main findings of our work are two-fold: first, we show that there is a statistically significant correlation between the cnns and lstms model performances and similarity to human attention. second, we show that the behavior of lstm models is significantly more similar to humans than the xlnet ones even though the latter perform best on the movieqa dataset. ## conclusion and future work our core contribution is a new method for comparing human visual attention versus neural attention distributions in machine reading comprehension. to the best of our knowledge, we are the first to do so with gaze data. our findings show that cnns and lstms have a statistically significant correlation between similarity to human visual attention distributions and system performance. interestingly, the same is not true for xlnets. moreover, the attention weights of the lstms are significantly different compared to the xlnets. although these pre-trained transformer networks are less similar to human visual attention, our fine-tuned model obtains the new sota on the movieqa benchmark dataset with 91% accuracy on the validation set. in addition, we extend the movieqa dataset with eye tracking data, release this as open source and present an attentive reading visualiza- nine best avg kl combo estimate std. error t-value p-value lstm 0.018 lstm vs. xlnet -0.003 0.001 -2.835 < 0.01 cnn 0.020 lstm vs. cnn -0.001 0.001 -1.098 0.27 xlnet 0.022 cnn vs. xlnet -0.001 0.001 -1.736 0.17 in future work we plan to extend our understanding of these large-scale pre-trained language models. it would be interesting to investigate whether the observed increase in performance but lack of similarity to humans in the xlnet models is because they are pre-trained on large external corpora or whether this is due to inherent properties in architecture, when compared to other pre-trained models (such as bert). lastly, to further disentangle token level saliency versus cognitive load of processing, additional analyses and metrics could be considered. yukun zheng, jiaxin mao, yiqun liu, zixin ye, min zhang, and shaoping ma. 2019. human behavior inspired machine reading comprehension. in proceedings of the 42nd international acm sigir conference on research and development in information retrieval, pages 425-434. acm.