Transformer Attention vs Human Attention in Anaphora Resolution (2024)

Abstract: Motivated by human cognitive processes, attention mechanism within transformer architecture has been developed to assist neural networks in allocating focus to specific aspects within input data. Despite claims regarding the interpretability achieved by attention mechanisms, the extent of correlation and similarity between machine and human attention remains a subject requiring further investigation.In this paper, we conduct a quantitative analysis of human attention compared to neural attention mechanisms in the context of the anaphora resolution task. We collect an eye-tracking dataset based on the Winograd schema challenge task for the Russian language. Leveraging this dataset, we conduct an extensive analysis of the correlations between human and machine attention maps across various transformer architectures, network layers of pre-trained and fine-tuned models. Our aim is to investigate whether insights from human attention mechanisms can be used to enhance the performance of neural networks in tasks such as anaphora resolution. The results reveal distinctions in anaphora resolution processing, offering promising prospects for improving the performance of neural networks and understanding the cognitive nuances of human perception.

## introduction the term attention describes both human cognitive processes, crucial for tasks like reading and comprehension, and the attention mechanism in neural networks @xcite , which dynamically adjusts focus to specific input data. despite their apparent differences, this paper aims to analyze the correlations between transformer attention and human attention during anaphora resolution task. successful language comprehension requires understanding the discursive connections in the sentences and the logical relationships between dis-course structures in the text. coreference resolution, a standard nlp task, determines which mentions in a text refer to the same entity. two mentions (i.e., textual phrases) are called coreferent if they refer to the same real-world objects or events. anaphora, one of the types of coreference resolution, highlights this challenge by requiring the matching of an anaphor (typically a pronoun) in a sentence with its antecedent (noun) in the preceding sentence. the winograd schema @xcite is a well-established method for evaluating language model performance in anaphora resolution tasks, assessing the model's logical reasoning and real-world knowledge in resolving coreference ambiguities. it is an evaluation dataset within the superglue @xcite suite across various languages. video oculography, known as eye-tracking, is a prevalent psycholinguistic method for studying reading processes. it involves recording the reader's eye movements via video and subsequent interpolation of their gaze onto a display screen. this method breaks down the reading process into fixations (periods of steady gaze) and saccades (rapid eye movements) between them with precision up to milliseconds. this approach enables a detailed examination of reading acquisition. we used eye-tracking techniques to gather information on human fixations and focuses during the anaphora resolution and create the eye-tracking winograd schema dataset. leveraging the dataset, we investigate the correlation between machine and human attention across various transformer architectures and network layers. the research aims to confirm whether integrating insights from human attention patterns can significantly improve the language model's ability to resolve anaphoras effectively. the contributions of the current study are the following: ## related work in the subsequent sections, we outline related works encompassing attention mechanisms in transformers, human attention datasets of eye-tracking data, methods of correlation analysis between human and machine attention, and the incorporation of eye-gaze data into models during training. ## eye-tracking data objective the anaphora resolution task was chosen to investigate the distinction between attention mechanisms in neural networks and humans. this study explores the potential benefits of integrating human-inspired attention mechanisms into transformer architectures. the research question seeks to confirm whether the language model's incorporation of information regarding human attention distribution during text reading improves its performance in the anaphora resolution task. experimental setup for the experiment, we employed eye-tracking via video oculography, utilizing the eyelink 1000 plus device. participants' gaze was calibrated until validation error values reached less than 1 (maximum) and 0.5 (average). the indication 0.5 is the maximum average deviation. when calibrating at 9 points, the error of each point is calculated. if the average is not more than 0.5 and the total error is not more than 1, then the calibration is considered successful, and incentives are presented to the participants. the eyelink 1000 plus device is one of the most accurate systems, with a validation error of about 0.2-0.5, within the standard protocol according to the system manual @xcite . the russian winograd schema challenge dataset from tape @xcite was utilized for the anaphora resolution task to gather information on participants' eye movements. the experiment comprised 150 complex or compoundcomplex sentences extracted from the winograd schema challenge dataset, each containing an anaphoric pronoun and its antecedent. each participant was shown a sentence with an anaphoric pronoun highlighted in red on the screen, followed by a question about the presumed antecedent of the anaphora. the question was the following: "does the highlighted pronoun refer @xmath0. an example of the participants' screen is presented in app. a. for each sentence, two presumed antecedents (one correct, one incorrect) were identified for each sentence. thus, each screen was read by fifty participants. the sentences were randomized for each participant to ensure balanced conditions. one hundred people (81 women, average age -22.68, standard deviation -4.27) who are native speakers of russian participated in the experiment. they were instructed to read the provided sentences carefully and answer the question using the keyboard (key 1 for agreement, key 0 for disagreement). participants completed three training sentences to ensure task comprehension before proceeding to three blocks of 50 sentences each, with breaks provided between blocks. to enhance recording quality, each trial began with a calibration check, requiring participants to focus precisely on the point where the first word of the text would appear. upon successful calibration, the text was displayed; otherwise, recalibration commenced. after responding to each question, participants automatically advanced to the next trial. the experiment duration averaged 45 minutes. dataset statistics observations with missing values and parsing errors were excluded from the dataset. the final dataset consists of 296 sentencequestion pairs, which contain 9319 words and 148 unique sentences. the average number of participants per word is 48. the total number of observations for each variable is 448047. the resulting fields of the dataset are presented in the app. b. ## comparative analysis of attention mechanisms in order to investigate the potential advantages of incorporating attention mechanisms similar to human processes into transformer architecture, we first need to examine and compare different attention mechanisms. we carried out a set of experiments on various architectures, fine-tuned using the data, and compared them with data on human attention. our aim was to provide a detailed comparative analysis of human and neural attention mechanisms on the winograd schema challenge. ## results correlations of human attention with model attention are reported in tab. 4. we found significant correlations (@xmath1.05) for all experiments. there are moderate correlations for t5-based and robertabased models and strong correlations for bertbased models on the first layer. the comparison of different aggregation setups for t5-based architectures underscores the prevalence of the mean aggregation with high correlations. conversely, for other architectures, we noted a contrasting trend where row aggregation predominates. the first layers have high correlations in comparison to the last layer in most setups. for example, there are extremely small correlations for ruroberta-large on the last layer; meanwhile, the maximum correlation is almost even with other values. for several models, the highest correlation is noted on a particular layer. moreover, the layer demonstrating the highest correlation varies notably across different architectures. for most of the models, there is no difference between pre-trained and fine-tuned versions, except for a slight correlation decrease for multilingual mt5-base after tuning on the winograd schemas. furthermore, we can compare the outcomes across different eye-gaze metrics and observe minimal discrepancies among them in terms of correlation analyses. finally, we highlight the model mbert-base, which demonstrates the highest correlation with human attention. we conclude that task-specific fine-tuning did not enhance the correlation between human attention and machine attention. the analysis suggests that encoder-only models provide more significant insights for evaluating attention correlation. a detailed visualization of the correlation between human attention and models' attention on different layers is presented in app. c. additionally, app. d provides a visualization of the important words for the human and the models for one example from the eye-tracking dataset. ## integrating human gaze into transformers based on the results obtained in sec. 5, there are significant correlations between human attention and the models' attention during the task of anaphora resolution. it can be assumed that using eye movement data when training models for this task will increase their performance. we conducted experiments to integrate eye movement data into the model training process by using an additional term in the loss function to bring the model's attention closer to human attention. ## conclusion in summary, this paper examines the transformer and human attention mechanisms in the anaphora resolution task. we collected a dataset for the anaphora resolution task using video oculography and released it under the mit license 2 . we used this dataset to analyze the correlation between machine and human attention across various transformer architectures and network layers. the results show a strong correlation between human and machine attention, but fine-tuning did not en-2 https://huggingface.co/datasets/russiannlp/eyewino hance this correlation. therefore, we conducted experiments integrating eye movement data into the model training process. this was done by adding an extra term to the loss function to align the model's attention more closely with human attention. however, the results did not show a consistent trend in the proposed setup, indicating that further research is needed for incorporation approaches.