An Eye Opener Regarding Task-Based Text Gradient Saliency (2024)

Abstract: Eye movements in reading reveal humans’ cognitive processes involved in language understanding. The duration a reader’s eyes fixate on a word has been used as a measure of the visual attention given to that word or its significance to the reader. This study investigates the correlation between the importance attributed to input tokens by language models (LMs) on the one hand and humans, in the form of fixation durations, on the other hand. While previous research on the internal processes of LMs have employed the models’ attention weights, recent studies have argued in favor of gradient-based methods. Moreover, previous approaches to interpret LMs’ internals with human gaze have neglected the tasks readers performed during reading, even though psycholinguistic research underlines that reading patterns are task-dependent. We therefore employ a gradient-based saliency method to measure the importance of input tokens when LMs are targeted on specific tasks, and we find that task specificity plays a crucial role in the correlation between human- and model-assigned importance. Our implementation is available at https://github.com/gjwubyron/Scan.

## introduction human eye movements during reading reflect cognitive processes involved in language processing @xcite : the fixation duration on a word correlates with reading comprehension @xcite . as such, fixation duration has been employed as proxy of the relative importance of a word to a reader @xcite . the introduction of neural attention mechanisms @xcite and the transformer architecture @xcite , which relies on self-attention to compute input and output representations, has given fresh impetus to research into how language models (lms) process language. attention mechanisms assign dynamic weights to input tokens, offering a method to understand a model's internal functioning and decision-making processes @xcite . recent research has compared model and human language comprehension by aligning model attention weights with human reading metrics, such as fixation durations @xcite @xcite , presuming model attention effectively signifies the relative importance of input tokens. however, the findings are mixed (cf section 2). while some studies @xcite ) observed significant differences between transformer lms' attention patterns and human fixation patterns, other studies @xcite found strong correlations. besides, research on attention @xcite @xcite has questioned the reliability of attention weights in accurately reflecting token significance. in contrast, hollenstein and beinborn (2021) utilized gradient-based saliency @xcite to approximate relative importance in lms through iterative token masking and discovered strong correlation between lms gradient-based saliency and human fixation durations. however, the output space of this approach comprises tens of thousands of tokens, which could make gradient-based saliency uninformative @xcite . moreover, their work focused on natural reading. since psycholinguistic studies show that human reading strategies vary with the task and differ from normal reading @xcite @xcite , it is crucial to take task specificity into account. in this work, we align the lms with the same tasks performed by human participants during taskspecific reading and measure the importance of input tokens using gradient-based saliency. additionally, we expand our analysis to include decoderbased lms, which, due to their auto-regressive nature, more closely mirror the incremental nature of human processing. we find strong correlations between lms and humans in this task-specific setting, and further fine-tuning on the task can enhance these correlations. ## related work model attention and human attention research comparing model attention to human visual attention, using fixation locations and durations as proxies, has produced mixed findings. @xcite observed distinct differences between transformer lm attention patterns and human fixation patterns. conversely, studies by @xcite and @xcite found strong correlations between early transformer layer attention weights, like those in bert @xcite , and human visual attention, contrasting with earlier results. this discrepancy can be attributed to methodological differences in processing attention weights: @xcite analyzed maximum attention values from the last layer's sub-word tokens, while @xcite averaged attention across sub-word tokens in the first layer. ## method consider an input sentence, formalized as @xmath0, . . . , x n ⟩ of n tokens, where x j is the j th token (word) in the sentence, and two corresponding token importance vectors of the same length: the human importance vector @xmath1, . . . , h n ⟩ and the model importance vector @xmath2, . . . , m n ⟩, where h j and m j are the human and model importance attributed to token x j . we obtain the mean spearman correlation between model and human importance by computing the by-token spearman correlations between the vectors m and h for all sentences x, then dividing the sum of these correlations by the number of sentences x. ## participant-level analysis to investigate whether the models correlate more with certain participants, we perform an additional participant-level analysis in which we compute correlations between the model-extracted saliency values and relative fixation durations for each participant individually. we also extract the participants' response accuracies for both their sa and re, averaged over sentences. the underlying intuition is that the models possibly correlate more with participants that have a higher task accuracy.