Relative Importance in Sentence Processing (2021)

Abstract: Determining the relative importance of the elements in a sentence is a key factor for effortless natural language understanding. For human language processing, we can approximate patterns of relative importance by measuring reading fixations using eye-tracking technology. In neural language models, gradient-based saliency methods indicate the relative importance of a token for the target objective. In this work, we compare patterns of relative importance in English language processing by humans and models and analyze the underlying linguistic patterns. We find that human processing patterns in English correlate strongly with saliency-based importance in language models and not with attention-based importance. Our results indicate that saliency could be a cognitively more plausible metric for interpreting neural language models. The code is available on github: https://github.com/beinborn/relative_importance .

## introduction when children learn to read, they first focus on each word individually and gradually learn to anticipate frequent patterns @xcite . more experienced readers are able to completely skip words that are predictable from the context and to focus on the more relevant words of a sentence @xcite . psycholinguistic studies aim at unraveling the characteristics that determine the relevance of a word and find that lexical factors such as word class, word frequency, and word complexity play an important role, but that the effects vary depending on the sentential context @xcite . in natural language processing, the relative importance of words is usually interpreted with respect to a specific task. emotional adjectives are most relevant in sentiment detection @xcite , relative frequency of a term is an indicator for information extraction @xcite , the relative position of a token can be used to approximate novelty for summarisation @xcite , and function words play an important role in stylistic analyses such as plagiarism detection @xcite . neural language models are trained to be a good basis for any of these tasks and are thus expected to represent a more general notion of relative importance @xcite . relative importance of the input in neural networks can be modulated by the so-called "attention" mechanism @xcite . analyses of image processing models indicate that attention weights reflect cognitively plausible patterns of visual saliency @xcite . recent research in language processing finds that attention weights are not a good proxy for relative importance because different attention distributions can lead to the same predictions @xcite . gradient-based methods such as saliency scores seem to better approximate the relative importance of input words for neural processing models @xcite . in this work, we compare patterns of relative importance in human and computational english language processing. we approximate relative importance for humans as the relative fixation duration in eye-tracking data collected in naturalistic language understanding scenarios. in related work, @xcite measure the correlation between attention in neural networks trained for a documentlevel question-answering task and find that the attention in a transformer language model deviates strongly from human fixation patterns. in this work, we instead approximate relative importance in computational models using gradient-based saliency and find that it correlates much better with human patterns. ## determining relative importance the concept of relative importance of a token for sentence processing encompasses several related psycholinguistic phenomena such as relevance for understanding the sentence, difficulty and novelty of a token within the context, semantic and syntactic surprisal, or domain-specificity of a token. we take a data-driven perspective and approximate the relative importance of a token by the processing effort that can be attributed to it compared to the other tokens in the sentence. ## methodology we extract relative importance values for tokens from eye-tracking corpora and language models as described in section 2 and calculate the spearman correlation for each sentence. 2 we first average the correlation over all sentences to analyze whether the importance patterns of humans and models are comparable and then conduct token-level analyses. ## results the results in length frequency sent tok sent tok geco human .69 .31 -.36 -.25 bert .65 .27 -.48 -.28 zuco human .75 .47 -.52 -.36 bert .72 .36 -.65 -.40 ## conclusion we find that human sentence processing patterns in english correlate strongly with saliency-based importance in language models and not with attentionbased importance. our results indicate that saliency could be a cognitively more plausible metric for interpreting neural language models. in future work, it would be interesting to test the robustness of the approach with different variants for calculating saliency @xcite . as we conducted our analyses only for english data, it is not yet clear whether our results generalize across languages. we will address this in future work using eye-tracking data from non-english readers @xcite and comparing monoand multilingual models @xcite . we want to extend the token-level analyses to syntactic phenomena and cross-sentence effects. for example, it would be interesting to see how a language model encodes relative importance for sentences that are syntactically correct but not se-mantically meaningful @xcite . previous work has shown that the inductive bias of recurrent neural networks can be modified to obtain cognitively more plausible model decisions @xcite . in principle, our approach can also be applied to left-to-right models such as gpt-2 @xcite . in this case, the tokens at the beginning of the sentence would be assigned disproportionately high importance as the following tokens cannot contribute to the prediction of preceding tokens in incremental processing. it might thus be more useful to only use the first fixation duration of the gaze signal for analyzing importance in left-to-right models. however, we think that the regressions by the readers provide valuable information about sentence processing.