Assessing Word Importance Using Models Trained for Semantic Tasks (2023)

Abstract: Many NLP tasks require to automatically identify the most significant words in a text. In this work, we derive word significance from models trained to solve semantic task: Natural Language Inference and Paraphrase Identification. Using an attribution method aimed to explain the predictions of these models, we derive importance scores for each input token. We evaluate their relevance using a so-called cross-task evaluation: Analyzing the performance of one model on an input masked according to the other model’s weight, we show that our method is robust with respect to the choice of the initial task. Additionally, we investigate the scores from the syntax point of view and observe interesting patterns, e.g. words closer to the root of a syntactic tree receive higher importance scores. Altogether, these observations suggest that our method can be used to identify important words in sentences without any explicit word importance labeling in training.

## introduction the ability to decide which words in a sentence are semantically important plays a crucial role in various areas of nlp (e.g. compression, paraphrasing, summarization, keyword identification). one way to compute (semantic) word significance for compression purposes is to rely on syntactic patterns, using integer linear programming techniques to combine several sources of information @xcite . @xcite exploit the same cues, with significance score computed as a mixture of tf-idf and surface syntactic cues. a similar approach estimates word importance for summarization @xcite or learns these significance scores from word embeddings @xcite . significance scores are also useful in an entirely different context, that of explaining the decisions of the interpreter takes both text inputs x p , x h , and hidden states h p of the nli model's encoder. it generates a binary mask z p which is used to mask x p , resulting in xp . the second pass (green dashed arrows): xp is passed to the nli model together with the original hypothesis. the divergence d * minimizes the difference between predicted distributions y and ŷ of these two passes. deep neural networks (dnns). this includes investigating and interpreting hidden representations via auxiliary probing tasks @xcite ; quantifying the importance of input words in the decisions computed by dnns in terms of analyzing attention patterns @xcite ; or using attribution methods based on attention @xcite , back-propagation @xcite or perturbation techniques @xcite . along these lines, deyoung et al. (2020) present a benchmark for evaluating the quality of modelgenerated rationals compared to human rationals. in this study, we propose to use such techniques to compute semantic significance scores in an innovative way. we demand the scores to have these intuitive properties: (a) content words are more important than function words; (b) scores are contextdependent; (c) removing low-score words minimally changes the sentence meaning. for this, we train models for two semantic tasks, natural lan- to explain the models' predictions. we evaluate the relevance of scores using the so-called crosstask evaluation: analyzing the performance of one model on an input masked according to the other model's weights. we show that our method is robust with respect to the choice of the initial task and fulfills all our requirements. additionally, hinting at the fact that trained hidden representations encode a substantial amount of linguistic information about morphology @xcite , syntax @xcite , or both @xcite , we also analyze the correlations of our scores with syntactic patterns. ## method we assume that sentence-level word significance (or word importance) is assessed by the amount of contribution to the overall meaning of the sentence. this means that removing low-scored word should only slightly change the sentence meaning. the method we explore to compute significance score repurposes attribution techniques originally introduced to explain the predictions of a dnn trained for a specific task. attribution methods typically compute sentence level scores for each input word, identifying the ones that contribute most to the decision. by explicitly targeting semantic prediction tasks, we hope to extract attribution scores that correlate well with semantic significance. our significance scoring procedure thus consists of two main components: an underlying model and an interpreter. the underlying model is trained to solve a semantic task. we select two tasks: natural language inference (nli) -classifying the relationship of a premise-hypothesis pair into entailment, neutrality or contradiction -and paraphrase identification (pi) -determining whether a pair of sentences have the same meaning. the interpreter relies on the attribution method proposed by de cao et al. (2020), seeking to mask the largest possible number of words in a sentence, while at the same time preserving the underlying model's decision obtained from the full sentence pair. the interpreter thus minimizes a loss function comprising two terms: an l 0 term, on the one hand, forces the interpreter to maximize the number of masked elements, and a divergence term d * , on the other hand, aims to diminish the difference between the predictions of the underlying model when given (a) the original input or (b) the masked input. we take the outputs of the interpreter, i.e. the attribution scores, as probabilities that given words are not masked. following de cao et al. (2020), these probabilities are computed assuming an underlying hard concrete distribution on the closed interval [0, 1], which assigns a non-zero probability to extreme values (0 and 1) (fig. 9 , de cao et al., 2020). during interpreter training, a reparametrization trick is used (so that the gradient can be propagated backwards) to estimate its parameters. given the hard concrete distribution output, the attribution score for a token expresses the expectation of sampling a non-zero value, meaning that the token should be masked (section 2, stochastic masks, de @xcite . we illustrate the process in figure 1 . ## analysis in our analysis of the predicted masks, we only consider the last-layer classifier, rescaling the values so that the lowest value and the highest value within one sentence receive the scores of zero and one, respectively. all results use the snli validation set. ## conclusion in this paper, we have proposed a novel method to compute word importance scores using attribution methods, aiming to explain the decisions of models trained for semantic tasks. we have shown these scores have desired and meaningful properties: content words are more important, scores are context-dependent and robust with respect to the underlying semantic task. in our future work, we intend to exploit these word importance scores in various downstream applications.