Leveraging Affirmative Interpretations from Negation Improves Natural Language Understanding (2022)

Abstract: Negation poses a challenge in many natural language understanding tasks. Inspired by the fact that understanding a negated statement often requires humans to infer affirmative interpretations, in this paper we show that doing so benefits models for three natural language understanding tasks. We present an automated procedure to collect pairs of sentences with negation and their affirmative interpretations, resulting in over 150,000 pairs. Experimental results show that leveraging these pairs helps (a) T5 generate affirmative interpretations from negations in a previous benchmark, and (b) a RoBERTa-based classifier solve the task of natural language inference. We also leverage our pairs to build a plug-and-play neural generator that given a negated statement generates an affirmative interpretation. Then, we incorporate the pretrained generator into a RoBERTa-based classifier for sentiment analysis and show that doing so improves the results. Crucially, our proposal does not require any manual effort.

## introduction natural language understanding is a crucial component to build intelligent systems that interact with humans seamlessly. while recent papers sometimes report so-called superhuman performance, simple adversarial attacks including adding negation and other input modifications remain a challenge despite they are obvious to humans @xcite . further, many researchers have found that state-of-the-art systems struggle with texts containing negation. for example, @xcite show that pretrained language models such as bert @xcite do not differentiate between negated and non-negated cloze questions (e.g., birds cannot [mask] vs. birds can @xcite ). other studies show that transformers perform much worse in many other natural language understanding figure 1: parallel sentences from bitext corpora (english-norwegian and english-spanish) and backtranslations into english. either the original english sentence or the backtranslation contains a negation, and the other one is an affirmative interpretation. in this paper, we show that leveraging sentences with negation and their affirmative interpretations is beneficial for several natural language understanding tasks including natural language inference and sentiment analysis. in this paper, we address this challenge building upon the following observation: negation often carries affirmative meanings @xcite . for example, people intuitively understand that john read part of the book from john didn't read the whole book. our fundamental idea is to leverage a large collection of sentences containing negation and their affirmative interpretations. we define an affirmative interpretation as a semantically equivalent sentence that does not contain negation. we explore this idea by automatically collecting pairs of sentences with negation and their affirmative interpretations from parallel corpora and backtranslating. figure 1 exemplifies the idea with english-norwegian and english-spanish parallel sentences. note that (a) either the original english sentence or the backtranslation have a negation (the one that does not is the affirmative interpretation) and (b) the meaning of both is equivalent. armed with the large collection of sentences containing negation and their affirmative interpretations, we show that leveraging them yields improvements in three natural language understanding tasks. first, we address the problem of generating affirmatively interpretations in the afin benchmark @xcite , a collection of sentences with negation and their manually curated affirmative interpretations. second, we address natural language inference using three common benchmarks: rte @xcite @xcite , @xcite , and mnli @xcite . third, we address sentiment analysis using sst-2 @xcite . the main contributions of this paper are: foot_0 1. a large collection (153,273) of pairs of sentences containing negation and their affirmative interpretations. we present an automated procedure to get these pairs and an analysis of the negation types (single tokens, morphological, lexicalized, etc.). 2. experimental results with the t5 transformer @xcite showing that blending our pairs during the fine-tuning process is beneficial to generate affirmative interpretations from the negations in afin. ## related work solving natural language understanding tasks when the input text contains negation is challenging. researchers have approached negation processing mainly by identifying the scope @xcite and focus @xcite . scope refers to the part of the meaning that is negated and focus refers to the part of the scope that is most prominently negated @xcite . there are many works targeting scope detection @xcite @xcite @xcite and focus detection @xcite @xcite . while scope and focus pinpoint what is and what is not negated, they do not reveal affirmative interpretations as defined in this paper. additionally, there is limited empirical evidence showing that scope or focus is beneficial to solve a natural language understanding task. @xcite show that scope improves sentiment analysis, but they do not experiment with modern networks that may not benefit from explicit scope information. outside of scope and focus, jiang et al. (2021) work with commonsense implications involving negations (e.g., "if x doesn't wear a mask" then "x is seen as carefree"). closer to our work, @xcite pretrain bert with an unlikelihood loss calculated with automatically obtained negated statements. their negated statements do not preserve meaning. the authors show that their method, bertnot, outperforms bert with lama (kassner and schütze, 2020) and the same natural language inference corpora we work with. the work proposed here outperforms theirs (section 4.2) and does not require any manual effort. ## collecting sentences with negation and their affirmative interpretations this section outlines our approach to create a large collection of sentences containing negation and their affirmative interpretations. first, we present the sources of parallel corpora we work with. second, we describe our multilingual negation cue detector to identify negation cues in the parallel sentences. third, we describe the backtranslation step and a few checks to improve quality. lastly, we present an analysis of the resulting sentences with negation and their affirmative interpretations. ## conclusions negation poses a challenge for natural language understanding. understanding negation requires humans to infer affirmative meanings @xcite (e.g., the lot has not been vacant conveys the lot has been occupied). inspired by this insight, we collect a large collection (153k) of pairs of sentences containing negation and their affirmative interpretations. we define the latter as a semantically equivalent sentence that does not contain negation. our collection process relies on parallel corpora and backtranslation and is automated. we show that leveraging our collection is beneficial to solve three natural language understanding tasks: (a) generating affirmative interpretations, (b) natural language inference, and (c) sentiment analysis. all our experiments use out-of-domain, manually curated corpora. crucially, our proposal yields better results when negation is present in the input while slightly improving or obtaining comparable results when it is not. additionally, our proposal does not require any manual annotations.