Paraphrasing in Affirmative Terms Improves Negation Understanding (2024)

Abstract: Negation is a common linguistic phenomenon. Yet language models face challenges with negation in many natural language understanding tasks such as question answering and natural language inference. In this paper, we experiment with seamless strategies that incorporate affirmative interpretations (i.e., paraphrases without negation) to make models more robust against negation. Crucially, our affirmative interpretations are obtained automatically. We show improvements with CondaQA, a large corpus requiring reasoning with negation, and five natural language understanding tasks.

## introduction negation is a fundamental linguistic phenomenon present in all human languages @xcite . language models underperform in various natural language understanding (nlu) tasks when the input includes negation. for example, @xcite and @xcite show that bert @xcite fails to distinguish between negated and non-negated cloze questions. researchers have also shown that large language models such as gpt-3 @xcite and in-structgpt @xcite are insensitive to negation and fail to reason under negation @xcite . @xcite point out that language models violate the logical negation property (p is true iff ¬p is false). @xcite analyze negation in eight popular corpora for six nlu tasks. they conclude that (a) nlu corpora have few negations compared to general-purpose texts and (b) the few negations in them are often unimportant. to our knowledge, condaqa @xcite is the largest benchmark (14,182 question-answer pairs from wikipedia) requiring reasoning over the implications of negations. in this paper, we paraphrase sentences with negation without using negation to make models for natural language understanding more robust when negation is present in the input. we will use the term affirmative interpretation to refer to paraphrases without negation (e.g., i am not sad: i am just ok, i am happy, etc.). appendix a provides examples of how affirmative interpretations differ from simple paraphrases. the main contributions of this paper are (a) strategies to generate and incorporate affirmative interpretations and (b) experimental results demonstrating that doing so yields better results. 1 in addition to condaqa, we experiment with five of the eight corpora analyzed by @xcite : commonsenseqa @xcite , @xcite , qnli @xcite , wic (pilehvar and camacho-collados, 2019), and wsc @xcite . 2 we do not experiment with the other three corpora because they do not contain any negation @xcite , there is no difference in results when negation is present @xcite 0.01 in macro f1), or has already been shown @xcite to benefit from affirmative interpretations @xcite . the corpora we experiment with are in english. related work early research on negation targeted detecting negating cues and generating semantic representations, usually by identifying the scope and focus @xcite @xcite @xcite . more recent works bypass formal representations. instead, they make neural models robust when the input contains negation. @xcite combine unlikelihood training and syntactic data augmentation to enhance the ability of bert to understand negation with negated lama (kass-ner and schütze, 2020). @xcite present a pretraining strategy designed for negation. unlike these works, we couple original inputs containing negation with affirmative interpretations. the first work on affirmative interpretations was by @xcite . @xcite present afin, a corpus of ≈3,000 sentences with negations and their affirmative interpretations. these two previous works are limited to generating affirmative interpretations from negations; they do not provide extrinsic evaluations. more recently, hossain and blanco (2022) present large-afin, over 153,000 pairs of sentences with negation and their affirmative interpretations obtained from parallel corpora via backtranslation. in this paper, we present strategies to generate affirmative interpretations that do not require parallel corpora or a machine translation system. moreover, we demonstrate that incorporating affirmative interpretations yields better results with condaqa and five other natural language understanding tasks. ## generating affirmative interpretations an affirmative interpretation generator is a system that takes a sentence with negation as its input and outputs an affirmative interpretation. the task is similar to paraphrase generation with an additional constraint: the output must not contain negation. we use two approaches to generate affirmative interpretations. the first one is an off-the-shelf t5 @xcite fine-tuned by @xcite with large-afin (section 1) to generate affirmative interpretations. we refer to this model as t5-hb, and to the affirmative interpretations generated by t5-hb as a hb . the second approach bypasses the need for a large collection of pairs of sentences with negation and their affirmative interpretations. it is based on the work by @xcite , who fine-tuned t5 on a paraphrase dataset obtained with chatgpt (419,197 sentences and five paraphrases per sentence). we refer to this model as t5-cg. note that it is trained to generate paraphrases-not affirmative interpretations. we obtain affirmative interpretations with t5-cg by generating five paraphrases and selecting the first one that does not contain negation. we refer to these affirmative interpretations as a cg . 3 for examples of a hb and a cg , see appendix d. we use all negation cues in condaqa to identify negation cues in our experiments. condaqa contains over 200 unique cues, including single words (e.g., inaction, unassisted, unknown), affixal negations (e.g., dislike, unmyelinated, unconnected, inadequate, impartial), and multiword expressions (e.g., a lack of, in the absence of, no longer, not at all, rather than). they also include multiple part-of-speech tags such as nouns (e.g., absence, nobody, inability), adverbs (e.g., indirectly, involuntarily, unexpectedly), determiners (e.g., neither, no, none), and verbs (e.g., cannot, refuse, exclude). ## experimental results we use roberta-large @xcite as the base model. in addition to experimenting with the original inputs for a task (e.g., passage and question from condaqa), we couple the original input with one affirmative interpretation of the sentence with negation (if any; no change otherwise). affirmative interpretations are concatenated to the original input after @xmath0. our approach is the same regardless of the type of negation. for implementation details, see appendix e and f. ## conclusion we have presented two strategies to generate and incorporate affirmative interpretations into models for natural language understanding. the idea is simple yet effective: complement inputs that contain negation with a paraphrase that does not contain negation. crucially, we have demonstrated that automatically obtained (noisy) affirmative interpretations yield improvements with (a) condaqa compared with a model with twice as many parameters pre-trained with ≈1m question-answer pairs from 20 existing corpora and (b) five nlu tasks. our methodology is architecture-and task-agnostic. in fact, the model to generate affirmative interpretations was tuned with out-of-domain corpora. future work. the methods we have presented are simple and effective, but they are not the only way to incorporate or generate affirmative interpretations. for example, one might be able to use llms such as gpt-4 or llama to generate affirmative interpretations. another interesting direction is to investigate the effect of affirmative interpretations on other nlu tasks, such as sentiment analysis or text classification. finally, it would be interesting to investigate the effect of affirmative interpretations on other languages, especially those with different word order or negation structures.