Language models are not naysayers: An analysis of language models on negation benchmarks
Year: 2023

Abstract:
Negation has been shown to be a major bottleneck for masked language models, such as BERT. However, whether this finding still holds for larger-sized auto-regressive language models (“LLMs”) has not been studied comprehensively. With the ever-increasing volume of research and applications of LLMs, we take a step back to evaluate the ability of currentgeneration LLMs to handle negation, a fundamental linguistic phenomenon that is central to language understanding. We evaluate different LLMs — including the open-source GPTneo, GPT-3, and InstructGPT — against a wide range of negation benchmarks. Through systematic experimentation with varying model sizes and prompts, we show that LLMs have several limitations including insensitivity to the presence of negation, an inability to capture the lexical semantics of negation, and a failure to reason under negation.

heading: 1 Introduction
text: Despite being a core linguistic phenomenon, negation remains a major stumbling block for modern NLP architectures (Kassner and Schütze, 2020; Hossain et al., 2022). A reason for this could be that texts containing negation are underrepresented in training data of language models, as humans tend to express themselves using affirmative rather than negative expressions (Ettinger, 2020). Regardless, negation has been shown to be challenging even for humans to correctly interpret due to the diversity of forms across domains (Truong et al., 2022a). For instance, in clinical documents, many acronyms are used to denote negation such as NAD (no abnormality detected), and implicit negation abounds, such as normal chest x-ray scan, which implies the absence of an abnormality. Even more complex is the use of negation in combination with other linguistic phenomena such as quantifiers, gradable adjectives (not unattractive does not imply attractive)
∗Now at Google DeepMind.
(Truong et al., 2022b); licensing context (negative polarity items, e.g. any, either, yet, normally appear in certain negative grammatical contexts Warstadt et al. (2019)); downward entailment (A man owns a dog entails A man owns an animal but A man does not own a dog does not entail A man does not own an animal) (Geiger et al., 2020).
Traditionally, negation has been treated as a standalone problem, e.g. as negation detection (Chapman et al., 2001). The investigation of the impact of negation in various downstream tasks (Hossain et al., 2022; Hossain and Blanco, 2022a), or through probing (Ettinger, 2020) has revealed several limitations of modern large language models (“LLMs”) in handling negation. Given that LLMs are being adopted in an ever-growing range of tasks and have been shown to display emergent abilities for high-level tasks that require complex reasoning (Wei et al., 2022a), we are interested in exploring how the handling of negation has progressed.
In this work, we investigate the performance of auto-regressive language models on different negation-focused benchmarks. Instead of just looking at samples containing negation in common NLP datasets, we consider datasets in which negation plays an important role in making the correct judgement. In particular, we classify the benchmarks into three categories corresponding to the requisite negation reasoning abilities: (1) sensitivity to negation through cloze completion (fill-inthe-blank) queries of factual statements; (2) lexical semantics of negation through classification of antonym/synonym relationships; and (3) ability to reason with negation through language inference tasks.
We conduct extensive experiments using promptbased learning to facilitate zero- and few-shot evaluation of LLMs, and find the following:
• larger LMs are more insensitive to negation compared to smaller ones (Section 3);
101
• LLMs lack lexical semantic knowledge about negation, yielding almost random performance for synonym/antonym classification (Section 3);
• LLMs have limited ability to reason under negation, performing worse than random across most NLI datasets (Section 3). Only with the latest instruction fine-tuned model (Ouyang et al., 2022; Chung et al., 2022) do we observe above-chance performance (Section 3);
• For each dataset, we also experiment with prompt variations and find that in most cases, providing more information (context, instruction, simple wording) leads to a degradation in performance.

heading: 2 Experimental settings
text: In this section, we outline the settings that , including benchmark datasets, models to evaluate, and the prompts that were used. Our code is available at https://github.com/ joey234/llm-neg-bench.

heading: 2.1 Benchmarks
text: We use a range of benchmark datasets that exhibit the effects of negation across a wide range of tasks, in the form of either cloze completion or classification tasks. An overview of the datasets is presented in Table 1, categorized according to purpose and the type of negation they contain. Specifically, we focus on: (1) investigating whether LLMs are sensitive to the presence of negation in factual statements; (2) testing whether LLMs capture negation
in lexical semantics relations (synonym/antonym relations); and (3) investigating whether LLMs are able to reason under negation through multiple natural language inference benchmarks. We discuss the datasets in greater detail in Section 3.

heading: 2.2 Models
text: For the LLMs, we primarily focus on open-source auto-regressive LLMs with up to 6.7B parameters, including GPT-Neo (Black et al., 2021), and OPT (Zhang et al., 2022), which are claimed to be comparable in performance to similar-sized GPT-3 class models. Architecture-wise, they are both decoder-only PLMs pre-trained with a causal LM objective, with the main difference being in their pre-training corpora: GPT-neo was trained solely on the Pile dataset (Gao et al., 2020) consisting of 22 sub-datasets spanning different sources, whereas OPT was trained on the combination of datasets used in RoBERTa (Liu et al., 2019), Pile, and PushShift Reddit (Baumgartner et al., 2020). We use the official model checkpoints from HuggingFace hub,1 as detailed in Appendix A. We experiment with smaller-sized variants of these two classes of models to observe the effect of scaling on their performance over different benchmarks.
We also consider base GPT-3 (175B) (Brown et al., 2020), and its instruction fine-tuned variant InstructGPT (Ouyang et al., 2022), as well as a strong open-source instruction-tuned model FLANT5-XXL (11B) (Chung et al., 2022), to examine how recent commercial LLMs perform on negation.
1https://huggingface.co/models

heading: 2.3 Prompts
text: We adopt prompt-based learning to enable zeroand few-shot evaluation of LLMs (Radford et al., 2019). Given that LLMs have been found to be sensitive to prompt variation (Wei et al., 2021), and that more natural-sounding prompts correlate with model performance (Gonen et al., 2022), we also experiment with different types of prompts (see Table 2).
We use GPT-3 style prompts (Brown et al., 2020) as the Default setting. As handling negation plays an important role in all tasks, we also design prompts to prime the LLMs to focus more on the negation context, by introducing modifications specific to each task. In detail, for the cloze completion task MKR-NQ, we investigate whether a given model can detect the difference between two contrasting sentences (with/without negation). To achieve this, we prepend the prompt with the corresponding sentence without negation (Contrasting prompt). In addition, we also evaluate alternative prompts where we connect the two sentences with a discourse marker (Discourse prompt), or mask the main subject to encourage the model to attend more to negation cues (Mask prompt).
For antonym/synonym-related tasks (MWR,
SAR), we also experiment with simplifying the prompt and use descriptive terms rather than the formal names of the relations (e.g. antonyms, synonyms → opposite of, similar to), based on the intuition that these terms will appear more frequently in the pre-training data.
Finally, for classification tasks, we propose negation-aware prompting (Negation prompt) by modifying the prompts to explicitly state that the task involves reasoning about negation. Note that we explicitly include class options in the prompts to help reduce the effect of the surface form competition causing LLMs to assign lower probabilities to the correct answers (Holtzman et al., 2021).
For datasets with an accompanying training set (SAR, MoNLI), we also experiment with few-shot evaluation formulated as in-context learning by prepending the input prompts with 10 random samples from the training set.

heading: 2.4 Metrics
text: To evaluate cloze completion tasks, we employ Weighted Hit Rate (WHR) (Jang et al., 2022b), which measures the number of matches between the top-k predicted words and a given set of target wrong predictions, taking into account the predic-
tion probabilities:
WHRk(x,W ) = ∑k
i=1 ci × 1(wi ∈ Wx)∑k i=1 ci (1)
where Wx is the wrong prediction set of the input query x, and wi is the top i-th prediction with confidence score ci, obtained by taking the softmax of log probabilities p(wi|x) from the LM. Note that the model performance is better if there are fewer matches between models’ predictions and wrong completions, WHR is an error metric (lower is better). One problem with the WHR metric is that we can only evaluate using a fixed set of wrong predictions. Regardless, we believe the relative performance numbers are indicative of model performance.
For classification tasks, we evaluate using Accuracy, noting that all datasets are reasonably balanced.

heading: 3 Main findings
text: We summarize the main findings in this section. In general, the performance of GPT-neo and OPT follows a similar trend across all benchmarks (we present GPT-neo results; results of OPT models are in Appendix B).
Finding 1: Larger LMs are more insensitive to negation
MKR-NQ (Jang et al., 2022b) Masked Knowledge Retrieval – Negated Query (MKR-NQ) is a negated version of the LAMA dataset (Petroni et al., 2019), which contains lexicalized statements of triples in ConceptNet (Speer et al., 2017). This dataset contains factual statements with verbal negations (i.e. negators not, don’t are associated with the main verb of the sentence), e.g. Iburofen is a type of medicine. → Iburofen isn’t a type of medicine.
Each sample contains the query along with a set of wrong word completions, supporting the evaluation of the sensitivity of the model to negation by measuring how likely a model will generate incorrect completions. Note that MKR-NQ only considers sample sentences that contain a single verb, making it trivial to negate the original sentences.
Findings From Figure 1, which is based on LLMs with a negated factual statement (Default prompt), we observe relatively low hit rates (<
0.15) across all model sizes, and a clear inverse scaling trend between model sizes and their performance. The smallest variant (GPT-neo-125M) has the best performance, which is comparable to that of masked language model of a similar size (BERT-base, 110M parameters) (Jang et al., 2022b). This phenomenon reflects the finding that larger models tend to memorize the training data more (McKenzie et al., 2022; Jang et al., 2022a). Moreover, higher hit rates for top-1 predictions suggest that models predict wrongly with high confidence.
For Contrasting prompts, in which we prepend the negated statement with its non-negated version, we notice a drastic increase in WHR, showing that models are prone to repeating what is presented in the prior context, confirming the finding of Kassner and Schütze (2020). When a discourse term is added to connect the two sentences (Discourse prompt), we do not observe any improvement, and the performance of the largest model is even worse. To investigate whether this phenomenon is attributable to models not being able to detect the presence/absence of negation, we experiment with masking out the main noun/verb of the queries (Mask prompt). We observed even higher WHR, especially for the top-1 prediction in this setting. The results suggest that repetitions are caused more by LLMs being easily primed by repeating what is present in the previous context, than by generating words that are closely associated with the main subject of interest. This again shows that the models cannot differentiate between identical contexts, differing only on whether negation is present or absent (i.e., outputs tend to be similar with or without negation).
To further analyze the outputs, we calculate the perplexity (PPL) of the generated predictions to determine their plausibility (Wilcox et al., 2020). Here, we choose the model with the best WHR5 score on the MKR-NQ benchmark, and calculate the mean perplexity over all queries for each prompt type (5 completions for each query). PPL is calculated as the exponentiated average negative log-likelihood of a sequence, with exponent base e. As a point of reference, we calculated the average perplexity of the provided completion of the original non-negated dataset (denoted Corpus). From the reported perplexities (Table 3), we can see that Default output are the most plausible (with PPL markedly lower than Corpus), while Contrast-
ing is the least natural. The remaining prompts types (Discourse, Mask) are comparable to Corpus. These results show that LLMs can indeed generate plausible and human-like output for this task.
Finding 2: LMs fail to capture synonym/antonym lexical relations
MWR (Jang et al., 2022b) To test the ability of LMs to capture negative lexical semantics, we use MWR dataset, where models are asked to predict the antonym/synonym of a target word. The dataset was constructed by using the most frequent nouns, adjectives, and adverbs that appear in SNLI (Bowman et al., 2015), then choosing their corresponding synonyms and antonyms from ConceptNet (Speer et al., 2017). The dataset also contains different wordings for antonym-asking and synonym-asking queries (e.g. is the opposite of, is different from and is similar to, is a rephrasing of ) to test model sensitivity to prompt variations.
Findings From Figure 2, we can observe the same inverse scaling trend as for MKR-NQ using
the Default prompt, where the hit rate of the smallest model is around 0.02, better than previouslyreported SOTA results (Jang et al., 2022b). With a more natural query with more focus on the target words via quotation marks (Quote prompt), surprisingly, we noticed a drastic jump in hit rates. However, MWR may not be a good indicator of model performance due to how the task is framed. One problem is that models can generate words that are not in the given wrong prediction set, but are also irrelevant, and are also neither antonyms nor synonyms of the given target words, as demonstrated in Table 4.
SAR (Jang et al., 2022b) To further investigate the ability of LLMs to capture negative lexical semantics, we consider the antonym/synonym relation classification task (SAR). Different from the MWR cloze-style synonym/antonym prediction task, this benchmark is framed as a binary classification task of predicting the correct antonym or synonym relationship between two given words. Data is once again taken from ConceptNet, where triplets with synonym and antonym relations are extracted in equal numbers (1000 samples for each relation).
Findings In contrast to the high results for MWR, we find that for this task, model performance is equivalent to random, with accuracy fluctuating around 0.5 (Figure 3). For prompt variants, we do not observe any meaningful improvement, in that Simple follows a similar trend to Default and Negation performs better for larger models (2.7B and 6B). This is a huge degradation from previous fully fine-tuned results over encoder models. For instance, Jang et al. (2022b) reported that BERTlarge achieves 92.5% accuracy on SAR. We argue that this is a specific task that is not captured in the next token prediction training objective of LLMs and thus, requires explicit supervision.
Finding 3: LLMs are unable to reason under negation
NegNLI (Hossain et al., 2020) NegNLI contains 4500 premise–hypothesis pairs with important negation, where negation is essential in making the correct judgement about the relationship between the premise–hypothesis pairs. Samples are extracted from the commonly-used NLI datasets
(RTE Dagan et al. (2005), SNLI Bowman et al. (2015), MNLI Williams et al. (2018)), then the negator not is added to the main verb either in the premise, hypothesis, or both. Here, we consider each subset separately, as the number of classes are not the same, and denote them SNLI-neg, MNLIneg, RTE-neg.
MoNLI (Geiger et al., 2020) MoNLI is an NLI dataset focused on lexical entailment and negation. Specifically, the dataset investigates the downward monotonicity property where negation reverses entailment relations (e.g. dance entails move, but not move entails not dance). MoNLI was created by extending samples from SNLI by substituting the nouns by their hypernyms/hyponyms from WordNet (Miller, 1998).
NaN-NLI (Truong et al., 2022b) NaN-NLI is a test suite which focuses on sub-clausal negation, in which only part of the sentence’s meaning is negated, thus making it harder to correctly determine the correct negation scope (e.g. in Not the first time that they pulled that off the negation scope is only Not the first time and the main clause of the sentence they pulled that off is not negated). Each premise–hypothesis pair is constructed so that the corresponding hypotheses are constructed to reflect different interpretations that the negated instance in the premise are likely to be misunderstood for.
Findings Similar to the antonym/synonym classification task, the performance for most negationfocused NLI benchmarks is low. In particular, for all NLI datasets, the performance is generally lower than baseline. As shown in Figure 4, scaling up model size has almost no effect, and the largest model performs worse in many cases, even when the prompt explicitly states that the task requires reasoning about negation (Negation prompt). For datasets which include a training set (SAR, MoNLI), we also experimented with fewshot learning but did not observe any noticable improvement (Figure 5). One exception is that the 2.7B model seems to pick up some signal from the provided MoNLI training samples, but falls back again when we increase the model size to 6B.
Even with general NLI datasets, zero-shot applications of LLMs were previously shown to be roughly equivalent to a random baseline (Wei et al., 2021). When negation is involved, the task becomes even more complex. As pointed out in Brown et al. (2020), one possible reason that LLMs
struggle with NLI is that the samples consist of two disjoint sentences, which are unlikely to appear naturally in standard training corpora. We hypothesise that NLI is a generally hard task that requires substantially more supervision in order for models to detect meaningful patterns.
Finding 4: Instruction fine-tuning improves reasoning under negation
We further evaluate with GPT-3 class models of significantly larger scale (175B), which have been shown to achieve strong results in zero- and few-shot settings across a wide range of tasks (Brown et al., 2020). In detail, we benchmark the largest GPT-3 model (text-davinci-001: Brown et al. (2020)) and its variant InstructGPT, which is trained to follow human instructions using reinforcement learning (text-davinci-003: Ouyang et al. (2022)). The results can be found in Table 5.
For the base GPT-3 model, the results over most benchmarks are no better than much smaller language models (GPT-neo-125M). For clozecompletion tasks, consistent with the earlier-
observed trend of larger models performing worse, we observe higher (worse) WHR scores compared to that of smaller language models, confirming our finding that larger models are more insensitive to the presence of negation. Results get even worse with using the instruction fine-tuned model.
On the other hand, for most classification tasks, InstructGPT achieves better zero-shot results than other models. In addition, using this model in combination with explicit instruction about negation (Negation prompt) further improves performance, which we did not observe for other LLMs. It is, however, unclear what data the instruction-tuning process was performed on. Thus, the huge gain in performance could be attributed to the existence of similar patterns in the training set (i.e. explicit supervision over similar tasks). Interestingly, InstructGPT performance on MoNLI did not increase (it underperfomed other models). We hypothesize that this is due to an inductive bias from model’s ability to reason with hypernymy. For instance, the model can understand that “dog is an animal” (and therefore own an animal entails own a dog), but incorrectly generalizes this logic to a similar sample containing negation (not own a dog entails not own an animal). This is indeed true when we look at the explanation generated by ChatGPT, the subsequent model to InstructGPT (Figure 6).
We also experiment with the instruction-tuned FLAN-T5-XXL model (Chung et al., 2022) and find that the results are better than GPT-3 for most NLI tasks, despite being ∼16x smaller. These results suggest that instruction fine-tuning has much greater impact than model scaling in terms of models developing the ability to perform reasoning tasks under negation.

heading: 4 Related work
text: Our work builds upon previous research on negation. In particular, we were inspired by the pioneering works of Kassner and Schütze (2020) and Ettinger (2020), which reveal that pre-trained language models have a major issue in being insensitive to the presence of negation, based on evaluation over a set of cloze-style queries. Following this line of research, Jang et al. (2022b) also explored negation in a cloze completion context by negating factual statements extracted from ConceptNet and come to a similar finding.
In a broader context, Hossain et al. (2020, 2022) investigated the performance of BERT-based methods on samples containing negation in the GLUE (Wang et al., 2018) and SuperGLUE (Wang et al., 2019) datasets. Their main finding is that the results for the subsets containing only negation are lower than those without, as well as the whole test set, showing that models struggle with negation, even when fine-tuned on relevant training data. Ravichander et al. (2022) proposed the challenging CONDAQA dataset to test the ability of models to reason about the implications of negation. The authors conducted comprehensive analysis of different types of LLMs under different settings, and found that the best-performing models were still well below human performance. Negation has also been investigated as part of psycholinguistic probing datasets (Lialin et al., 2022; Jumelet et al., 2021; Staliūnaitė and Iacobacci, 2020). Contrasting previous finding, Gubelmann and Handschuh (2022) found that the ability to understand negation of LMs is underestimated in previous studied. Through designing a controlled dataset with minimal pairs varying in syntactic structure, gender, profession, and first name, they concluded that the models are indeed sensitive to negation and thus, their struggle comes more from the contextualization of the tasks.
As part of the analysis on emergent abilities of LMs, negation has been shown to be one of the tasks that displays a flat scaling curve (Wei et al., 2022a) or even inverse-scaling (McKenzie et al., 2022). This behaviour was later shown to be alleviated by instruction fine-tuning (Wei et al., 2022b). The effectiveness of instruction fine-tuning is further supported in Jang and Lukasiewicz (2023). The authors investigated the logical consistency of ChatGPT and found that ChatGPT understands negation and antonyms much better than previous
models. Beside probing and evaluation, there have also been works on making language models more robust to negation, including unlikelihood training (Hosseini et al., 2021), adaptive pre-training on relevant data (Truong et al., 2022a), leveraging affirmative interpretations from negation (Hossain and Blanco, 2022b), and learning better representation of negation through contrastive learning (Jiang et al., 2022; Wang et al., 2022).

heading: 5 Conclusion
text: We have shown that LLMs still struggle with different negation benchmarks through zero- and fewshot evaluations, implying that negation is not properly captured through the current pre-training objectives. With the promising results from instructiontuning, we can see that rather than just scaling up model size, new training paradigms are essential to achieve better linguistic competency. Through this investigation, we also encourage the research community to focus more on investigating other fundamental language phenomena, such as quantification, hedging, lexical relations, and downward entailment.

heading: 6 Limitations
text: First, regarding the experimental settings, the WHR metrics used to evaluate cloze completion tasks are imperfect, as we discussed. Framing cloze completion tasks in the style of multiple-choice question answering to limit the options that models are evaluated on would be a good direction to follow (Robinson et al., 2022). In addition, the prompt engineering in this work is in no way exhaustive, and could be extended using different prompt engineering strategies such as soft prompt tuning (Lester et al., 2021), or mining- and paraphrasing-based methods to generate high quality prompts (Jiang et al., 2020).
Second, due to computational constraints, we could not perform an extensive set of experiments for larger models like PaLM (with up to 540B parameters) (Chowdhery et al., 2022). Recent work by Wei et al. (2022b) has shown that the inverse scaling trend on several benchmarks can be alleviated using the large instruction fine-tuned models such as FLAN-PaLM-540B, which is largely in line with our findings regarding InstructGPT and FLAN-T5. With a small-scale experiment, we found that ChatGPT displayed strong performance
on challenging samples in the investigated benchmark, so the main findings of the paper may not hold true for newer LLMs.
Finally, this work only considers negation in the English language. There is every reason to believe that negation is an equally challenging problem in other languages. As this is a linguistically-intensive task, and requires native speakers to conduct thorough analysis of the results, we leave this for future work.

heading: Acknowledgement
text: The authors would like to thank the anonymous reviewers for their detailed, kind, and constructive reviews. This research was conducted by the Australian Research Council Training Centre in Cognitive Computing for Medical Technologies (project number ICI70200030) and funded by the Australian Government. This research was undertaken using the LIEF HPC-GPGPU Facility hosted at the University of Melbourne. This Facility was established with the assistance of LIEF Grant LE170100200.

heading: A Model checkpoints
text: For open-sourced LMs, we consider the official released checkpoints on the HuggingFace hub at:
• https://huggingface.co/EleutherAI/x
• https://huggingface.co/facebook/y
where x in {gpt-neo-125M,gpt-neo-1.3B,gpt-neo2.7B,gpt-j-6B}, and y in {opt-125m,opt-350m,opt1.3b,opt-2.7b,opt-6.7b}.
For GPT-3 models, we access them through the official API at https://openai.com/api/, using the Text completion endpoint. The considered model identifiers along with their sizes are:
• text-ada-001: 350M
• text-babbage-001: 1.3B
• text-curie-001: 6.7B
• text-davinci-001: 175B
• text-davinci-003: 175B

heading: B OPT results
text: For MWR, although we observe improvements with increasing model sizes, the WHR scores are much higher than those of GPT-neo, showing that OPT is worse at predicting antonyms and synonyms of words. The gap in performance may lie in differences in training data between the two types of models.
C Model outputs

heading: Future Work
text: As this is a linguistically-intensive task, and requires native speakers to conduct thorough analysis of the results, we leave this for future work