Investigating and Addressing Hallucinations of LLM s in Tasks Involving Negation (2025)

Abstract: Large Language Models (LLMs) have achieved remarkable performance across a wide variety of natural language tasks. However, they have been shown to suffer from a critical limitation pertinent to ‘hallucination’ in their output. Recent research has focused on investigating and addressing this problem for a variety of tasks such as biography generation, question answering, abstractive summarization, and dialogue generation. However, the crucial aspect pertaining to ‘negation’ has remained considerably underexplored. Negation is important because it adds depth and nuance to the understanding of language and is also crucial for logical reasoning and inference. In this work, we address the above limitation and particularly focus on studying the impact of negation in LLM hallucinations. Specifically, we study four tasks with negation: ‘false premise completion’, ‘constrained fact generation’, ‘multiple choice question answering’, and ‘fact generation’. We show that open-source state-of-the-art LLMs such as LLaMA-2-chat, Vicuna, and Orca-2 hallucinate considerably on all these tasks involving negation which underlines a critical shortcoming of these models. Addressing this problem, we further study numerous strategies to mitigate these hallucinations and demonstrate their impact.

## introduction despite the impressive performance achieved by recently developed large language models @xcite @xcite @xcite @xcite , their tendency to 'hallucinate' in the output critically hampers their reliability and trustworthiness. hallucination in the llm context corresponds to the generation of text that seems syntactically sound and correct but is factually incorrect or unfaithful to the source input @xcite @xcite . prior work has studied hallucination of llms in various scenarios such as open-ended text generation @xcite , question answering @xcite , abstractive summarization @xcite @xcite , machine translation @xcite , and dialogue generation @xcite . while the above studies are important, investigating the impact of 'negation' in llm hallucinations has remained underexplored. negation is important because it adds depth and nuance to the understanding of language. it helps understand the opposite or absence of a statement, providing a more precise and nuanced interpretation and it is also crucial for logical reasoning and inference. furthermore, we humans arguably use affirmative expressions (without negation) more often than expressions with negation @xcite ; this implies that texts containing negation could be underrepresented in the training/tuning data of the models making it even more important to study. with the aforementioned motivation, in this work, we focus on 'negation' and study its impact on llm hallucinations. prior work on negation has primarily studied classification tasks such as natural language inference and masked word prediction @xcite @xcite @xcite . however, it is also important to study generative tasks with state-of-the-art llms. to this end, we study negation in four tasks: (i) false premise completion (fpc), (ii) constrained fact generation (cfg), (iii) multiple-choice question answering (mcqa), and (iv) fact generation (fg). figure 1 illustrates examples of all four tasks. we provide a detailed description and the rationale behind ronaldinho has not been awarded the fifa world player of the year. (✗) golf balls are not smaller than basketballs because write five statements about rabindranath tagore, each containing a negation and presenting factual information. he was not a political activist, but he did write a poem called "chitto jetha bhayo shunno" (...), which was adopted as the anthem of the indian national congress (✗) ## related work we note that the above works investigating hallucinations lack comprehensively studying the crucial aspect of 'negation'. in addition to the reasons mentioned in section 1 for studying negation, we additionally note that negation also helps prevent misinterpretation of statements, i.e., without the ability to recognize negation, one might misunderstand the intended meaning of a sentence, leading to inaccurate responses. in summary, negation is a fundamental aspect of linguistic expression and thus comprehensively studying it is important. prior studies on negation have primarily focused on classification tasks like natural language inference and masked word prediction. @xcite propose to fine-tune bert with an unlikelihood objective and evaluate on negated lama dataset and show that by training bert with the resulting combined objective reduces the mean top 1 error rate to 4%. hossain et al. (2020) present an nli benchmark where the instances involve negation and evaluate language models. they show the models trained on the original benchmarks are not robust when negation is present in the evaluation instances. hossain and blanco (2022) collect pairs of sentences with negation and their affirmative interpretations and show that leveraging these pairs help roberta-based classifier improve the performance on natural language inference. they also use this data to develop a generator model that takes a negated statement and generates its affirmative interpretation. then, they use this generator with the roberta model to improve the performance on sentiment analysis dataset. @xcite study negation in logical reasoning context and inspect the step-by-step reasoning ability of the llms. the finding on this work is that the llms are not robust against lexical negation when performing cot-style reasoning. a more recent work @xcite study the performance of llms on transformed prompts of various datasets where the transformation is performed by replacing words like 'correct' with 'incorrect', 'appropriate' with 'inappropriate', and 'natural' with 'unnatural'. this transformation results in prompts such as "complete the given sentence with the inappropriate ending". different from these studies, in our work, we focus on hallucinations of llms and conduct a systematic evaluation and analysis with four different generative tasks motivated from real-world settings. additionally, to address this hallucination problem, we also study various mitigation strategies. ## evaluation tasks in this section, we provide a detailed description and the rationale behind studying all the tasks. ## experiments and results we experiment with various open-source state-ofthe-art llms including llama-2-chat @xcite , vicuna @xcite , and orca-2 @xcite . we experiment with the 13b parameter models and the evaluation set contains 300 instances in fpc task (150 each of false premise and correct premise prompts as detailed in section 4.1), 100 instances each in cfg and mcqa tasks, and 300 instances (5 each of without negation and with negation for all the 30 personalities) in fg task. note that all the tasks described in section 3 are focused on factuality; thus, we use the temperature of 0 during inference. ## conclusion motivated by the importance of 'negation' in language, we comprehensively studied hallucinations of llms with four evaluation tasks that deal with negation. specifically, we studied false premise completion, constrained fact generation, multiplechoice qa, and fact generation tasks. through extensive experiments, we demonstrated that opensource state-of-the-art llms including llama-2chat, vicuna-v1.5, and orca-2 hallucinate considerably on all these tasks, showing a critical shortcoming of these models. addressing this hallucination problem, we also studied various mitigation strategies and analyzed their effectiveness. our analysis also revealed an important negative impact of employing these mitigation strategies. in summary, our work underlined a critical shortcoming in the abilities of existing llms to deal with negation and studied ways to address that limitation.