Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge (2023)

Abstract: Large language models (LLMs) have been widely studied for their ability to store and utilize positive knowledge. However, negative knowledge, such as “lions don’t live in the ocean”, is also ubiquitous in the world but rarely mentioned explicitly in text. What do LLMs know about negative knowledge?This work examines the ability of LLMs on negative commonsense knowledge. We design a constrained keywords-to-sentence generation task (CG) and a Boolean question answering task (QA) to probe LLMs.Our experiments reveal that LLMs frequently fail to generate valid sentences grounded in negative commonsense knowledge, yet they can correctly answer polar yes-or-no questions. We term this phenomenon the belief conflict of LLMs.Our further analysis shows that statistical shortcuts and negation reporting bias from language modeling pre-training cause this conflict.

## introduction most of the world knowledge exists in a positive and affirmative form @xcite @xcite . as a result, large language models (llms) pre-trained on a colossal amount of texts, such as gpt-3 @xcite and palm @xcite , have demonstrated their remarkable abilities for storing and utilizing positive knowledge in downstream tasks. in contrast, negative knowledge, such as the commonsense statement that "lions do not live in the ocean", is rarely mentioned in the textual world @xcite . 2 such negative knowledge also exists in the real world, and is important figure 1 : an example of the probing tasks studied in this paper. for the same negative commonsense @xmath0, located at, @xmath1, we find llms often fail to generate texts grounded in such negative knowledge while knowing its validity according to question answering. for cognitive skills such as knowing what is not true or what not to think @xcite @xcite . therefore, we ask this question: do llms (such as gpt-3 models) acquire such implicit negative knowledge through extensive language modeling pre-training? one important way of probing llms, which are mostly generative models, is checking whether the generated texts are knowledge-grounded. this is because the generation of texts is a direct manifestation of a model's internal beliefs towards world knowledge @xcite @xcite . 3 knowledgegrounded text generation has been a focus of nlp research @xcite . for example, the com-mongen benchmark @xcite evaluates generative commonsense reasoning that organizes concepts as keyword input and generates a sentence grounded in commonsense knowledge. however, previous work does not consider negative knowledge, nor do they probe the consistency between what models know and what they generate. another line of work on probing @xcite @xcite is conducted through the mask-infilling task. however, this task mainly evaluates bidirectional models @xcite , and is not natural for unidirectional llms. also, this task suffers from the open-world problem in evaluation, i.e., there could be multiple valid answers to fill the mask. this is vital for evaluating negative knowledge, which has an infinite answer space, e.g., lions don't live in the sky, water, desk, car, etc. in this study, we investigate the belief of llms about negative commonsense knowledge through the lens of text generation. since llms have become a foundational service @xcite and cannot be easily trained, we apply incontext learning @xcite for the probing tasks, which is tuning-free. we design a constrained sentence generation (cg) probing task, following @xcite , where the model must generate a knowledge-grounded sentence based on a given @xmath2, r, o>. for example, given a triple "<lion, located at, @xmath3, a model should generate "lions do not live in the ocean". this task is rather simple and clear. the output sentence basically contains the same information as the input keywords. thus, the generated texts are easy to evaluate according to the appearance of negation. we also add a boolean question answering (qa) task that asks llms whether a knowledge triple is valid, which shows their beliefs about this piece of knowledge. an example is given in figure 1 . in our experiments, we find that llms of different sizes and shapes often produce hallucinated claims of negative knowledge, even if they answer yes-or-no questions about it correctly. we term this phenomenon the belief conflict, i.e., actions (generating texts with it) conflict with its belief (answering question about it). hallucinated generation of negative knowledge is seen in both our probing tasks and downstream tasks, such as explanation generation @xcite , where negative knowledge plays an important role in the argumentation of refutation. further analysis shows that this problem stems from the statistical shortcuts and reporting bias of negation during pretraining. moreover, such implicit biases can be alleviated through explicit reasoning with chainof-thought prompting @xcite , such as syllogistic deduction and related fact comparison. the main contributions of this paper are summarized as follows: 1) we are the first to investigate llms' belief about negative knowledge in the commonsense domain, which may shed light on a previously unstudied aspect of llms' abilities. ## related work negative knowledge negative knowledge refers to information that describes what is not true, what cannot be done, or what does not exist, while everything that exists is positive @xcite . it plays an important role in the human reasoning process, because to think effectively, we need to know what "not to think" @xcite . current research of negative knowledge in nlp mainly focuses on developing negative knowledge bases that store relational negative commonsense knowledge @xcite @xcite and utilizing negative knowledge within arguments or explanations to refute a candidate @xcite @xcite . this paper is based on these resources to probe the belief of llms about the relations of everyday concepts that are not true. ## probing protocol in this section, we set up an evaluation protocol to understand what llms know about (negative) commonsense knowledge of everyday concepts. ## closing remarks in this study, we explored and quantified the limitations of llms in generating texts grounded in negative commonsense knowledge that they seem to know, a phenomenon we term as "belief conflict". to investigate this, we probe llms with a constrained sentence generation (cg) task, coupled with a qa task. our experiments demonstrated the existence of the belief conflict in all llms when it comes to negative knowledge, which is mostly brought by quantifiable statistical shortcuts such as keywords co-occurrence. we also see that this can be lessened by giving more in-context examples of negative knowledge or by using a chain-of-thought (cot) prompting method to explain the explicit reasoning process for deriving negative knowledge. with the rapid increase of the study on languagebased reasoning @xcite @xcite , there would be cause for concern if llms have trouble generating proofs or reasoning steps with negative knowledge. with all the good scores they achieve at qa tasks, whether they can be trusted with their knowledge expressed during generation, which is one of the most prominent way of human-ai interaction, is still questionable. in this sense, the study of negative knowledge creates a good testbed for assessing real languagebased reasoning skills for llms without the statistical heuristics they memorized. we hope that the findings in this work could raise the awareness of the community on negative knowledge for llms in downstream text generation tasks.