When Truth Matters - Addressing Pragmatic Categories in Natural Language Inference ( NLI ) by Large Language Models ( LLM s) (2023)

Abstract: In this paper, we focus on the ability of large language models (LLMs) to accommodate different pragmatic sentence types, such as questions, commands, as well as sentence fragments for natural language inference (NLI). On the commonly used notion of logical inference, nothing can be inferred from a question, an order, or an incomprehensible sentence fragment. We find MNLI, arguably the most important NLI dataset, and hence models fine-tuned on this dataset, insensitive to this fact. Using a symbolic semantic parser, we develop and make publicly available, fine-tuning datasets designed specifically to address this issue, with promising results. We also make a first exploration of ChatGPT’s concept of entailment.

## introduction: "i didn't say that!' committing oneself to the truth of a certain claim always implies or suggests one's commitment to the truth of a number of other claims, and it precludes one's commitment to the truth of a second set of claims. this is the essence of the logical notion of entailment (here used synonymously with inference) and contradiction. for instance, somebody who claims "loral did harm national security" is also committed to the less specific claim "something or somebody did harm national security". the concept of inference is itself quite complex. as @xcite detail, valid inferences can be divided into deductively valid inferences, where it is not logically (see @xcite possible that the premise is true while the conclusion is false, and inductively valid inferences (also called abductions), where it is possible that the premise is true while the conclusion is false, but where the truth of the premise is in general a good reason for the truth of the conclusion. for two utterances to be able to (deductively or inductively) entail or contradict each other, they have to be of the correct pragmatic category. while assertions or claims are able to stand in these logical relationships, the same does not hold for other pragmatic kinds of utterances, such as questions or commands, as they do not involve making a claim that could be true or false and hence commit their author to the truth of certain claims that could then entail or contradict other claims. for instance, uttering (p) in example (1) does not commit the speaker to loral's harming of national securityshe's simply asking a question, not making a claim. hence if, after uttering (p) from (1), somebody replies with "so you claim that somebody did harm national security", the appropriate response would be "i didn't say that!". neither does she commit herself to everybody's, as a matter of fact, having a happy hanukkah when uttering (p) in (2), that is, wishing everyone a happy hanukkah: rather, she is expressing the wish that everyone is going to have a happy hanukka. committing oneself to some state of affairs, i.e., to express a claim that can be true or false, is not the kind of thing one does when uttering a question or a command (which is not to dispute that questions and commands come with specific presuppositions, including factual presuppositions, that need to be fulfilled for the speech act in question to succeed). most recently, the introduction of generaldomain, openly-available conversational systems such as chatgpt (openai) shows the need for such a distinction even more clearly: a chatbot, which collects its information from web resources but does not perform any reasoning steps itself, can falsely spread non-claims as claims if it cannot differentiate between the two. current nlp research conceives natural language inference (nli) as a three-way classification task between two sentences (or sentencefragments), called premise (p) and hypothesis (h). llms are trained to predict contradiction (p and h cannot both be correct), entailment (if p is correct, then h must be correct as well), or neutral (neither of the two). while much of the very early research focused on deductively valid inferences, more recent research has also taken into account inductive inference, which are called applied entailments @xcite or informal reasoning @xcite . the multi-genre natural language inference dataset (mnli) @xcite has arguably become the most widely used dataset for fine-tuning llms for nli. this means that many (perhaps the majority of) llms that are fine-tuned for the task of nli are fine-tuned using mnli and thereby pick up mnli's concept of inference. the instructions given to the crowdworkers who worked to create the dataset as well as explicit comments by the authors support the conclusion that mnli's target notion of entailment dovetails with the one detailed here, applicable to claims but not to questions and commands. however, mnli contains prompts that are questions, such as in (1), or commands, such as in (2), as well as fragments such as (3), which are entirely incomprehensible if they are presented, as in mnli, without any context. in fact, all examples are from mnli's training split with their respective gold-labels in brackets. hence, there seems to be a conceptual gap between the notion of entailment as explicitly embraced by the authors of mnli and the pragmatic kinds of some of the prompts used to create the dataset. in this paper, we study the extent of the phenomenon, the consequences that this set-up of mnli has for llms that are fine-tuned on mnli, and we explore ways to acquaint the llms with these core pragmatic categories. our paper makes three contributions. first, after detailing the notion of inference as well as the conceptual gap in mnli on a theoretical level (section 3), we empirically assess the extent of the phenomenon of non-assertive premises in mnli (section 4). second, relying on the existing semantic parser gkr, we show a promising path towards acquainting llms with these pragmatic categories (section 5). third, we publish both an expert-curated gold-standard evaluation dataset as well as 7 different fine-tuning datasets to further research in this field. 1 additionally, we also take the very first steps toward exploring chatgpt's concept of entailment. ## analyzing mnli's concept of inference after describing how current research in linguistics and logic conceives inference and separates it from presupposition, implicature, and paraphrase, and after situating mnli in the current way how nlp approaches the task of nli, we now detail the tension that we see in mnli's concept of inference. given how @xcite specify the tasks for the crowdworkers creating mnli, the goal seems to be premise-hypothesis pairs that are deductively valid. we give the part of the instruction that is relevant for entailment in bold (for the full instructions, see the appendix, section a): [...] the line will describe a situation or event. using only this description and what you know about the world: write one sentence that is definitely correct about the situation or event in the line. according to this passage, the hypothesis to be written should be such that it is definitively correct about the situation or event described in the premise. world knowledge is allowed to be used, presumably to make room for implicit but uncontroversial premises. from a logical point of view, this means that whenever the state of affairs described in the premise obtains, the one described in the hypothesis must obtain as well. hence, mnli seems indeed to follow earlier nli research and aim at inference in the deductive or inductive sense detailed above (sections 1 and 2.1). this also agrees with the stated goal of @xcite , according to which they are aiming at pairs where the hypothesis is "necessarily true or appropriate whenever the premise is true". however, a manual inspection of the collected examples shows a different picture: the dataset contains entailment and contradiction pairs with premises that are non-assertive because they express commands or questions, or because they are fragmentary beyond comprehension (see examples above (1), (2), and (3)). note that these examples are not resulting from cherry-picking: the creators of mnli deliberately selected bits of text at random from 10 different genres, emphasizing that they only applied minimal pre-processing (e.g., removing sentences with less than eight characters, mathematical formulae, bibliographical references, see @xcite ). no grammaticality checks or parsing of sentence types are done. hence, including incomprehensible fragments as well as questions and commands results from an explicit design decision by the authors. unfortunately, this design decision seems to be in tension with the instructions to the crowdworkers as well as with the stated goal to find premises that are true or appropriate whenever the premise is true. the examples (1), (2), and (3) do not contain premises that can be true or false, making it exceedingly difficult for the crowdworkers to follow the instructions and write a sentence that is definitively correct about the situation or event in the line: commands and questions do not aim to describe situations, incomprehensible fragments cannot describe such situations. the crowdworkers did their best. sometimes, as in (1), they developed a hypothesis that contradicts one of the presuppositions of a question (see, for example, (1)), developed a largely synonymous command for a premise containing a command (see example (2)), or just wrote down a concept that differs from the concept in the premise (see (3)). none of this, of course, amounts to developing entailment or contradiction pairs. in our pre-study, we try to quantitatively assess the extent of this problem and develop a solution for it. ## pre-study: non-assertive premises in mnli with this pre-study, we pursue two goals. first, we would like to obtain a more reliable estimate for the amount of non-assertive premises (that is, premises that do not express a determinate claim because they violate c1 or c2 from section 2.1 above) in mnli. second, our main experiment relies on gkr correctly categorizing premises from mnli that violate c1 or c2 (for details, see section 2.1) or that don't violate them and thus represent assertive sentences. thus, before starting with the main experiment, we conduct a pre-study to evaluate our choice of using gkr. we randomly select 1000 premise-hypothesispairs from mnli and submit the premises of each of the samples to the parser. the output of gkr (more specifically its context graph, where the type of sentence is specified) is then compared to our gold-standard annotations. the precision (p) and recall (r) results of this pre-study are shown in with our pre-study we find that 153 of the 1000 samples are non-assertive (based on their gold label). this suggests that approximately 15% of all pairs in mnli are indeed not assertions, meaning that they cannot entail or contradict any other assertions foot_3 . this however also means that any of these pairs having an entailment or a contradiction label (assuming a balanced dataset, this would mean around 10%) is indeed mislabeled as there can only be the neutral relation for non-assertions. note that even if we do not consider the gold labels but only the true positives of gkr (since gkr's output is what will be considered in the main experiment), the percentage of non-assertions in mnli only drops to 10% (97 out of 1000 samples are true positives). this would again mean that around 2/3 of these 10%, that is, some 6.6%, of mnli is incorrectly annotated. ## main experiment: probing llms for pragmatic understanding for our main experiment, based on the tension found in mnli's concept of inference (see above, section 3), we hypothesize that models fine-tuned on mnli lack any sensitivity to the fact that nonassertive premises cannot entail or be contradicted by other premises (research hypothesis 1), and that this deficit can be amended using properly composed fine-tuning datasets (research hypothesis 2). finally, we hypothesize that this does not significantly harm performance on the original mnli evaluation dataset (research hypothesis 3). to empirically test these hypotheses, we compile a number of fine-tuning datasets and evaluate llms fine-tuned on them both on a specific, hand-corrected dataset that only contains neutral premises as well as on the original mnli-matched evaluation dataset. ## outlook: exploration of chatgpt following recent advances in the area we are curious to see whether chatgpt, a general-purpose chatbot trained by openai foot_6 a) has a better notion of entailment than other llms, and b) can correctly identify non-assertive statements and treat them accordingly. to address these questions, we manually explore chatgpt: we manually prompt the chatbot with 96 premise-hypothesis-pairs with non-assertive premises according to gkr that were originally intended as entailment pairs by the creators of mnli. we join premise and hypothesis to obtain a question (see also appendix c). we find that, although chatgpt gets the general definition of logical entailment perfectly right ("entailment is a relationship between two propositions, in which the truth of the premise guarantees the truth of the conclusion", something that it tends to assert quite often, see the appendix, section b), it often fails to apply it to the given examples: it states that the hypothesis is logically implied (entailed) by the non-assertive premise in 54% of the cases. interestingly, from the remaining 46% of the cases, where chatgpt indeed answers negatively, i.e., that there is no entailment, we observe that in 27% of the samples the non-assertiveness of the premise stems from its being too fragmentary to express a specific proposition. this indicates that chatgpt is indeed able to tell when a premise is too incomplete to express a determinate claim. hence, this very small sample might suggest that chatgpt has a more accurate notion of entailment than the best models tested in the main experiment, but this does not fundamentally alter the scene: chatgpt can perform better with incomplete sentences, realizing that nothing can be logically inferred from them, but relatively poorly with questions or commands. ## conclusion in sum, we take the results of our experiments to be very encouraging. while llms that have been finetuned only on mnli show no sensibility for the fact that nothing follows logically from questions, commands, or incomprehensible fragments, fine-tuning on our datasets can address this potentially consequential shortcoming without losing too much accuracy on mnli-matched. in the future, we plan to experiment with more sophisticated fine-tuning methods and datasets to maximize performance on non-assertive prompts. furthermore, we hope to probe chatgpt using larger datasets, as the preliminary results obtained here are very interesting.