When Truth Matters – Addressing Pragmatic Categories in Natural Language Inference (NLI) by Large Language Models (LLMs)
Year: 2023

Abstract:
In this paper, we focus on the ability of large language models (LLMs) to accommodate different pragmatic sentence types, such as questions, commands, as well as sentence fragments for natural language inference (NLI). On the commonly used notion of logical inference, nothing can be inferred from a question, a command, or an incomprehensible sentence fragment. We find MNLI, arguably the most important NLI dataset, and hence models fine-tuned on this dataset, insensitive to this fact. Using a symbolic semantic parser, we develop and make publicly available, fine-tuning datasets designed specifically to address this issue, with promising results. We also make a first exploration of ChatGPT’s concept of entailment.

heading: 1 Introduction: “I didn’t say that!’
text: Committing oneself to the truth of a certain claim always implies or suggests one’s commitment to the truth of a number of other claims, and it precludes one’s commitment to the truth of a second set of claims. This is the essence of the logical notion of entailment (here used synonymously with inference) and contradiction. For instance, somebody who claims “Loral did harm national security” is also committed to the less specific claim “Something or somebody did harm national security”.
The concept of inference is itself quite complex. As Gubelmann et al. (2022) detail, valid inferences can be divided into deductively valid inferences, where it is not logically (see Plantinga 1974) possible that the premise is true while the conclusion is false, and inductively valid inferences (also called abductions), where it is possible that the premise is true while the conclusion is false, but where the truth of the premise is in general a good reason for the truth of the conclusion.
For two utterances to be able to (deductively or inductively) entail or contradict each other, they have to be of the correct pragmatic category. While
assertions or claims are able to stand in these logical relationships, the same does not hold for other pragmatic kinds of utterances, such as questions or commands, as they do not involve making a claim that could be true or false and hence commit their author to the truth of certain claims that could then entail or contradict other claims. For instance, uttering (P) in example (1) does not commit the speaker to Loral’s harming of national security – she’s simply asking a question, not making a claim. Hence if, after uttering (P) from (1), somebody replies with “So you claim that somebody did harm national security”, the appropriate response would be “I didn’t say that!”.
Neither does she commit herself to everybody’s, as a matter of fact, having a happy Hanukkah when uttering (P) in (2), that is, wishing everyone a happy Hanukkah: Rather, she is expressing the wish that everyone is going to have a happy Hanukka. Committing oneself to some state of affairs, i.e., to express a claim that can be true or false, is not the kind of thing one does when uttering a question or a command (which is not to dispute that questions and commands come with specific presuppositions, including factual presuppositions, that need to be fulfilled for the speech act in question to succeed).
(1) (P) Did Loral harm national security? (H) National security was not in danger. (contradiction)
(2) (P) Happy Hanukkah, everybody! (H) Everyone, have a happy Hanukkah! (entailment)
Properly distinguishing between pragmatic kinds of utterances that can and cannot stand in logical relations is important in several areas of application. First, we can consider the legal context, where fact and claim verification is of critical importance. In the setting of the automatic extraction of claims from testimonies, a system should be able to dis-
24
tinguish between claims and statements with other pragmatic functions such as questions, which do not commit their speakers to the truth of any claims (see, e.g., the overview in Ashley (2018)). Additionally, an essential application area is education: Using large language models (LLMs) to give formative feedback on students’ arguments requires that the LLMs be able to distinguish between claims made in the text, which can be used to infer other claims, and questions and commands, which cannot (see Rapanta et al. (2013) for an illustration of the importance for this logical concept of entailment in education science).
Most recently, the introduction of generaldomain, openly-available conversational systems such as ChatGPT (OpenAI) shows the need for such a distinction even more clearly: a chatbot, which collects its information from web resources but does not perform any reasoning steps itself, can falsely spread non-claims as claims if it cannot differentiate between the two.
Current NLP research conceives natural language inference (NLI) as a three-way classification task between two sentences (or sentencefragments), called premise (P) and hypothesis (H). LLMs are trained to predict contradiction (P and H cannot both be correct), entailment (If P is correct, then H must be correct as well), or neutral (neither of the two). While much of the very early research focused on deductively valid inferences, more recent research has also taken into account inductive inference, which are called applied entailments (Dagan et al., 2005) or informal reasoning (MacCartney, 2009).
The Multi-Genre Natural Language Inference Dataset (MNLI) Williams et al. (2018) has arguably become the most widely used dataset for fine-tuning LLMs for NLI. This means that many (perhaps the majority of) LLMs that are fine-tuned for the task of NLI are fine-tuned using MNLI and thereby pick up MNLI’s concept of inference. The instructions given to the crowdworkers who worked to create the dataset as well as explicit comments by the authors support the conclusion that MNLI’s target notion of entailment dovetails with the one detailed here, applicable to claims but not to questions and commands.
However, MNLI contains prompts that are questions, such as in (1), or commands, such as in (2), as well as fragments such as (3), which are entirely incomprehensible if they are presented, as in
MNLI, without any context. In fact, all examples are from MNLI’s training split with their respective gold-labels in brackets. Hence, there seems to be a conceptual gap between the notion of entailment as explicitly embraced by the authors of MNLI and the pragmatic kinds of some of the prompts used to create the dataset.
(3) (P) The kids. (H) The adults. (contradiction)
In this paper, we study the extent of the phenomenon, the consequences that this set-up of MNLI has for LLMs that are fine-tuned on MNLI, and we explore ways to acquaint the LLMs with these core pragmatic categories. Our paper makes three contributions. First, after detailing the notion of inference as well as the conceptual gap in MNLI on a theoretical level (section 3), we empirically assess the extent of the phenomenon of non-assertive premises in MNLI (section 4). Second, relying on the existing semantic parser GKR, we show a promising path towards acquainting LLMs with these pragmatic categories (section 5). Third, we publish both an expert-curated gold-standard evaluation dataset as well as 7 different fine-tuning datasets to further research in this field.1 Additionally, we also take the very first steps toward exploring ChatGPT’s concept of entailment.

heading: 2 Related Research

heading: 2.1 Inference in Logic and Semantics
text: Both deductive and inductive inferences require claims with determinate truth-values for their functioning. This means that it is necessary for any relationship of inference to be possible that both relata are constituted by a claim with determinate truth conditions: it needs to be clear in which situations premise and hypothesis are true. Otherwise, it would be impossible to assess whether the truth of the premise guarantees/makes reasonable the truth of the hypothesis, which is the essence of both deductive and inductive inferences. We propose that only sentences fulfilling the following conditions C1 and C2 can express such determinate claims.
C1 Only sentences whose pragmatic force is assertive can express determinate claims.
C2 Only assertions which are sensible (that is, where it is clear what has to be the case for
1To access the datasets, please consult: https://github. com/retoj/whentruthmatters.
the claim to be true) can express determinate claims.
C1 is violated by questions and commands, such as the premises in examples (1) and (2). The question “Did Loral harm national security?” lacks determinate truth conditions because questions cannot be true or false, but rather sensible or nonsensical. Similarly, uttering a command like “Happy Hanukkah!” does not aim to make a determinate claim about the state of affairs but rather aims to bring about a certain state of affairs.
We owe the insight that one can do different things with different types of sentences (the theory of so-called “speech acts") to Austin (1962, 1975) and Searle (1969, 1985), continuing a basically Wittgensteinian outlook Wittgenstein (2006/1953, §43).2 For a more recent survey of this approach, see Levinson (2017). Speech acts, like any actions, can succeed or fail to reach the goal that the agent intends with it. If some presuppositions for an act are not met, then it cannot possibly succeed.
With regard to fulfilling C2, the bare minimum needed for a sentence to express a claim with determinate truth conditions is some entity that is identified with sufficient precision (call it “subject”) as well as something, again sufficiently precise, that is predicated of that entity (call it “predicate”). This conception of a minimal claim as consisting of some specific entity of which something is said is a standard in logical textbooks, see, e.g., Smullyan (1968, 43) or Garson (2006, 29), but also in everyday human communications. For an overview, see Shapiro and Kouri Kissel (2021, sec. 2.2). For an influential contemporary statement of this minimal notion of a determinate claim, see Burge (2010, 537-547). For example, C2 is clearly violated by the premise of (3). Without any further context, and MNLI does not provide any such context, it is not clear whether the fragment “the kids” is intended as subject or predicate, but it is clear that one of the two is missing.
To see that nothing can be inferred from anything that violates C1 and C2, it is crucial to be aware of the distinction between inference and presupposition. For instance, one might be tempted to say that from the question (1), it can be inferred that Loral potentially endangers national security, which would contradict the hypothesis of this exam-
2Mastery of this multitude of language games in a flexible and adaptive manner is a key challenge for AI systems to come to really understand language, see Gubelmann (2023).
ple. This, however, would be to confuse inference with presupposition (the subtleties of the notion of presupposition, going back to Russell 1905 are still lively discussed in linguistics, see Dryer 1996 for a more recent influential contribution).
To claim that national security was never in danger would not, as the gold label for example (1) suggests, contradict the question: questions can be answered, rejected, ridiculed, etc., but not contradicted in the relevant logical sense. Rather, the claim would (at least on some readings of the question) show that the question fails to make proper sense, as one of its presuppositions, that national security was ever endangered, is not met.
A phenomenon similar to presuppositions has been described by Grice (1975) as conventional implicature.3 Roughly, conventional implicatures, unlike presuppositions, do not affect the sensibility of the utterance in question (this follows (Potts, 2015, 31), who argues that an implicature, unlike a presupposition, is independent from the primary content of the utterance.4 Consider example (4).
(4) (a) Bern, the capital of Switzerland, is the largest city of the country. (b) This is not true.
On a first level, the claim expressed by sentence (a) in example (4) is simply wrong: Zurich, not Bern, is the largest city of Switzerland. On the second level, however, it is also not the case that Bern is the capital of Switzerland: The founders of Switzerland deliberately avoided designating an official capital city due to existing rivalries between the candidates for such a role. This second level is beyond the reach of the challenge (b), it only reaches the actual claim being made about the relative population of Bern.
In fact, this availability for direct challenge is what helps to clearly identify the determinate, claimed content in an utterance – and it also helps to establish whether there is any such claimed content in the first place. With commands such as the one in example (2), you cannot respond with “This is not true”, nor can you do so in response to a question such as in (1), or in response to an incomprehensible fragment such as in (3). In contrast, to
3See Davis (2019) for an overview. For more recent discussions of the concept, see Potts (2004); Lepore and Stone (2010). For an original perspective on the concept from formal semantics, see Peters (1979).
4Karttunen and Peters 1979, in contrast, use the two concepts almost equivalently.
access and reject the presuppositions behind some of these utterances, one has to do more linguistic work. For instance, you could reject a possible presupposition of the question in example (1) with: “Your question is beside the point because Loral has newer produced anything else than toilet paper; therefore, the very idea that it could have been even a potential danger for national security is misguided.”
The second important distinction that we want to point out is that between inference and meaningpreserving paraphrase. The two relationships are orthogonal: If one claim with determinate truthconditions is the meaning-preserving paraphrase of another such claim, than they can be mutually inferred from each other. If, in contrast, what is being paraphrased is a question, a command, or an incomprehensible fragment, then no relationship of inference exists between the original and the paraphrase – regardless of how synonymous they are.
We would, finally, like to note that we recognize the usefulness of a broader, non-truthfunctional notion of entailment for uses beyond NLI. For instance, Groenendijk and Stokhof (1984, p.47f.,p481f.) define entailment between questions by resorting to a very general, non-truth-functional notion of entailment as a kind of semantic inclusion. Such a notion is very useful for question-answering or information retrieval tasks, but it is not how the NLI task was originally defined Dagan et al. (2005); MacCartney (2009), nor what the MNLI instructions to the crowdworkers specify, as we shall see.

heading: 2.2 Inference in NLP
text: LLMs based on the transformer architecture (Vaswani et al., 2017) have become the de facto standard in a variety of NLP tasks, including NLI. Highly successful architectures, starting with BERT (Devlin et al., 2019) and followed by others such as RoBERTa (Liu et al., 2019), XLNet (Yang et al., 2019), DeBERTa (He et al., 2020) as well as smaller versions such as DistilBERT (Sanh et al., 2019) and Albert (Lan et al., 2019), but also sequence-to-sequence architectures, e.g., T5 (Raffel et al., 2019) and BART (Lewis et al., 2020), have shown state-of-the-art performance on NLI.
Thanks to their sheer size, SNLI (Bowman et al., 2015), 570k premise-hypothesis pairs from image captions, and MNLI (Williams et al., 2018), 433k premise-hypothesis-pairs from 10 genres, written
and spoken, dominate the field, as their size is suitable for fine-tuning large LLMs. There is a number of studies that critically assess SNLI and MNLI for their bias. Williams et al. (2018) themselves note that their dataset contains a negation bias: if the hypothesis contains a negation, then it is more likely to be part of a contradiction pair (this bias is most likely due to the fact that simply negating the premise provides an efficient way for crowdworkers to create contradiction pairs). Poliak et al. (2018) systematically investigate the prospects of hypothesis-only approaches (methods that only consider the hypothesis for predicting the label) to NLI in different datasets, finding betterthan-random performance at most of them, which suggests the broad presence of statistical irregularities. Gururangan et al. (2018) show that SNLI and, to a lesser extent, MNLI, contain clues that make hypothesis-only approaches quite successful. Chien and Kalita (2020) focus on syntactic bias for LLMs fine-tuned on SNLI and MNLI, also finding that these bias are strong. Bernardy and Chatzikyriakidis (2019) argue that both SNLI and MNLI only cover a part of the entire range of human reasoning. In particular, they suggest that they do not cover quantifiers, nor strict logical inference. Furthermore, Pavlick and Kwiatkowski (2019), Zhang and de Marneffe (2021), and Jiang and de Marneffe (2022) all address the topic of disagreement among annotators. Jiang and de Marneffe (2022) focus on MNLI and suggest using a fourth category, namely “complicated”, along with the known ones of entailment, contradiction, and neutral. Similarly, Kalouli et al. (2019, 2023) discuss the annotation artifacts and quality of such datasets, especially concerning the distinction between neutral and contradiction pairs, and propose a refinement of the task.
We contribute to this ongoing research by focusing on the pragmatic categories of sentences (questions, commands, claims) which determine whether they can stand in the logical relationships introduced above (section 2.1). We use GKR (Kalouli and Crouch, 2018; Kalouli, 2021) to automatically categorize premises from MNLI that violate C1 or C2. GKR (Graphical Knowledge Representation) is the semantic representation generated by the corresponding parser. In GKR the sentence information is split into six subgraphs: a) the dependency graph holding the syntactic dependencies, b) the lexical graph holding lexical information such as synonyms and antonyms of the words of the
sentence, c) the properties graph holding morphosyntactic information such as the numerus of nouns and quantifiers, d) the concept graph holding the basic predicate-argument-structure of the sentence, the “who-is-doing-what-to-whom" information, e) the context graph making existential commitments over the concepts of the concept graph, e.g., for the sentence “the dog is not eating the bone" it says not only that there is the concept of eating involved in the sentence, but it also commits to its non-existence, its non-instantiation (due to the negation) and f) the coreference graphs capturing coreference links between entities.
In addition to its performance, what makes this parser particularly suitable for our goal is that it also identifies the type of sentence that is being parsed (assertion, question, or command). By default, the parser also categorizes subject-less sentences as imperatives (which helps to identify incomprehensible sentence fragments).

heading: 3 Analyzing MNLI’s Concept of Inference
text: After describing how current research in linguistics and logic conceives inference and separates it from presupposition, implicature, and paraphrase, and after situating MNLI in the current way how NLP approaches the task of NLI, we now detail the tension that we see in MNLI’s concept of inference.
Given how Williams et al. (2018, 1114) specify the tasks for the crowdworkers creating MNLI, the goal seems to be premise-hypothesis pairs that are deductively valid. We give the part of the instruction that is relevant for entailment in bold (for the full instructions, see the Appendix, section A):
[...] The line will describe a situation or event. Using only this description and what you know about the world: Write one sentence that is definitely correct about the situation or event in the line.
According to this passage, the hypothesis to be written should be such that it is definitively correct about the situation or event described in the premise. World knowledge is allowed to be used, presumably to make room for implicit but uncontroversial premises. From a logical point of view, this means that whenever the state of affairs described in the premise obtains, the one described in the hypothesis must obtain as well. Hence, MNLI seems indeed to follow earlier NLI research and aim at inference in the deductive or inductive sense detailed
above (sections 1 and 2.1). This also agrees with the stated goal of Williams et al. (2018), according to which they are aiming at pairs where the hypothesis is “necessarily true or appropriate whenever the premise is true”.
However, a manual inspection of the collected examples shows a different picture: the dataset contains entailment and contradiction pairs with premises that are non-assertive because they express commands or questions, or because they are fragmentary beyond comprehension (see examples above (1), (2), and (3)). Note that these examples are not resulting from cherry-picking: The creators of MNLI deliberately selected bits of text at random from 10 different genres, emphasizing that they only applied minimal pre-processing (e.g., removing sentences with less than eight characters, mathematical formulae, bibliographical references, see (Williams et al., 2018, 114f.)). No grammaticality checks or parsing of sentence types are done. Hence, including incomprehensible fragments as well as questions and commands results from an explicit design decision by the authors.
Unfortunately, this design decision seems to be in tension with the instructions to the crowdworkers as well as with the stated goal to find premises that are true or appropriate whenever the premise is true. The examples (1), (2), and (3) do not contain premises that can be true or false, making it exceedingly difficult for the crowdworkers to follow the instructions and write a sentence that is definitively correct about the situation or event in the line: Commands and Questions do not aim to describe situations, incomprehensible fragments cannot describe such situations.
The crowdworkers did their best. Sometimes, as in (1), they developed a hypothesis that contradicts one of the presuppositions of a question (see, for example, (1)), developed a largely synonymous command for a premise containing a command (see example (2)), or just wrote down a concept that differs from the concept in the premise (see (3)). None of this, of course, amounts to developing entailment or contradiction pairs.
In our pre-study, we try to quantitatively assess the extent of this problem and develop a solution for it.

heading: 4 Pre-Study: Non-Assertive Premises in MNLI
text: With this pre-study, we pursue two goals. First, we would like to obtain a more reliable estimate for the amount of non-assertive premises (that is, premises that do not express a determinate claim because they violate C1 or C2 from section 2.1 above) in MNLI. Second, our main experiment relies on GKR correctly categorizing premises from MNLI that violate C1 or C2 (for details, see section 2.1) or that don’t violate them and thus represent assertive sentences. Thus, before starting with the main experiment, we conduct a pre-study to evaluate our choice of using GKR.
We randomly select 1000 premise-hypothesispairs from MNLI and submit the premises of each of the samples to the parser. The output of GKR (more specifically its context graph, where the type of sentence is specified) is then compared to our gold-standard annotations. The precision (P) and recall (R) results of this pre-study are shown in Table 1. The table also compares the results to a simplistic baseline approach, where we only count as non-assertive all premises that end with a question mark (?) or an exclamation mark (!).
Table 1 shows that GKR’s precision is at 83.6% and its recall at 63.3%, when it comes to identifying non-assertive premises that cannot stand in a logical relationship. Identifying assertive premises is achieved with an even higher precision and recall (93.6% and 97.7%, respectively). These results heavily outperform the simplistic baseline approach, in which both the precision and the recall for non-assertive premises do not overcome chance. This is not surprising though. First, in MNLI, questions and commands are not always marked with question and exclamation marks, respectively, so this is no reliable method. Second, many questions do have a question mark, but are direct speech embedded in indirect speech, e.g., How much? asked the northerner. In these cases, the premises are indeed assertions (with an embedded non-assertive content). Third, there are premises with exclamation marks, which are no commands, e.g., You were
just wonderful!. Finally, this simplistic approach cannot capture any cases of fragmentary premises. These results confirm the quality of the GKR parser and the need for such a tool.
With our pre-study we find that 153 of the 1000 samples are non-assertive (based on their gold label). This suggests that approximately 15% of all pairs in MNLI are indeed not assertions, meaning that they cannot entail or contradict any other assertions5. This however also means that any of these pairs having an entailment or a contradiction label (assuming a balanced dataset, this would mean around 10%) is indeed mislabeled as there can only be the neutral relation for non-assertions. Note that even if we do not consider the gold labels but only the true positives of GKR (since GKR’s output is what will be considered in the main experiment), the percentage of non-assertions in MNLI only drops to 10% (97 out of 1000 samples are true positives). This would again mean that around 2/3 of these 10%, that is, some 6.6%, of MNLI is incorrectly annotated.

heading: 5 Main Experiment: Probing LLMs for Pragmatic Understanding
text: For our main experiment, based on the tension found in MNLI’s concept of inference (see above, section 3), we hypothesize that models fine-tuned on MNLI lack any sensitivity to the fact that nonassertive premises cannot entail or be contradicted by other premises (research hypothesis 1), and that this deficit can be amended using properly composed fine-tuning datasets (research hypothesis 2). Finally, we hypothesize that this does not significantly harm performance on the original MNLI evaluation dataset (research hypothesis 3). To empirically test these hypotheses, we compile a number of fine-tuning datasets and evaluate LLMs fine-tuned on them both on a specific, hand-corrected dataset that only contains neutral premises as well as on the original MNLI-matched evaluation dataset.

heading: 5.1 Models
text: We use three transformer-based models that are already fine-tuned on MNLI, delivering very good performance on this dataset, and that differ substantially in their architecture. We deliberately choose
5Since this was tested on a random sample, we can expect that this percentage will be similar for any other random sample.
models based on different architectures, sizes and fine-tuning methods.
The reason why we fine-tune models that are already fine-tuned on MNLI is that we assume that our non-assertive dataset is not large enough on its own for learning inference. Thus, we use models that are first fine-tuned on the large MNLI dataset and have thereby acquired a basic understanding of the task. Then, we fine-tune them further on the smaller non-assertive dataset to sharpen their conceptions of entailment and contradiction so that they do not predict entailment or contradiction, but rather neutral, when the premise is non-assertive. To test our assumption, we also include a version of RoBERTa-large that is not already fine-tuned to MNLI in our fine-tuning experiments. If it performs better than the others who are already finetuned to MNLI, our assumption is falsified, otherwise, it is verified.
The models are DeBERTa-base (He et al., 2020), XLNET-base (Yang et al., 2019), both 110M parameters, and RoBERTa-large (Liu et al., 2019), 355M parameters. Our DeBERTa-model is fine-tuned to MNLI using the method proposed by Reimers and Gurevych (2019), the XLNET-model by the adversarial method proposed in Morris et al. (2020), and for RoBERTa, we use the original fine-tuned version by Liu et al. (2019). We obtain all of our models from Huggingface (Wolf et al., 2019).

heading: 5.2 Datasets
text: We run GKR over randomly chosen premises of the train split of MNLI. As compute time per sample is rather high (about 30 sec per sample in our setting), we stopped the process after receiving 1875 premises that GKR classified as either interrogative or imperative (a label also given to sentence fragments lacking a subject), and hence non-assertive. The same run also yielded 8546 premises that GKR classified as assertions. Based on this, we develop a manually corrected evaluation dataset as well as a number of systematically varied fine-tuning datasets.
Evaluation-Datasets We use a subpart of the 1875 non-assertive premises, namely 636 premises, to compile an evaluation dataset: We manually verify that these premises are indeed not expressing a determinate claim (either because they are questions, commands, or incomplete beyond understanding), resulting in 406 premises. We then select the three premise-hypothesis-pairs correspond-
ing to each premise in the MNLI dataset (for each premise there was an entailment, a contradiction and a neutral pair created). This results in 1218 pairs whose correct relationship should be neutral due to their premise, but which were written up by crowdworkers to be evenly split among the labels of entailment, contradiction, and neutral.6 We call this evaluation dataset “GKR-n” for “GKRneutral”. The second dataset that we use to evaluate our fine-tuned models is MNLI-Matched (“MNLIM”), the matched evaluation dataset provided by Williams et al. (2018).
Fine-Tuning-Datasets We compile 8 different fine-tuning datasets, each consisting of a train split containing 6000 samples and a validation split containing 600 samples, evenly distributed across the three labels. In addition to a dataset that solely consists of unfiltered MNLI-train samples (mnli_u) as well as a dataset consisting solely of samples whose premises GKR classified as assertive (GKR_a), we compile six datasets combining these two sources (see Table 2). These six datasets are combinations from two different datasets for entailment and contradiction labels (entailment-contradiction 1 & 2, in short, ec1 & ec2) with three different datasets for neutral labels (neutral 1,2, 3: n1,n2,n3).
For example, the dataset ec2_n2 in Table 2 is composed of 6600 samples in total, of which 2200 are entailment, 2200 contradiction, and 2200 neu-
6Note that some premises in MNLI are used in more than three pairs, leading to minimal distortions in the scaling up.
tral. Of the entailment and contradiction samples, making up 4400 samples, 2/3 are such that GKR has identified their premises as assertions (2nd column in T 2). Furthermore, 2/3 of the neutral samples are such that GKR has identified their premise as non-assertive (3rd column in Table 2).
The basic idea behind fine-tuning on these 8 different datasets is to see what factors influence performance on the evaluation dataset: ec1 differs from ec2 in containing only 1/3 as opposed to 2/3 of pairs whose premises have been confirmed by GKR to be assertive. From n1 over n2 to n3, the portion of neutral pairs with non-assertive premises increases from 1/3 over 2/3 to 3/3. Testing all combinations of these datasets allows us to determine whether assertive premises in entailment and contradictions samples are more important than non-assertive premises in neutral samples when it comes to performance on the evaluation dataset, where all of the labels should be neutral due to the non-assertive premises of all the samples used there. Including the mnli_u dataset as fine-tuning dataset allows us to test whether our fine-tuning method leads to any distortions: unless our fine-tuning method is flawed, models fine-tuned on mnli_u should perform on MNLI-M approximately as they did before fine-tuning. After all, here we just perform further fine-tuning with the same data that has been used for the original fine-tuning run.7

heading: 5.3 Fine-Tuning Experiment
text: We fine-tune the four different transformer-based LLMs on a single GPU of a DGX-2 cluster. We fine-tune each model on each dataset for two epochs, using the trainer API provided by Huggingface. The results shown are the average over three fine-tuning runs per model and dataset. We use a batch size of 8 throughout, and we begin with an initial learning rate of 2e-5.

heading: 5.4 Results & Discussion
text: The results of our experiment are shown in Figure 1. We give the individual models’ results on the two evaluation datasets sorted by the fine-tuning dataset that was used.
With regard to our decision to use models previously fine-tuned to MNLI (see above, 4.1), the
7Note that it would not be useful for our purposes to convert non-assertive premises to assertions. We need to have them marked as neutrals/non-assertive as systems need to be able to differentiate between these pragmatic functions (see discussion in section 1).
results confirm our assumptions: roberta-large performs overall 28% worse than the version of the model that was previously fine-tuned to MNLI (abbreviated by roberta-lmnli in figure 1). As a consequence, we do not consider it in our presentation and discussion of results anymore.
Figure 1 shows that the most important factor for performance on the GKR-n evaluation dataset is the portion of neutral samples that are neutral because their premise is non-assertive. The accuracy of all models is 32% on average, and hence almost exactly random, if no such samples have been specifically selected and added to the fine-tuning dataset (as is the case in mnli_u as well as GKR_a, see the first and the penultimate columns, respectively). This accuracy increases steadily if the portion of neutral samples of said kind is increased from n1, n2, to n3, where it reaches 89% for roberta-largemnli. Performance on MNLI-matched decreases from n1 to n2 and n3, but in much smaller steps: from 86% to 82% to 77%.
These results allow for three main insights. First, without fine-tuning on our datasets, the LLMs do indeed fail to show any sensitivity for the fact that questions, orders, or incomprehensible fragments cannot entail or contradict anything. This follows from the random accuracy that the models reach after being fine-tuned on mnli_u, it confirms our first hypothesis, and it lends further support to the tension found in MNLI’s concept of inference. Considering the fact that MNLI is the de facto standard fine-tuning dataset, this means that the standard method used currently to fine-tune LLMs to NLI tasks very likely results in models that falsely classify pairs such as (1), (2), or (3) as contradicting or entailing each other.
The second insight, confirming our second hypothesis, is that our fine-tuning approach shows much promise in getting the models to understand that nothing follows logically from non-assertive premises. In particular, this applies to the models fine-tuned to ec1_n3, i.e., to a dataset that contains 1/3 of pairs with a premise that is assertive according to GKR in entailment and contradiction and 3/3 of pairs with non-assertive premises in neutral.
The third insight is that acquiring this sensitivity does not take a heavy toll on the accuracy of MNLI-matched, with an effective accuracy difference of 3.3%. While the performance difference seems greater at first sight (10% from 87% to 77%), it must be noted that 10% of the MNLI-premises
are anyway non-assertive (see Section 4) and thus those of them (approximately 2/3, 6.6%) that are labeled as entailments or contradictions are mislabeled, yielding an actual loss in accuracy of 3.3%. We emphasize that performing well at GKR-n requires that the LLMs predict neutral for any of the pairs in that dataset (because their premises are all non-assertive), while performing well at MNLI-M of course requires to predict all of the three labels with equal frequency. It is reassuring that the same LLMs manage to perform well at both evaluation datasets, confirming our research hypothesis 3.

heading: 6 Outlook: Exploration of ChatGPT
text: Following recent advances in the area we are curious to see whether ChatGPT, a general-purpose chatbot trained by OpenAI8 a) has a better notion of entailment than other LLMs, and b) can correctly identify non-assertive statements and treat them accordingly. To address these questions, we manually explore ChatGPT: We manually prompt the Chatbot with 96 premise-hypothesis-pairs with non-assertive premises according to GKR that were originally intended as entailment pairs by the creators of MNLI. We join premise and hypothesis to obtain a question (see also Appendix C).
We find that, although ChatGPT gets the general definition of logical entailment perfectly right (“entailment is a relationship between two propositions, in which the truth of the premise guarantees the truth of the conclusion", something that it tends to
8See this blogpost, last consulted on December 14, 2023. All interaction occurred on December 14, 2022, via OpenAI’s research interface: https://chat.openai.com/chat.
assert quite often, see the Appendix, section B), it often fails to apply it to the given examples: it states that the hypothesis is logically implied (entailed) by the non-assertive premise in 54% of the cases. Interestingly, from the remaining 46% of the cases, where ChatGPT indeed answers negatively, i.e., that there is no entailment, we observe that in 27% of the samples the non-assertiveness of the premise stems from its being too fragmentary to express a specific proposition. This indicates that ChatGPT is indeed able to tell when a premise is too incomplete to express a determinate claim. Hence, this very small sample might suggest that ChatGPT has a more accurate notion of entailment than the best models tested in the main experiment, but this does not fundamentally alter the scene: ChatGPT can perform better with incomplete sentences, realizing that nothing can be logically inferred from them, but relatively poorly with questions or commands.

heading: 7 Conclusion
text: In sum, we take the results of our experiments to be very encouraging. While LLMs that have been finetuned only on MNLI show no sensibility for the fact that nothing follows logically from questions, commands, or incomprehensible fragments, fine-tuning on our datasets can address this potentially consequential shortcoming without losing too much accuracy on MNLI-matched. In the future, we plan to experiment with more sophisticated fine-tuning methods and datasets to maximize performance on non-assertive prompts. Furthermore, we hope to probe ChatGPT using larger datasets, as the preliminary results obtained here are very interesting.
Limitations
We see two main limitations of this work. First, it is clear that the quality of the proposed fine-tuning method (with the non-assertive datasets) heavily depends on the quality of the GKR parser, used for the detection of non-assertive statements. Since the parser has an F-score of 84.3%, as evaluated within this work, it is to be expected that some of the examples used for fine-tuning will be faulty (mistakenly classified as non-assertive and thus mistakenly classified as neutrals). This means that the models are not given entirely clean input to learn from and thus, their performance could further improve if the quality of the GKR parser (or another similar parser) could improve. Note however that this only concerns the fine-tuning datasets and not the evaluation dataset, as this was manually checked for correctness (and thus, there is no impact on the results we report). The second limitation concerns our experimentation with ChatGPT: we were only able to experiment with a small amount of samples (due to the extremely expensive manual effort required) and thus, our findings will need to be reproduced through a larger-scale study. However, our findings agree with Bang et al. (2023), who find that ChatGPT performs reasonably well on deductive and abductive inference tasks.
Ethics Statement
For fine-tuning our models, we have used parts of the existing MNLI dataset. Thus, we see no additional risk of adding personally protected information, offensive material, or biases that could discriminate against marginalized groups.

heading: A Full Instructions Given to Crowdworkers
text: Williams et al. (2018, 1114) specify the following tasks for the crowdworkers:
“This task will involve reading a line from a non-fiction article and writing three sentences that relate to it. The line will describe a situation or event. Using only this description and what you know about the world:
• Write one sentence that is definitely correct about the situation or event in the line.
• Write one sentence that might be correct about the situation or event in the line.
• Write one sentence that is definitely incorrect about the situation or event in the line. "

heading: B ChatGPT on logical relationships involving commands and hopelessly incomplete premises
text: All interaction occurred on December 14, 2022, via OpenAI’s research interface: https://chat. openai.com/chat.
Human: Is there a relation of logical entailment between "What is the capital of france?" and "Paris"?
ChatGPT: Yes, there is a relation of logical entailment between the question "what is the capital of France?" and the answer "Paris". In logic, entailment is a relationship between two propositions (statements or sentences) in which the truth of one proposition (the premise) guarantees the truth of the other proposition (the conclusion). In other words, if the premise is true, then the conclusion must also be true.
In the case of the question "what is the capital of France?" and the answer "Paris", the premise is the information about the country of France, and the conclusion is the name of its capital city. Since we know that Paris is the capital of France, the truth of the premise (the information about France) logically entails the truth of the conclusion (the name of its capital city). In other words, if the premise is true (i.e. if we are asking about the capital of France), then the conclusion must also be true (i.e. the answer is "Paris").
Human: Is there a relation of logical entailment between "the kids" and "the adults"?
ChatGPT: No, there is no relation of logical entailment between the phrases "the kids" and "the adults". In logic, entailment is a relationship between two propositions (statements or sentences) in which the truth of one proposition (the premise) guarantees the truth of the other proposition (the conclusion). In other words, if the premise is true, then the conclusion must also be true. In the case of the phrases "the kids" and "the adults", there is no logical connection or relationship between the two. The phrases refer to two different groups of people, and there is no inherent logic that would guarantee that one group must be true if the other is true. For example, if we know that "the kids" are present, it does not necessarily follow that "the adults" are also present. Therefore, there is no relation of logical entailment between these two phrases.
Human: Is there a relation of logical entailment between "Close the door now!" and any other sentence?
ChatGPT: Yes, there is a potential relation of logical entailment between the order "Close the door now!" and any other sentence that provides information about the intended action or outcome of the order. In logic, entailment is a relationship between two propositions (statements or sentences) in which the truth of one proposition (the premise) guarantees the truth of the other proposition (the conclusion). In other words, if the premise is true, then the conclusion must also be true. In the case of the order "Close the door now!", the premise is the command to close the door, and the conclusion is the intended action or outcome. Therefore, any sentence that provides information about the intended action or outcome of the order can be considered the conclusion of the entailment relationship. For example, if the order "Close the door now!" is followed by the sentence "The door is now closed", this sentence can be considered the conclusion of the entailment relationship, because the truth of the premise (the command to close the door) logically entails the truth of the conclusion (the door is now closed).
In general, the relation of logical entailment between "Close the door now!" and any other sentence will depend on the specific information provided by the other sentence. If the sentence provides information that is logically entailed by the premise of [here the response from ChatGPT ended rather abruptly].

heading: C Full Output from ChatGPT Prompting with non-assertive premises
text: In tables 3, 4, and 5, we can see the predictions of ChatGPT for a selection of 96 pairs with nonassertive premises from the validation split of our dataset. For each of the pairs, we prompted ChatGPT with the following template:
(5) Does "PREMISE" entail "HYPOTHESIS"?
ChatGPT always responded with a clear, albeit sometimes somewhat verbose, yes (represented with “1” in the tables) or no (represented with “0” in the tables). The tables show that ChatGPT correctly predicts neutral (“0”) in 44 of 96 prompts.
For example, here is the full turn with ChatGPT’s explanation of its response for one sample with an incomplete sentence as premise.
(6) (P) : Heaven’s Highway (H) Paradise’s Road.
Human: Does “: Heavaen’s Highway” entail “Paradise’s Road?”
ChatGPT It is not clear whether the phrase ": Heaven’s Highway" entails that "Paradise’s Road". The phrase ": Heaven’s Highway" is not a complete sentence and is difficult to interpret or understand. It appears to describe a specific location or path (Heaven’s Highway), but it is not clear what this location or path entails or whether it relates to the second phrase ("Paradise’s Road"). In order to determine whether the first phrase entails the second phrase, you would need to provide more information about the context in which the phrase is used and the intended meaning of the speaker.

heading: Future Work
text: In the future, we plan to experiment with more sophisticated fine-tuning methods and datasets to maximize performance on non-assertive prompts. Furthermore, we hope to probe ChatGPT using larger datasets, as the preliminary results obtained here are very interesting. Limitations
We see two main limitations of this work. First, it is clear that the quality of the proposed fine-tuning method (with the non-assertive datasets) heavily depends on the quality of the GKR parser, used for the detection of non-assertive statements. Since the parser has an F-score of 84. 3%, as evaluated within this work, it is to be expected that some of the examples used for fine-tuning will be faulty (mistakenly classified as non-assertive and thus mistakenly classified as neutrals). This means that the models are not given entirely clean input to learn from and thus, their performance could further improve if the quality of the GKR parser (or another similar parser) could improve. Note however that this only concerns the fine-tuning datasets and not the evaluation dataset, as this was manually checked for correctness (and thus, there is no impact on the results we report). The second limitation concerns our experimentation with ChatGPT: we were only able to experiment with a small amount of samples (due to the extremely expensive manual effort required) and thus, our findings will need to be reproduced through a larger-scale study. However, our findings agree with Bang et al. (2023), who find that ChatGPT performs reasonably well on deductive and abductive inference tasks