For Generated Text, Is NLI -Neutral Text the Best Text? (2023)

Abstract: We explore incorporating natural language inference (NLI) into the text generative pipeline by using a pre-trained NLI model to assess whether a generated sentence entails, contradicts, or is neutral to the prompt and preceding text. First, we show that the NLI task is predictive of generation errors made by GPT-3. We use these results to develop an NLI-informed generation procedure for GPT-J. Then, we evaluate these generations by obtaining human annotations on error types and overall quality. We find that an NLI strategy of maximizing entailment improves text generation when the nucleus sampling randomness parameter value is high, while one which maximizes contradiction is in fact productive when the parameter value is low. Overall, though, we demonstrate that an NLI strategy of maximizing the neutral class provides the highest quality of generated text (significantly better than the vanilla generations), regardless of parameter value.

## introduction a perfectly informative gricean agent (that is, one who strives to be maximally relevant to their interlocutor, as in @xcite would eschew utterances that are redundant or which contradict that which they have already said. @xcite give a theoretical argument that text generated by a perfectly informative agent allows for the learning of semantic entailment information. 1 a prediction from this gricean approach is that most of what speakers say is not entailed by what came before it (since that would not be very informative), nor would it be contradicted by what came before it. our goal here is to explore this idea using llms, asking whether generated text classified as neutral on a natural language inference (nli; @xcite @xcite task is less likely to show various kinds of figure 1 : average holistic ratings for generations from vanilla gpt-j (control), vs. nli strategies of maximizing for neutral, contradiction, or entailment, for 2 different choices of parameter values. neutral performs best in all cases (significantly better than control), but maximizing contradictions is better than the control when randomness is low, and maximizing entailment is better than the control when randomness is high. the nli task takes a premise and a hypothesis and queries whether the hypothesis is entailed by the premise, contradicts the premise, or is neutral with respect to the premise @xcite . we propose that we can use these models, and the extensive body of work around them, in order to guide model generationessentially using nli ratings as a way of reranking @xcite the "fast" output of an llm, using a well-established logical inference task. in this vein, @xcite used an nli approach, alongside other approaches, to try to get rnns to generate more coherent text. specifically, they focused on selecting sentences that maximize the probability of being neutral to what preceded them, but did not conduct a thorough evaluation of whether this was the best strategy. while they showed that their models helped generations overall, the nli part showed mixed results and, on its own, did not help much. @xcite show that using an nli task (and particularly using contradictions) can be taken as a measure of semantic diversity. @xcite use an nli task to increase the consistency of the persona in generative models. @xcite use this insight to develop a system that combines llm generation with a symbolic reasoning model, essentially using the llm as a fast generation system and then using the symbolic reasoner to refine the output. we investigate how nli task judgments relate to text generation errors by systematically investigating whether an nli task can predict the kinds of errors made by gpt-3 in the publicly-available-forresearch scarecrow dataset @xcite . since generated text has different failure modes depending on the parameters (i.e. for the nucleus sampling parameter p, text generated with high values is more likely to be incoherent/off-prompt, text generated with lower values is more likely to be redundant), we pay particular attention to how the nli task interacts with the nucleus sampling parameter. we use the results of this analysis to motivate a number of possible ways in which the nli task can be used to improve generated outputs. comparing across conditions, we generate new text using gpt-j (wang and komatsuzaki, 2021) and evaluate it on a subset of the scarecrow criteria. as proof of concept, we show that using an nli filter can improve gpt-j generations. in this work, as in previous work using the nli task discriminatively, the domain is different than in traditional nli tasks. in traditional nli data sets, the premise/hypothesis sentence pairs are typically formulated so as to explicitly stand in a particular logical relation to each other-even when the hypothesis is neutral to the premise. for instance, consider this neutral premise/hypothesis pair from the mnli homepage: "your gift is appreciated by each and every student who will benefit from your generosity. / hundreds of students will benefit from your generosity." while the label is neutral, the sentences stand in a very particular relationship to each other. in contrast, we assign nli labels (contradiction, neutral, or entailment) to pairs of sentences that are not explicitly constructed for the task and are thus out-of-domain relative to the nli training sentences. as such, the nli labels have different interpretations than they do on the mnli data sets: e.g., text labeled "entailment" in our data should not be interpreted as "entailment" in the way it would be in an mnli data set. but, for our purposes, this is not a problem since our goal is to use the nli models as tools. that is, our goal is to take an nlituned model tuned on nli data (which is specific and idiosyncratic) and apply it to general-purpose text, treating the nli label as a property of the text that can be used downstream in constraining generations--not because every pair of sentences can be thought of as an nli task but because, even without that assumption, the nli labels on sentence pairs are useful. 2 analysis of gpt-3 text through ## conclusion while the use of an iterative nli task during generation is likely not practical at scale, we have shown that, as a proof of concept, using nli as a mechanism for choosing among possible generations seems to be able to imbue text generation systems with a more sophisticated reasoning system (analogous to the idea of using a discriminator to incorporate "system 2" reasoning into generative systems, as described by @xcite . in particular, maximizing neutral sentences seems most promising. but it also seems that, in cases where one wants to generate text with a high randomness parameter, maximizing entailments could be productive. similarly, in a low randomness setting, maximizing contradictions could actually make the text better by avoiding redundancy. as such, this work has implications for understanding the ways in which logical relations among sentences are reflected in text generation using llms.