For Generated Text, Is NLI-Neutral Text the Best Text?
Authors: Michail Mersinias
Kyle Mahowald
Venue: EMNLP 2023 Findings (2023)
Keywords: generation
natural language inference
TL;DR: We run an NLI task as a tool for guiding generation and find that neutral tags are predictive of the best generations.

Abstract: We explore incorporating natural language inference (NLI) into the text generative pipeline by using 
a pre-trained NLI model to assess whether a generated sentence entails, contradicts, or is neutral to the prompt and preceding text. 
First, we show that the NLI task is predictive of generation errors made by GPT-3.
We use these results to develop an NLI-informed generation procedure for GPT-J.
Then, we evaluate these generations by obtaining human annotations on error types and overall quality.
We find that an NLI strategy of maximizing entailment improves text generation when the nucleus sampling randomness parameter value is high, while one which maximizes contradiction is in fact productive when the parameter value is low. 
Overall, though, we demonstrate that an NLI strategy of maximizing the neutral class provides the highest quality of generated text (significantly better than the vanilla generations), regardless of parameter value.

For Generated Text, Is NLI-Neutral Text the Best Text?
Michail Mersinias
Kyle Mahowald
Department of Computer Science The University of Texas at Austin mmersinias@utexas.cmm edu
Department of Linguistics
The University of Texas at Austin
mahowaldeutexas.com edu
Abstract
param 0.4
param p 0.96
We explore incorporating natural language in- ference (NLI) into the text generative pipeline by using a pre-trained NLI model to assess whether a generated sentence entails, contra- dicts, is neutral to the prompt and preceding text. First, we show that the NLJ task is predic- tive of generation eπrоrs made by GPT-3. We use these results to develop NLI-informed generation procedure for GPT-J. Then, we eval- uate these generations by obtaining human an- notations on error types and overall quality. We find that an IL. strategy of maximizing entail- ment improves text generation when the nu- cleus sampling randomness parameter value is high, while one which maximizes contradic- tion is in fact productive when the parameter value low. Overall, though, we demonstrate that NIII strategy of maximizing the neutral class provides the highest quality of generated text (significantly better than the vanilla gener- ations), regardless of parameter value.
CON
CON
0
avg. rating (1-5)
Figure Average holistic ratings for generations from vanilla GPT-J (control), vs. NLI STRATEGIES of max- imizing for neutral, contradiction, or entailment, for different choices of parameter values. Neutral per- forms best in all cases (significantly better than control), but maximizing contradictions is better than the control when randomness is low, and maximizing entailment is better than the control when randomness is high.
errors text generation, using an LLM like GPT- 3. We also ask whether an NLI task could then be used to improve generation. We do not aim for a performant state-of-the-ae generation system: the quality of GPT-4 generations are likely higher and more efficient than those here (which involve multiple NLI judgment tasks). Rather, we seek improve our basic theoretical understanding of the properties of text generated by LLMs, and how they relate to theoretical questions about the emergence of semantic information from text alone.
1 Introduction
A perfectly informative Gricean agent (that is, one who strives to be maximally relevant to their inter- locutor, as in Grice, 1975) would eschew utterances that are redundant or which contradict that which they have already said. Merrill et al. (2022) give a theoretical argument that text generated by a per- fectly informative agent allows for the learning of semantic entailment information.' A prediction from this Gricean approach is that most of what speakers say is entailed by what came before it (since that would be very informative), nor would it be contradicted by what came before it.
The NLI task takes a premise and hypothe- sis and queries whether the hypothesis is entailed by the premise, contradicts the premise, tral with respect to the premise (Nie al., 2020: Bowman et al., 2015). We propose that we can use these models, and the extensive body of work around them, in order to guide model generation essentially using NLI ratings as a way of reranking (Shen et al., 2019; Holtzman et al., 2018) the "fast" output of an LLM, using well-established logical inference task.
neu-
Our goal here is to explore this idea using LLMs, asking whether generated text classified as neutral on a Natural Language Inference (NLI; Conneau et al., 2018; Williams et al., 2018; Bowman et al., 2015) task is less likely to show various kinds of 'Our code and results are available at tttps://githhb. com/Michael- -Maesaaai--ee-iiiiiaeaaa aetiiiisssnniiiiiieeieeiiieiiiie welptteetttt_tieeeeeeee tet_tteeet___teeeeeeeeeeeeeeeeee generrtnne
In this vein, Holtzman et al. (2018) used an NLI approach, alongside other approaches, to try to get RNNs to generate more coherent text. Specifically, they focused on selecting sentences that maximize

---

p (nucleus sampling param)
CON
ENT
NEU
0.5
parameter p
40
3,44
12.93
83.62
proponton of error type
12.41
1.37
86.20
0.4
0.4
0.96
Table I: For high and low p parameters in SCARECROW, breakdown of NLI classes for generated text. Neutral is by far the most common in both settings, but entailment is more common than contradiction when randomness is low, and vice versa when randomness is high.
0.-
0.1
0.0
Incoherent
Of Prompt
Redundant Salt Contradictory error types
Low (0.4)
High P (0.96)
CON
ENT
NEU
CON
ENT
NEU
All
3,44
12,93
83.62
12.41
1.37
86.20
Figure 2: Proportion of erroneous examples in Scare-
CO
1.66
3.33
95.00
7.85
0.52
91.62
crow per error type for high and low p parameter.
OF
12.50
0.00
87.50
23.07
0.00
76.92
SC
25.00
0.00
75.00
20.00
0.00
80.00
0.00
0.00
0.00
31.81
4.54
63.63
RD
2.22
28.88
68.88
16.66
6.66
76.66
Comparing across conditions, generate new text using GPT-J (Wang and Komatsuzaki, 2021) and evaluate it subset of the SCARECROW criteria, As proof of concept, we show that using an NLI filter can improve GPT-J generations.
Table 2: Distribution of spans in SCARECROW text marked with each error type (CO: correct, OP: off- prompt, SC: self-contradiction;; IN: incoherent, RD: re- dundant) by at least half of annotators, broken down by NLI class.
In this work, as in previous work using the NLI task discriminatively, the domain is different than in traditional NLI tasks. traditional NLI data sets, the premise/hypothesis sentence pairs are typically formulated so as to explicitly stand in a particu- lar logical relation to each other- even when the hypothesis is neutral to the premise. For instance, consider this neutral premisc/hypothessis pair from the MNLI homepage: "Your gift is appreciated by each and every student who will benefit from your generosity. / Hundreds of students will benefit from your generosity." While the label is neutral, the sentences stand in a very particular relationship to each other.
the probability of being neutral to what preceded them, but did not conduct thorough evaluation of whether this was the best strategy. While they showed that their models helped generations over- all, the NLI part showed mixed results and, on its own, did not help much. Stasaski and Hearst (2022) show that using an NLI task (and particularly using contradictions) can be taken as a measure of seman- tic diversity. Song et al. (2020) use an NLI task to increase the consistency of the persona in genera- tive models. Nye et al. (2021) use this insight to develop system that combines LLM generation with symbolic reasoning model, essentially using the LLM as fast generation system and then using the symbolic reasoner to refine the output.
In contrast, we assign NLI labels (contradiction, neutral, or entailment) to pairs of sentences that are not explicitly constructed for the task and are thus out-of- relative to the NLI training sen- tences. As such, the NLI labels have different inter- pretations than they do the MNLI data sets: e.g., text labeled "entailment" in our data should not be interpreted as "entailment" in the way it would be in an MNLI data set. But, for our purposes, this is not a problem since our goal is to use the NLI models as tools. That is, our goal is to take an NLI- tuned model tuned on NLI data (which is specific and idiosyncratic) and apply it to general-purpose text, treating the NLI label as a property of the text that can be used downstream in constraining generations- because every pair of sentences can be thought of an NLI task but because, even without that assumption, the NLI labels on sentence pairs are useful.
We investigate how NLI task judgments relate to text generation errors by systematically investigat- ing whether an NLI task can predict the kinds of errors made by GPT-3 in the abblicly-vailabbee.. research SCARECROW dataset (Dou et al., 2022). Since generated text has different failure modes depending on the parameters (i.e. for the nucleus sampling parameter p. text generated with high values is more likely to be incoherent/off-prome prompt, text generated with lower values is more likely to be redundant), we pay particular attention how the NLI task interacts with the nucleus sampling parameter. We use the results of this analysis to motivate a number of possible ways in which the NLI task can be used to improve generated outputs.

---

vanilla GPT-J randomness: 0.4
rendommess: 0.98
param p 0.4
param p 0.96
Incoherent
10.21
-0.22
0.02
0.11
0.08
-0.08
CON
CON
NEU cnntool
NEU
commll
000.4
-0.36
Off-prompt
CON
0.11
-0.02
-.0026
ENI
NEU
pontrol
ont
IN- 0.07
0
-0.04
"00.17
0.07
0.1017
Redundant
CON
CON
CON
ENT NEU
CON
ENT
NEU
ENT
NEU
oontral
ontrol
Figure 3: For high and low p (randomness) parameters in SCARECROW, rank correlation between proportion of text showing an error type (y-axis) and the probability of the given NLI class (x-axis).
COO
OON
Self- Self-Cont
NPU
NFU
pontrol
pontroll
0.0 0.1 0.2 0.3 0.4 0.50 0 0.1 0.2 0.3 0.4 0.5 mean proportion error span for each type
2 Analysis of GPT-3 Text Through Natural Language Inference
Figure 4: For our text generation task, the average hu- man annotator ratings for each SCARECROW er- types, broken up by whether output (control), maximize neutral NLI relationships generated text, maximize entailments, or maximize contradictions. Maximizing neutral is best overall, but maximizing entailment is better than maximizing con- tradiction when randomness high and vice versa when randomness is low.
First, we conduct a systematic exploration on a sub- set of the SCARECROW (Dou et al., 2022) dataset (GPT-3 generations with static temperature pa- rameter value of 1, and either high or low nucleus sampling parameter as described). Each dataset entry provides the prompt, the generated text and list of errors identified by human annotators. For analysis and evaluation, focus on "language errors" in SCARECROW specifically the categories off-prompt (OP), self-contradiction (SC), incoher- ent (IC), and redundant (RD) error types, as as- signed by at least 50% of human annotators.
man rank correlations between the proportion of text marked with each error type and the NLI probability assigned each category (entail- ment/heuutaa/coonraaccttt)))) In the low random- ness setting, the contradiction NLI class was signif- icantly associated with more seff-contradictions and the entailment category with fewer self- contradictions but more redundancy, and the neu- tral category with less redundancy. In the high randomness setting, the contradiction NLI class significantly associated with more off off-prompt and incoherent errors, entailments with more off- prompt errors, and neutral with fewer off off- prompt and incoherent errors. All other correlations were not significant at < .05.
For attaining NLI ratings, we an off-the-shelf pre-trained BART-large-mn1 model for natural language inference. We treat the SCARECROW prompt as the premise and the GPT-3-generated text as the hypothesis and run the NLI task. The distribution NLI ratings appears in Table 1.
We present the distribution of error types across each NLI class in Table 2. It shows that Correct (CO) segments are very likely be neutral (95% of the time for low randomness, 92% of the time for high randomness). But, when there a redundancy (RD) error, the probability of entailment goes way up: to 29% the low randomness condition and 7% in the high randomness condition (although in all cases, the neutral class remains dominant). off-prompt or self-contradictory errors, in both settings, the probability of the con- tradiction class increases sharply.
Overall, we observe that a low randomness set- ting leads to higher probability of text classified as entailment, and that such text is also more likely to be redundant. In contrast, a high randomness setting leads to higher probability of text clas- sified as contradiction, which is also more likely to be off-prompt or incoherent. both settings,
When there
As shown in Figure 3, we computed Spear-

---

with no errors is significantly more likely be classified as neutral-lending support the idea that the neutral class is preferable.
are typically out-of-domain the NLI task. As such, many sentence pairs of interest us would be rated neutral. So these NLI STRATEGIESwere chosen so that only pairs with neutral probability greater than 0.85 are rated NEU. There is no such threshold for the strategies ENT or CON-which are determined purely by comparing whether the entailment probability is greater than the contra- diction probability vice versa. There are, of course, other ways that we could have operational- ized these strategies but we found that these gave us the kind of varied behavior we desired (whereas a strategy that, e.g., only called pairs CON if the con- Iradiction probability was greater than 0.85 would rarely have been triggered).
3 Realtime NLI to Improve Generation
Method Motivated by this finding, we propose an approach for overcoming the issues present in the generated text of LLMs in order to improve its quality by incorporating natural language in- ference in the text generation pipeline. For text generation, we use the open-source GPT-J (Wang and Komatsuzaki, 2021) while for NLI we use pre-trained BART-Large-mn1i model, above. Using a random subset of 50 prompts contained in the SCARECROW dataset, we generate a contin- uation for each of nucleus sampling parameters p (0.4 or 0.96) x 4 conditions (one for vanilla GPT with no NLI constraints, and one for each of our three NLI STRATEGIES as described below), for total of 8 continuations for each of the 50 prompts (n=400 total).
In order ensure sufficient length of generated text for our evaluation, we restarted the process from the initial prompt for any examples whose produced text included less than 2 sentences. After running this process twice, in all cases, at least one sentence was generated that passed the NLI check. See Appendix A for details on the compute set-up-
For the vanilla GPT-J setting, we generate using the relevant parameter for nucleus sampling (0.4 or 0.96), with a max length of 256 tokens includ- ing the initial prompt. For the NLI ATRATES we first generate a continuation with a maximum length of 128 tokens including the initial prompt.
Two annotators (both undergraduate students at our institution, paid $15 per hour and informed of how the data would be used) evaluated the gener- ations using the SCARECROW annotation frame- work, as well as a 1-5 holistic rating of overall gen- eration quality (see Appendix B for guidelines). Be- cause some of the prompts were either erroneously not annotated by both annotators or included sen- sitive content that we removed from our final data set, we ended up with 326 examples with ratings by both annotators and used those for analysis. (The number is relatively small due to a limited bud- get.) To assess inter-annotator agreement between the two annotators, we measured the correlation of proportion of spans highlighted for each error type across all error types and sentences and found Pearson correlation of 0.70. Treating error type in a particular category a binary score, there was 82% agreement between annotators (chance agreement rate would be 56%). Inter-annotator agreement for the holistic ratings were also high, with a Pearson correlation of 0.77 between the two annotators.
Then, the chosen NLI STRATEGY is applied to each candidate sentence of the generated text. The candidate sentence is treated as the "hypothesis", while each sentence of the prompt is used as the "premise". The candidate sentence is appended to the continuation (and therefore becomes part of the hypothesis for subsequent candidates) iff it satisfics the chosen NLI STRATEGY for every sentence in the preceding text. Otherwise it is discarded along with the remaining candidate sentences. The NLI STRATEGY conditions are:
ENT: P(entailment) > P(contradiction)
CON: P(contradiction) P(entailment)
NEU: P(neutral) > 0.85
For instance, if we are applying the NEU strategy, we reject a sentence assigned neutral probability < 0.85, relative to any sentence in the prompt or any previously generated sentence appended to the continuation. We also reject all sentences after it. The process is repeated until there are seven con- secutive failed attempts to expand the continuation until the continuation exceeds 256 characters and has 3 or more sentences.
Focusing first the average holistic rat- ing assigned the generation, we observe that max- imizing neutral NLI status improves generation. We ran a regression predicting average holistic rat- ing for a prompt, based on the NLI STRATEGY used, with the control (vanilla GPT-J) as the base-
Results
As discussed in the Introduction, our sentences

---

Second, as Nye et al. (2021) show, there is much the NLI task does not capture. NLI tasks capture only a fraction of the possible real-world use cases that might want use for guiding generation. line. When p was 0.4 (low randomness), NEU (the neutral strategy) was rated significantly higher than the control (3 = .1, < .05), as was the case in the high randomness condition (3 = -70,p < .01). As shown in Figure 1, when randomness low, the CON (contradiction strategy) outperforms the con- trol and ENT (entailment strategy) underperforms it, but neither significantly so. In the high random- ness condition, ENT is significantly better than the control (3 .49, p < .01) but still worse than neutral, while CON performs similarly to control.
Limitations
First, while our method incorporates NLI into the text generation process in real-time and zero-shot fashion, there is still the issue of computational efficiency. Specifically, the NLI STRATEGIES that maximize entailment and contradiction often quire multiple generations produce sentence which passes their respective NLI checks. Because LLM text generation is already slow for some use cases, this process may cause a bottleneck.
To better understand the source of the differ- ence, we considered the specific error annotations, as shown in Figure 4. We treated the control as the baseline and ran regressions iteratively for error types and the randomness parameter p. For parame- ter p= 0.96 (high randomness), relative control, NEU showed significantly fewer off- fiffprompt errors (p < .001) and incoherent errors (p .05). For parameter p- = 0.40 (low randomness), relative to control, NEU did not show significant differences but was particularly better on redundancy errors. When randomness is high, we also observe that ENT is significantly better off- prompt errors but significantly worse redundancy errors. When randomness is low, CON is significantly worse for off-prompt errors.
Future work might explore using additional kinds of tasks, with the caveat that increasing the com- plexity of the task could slow generation down even more.
Finally, we tested the generation procedure on only GPT-J, but are open the possibility that more sophisticated models (especially those like ChatGPT that already include human feedback) might already do better some of the errors iden- tified in SCARECROW, and so could benefit less from our procedure.
Acknowledgements
We thank Clara Meister and Jessy Li for helpful conversations. K.M. acknowledges funding from NSF Grant 2104995.
4 Conclusion
While the use of an iterative NLI task during gener- ation likely not practical at scale, we have shown that, as proof of concept, using NLI as a mech- anism for choosing among possible generations seems to be able to imbue text generation systems with more sophisticated reasoning system (anal- ogous the idea of using discriminator to in- corporate "System 2" reasoning into generative systems, as described by Nye et al., 2021). In par- ticular, maximizing neutral sentences seems most promising. But it also seems that, in cases where one wants to generate text with high randomness parameter, maximizing entailments could be pro- ductive. Similarly, in low randomness setting, maximizing contradictions could actually make the text better by avoiding redundancy. As such, this work has implications for understanding the ways in which logical relations among sentences are re- flected in text generation using LLMs.
References
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large amno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empiri- cal Methods in Natural Language Processing, pages 632-642, Lisbon, Portugal. Association for Compu- tational Linguistics.
Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating cross- lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Nat- ural Language Processing, pages 2475-2485, Brus- sels, Belgium. Association for Computational Lin- guistics.
Yao Dou, Maxwell Forbes, Rik Koncel-Kedziorse Noah A Smith, and Yejin Choi. 2022. Is gpt-3 text indistinguishable from human text? scarecrow: A framework for scrutinizing machine text. Proceed- ings of the 60th Annual Meeting of the Association for

---

Computettional Linguistics (Volume 1: Long Papers), pages 7250-7274.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceed- ings of the 2018 Conference the North American Chapter of the Association for Computational Lin- guisties: Human Language Technologies, Volume (Long Papers), pages 11 12-1122, New Orleans, Louisiana. Association for Computational Linguis- tics.
Herbert Grice. 1975. Logic and conversation. In Speech acts, pages 41-58, Brill.
Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. 2018. Lear- ing to write with cooperative discriminators. In Pro- ceedings of the 56th Annual Meeting of the Associa- tion for Computational Linguistics (Volume 1: Long Papers), pages (638-1649 Melbourne, Australia. As- sociation for Computational Linguistics.
A Appendix: Compute
The experimental evaluation was conducted in Google Colab Prot (Python 3.8.10, Pandas 1.3.5, Numpy 1.21.6, Transformers 4.25.1, Torch 11131+ccu166
William Merrill, Alex Warstadt, and Tal Linzen. 2022. Entailment semanties can be extracted from ideal language model. In Proceedings of the 26th Confer- ence on Computational Natural Language Learning (CoNLL), pages 176-193, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
We used off aff-the-shelf models from HuggingFace for our research purposes, consistent with their intended use.
We make our code and data publicly available for research use.
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversarial NLI: A new benchmark for natural language under- standing. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 4885-4901, Online. Association for Computa- tional Linguistics.
For a single pass of the 326 examples, 35 com- pute units from an A A100-SXM4440GB GPU and about 3 hours were required.
B Appendix: Annotation Guidelines
Maxwell Nye, Michael Tessler, Josh Tenenbaum, and Brenden M Lake. 2021. Improving coherence and consistency in neural sequence models with dual- system, neuro-symbolic reasoning. Advunces in Neural Information Processing Systems, 34:25192- 25204.
Below, we reproduce our annotation guidelines and in Figure 5 the SCaReCROw guidelines given to our annotators.
Dataset In this research regarding natural lan- guage inference (NLI) application in text genera- tion, we have made use of 50 prompts in order to generate 400 picces of text. For the purpose of text generation, we have used the open nour-edd GPTJ model and the nucleus sampling decoding strategy. There totally eight different combinations of NLI conditions and nucleus sampling parameter values.
Sheng Shen, Daniel Fried, Jacob Andreas, and Dan Klein. 2019. Pragmatically informative text gener- ation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long and Short Papers), pages 4060-4067, Minncapolis, Minnesota. Association for Computational Linguistics.
Haoyu Song, Wei-Nan Zhang, Jingwen Hu, and Ting Liu. 2020. Generating persona consistent dialogues by exploiting natural language inference. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8878-8885,
Error Types For each example, an annotation has to be completed in accordance with the SCARE- CROW annotation guidelines. The error types of interest are the "redundant", "off-prompt", "self- contradiction" and "incoherent" ones. An example may have zero, one or multiple error types assigned to it. Furthermore, a rating (1 5 stars) should be given to each example in order to represent the overall quality of text generation according to the annotator's judgment. A description of cach error type it is defined by SCARECROW, along with corresponding example, is presented in Figure 5.
Katherine Stasaski and Marti Hearst. 2022. Semantic diversity dialogue with natural language inference. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 85-98, Seattle, United States. Association for Computational Linguistics.
Ben Wang and Aran Komatsuzaki. 2021. GPT-J- 6B: A 6 Billion Parameter Autoregressive Lan- guage Model. https: (hgb. http://gtthuuu.mmmig//gg//////// com//ingofllll//
Annotation Process The goal of the annotation process is to explore the quality of the text gener-

---

cookriant tocreposty nott onnds sario
they docsiitts 3ysed
Honkkra
Recundent
rppedied Olten
stands.
Prompp place
ttall expease
Prongt Chira recard Eocrare Gaowth
Sonearree
Offoronall
conciimly
Then: The Citress comommy ronth year.
unmvisted Other
tiree ressect
contrediio
MuDusaddfo aifo, conaker mi
0:00Us when TheAl willin sfoce but tut hod
condhoor pathaging Conn-
canunn cacund moses
revinccs Tee pardi 201
The bod thail obsarh 10 the
Sear for mem breuting apar
Demen mode ang ae
sterel camp Bas dlterr
Figure 5: Description of redundant, off-prompt, self- contradiction and incoherent error types along with cor- responding examples, according the SCARECROW annotation framework.
ation which resulted from the respective prompt. Each example consists of the prompt, -IGENI- sep- aration token and the GPTJ generated text. Thus, it is the form: <PROMPT> -IGENI- <GENER- ATION>. In order to perform the annotation of an example, the procedure is as follows.
1. Select an error type, then perform drag selec- tion in the sequence of words that corresponds to it.
2. Repeat the process needed. An example may have zero, multiple error types assigned to it. An error type may also be assigned multiple times.
3. Finally, assign a rating (1 5 stars) in order to represent the overall quality of text generation.
Furthermore, a mistake is made, the "Delete all Regions" button may be selected in order to redo the annotation task for this example. Finally, when the annotation task for an example is complete, the blue "Submit" button should be selected on the top right in order to register the annotation.


Reviews:
[{"review_id": "ntVA20yDwA", "reviewer": "Reviewer_XZPF", "summary": "", "questions": "A. What are the significance levels in Figure 3?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*Q: \"The three categories of NLI relations are not sufficient to cover all possible relations between generated text and prompt….The reliability of the NLI classifier is also a concern. Specifically, how many examples are there that cannot be categorized into any of the three categories? How many of them are misclassified as 'neutral'?\"*\n\nA: This Reviewer raises an important theoretical point that we will aim to clarify in a revision. The Reviewer is exactly right that most sentence pairs are not like the simple pairs typically used in NLI tasks. Our claim does not depend on a close match between the NLI training examples and the test examples. Rather, the idea is to use the NLI models as tools. So, if the text labeled “entailment” was only sometimes entaliment, but it *was* consistently redundant, that would still be informative. In that sense, even the accuracy of the NLI task on these sentences is not crucial: what matters is that the ratings are useful for constraining generation.\n\nIn other words, the task is to take an NLI-tuned model tuned on NLI data (which is specific and idiosyncratic) and apply it to general text—not because every pair of sentences is equivalent to an NLI task but because, even without that assumption, we think the NLI-tuned data is useful.\n\nWriting this explicitly would, we think, help clarify the theoretical goals of the paper and we will add a similar discussion for the Camera Ready. Thank you for this review: we think doing this will help clarify our position.\n\n*Q: \"Significance levels in Figure 3?\"*\n\nA: In Figure 3, we display Spearman Rank Correlations where color represents correlation magnitude and significance is represented by the number of stars which correspond to p values. Specifically, one star corresponds to p < 0.05, two stars correspond to p < 0.01, and three stars correspond to p < 0.001.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Using NLI to guide language generation is an interesting idea. The correlation between NLI relations and the quality of generated text is significant in the experiments. The paper is well-written.", "reasons_to_reject": "I'm not sure to what extent the findings in this paper are generalizable to all language generation tasks. As the authors pointed out in the Limitation section, it seems impossible to justify that the three categories of NLI relations are sufficient to cover all possible relations between generated text and prompt. In this sense, it is important to include discussions on the generalizability of the method and why the chosen dataset (i.e., SCARECROW) is suitable for this study.\n\nIn addition, the reliability of the NLI classifier is also a concern.\nSpecifically, how many examples are there that cannot be categorized into any of the three categories? How many of them are misclassified as 'neutral'? This matters because a major claim of this paper is that 'neutral' is an indicator of high quality.", "questions_to_authors": "A. What are the significance levels in Figure 3?", "paper_topic_and_main_contributions": "This paper investigates the relationship between NLI relations and the quality of machine-generated text. The authors find that neutral NLI relation between generated text and input prompt is often an indicator of high quality. Contradiction and entailment relations might indicate high quality under low and high-randomness sampling strategies, respectively.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yvolTWp82k", "reviewer": "Reviewer_GiAa", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the review!\n\n*Q: \"Only 50 sampled are evaluated, thus the reliability is not good.\"*\n\nA: There were 50 prompts used in order to generate 400 examples, and the evaluation was done on an a subset of those examples, with exclusions as described. To deal with the small data size, we were careful to conduct inferential statistics. Specifically, Figure 1, Figure 3 and Figure 4 display a significant effect, with 95% confidence intervals for Figures 1 and 4, and with significant p-values in Figure 3. Thus, while we agree the experimental setup could ideally benefit from a larger sample size, we were constrained by having to get trained annotators to annotate and think there is enough useful signal here to be informative, based on the statistics. We also note that reliability is certainly correlated with sample size in the population, but it is certainly statistically possible for relatively small samples to be reliable.\n\n*Q: \"A performance comparesion between using NLI and not using should be given in table or figure.\"*\n\nA: The performance of vanilla generations without NLI is annotated as \"Control\" in Figure 1 and Figure 4, where it it shown that methods using NLI appropriately outperform Control significantly. [Apologies if we misunderstood this query and the Reviewer was asking for something else!]", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1 Classify the different errors caused by different probabiltiy settings.\n2 By using nature langauge inference module in pipeline, they improve the generation. ", "reasons_to_reject": "Only 50 sampled are evaluated, thus the reliability is not good.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper investigated the impact of different p-value in nucleus sampling. The findings are that high probabiltiy causes contradiction while low p-value causes redundant. Further, the authors designed real NLI to improve generation. The results show the gain of the proposed method. ", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "A performance comparesion between using NLI and not using should be given in table or figure. "}, "extra_text": {}}, {"review_id": "7dK5ka8XXf", "reviewer": "Reviewer_gehN", "summary": "", "questions": "Q1: For the NEU scenario, how do you prevent the model from generating completely unrelated sentences (i.e. not even the same topic)?\n\nQ2: For the ENT/CON scenarios, can't it happen that the (ent, con, neu) probabilities are e.g. $(0.11, 0.8, 0.09)$? If this happens, then it is accepted as \"entailment\" but likely it is a neutral continuation. What was the rationale for simply setting the condition for each ENT/CON/NEU to be $\\text{arg}\\max p$ = ENT/CON/NEU?", "limitations": "", "rating": 3, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for this review, which we found helpful. \n\n*Q: \"While the execution is great, the novelty of this work is disputable as most of the ideas and conclusions could be synthesized from existing works. That said, I do not believe that this should be a reason to reject this paper.\"*\nA: We’re glad the Reviewer appreciates the execution and recommends acceptance. We do not disagree with the claim that many of the ideas here could indeed be synthesized from earlier work. But we do think that the particular framing and execution here is novel in how we are thinking about the relationship between NLI data and “normal text”. Given ongoing interest on the relationship between logical entailiment tasks (see, for instance, Merrill et al. on learning entailment relationships from perfectly communicative text), we think this could be a useful theoretical and empirical contribution.\n\n*Q: \"For the NEU scenario, how do you prevent the model from generating completely unrelated sentences (i.e. not even the same topic)?\"*\nA: The model is still generating conditional on the existing prompt and sentences so typically doesn’t go too far off the rails, even in the NEU and CON scenarios.\n\n*Q: \"For the ENT/CON scenarios, can't it happen that the (ent, con, neu) probabilities are e.g. (0.11, 0.8, 0.09)? If this happens, then it is accepted as \"entailment\" but likely it is a neutral continuation. What was the rationale for simply setting the condition for each ENT/CON/NEU to be argmax p = ENT/CON/NEU?\"*\nA: Yes, good point. The distribution of text generation by LLMs is different than the distribution of the MNLI dataset that the BART model for NLI was trained on, specifically there is an increased probability of Neutral compared to the artificially balanced NLI data. This is shown in Table 1. As we explained in responses to R1, our goal was to use the NLI-tuned classifier to generate useful judgments about naturalistic text (not just NLI text), and simply classifying the vast majority of text as Neutral would not be useful. Rather, we wanted more balanced categories and so designed a thresholding approach that would create a more balanced set. Note that the thresholding procedure was decided independently from the annotations. So, for us, the ENT/CON/NEU labels have a slightly different (but related) meaning than they do in an NLI task.\n\nResponding to these reviews has been useful. We are glad that the reviewers largely appreciated the somewhat different perspective we take on using the NLI task to constrain generations. But we think we can make this theoretical approach even clearer for the Camera Ready.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is written very clearly, is placed well within existing literature and has a clear and well-described experiment design. It also shuns away from pursuing the SotA and instead focuses on more fundamental links between text generation and information theory. I believe that as it is, the paper is a good contribution to the community.", "reasons_to_reject": "While the execution is great, the novelty of this work is disputable as most of the ideas and conclusions could be synthesized from existing works. That said, I do not believe that this should be a reason to reject this paper.", "questions_to_authors": "Q1: For the NEU scenario, how do you prevent the model from generating completely unrelated sentences (i.e. not even the same topic)?\n\nQ2: For the ENT/CON scenarios, can't it happen that the (ent, con, neu) probabilities are e.g. $(0.11, 0.8, 0.09)$? If this happens, then it is accepted as \"entailment\" but likely it is a neutral continuation. What was the rationale for simply setting the condition for each ENT/CON/NEU to be $\\text{arg}\\max p$ = ENT/CON/NEU?", "paper_topic_and_main_contributions": "The paper revisits the link between redundancy and statements not being concordant. While this has been studied in NLP already, this paper explores it again with focus on different sampling strategies and the associated decoder scores. Specifically, the authors design a rejection mechanism based on NLI predictions, which they show, through small-scale human evaluation, to reduce errors in the generated text.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "The writing is excellent. I only feel like the title is too vague and could be made more concrete whilst retaining its brevity."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper explores leveraging a pretrained NLI model to assess the entailment relationship between generated text, and the prompt and the preceding text. While all reviewers appreciated the execution of the paper, some reviewers expressed concerns regarding the generalizability of the proposed method.
Decision: Accept-Findings