With a Little Push, NLI Models can Robustly and Efficiently Predict Faithfulness
Year: 2023

Abstract:
Conditional language models still generate unfaithful output that is not supported by their input. These unfaithful generations jeopardize trust in real-world applications such as summarization or human-machine interaction, motivating a need for automatic faithfulness metrics. To implement such metrics, NLI models seem attractive, since they solve a strongly related task that comes with a wealth of prior research and data. But recent research suggests that NLI models require costly additional machinery to perform reliably across datasets, e.g., by running inference on a cartesian product of input and generated sentences, or supporting them with a question-generation/answering step. In this work we show that pure NLI models can outperform more complex metrics when combining task-adaptive data augmentation with robust inference procedures. We propose: (1) Augmenting NLI training data to adapt NL inferences to the specificities of faithfulness prediction in dialogue; (2) Making use of both entailment and contradiction probabilities in NLI, and (3) Using Monte-Carlo dropout during inference. Applied to the TRUE benchmark, which combines faithfulness datasets across diverse domains and tasks, our approach strongly improves a vanilla NLI model and significantly outperforms previous work, while showing favourable computational cost.

heading: 1 Introduction
text: Conditional language models suffer from a tendency to hallucinate information (Maynez et al., 2020), resulting in generations that are not faithful to their input documents, which limits the trustworthiness of such models. This raises a need for automatic faithfulness metrics. In this context, models trained on natural language inference (NLI) (Bowman et al., 2015) are attractive since, intuitively, a generation being faithful implies it must be entailed by the source (Falke et al., 2019).
However, pure NLI models have seen mixed success in faithfulness evaluation (Falke et al., 2019; Kryscinski et al., 2020; Wang et al., 2020; Maynez et al., 2020). While in recent evaluation on the TRUE benchmark (Honovich et al., 2022), which contains datasets from knowledge-grounded dialogue, summarization and paraphrasing, NLIderived metrics perform best overall, they require impractically large models, or costly additional machinery such as question generation and answering models at inference, while still showing robustness issues. Thus we ask: What is still needed for pure NLI models to perform robustly across faithfulness datasets – while remaining cheap enough to serve as a lean and practical evaluation tool?
We enhance a relatively small NLI model to make it work robustly across tasks in three ways:
Task-Adaptive Data Augmentation. In NLI, a hypothesis must be fully entailed by its supporting premise. However, in faithfulness, not all parts of the generation always need to be grounded. We identify an instance of this phenomenon in dialogue where parts of a turn can fulfill communicative functions such as hedging or establishing emotional connection and are often disregarded in faithfulness annotation. Hence, when applying NLI models to complete dialogue turns that may include statements irrelevant for grounding, we run a risk of producing incorrect unfaithfulness predictions.
To alleviate this issue, we propose a simple data augmentation method to adapt NLI models to genres where they need to be aware of statements that must be exempt from NLI-based faithfulness evaluation. Our approach is computationally attractive, as it avoids an increase of cost at inference time.
Integration of NLI Contradiction Scores. Existing NLI faithfulness metrics typically use the entailment score for their predictions (Honovich et al., 2022; Falke et al., 2019; Kryscinski et al., 2020). However, Chen and Eger (2022) show that subtracting the contradiction score from the entail-
914
ment score (referred to as e-c ) can improve NLI performance in certain evaluation tasks. We show that there also is a strong positive effect of e-c for faithfulness prediction, and demonstrate that this is due to a high contradiction probability being a more reliable predictor of unfaithfulness than low entailment probability.
Monte-Carlo Dropout Inference. Applying NLI models to faithfulness prediction involves a domain shift from largely human-written data to automatically generated text. To make NLI model scores more robust under this shift, we propose to use Monte-Carlo dropout during inference (Srivastava et al., 2014). This essentially creates a cheap ensemble and has been shown to deal better with noisy labels (Goel and Chen, 2021). This approach leads to consistent score improvements in our tasks.
The combination of all modifications not only strongly improves over a baseline NLI model, but also outperforms all other metrics on TRUE, on average, while being cheaper and smaller.1

heading: 2 Method Details

heading: 2.1 Task-adaptive Data Augmentation
text: To illustrate that task requirements can be incompatible between faithfulness and NLI, consider the following instance from the Q2 dialogue corpus (Honovich et al., 2021) that is labelled as faithful:
Grounding: American pancakes are similar to Scotch pancakes or drop scones. Generation: yes , i love american pancakes , they are like scotch pancakes
From an NLI perspective, the generation is clearly not entailed, since the statement “I love american pancakes” is not supported by the input.
To better prepare an NLI system for such genre or task-specific cases, we manually curate a small list of statements that should not influence the faithfulness prediction. We augment NLI data from the ANLI corpus (Nie et al., 2020) by adding a randomly chosen phrase from this set to each instance, while preserving the label. We then train an already fine-tuned NLI model on a concatenation of these augmented samples and original ANLI data. For training details see Appendix A.
1All code is available at https://github.com/julmaxi/ with_a_little_push

heading: 2.2 Monte-Carlo Dropout
text: To compute scores under Monte-Carlo dropout, we randomly sample k dropout masks and compute the average of the model predictions. We set k = 15, since preliminary experiments showed that performance did not profit from additional samples.

heading: 3 Experimental Setup
text: We run experiments on TRUE (Honovich et al., 2022), a benchmark that compiles a wide variety of faithfulness tasks in a standardized format. It contains summarization (Pagnoni et al., 2021; Maynez et al., 2020; Wang et al., 2020; Fabbri et al., 2021), knowledge-grounded dialog (Honovich et al., 2021; Gupta et al., 2022; Dziri et al., 2022)2 and paraphrasing (Zhang et al., 2019) datasets.3 Following recommendations in TRUE, we evaluate using Area under the ROC Curve (AUC).
As our BASE model, we use the DeBERTa-large (He et al., 2020) model of Laurer et al. (2022), trained on MultiNLI (Williams et al., 2018), FeverNLI (Thorne et al., 2018), ANLI (Nie et al., 2020), LingNLI (Parrish et al., 2021) and WANLI (Liu et al., 2022). The metric All uses all three of our proposed modifications to Base. We also investigate a variant without MC dropout inference (-MC) as a more cost efficient alternative.
We compare to the strongest models on TRUE: T5 ANLI (Honovich et al., 2022) is a T5-11B
(Raffel et al., 2020) model trained on ANLI.4
SummacZS (Laban et al., 2022) evaluates an NLI model on all pairs of input and generated sentences and then averages maximum entailment probabilities for each generated sentence. Q2 (Honovich et al., 2021) combines a question generation/answering pipeline with an NLI score. Finally, Honovich et al. (2022) introduce a strong ensemble of these 3 methods (Eorig). To further verify our approach, we construct a new ensemble (Eour) by replacing T5 with All.

heading: 4 Results
text: Table 1 shows the AUC scores for each metric. Our model All not only significantly improves over
2TRUE uses an earlier variant of BEGIN that is described in https://arxiv.org/pdf/2105.00071v1.pdf
3TRUE also has a fact-checking part, which was not included in average metric performance. We also exclude it here, as our base NLI model was trained on parts of it.
4The original T5 model is also pretrained on GLUE (Wang et al., 2018) and SuperGLUE (Wang et al., 2019) data, which contains additional NLI data.
Base on six out of nine corpora, but also significantly outperforms all other competitors on average, while being more computationally efficient.
As expected, we find the biggest gains in dialogue, where the All model even outperforms Eorig on 2 out of 3 corpora. We do not improve on BEGIN, which is likely due to bias in the dataset construction, which we elaborate on in Section 5.1. On the summarization part, All improves significantly over Base on 3 out of 5 corpora, while not significantly harming performance on any corpus. However, it still falls short of the best models in TRUE. The strong showing of T5 on these corpora suggests that this might be alleviated with a stronger base model.
Overall, a very similar behaviour is exhibited by -MC, presenting an attractive option when the added overhead of multiple samples is undesirable. Eour is on par with Eorig, despite massively
reduced costs; it even significantly outperforms it on two dialog and the paraphrasing corpora.
We also investigate the performance of each individual modification to our model (Table 2). They all improve average scores, while only leading to a notable decrease on BEGIN for both e-c and dialogue augmentations and on MNBM for e-c .
Outside of dialogue, we find that the augmentation methods have a positive impact on PAWS, as well as all summarization corpora that are at least partially based on summaries for the CNN/DM dataset (Hermann et al., 2015) (Frank, QAGS-C, and SummEval). While we do not have a definitive explanation for this phenomenon, we hypothesize that on these datasets our augmentations aid in making the model robust in the presence of noise
or irrelevant context since our augmentations are label-neutral and must similarly be ’ignored’ during training.

heading: 5 Analysis

heading: 5.1 Effect of Dialogue Adaptation
text: We investigate whether the improvements via our augmentation approach are indeed due to them improving the handling of personal statements.
We use the occurrences of the pronoun I in a generation as a proxy measure5 and compute its correlation with human labels and metrics (see Table 3). On both Q2 and Dialfact, our proxy measure, while uncorrelated with human labels, is strongly correlated with the scores of both Base and T5. This indicates these metrics indeed tend to incorrectly reject generations with personal statements. All on the other hand reduces this dependency.
Our results also help explain why All fails to improve on BEGIN, since BEGIN gold labels are
5We use spacy (spacy.io) for POS tagging to identify pronouns.
negatively correlated with first person pronouns. This is likely due to a bias in dataset construction: The BEGIN dataset used in TRUE has generations from two models, one of which is both more likely to generate pronouns and more likely to generate unfaithful output (see Appendix B).

heading: 5.2 Effect of integrating contradiction scores
text: To isolate the effect of e-c we compare score distributions of Base and Base+e-c in Figure 1. The lefthand side of the figure shows that in Base ca. 2700 faithful instances are predicted as non-entailed (i.e., e-score near 0), which implies they are labelled as contradictory or neutral. e-c , on the other hand, further differentiates these instances into instances with high contradiction (negative e-c score) and high neutral probability (e-c score near 0). We observe that almost all low-scoring faithful generations are classified as neutral, whereas nearly all instances that are classified as contradictory are indeed unfaithful. Where Base has no way to make use of this information, e-c allows to reliably label contradictory instances as unfaithful.

heading: 5.3 Cost comparison to other approaches
text: There is increasing awareness of the resource-hungriness of deep learning (Strubell et al., 2019). Especially for faithfulness, cheap and reliable metrics are critical, given rising demands for NLG in research and industry. Table 4 shows that our model
requires fewer parameters than any other metric, including a more than 30x reduction compared to T5. During inference our model always requires a constant number of calls which can be reduced to a single call when ablating MC dropout. On the other hand, the number of calls in SummacZS scales with the number of input and output sentences. Q2 needs to generate questions by calling an auto-regressive QG model n times, where n factors in the amount and length of questions (#Q×Ql), answer #Q questions with the QA model and finally check #Q answers with an NLI model (#Q × 2).
In sum, our model compares favourably with other approaches, while also allowing for a performance/cost tradeoff by forgoing MC dropout.

heading: 5.4 Phrase Selection Robustness
text: To ensure that our augmentation is robust and not overly reliant on any particular choice of phrases, we repeat our dataset augmentation process multiple times with five randomly chosen augmentation phrases out of the original ten. We sample ten such datasets and retrain our model for each. Table 5 shows the average score, minimum and maxi-
mum score, as well as the standard deviation of the scores. We also report results of a model with both MC dropout and e-c but without any additional training and augmentations to directly quantify whether the augmentations are still helpful in their reduced form. This corresponds to applying MC dropout and e-c to Base.
As expected, we find that reducing the variety of available phrases leads to a drop in performance across almost all datasets, compared to All. The only exception is BEGIN, where we instead see a slight improvement. This is likely to be related to the construction of BEGIN (see the discussion in Section 5.1).
When comparing our limited augmentation models to the non-augmented model, we find that they still outperform the non-augmented model in almost all cases. In particular for Q2 and DialFact, for which we expect the strongest impact of our augmentations, we find that even the worst run still outperforms non-augmented model. This suggests that our augmentations can robustly adapt the model to the dialogue task.
Finally, we observe a relatively large drop in scores for all datasets that are at (least partially) derived from CNN/DM (Frank, SummEval and QAGS-C). This mirrors our earlier observation in Section 4 that these datasets profit from our augmentation procedure.

heading: 6 Related Work
text: Previous work on the utility of NLI for faithfulness led to mixed conclusions. In summarization, Falke et al. (2019) and Kryscinski et al. (2020) find out-of-the-box models have only limited utility in a faithfulness setting. In Wang et al. (2020), an NLI model is outperformed by a question generation/answering (QA/QG)-based method. In contrast, Maynez et al. (2020) find that a similar NLI model vastly outperforms a QA/QG metric on their data. In knowledge-grounded dialogue, Dziri et al. (2022), Gupta et al. (2022) and Honovich et al. (2021) find out-of-the-box models underperform.
To improve NLI models for faithfulness in summarization, Kryscinski et al. (2020) propose FactCC, which is trained on artificially noised summaries. Utama et al. (2022) propose a controllable generation model to generate artificial faithfulness data. In knowledge-grounded dialogue, Dziri et al. (2022) and Gupta et al. (2022) combine noising techniques to generate additional training data for
NLI-based faithfulness models. In contrast to our work, these approaches a) generate training data from external sources, instead of directly augmenting NLI data, and b) do not explicitly focus on reconciling differences between NLI and faithfulness with their augmentation. Outside of augmentationbased approaches, Goyal and Durrett (2020) propose to train NLI models to label faithfulness at the dependency arc level.

heading: 7 Conclusion
text: We have demonstrated that with a small number of focused adaptations, even a relatively small NLI model can robustly predict faithfulness. We have:
1. Shown that NLI-based metrics can be incompatible with task-specific requirements and identified and fixed one such incompatibility in dialogue with an augmentation strategy.
2. Demonstrated the importance of contradiction probability for scoring and that the underlying mechanism is the high reliability of NLI contradiction scores for detecting unfaithfulness
3. Shown that using Monte-Carlo dropout improves metric performance.
Our improved NLI model significantly improves over its baseline across many corpora and outperforms all competitors in average score on TRUE, while being much more efficient at inference.
Our work suggests that strong improvements are possible for NLI-based faithfulness metrics, by combining data augmentation with adapted NLI score computation. We hope this finding will spurn advances in cheap and robust NLI for faithfulness.

heading: 8 Limitations
text: Some of the summarization datasets annotated for faithfulness are relatively small, which makes score estimates uncertain. Furthermore, many datasets contain only output from a limited number of generation systems, which makes it hard to properly account for potential biases towards certain generation systems that may confound scores (see Pagnoni et al. (2021)). These concerns are, however, alleviated to some extent since we study trends across many independently created datasets, which makes it less likely for a single bias to persist in all of them. Furthermore the availability of generation and thus annotated faithfulness data limits our experiments to English. Finally, it remains
unclear whether our results would still provide advantages when applied to larger models such as T5-11B, whose parameter count makes experimentation infeasible on the hardware available to us.

heading: 9 Ethics Statement
text: Faithfulness metrics help reduce the amount of incorrect information generated by NLG systems, reducing the risk associated which such generations. However, faulty or unreliable faithfulness metrics might cause harm by incorrectly classifying faithful content as unfaithful and vice versa.
We run all experiments on publicly available data that has been specifically constructed for faithfulness evaluation. The underlying publication has been published at a conference whose review process involved an ethics review. For a specific discussion of the human effort involved in creation of the datasets we refer the reader to the original publications.

heading: A Augmentation Training Details
text: A.1 Augmentation Phrases Table 6 lists our manually curated list of phrases inserted during data augmentation. All phrases were derived via a small manual error analysis on the Base model.
We broadly divide our phrases into three categories: introductory statements, hedging, and sentiment statements. For each instance in ANLI, one random phrase from the list is prepended to the hypothesis. We use all three rounds of ANLI annotations. This results in 162,865 augmented instances
which, together with the original ANLI instances, leads to a total of 325,730 training instances.
A.2 Hyperparameters Table 7 lists the hyperparameter settings for our model. We use the same optimizer hyperparameters as Laurer et al. (2022) except for an increased batch size and the learning rate. For the latter we tested three learning rates (5e− 6, 5e− 2, 5e− 1) and select the one that provided the best loss on the augmented ANLI validation set. We initially ran models for 10,000 steps with a checkpoint every 1,000 steps and selected the checkpoint with the lowest loss on the augmented ANLI validation set. Later we reduced the number of training steps to 2,000 since we found we would usually select an early checkpoint as validation loss increased later in training, likely related to overfitting on the augmented data.
A.3 Training We use the DeBERTa implementation in the huggingface transformers library (Wolf et al., 2020) and trained our model on a single node using two RX6800 GPUs, with one training run taking about three hours. Later experiments with fewer steps cut that time by 80%.

heading: B Dataset Bias in BEGIN
text: BEGIN is the only dialogue corpus on which first person pronoun occurrence shows a strong (negative) correlation with faithfulness (see Table 3). Since there is nothing in the annotation guidelines that would explain this correlation, we instead hypothesize that this is the consequence of a model induced bias in the data. Specifically, we hypothesize that one of the two models in BEGIN is (1) more likely to generate personal statements and (2) less likely to generate faithful responses.
To avoid confusion in the remainder of this section, we highlight that there are two variants of BEGIN:
BEGIN-v1 is the variant used in TRUE. It contains labeled generations by a fine-tuned GPT-
2 base (Radford et al., 2019) and a fine-tuned T5 base model (Raffel et al., 2020) on the Wizard of Wikipedia dataset (Dinan et al., 2019).6
BEGIN-v2 is a more recent variant of BEGIN that is not part of TRUE. In addition to new instances generated by T5 and GPT-2 it contains outputs from two additional models. It also has a revised annotation procedure. When we refer to BEGIN-v2, we exclusively mean the Wizard of Wikipedia subset.
Unfortunately, BEGIN-v1 does not allow us to retrieve which model generated which instance. This makes it impossible to directly investigate for model bias. However, BEGIN-v2 includes outputs by the same two models, fine-tuned on the same data. Since we only need corpus level statistics to verify our assumptions, we conduct our analysis on the GPT-2 and T5 instances in BEGIN-v2.
To verify (1), we compute the correlation between a binary variable indicating which model generated each instance (T5: 0, GPT-2: 1) and firstperson pronoun occurrence. We find a positive correlation (Kendall’s τ wrt. to I-pronoun occurrence: 0.18, p < 0.001), indicating that GPT-2 generates outputs including more first-person pronouns.
To investigate whether GPT-2 is also more likely to be unfaithful, i.e. to verify (2), we compute the correlation between the binary model indicator variable and a faithfulness variable that is 1 when the output is labelled as Fully attributable and 0 otherwise. We find a negative correlation (Kendall’s τ wrt. to Faithfulness: −0.25, p < 0.001), supporting our hypothesis that GPT-2 is also overall less faithful. To ensure that this is not an effect of additional personal statements leading to more unfaithful generations, we conduct the same analysis only on instances where we identify no first-person pronouns. We find a similarly strong negative correlation of −0.29 (p < 0.001).
Our analysis shows that GPT-2 produces both overall less faithful outputs and more first-person pronouns than T5. Since BEGIN-v1 contains only outputs from T5 and GPT-2 this suggests that the root cause for the negative correlation between faithfulness label and first-person pronoun occurrence in BEGIN-v1 is model bias confounding faithfulness and first-person pronoun occurrence.
6The relevant data can be found at https://raw. githubusercontent.com/google/BEGIN-dataset/ 5fa0cb0dde0e653d2016724a52a5ca27fe8b6a3f/dev_05_ 24_21.tsv
B.1 Dataset Bias in BEGIN-v2 We conduct a preliminary study to investigate whether similar biases also exist in BEGIN-v2.
We observe that while BEGIN-v2 uses data from four dialogue systems, a majority of faithful generations is produced by a single system called CTRLDIALOG (Rashkin et al., 2021). CTRL-DIALOG is specifically trained to generate less subjective text, which we hypothesize might result in fewer first person pronouns. Since CTRL-DIALOG also produces more faithful texts, this would lead to a negative correlation between faithfulness and first person pronouns, similar to what we observe on BEGIN-v1.
We verify this assumption by computing the correlation of a binary variable indicating an instance has been generated by CTRL-DIALOG with a) the faithfulness labels on BEGIN-v2 and b) first-person pronoun occurrence. We find that an instance being generated by CTRL-DIALOG is positively correlated with it having a faithful label (Kendall τ w.r.t. faithfulness: 0.48, p< 0.001) while being negatively correlated with the number of pronouns (Kendall τ w.r.t. I-pronoun occurrence: - 0.34, p< 0.001). This suggests future evaluations on the BEGIN-v2 might run into similar bias issues.

heading: C Dataset Statistics
text: We report the number of instances, as well as the class distribution of TRUE in Table 8.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
8
3 A2. Did you discuss any potential risks of your work? 9
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? 1,3

heading: 3 B1. Did you cite the creators of artifacts you used?
text: 1,3

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: 1,9

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? 9
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Most data is machine generated and thus unlikely to reveal personal information. All data is also already publicly available and has been introduced in peer-reviewed publications, providing an additional safeguard.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? We discuss the limitation to English in Section 9.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Appendix C
C 3 Did you run computational experiments? 3,4

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Appendix A
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Appendix A

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 4

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 5.2,Appendix A
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: This suggests future evaluations on the BEGIN-v2 might run into similar bias issues