SlowGuess/NAACL_2022_OCR huggingface dataset:
Katherine Stasaski and Marti A. Hearst
{katie_stasaski, hearst}@berkeley.edu
Generating diverse, interesting responses to chitchat conversations is a problem for neural conversational agents. This paper makes two substantial contributions to improving diversity in dialogue generation. First, we propose a novel metric which uses Natural Language Inference (NLI) to measure the semantic diversity of a set of model responses for a conversation. We evaluate this metric using an established framework (Tevet and Berant, 2021) and find strong evidence indicating NLI Diversity is correlated with semantic diversity. Specifically, we show that the contradiction relation is more useful than the neutral relation for measuring this diversity and that incorporating the NLI model's confidence achieves state-of-the-art results. Second, we demonstrate how to iteratively improve the semantic diversity of a sampled set of responses via a new generation procedure called Diversity Threshold Generation, which results in an average  $137\%$  increase in NLI Diversity compared to standard generation procedures.
Dialogue models often struggle to produce engaging utterances in conversations, tending to generate responses which are common in the training data, such as "OK," "Yeah," or "I don't know" (Li et al., 2016). While these responses are appropriate for a wide variety of contexts, their over-production can result in a dull conversation (See et al., 2019).
An evaluation task has emerged that consists of measuring the diversity of chitchat model responses over a test set. While some past work uses human evaluation to measure model response diversity according to engagingness, specificity, or interestingness (Li et al., 2016; See et al., 2019; Ghandeharioun et al., 2019), several automated metrics have also been proposed to measure diversity of model responses. Some metrics measure lexical diversity, typically via n-gram overlap (Li
The Original Conversation
Speaker 1: I hated the hand-medowns to wear when I was a kid.
Speaker 2: That's understandable. Every kid likes new clothes.
A Set of Next Responses
Figure 1: Illustration of NLI Diversity using human responses from DailyDialog++. Contradictions are weighted by 1, entailments by -1, and neutrals by 0, so the score is  $(2 \times 1) + (3 \times 0) + (1 \times -1) = 1$ .
et al., 2016) or computing the BLEU score (Zhu et al., 2018) among model responses generated from the test set. Other past work attempts to measure semantic diversity via repurposing sentence similarity metrics (Tevet and Berant, 2021; Zhang et al., 2020a; Cer et al., 2017).
We propose a new metric aimed at measuring semantic diversity by leveraging a Natural Language Inference (NLI) model to score a set of multiple dialogue model responses for a single conversation, as illustrated in Figure 1. NLI is a three-way classification task to determine whether one sentence entails, contradicts, or is neutral toward a second sentence. We hypothesize that a diverse set of responses for a conversation captures contradictory ways one could respond, which can be measured by the NLI model. We aggregate the contradiction, neutral, and entailment predictions among pairs of
Human Language Technologies, pages 85 - 98
responses from the set and combine the predictions into a new diversity metric, called NLI Diversity.
We additionally explore two modifications of NLI Diversity. First, because the neutral prediction may be indicative of diversity, we propose Neutral NLI Diversity, where neutral predictions are weighted the same as contradiction predictions. Second, since our Baseline NLI Diversity method does not take into account the confidence of the model's prediction, we propose Confidence NLI Diversity, which aggregates the probability mass of the model's predicted class instead of aggregating the number of predictions for each class.
We assess NLI Diversity using Tevet and Berant (2021)'s diversity metric evaluation framework, finding that NLI Diversity is highly correlated both with human judgments of diversity and with the diversity parameter, a gold standard diversity value used to generate the set of responses. Confidence NLI Diversity achieves state-of-the-art performance in terms of correlation with semantic diversity. Also, through an ablation study, we find positive, neutral, and negative correlations between human judgments and the number of contradiction, neutral, and entailment predictions, respectively.
We next explore the use of a dialogue model to generate a set of candidate responses with a minimum target level of semantic diversity, such as 10 Contradictions. Our new generation procedure, *Diversity Threshold Generation*, iteratively improves a set of model responses until this intended threshold is reached. If a set of sampled responses does not meet the intended threshold, the lowest-scoring response is thrown out and a new response is sampled until the diversity threshold is reached. We show this procedure results in a more diverse set of responses than the original sampled set, often with only a few resampled responses. Results of automated analysis shows relevancy is maintained from initial to final sets of responses.
- A novel diversity metric, NLI Diversity, evaluated using Tevet and Berant (2021)'s framework, that measures semantic diversity and interrogates the relationship between Contradiction and Neutral predictions and diversity,
- Confidence NLI Diversity, a diversity metric which obtains state-of-the-art performance on semantic diversity,
- A new dialogue generation procedure, Diversity Threshold Generation, which continues
sampling responses until an intended diversity threshold, defined using NLI Diversity, is reached,
- Experimental results indicating dialogue models are able to generate diverse responses using Diversity Threshold Generation with minimal loss in relevancy.
Past work has explored lexical and semantic diversity metrics as well as ways of evaluating these metrics. We also draw from work in NLI and generating diverse sets of hypotheses.
2.1 Measuring Model Response Diversity
Traditionally, a model's diversity has been measured in terms of its predictions over the test set (Li et al., 2016), which we call Test Set Diversity. In this setup, the model predicts one response for each conversation in the test set (containing  $n$  conversations), resulting in  $n$  predictions. The diversity measure is computed over these  $n$  predictions, resulting in a score over the entire test set.
The notion of diversity we investigate, however, measures the model's ability to generate a set of responses for a single conversation (Zhang et al., 2019; Tevet and Berant, 2021), which we call Multi-Response Diversity. Instead of generating one response for each of the conversations in the test set, we evaluate a model's ability to generate  $m$  responses for each of the  $n$  conversations.
As shown by Tevet and Berant (2021), metrics which have been proposed in the Test Set Diversity setting can still be applied in the Multi-Response Diversity setting, however, by treating each set of  $m$  responses as its own "test set" and averaging over the  $n$  total sets.
2.2 Diversity Metrics
Lexical diversity metrics measure differences in word choice, as opposed to diversity of content. Li et al. (2016) propose distinct-n, which measures the number of unique n-grams generated divided by the total number of n-grams generated in the Test Set Diversity setting. Some past work has applied this metric to the Multi-Response Diversity setting (Tevet and Berant, 2021). Cao and Clark (2017) propose examining the percent of unique responses over the test set. Other past work has proposed using BLEU score over a set of model responses in the Test Set Diversity setting (Zhu et al., 2018).
Semantic diversity metrics, on the other hand, compare diversity of the content present in each response. Many of these measures are adapted from semantic similarity scores, since lower similarity can indicate higher diversity (Tevet and Berant, 2021). BERTScore measures the similarity of BERT embeddings for each token in two sentences (Zhang et al., 2020a). Bert-STS assigns a score based on the semantic similarity of two sentences (Tevet and Berant, 2021). The Sent-BERT metric computes cosine similarity between BERT sentence embeddings (Reimers and Gurevych, 2019). Larson et al. (2019) propose identifying diverse paraphrases by identifying embedding outliers.
Other past work has used human evaluation to measure a model's diversity. Li et al. (2016) ask humans to choose the better of two responses based on specificity to the past conversation. See et al. (2019) ask humans to rank dialogue responses on a variety of factors, including interestingness and inquisitiveness. Tevet and Berant (2021) compare participants' ability to judge diversity of a set of responses in two ways: (i) by ranking one response as more diverse than a second response and (ii) by judging the diversity of a single response on a Likert scale, finding that participants were equally able to judge diversity in both conditions. They also find that human judges are better at distinguishing semantic diversity than lexical diversity.
Other past work has incorporated diversity metrics into the dialogue dataset creation pipeline. Stasaski et al. (2020) propose a method which measures the diversity of a crowdworker's contributions compared to a corpus, using that information to determine when to stop collecting data from the worker. This results in a more diverse dataset.
2.3 Evaluation of Diversity Metrics
Tevet and Berant (2021) propose a framework to examine the reliability of diversity metrics. They propose the notion of a diversity parameter, which is used to generate a set of model responses, e.g., the  $p$ -value in nucleus sampling, which specifies the vocabulary probability distribution cutoff used to restrict sampling to the most-likely words whose combined likelihood  $\geq p$ . If  $p$  is higher, the set of responses should have higher diversity, and vice-versa. This diversity parameter is treated as a gold standard for a set of responses' diversity. Diversity metrics assign scores in the Multi-Response Diversity condition and are evaluated in terms of
correlation to the diversity parameter. They further propose two datasets to evaluate diversity metrics: one which includes model responses and contains varying levels of lexical diversity and one which is human-created and maintains high lexical diversity to allow focused evaluation of semantic diversity.
2.4 Natural Language Inference
Natural Language Inference is a task aimed at predicting whether one sentence contradicts, entails, or is neutral towards a second sentence. Models for NLI are typically trained using one of two datasets: Stanford Natural Language Inference (SNLI) (Bowman et al., 2015) or Multi-Genre NLI (MNLI) (Williams et al., 2018). More recent datasets include FEVER (Thorne et al., 2018; Nie et al., 2019), adapted from a fact-checking dataset, and ANLI (Nie et al., 2020), collected in an adversarial human-in-the-loop procedure. With the rise of transformer architectures, models have achieved high performance on NLI tasks (Liu et al., 2019).
In a dialogue setting, NLI has been used to improve consistency between a persona and model responses over the course of a conversation by integrating an NLI-based reward into a reinforcement learning training procedure (Song et al., 2020).
To our knowledge, however, NLI has not been used to measure the diversity of model responses in either the Test Set Diversity or the Multi-Response Diversity setting.
2.5 Generating Diverse Sets of Hypotheses
While work has only recently begun to explore the task of generating multiple dialogue responses to a conversation (Zhang et al., 2019; Tevet and Berant, 2021), past work has explored generating diverse sets of hypotheses in some other application areas. Carbonell and Goldstein (1998) explored using Maximal Mutual Relevance to reduce redundancy without sacrificing relevancy in document selection for summarization. Batra et al. (2012) proposed a greedy iterative algorithm to generate diverse, probable hypotheses for multiple vision tasks. Most related to our work is Gimpel et al. (2013), which applied Batra et al. (2012)'s approach to machine translation, generating a set of translations instead of a single translation. In contrast to Gimpel et al. (2013), by holding the sampling procedure constant throughout the iterative process, our method can explore the extent to which diversity can be increased without altering standard decoding practices.
3 NLI Diversity Metric
We propose three diversity metrics in the Multi-Response Diversity setting which leverage the predictions of an NLI model. Two metrics (Baseline and Neutral) aggregate the NLI model's class predictions and one metric (Confidence) aggregates the weight of these predictions.
3.1 Baseline NLI Diversity
We propose a new metric, called Baseline NLI Diversity, which uses an NLI model's predictions to measure diversity. More formally, for a given conversation,  $c$ , and a dialogue generation model  $M$ , a set of utterances  $u_{1}, \ldots, u_{n}$  is produced by the model. Each pair of utterances is compared in both directions using an NLI model,  $NLI(u_{1}, u_{2}), NLI(u_{2}, u_{1}), \ldots, NLI(u_{n}, u_{n-1})$ .
The NLI model predicts a distribution over the three potential classes: contradiction, neutral, and entailment. We take the argmax over these classes, resulting in a list of NLI predictions,  $NLI_{preds}(NLI(u_1, u_2), \dots, NLI(u_{n-1}, u_n))$  of size  $n(n-1)$ . To produce an overall diversity score for  $NLI_{preds}(u_1, \dots, u_n)$ , we assign each of these classes a value representing their diversity, denoted  $NLI_{score}(NLI_{preds}(u_1, \dots, u_n))$ .
We hypothesize that larger numbers of entailment predictions found in a set of model-generated utterances is indicative of a lack of diversity; similarly, larger number of contradiction predictions is indicative of a larger amount of diversity. Because we want a higher value of  $NLI_{score}$  to indicate higher diversity, we assign values as:
$$
N L I _ {s c o r e} = \left\{ \begin{array}{l l} 1 & \text {i f c o n t r a d i c t i o n} \\ 0 & \text {i f n e u t r a l} \\ - 1 & \text {i f e n t a i l m e n t} \end{array} \right.
$$
The sum of the  $NLI_{score}$  values for the set of utterances results in the final NLI Diversity score, formally defined as:
Baseline NLIDiversity  $=$
$$
\sum_ {u _ {i}, u _ {j} \in u _ {1},.., u _ {n}} N L I _ {s c o r e} (N L I _ {p r e d} (N L I (u _ {i}, u _ {j}))
$$
While the Baseline NLI Diversity metric aggregates all classes, we also investigate the separate number of entailment, contradiction, and neutral predictions in  $NLI_{preds}$ , denoted # Entailment, # Contradiction, and # Neutral, respectively.
3.2 Neutral NLI Diversity
Our primary hypothesis is that contradictions indicate diversity and entailments indicate lack of diversity. Because it is unclear what the role of neutrals might be, we explore a version of NLI Diversity which weights neutral and contradiction predictions as equally diverse. This metric is the same as Baseline NLI Diversity except the  $NLI_{score}$  used to assign values is:
$$
N L I _ {s c o r e \_ n e u t r a l} = \left\{ \begin{array}{l l} 1 & \text {i f c o n t r a d i c t i o n} \\ 1 & \text {i f n e u t r a l} \\ - 1 & \text {i f e n t a i l m e n t} \end{array} \right.
$$
3.3 Confidence NLI Diversity
Because the prior two NLI Diversity metrics do not incorporate the confidence of the NLI model's class predictions, we explore an additional metric which incorporates this value. Letting  $conf_{class}(u_1, u_2)$  represent the model's probability mass assigned to the predicted NLI class after softmax, the function is defined as:  $NLI_{score\_confidence} =$
$$
\left\{ \begin{array}{l l} 1 \times c o n f _ {c o n} (u _ {1}, u _ {2}) & \text {i f c o n t r a d i c t i o n} \\ 0 & \text {i f n e u t r a l} \\ - 1 \times c o n f _ {e n t} (u _ {1}, u _ {2}) & \text {i f e n t a i l m e n t} \end{array} \right.
$$
Intuitively, instead of assigning a 1 value for a contradiction prediction, this metric assigns the probability of the contradiction class. Likewise, instead of a -1 for an entailment prediction, this metric assigns the negative probability mass of the entailment class.
4 Evaluation of NLI Diversity
We evaluate NLI Diversity by computing the correlation between the metric and both human labels and diversity parameter labels. Below we first describe the models and data and then present the results of the evaluation.
We explore two NLI models: a Roberta-large model (Liu et al., 2019) fine-tuned on the Multi-Genre NLI (MNLI) Corpus (Williams et al., 2018) $^{1}$  and a Roberta-large model fine-tuned on a combination of MNLI, SNLI, FEVER, and ANLI $^{2}$ , both
1https://huggingface.co/ roberta-large-mnli 2https://huggingface.co/ynie/ roberta-large-snli_mnli_fever_anli_R1_ R2_R3-nli
decTest	Mixed Lexical Diversity;
Mixed Semantic Diversity;
Model Generated
Examples:
temp 0.28	“I think he is the most aw-
some guy ever”
“He is the most awesome guy
ever”
temp 0.55	“The unemployment rate is
lower than what it is”
“No but it does make it more
likely to be higher than what
it is”
conTest	High Lexical Diversity;
Mixed Semantic Diversity;
Human Generated
Examples:
high lexical and
low semantic	“Sorry, but I don’t agree.”
“I think you are wrong about
that.”
“Dont be so judgemental, try
to see
things her way.”
high lexical and
high seman-
tic	“You are right that is insane.”
Table 1: Descriptions of diversity datasets from Tevet and Berant (2021). Corresponding temperature parameter (higher is more diverse) or semantic and lexical diversity levels accompany each example.
containing 300M parameters. We refer to these models as NLI Diversity - MNLI and NLI Diversity - Combined, respectively. We do not employ additional fine-tuning of these models.
There are two different English datasets released to evaluate diversity metrics in Tevet and Berant (2021): conTest and decTest, described in Table 1. The conTest dataset is human-created and captures content, or semantic, diversity independent of lexical diversity. Low-diversity examples in this dataset have high lexical diversity but low semantic diversity. This dataset was created by asking crowdworkers to generate sets of utterances with either low or high semantic diversity using varied
language, in order to keep a high level of lexical diversity constant across both conditions.
The decTest dataset includes model-generated responses, with diversity controlled by a decoding parameter, such as a temperature parameter. The dataset can include duplicate responses, and does not attempt to mediate lexical diversity; therefore, low-diversity examples in this dataset may reflect low lexical as well as low semantic diversity.
While the original dataset includes multiple generation tasks, we evaluate on the dialogue task, respGen, which is drawn from Reddit conversations (Hashimoto et al., 2019) $^3$ . There are 200 conversations for each of conTest and decTest for the respGen task, with multiple responses for each conversation (5 for conTest, 10 for decTest).
4.3 Diversity Parameter Correlation
The diversity parameter from Tevet and Berant (2021) represents either a parameter directly used to generate responses via a dialogue model, such as  $p$  in nucleus sampling, or a binary value indicating whether crowdworkers were instructed to generate a high- or low-diversity set of responses. A measure which is able to capture diversity will be positively correlated with this diversity parameter.
Table 2 shows Spearman's correlations between NLI Diversity and the diversity parameter. On the conTest semantic diversity dataset, Confidence NLI Diversity achieves the highest correlation of all metrics (0.62) and approaches human performance. Baseline NLI Diversity performs comparably to the top-performing automatic metric from Tevet and Berant (2021), at 0.59 correlation. We note the  $95\%$  confidence intervals overlaps between Baseline NLI Diversity, Confidence NLI Diversity, SentBERT, and human judgements, indicating a lack of significant differences (see Appendix A). Although Neutral NLI Diversity does relatively poorly on conTest (0.24), it is the highest-performing NLI metric on decTest (0.72), suggesting that incorporating neutral predictions may capture lexical instead of semantic diversity.
A histogram of Confidence NLI Diversity values for low and high semantic diversity sets of responses is shown in Figure 2. We note the lack of large overlap between the distributions of low and high semantic diversity data. In addition to
3In the data released from Tevet and Berant (2021), these files are called con_test_200_with_hds_resp_gen.csv and dec_test_200_with_hds_resp_gen.csv for conTest and decTest, respectively.
Metric	decTest ρ	conTest ρ
Human Performance (ab-sHDS)	0.81	0.63
distinct-n	0.89	0.34
cos-sim	0.89	0.33
BERT-STS	0.81	0.46
Sent-BERT	0.80	0.59
BERTScore	0.87	0.49
Baseline NLI Diversity - MNLI	0.58	0.59
Baseline NLI Diversity - Combined	0.39	0.59
Neutral NLI Diversity	0.72	0.24
Confidence NLI Diversity	0.44	0.62
the correlation results in Sections 4.3 and 4.4, this result indicates the Confidence NLI Diversity metric distinguishes between low and high semantic diversity.
The higher correlation to the diversity parameter leads us to choose NLI Diversity - MNLI instead of Combined for all further experimentation.
4.4 Human Correlation
In this subsection, we examine the NLI Diversity metric's correlation to the human annotations collected by Tevet and Berant (2021). Each set of responses in  $\text{conTest}$  and  $\text{decTest}$  is scored by 10 annotators from 1 (not diverse at all) to 5 (very diverse) with half-point increments. We compute correlation with respect to the averaged rating.
In addition to NLI Diversity, we explore the prediction counts for each category. We expect that a higher # Entailment value will be negatively correlated with diversity because the more pairs of responses that entail each other, the more similar the set of responses is. Similarly, we expect that a higher # Contradiction value will be positively correlated with diversity. Since the NLI Diversity metric incorporates both # Entailment and # Contradiction, we would expect this metric to be highly correlated with human judgments as well.
Figure 2: Histogram of Confidence NLI Diversity for high and low semantic diversity examples.
Table 2: Spearman's  $\rho$  correlations between NLI Diversity metrics and the diversity parameter. Results above the double line are reproduced from Tevet and Berant (2021). Both the best automatic metric and human performance for each dataset are in boldface.
Metric	decTest ρ	conTest ρ
Baseline NLI Diver-sity	0.48	0.63
Neutral NLI Diver-sity	0.69	0.40
Confidence NLI Di-versity	0.41	0.64
# Contradiction	0.26	0.46
# Neutral	0.05	-0.08
# Entailment	-0.48	-0.65
Table 3: Spearman's  $\rho$  correlation between NLI Diversity metrics (MNLI) and human judgments. Negative values indicate higher # Entailment is negatively correlated with diversity.
Spearmean's  $\rho$  rank correlation results between our metrics and the human diversity scores are shown in Table 3. The highest-performing correlation for lexical diversity is the Neutral NLI Diversity (0.69). The highest-performing semantic diversity correlation is Confidence NLI Diversity (0.64). Additionally, Baseline and Confidence NLI Diversity correlations are stronger when evaluating with the conTest dataset than the decTest dataset (an increase of 0.48 to 0.63 for Baseline MNLI and 0.41 to 0.64 for Confidence NLI), indicating these metrics are more correlated with human ratings of semantic diversity than lexical diversity.
Across both datasets, # Entailment is negatively correlated with diversity, # Neutral does not have a strong correlation, and # Contradiction is positively correlated, as hypothesized. This supports
our motivation to use NLI as a diversity metric.
5 Diversity Threshold Generation