Semantic Diversity in Dialogue with Natural Language Inference (2022)

Abstract: Generating diverse, interesting responses to chitchat conversations is a problem for neural conversational agents. This paper makes two substantial contributions to improving diversity in dialogue generation. First, we propose a novel metric which uses Natural Language Inference (NLI) to measure the semantic diversity of a set of model responses for a conversation. We evaluate this metric using an established framework (Tevet and Berant, 2021) and find strong evidence indicating NLI Diversity is correlated with semantic diversity. Specifically, we show that the contradiction relation is more useful than the neutral relation for measuring this diversity and that incorporating the NLI model’s confidence achieves state-of-the-art results. Second, we demonstrate how to iteratively improve the semantic diversity of a sampled set of responses via a new generation procedure called Diversity Threshold Generation, which results in an average 137% increase in NLI Diversity compared to standard generation procedures.

## introduction dialogue models often struggle to produce engaging utterances in conversations, tending to generate responses which are common in the training data, such as "ok," "yeah," or "i don't know" @xcite . while these responses are appropriate for a wide variety of contexts, their over-production can result in a dull conversation @xcite . an evaluation task has emerged that consists of measuring the diversity of chitchat model responses over a test set. while some past work uses human evaluation to measure model response diversity according to engagingness, specificity, or interestingness @xcite @xcite , several automated metrics have also been proposed to measure diversity of model responses. some metrics measure lexical diversity, typically via n-gram overlap (li @xcite or computing the bleu score @xcite among model responses generated from the test set. other past work attempts to measure semantic diversity via repurposing sentence similarity metrics @xcite @xcite . we propose a new metric aimed at measuring semantic diversity by leveraging a natural language inference (nli) model to score a set of multiple dialogue model responses for a single conversation, as illustrated in figure 1 . nli is a three-way classification task to determine whether one sentence entails, contradicts, or is neutral toward a second sentence. we hypothesize that a diverse set of responses for a conversation captures contradictory ways one could respond, which can be measured by the nli model. we aggregate the contradiction, neutral, and entailment predictions among pairs of responses from the set and combine the predictions into a new diversity metric, called nli diversity. we additionally explore two modifications of nli diversity. first, because the neutral prediction may be indicative of diversity, we propose neutral nli diversity, where neutral predictions are weighted the same as contradiction predictions. second, since our baseline nli diversity method does not take into account the confidence of the model's prediction, we propose confidence nli diversity, which aggregates the probability mass of the model's predicted class instead of aggregating the number of predictions for each class. we assess nli diversity using tevet and berant (2021)'s diversity metric evaluation framework, finding that nli diversity is highly correlated both with human judgments of diversity and with the diversity parameter, a gold standard diversity value used to generate the set of responses. confidence nli diversity achieves state-of-the-art performance in terms of correlation with semantic diversity. also, through an ablation study, we find positive, neutral, and negative correlations between human judgments and the number of contradiction, neutral, and entailment predictions, respectively. we next explore the use of a dialogue model to generate a set of candidate responses with a minimum target level of semantic diversity, such as 10 contradictions. our new generation procedure, diversity threshold generation, iteratively improves a set of model responses until this intended threshold is reached. if a set of sampled responses does not meet the intended threshold, the lowest-scoring response is thrown out and a new response is sampled until the diversity threshold is reached. we show this procedure results in a more diverse set of responses than the original sampled set, often with only a few resampled responses. results of automated analysis shows relevancy is maintained from initial to final sets of responses. in summary, our contributions are: tevet and berant (2021)'s framework, that measures semantic diversity and interrogates the relationship between contradiction and neutral predictions and diversity, • confidence nli diversity, a diversity metric which obtains state-of-the-art performance on semantic diversity, • a new dialogue generation procedure, diversity threshold generation, which continues sampling responses until an intended diversity threshold, defined using nli diversity, is reached, • experimental results indicating dialogue models are able to generate diverse responses using diversity threshold generation with minimal loss in relevancy. ## related work past work has explored lexical and semantic diversity metrics as well as ways of evaluating these metrics. we also draw from work in nli and generating diverse sets of hypotheses. ## nli diversity metric we propose three diversity metrics in the multi-response diversity setting which leverage the predictions of an nli model. two metrics (baseline and neutral) aggregate the nli model's class predictions and one metric (confidence) aggregates the weight of these predictions. ## evaluation of nli diversity we evaluate nli diversity by computing the correlation between the metric and both human labels and diversity parameter labels. below we first describe the models and data and then present the results of the evaluation. ## diversity threshold generation we have verified that nli diversity is both able to capture semantic diversity and aligns with human judgements. we can additionally use nli diversity to define a straightforward desired diversity threshold, div thresh for a set of model-generated responses, u 1 , ..., u n . for example, we might intend there to be 10 contradictions within the set. we propose a generation procedure, diversity threshold generation, designed to iteratively increase the diversity of a set of responses for a conversation. for a conversation, diversity threshold generation begins by sampling n responses. we score the diversity of these responses using a diversity metric, div_metric(u 1 , ..., u n ). if the diversity score falls above div thresh , the process is finished. if, however, the score falls below div thresh , we identify the model response which contributes least to the diversity score by calculating div_metric(u 1 , ..., u n-1 ) for each sub-group of model responses of size n -1. we discard the model response not present in the highestscoring subgroup and resample a new response. we re-calculate div_metric(u 1 , ..., u n ) and if div_metric(u 1 , ..., u n ) > div thresh , the process finishes. we continue resampling until the maximum cutoff of s is reached. ## discussion limitations. while nli diversity is highlycorrelated with human judgements of diversity, it is limited by the nli model chosen. compared to sent-bert, the dataset used to train the nli model is limited in scope. while our experiments showed that an nli model trained on more datasets (combined) did not perform better than mnli, future work can more explicitly explore the effect of more generalized data on nli diversity. this work is limited by automatic evaluation metrics for diversity and relevance. future work should conduct additional human validation of model responses. more work could also be done to examine cases where the model was not able to generate diverse set, such as when humans also find creating a diverse set of responses difficult. future work. our results showed confidence nli diversity was highly correlated with both future work should compare the resulting diverse responses in a conversational context. studies could be conducted where chatbot users or dialogue writers can choose the way they want the model to respond, similar to @xcite . ## conclusion we propose a novel semantic diversity metric, nli diversity, which is highly correlated to human judgments. confidence nli diversity achieves state-ofthe-art results on measuring semantic diversity. we propose diversity threshold generation to incentivize production of diverse sets of responses for a conversation. this results in more diverse sets of responses than originally sampled for multiple models, datasets, and metrics while maintaining relevancy, and can also be used to investigate a model's ability to produce diverse responses. metric model dataset utterance overlap nli dg daily 2.63 emp 2.42 bb daily 1.78 emp 1.73 distinctn dg daily 2.89 emp 0.87 bb daily 1.51 emp 1.65 sent-bert dg daily 3.11 emp 3.0 bb daily 1.56 emp 1.64