DE nsity: Open-domain Dialogue Evaluation Metric using Density Estimation (2023)

Abstract: Despite the recent advances in open-domain dialogue systems, building a reliable evaluation metric is still a challenging problem. Recent studies proposed learnable metrics based on classification models trained to distinguish the correct response. However, neural classifiers are known to make overly confident predictions for examples from unseen distributions. We propose DENSITY, which evaluates a response by utilizing density estimation on the feature space derived from a neural classifier. Our metric measures how likely a response would appear in the distribution of human conversations. Moreover, to improve the performance of DENSITY, we utilize contrastive learning to further compress the feature space. Experiments on multiple response evaluation datasets show that DENSITY correlates better with human evaluations than the existing metrics.

## introduction automatic evaluation is essential in developing various natural language generation systems, such as machine translation @xcite and summarization @xcite . a common practice for evaluating the generation quality is to compute the similarity of the generated outputs against ground-truth references @xcite @xcite . in open-domain dialogue areas, however, the set of potential responses for a single dialogue history is extremely large, as a conversation can evolve in many ways. due to this very nature of dialogues, reference-based metrics show a poor correlation with human evaluations @xcite . to remedy this, recent studies proposed various reference-free and model-based metrics for dialogue evaluation. many of them focus on estimating the relevance of a response to a dialogue history. for instance, @xcite propose a classification model that is trained to distinguish a correct response for a dialogue history from irrelevant responses. after training, the model is used to evaluate responses by predicting how likely the response would follow the dialogue history. these classifierbased metrics have shown a higher correlation with the human evaluations than the reference-based metrics @xcite @xcite . however, the goal of training such metrics is to find the decision boundary of classifying the training examples. as shown in fig. 1 , if an irrelevant example from an unseen distribution is far from the decision boundary, but is on the same side as the correct train responses, a neural classifier will incorrectly give a high score @xcite . therefore, a metric that assumes that a generated response comes from a distribution similar to the training data is not reliable. due to this misalignment of goals, classifier-based metrics may not be suitable for evaluating open-domain dialogue systems. a similar misalignment is found in different tasks that utilize neural classifiers, such as out-of-distribution (ood) detection. instead of using a prediction score from classifiers, studies utilized an alternative approach, in which the goal is to detect abnormal examples by estimating their densities on the feature space of a classifier @xcite @xcite , and showed impressive results in ood detection. inspired by the benefits of the density estimation approaches in ood detection, we propose density, a new dialogue evaluation metric using density estimation. density measures the density of the response on the feature distribution of human conversations. specifically, a response selection model is utilized as a feature extractor to obtain representations of both the human responses in the dialogue corpus and the system generated response. human response features are fitted on a multivariate gaussian distribution, and the density of the generated response on the human distribution is estimated using the mahalanobis distance. moreover, we adopt contrastive learning to further compress the features of appropriate human responses. looking at the right figure on fig. 1 , unlike the classifier-based metric, density estimation properly assess the evaluation examples, and assign correct scores to relevant and irrelevant responses. preliminary studies suggest that our density estimation based metric can be more robust to various failures of dialogue systems than the classifier-based metrics. experiments on four turn-level response evaluation datasets show that density correlates better with the human evaluation than other metrics. we summarize our contributions as follows: ## related work building a reliable automatic metric for opendomain dialogue systems is a difficult task. traditional reference-based metrics, such as bleu @xcite , rouge @xcite , or me-teor @xcite , show a low correlation with human evaluations @xcite . due to the lack of a reliable automatic metric, many studies rely on human annotators to manually evaluate their systems, which can be expensive and time-consuming. to resolve this, lowe et al. (2017) propose a supervised regression model to estimate the quality of dialogue response directly. despite its superior performance, the supervised regression model requires human-annotated quality scores for training, which reduces the overall generalizability of such models. therefore, recent studies propose unsupervised learning-based metrics to evaluate the relevance of a response to a given dialogue history. for instance, tao et al. (2018) train an classification model that learns to discriminate the original response from randomly sampled negative responses. furthermore, researchers have extended these classifier-based metrics with various techniques. ghazarian et al. ( 2019 ) leverage pretrained language models (lms) to improve the evaluation performance. several works aim to make hard negative samples used in training through various strategies @xcite @xcite @xcite . sai et al. (2020) suggest a pre-training strategy for dialogue evaluation along with a public release of an human-annotated adversarial dataset. @xcite leverage an external knowledge source @xcite to augment an evaluation model. @xcite propose a new graph-based model to focus on the interactive and multi-turn natures of a dialogue. another line of research evaluates a response by measuring the likelihood of words in a response @xcite . this approach usually employs pre-trained lms @xcite to estimate the likelihood. our work is distinct from previous studies in that we evaluate a response by measuring its similarity to human responses by exploiting the rich information presented in the feature space of a neural network. numerous studies propose to understand and evaluate artificial responses from neural generation models. for instance, @xcite report that neural language models often create incoherent and repetitive sequences. @xcite compare the distribution of a generated text against the ones written by humans in a quantized embedding space. in the dialogue domain, @xcite measure the distance between the distributions of generated conversations and real-world conversations to compare at the system-level. unlike the previous studies, we focus on measuring the extent to which a generated conversation is similar to real human conversations. our work is inspired by previous studies that leverage the representation space of neural networks to detect out-of-distribution (ood) or adversarially curated examples @xcite @xcite . instead of detecting such abnormal instances, however, we aim to judge the quality of generated responses by considering their representations. ## density: open-domain dialogue evaluation using density estimation we present density, which evaluates a response by measuring its density on a distribution of human responses. we first train a response selection model that learns to distinguish a correct response from random responses ( §3.1). the selection model is employed as a feature extractor to obtain features of both human responses in a dialogue corpus and a generated response ( §3.2). we then evaluate the generated response on the distribution of human features with gaussian discriminant analysis and mahalanobis distance ( §3.3). we also introduce our contrastive loss to obtain better features in §3.4. fig. 2 illustrates the overall pipeline of density. ## conclusion in this paper, we present density, a new learnable metric for open-domain dialogue systems. den-sity evaluates a response by estimating its density on the distribution of human conversations. empirical results on multiple datasets demonstrate that our metric has a higher correlation with human evaluations than other metrics. we hope that density, a reliable and robust metric for evaluating dialogue systems, contributes to improving evaluation of natural language generation tasks.