DENSITY: Open-domain Dialogue Evaluation Metric using Density Estimation
Year: 2023

Abstract:
Despite the recent advances in open-domain dialogue systems, building a reliable evaluation metric is still a challenging problem. Recent studies proposed learnable metrics based on classification models trained to distinguish the correct response. However, neural classifiers are known to make overly confident predictions for examples from unseen distributions. We propose DENSITY, which evaluates a response by utilizing density estimation on the feature space derived from a neural classifier. Our metric measures how likely a response would appear in the distribution of human conversations. Moreover, to improve the performance of DENSITY, we utilize contrastive learning to further compress the feature space. Experiments on multiple response evaluation datasets show that DENSITY correlates better with human evaluations than the existing metrics.1

heading: 1 Introduction
text: Automatic evaluation is essential in developing various natural language generation systems, such as machine translation (Sutskever et al., 2014) and summarization (See et al., 2017). A common practice for evaluating the generation quality is to compute the similarity of the generated outputs against ground-truth references (Papineni et al., 2002; Lin, 2004; Banerjee and Lavie, 2005; Zhang et al., 2019). In open-domain dialogue areas, however, the set of potential responses for a single dialogue history is extremely large, as a conversation can evolve in many ways. Due to this very nature of dialogues, reference-based metrics show a poor correlation with human evaluations (Liu et al., 2016).
To remedy this, recent studies proposed various reference-free and model-based metrics for dialogue evaluation. Many of them focus on estimating the relevance of a response to a dialogue history.
1Our code is available at https://github.com/ ddehun/DEnsity.
For instance, Tao et al. (2018) propose a classification model that is trained to distinguish a correct response for a dialogue history from irrelevant responses. After training, the model is used to evaluate responses by predicting how likely the response would follow the dialogue history. These classifierbased metrics have shown a higher correlation with the human evaluations than the reference-based metrics (Ghazarian et al., 2019; Mehri and Eskenazi, 2020b; Sinha et al., 2020; Zhang et al., 2021).
However, the goal of training such metrics is to find the decision boundary of classifying the training examples. As shown in Fig. 1, if an irrelevant example from an unseen distribution is far from the decision boundary, but is on the same side as the correct train responses, a neural classifier will incorrectly give a high score (Hendrycks and Gimpel, 2016; Liang et al., 2018). Therefore, a metric that assumes that a generated response comes from a distribution similar to the training data is not reliable. Due to this misalignment of goals, classifier-based metrics may not be suitable for evaluating open-domain dialogue systems.
A similar misalignment is found in different tasks that utilize neural classifiers, such as out-of-
14222
distribution (OOD) detection. Instead of using a prediction score from classifiers, studies utilized an alternative approach, in which the goal is to detect abnormal examples by estimating their densities on the feature space of a classifier (Lee et al., 2018; Winkens et al., 2020; Zhou et al., 2021), and showed impressive results in OOD detection.
Inspired by the benefits of the density estimation approaches in OOD detection, we propose DENSITY, a new Dialogue Evaluation metric using DeNSITY Estimation. DENSITY measures the density of the response on the feature distribution of human conversations. Specifically, a response selection model is utilized as a feature extractor to obtain representations of both the human responses in the dialogue corpus and the system generated response. Human response features are fitted on a multivariate Gaussian distribution, and the density of the generated response on the human distribution is estimated using the Mahalanobis distance. Moreover, we adopt contrastive learning to further compress the features of appropriate human responses. Looking at the right figure on Fig. 1, unlike the classifier-based metric, density estimation properly assess the evaluation examples, and assign correct scores to relevant and irrelevant responses. Preliminary studies suggest that our density estimation based metric can be more robust to various failures of dialogue systems than the classifier-based metrics. Experiments on four turn-level response evaluation datasets show that DENSITY correlates better with the human evaluation than other metrics. We summarize our contributions as follows:
1. We introduce a new reference-free learnable metric for open-domain dialogue systems, which estimates the density of responses on the distribution of human conversations.
2. Extensive experiments on multiple datasets demonstrate that, compared to other baseline metrics, DENSITY correlates better with human evaluations, confirming its superiority.

heading: 2 Related Work
text: Building a reliable automatic metric for opendomain dialogue systems is a difficult task. Traditional reference-based metrics, such as BLEU (Papineni et al., 2002), ROUGE (Lin, 2004), or METEOR (Banerjee and Lavie, 2005), show a low correlation with human evaluations (Liu et al., 2016). Due to the lack of a reliable automatic metric, many
studies rely on human annotators to manually evaluate their systems, which can be expensive and time-consuming. To resolve this, Lowe et al. (2017) propose a supervised regression model to estimate the quality of dialogue response directly. Despite its superior performance, the supervised regression model requires human-annotated quality scores for training, which reduces the overall generalizability of such models. Therefore, recent studies propose unsupervised learning-based metrics to evaluate the relevance of a response to a given dialogue history. For instance, Tao et al. (2018) train an classification model that learns to discriminate the original response from randomly sampled negative responses. Furthermore, researchers have extended these classifier-based metrics with various techniques. Ghazarian et al. (2019) leverage pretrained language models (LMs) to improve the evaluation performance. Several works aim to make hard negative samples used in training through various strategies (Bak and Oh, 2020; Sinha et al., 2020; Gupta et al., 2021; Park et al., 2021; Lee et al., 2022). Sai et al. (2020) suggest a pre-training strategy for dialogue evaluation along with a public release of an human-annotated adversarial dataset. Huang et al. (2020) leverage an external knowledge source (Speer et al., 2017) to augment an evaluation model. Zhang et al. (2021) propose a new graph-based model to focus on the interactive and multi-turn natures of a dialogue. Another line of research evaluates a response by measuring the likelihood of words in a response (Mehri and Eskenazi, 2020b; Pang et al., 2020). This approach usually employs pre-trained LMs (Devlin et al., 2019; Radford et al., 2019) to estimate the likelihood. Our work is distinct from previous studies in that we evaluate a response by measuring its similarity to human responses by exploiting the rich information presented in the feature space of a neural network.
Numerous studies propose to understand and evaluate artificial responses from neural generation models. For instance, Holtzman et al. (2019) report that neural language models often create incoherent and repetitive sequences. Pillutla et al. (2021) compare the distribution of a generated text against the ones written by humans in a quantized embedding space. In the dialogue domain, Xiang et al. (2021) measure the distance between the distributions of generated conversations and real-world conversations to compare at the system-level. Unlike the previous studies, we focus on measuring
the extent to which a generated conversation is similar to real human conversations. Our work is inspired by previous studies that leverage the representation space of neural networks to detect out-of-distribution (OOD) or adversarially curated examples (Lee et al., 2018; Winkens et al., 2020; Xu et al., 2020; Zhou et al., 2021). Instead of detecting such abnormal instances, however, we aim to judge the quality of generated responses by considering their representations.

heading: 3 DENSITY: Open-domain Dialogue Evaluation using Density Estimation
text: We present DENSITY, which evaluates a response by measuring its density on a distribution of human responses. We first train a response selection model that learns to distinguish a correct response from random responses (§3.1). The selection model is employed as a feature extractor to obtain features of both human responses in a dialogue corpus and a generated response (§3.2). We then evaluate the generated response on the distribution of human features with Gaussian discriminant analysis and Mahalanobis distance (§3.3). We also introduce our contrastive loss to obtain better features in §3.4. Fig. 2 illustrates the overall pipeline of DENSITY.

heading: 3.1 Training Response Selection Model for Feature Extraction
text: The response selection model learns to find the next utterance for a given dialogue history among the response candidates. The response candidates contain multiple negative responses that are incorrect and not suitable as the next utterance for the given dialogue history. Formally, c represents a dialogue context that consists of multiple utterances between two speakers. The response candidate C
contains one answer response rp and |C| − 1 negative responses {rni} |C|−1 i=1 that are randomly sampled from a dialogue corpus. The selection model is trained to distinguish a positive pair (c, rp) from the negative pairs (c, rni) as follows:
LRS = − log ef(c,rp)
ef(c,rp) + ∑|C|−1
i=1 e f(c,rni )
(1)
where f(·, ·) denotes the prediction score of the selection model to a context-response pair. We implement the selection model with transformer (Vaswani et al., 2017)-based cross-encoder architecture, where the concatenation of a dialogue history and a response [c; r], along with the [SEP] token, is fed into a pre-trained transformer encoder g. The d-dimensional output representation from the transformer encoder h = g(c, r) ∈ Rd is then transformed into a single scalar value f(c, r) = Whr with a linear layer W ∈ R1×d. In this work, we use the [CLS] vector from the transformer encoder as the output representation.

heading: 3.2 Fitting Gaussian Distribution on Features
text: After training the selection model, we encode all positive training pairs in a dialogue corpus with the encoder g to obtain their output representations H = {hi}Ni=1 = {g(ci, ri)}Ni=1, where N denotes the total number of positive pairs. We then train a single-class generative classifier by fitting the multivariate Gaussian distributions as follows:
µ = 1
N
N∑
i=1
hi,Σ = 1
N
N∑
i=1
(hi−µ)(hi−µ)T (2)
where µ and Σ denote the empirical mean and covariance matrix of features, respectively. Note that both µ and Σ can be calculated only once, and be used at any time for response evaluation.

heading: 3.3 Response Evaluation with Density Estimation
text: To evaluate a response r generated by a dialogue system for a dialogue context c, we first obtain its feature with encoder g. We then estimate its density on the Gaussian distribution N (µ,Σ) using a Mahalanobis distance as follows2:
s(r) = −(g(c, r)− µ)Σ−1(g(c, r)− µ)T (3)
where Σ−1 denotes a pseudo-inverse matrix of Σ. The distance value s(r) of a response r is used as the score assigned by our metric. In other words, we regard the distance between the response and the distribution of human responses as an indicator of its quality. Therefore, a high-quality response will receive a high score from our metric, while a low quality response will receive a low score. Does a density estimation based metric actually work? Note that many previous studies leverage the prediction of a classifier that is trained to distinguish a correct response from others (Tao et al., 2018; Mehri and Eskenazi, 2020b; Sinha et al., 2020), which is similar to our selection model’s prediction score f(c, r). In contrast, our metric
2We omit c from s(r) for simplicity.
only uses the intermediate output of the model to estimate the density of a generated response. In our preliminary study, we compare the classifier-based approaches with density estimation by observing their behaviors on adversarially manipulated responses. Specifically, we use a dataset released by Khalid and Lee (2022) that is designed to probe the robustness of dialogue evaluation metrics. The dataset consists of three components: (1) dialogue history, (2) answer response, and (3) adversarial response. The adversarial response is created by manipulating the original answer response with various strategies. An evaluation metric should assign a higher score to the answer response than the adversarial response. We use the following three adversarial types, which reflect errors that frequently occur in dialogue systems: (1) Repetition: repeats words in the answer, (2) SpeakerSensitiveness: concatenates the last utterance from the dialogue history to the answer, (3) Contradiction: corrupts the contextual information in the answer. We also compute the accuracy on a random negative example to check whether the density estimation based metric can perform well in the original training task. Fig. 3a presents sample responses of different attack types.
From the results in Fig. 3b, we first observe that our density estimation based metric can distinguish random responses on par with a classifier-based metric. In terms of accuracy on the adversarial responses, the density estimation based metric performs better than the classifier-based metric. Previous literature on OOD detection task (Lee et al., 2018; Xu et al., 2020) reports similar trends to our observation, where the softmax-based neural classifiers are prone to make overly confident predictions (Hendrycks and Gimpel, 2016; Liang et al., 2018) on abnormal instances. These results imply that the feature space derived by g contains sufficient information to perform the original task, and the density estimation based metric can be more effective than the classifier-based metric in detecting various failures made by dialogue systems.

heading: 3.4 Enhanced Feature Space with Contrastive Learning
text: Our feature extractor g is trained to make features that can discriminate the correct response from the incorrect ones for the same dialogue history. Therefore, features of a positive pair from a different dialogue history may not be easily distinguished
from features of negative responses, which can reduce the performance of our metric. To resolve this, we adopt a supervised contrastive loss (Khosla et al., 2020; Gunel et al., 2020; Zhou et al., 2021), which regards all positive pairs in the training set as the same class. The loss function encourages the representations of positive pairs to be closer, while increasing the discrepancy from the representations of negative pairs. Formally, given a batch of |B| dialogue history, the training objective for contrastive learning is:
LCL = |B|∑
i=1
−1 P (i)
∑
p∈P (i) log ezpi ·zp/τ∑ a∈B(i) ezpi ·za/τ (4)
where z denotes a L2-normalized [CLS] vector of a context-response pair from the transformer encoder g, and pi denotes the ith positive pair in the batch. P (i) denotes all positive context-response pairs in the batch except for pi, and B(i) denotes all context-response pairs in the batch except for pi.3 τ > 0 is a temperature scaling hyperparameter. The final training objective of selection model is
Ltotal = LRS + λLCL (5)
where λ > 0 is a hyperparameter.

heading: 4 Experiments

heading: 4.1 Dataset
text: We conduct experiments on two representative open-domain dialogue datasets. DailyDialog (Li et al., 2017) is a multi-turn dialogue dataset written by human annotators. The dataset is designed to cover various topics and relationships in our daily life. The dataset consists of 13,118 multi-turn conversations. ConvAI2 (Dinan et al., 2020) is a crowd-sourced dataset, where two speakers continue a conversation with their assigned personal information. The dataset consists of 17,878 multi-turn conversations.
We use four turn-level dialogue evaluation datasets to compare the performance of different evaluation metrics. In the dataset, each example consists of a dialogue history, an answer response, a generated response by a dialogue system, and a quality score judged by human annotators. The details of evaluation datasets are as follows: DailyDialog-Zhao (Zhao et al., 2020) contains 900 evaluation examples from six different dialogue
3|P (i)| and |B(i)| are |B| − 1 and |B| × |C| − 1.
systems. The dataset is derived from the DailyDialog dataset, which is used as a training corpus of dialogue systems and a source of dialogue histories. The “overall” score is used in our experiments. ConvAI2-USR (Mehri and Eskenazi, 2020b) is derived from the ConvAI2 dataset and contains 240 evaluation examples from three dialogue systems. The “Overall Quality” score is used in our experiments. DailyDialog-GRADE and ConvAI2-GRADE are datasets released by Huang et al. (2020), each of which is based on the DailyDialog and ConvAI2 datasets. DailyDialogGRADE and ConvAI2-GRADE contain 300 examples from two systems and 600 examples from four systems, respectively.

heading: 4.2 Baselines
text: We compare our method against the following baseline metrics. Note that the training dataset of reference-free metrics is the same as the original dialogue dataset from which the evaluation dataset is derived. Further implementation details of the baseline metrics are available in Appendix A. BLEU, ROUGE, and METEOR measure word overlap of hypothesis against references. Embedding Average/Greedy/Extrema (Liu et al., 2016) compute the similarity between an answer and generated responses with a distributed word representation. BERTScore (Zhang et al., 2019) use a pre-trained LM to obtain the contextualized embedding of responses for similarity comparison. SimCSE (Gao et al., 2021) adopts a self-supervised contrastive learning for better sentence embeddings. We calculate the cosine similarity between the embeddings of an answer and generated responses. BLEURT (Sellam et al., 2020) is a reference-based metric pretrained on a synthetic dataset for an evaluation of machine translation systems. USR-MLM (Mehri and Eskenazi, 2020b) replaces each token in a response to [MASK], and aggregates the likelihood of each token when they are conditioned on the context by using masked language modeling by BERT (Devlin et al., 2019). GPT2-Coherency (Pang et al., 2020) measures the perplexity of a response conditioned on its dialogue history by using GPT-2 (Radford et al., 2019). FED (Mehri and Eskenazi, 2020a) evaluates a response by measuring the perplexity of follow-up utterances designed by the authors. BERT-RUBER (Ghazarian et al., 2019) replaces
the word embedding layer in the original RUBER (Tao et al., 2018) with BERT. USR-Retrieval (Mehri and Eskenazi, 2020b) learns to distinguish the next utterance of a given dialog history from a random response. After training, the score for which the response is predicted to be the next utterance is used for evaluation. USL-H (Phy et al., 2020) combines the predictions from multiple models trained on different tasks for configurable response evaluation. FlowScore (Li et al., 2021) compares a response’s semantic influences and those expected by a response generation model. DynaEval (Zhang et al., 2021) adopts a graphbased architecture to reflect the interaction between speakers in a dialogue, and is trained to distinguish the original dialogue from the corrupted ones.

heading: 4.3 Implementation Details
text: We use BERT (Devlin et al., 2019) released by Wolf et al. (2020) to initialize our response selection model.4 We set τ and λ in Eq. 4 and Eq. 5 to
4bert-base-uncased is used.
0.1 and 1.0, respectively. The selection model is trained for 10 epochs, and the initial learning rate is set to 5e-5. AdamW (Loshchilov and Hutter, 2018) optimizer is used for optimization, and the linear learning rate scheduler is used with 1000 warm up steps. The maximum number of tokens in the input sequence is set to 256, and the batch size is set to 16. The number of negative responses for a dialogue history is set to 15 for DailyDialog dataset, and 19 for ConvAI2 dataset. We use the square root value of the distance in Eq. 3, since it empirically shows a better performance. Further implementation details are in Appendix B.

heading: 5 Results

heading: 5.1 Main Results
text: Table 1 shows the results of the response evaluation task on each dataset. Pearson correlation coefficient (r) and Spearman’s rank correlation coefficient (ρ) are used to compare the model predictions with human scores. We first observe that DENSITY achieves the highest correlation with human scores in all evaluation datasets. In terms of baseline met-
rics, n-gram overlap-based metrics like BLEU usually show a low correlation with human scores. These observations are consistent with previous studies (Liu et al., 2016), where the metrics relying on a comparison with the answer response are not appropriate in the dialogue domain. Referencebased metrics like SimCSE and BLEURT, which do not rely on the n-gram overlap, show an improved performance, but their correlations are still relatively lower than the reference-free metrics. The reference-free metrics that either measure the likelihood of words in a response (e.g., USR-MLM and GPT2-Coherency) or use the classification model (e.g., BERT-RUBER, USR-Retrieval) usually show a better performance than the reference-based metrics. Notably, classifier-based metrics like BERTRUBER, USR-Retrieval, and USL-H often show competitive performance with DENSITY. DynaEval, a well-known metric for dialogue-level evaluation, usually performs worse than other referencefree metrics, which can be attributed to the different characteristics of turn-level and dialogue-level evaluations. Based on these results, we can confirm the validity and the effectiveness of DENSITY.

heading: 5.2 Correlation Visualization
text: Scatter plots in Fig. 4 present human scores and the prediction scores of each metric. Each point indicates an evaluated response, and the x-axis and y-axis values indicate the human score and metric score, respectively. BLEU, a metric that relies on
word overlap similarity, usually gives low scores close to zero, which makes it hard to be adopted as a reliable metric. Embedding-based metrics like Embedding Extrema and BERTScore tend to give similar scores to responses of different human scores, which fails to discriminate between highquality responses. Several reference-free metrics like RUBER-BERT and GPT2-Coherency make predictions that positively correlate with human scores. USR-Retrieval successfully gives high scores to responses with high human scores, but it also frequently makes overly confident predictions to responses with low human scores. This false-positive problem is relatively alleviated in DENSITY, as the number of high predictions for low human scores is decreased. One possible limitation of our metric is that it makes confident predictions when responses receive a score higher than 0.8 from human annotators, which implies that our model might not be calibrated well. This behavior would be undesirable to ones that hope to find the best response among reasonably well-written responses. Building a well-calibrated metric for general purposes is left as future work.

heading: 5.3 Ablation Study
text: We conduct an ablation study to investigate the impact of each component in our metric. Results are shown in Table 2. We first observe that the correlations of distance-based scoring functions are largely increased with contrastive learning. Such
gain is considerably larger for Mahalanobis distance, while there are cases in which the correlation dropped with Euclidean distance. Regarding the comparison between different approaches for response evaluation (classifier vs density estimation), the classifier-based metric performs better than density estimation based metrics when the contrastive loss is not applied. These results imply that contrastive learning is indeed helpful in improving the performance of our metric, by enhancing the quality of the feature space from the selection model. In terms of the two different distance functions, the Mahalanobis distance with LCL usually shows a higher correlation than the Euclidean distance.
To further investigate the impact of contrastive learning on a feature space from the selection model, we compute DENSITY score s(r) of answer responses and random negative responses on the validation set of the original DailyDialog corpus. The score distributions of the selection models trained with or without LCL objective are compared. The score distributions in Fig. 5 confirm that contrastive learning encourages the model to
make a more discriminative feature space.

heading: 5.4 Case Study
text: We present sample evaluation results of few selected metrics on the DailyDialogue-Zhao dataset in Table 3. While the system responses in the first and second examples look okay, taking a closer look at the dialog history, they are both incoherent. In the first example, the question asked gave two options, but the system responded with an option that was not in the original question. In the second example, the system asks if he (or she) should
take the bus, even though he is already waiting for the bus. The low scores from human annotators highlight this incoherency. Baseline metrics like USR-Retrieval and GPT2-Coherency often give high scores to such responses. In contrast, DENSITY outputs low scores, similar to human scores. In the third and fourth example, the system responses receive high scores from the human annotators. BLEU gives low scores to both responses, as there are little word overlap between the answers and system responses. In both examples, DENSITY gives relatively similar scores to human scores.

heading: 5.5 Experiments on Dialogue-level Evaluation
text: While our experiments generally focus on turnlevel response evaluation, we also conduct experiments on dialogue-level evaluation to probe the extensibility of DENSITY on such tasks. To this end, we use the FED dialogue-level evaluation dataset (Mehri and Eskenazi, 2020a), and the “Overall” score is used to calculate the Spearman correlation. We compare DENSITY against some turn-level metrics that were competitive with DENSITY in our turn-level evaluation (USR-Rtv. and GPT2-Coh.). To extend the turn-level evaluation metrics to a dialogue-level evaluation, we simply evaluate every turn in a dialogue, and average their scores. We also include the results of the FED (Mehri and Eskenazi, 2020a) and DynaEval (Zhang et al., 2021) models from the original papers.5 The results are shown in Table 4.
5We use the results of FED model with 345M parameters, which has a comparable size with other models.
The results show that DENSITY shows a higher correlation than other turn-level metrics. This result shows that although our current metric is not explicitly designed to handle dialogue-level evaluations, there is potential for utilizing a density estimationbased evaluation for dialogue-level evaluations. We leave this as our future work.

heading: 6 Conclusion
text: In this paper, we present DENSITY, a new learnable metric for open-domain dialogue systems. DENSITY evaluates a response by estimating its density on the distribution of human conversations. Empirical results on multiple datasets demonstrate that our metric has a higher correlation with human evaluations than other metrics. We hope that DENSITY, a reliable and robust metric for evaluating dialogue systems, contributes to improving evaluation of natural language generation tasks.
Limitations
Our proposed metric is mainly designed for a turnlevel evaluation of dialogue systems. We recognize that our metric may not generalize to other evaluation scenarios directly, such as dialogue-level evaluation or human-chatbot interactive setups. As shown in Section 5.5, the easiest way to extend our metric to a multi-turn dialogue evaluation is by evaluating every turn in a dialogue individually, and then aggregating their scores. However, as the dialogue-level evaluation is not considered during the development process of our metric, it is not clear whether such a simple extension would be applicable without a decrease in performance. Nevertheless, as turn-level evaluation is a fundamental component to build a holistic evaluation framework for a dialogue, we believe that it is an important task to investigate better evaluation metrics for individual responses.
Ethics Statement
All experiments are conducted on English datasets only, so the generalizability toward other languages is not verified. Besides, as the current automatic evaluation metrics, including ours, are imperfect, they may introduce unintended favor toward a certain type of responses. Future research should focus on detecting and mitigating such undesirable biases of learnable metrics.

heading: Acknowledgements
text: This work was supported by Institute for Information & communications Technology Planning & Evaluation(IITP) grant funded by the Korea government(MSIT) (No. 2020-0-00368, A Neural-Symbolic Model for Knowledge Acquisition and Inference Techniques and No.2019-000075, Artificial Intelligence Graduate School Program (KAIST)).

heading: A Baseline Details
text: We present further implementation details in baseline metrics. For BLEU, we use BLEU2 score with NLTK library6. For ROUGE, we use F-score of ROUGE-L. For METEOR, we use NLTK library7. For Embedding Average/Greedy/Extrema, we use an evaluation toolkit released by Sharma et al. (2017) with GloVe (Pennington et al., 2014) embedding. For BERTScore, we use the default roberta-large model in the official implementation8, and use F1 score between answer and generated responses for evaluation. For SimCSE, we use princeton-nlp/ sup-simcse-bert-base-uncased model in the Huggingface Hub9, and compute the cosine similarity between answer and generated responses for evaluation. For BLEURT, we use Elron/bleurt-tiny-512 model in the Huggingface Hub10. For USR-MLM, we train bertbase-uncased model with learning rate, train epochs, and batch size as 5e-5 and 1, and 16, respectively. For GPT2-Coherency, we train 12-layer gpt2 model with learning rate, maximum train epochs, and batch size as 5e-5, 10, and 16, respectively. For BERT-RUBER, we use bert-base-uncased as an contextualized word embedding, and train the model with learning rate, maximum train epoch, and batch size as 1e-4, 10, and 16, respectively. For USR-Retrieval, we train bert-base-uncased model with learning rate, train epochs, and batch size as 5e-5, 10, and 16, respectively. For USL-H, we use an official implementation11 to train and evaluate models. For FlowScore and DynaEval, we use official models1213 for evaluation. All baseline models trained in our experiments utilize the same training environments with our metric (e.g., optimizer, max sequence length) unless specified otherwise. When evaluating reference-free models, the dialogue corpus for the training of models and the original dia-
6https://www.nltk.org/ 7https://www.nltk.org/ 8https://github.com/Tiiiger/bert_score 9https://huggingface.co/princeton-nlp/
sup-simcse-bert-base-uncased 10https://huggingface.co/Elron/ bleurt-tiny-512 11https://github.com/vitouphy/usl_ dialogue_metric 12https://github.com/ictnlp/DialoFlow 13https://github.com/e0397123/DynaEval
logue corpus that derives an evaluation dataset are matched. For instance, we use DynaEval model trained on DailyDialog dataset for the evaluations on DailyDialog-Zhao and DailyDialog-GRADE datasets.

heading: B Further Implementation Details
text: We evaluate our selection model after every train epoch, and select the best model based on its recall@1 score on the validation set of the original dialogue corpus. All the learnable metrics are implemented with PyTorch (Paszke et al., 2019). We use Transformers framework (Wolf et al., 2020) from Huggingface14 to implement transformer (Vaswani et al., 2017)-based models. In overall experiments, two 3090 RTX GPU with 24GB of memory are used.

heading: C Additional Results
text: C.1 Impacts of Contrastive Learning on Response Selection Task
To more comprehensively understand the effect of contrastive learning on our feature extractor (g), we report the impacts of our contrastive learning objective on the performance of the response selection task. Specifically, we compare the selection model trained with both the response selection task and contrastive learning (LRS+LCL) against the model trained solely with response selection task (LRS). Recall@1 and mean reciprocal rank (MRR) metrics are used to measure the selection accuracy. We evaluate both models on the test splits of DailyDialog and ConvAI2 datasets. As shown in Table 5, we observe that contrastive learning improves the performance on the original task on which the selection models are trained.
14https://huggingface.co/
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: "Limitations" section on page 9

heading: 3 A2. Did you discuss any potential risks of your work?
text: "Ethics Statement" section on page 9
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts?

heading: 3, 4, 5, and Appendix

heading: 3 B1. Did you cite the creators of artifacts you used?
text: 3, 4, 5, and Appendix

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: 4, and Appendix
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 4
C 3 Did you run computational experiments? 3, 4, 5

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? 4, Appendix
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? 4, Appendix

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 5

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 4, Appendix
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Future research should focus on detecting and mitigating such undesirable biases of learnable metrics