Venue: NAACL 2024

# Leveraging LLMs for Dialogue Quality Measurement

Jinghan Jia\({}^{1}\), Abi Komma\({}^{2}\), Timothy Leffel\({}^{2}\), Xujun Peng\({}^{2}\)

**Ajay Nagesh\({}^{2}\), Tamer Soliman\({}^{2}\), Aram Galstyan\({}^{2}\), Anoop Kumar\({}^{2}\)**

###### Abstract

In task-oriented conversational AI evaluation, unsupervised methods poorly correlate with human judgments, and supervised approaches lack generalization. Recent advances in large language models (LLMs) show robust zero-shot and few-shot capabilities across NLP tasks. This paper explores using LLMs for automated dialogue quality evaluation, experimenting with various configurations on public and proprietary datasets. Manipulating factors such as model size, in-context examples, and selection techniques, we examine "chain-of-thought" (CoT) reasoning and label extraction procedures. Our results show that (1) larger models yield more accurate dialogue labels; (2) algorithmic selection of in-context examples outperforms random selection; (3) CoT reasoning where an LLM is asked to provide justifications before outputting final labels improves performance; and (4) fine-tuned LLMs outperform out-of-the-box ones. Our results indicate that LLMs that are suitably fine-tuned and have sufficient reasoning capabilities can be leveraged for automated dialogue evaluation.

+
Footnote †: _Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 6: Industry Track)_, pages 359–367

## 1 Introduction

Evaluating conversational system performance in NLP is challenging. Automating effective evaluation is crucial for enhancing dialogue systems. However, automatic metrics like BLEU Papineni et al. (2002) and ROUGE Lin (2004) fall short in accurately measuring perceived quality due to complex mappings Liu et al. (2016). Advanced methods USR Mehri and Eskenazi (2020), FED Mehri and Eskenazi (2020), DialogRPT Gao et al. (2020)) address this but often require extensive training data and human references, making them costly and limited in generalization to new datasets.

Recent advancements in Large Language Models (LLMs) Bubeck et al. (2023) have demonstrated robust zero- or few-shot capabilities and reasoning skills across a range of tasks Brown et al. (2020). Consequently, researchers have begun to explore the application of LLMs to classification problems such as dialogue evaluation Lin and Chen (2023); Huynh et al. (2023).

Recent studies Lin and Chen (2023); Huynh et al. (2023) demonstrate that LLMs excel on diverse datasets. However, uncertainties persist regarding the impact of factors like model size and in-context examples on open-sourced LLM performance. This paper aims to clarify these influences. Additionally, there is a lack of comprehensive literature on deploying LLMs for dialogue evaluation. To fill this gap, we propose two common evaluation strategies, providing a comparative analysis of their pros and cons.

In this paper, we systematically study different aspects of LLM-based dialogue evaluation by conducting extensive experiments on two benchmark datasets, one publicly available and the other proprietary, Amazon-internal datasets. We initially explore the connection between different attributes such as model size and in-context examples, and their impact on dialogue evaluation performance. Additionally, we present a dialogue evaluation that leverages "chain-of-thought" (CoT) reasoning abilities of LLMs Wei et al. (2022); Wang et al. (2023).

Our experiments demonstrate that larger model sizes and instruction tuning generally helps with zero shot dialogue evaluation. Furthermore, in few-shot scenario, we find that algorithmic selection of in-context examples yields better results than random selection. Next, we demonstrate that supervised fine-tuning can substantially improve the performance of LLMs on dialogue evaluation task. Finally, we explore and validate a CoT-based evaluation framework which is capable of returning not only dialogue labels but comprehensive explanations and justifications, thereby offering a more coherent and holistic evaluation. Remarkably, our results indicate that CoT-based evaluation is more

accurate when the LLM is prompted to first analyze the dialogue and then produce labels. Combined, our findings confirm the applicability and effectiveness of LLM-based automated dialogue evaluation.

## 2 Related work

**Dialogue evaluation.** Evaluating dialog systems poses challenges like accounting for multiple interlocutors, contextual dynamics, and the one-to-many relationship, as highlighted by Zhang et al. (2021) and Zhao et al. (2017). Metrics such as USR and FED address these challenges, showing strong correlation with human evaluation standards. Utilizing models like RoBERTa and DialoGPT, notable for their smaller yet effective versions fine-tuned for specific dialog tasks, these metrics excel in capturing nuanced dialog attributes. Other evaluation metrics such as GRADE and DEB Huang et al. (2020); Mehri and Eskenazi (2020) attempt to measure text coherence, response diversity, engagement, and common sense. With the exponential growth in parameter count of contemporary LLMs and their promising generalization capabilities in NLP tasks, it is anticipated that these model-centric evaluative metrics will undergo further enhancements.

**Large language models for evaluations.** Recent studies explore LLMs in dialogue evaluation. GPTScore Fu et al. (2023) uses models such as GPT-3 Brown et al. (2020), assigning higher probabilities to superior-quality content and employing diverse prompts for holistic evaluation. Similarly, Huynh et al. (2023) investigate using ChatGPT and InstructGPT Ouyang et al. (2022) for reference-independent text quality assessment, contrasting various LLM methodologies, including explicit scoring, leveraging model confidence for implicit score allocation, and direct pairwise text comparison.

The G-EVAL framework Liu et al. (2023) is a notable advancement, synergistically integrating LLMs with the chain-of-thought (CoT) paradigm and a form-filling strategy. Notably, using GPT-4 Bubeck et al. (2023) as its foundational model, G-EVAL shows strong correlation with human evaluations in summarization tasks.

**Parameter-efficient fine-tuning (PEFT).** As base language models grow in size Touvron et al. (2023); Zhang et al. (2022), researchers frequently turn to parameter-efficient fine-tuning techniques to tailor models for specific downstream tasks. These fine-tuning approaches typically fall into three main categories:

**(1) Prefix-Tuning:** This method inserts special tokens among input tokens with trainable embeddings for the task at hand Li and Liang (2021).

**(2) Adapter Tuning:** This approach inserts adapter layers between self-attention and MLP modules, providing nuanced control over the model's behavior without altering the core architecture Houlsby et al. (2019); Zhang et al. (2023).

**(3) Low-Rank Adaptation:** This technique uses trainable low-rank decomposition matrices in each network layer, simplifying the model for efficient fine-tuning Hu et al. (2021). It shows promise in adapting large generative models for specific applications Cuenca and Paul (2023); Zhang et al. (2023).

These strategies reflect ongoing efforts to make large-scale models more adaptable and efficient, leveraging their vast capacities while mitigating computational and practical challenges.

**In-context learning.** In-context learning is a prompting technique for LLMs in which example input-output pairs for some task are injected into the prompt before the target input is presented. The idea is that seeing correct examples of the task will help the model to provide a correct target output.

Selecting in-context examples is crucial for effectively prompting LLMs, enabling them to pivot to new tasks without extensive fine-tuning. Examples play a pivotal role in guiding LLMs' predictive capabilities, with research exploring methods such as semantic proximity evaluation Liu et al. (2021) and retrieval mechanisms such as BM25 Robertson et al. (2009), used independently or in an initial training phase for a selector retriever.

These selection approaches excel in few-shot NLP tasks. For instance, in Su et al. (2022), a bifurcated framework effectively annotates and selects in-context samples from unlabeled repositories, achieving impressive performance across various tasks. Similarly, Liu et al. (2021) suggested that choosing examples with congruent sentence embeddings optimizes GPT-3's efficacy. Despite positive outcomes, there is a need for deeper explorations to discover more general in-context example retrieval methodologies.

## 3 Methodology

**Dialogue evaluation with logits.** LLMs like GPT Radford et al. (2018) with a decoder-only architec 

ture are autoregressive. They generate sequences one element at a time, each conditioned on the preceding ones. The probability of a token sequence \(x=(x_{1},x_{2},...,x_{T})\) is modeled as the product of conditional probabilities for each token given its history.

\[p(x)=\prod_{t=1}^{T}p(x_{t}|x_{<t}) \tag{1}\]

where \(x_{<t}=(x_{1},...,x_{t-1})\) is the history before \(x_{t}\), and \(p(x_{t}|x_{<t})\) is typically modeled by a softmax over the vocabulary. LLMs can be prompted to provide a score (example prompt in section A.1), using the returned probabilities to generate ratings.

Building on methods from Huynh et al. (2023), we use these properties to select the top-\(K\) ratings \(r_{1},r_{2},...,r_{K}\), based on their corresponding log probabilities \(p_{1},...,p_{K}\). We then perform a weighted sum of these ratings, as illustrated in the left panel of Figure 1. The weights can be calculated using Equation (2):

\[w_{i}=\frac{p_{i}}{\sum_{j=1}^{K}p_{j}} \tag{2}\]

The final rating can be represented by Equation (3):

\[r=\sum_{i=1}^{K}r_{i}*w_{i} \tag{3}\]

**Dialogue evaluation with generation.** In addition to the mentioned method, Lin and Chen (2023) proposed a novel framework where LLMs are prompted to directly generate responses for dialogue evaluation. Ratings for the dialogue can then be extracted from the produced LLM responses. See prompts in section A.2.

## 4 Experiment setup

**Model.** In this study, we utilize models from the Llama family Touvron et al. (2023) and the Falcon series Almazrouei et al. (2023). We also incorporate the instruction-tuning variants of these models as proposed in the Alpaca study Taori et al. (2023). Temperature is fixed at \(0.7\) during generation.

**Dataset.** We experiment on two datasets: the publicly available USS Sun et al. (2021) dataset includes SGD, MultiWOZ, ReDial, and CCEP subsets, with a 1-5 quality score scale. We randomly allocate \(10\%\) as test data, using the rest for training (supervised fine-tuning or in-context learning). We also evaluate our methods on two versions of an Amazon-internal dialogue-quality dataset, which has a human-annotated quality rating on a scale of 1-5 (similar in format to the data described in Komma et al. (2023)). The rating distribution is shown in Figure 2, with the smaller training set's rating distribution resembling the test set more closely than the larger one.

We binarize the datasets, considering scores of three and below as "defect" (unsatisfactory) and scores of four or five as "non-defect" (satisfactory). This simplified scheme enables us to use standard binary classification metrics to compare the effectiveness of different methods.

**In-context example selection methods.** Here we use three methods to select in-context examples. The first is to simply source examples randomly from the training dataset. The second employs a probabilistic algorithm for information retrieval (IR) Robertson et al. (2009), selecting similar examples based on a defined similarity metric. The third relies on BERT Devlin et al. (2018), extracting representations from the input dialogue and identifying similar examples using cosine similarity computations. Together, these three methods offer diverse strategies for in-context example selection. We conducted three runs for the random selection method and report the mean value.

Figure 1: Schematic overview of LLM dialogue evaluation methods. Left: Pipeline using logits method for generating scores from LLMs. Right: Pipeline employing generation method to produce ratings from LLMs.

Figure 2: Score distribution in train and test splits from the Amazon-internal dataset.

 



**Supervised fine-tuning method.** In this study, we also explore supervised fine-tuning to adapt LLMs to the task of dialogue quality evaluation. To manage cost, we adopt the LoRA setting Hu et al. (2021), fine-tuning a relatively small number of parameters compared to full-rank fine tuning. LoRA fine-tuning efficiently enhances the model's effectiveness at the target task while addressing computational challenges.

## 5 Results

We now describe the results of our experiments designed to analyze various aspects of LLM-based dialogue evaluation.

### Larger models help zero-shot dialogue evaluation.

Table 1 shows the relationship between model size and zero-shot ability in dialogue quality evaluation, comparing Spearman and Pearson correlation values with human annotation for different model configurations across the Alpaca, Llama, and Falcon series.

Table 1 suggests a positive relationship between model size and zero-shot ability in dialogue quality evaluation. Notably, the Falcon series shows a significant improvement (Spearman: -0.02 to 0.41 for Falcon-7b versus Falcon-40b). Alpaca 13b sees marginal improvements compared to 7b, but the Llama series exhibits no significant improvement as size increases, with weak correlations across sizes (possibly due to the original Llama's poor instruction understanding).

This conclusion underscores the potential benefits of employing larger models in dialogue systems, particularly for applications that require zero-shot flexibility.

### Instruction-tuning helps zero-shot dialogue evaluation.

Table 1 also provides insight into the impact of instruction fine-tuning on models' ability to do zero-shot dialogue quality evaluation. The Alpaca series underwent instruction fine-tuning based on the foundational Llama models, an important modification to enhance task-specific performance. Our empirical results illustrate the superiority of the Alpaca models in the realm of zero-shot dialogue quality evaluation, as evidenced by consistently higher Spearman and Pearson correlation coefficients. This observation underscores the significance of instruction tuning as a critical technological approach for augmenting dialogue evaluation performance. For example, the improvement from Alpaca-13b compared to the Llama-13b models is 0.47 on the Spearman correlation shown in Table 11. We hypothesize that instruction tuning may serve to refine LLMs comprehension of instructions or prompts, thereby optimizing their ability to execute the specified tasks with greater accuracy. The results suggest that such tuning may be instrumental in facilitating a more nuanced understanding of dialogues, opening avenues for further research and development in this domain.

Footnote 11: Llama–7b having a Spearman correlation value of 0 is actually rounded value of a tiny correlation of ¡ 0.005.

### In-context examples enhance the performance of dialogue evaluation.

In this section, we evaluate the influence of innContext examples on the base model Falcon-7b-instruct, utilizing three distinct in-context example selection methods: Random, BM25, and BERT. For the BM25 and BERT-based ICL approaches, we selected either 1 or 4 of the most semantically similar samples from the training set. The selection was based on the semantic similarity between the examples, as determined by the BM25 and BERT models. In contrast, for the random-selection ICL approach, we randomly picked either 1 or 4 examples from the training set to use as the in-context examples. These experiments are conducted on the open-sourced dataset USS shown in Table 2.

First, we observe that in general, the few-shot performance is better than the zero-shot performance. A closer examination reveals that the best performance across various evaluation metrics is primarily concentrated within the few-shot settings in the first three datasets. If we take a closer look at the MWOZ dataset, all highlighted numbers are in the few-shots settings. This observation substantiates the notion that providing in-context examples can indeed enhance the performance in dialogue evaluation. However, it is worth noting that an excessive provision of examples does not necessar

\begin{table}
\begin{tabular}{l|c c c c c c c} \hline Methods & Alpha Test & Albacus/Llama & Llama & Llama/Llama & Flame-tuned & Riste-bit-matter \\ \hline Separate confidence & 0.67 & **0.89** & 0.00 & 0.01 & **0.01** & 4.02 & **0.04** \\ Power correlation & 0.27 & **0.89** & 0.00 & 0.00 & **0.00** & 4.02 & **0.05** \\ \end{tabular}
\end{table}
Table 1: Performance comparison across model with different sizes on Amazon-internal datasets. Spearman and Pearson correlation values presented, with the best results highlighted in bold.



ily lead to further improvement. We hypothesize that this limitation may stem from the capacity constraints of LLMs, which can struggle to process overly lengthy inputs, occasionally resulting in performance degradation. This pattern is particularly evident in the Redial dataset, where the performance of the 4-shot approach does not surpass the results obtained from the 1-shot experiment. In addition to these findings, Table 2 reveals that zero-shot outperforms the few-shot settings in the SGD dataset. This is likely attributable to the capacity constraints of LLMs; the dialogue lengths should not be excessively long. Notably, the dialogues in the SGD dataset have more turns (\(26.7\) turns per dialogue) compared to the other three datasets from Sun et al. (2021). Such findings further emphasize the nuanced relationship between the number of in-context examples and the resulting performance, highlighting the importance of careful selection in few-shot learning.

Second, when comparing the performance across different in-context selection methods, we find that algorithmic selection methods result in notable performance improvements over random selection for in-context examples. For instance, the BM25 and BERT methods consistently perform best across all datasets. Upon closer examination of the CCPE dataset in Table 2, we observe that \(85\%\) of the highest values across all metrics are derived from the algorithm's selected method. What is more, the optimal choice for selecting different in-context example methods varies based on the dataset. In the first two datasets, the BM25 method excels, while in the third, the BERT method stands out.

In conclusion, our results suggest that in-context examples can significantly enhance the quality of dialogue evaluation. Our findings also underscore the importance of employing algorithmic methods for selecting these examples, as the right selection strategy can lead to meaningful performance gains. By revealing these patterns, our study contributes to a deeper understanding of how few-shot learning can be best utilized in dialogue systems.

### Supervised fine-tuning improves dialogue evaluation quality

Here we examine the influence of supervised finetuning (SFT) on the performance of LLMs for the dialogue evaluation task. Specifically, we fine-tuned the models using both Likert-scale and binary label data. We leverage two internal datasets

\begin{table}
\begin{tabular}{l l c c c c c c c c c c c c c c} \hline \hline Dataset & Model & DevRet & Budget & DirectClass & \multicolumn{3}{c}{Non-Defect Class} & \multicolumn{3}{c}{Wogled Average} & \multicolumn{3}{c}{Macrew Average} & Spearman & Pearson \\  & Precision & Recall & F1-Score & Precision & Recall & F1-score & Precision & Recall & F1-score & Precision & Recall & F1-score & \\ \hline \multirow{9}{*}{CCPE} & Zenit-date & 62.5k & 0.71 & 0.39 & 0.5 & 0.42 & 0.74 & 0.54 & 0.6 & 0.52 & 0.51 & 0.57 & 0.56 & 0.52 & 0.21 & 0.27 \\  & Random-1 & 62.8k & 0.75 & 0.19 & 0.31 & 0.4 & **0.89** & 0.56 & 0.62 & 0.6 & 0.46 & 0.58 & 0.54 & 0.43 & **0.44** & 0.25 \\  & BME2-1 & 62.5k & **0.79** & 0.35 & 0.49 & 0.44 & 0.84 & **0.88** & 0.66 & 0.54 & 0.52 & 0.62 & 0.6 & 0.54 & 0.4 & **0.31** \\  & BME2+1 & 62.5k & 0.73 & 0.26 & 0.38 & 0.41 & 0.55 & 0.61 & 0.58 & 0.48 & 0.45 & 0.57 & 0.55 & 0.47 & 0.18 & 0.19 \\  & Random-1 & 62.5k & 0.71 & 0.55 & 0.62 & 0.46 & 0.63 & 0.53 & 0.61 & 0.58 & 0.59 & 0.58 & 0.59 & 0.58 & 0.16 & 0.17 \\  & BME2+4 & 62.5k & 0.73 & **0.77** & **0.78** & **0.59** & 0.53 & 0.61 & **0.67** & **0.68** & **0.68** & **0.66** & **0.58** & **0.48** & 0.20 & 0.2 \\  & BREK4 & 62.3k & 0.68 & 0.68 & 0.68 & 0.47 & 0.47 & 0.47 & 0.6 & 0.6 & 0.6 & 0.58 & 0.58 & 0.58 & 0.58 & -0.07 & -0.13 \\ \cline{2-13}  & Zenet-date & 55.5k & 0.5 & 0.29 & 0.37 & 0.43 & 0.64 & 0.51 & 0.47 & 0.45 & 0.43 & 0.46 & 0.47 & 0.44 & -0.05 & 0.0 \\  & Random-1 & 55.9k & 0.55 & 0.11 & 0.18 & 0.45 & **0.89** & **0.66** & 0.5 & 0.66 & 0.37 & 0.5 & 0.5 & 0.39 & 0.09 & **0.41** & **0.38** & 0.06 \\  & MMD2 & SBE7+1 & 55.9k & 0.53 & 0.15 & 0.23 & 0.45 & 0.84 & 0.58 & 0.49 & 0.43 & 0.54 & 0.52 & 0.44 & 0.13 & 0.0 \\  & Random-1 & 55.9k & **0.63** & 0.18 & 0.28 & 0.46 & 0.67 & **0.66** & 0.55 & 0.49 & 0.43 & 0.54 & 0.24 & 0.44 & 0.13 & 0.0 \\  & Random-1 & 55.9k & 0.53 & 0.35 & 0.42 & 0.44 & 0.42 & 0.51 & 0.49 & 0.47 & 0.46 & 0.48 & 0.48 & 0.47 & 0.09 & 0.08 \\  & BME2+4 & 55.9k & 0.61 & **0.64** & **0.82** & 0.53 & 0.53 & **0.57** & **0.57** & **0.57** & **0.57** & **0.57** & **0.57** & **0.57** & **0.57** & 0.08 & 0.04 \\  & BME4+ & 55.9k & 0.56 & 0.58 & 0.57 & 0.47 & 0.44 & 0.45 & 0.52 & 0.52 & 0.52 & 0.51 & 0.51 & 0.51 & 0.06 & 0.05 \\ \hline \multirow{9}{*}{Fashion} & Zenit-date & 44.9k & 0.49 & **0.55** & **0.52** & **0.61** & 0.55 & 0.86 & 0.56 & 0.55 & 0.55 & 0.55 & 0.55 & **0.55** & 0.09 & 0.11 \\  & Random-1 & 44.9k & 0.5 & 0.2 & 0.29 & 0.57 & 0.84 & 0.64 & 0.54 & 0.56 & 0.51 & 0.54 & 0.52 & 0.49 & 0.18 & 0.24 \\  & BME2-1 & 44.6k & 0.48 & 0.27 & 0.35 & 0.57 & 0.77 & 0.66 & 0.53 & 0.55 & 0.52 & 0.53 & 0.52 & 0.5 & 0.17 \\  & Random-1 & 44.9k & **0.58** & 0.32 & 0.41 & 0.61 & 0.62 & **0.77** & **0.66** & **0.67** & **0.57** & **0.59** & **0.57** & **0.58** & **0.24** & **0.29** \\  & Random-1 & 44.9k & 0.5 & 0.41 & 0.45 & 0.59 & 0.08 & 0.63 & 0.55 & 0.56 & 0.55 & 0.55 & 0.54 & 0.54 & 0.12 & 0.11 \\  & BME2+4 & 44.9k & 0.4 & 0.45 & 0.43 & 0.52 & 0.46 & 0.49 & 0.47 & 0.46 & 0.46 & 0.46 & 0.46 & 0.46 & 0.1 & 0.15 \\  & BME2+4 & 44.9k & 0.4 & **0.55** & 0.49 & 0.57 & 0.46 & 0.51 & 0.51 & 0.5 & 0.5 & 0.5 & 0.5 & 0.5 & 0.05 & 0.05 \\ \hline \multirow{9}{*}{SGD} & Zenit-date & 44.8k & **0.64** & **0.63** & **0.63** & **0.63** & **0.66** & **0.67** & **0.67** & **0.65** & **0.65** & **0.65** & **0.65** & **0.65** & **0.65** & **0.65** & **0.65** & **0.65** \\  & Random-1 & 44.9k & 0.40 & 0.08 & 0.14 & 0.51 & 0.88 & 0.65 & 0.46 & 0.50 & 0.40 & 0.46 & 0.48 & 0.39 & 0.20 & 0.09 \\ \cline{1-1}  & BAM2-1 & 48.9k & 0.39 & 0.15 

used for training dialogue quality estimation models. The first is a small version of the dataset, resulting in models denoted as "xxx-small," while the second is a larger version, leading to models labeled as "xxx-large."

We report on several classification metrics, including precision, recall, F1-score, and F1-micro. Our findings from Table 3 reveal a consistent improvement in the original model's performance after SFT, especially in the Spearman and Pearson correlations with human annotation. For instance, the model "Falcon-40b-instruct-sft-large" exhibits a \(48\%\) relative improvement compared to the original "Falcon-40b-instruct." This indicates that SFT enhances the alignment of the model's scoring with human evaluation.

Analysis of F1-micro shows that SFT generally improves performance in comparison to the original model. When comparing the Llama-7b-sft-small to the original Llama-7b, there is a \(75\%\) relative improvement. A closer examination also suggests that utilizing a larger dataset can boost overall performance. For example, the highest correlations in the first two columns are consistently associated with models trained on the larger dataset. Interestingly, models trained on smaller datasets occasionally exhibit superior F1-micro scores, as observed for the Llama series. For example, when comparing the Llama-13b-sft-small to the original Llama-13b-sft-large, there is a \(15\%\) relative improvement. We hypothesize that this may occur when the score distribution between the small dataset and the test dataset aligns more closely shown in Figure 2.

It is important to note that Llama-7b and Llama-13b predict defects for all test samples, which results in a recall of \(1.0\) and a precision of \(0.67\)--this obviously does not indicate optimal performance. To better understand the relationship between model predictions and the ground truth, we should consider the Spearman and Pearson correlations over likert scores, which provide more insight into the linear relationship between the predictions and the human labels.

In conclusion, our findings show that SFT can substantially enhance the performance of LLMS on dialogue evaluation. This study underscores the value of fine-tuning and dataset selection in achieving more accurate and human-aligned evaluations in the context of dialogue systems.

### Chain-of-thought for generation of scores and reasons

In this section, we shift our focus to the "generation and chain-of-thoughts" approach Wei et al. (2022), which not only generates scores but also provides natural language _reasons_ for selecting those scores. We explore two distinct paradigms to accomplish this task. The first paradigm, **Analysis-first**, involves prompting the model to generate an analysis first and then derive ratings based on that analysis. The second paradigm, **Rating-first**, prompts the model to generate a rating first and then elucidate the reasons for choosing that particular score (see Section A.2 for respective prompts).

Interestingly, our findings suggest that the first paradigm--Analysis-first--provides more aligned scores and reasons, as shown in Table 4. We observe consistent improvement compared to the Rating-first approach. For example, Analysis-first methods outperform Rating-first in \(85\%\) of all evaluation metrics. Upon conducting a failure analysis, we discovered that for Rating-first, the scores do not always align with the subsequent reasons. However, in the Analysis-first paradigm, there is consistent alignment between ratings and scores. We attribute the observed metric improvement to this consistency.

The implications of these findings may extend to various applications where the alignment between scores and reasoning is essential. Further exploration of these paradigms and their potential advantages and limitations may provide valuable insights into the optimal utilization of LLMs for complex tasks such as rating and explanation generation.

## 6 Conclusion

This paper explores the application of LLMs to evaluation of task-oriented dialogue systems. Key findings include: the impact of pretrained model size; the importance of instruction fine-tuning; the effectiveness of in-context examples; consistent performance improvement through supervised fine

\begin{table}
\begin{tabular}{l c r r r r r r r r r r r r} \hline Model & Defect Rate & \multicolumn{4}{c}{Defect Class} & \multicolumn{4}{c}{Non-Defect Class} & \multicolumn{4}{c}{Weighted Average} & \multicolumn{4}{c}{Mean Average} & \multirow{2}{*}{Spearman} \\  & Precision & Recall & F1-Score & Precision & Recall & F1-score & Precision & Recall & F1-score & Precision & Recall & F1-score \\ \hline Rating-first & 599 & 0.59 & **1.00** & 0.74 & **6.67** & 0.01 & 0.02 & 0.62 & 0.59 & 0.45 & 0.63 & 0.50 & 0.38 & 0.10 & 0.07 \\ Analysis-first & 599 & **0.67** & 0.86 & **0.75** & 0.66 & **0.39** & **0.49** & **0.67** & **0.65** & **0.67** & **0.63** & **0.62** & **0.23** & **0.28** \\ \hline \end{tabular}
\end{table}
Table 4: Comparison of CoT methods on the internal dataset over Falcon–7b-instruct. “Rating-first” refers to generating the score first, followed by the reasons, while “Analysis-first” involves generating an analysis first, then determining the scores.

 

tuning; and that the chain-of-though paradigm is most effective with the Analysis-first approach.

## 7 Limitations

Our experiments provide valuable insights, but there are limitations. We focus on open-sourced models, excluding closed ones like ChatGPT and Claude. Evaluation primarily centers on user satisfaction, lacking metrics for interestingness and coherence. Performance is influenced by prompt designs, and suboptimal prompts may lead to decline.

## 8 Ethics Statement

We acknowledge ethical concerns in using LLMs in our evaluation. Firstly, LLMs may carry biases that could impact dialogue evaluation negatively. Secondly, our focus on user satisfaction might overlook issues like toxic responses, leading to inadequate evaluations. Lastly, concerns arise about unintentional release of private information during reason and rating generation. To address these concerns, researchers should exercise caution in using LLMs for reasons and ratings, ensuring accuracy and fairness in interpretation.

## References

* A. Almazrouei, H. Alobeidli, A. Albashami, A. Cappelli, R. Cojocaru, M. Debbah, E. Goffinet, D. Heslow, J. Launay, Q. Malartic, B. Noune, B. Pannier, and G. Penedo (2023)Falcon-40B: an open large language model with state-of-the-art performance. External Links: Link Cited by: SS1.
* T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. (2020)Language models are few-shot learners. Advances in neural information processing systems33, pp. 1877-1901. External Links: ISSN 1563-690X, Document, Link Cited by: SS1.
* S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, C. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, et al. (2023)Sparks of artificial general intelligence: early experiments with gpt-4. arXiv preprint arXiv:2303.12712. Cited by: SS1.
* P. Cuenca and S. Paul (2023)Using lora for efficient stable diffusion fine-tuning. External Links: Link Cited by: SS1.
* J. Devlin, M. Chang, K. Lee, and K. Toutanova (2018)Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. Cited by: SS1.
* J. Fu, See-Kiong Ng, Z. Jiang, and P. Liu (2023)Gptscore: evaluate as you desire. arXiv preprint arXiv:2302.04166. Cited by: SS1.
* X. Gao, Y. Zhang, M. Galley, C. Brockett, and B. Dolan (2020)Dialogue response ranking training with large-scale human feedback data. arXiv preprint arXiv:2009.06978. Cited by: SS1.
* N. Houlsby, A. Giurgiu, S. Jaystrzebski, B. Morrone, Q. Le Laromussilhe, A. Gesmundo, M. Attariyan, and S. Gelly (2019)Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pp. 2790-2799. Cited by: SS1.
* E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2021)Lora: low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. Cited by: SS1.
* L. Huang, Z. Ye, J. Qin, L. Lin, and X. Liang (2020)GradE: automatic graph-enhanced coherence metric for evaluating open-domain dialogue systems. arXiv preprint arXiv:2010.03994. Cited by: SS1.
* J. Huynh, C. Jiao, P. Gupta, S. Mehri, P. Bajaj, V. Chaudhary, and M. Eskenazi (2023)Understanding the effectiveness of very large language models on dialog evaluation.. arXiv preprint arXiv:2301.12004. Cited by: SS1.
* A. Komma, N. P. Chandrasekarasatury, T. Leffel, A. Goyal, A. Metallinou, S. Matsoukas, and A. Galstyan (2023)Toward more accurate and generalizable evaluation metrics for task-oriented dialogs. In The 61st Annual Meeting Of The Association For Computational Linguistics (Industry Track), Cited by: SS1.
* X. L. Li and P. Liang (2021)Prefix-tuning: optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190. Cited by: SS1.
* C. Lin (2004)Rouge: a package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-81. Cited by: SS1.
* Y. Lin and Y. Chen (2023)LLM-eval: unified multi-dimensional automatic evaluation for open-domain conversations with large language models. arXiv preprint arXiv:2305.13711. Cited by: SS1.
* C. Liu, R. Lowe, I. V. Serban, M. Noseworthy, L. Charlin, and J. Pineau (2016)How not to evaluate your dialogue system: an empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:1603.08023. Cited by: SS1.
* J. Liu, D. Shen, Y. Zhang, B. Dolan, L. Carin, and W. Chen (2021)What makes good in-context examples for gpt-3?. arXiv preprint arXiv:2101.06804.

 

* Liu et al. (2023) Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. Gpteval: Nlg evaluation using gpt-4 with better human alignment. _arXiv preprint arXiv:2303.16634_.
* Mehri and Eskenazi (2020a) Shikib Mehri and Maxine Eskenazi. 2020a. Unsupervised evaluation of interactive dialog with dialogpt. _arXiv preprint arXiv:2006.12719_.
* Mehri and Eskenazi (2020b) Shikib Mehri and Maxine Eskenazi. 2020b. USR: An unsupervised and reference free evaluation metric for dialog generation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 681-707, Online. Association for Computational Linguistics.
* Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. _Advances in Neural Information Processing Systems_, 35:27730-27744.
* Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In _Proceedings of the 40th annual meeting of the Association for Computational Linguistics_, pages 311-318.
* Radford et al. (2018) Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training.
* Robertson et al. (2009) Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: Bm25 and beyond. _Foundations and Trends& in Information Retrieval_, 3(4):333-389.
* Su et al. (2022) Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, et al. 2022. Selective annotation makes language models better few-shot learners. _arXiv preprint arXiv:2209.01975_.
* Sun et al. (2021) Weiwei Sun, Shuo Zhang, Krisztian Balog, Zhaochun Ren, Pengjie Ren, Zhumin Chen, and Maarten de Rijke. 2021. Simulating user satisfaction for the evaluation of task-oriented dialogue systems. In _Proceedings of the 44rd International ACM SIGIR Conference on Research and Development in Information Retrieval_, SIGIR '21. ACM.
* Taori et al. (2023) Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_alpraca](https://github.com/tatsu-lab/stanford_alpraca).
* Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lacchaux, Timothee Lacroix, Baptiste Roziere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llana: Open and efficient foundation language models. _arXiv preprint arXiv:2302.13971_.
* Wang et al. (2023) Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2023. Towards understanding chain-of-thought prompting: An empirical study of what matters. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 2717-2739.
* Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In _Advances in Neural Information Processing Systems_, volume 35, pages 24824-24837. Curran Associates, Inc.
* Zhang et al. (2023) Renrui Zhang, Jiaming Han, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, Peng Gao, and Yu Qiao. 2023. LLama-adapter: Efficient fine-tuning of language models with zero-init attention. _arXiv preprint arXiv:2303.16199_.
* Zhang et al. (2022) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shubhui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. _arXiv preprint arXiv:2205.01068_.
* Zhang et al. (2021) Zhenyu Zhang, Tao Guo, and Meng Chen. 2021. Dialoguebert: A self-supervised learning based dialogue pre-training encoder. In _Proceedings of the 30th ACM International Conference on Information & Knowledge Management_, pages 3647-3651.
* Zhao et al. (2017) Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. _arXiv preprint arXiv:1703.10960_.

 

Prompts

In this section, we elaborate on the prompts and instructions we used in the logits-based and generation-based evaluation methods.

### Prompts for logits method

The prompts we used in the logits method are formulated in the following manner:

**Instruction: Could you please evaluate the subsequent dialogue by assigning a score from the given set [1,2,3,4,5]? A score of 1 implies dissatisfaction, while a 5 signifies high satisfaction.**

### Prompts for generation method

**Rating-first In this section, we outline the prompts utilized in the Rating-First generation method. This process begins with an instruction that directs the Large Language Models (LLMs) to provide both reasons and ratings for a given dialogue. To enhance the LLMs' comprehension, we also supply evaluation criteria standards. Finally, we detail the evaluation steps necessary to complete the entire procedure for dialogue evaluation. The prompt is shown as follows:**

**Instruction: Could you please evaluate the subsequent dialogue, providing a score from the set [1,2,3,4,5], and give an explanation for choosing that score?**

**To help you better evaluate, here is the evaluation Criteria:**

A score of 1 means very dissatisfied, where the user repeatedly has to stop or cancel bad responses and repeat their request again;

A score of 2 means dissatisfied, where None of the user goals are achieved,the user expresses negative feedback,steps towards a user goal succeeds but the goal fails;

A score of 3 means normal, where At least one of a user goals succeed, and no negative feedback

A score of 4 means satisfied, where the majority of turns succeeded or moved the user closer to their goal

A score of 5 means very satisfied, all turns either succeeded or moved the user closer to their goal(s), and the user

expressed no dissatisfaction, and goal was achieved without unnecessary steps

**Steps to conduct the evaluation are:**

1.Read the dialog, and the response carefully

2.Rate the response on a scale of 1-5 for satisfaction level from user, according to the criteria above and

3.Provide a brief explanation for your rating, referring to specific aspects of the response and the dialog.

**Analysis-first In this section, we describe the prompts used in the Analysis-First generation method. Similar to the Rating-First approach, we begin by providing instructions for the LLMs to carry out dialogue evaluation. We then present specific aspects for the LLMs to analyze. To facilitate better understanding, we also supply the evaluation criteria. Finally, we detail the specific steps for the LLMs to follow.**

**Instruction: Could you please evaluate the subsequent dialogue overall quality, first analysis the dialogue from the following aspects: 1.User goal. 2.User feedback. 3.System response. 4.System feedback. Based the above analysis provide a user satisfactory score from the set [1,2,3,4,5] To help you better evaluate, here is the evaluation Criteria:**

A score of 1 means very dissatisfied, where the user repeatedly has to stop or cancel bad responses and repeat their request again;

A score of 2 means dissatisfied, where None of the user goals are achieved,the user expresses negative feedback,steps towards a user goal succeeds but the goal fails;

A score of 3 means normal, where At least one of a user goals succeed, and no negative feedback

A score of 4 means satisfied, where the majority of turns succeeded or moved the user closer to their goal goal(s), and the user expressed no dissatisfaction, and goal was achieved without unnecessary steps Steps to conduct the evaluation are: 1.Read the dialog, and the response carefully

2.Give some brief analysis from the aspects mentioned before

3.Rate the response on a scale of 1-5 for satisfaction level from user, according to the criteria above and the analysis.