On the Benchmarking of LLM s for Open-Domain Dialogue Evaluation (2024)

Abstract: Large Language Models (LLMs) have showcased remarkable capabilities in various Natural Language Processing tasks. For automatic open-domain dialogue evaluation in particular, LLMs have been seamlessly integrated into evaluation frameworks, and together with human evaluation, compose the backbone of most evaluations. However, existing evaluation benchmarks often rely on outdated datasets and evaluate aspects like Fluency and Relevance, which fail to adequately capture the capabilities and limitations of state-of-the-art chatbot models. This paper critically examines current evaluation benchmarks, highlighting that the use of older response generators and quality aspects fail to accurately reflect modern chatbot capabilities. A small annotation experiment on a recent LLM-generated dataset (SODA) reveals that LLM evaluators such as GPT-4 struggle to detect actual deficiencies in dialogues generated by current LLM chatbots.

## introduction the emergence of large language models has brought about a significant transformation in the field of natural language processing (nlp). through advancements in model and data scale and clever prompting strategies that leverage in context learning and chain-of-thought reasoning @xcite , these models have exhibited remarkable capabilities, marking a notable shift in the landscape of nlp research and development. for the task of dialogue evaluation in particular, several works have proposed the use of llms as a standalone or as part of a dialogue evaluation framework for direct assessment of a response or full dialogue @xcite @xcite . by simply providing inside the prompt the quality aspects under evaluation, together with a few examples, these models achieve state of the art correlations with human judgements on several benchmarks. despite the promising results heralded by this recent approach, we argue that the methods used to benchmark dialogue evaluation are not adequate to accurately assess the evaluation capabilities of current open-domain dialogue evaluation metrics. in this paper, we investigate existing commonly used human-annotated datasets and identify their shortcomings when used as benchmarks for assessing llm-based evaluators. in particular, these datasets often rely on the use of weak chatbots to evaluate the proposed framework/metric (as illustrated in to prioritise these outdated criteria, leading to a disconnect between evaluation practices and the capabilities of modern chatbots. in support of our argument, we present a small qualitative analysis of evaluations provided by these models on dialogues that better approximate current chatbot performance. on the one hand, our analysis shows that dialogues that lack fluency are both easy to detect, and hard to find. on the other hand, llms struggle to correctly identify coherence and commonsense issues, which are aspects where the current generation of chatbots still underperform and where better detection and evaluation would be desirable. ## benchmark datasets this section presents a brief survey of datasets that have been used as a benchmark for llm-based open-domain dialogue evaluation metrics. these datasets are summarised in the fed dataset @xcite ) consists of turn level and dialogue level annota-tions of conversations conducted between a human (40 dialogues) and two chatbot engines (meena with 40 dialogues @xcite and 44 from mitsuku foot_0 ) targeting eighteen quality aspects. each conversation received one annotation at the dialog level and three annotations at the turn level, randomly selected from the conversation. in total, the fed dataset comprises 3,348 turn-level and 1,364 dialog-level data points, amounting to 4,712 annotations. for usr @xcite , annotations were collected for models trained on the topicalchat @xcite and per-sonachat @xcite dialogue datasets. generated responses were obtained from models including transformer @xcite , rnn seq2seq @xcite , lstm @xcite , and kv-memnn @xcite . for each dialog context, an additional human response was also collected. human annotation was then carried out on sixty dialog contexts, with six responses per context for topical-chat (four transformer outputs with different decoding strategies, one newly-annotated human output, and the original ground-truth response) and five for personachat (seq2seq, lstm, kv-memnn, one newly-annotated human output, and the original ground-truth response). dstc10 @xcite . the principal goal of the "automatic evaluation and moderation of open-domain dialogue systems" track was to offer a competitive venue for participants in this challenge to design robust automatic dialogue evaluation metrics that correlate well with human judgements across multiple dialogue domains as well as across different quality aspects. for the development set, 14 publicly available datasets were collected: (1-3) grade datasets @xcite , (4-5) dailydialog/persona-zhao @xcite , (6) dailydialog-gupta @xcite , (7-8) usr, (9) humod (merdivan et al., 2020), (10) twitter-dstc6 @xcite , (11) reddit-dstc7 @xcite , (12) persona-see @xcite and (13-14) fed. in total, over 35k turn-level human annotations were compiled. for testing, 3 sources of data were used: (1) chanel-jsalt2020, (2) chateval @xcite and (3) an additional annotation conducted on topicalchat @xcite and personachat @xcite . eight systems, a human baseline, and a random utterance were used as response generators. specifically, the eight systems are lstm seq2seq, attention-based lstm seq2seq @xcite , @xcite , vhred, blenderbot (400m-distill) @xcite , dialogpt-medium @xcite , t5-base @xcite , and gpt-3 @xcite . dstc11 @xcite . similar to dstc10, the "robust and multilingual automatic evaluation metrics for open-domain dialogue systems" track is split into development and test sets. for the development set, the organisers provide data from two clusters of datasets from dstc10 and 4,470 dialogues (approximately 130k turns) open-domain human-human dialogues which are originally in chinese. since the goal of the shared task was to evaluate mulitlinguality and robustness of metrics, development data is translated into english, chinese, spanish, and backtranslated. for testing, the organisers combine a portion of the dstc10 test set, and include new human-chatbot dialogues generated by sota chatbots. these are: chatgpt (a platform powered by gpt-3.5-turbo), gpt-3.5 @xcite and blenderbot3 @xcite . similar to the development set, the test set was also translated. in total, 4,839 turn level and 277 dialogue level annotations were conducted. ## llms as evaluators most automatic evaluation in the literature up until recently was conducted with word-overlap metrics or encoder-based metrics trained using selfsupervised training objectives @xcite . @xcite proposed an alternative approach called fed (fine-grained evaluation of dialog), which measures dialogue quality by computing the likelihood that dialogpt @xcite will respond to it with a particular set of follow-up utterances that are constructed. despite the unsupervised nature, it was only with the introduction of llms that these approaches fully replaced encoder-based metrics. the first documented systematic evaluation of llms was conducted by @xcite , where they evaluate training and few-shot strategies for this task. the authors evaluate several llms including bloom @xcite , opt @xcite , @xcite , instructdial @xcite and tnlgv2 @xcite on the dstc10 and fed benchmarks. the authors report good correlation results with human judgements and confirm the appropriateness of few-shot learning for dialogue evaluation. gptscore @xcite is based on the assumption that a generative pre-training model will assign a higher probability to high-quality generated text than low quality one following a given instruction and context. several llms are tested, including gpt-3 and flan-t5 on the fed-turn dataset. g-eval @xcite studies gpt-3.5-turbo and gpt-4 for the evaluation of generation models. in detail, the framework comprises (1) a prompt defining the evaluation task and criteria, (2) a chain-of-thoughts step containing intermediate instructions generated by the llm outlining evaluation steps, and (3) a scoring function based on return token probabilities estimated by generating multiple times. for the task of dialogue evaluation, g-eval is benchmarked on the usr-topicalchat dataset covering naturalness, coherence, engagingness and groundedness. dialevalml @xcite is a hybrid framework combining encoder-based models (in this case xlm-roberta-large @xcite ) trained with self-supervised objectives and direct prompting and score extraction from gpt-3.5-turbo. the authors combine the predictions using a correlation rescaling method obtained from the development set, achieving first place in all tracks of dstc11 @xcite . llm-eval @xcite ) is a singleprompt-based evaluation method that leverages a unified evaluation schema to cover multiple dimensions of conversation quality in a forward pass. the authors evaluate claude-v1.3 @xcite , chatgpt and gpt-3.5 on the dstc10 hidden set. xdial-eval @xcite focuses on probing the evaluation capabilities of several open access llms against gpt-3.5-turbo. the authors focus on context relevance and coherence by combining a selection of subsets from dstc11 development set. they additionally translate the original english data to 9 additional languages. unlike other approaches, the llms were evaluated in (1) zero and few shot learning scenario; (2) instruction finetuning; and (3) ensemble with a strong encoder-based framework. zhang et al. ( 2024 ) conduct a comprehensive study of 30 recently emerged llms for automatic dialogue evaluation using a smaller subset than the one from xdial-eval. in particular, the authors assess relevance, understandability, specificity, interestingness, and overall quality at the turn level, while at the dialogue level, they evaluate coherence, engagingness, informativeness, diversity, and overall quality. ## limitations in current benchmarking given the datasets identified in section 2 used to assess llm-based evaluators (section 3), we identify several limitations in the benchmarking of automatic open-domain dialogue evaluation, which we enumerate below. use of outdated generative models with the exception of dstc11-test (which was only used as a benchmark by dialevalml), most benchmarks contain responses from older generative models such as lstms or hred. as a result, a substantial amount of low quality responses are easily identifiable (lacking basic quality aspects such as fluency, contextual relevance or specificity). concurrently, responses that are relevant but contain contradictions, coherence issues or are factually incorrect are overvalued by evaluators due to biased guidelines. this tendency to rate flawed responses can skew the perception of evaluator performance, leading to misleading conclusions about their effectiveness in practice. irrelevance of quality aspects in current chatbots dialogue evaluation guidelines are focused on detecting issues that were prevalent in older generation models. for instance, all benchmarks have a quality aspect that targets fluency and relevance. given current llm-based chatbots, these quality aspects are no longer informative to differentiate output quality between different contemporary dialogue systems: most if not all models now output fluent and relevant responses (e.g., focus on english an overarching trend on the benchmarks being used is that they exclusively focus on the english language. although dstc11 does provide annotations in chinese and spanish, they are only partially available for the test set. moreover, in the development set, only translated versions of the original english dialogues are included, thereby introducing english bias into the assessment of quality. this bias further extends to the test set, where, even if evaluated by native annotators, the aspects being measured fall short of capturing the linguistic and cultural nuances present in dialogues. these nuances can include the use of formal versus informal language, expressions of politeness, cultural references, and idiomatic expressions foot_1 that may not directly translate into english. ## qualitative analysis informed by the issues highlighted in section 4, we conduct a small scale annotation experiment. the goal of this annotation is twofold. firstly, we aim to understand whether annotations such as fluency are still relevant. secondly, the annotation of more complex aspects such as coherence or commonsense in this dataset allows us to understand the performance of llms when evaluating responses generated by sota chatbots on quality aspects that require a stronger understanding of conversational dynamics. we use soda @xcite as our dialogue dataset since it leverages a llm (in this case gpt-3.5) for the generation of dialogues. as such, soda will exhibit most of the typical issues associated with llms, thereby making its use as a contemporary benchmark more relevant than benchmarks relying on weaker response generators (as identified in section 4). human evaluation conducted on soda shows that its dialogues are more consistent, specific, and natural than daily-dialog @xcite , a frequently used dialog dataset used for the development of evaluation metrics @xcite . ## conclusion this paper conducts an inventory of open domain dialogue evaluation benchmarks being currently used by llm evaluation frameworks. we show that these benchmarks have several limitations that hinder the progress in the field. in particular, we argues they lack (1) responses generated by strong llm chatbots; (2) aspects that identify their weaknesses; (3) representation of other languages and cultures. in order to illustrate these limitations, we also conducted a small scale experiment on soda and show that even gpt-4 shows limitations in the detection of low quality responses. however, these findings underscore one critical limitation in how direct assessment benchmarks are currently being developed: they are mostly concerned with evaluating contemporary chatbot capabilities. as it stands, the current evaluation research environment is one where the driver of progress is the advancement in generation, and not the converse. ultimately, evaluation benchmarks should possess the flexibility to remain relevant as newer chatbots emerge, thereby pushing the envelope of dialogue generation. embracing this goal would not only foster greater comparability and reproducibility in research, but also facilitate continuous improvement in the field, leading to the development of better chatbots.