Approximating Online Human Evaluation of Social Chatbots with Prompting (2023)

Abstract: With conversational models becoming increasingly available to the general public, developing scalable and robust evaluation metrics is crucial to minimize potential social and psychological risks for the users. Existing evaluation metrics aim to automate offline user evaluation and approximate human judgment of pre-curated dialogs. However, they are limited in their ability to capture subjective perceptions of users who actually interact with the chatbots and might not generalize to real-world settings. To address this limitation, we propose an approach to approximate online human evaluation, leveraging large language models (LLMs) from the GPT-family. We introduce a new Dialog system Evaluation framework based on Prompting (DEP), which enables a fully automatic evaluation pipeline that replicates live user studies and achieves an impressive correlation with human judgment (up to Pearson r=0.95 on a system level). The DEP approach involves collecting synthetic chat logs of evaluated bots with an LLM in the other-play setting, where the LLM is carefully conditioned to follow a specific scenario. We further explore different prompting approaches to produce evaluation scores with the same LLM. The best-performing prompts, which contain few-shot demonstrations and instructions, show outstanding performance on the tested dataset and demonstrate the ability to generalize to other dialog corpora.

## introduction the recent arrival of conversational ai, marked by the public release of chatgpt from openai, foot_0 initiated unprecedented user engagement with conversational chatbots in a real-world setting. with the impressive naturalness of machines' responses, users are going beyond traditional transactional exchanges and start exploring more social interaction scenarios with increasing curiosity @xcite . in such situations, users might be subject to social and psychological harms if dialog systems fail to follow commonsense social rules @xcite . several instances of alarming social behavior of this technology have already been discussed in the media @xcite @xcite . in this context, developing meaningful and robust evaluation metrics for these systems has become particularly urgent to ensure that the models are safe and acting in the best interest of the users before their release. initially, human evaluation was considered a de facto standard for evaluating dialog systems @xcite . as running human evaluation is timeand resource-consuming, a number of automatic evaluation metrics for dialog systems have been proposed @xcite . the majority of these approaches aim to automate the offline user evaluation. in this setting, dialog evaluation is performed by a human judge who is distinct from the one conversing with the bot (figure 1 , offline). the metrics proposed for this case approximate the evaluation scores provided by this third-party human judge for the pre-produced dialogs (e.g. @xcite . despite its popularity, offline user evaluation is limited in its ability to capture subjective perceptions of users who actually interacted with the bots @xcite @xcite . this limitation of relying on second-hand evaluation can be illustrated by an analogy from the realm of restaurant critique when one tries to evaluate a restaurant solely by reading consumer reviews but having never actually eaten there. conducting online user evaluation, where the same individual interacts with the bot and assesses its performance, is more likely to produce or by the same conversational partner (online). in the third step, the system ranking is obtained by aggregating the dialog scores of each chatbot. grey bot icons indicate steps that are intended to be approximated by means of automatic evaluation. pink boxes mark the steps in the process where the correlation (r.) with the ground truth human judgment is computed to validate the automatic evaluation metric during its development process. to address this gap, we propose a novel automatic dialog system evaluation framework based on prompting, dep. our framework automates the whole pipeline of dialog system evaluation in an interactive setting, replicating live user studies. as the first step towards this goal, we leverage a large language model (llm) from the gpt-family models to collect synthetic chat logs of evaluated bots with the llm. second, we prompt the same llm to produce the resulting evaluation scores for generated chat logs and, finally, rank the chatbots based on their overall performance (figure 1 , online). while using bot-play is not a new idea per se, we emphasize the importance of carefully choosing a dialog partner for the evaluated chatbots specifically for social conversational contexts where the roles of two interlocutors can differ significantly. for example, it was shown that the emotion/intent distributions in conversations between an emotional speaker and an empathetic listener are very different for the two dialog partners @xcite . to account for it, in the first step of our framework, we propose prompting llms to play a particular social role over the course of the interaction with the chatbots to be evaluated. for the second step, we draw inspiration from the fact that llms demonstrate solid performance improvement when their generation process is augmented with instructions @xcite . we demonstrate that prompting the model with appropriate instructions that explain how fine-grained evaluation dimensions relate to the overall dialog score leads to substantial performance improvement, reaching up to @xmath0.95 pearson correlation with the human judgment on a system level. overall, our contributions include the following. 1) we describe an end-to-end prompting-based evaluation framework for dialog systems, specifically targeting social interaction scenarios (section 3). 2) our experiments showcase the effectiveness of prompting for assigning a desired social role to llms and, thus, collecting machine-generated dialogs that better approximate real interpersonal communication (section 4.1.2). 3) we consider different prompt designs and conclude that including demonstrations together with instructions results in the best performance (sections 4.1.3, 4.2.2). ## proposed method: dep we introduce our dep framework, which consists of two consecutive steps. first, it requires collecting interactive chat logs between the llm and evaluated chatbots, which we denote as llm-tobot play. second, the llm is prompted to generate scores for these chat logs. the generated scores are further aggregated to produce a final ranking of the systems. we describe each of the steps below. ## results for all reported experiments, we used the most capable version of the instructgpt model (textdavinci-003) available at the moment of initiation of our experiments in early q1 2023. we used this model as it was easily accessible through ope-nai api 2 and was expected to have superior performance for social scenarios as it was trained based on human feedback, which captures subjective human judgment of interactive outputs @xcite . following previous works that considered system-level evaluation @xcite , we report pearson correlation for our experiments, unless specified otherwise. we also opted for this type of correlation coefficient as it performed better for capturing whether the automated metric succeeds in preserving the gap in scores for the best-and least-performing chatbots, the information which gets lost with rank correlation. we start by demonstrating the application of our evaluation framework to empathetic dialog systems as in these interactive scenarios two conversational partners have clearly distinct social roles: an emotional speaker and an empathetic listener. further, we consider the generalizing ability of the framework to other social domains. ## discussion dialog system evaluation with prompting showed its usefulness both for generating new interactive exchanges with the evaluated systems and for judging their performance, therefore, allowing for a reasonable approximation of the online user evaluation pipeline. we deem this approach particularly promising for the evaluation of social aspects of conversations. llms used for prompting suffer from occasional hallucinations, i.e., a tendency to make up factual information @xcite . it might be difficult to keep track of all specific factual items of information that come up in the interactively created dialog between two conversational models and search for ground truth references for each of them to construct objective metrics such as the model's accuracy or truthfulness @xcite . whereas, prompting the llm to establish a specific behavior and providing instructions about commonsense social norms appears more feasible once these instructions are established. drawing from the visualization of discourse patterns in our newly collected dataset of dialogs between the llm and empathetic chatbots, we observed that the prompted llm largely mirrors the conversational patterns of humans. however, there are also some differences. for example, in figure 4 there is an apparent sub-flow with a grateful emotion, increasingly displayed by the llm. we believe the llm might have developed an agreeable "personality" due to its training procedure based on reinforcement learning from human feedback, which optimized llm's responses to satisfy human labelers. differences in speakers' behavior led to the difference in the responses of the evaluated chatbots. while their most frequently produced intents are similar, their frequency distributions are statistically identical only for the second turn (first response of the evaluated chatbots) according to the permutation and chi-square tests. future research can consider alternative prompting techniques to make the emotion/intent distribution of llms' and chatbots' responses even more balanced and representative. it might be beneficial to conduct additional experiments to compare original and generated dialogs, which can, for example, include testing the human ability to distinguish the dialogs created with the help of an llm and dialogs with human speakers. we conducted our experiments with only one llm and explored the few-shot prompting scenarios with a fixed number of demonstrations. future studies could explore the applicability of other llms for the dep framework, as it has been already initiated by @xcite ). an area of particular interest would be to study the efficacy of the framework working with open-source llms, such as llama @xcite . additional investigation is necessary to analyze the capability of the framework to handle longer dialogs, which might be challenging to fit into a context window of an llm. we would also like to explore how dep generalizes to evaluating other phenomena in social conversations, apart from generic open-domain interactions and empathetic dialogs. for example, further studies might focus on applying the framework to evaluate toxicity or humor in dialogs. however, this research direction requires the curation of appropriate calibration datasets. last but not least, evaluation artifacts produced by dep may be used to assist designers of chatbots as they allow for both analyzing the synthesized logs and comparing quality ratings. these insights may be integrated into assistive chatbot design tools, such as ichatprofile @xcite , to offer a faster prototyping cycle due to the automatic generation of chat logs and richer insight about chatbot profiles due to additional rating information provided by the last step of dep. ## conclusion in this paper, we proposed dep -a framework for evaluating social chatbots using prompting. our framework addresses the limitations of evaluation approaches using benchmark datasets in an offline setting. we describe how llms can be leveraged to synthesize realistic conversational logs with the evaluated chatbots in an online interactive manner. we further outline how the knowledge about the desired fine-grained qualities of a conversational partner can be translated into the prompting instructions to generate reliable overall scores for the collected dialogs. the proposed framework streamlines the evaluation process, making it highly efficient in terms of both time and cost, by removing the need for human involvement at every step. our experiments demonstrated that the promptingbased evaluation results achieve a high correlation with human judgment, reaching an impressive pearson @xmath1.95 system-level correlation for the ieval dataset, which features dialogs with empathetic chatbots. we explain our vision of why this framework is well-suited for the evaluation of social phenomena in conversations and lay out future research directions. we also publicly release all freshly curated chat logs between the llm and evaluated chatbots, as well as all additional annotations for the ieval, fed, and dstc9 datasets created for this study. 5