Leveraging Large Language Models for Automated Dialogue Analysis (2023)

Abstract: Developing high-performing dialogue systems benefits from the automatic identification of undesirable behaviors in system responses. However, detecting such behaviors remains challenging, as it draws on a breadth of general knowledge and understanding of conversational practices. Although recent research has focused on building specialized classifiers for detecting specific dialogue behaviors, the behavior coverage is still incomplete and there is a lack of testing on real-world human-bot interactions. This paper investigates the ability of a state-of-the-art large language model (LLM), ChatGPT-3.5, to perform dialogue behavior detection for nine categories in real human-bot dialogues. We aim to assess whether ChatGPT can match specialized models and approximate human performance, thereby reducing the cost of behavior detection tasks. Our findings reveal that neither specialized models nor ChatGPT have yet achieved satisfactory results for this task, falling short of human performance. Nevertheless, ChatGPT shows promising potential and often outperforms specialized detection models. We conclude with an in-depth examination of the prevalent shortcomings of ChatGPT, offering guidance for future research to enhance LLM capabilities.

## introduction one crucial aspect of developing high-performing dialogue systems is the automated identification of errors in system responses. these errors can result from various behaviors, including incorrect information retrieval or illogical semantics (figure 1 ). identifying such errors enhances dialogue system development and complements dialogue-level evaluation methods by providing finer-grained metrics for comparison @xcite . to capitalize on these benefits, recent research has focused on training classifiers for specific dialogue behaviors. while certain behaviors have received considerable attention, this is not the case do you speak spanish? yes, the population of miami-dade county, the most populous county in florida, is estimated to be 5.3 million. yes, and miami is also the second-largest us city (after el paso, texas) with a spanish-speaking majority. yes, i do. i'm a native speaker of english, but my mother tongue is spanish. although there is much effort towards opensourcing competitive llms, openai's chatgpt remains the most successful llm to date @xcite . thus, we focus our experiments on chatgpt to assess the current best-case performance on automated dialogue behavior detection using llms. with its wide accessibility and low costs, chatgpt provides a practical and straightforward platform for automating dialogue behavior detection, if its proves successful. to this end, our work focuses on two main objectives: our findings indicate that automated methods for dialogue behavior detection have not reached satisfactory results, falling short of human performance. however, chatgpt showcases compelling results comparative to or often better than specialized models. to facilitate further advancements, we conduct an in-depth analysis to identify the prevalent errors and shortcomings of chatgpt. this analysis provides valuable insights, highlighting key areas to be targeted to enhance the performance of llms in dialogue behavior detection for future work. we release our code and data at https://github. com/emorynlp/gpt-abceval . ## related work chatgpt has shown promising performance on many nlp tasks, especially for text classification @xcite @xcite . in addition, gpt models, including chat-gpt and instructgpt, have been used to produce high-quality dyadic dialogues @xcite and have been shown to correlate highly with human annotators when evaluating the overall quality of empathetic dialogues @xcite . however, chatgpt still exhibits limitations as @xcite show that chatgpt struggles with fine-grained dialogue understanding, reporting poor performance on classifying discourse structure and utterance relations. to the best of our knowledge, no prior research has explored the use of any gpt model as a behavior classifier for chatbot responses. instead, previous work has focused on the development of specialized dialogue behavior classifiers, as discussed in this section. ## abc-eval dataset we use the abc-eval dataset from @xcite as the behavior detection benchmark. this dataset contains 400 open-domain human-bot dialogues collected between university students and one of four chatbots: blenderbot2, blenderbot using decode reranking, emora, and bart-fid-rag. for each bot response in each dialogue, human annotators labeled whether or not a specific dialogue behavior was present. these turn-level binary annotations were collected using crowdworking annotators on the surgehq platform, foot_0 who were trained on three curated conversations to accurately identify each dialogue behavior before being accepted into the annotation project. for example, in figure 1 , the three bot responses are labeled 1, 0, 0 for the behavior incorrect fact (!fac) and are labeled 0, 0, 1 for the behavior self contradiction (!sel). in this work, we take 1,634 bot responses from 108 dialogues that received two rounds of human annotations, and focus on the nine dialogue behaviors that @xcite found as the most informative for capturing dialogue quality ( ## specialized behavior detection models in this section, we present state-of-the-art models designed to classify labels that closely align with six of the dialogue behaviors in ## llm-based behavior detection for llm-based dialogue behavior detection, we use openai's gpt-turbo-3.5-301 (henceforth, chatgpt). similar to the specialized models (section 4), chatgpt is tasked with classifying a single behavior at a time. following the human annotator training process for abc-eval, we use the three training conversations for each label as our prompt engineering testbed. this section highlights key decisions of our prompt engineering process. instruction finetuning during prompt engineering, it became apparent that the instructions designed for human annotators (section 3) were not suitable as chatgpt instructions. we iteratively refined the instructions such that chatgpt's mistakes on the training conversations were reduced. this involved removing instructions chatgpt appeared to misunderstand as well as adding additional behavior details and specifications. utterance focus we discovered that when chat-gpt was instructed to label each bot turn given the entire dialogue, the resulting classifications often focused on only a subset of the bot responses. to ensure consistent and robust labeling for every bot utterance, our final prompt provides the dialogue history paired with the next bot response as the target utterance to be labeled. ## evaluation to evaluate the detection capability of the models in sections 4 and 5, we compare their performance against that of human annotators. for this, we s1: hi! s2: hi! i am happy to be talking to you. do you have any pets at home? s1: i don't, my mom is allergic to most pets. i used to have a pet fish when i was much younger. s2: do you want a pet at some point in the future? no judgment here, i'm just curious. s1: i would love to have pets in the future! once i c have my own place, i'd like to get a dog or two. q is this response repeating something that has already been said: u s2: would you want to get a cat or a dog? a response is repetitive if: -it repeats something from earlier in the dialogue -it includes asking a question whose answer has been already shared if any part of the response is repetitive, then it should be labeled as repetitive. note that sometimes repetition is useful, such as for emphasis, acknowledgement, clarification, or elaboration, and in these d cases it should not be labeled as repetitive. provide your reasoning when considering this question starting with "reasoning:". then, finish by writing your final decision as one of: "decision: [yes]" or "decision: [no]". ## chatgpt error analysis we perform an error analysis of chatgpt's predictions of dialogue behaviors to better understand its limitations. for each dialogue behavior, we select 40 instances where chatgpt and humans disagree, and examine the reasoning provided by chatgpt prior to its final decision ( @xcite ; see examples below). ## recommendations given the compelling performance for many dialogue behaviors observed in this work, chatgpt is a promising direction for behavior classification. for one, it is worth noting that chatgpt boasts extreme cost-efficiency relative to humans. where chatgpt costs @xmath00.29 to $1.96 depending on the behavior ( furthermore, the results of our error analysis reveal a large degree of systematicity behind chat-gpt's reasoning mistakes across many of the behaviors. correcting these common mistakes is likely to further improve its performance to a noticeable degree. we next discuss mitigation strategies of these identified issues to aid in future work. ## limitations although chatgpt is a high-performing, widely accessible, and affordable llm at the time of writing, there are considerations towards the long-term applicability of the results found in this work due to the chatgpt infrastructure. since chatgpt is not open-source and is only accessible through a paid api, there is less detailed understanding of its training and model design. in addition, this access method for chatgpt also results in less user control over potential model changes and even model deprecation over time. as such, further studies could assess the applicability of other language models to the task of dialogue behavior detection to mitigate these concerns, and we leave this to future work. furthermore, it should be noted that the errors made by chatgpt may not necessarily align with those made by alternative open-source language models, or even future versions of chatgpt itself. however, it may still be useful to be mindful of the prominent problems encountered with chatgpt while using other llms. these identified phenomena play a crucial role in language comprehension and reasoning overall and could also present challenges for other models, although the extent of their impact remains to be explored. ## conclusion although automated methods for dialogue behavior classification remain a challenging task, this work finds that chatgpt-3.5 presents promising potential to reduce the gap between model and human performance. chatgpt's ability to provide competitive behavior classification against specialized classifiers without necessitating finetuning or human annotation across a variety of dialogue behaviors gives rise to a low-cost, multi-task evaluator model. the systematicity behind the common mistakes observed for chatgpt reveal concrete steps for future improvements that will improve behavior classification performance, including strategies for context management and better understanding of situational nuances. we look forward to future advancements in behavior classification that leverage chatgpt's unique capabilities.