PANDA : Persona Attributes Navigation for Detecting and Alleviating Overuse Problem in Large Language Models (2024)

Abstract: In the persona-grounded dialogue (PGD) task, it is required not only to respond fluently, but also to ground the attributes according to the current conversation topic properly. However, due to their tendency to overly ground given attributes, LLMs often generate unnatural responses provoked by using attributes that deviate from the flow of the conversation or by exploiting too many attributes at once. We term this phenomenon the *overuse* problem of LLMs. Unfortunately, research devising precise criteria and frameworks to quantitatively verify LLMs’ *overuse* problem is obviously insufficient. To address this issue, we propose **P**ersona **A**ttributes **N**avigation for **D**etecting and **A**lleviating the *overuse* problem (**PANDA**) framework. **PANDA** is the first study to quantify the persona *overuse* problem of LLMs by establishing clear standards of the problem and verifying various LLMs based on them. Moreover, this framework navigates us into understanding persona attributes by introducing diverse and detailed dialogue topics that consider practical conversation situations. We provide insights related to LLMs’ persona attribute *overuse* problem through comprehensive verification and analysis with **PANDA** in the PGD task. Our code and resources can be found at http://github.com/jin62304/PANDA.

## introduction the persona-grounded dialogue (pgd) task aims to generate responses that fit the conversation's flow based on pre-defined personas. this task is practical because it is effectively used in the real world, for example, empathetic chatbots in mental health care or education @xcite . for the pgd task, the model is required to be able to continue a conversation appropriate to the context while appropriately utilizing its own persona (self persona) and the persona of the conversation partner (partner persona). previously, relatively moderate-sized pre-trained language models (plms) have suffered from a lack of consistent personality regarding persona grounding, such as not actively leveraging persona attributes @xcite @xcite . this challenge has led to the revitalization of several existing studies focusing on how faithfully persona attributes are grounded. for example, there have been attempts to consider how diligently a model grounds persona attributes, based on the cosine similarity or f1 score between given persona attributes and the model's response @xcite @xcite . conversely, with the advent of the era of large language models (llms), which have greatly improved the overall fluency of responses and instruction-following abilities, a phenomenon is often observed in which llms tend to excessively utilize persona attributes in conversational situations, generating unnatural responses. we term this phenomenon the overuse problem, where the model grounds overfull persona attributes at once or exploits those that deviate from the dialogue topic. as shown in figure 1 , which contains the actual llm's response, the llm recklessly uses several personas, including off-topic attributes from the conversation context in the partner's utterance. this overuse phenomenon hinders the interaction between the system and the user in actual conversation situations and reduces end-user satisfaction, so it needs to be detected and alleviated. considering examples from a similar research field, numerous studies on emotional support conversation (esc), particularly those involving roleplaying tasks such as the pgd task-a significant area of research in real-world mental health care-have regarded problems that deviate from behavioral strategies appropriate for specific stages of the conversation as critical. this is due to the structured roles that participants are expected to fulfill at each stage @xcite . however, there is insufficient research to quantify and formalize the problem and verify the status of llms to alleviate the persona attributes overuse phenomenon of llms in the pgd task. this is because existing persona attributes grounding studies focus on being as similar to persona attributes as possible and grounding them to the fullest extent, which is the opposite direction of the overuse phenomenon. to address this issue, this study is conducted with the following research question: accordingly, we propose persona attributes navigation for detecting and alleviating overuse problem (panda), a verification framework with a dialogue flow-aware task and measurement method that becomes the basis for alleviation by detecting llms' persona overuse problem. moreover, we devise a verification resource construction method to execute the framework. the panda framework consists of multiple steps: dialogue labeling, personatopic mapping, and overuse measurement. in this study, the following phases for panda framework design are explained: i) two criteria for quantifying the overuse problem-'off-topic' and 'excess of quantity'-are clearly set ( § 2.2), and ii) to typify persona attributes, from the dialogue topic perspective, 14 fine-grained topics are introduced that take into account the actual conversation situation. ( § 3.1). afterward, based on i) and ii), iii) we design a task based on multiple steps for detection of the overuse problem ( § 3.3). iv) additionally, the creation method of the verification dataset to perform this task is explained ( § 4.1). our contributions are threefold: (1) we conduct the first comprehensive study to formalize the overuse problem that occurs in persona-grounded dialogue tasks and verify various llms based on this. (2) we propose a panda framework that detects and verifies the overuse problem and a resource creation method for it. (3) we serve as a navigator to improve the interpretability of llms' use of persona attributes based on panda's fine-grained topic taxonomy and sophisticated quantification that considers real-world conversation situations. in this section, we describe why panda is significant for verifying the overuse phenomenon and set the criteria to quantify the overuse problem. ## dataset annotation for panda in this section, we describe the process of building verification resources to perform the panda ( § 5) task designed to quantify the persona overuse problem. we annotate the dataset according to the detailed dialogue topics defined in section 3.1. ## related works there have been increasing studies in dialogue responses based on explicit persona descriptions, focusing on the personachat dataset @xcite . traditionally, studies have focused on improving the fluency of model responses rather than grounding persona attributes due to the lack of a consistent personality in moderately sized pretrained language models (plms) @xcite @xcite . for instance, there have been attempts to improve fluency by integrating weights from persona and context information through attention mechanisms, or by utilizing knowledge graph knowledge @xcite @xcite . rather than focusing on individual persona attributes, these studies feed the persona sentences all together into the model to be implicitly considered during the generation phase @xcite @xcite . moreover, research on persona grounding has aimed to ground persona attributes faithfully. for instance, @xcite utilized cosine similarity between attributes and model responses to enhance the ability of personas grounding, @xcite integrate persona-aware prompt learning into models to enhance coherence between persona and response generation. also, @xcite and @xcite adopt evaluation methods such as f1 score between model response and persona attributes to assess how faithfully a given persona is reflected. however, in the era of advanced llms with enhanced instruction-following capabilities, the perspective of similarity to and obsessive faithfulness towards attributes from existing research falls short in addressing the overuse problem, where persona attributes are excessively employed out of context. therefore, to address the overuse problem of llms, we increase the explainability of model analysis through fine-grained categorization of persona attributes. for this purpose, inspired by a study that previously classified persona attributes into primarily five types @xcite , we expand upon this by defining a much broader range of categories for persona attributes. we lay the groundwork for detecting and mitigating overuse through navigating an understanding of persona attributes from a dialogue topic perspective. our study is the first to quantify and verify the important but not yet discussed overuse problem of llms. ## conclusion this work is the first study among personagrounded dialogue studies to address the overuse problem of llms, which generates unnatural responses due to models' excessive grounding tendency. we propose panda, a verification framework including task and measurement design to verify and alleviate the persona attributes overuse problem of llms, and also devise a resource construction method for this purpose. to this end, we establish 1) 'off-topic' and 2) 'excess of quantity' as criteria to quantify the overuse problem, and formulate the overuse detection task based on these criteria. to navigate the explainability of persona attributes, we present 14 detailed dialogue topic types that consider real-world conversation situations, and typify persona attributes based on them. by utilizing these types for the overuse detection considering the dialogue topic, the interpretability of llms' persona attributes grounding is improved. through a comprehensive analysis, it is confirmed that the panda framework is appropriate for addressing llms' overuse problem that actually occurs, compared to other existing evaluation approaches for the pgd task. 12013 our verification framework, panda, serves as a basis for alleviation by detecting the overuse problem in persona-grounded dialogue tasks, and hallucinations regarding personas are observed occasionally in the generated responses. however, since the case of hallucinations is a severe problem even in large language models with enormous parameter sizes, it is required for our nlp communities to continue to solve the challenge. also, due to the issues of api cost and gpu resources for gpt family models, experiments were conducted with examples randomly sampled from the entire data, and more advanced llms, such as gpt-4, were not adopted. the number of cases may be relatively small for evaluating the entire aspects of the capabilities. we plan to improve our framework for future work by conducting human evaluations with considerable cases and enhancing the way of qualitative analysis for addressing the model's hallucinated answers. as miniaturization technology advances, verification of sllms with more compressed parameter sizes is also a desirable direction for gpu resource issues.