Chamain: Harmonizing Character Persona Integrity with Domain-Adaptive Knowledge in Dialogue Generation (2024)

Abstract: Recent advances in large language models (LLMs) have shown their capacity for generating natural dialogues, leveraging extensive pre-trained knowledge. However, the seamless integration of domain-specific knowledge into dialogue agents, without undermining their personas or unique textual style, remains a challenging task. Traditional approaches, such as constructing knowledge-aware character dialogue datasets or training LLMs from the ground up, require considerable resources. Sequentially fine-tuning character chatbots across multiple datasets or applying existing merging techniques often leads to catastrophic forgetting, resulting in the loss of both knowledge and the character’s distinct persona. This compromises the model’s ability to consistently generate character-driven dialogues within a user-centric framework. In this context, we introduce a novel model merging method, Chamain, which effortlessly enhances the performance of character models, much like finding a “free lunch”. Chamain merges domain-specific knowledge into a character model by parameter-wise weight combination of instruction-tuned models and learns to reflect persona’s unique characteristics and style through Layer-wise merging. Our experiments demonstrate that Chamain effectively maintains style while also solving domain-specific problems to a certain extent compared to the baselines, even showing a higher style probability compared to the character model in legal QA.

## introduction the recent advancements in large language models (llms) have been driving innovation across various fields like open-domain conversational models @xcite . llms demonstrate their capacity not just by solving com-plex computational problems in mathematics @xcite or programming @xcite , but also by delivering expert-level performance in specialized knowledge areas @xcite @xcite . researches on open-domain chatbot focus on integrating personas to develop unique ai agents @xcite . the efforts to make chatbots more human-like are not just for the purpose of obtaining knowledge and information, but to enhance the close interaction between humans and machines @xcite . such efforts have achieved significant commercial applications, allowing users to craft custom ai agents with character-related information, enhancing user-ai interaction. however, it has been observed that relying solely on prompt design, without additional training, as seen in products like chatgpt and character.ai (character.ai., 2022), presents challenges in displaying a consistent persona throughout dialogues @xcite . furthermore, despite efforts to preserve style using character-related dialogue data, the necessity of assimilating new knowledge can lead to catastrophic forgetting @xcite , where the newly acquired information overshadows previously learned character traits @xcite . this indicates a need for a more robust approach to sustain both the acquisition of new knowledge and the preservation of unique character features in ai agents. the emergence of model merging as a prominent area of interest is largely due to the challenges associated with supervised fine-tuning (sft) and multi-task learning. for instance, while sft is an effective method for optimizing language models for specific tasks @xcite , it requires the storage and deployment of a separate model for each task. using sft would necessitate storing and managing distinct models per each task, increasing complexity and storage demands. additionally, models often fail to generalize beyond the data or domains they were trained on, presenting challenges in out-of-domain generalization. in contrast, multi-task learning, which strives to train a single model for multiple tasks, brings its own set of challenges. it offers a solution to the inefficiencies of sft by integrating training across different tasks into a single model. however, this approach necessitates retraining with large and diverse datasets to achieve a balanced representation of each task within the model @xcite . such a balance is critical to ensure that all tasks are learned effectively. the need of providing balanced, extensive, and varied data adds complexity and potential costs of multi-task learning, making it a sophisticated and sometimes expensive endeavor. model merging emerges as a response to these issues, offering a way to integrate the strengths of individual models trained on specific tasks or through multitask learning, while mitigating the limitations of each approach. based on the challenges identified, we introduce chamain, a novel approach that enables chatbots to acquire additional knowledge while maintaining their character and charm without additional extensive training (figure 1 ). chamain is based on the actively researched model merging method @xcite , focusing on maintaining the character and style of the model. chamain consists of three main stages: (1) preparing instruction-tuned models for merging, (2) combining task vectors and character vectors of instruction-tuned models, and (3) subsequently fusing the latter layers of the character model based on the layer selection method. it enhances the model's ability to generate utterances that embody the nuances of the character's persona. we merge three types of models, a conversation model trained on a self-created persona dataset, an instruction-tuned model on a domain-specific data, and a fine-tuned model for downstream tasks within the domain. to verify the integration of new knowledge, we selected law and finance as specific domains for testing. we applied established merging techniques to blend character-driven dialogue models with those designed for specific domains, and evaluated their effectiveness in downstream tasks within each domain. additionally, we assessed the character representation capabilities of these integrated models using a model developed to differentiate between characters. we conducted experiments to demonstrate that our approach is compatible with the most recent advancements in model merging techniques. using the chamain method, we retained about 80% of the performance of extensively task-specific finetuned models and maintained the ability of character models to portray personalities. this achieved persona portrayal on par with dedicated character models, improving upon previous merging methods. in summary, our key contributions are as follows: ## chamain we introduce chamain, a novel approach designed to accurately capture the essence of a specific character, including their unique traits and speaking style, while seamlessly integrating new knowledge. for a detailed depiction of the chamain architecture, refer to figure 2 . chamain combines the weights of existing models (f (θ 1 ), f (θ 2 ), f (θ 3 ), . . . , f (θ n )) at the parameter level without additional training, and integrates the weights of a model specialized for a specific character dataset (θ character ) at the layer level. as in prior studies @xcite , we recognize differences in each layer's impact. we have a neural network f (θ i ) with l i layers, where θ i denotes the parameters of the model. this network maps the input x i,@xmath0,@xmath1,j , y i,j ) in dataset d i (equation 1 ). that is, each model f (θ i ) is optimized independently for its corresponding dataset d i that consists of m i input-output pairs. in this work, we employ three datasets (@xmath2, namely alpaca style dataset (d alpaca , taori et al. ( 2023 )), domain-specific downstream task dataset (d knowledge ), and dialogue dataset reflecting character persona (d character ). let l @xmath3, that all tuned models (θ tuned ) share the same backbone, which refers to θ original . in the second step (equation 2 ), we begin by subtracting the parameter values of the backbone from those of the instruction-tuned model to get the task vector (τ = θ tunedθ original ). as one choice, we can then add all the resulting task vectors to the original model weights applying task-arithmetic @xcite . alternatively, we may choose to retain only the top-k% values and reset the rest to their initial values to remove redundant parameters @xcite . this process is expected to effectively remove the parameters that do not contribute to the model performance. subsequently, an elected sign vector is generated for the merged model by resolving discrepancies in the signs of parameters across the different models. finally, we compute a disjoint mean for each parameter by averaging the parameter values from models with matching signs in the merged model. f (θlayer-wise merged) = f (θcross merged), for @xmath4, for @xmath5 at the final step, we perform layer-wise merging as described in equation 3. for the index of each layer l in the range [l], where [l] denotes the set of integers from 1 to l, we merge f (θ cross merged ) up to a selected layer l s and then switch to f (θ character ) for the subsequent layers. the selection of the layer l s is based on a layer-wise comparative analysis of gradient discrepancies between the character-based (g character ) and knowledge-based (g knowledge ) representations. we calculate gradient difference accumulated through first l layers, denoted by ∆g character l and ∆g knowledge l as shown in equations 4. these differences signify the disparities in learned representations between the character-based and knowledgebased models. the optimal layer is determined by an iterative searching of minimum l satisfying the condition defined in equation 5. we traverse through the layers in descending order (@xmath6, l -1, . . . , 1) until we find the layer where the gradient discrepancies in the character-based model exceed those in the knowledge-based model. ## experimental setup all the baseline models and datasets for training and evaluation regards the korean language. ## conclusion we present chamain, a methodology that allows for the incorporation of domain knowledge into character-specific models without additional training while preserving the models' personas. chamain is designed to be easily integrated with existing model merging methods. it enhances downstream task performance across various domainspecific tasks, drawing enhancement directly from the character model. this offers a comprehensive solution for maintaining character consistency and domain accuracy simultaneously. through chamain, we aim to address the challenges of efficiently combining nuanced character traits with specialized domain knowledge in a unified model.