Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Models (2024)

Abstract: The growing interest in Large Language Models (LLMs) for specialized applications has revealed a significant challenge: when tailored to specific domains, LLMs tend to experience catastrophic forgetting, compromising their general capabilities and leading to a suboptimal user experience. Additionally, crafting a versatile model for multiple domains simultaneously often results in a decline in overall performance due to confusion between domains. In response to these issues, we present the RolE Prompting Guided Multi-Domain Adaptation (REGA) strategy. This novel approach effectively manages multi-domain LLM adaptation through three key components: 1) Self-Distillation constructs and replays general-domain exemplars to alleviate catastrophic forgetting. 2) Role Prompting assigns a central prompt to the general domain and a unique role prompt to each specific domain to minimize inter-domain confusion during training. 3) Role Integration reuses and integrates a small portion of domain-specific data to the general-domain data, which are trained under the guidance of the central prompt. The central prompt is used for a streamlined inference process, removing the necessity to switch prompts for different domains.Empirical results demonstrate that REGA effectively alleviates catastrophic forgetting and inter-domain confusion. This leads to improved domain-specific performance compared to standard fine-tuned models, while still preserving robust general capabilities.

## introduction large language models (llms) @xcite @xcite have revolutionized the field of natural language processing, demonstrating exceptional general capabilities, such as instruction-following @xcite and complex reasoning @xcite @xcite the models tuned by ft suffer from a severe drop in general performance as the training epoch increases. whereas, the counterparts tuned by rega are better at preserving general capacities while achieving comparable domain-specific performance. @xcite . however, general-purpose llms might fall short in some specific areas requiring professional knowledge, due to the lack of exposure to data in relevant domains. hence, there has emerged an increasing number of studies in developing domain-specific models by injecting domain knowledge into llms in some domains, e.g., medicine @xcite , law @xcite , and finance @xcite . nevertheless, adapting llms to specific areas risks triggering catastrophic forgetting @xcite @xcite @xcite . as shown in figure 1 , the enhancement of specialized abilities comes at the cost of the generic ability to follow diverse instructions. this dilemma underscores the need for effective solutions that uphold the balance between domain-specific mastery and general applicability. besides, directly adapting a single llm to multiple domains simultaneously through standard finetuning may cause inter-domain confusion @xcite , which negatively affects the model perfor-mance in each specific domain. to this end, we propose the role prompt guided multi-domain adaptation (rega) strategy. as shown in figure 2 , given the instructionfollowing pairs from multiple target domains and our collected general-domain instructions, rega reconstructs the training data for robust multidomain adaptation through three key steps. we conduct extensive experiments by adapting several llms in both chinese and english datasets spanning three domains, including medicine, law, and finance. the experiment results exhibit that llms trained with rega surpass other baselines in domain performance by a large margin while having a significant generic performance advantage. furthermore, our detailed analysis underscores the effectiveness of each component of rega. we reveal strong evidence that self-distillation is a reliable method for preventing the loss of general capabilities ( § 5.1). additionally, role prompting is critical in reducing inter-domain confusion ( § 5.2). lastly, role integration proves to be vital for the successful incorporation of knowledge from specific domain roles into a unified central role ( § 5.3), which is essential for the model's adaptability. ## related work catastrophic forgetting it has been observed that domain-specific tuning of llms can lead to catastrophic forgetting @xcite , where an llm loses its ability to perform previously learned tasks effectively. this suggests a balance must be struck between domain specialization and general proficiency. to mitigate catastrophic forgetting, particularly in the context of continual learning, researchers have explored three kinds of strategies. exemplar replay involves preserving and revisiting key training examples to maintain model performance @xcite . regularization methods introduce regulation functions in addition to the loss function to constrain the learning process @xcite . architectural methods adjust the model's structure by adding parameters specific to new tasks or domains @xcite . our task setting is to train an llm that can competently handle multiple domains concurrently, with minimal impairment to its generalist capabilities, differentiating from continual learning where the model is exposed to tasks sequentially, striving to prevent significant forgetting of earlier tasks @xcite . inter-domain confusion furthermore, training a single llm for multiple domains risks triggering inter-domain confusion where the llm may not perform as well in each domain due to the blending of domain-specific knowledge @xcite . therefore, some studies have been directed toward identifying commonalities across domains to maintain model performance while preserving the unique characteristics of each domain @xcite . in this paper, we propose to utilize role prompting to alleviate inter-domain confusion. role prompting previous works found that role prompting can significantly improve the performance of llms. for example, character.ai foot_0 proposes a dialogue agent mimicking diversified figures, which can bring enriched user experience. similarly, @xcite proposes cosplay to perform human-like conversations. moreover, wu et al. (2023b) found llms can effectively evaluate summarization results with diversified role prompts from varied perspectives. kong et al. (2023) found rega-llm 𝑰𝟏 𝑪 : you are a helpful generalist assistant … write a sentence using alliteration. 𝑰𝟏 𝑪 : you are a helpful generalist assistant … define the term "statute of limitations." 𝑰𝟏 𝑪 : you are a helpful generalist assistant … how do i know if i have an allergy to something? 𝑰𝟏 𝑪 : you are a helpful generalist assistant … what is the difference between a stock and a bond? 𝑹𝟏: peter piper picked a peck of pickled peppers. 𝑹𝟏: civil law handles disputes between individuals … 𝑹𝟏: allergy symptoms can include hives, itching … 𝑹𝟏: a stock represents partial ownership in a firm … multi-domain instructions inference procedure of rega-llm 𝑷𝑪: you are a helpful generalist assistant … step-2: role prompting law data 𝑷𝑳: you are an expert legal assistant … 𝑰𝟏 𝑳 : you are an expert legal assistant … define the term "statute of limitations." 𝑹𝟏: civil law handles disputes between individuals … … … med. data 𝑷𝑴: you are an expert medical assistant … 𝑰𝟏 𝑴 : you are an expert medical assistant … how do i know if i have an allergy to something? 𝑹𝟏: allergy symptoms can include hives, itching … … … fin. data 𝑷𝑭: you are an expert financial assistant … 𝑰𝟏 𝑭 : you are an expert financial assistant … what is the difference between a stock and a bond? 𝑹𝟏: a stock represents partial ownership in a firm … … … general data (self-distilled) 𝑰𝟏 𝑪 : you are a helpful generalist assistant … write a sentence using alliteration. 𝑹𝟏: peter piper picked a peck of pickled peppers. … ... 𝑷𝑪: you are a helpful generalist assistant … 𝑰𝟏: write a sentence using alliteration. 𝑰𝟐: how to make a sandwich? 𝑰𝟑: what is the capital of france? … … 𝑹𝟏: peter piper picked a peck … 𝑹𝟐: lay out two slices of bread … 𝑹𝟑: paris. … … step-1: self-distillation law sub-data fin. sub-data med. sub-data 𝑷𝑪 𝑰𝟏 𝑪 : you are a helpful generalist assistant … define the term "statute of limitations." 𝑹𝟏: civil law handles disputes between individuals … 𝑰𝟏 𝑪 : you are a helpful generalist assistant … how do i know if i have an allergy to something? 𝑹𝟏: allergy symptoms can include hives, itching … 𝑰𝟏 𝑪 : you are a helpful generalist assistant … what is the difference between a stock and a bond? 𝑹𝟏: a stock represents partial ownership in a firm … step-3: role integration vanilla llm general data (self-distilled) rega training data general instructions law data med. data fin. data role prompting can also boost the complex reasoning abilities of llms. inspired by these findings, we propose to utilize role prompting to help llms distinguish samples among domains and assign domain-specific abilities to each role. our experiments demonstrate that role prompting can effectively alleviate inter-domain confusion. ## further analysis in this section, we analyze the effects of the three components of rega. ## conclusion in this paper, we attempt to strike a balance between domain specialization and generic abilities while adapting llms to multiple domains. specifically, we propose the rega, which consists of self-distillation to alleviate the catastrophic forgetting, role prompting to separate each domain while assigning each role prompt with domain-specific abilities to avoid inter-domain confusion, and role integration to transfer the domain-specific abilities from the domain-specific role prompt to the central prompt. extensive experiments on plenty of datasets and llms demonstrate the effectiveness and efficiency of our proposed method.