M ed C are: Advancing Medical LLM s through Decoupling Clinical Alignment and Knowledge Aggregation (2024)

Abstract: Large language models (LLMs) have shown substantial progress in natural language understanding and generation, proving valuable especially in the medical field. Despite advancements, challenges persist due to the complexity and diversity inherent in medical tasks, which can be categorized as knowledge-intensive tasks and alignment-required tasks. Previous approaches either ignore the latter task or focus on a minority of tasks and hence lose generalization. To address these drawbacks, we propose a progressive fine-tuning pipeline. This pipeline employs a and a to encode diverse knowledge in the first stage and filter out detrimental information. In the second stage, we drop the to avoid the interference of suboptimal representation and leverage an additional alignment module optimized towards an orthogonal direction to the knowledge space to mitigate knowledge forgetting. Based on this two-stage paradigm, we proposed a Med ical LLM through decoupling C linical A lignment and Knowledge Agg re gation (), which is designed to achieve promising performance on over 20 medical tasks, as well as results on specific medical alignment tasks. Various model sizes of (1.8B, 7B, 14B) all demonstrate significant improvements over existing models with similar model sizes. Our code and datasets are available at https://github.com/BlueZeros/MedCare .

## introduction large language models (llms) @xcite @xcite have made significant strides in the realms of natural language generation and thereby finding extensive applications across a broad spectrum of disciplines @xcite . among these, the alignment-required tasks identify the entities in the sentence: premature infants have lower stores of vitamin a in their liver, limited fat absorption capabilities, and faster growth and development rates, which make them more prone to vitamin a deficiency.\nmedical entity options: medical tests, diseases, medications, clinical manifestations, body parts\nanswer: the entities in the sentence include: medical test entities: medical field has attracted considerable attention due to its importance and demand. much effort has been dedicated to researching and developing specialized llms for this domain @xcite @xcite . although progress has been made, llms still face challenges due to the complexity and diversity of medical tasks. this is particularly evident when data is intentionally structured to represent a diverse set of tasks @xcite @xcite . we categorize medical tasks into two types: knowledge-intensive tasks and alignment-required tasks. knowledgeintensive tasks, such as medical question answering @xcite and medical dialogues @xcite , primarily test the model's understanding of internal medical knowledge. conversely, alignment-required tasks, like clinical terminology standardization @xcite and medical entity recognition @xcite , demand not only medical knowledge but also strict adherence to output formats. figure 1 illustrates these two categories. in general scenarios, like medical question-answering or dialogue, the llms only need to answer the user's questions correctly. however, tasks in clinical applications like report generation or situations usually require the model to produce a correct response and adhere to specific formats. previous medical llms, such as hu-atuogpt @xcite , huatuogpt-ii @xcite , med-palm @xcite and biomistral @xcite , have primarily focused on enhancing the encoding of medical knowledge in llms, neglecting the requirements of alignment-required tasks @xcite . on the other hand, alignment-oriented methods @xcite @xcite ) that perform alignment through fine-tuning often suffer from an "alignment tax". this results in knowledge forgetting and performance drops in knowledge-intensive tasks @xcite . these issues limit the practical use of llms in healthcare. to create a more practical medical llm, in this paper we propose a novel training pipeline consisting of two progressive fine-tuning stages: miscellaneous knowledge aggregation (mka) and downstream alignment (da). in the first stage, we introduce two modules, knowledge aggrega-tor (ka) and noise aggregator (na), to encode advantageous knowledge and noisy contents, respectively. the noise aggregator is asymmetric with respect to knowledge aggre-gator, which explicitly induces knowledge aggregator to perform multi-task knowledge extraction. to avoid catastrophically parameterized knowledge disruption, we innovatively remove the updated noise aggregator but retain only the knowledge aggregator after training. following this stage, we introduce an alignment module to cater to the downstream alignment requirements from the specific task. we add an orthogonal regularization term to ensure non-overlapping between the optimization space of alignment and knowledge space in the first stage. built upon this pipeline, we propose medcare with three sizes (1.8b, 7b, 14b), a specialized llm tailored for both knowledge-intensive tasks and alignment-required tasks, derived from the qwen1.5 series. figure 2 provides a visual representation of med-care. in summary, our contributions are as follows: ## preliminaries low-rank adaptation (lora) given an input sequence @xmath0, lora @xcite proves that the update of original linear layers ∆w in large language models is of low-rank, and can be decomposed into the multiplication of two compact matrices ab. the pretrained weight w is frozen in the training phase and does not undergo gradient updates, while a and b are trainable parameters and contribute together to the forward pass: attention ffn knowledge aggregator knowledge aggregator add & norm add & norm noise aggregator knowledge-intensive tasks & alignment-required tasks ⊕ ℒ nll merged attention ffn knowledge aggregator add & norm add & norm alignment module ℒ orth ℒ nll pretrained parameter merged parameter fine-tuned parameter miscellaneous knowledge aggregation (mka) downstream alignment (da) ⊕ examinations treatments diagnosis name entity recognition image analysis report generation name entity recognition image analysis report generation alignment-required tasks beginning of training, b is initialized to an all-zero matrix and a uses gaussian initialization to make sure that the product ab is zero at initialization. ## medcare in this paper, we categorize the medical task into two primary genres: knowledge-intensive tasks and alignment-required tasks. knowledge-intensive tasks require llms to seamlessly transfer both medical knowledge and their general domain reasoning capabilities into the medical domain. models leverage the internal broad pre-trained knowledge without significantly disrupting the pretrained distribution, to fulfill domain-specific medical reasoning tasks. in contrast, alignment-required tasks pose a greater challenge. these tasks typically deviate more substantially from the model's original training paradigms, which can lead to hallucinations @xcite and the aliasing of pre-trained knowledge @xcite . to address this challenge, we propose medcare, a method designed to simultaneously acquire the necessary knowledge for the general adaptation of llms and to achieve specialized alignment for specific tasks. the fine-tuning process of med-care comprises two stages: miscellaneous knowledge aggregation (mka; §3.2) and downstream alignment (da; §3.3). the training procedure for medcare is illustrated in figure 2 . ## discussion in this section, we discuss the following research questions (rq) of the proposed medcare: response to rq1: noise aggregator interferes with models because of the routing mismatch. we investigate the effect of different combinations of molora experts in noise aggre-gator on the final performance to demonstrate the routing mismatch. we first compute the activation frequencies for each combination of the experts in vanilla molora inference. meanwhile, different expert combinations are designated to test the performance of the models. the results in figure 4 show the mismatch between the experts' activation times and their corresponding performance, which indicates that the router in the molora module failed to select the experts with optimal performance for each task and even caused the per20 40 60 80 100 datasize (%) 38 40 42 44 46 score medcare w/ na medcare base model figure 6: average performance on the medical knowledge examination with different knowledge aggregation learning data size. 'base model' indicates the performance of the qwen1.5-1.8b without fine-tuning. response to rq2: yes. symmetric structure failed to decouple the learning process into the common knowledge and task-specific requirements. to demonstrate this, we tested the models with two parallel lora modules to investigate the role of the noise aggregator. as shown in response to rq3: the ka stage improves the knowledge capacity of the models and the da stage adapts the models to learn the target formats. we explore the respective role of each finetuning stage and empirically validate each merit. figure 5 shows the results on knowledge examination, cblue, and ccte, respectively. for the first stage, it is obvious that the models with the mka fine-tuning stage not only achieve better performance on the knowledge examination tasks but also align with the format of the downstream dataset with greater ease. although discarding noise ag-gregator after the first stage of fine-tuning reduces the alignment performance of the model, the target format can be learned faster by retraining the model with the second-stage medical adaptation. as more second-stage aligned data is added, the model's alignment capability significantly improves without compromising its knowledge capacity. this suggests that the second stage of the fine-tuning process primarily helps the model align with the format, rather than learning new knowledge. response to rq4: yes. even with limited data, medcare can enhance its medical knowledge proficiency. to show the effective knowledge learning ability of medcare, we compare the performance of models with varying scales of the first-stage fine-tuning corpus, and present the results in figure 6 . the vanilla molora suffers from performance degradation when the size of the training-corpus is smaller than 50%, while med-care utilizes the knowledge in the corpus among all sizes of the data and surpasses the performance of the base model consistently, demonstrating that medcare has better knowledge utilization ability. response to rq5: medcare outperforms the commonly used peft methods on knowledgeintensive tasks. we choose four peft methods to show the effectiveness of medcare, including lora (hu et al., 2021), lora+ @xcite , mora @xcite , and dora @xcite . it is observed in response to rq6: medcare shows strong cross-lingual generalization ability with only fine-tuned with monolingual data. we choose three english medical datasets, medqa @xcite , medmcqa @xcite , and the medical part of mmlu @xcite , to validate the cross-lingual capacity of the llms. the results in ## conclusions in this paper, we first categorize medical tasks into knowledge-intensive tasks and alignment-required tasks to reinforce the practical usability requirements for medical llms. then, we propose a twostage fine-tuning framework to adapt llms to medical domains and mitigate knowledge performance degradation and propose medcare. the experiment results show that medcare can achieve promising performance on over 20 knowledgeintensive and alignment-required tasks.