Self-Specialization: Uncovering Latent Expertise within Large Language Models (2024)

Abstract: Recent works have demonstrated the effectiveness of self-alignment in which a large language model is aligned to follow general instructions using instructional data generated from the model itself starting from a handful of human-written seeds. Instead of general alignment, in this work, we focus on self-alignment for expert domain specialization (e.g., biomedicine, finance). As a preliminary, we quantitively show the marginal effect that generic instruction-following training has on downstream expert domains’ performance. To remedy this, we propose self-specialization - allowing for effective model specialization while achieving cross-task generalization by leveraging only a few labeled seeds. Self-specialization offers a data- and parameter-efficient way of “carving out” an expert model out of a generalist pre-trained LLM. Exploring a variety of popular open large models as a base for specialization, our experimental results in both biomedical and financial domains show that our self-specialized models outperform their base models by a large margin, and even larger models that are generally instruction-tuned or that have been adapted to the target domain by other means.

## introduction instruction-tuning @xcite @xcite of large language models (llms) offers a mechanism to adeptly guide models using specific directives, thereby enhancing their versatility across diverse tasks. however, as promising as this concept might seem, it poses an inherent challenge: the substantial need for quality data @xcite @xcite . the very premise of instruction-tuning hinges on the availability of well-crafted, human-annotated data, a resource that * work done during internship at mit-ibm watson ai lab. is both time-consuming and challenging to scale efficiently @xcite . ## preliminaries: benchmarking existing aligned models to motivate our exploration of self-specialization, we first begin by addressing a fundamental question: how well do generally aligned models perform on specialized domains? while popular models, such as alpaca @xcite and dromedary @xcite , have demonstrated effectiveness in following general instructions, it remains unclear whether general alignment can also elicit expertise for a certain domain. investigating this, we assess the capabilities of alpaca and dromedary against their base model, llama-65b @xcite , on a collection of benchmarks within the biomedical domain. we evaluate alpaca as an upper bound, due to its reliance on gpt-3.5-generated datasets @xcite via the self-instruct process @xcite , unlike dromedary, which generates instructional data from its base model. we use 10 biomedical nlp datasets (see section 4 for details), covering a diverse set of tasks to ensure a comprehensive mix of content and also to look at the cross-task generalization, the core of instructiontuning. we find that both alpaca and dromedary have only a slight (1.2 -2.5) advantage over llama in biomedicine. while they are aligned to handle a broad set of instructions, they do not seem to effectively improve their specialized domain expertise; intuitively trading their expertise for generality given finite parameters. in light of these findings, it becomes evident that for cases where we are only interested in expert domains for all our downstream tasks, there remains a large potential for improvement beyond the generic alignment. this underscores the need for a model or approach, like self-specialization, that could potentially uncover specialization while maintaining cross-task generalizability with minimal supervision. ## self-specialization in this section, we describe our method called selfspecialization illustrated in figure 2 . ## experimental settings datasets. for our primary evaluation, we employ various biomedical nlp datasets, most of which are curated in bigbio @xcite . a total of 10 different datasets are adopted to encompass a wide range of nlp tasks: question answering (qa), named entity recognition (ner), relation extraction (re), sentiment analysis (sa), and document classification (dc). following a prior work @xcite , all datasets are transformed into instructional data. additionally, we validate our method in the financial domain to showcase its generalizability. we adopt a total of 10 diverse datasets, covering numerous nlp tasks: summarization (sum), qa, ner, re, sa, and classification (cls), detailed in appendix a. models. we employ mpt-30b (team, 2023) as a base model for main experiments. for the retriever, we use simple yet effective bm25 @xcite , assuming human-labeled data is not sufficient. for benchmarking of generalpurpose aligned models, we evaluate alpaca-65b @xcite and dromedary-65b @xcite that are both based on llama @xcite . in addition to mpt-30b, we adopt llama-2 7b @xcite and falcon-40b @xcite to further validate the general applicability of self-specialization with different scales and base models. we additionally evaluate existing domain-specific models @xcite : medllama and pmc-llama (details are in section 5.2). metrics. in our study, all tasks are approached as a unified text generation problem, aiming to assess the capabilities of generative models. in alignment with an established convention @xcite , we adopt f 1 -score as our main evaluation metric, given an early observation that rouge-l @xcite , as shown in implementation details. for biomedical seeds, we use data sampled from box @xcite , encompassing 32 tasks, up to 5 instances for each dataset, resulting in a compact yet representative 80 seed samples in total, which are also used as demonstrations at inference. for optional external corpus, we leverage pubmed preprocessed in @xcite , which contains ≈30m abstracts. in the financial domain, based on our finding from @xmath0 ## results and analyses here, we provide a set of experimental results and analyses to address relevant research questions. ## related work the goal of instruction-tuning and alignment of large language models (llms) is to achieve crosstask generalization or to align with human preferences. this can be accomplished by either while enhancing general alignment, according to our presented evidence, these approaches are unlikely to induce specialization in expert domains, leaving different domain expertise in superposition inside the model. to the best of our knowledge, we are the first to show the potential for expert domain specialization through self-alignment, effectively "uncovering" a domain expert out of the model in a parameter-and data-efficient manner. recent studies highlight the benefits of employing instructions in different adaptation scenarios @xcite . instructor @xcite illustrated the adaptability of instruction-based text embeddings to various tasks and domains, while instructe @xcite demonstrated that incorporating instructions with a schema can yield robust results for table extraction across diverse domains. however, these require the use of costly human labels or extensively tuned large models (e.g., 175b). self-training has also been explored for different adaptation scenarios. for domain knowledge adapation, @xcite and @xcite proposed constructing synthetic data by generating in-domain question-answering data, but these data generators are trained with more than 80k human curated qa pairs and do not involve instructional ones that have the potential for cross-task generalization. instruction-tuning has been shown to adapt pre-trained llms to different modalities, including vision @xcite , audio @xcite , and programs @xcite , and enables the use of apis @xcite and search engines @xcite . unlike these works, our work focuses on uncovering target domain expertise latent within llms while promoting cross-task generalization with minimal supervision. ## conclusion our exploration into self-specialization aimed to elucidate the latent expertise within large language models (llms) with limited human supervision. this scheme demonstrated promising results in specialized domains. the self-specialized model exhibited remarkable performance, outperforming its base model, mpt-30b, and even surpassing larger generally aligned models (65b). this illuminates the intrinsic challenges of encoding vast general knowledge into limited parameters and underscores the efficiency of self-specialization. remarkably, the model's efficient training, marked by minimal data usage and the integration of qlora @xcite , adds another layer to its practicality in terms of parameter and data efficiency. these findings signify a promising pathway for leveraging inherent expertise in llms and offering a large variety of exciting opportunities for future work.