An Expert is Worth One Token: Synergizing Multiple Expert LLM s as Generalist via Expert Token Routing (2024)

Abstract: We present Expert-Token-Routing, a unified generalist framework that facilitates seamless integration of multiple expert LLMs. Our framework represents expert LLMs as special expert tokens within the vocabulary of a meta LLM. The meta LLM can route to an expert LLM like generating new tokens. Expert-Token-Routing not only supports learning the implicit expertise of expert LLMs from existing instruction dataset but also allows for dynamic extension of new expert LLMs in a plug-and-play manner. It also conceals the detailed collaboration process from the user’s perspective, facilitating interaction as though it were a singular LLM. Our framework outperforms various existing multi-LLM collaboration paradigms across benchmarks that incorporate six diverse expert domains, demonstrating effectiveness and robustness in building generalist LLM system via synergizing multiple expert LLMs.

## introduction large language models (llms)-notably, gpt-4 and llama @xcite -have demonstrated remarkable capabilities across a wide spectrum of tasks. however, their performance and reliability in certain specialized domains sometimes still fall short of expectations @xcite . motivated by this, the recent year has seen emergence in the development of "expert" llms, with a particular focus on tailoring a general llm to excel in specific domains or tasks. for example, there are efforts to refine the pretraining corpus to develop expert llms with enhanced capabilities in specific domains, such as codellama @xcite and deepseek-math @xcite . furthermore, applying continual training or supervised fine-tuning (sft) on specially selected corpora can lead to significant enhancements on targeted tasks like data analysis @xcite and financial analysis @xcite , making a general llm into a specialized expert. in this study, we focus on the following specific research question: how can we synergize various expert llms into a singular generalist framework? the practicality of this question lies in the fact that it's unrealistic to ask users to precisely switch between various expert llms, given the diversity and subtle difference in their capabilities. there are mainly two paradigm to build such a generalist llm system, termed in this paper as the prompt-based methods and the router-based methods. prompt-based methods @xcite @xcite treat expert llms as conversational agents. a "meta" llm guided by crafted instruction communicates with expert llms to acquire domain-specific information. for the router-based methods @xcite , a router model is trained to route each query to decide which expert is responsible to answer it. however, both of the aforementioned paradigms have some notable limitations ( in this work, we introduce a novel technique termed expert-token-routing (etr). etr employs a multi-expert llm collaboration framework, wherein a meta llm can seamlessly switch to a specific expert llm. the core idea of etr is encoding expert llms as special tokens within the vocabulary of the meta llm. inspired by toolkengpt @xcite , which captures tool semantics using special tokens, etr employs expert tokens to characterize the expertise of different expert llms. under this formulation, the meta llm can route to an expert llm like generating new tokens. this approach allows for expert routing without training extra router model, simply by appending trained expert tokens into a frozen meta llm. to train the expert tokens, we construct an automated pipeline to collect expert queries from existing datasets. etr can learn expert token embeddings from collected expert queries, which profiles the strengths of expert llms, thereby handling situations where the expertise of expert llms is implicit or undisclosed. our framework also supports a plug-and-play integration for new expert llms, allowing providers to add their expert llms and trained expert tokens to the framework as modular plug-ins, eliminating the need for modifications to any other part of the framework. another appealing characteristic of our method is that it conceals the collaboration process between the meta llm and expert llms from the user's perspective, facilitating interaction with the framework as though it were a singular llm. in our comprehensive experiments across six different domains, we compare etr with the existing paradigms for expert llm collaboration. our findings indicate that etr significantly outperforms the baselines as a generalist, with an overall performance improvement of 5.64%. it is worth noting that our framework also integrates the advantages of existing paradigms, as illustrated in the core contribution of this work is the introduction of a singular generalist framework that facilitates seamless integration and collaboration of expert llms. this framework not only supports learning the implicit expertise of expert llms from existing instruction dataset but also allows for dynamic extension of new expert llms in a plug-andplay manner. the effectiveness of our framework is showcased across benchmarks that incorporate six diverse expert domains. ## approach in this section, we present etr, which enables the integration of multiple expert llms into a singular generalist llm framework for leveraging expert knowledge without the need for heavily finetuning or designing crafted instruction templates. typically, an expert llm ϵ is derived from base pre-trained llms through continual pretraining or supervised fine-tuning (sft) on data of target domain. by careful data selection, an expert llm of smaller scale can exceed a larger, general-purpose llm in specific areas like coding @xcite or math problem-solving @xcite . given an expert llm library @xmath0, ϵ 2 , . . .}, our goal is to develop an llm framework which can utilizing the expert knowledge available in the library e when solving problems within specific domains. to determine when and which expert to corporate with, we use a general-purpose llm ω as a meta llm. and expert llm ϵ i is represented by a special token (expert token) witin the meta llm's vocabulary. ## related works expert llm collaboration a bunch of methods aim at enhancing the reasoning capabilities of llms in specific tasks by instructing llms into experts through the crafting of prompts. these methods mainly focus on boosting a single llm by simulating multiple expert llm instances, which can also be applied to the setting in this paper. expert-prompting @xcite elicits llms to generate expert-level responses using proper crafting of prompts. meta-prompting (suzgun and kalai, 2024) breaks down complex tasks into subtasks handled by expert instances of a single llm. multi-llm debate @xcite involves the use of multiple llm instances to propose and debate their individual responses and reasoning processes over several rounds in order to converge on a common final answer. there is also a line of methods highly relevant to this paper, which involves training an additional router model to integrate multiple llms into one framework. llm-blender @xcite uses a pair-ranker model for optimal llm output selection and a gen-fuser model for merging the best outputs. zooter @xcite is a reward-guided routing method for llm ensembles that efficiently assigns queries to the most expert llm. however, in scenarios where expert models are dynamically extended, these methods often require retraining the router model. ## conclusion in this study, we introduced a novel approach etr to integrating various expert llms into a unified generalist framework. this method facilitates seamless collaboration among llms without exposing the complexity to the user, effectively turning the framework into a single, versatile llm. through comprehensive experiments across multiple domains, etr demonstrated superior performance over existing paradigms, offering a promising direction for enhancing llm's generalist capabilities by combining the strengths of both general and expert models.