T ag R outer: Learning Route to LLM s through Tags for Open-Domain Text Generation Tasks (2025)

Abstract: Model routing allocates queries to the suitable model, improving system performance while reducing costs. However, existing routing methods face practical limitations that hinder scalability in large-scale applications and struggle to keep up with the rapid growth of the large language model (LLM) ecosystem. To tackle these challenges, we propose TagRouter, a training-free model routing method designed to optimize the synergy among multiple LLMs for open-domain text generation tasks. Experimental results demonstrate that TagRouter outperforms 13 baseline methods, increasing the accept rate of system by 6.15% and reducing costs by 17.20%, achieving optimal cost-efficiency. Our findings provides the LLM community with an efficient and scalable solution for model ensembling, offering users an evolvable “super model.”

## introduction large language models (llms) have revolutionized the landscape of natural language processing (nlp) by transforming a wide array of nlp tasks into text generation task, outperforming specialized models in various domains @xcite . the remarkable capabilities of llms have attracted significant investments from both academia and industry, accelerating their advancement and widespread application. in 2024, significant advancements were marked by the release of @xcite of openai, ernie 4.0 @xcite of baidu, and qwen2.5 @xcite of alibaba. presently, the hugging face platform hosts over 170,000 models employed in text generation, each varying in architecture, size, training data, and method, leading to a diverse range of capabilities @xcite the selection of llms for specific tasks and scenarios is often guided by their performance on relevant evaluation benchmarks @xcite . generally, models with larger parameter sizes tend to achieve higher scores on these benchmarks @xcite . however, these top scores typically represent the average performance across the benchmarks. given the diverse capabilities of models, which allow them to demonstrate varying strengths across different queries (tab. 1), it is crucial to evaluate their performance at the sample level @xcite . as the llm community advances, the integration of diverse models through model routing promises to enhance capabilities of model system and reduce the reliance on larger llms @xcite . the model routing system automates the selection of the optimal candidate model in model system for each query by capturing its semantic features and generating responses @xcite . the routing system streamlines user interaction by automatically selecting the suitable model, minimizing the complexity and effort involved in searching and testing multiple different models @xcite . most studies conceptualize model routing as a multi-label classification problem @xcite ), yet there remains substantial room for improve-ment. methods requiring the multiple call of candidate models can lead to increased latency and higher system costs @xcite @xcite . other methods fail to manage usage costs effectively, limiting their feasibility for large-scale deployment @xcite @xcite . methods like @xcite ; @xcite ; @xcite require access to logits during inference, complicating the routing control for proprietary models. furthermore, task-specific methods and those requiring specially designed loss functions present scalability challenges for opendomain tasks @xcite @xcite . although some methods address certain shortcomings, they require retraining whenever there are changes in the candidate models, which reduces their adaptability in the fast-evolving llm ecosystem @xcite @xcite . moreover, some methods only support routing between two models with different parameter scales, which limits their scalability for tasks involving multiple models or models with minimal differences in capabilities @xcite @xcite . this work introduces tagrouter, a practical routing method for llms that leverages self-aware tags. tagrouter captures key semantic features of user queries and controls the behavior of multiple models. it seamlessly ensembles models in a training-free manner, while controlling costs and meeting the requirements of open-domain text generation tasks. by leveraging these capabilities, tagrouter improves the efficiency of the increasingly complex model ecosystem, offering users a evolvable "super model." the contributions of this work are as follows: ## evaluation metrics accept rate (ar) quantifies the proportion of queries @xmath0, including both "win" and "tie" responses. ar is defined as: gpt-rank (rank) denotes the average ranking of model m * on dataset q. a value of 1 indicates that m * achieves the highest performance on q. area under curve (auc) evaluates the performance of the model system by computing the area under the curve defined by the routing ratio ρ to m max along the x-axis and the corresponding ar values along the y-axis. the auc is defined as: partial area under curve (pauc) measures the performance of model system in regions where the ar surpasses that of m llm . specifically, pauc represents the area under the auc curve where ar(ρ) > ar m llm , with ar m llm denoting the ar achieved by always routing to m llm . the pauc is defined as: a higher pauc score indicates that the routing system more effectively selects models m * (q) that outperform m llm (q). therefore, pauc serves as a key metric for evaluating the ability of the routing system f (t (q), m ) to enable the performance of the model system m to surpass that of m llm . ## discussions how does tagrouter perform among models with similar capabilities? fig. 6 presents the performance of tagrouter when glm-9b and qwen2.5-7b are selected as candidate models. experimental results demonstrate that tagrouter effectively assigns different queries to glm-9b and qwen2.5-7b, validating its routing capability among models with high similarity. can taggenerator generalize to other dataset? fig. 7 illustrates the performance of tagrouter, trained on the bcuq dataset, when applied to the alpaca and dolly datasets. results indicate that tagrouter identifies query charac-teristics effectively across diverse datasets. moreover, it requires only a small number of labeled samples from the target dataset to further enhance its performance. interestingly, even without datasetspecific optimization, tagrouter consistently outperforms existing routing methods that have been fine-tuned on the specific datasets, underscoring its strong generalization capability (fig. 8 ). how should the threshold of tagdecider be selected? extensive experiments indicate that the default setting of θ = 0 is generally effective. for further optimization, appx. g.2 presents a method for adapting θ to different datasets. how practical is tagrouter? tagrouter is applicable to model routing across text generation tasks and benefits from a training-free manner. when new candidate models are added to the model system, only a small number of samples need to be annotated using the llm-as-ajudge approach (tab. 7 presents the performance under varying sample sizes). the capability features of new candidate models are then stored and quantified in a key-value format. this mechanism enables efficient expansion of the routing system without requiring retraining, ensuring adaptability to the rapidly evolving llm ecosystem. moreover, tagrouter consistently outperforms baseline methods across different datasets and candidate model groups (tab. 3). how efficient is tagrouter? in tagrouter, we utilize a 500mb taggen-erator and a 33mb embedding model, with routing performed via simple key-value lookups. compared to existing routing methods, this design offers a competitive advantage in computational efficiency and latency. why does tagrouter exhibit superior performance? as shown in tab. 2 and fig. 5 , the four tag-based routing methods outperform 10 existing methods. we hypothesize that this superior performance stems from the ability of taggen-erator to extract the core semantic features of potentially redundant, high-dimensional textual information and encode them into a structured set of tags (appx. e.5 and appx. e.6). this process can be seen as a form of automatic dimensionality reduction or feature abstraction, allowing routing models like tagrouter to focus on the main features. therefore, the routing system achieves improved learning efficiency and performance. ## conclusions in this work, we introduce tagrouter, a trainingfree routing method designed to scale with the growth of llms and handle open-domain text generation tasks. extensive experimental evaluations demonstrate that tagrouter not only outperforms 13 baseline routing methods across a variety of datasets and tasks, but also exhibits strong adaptability and generalization. by dynamically orchestrating llms of varying scales and abilities, tagrouter allows users to benefit from highperformance llm services without always relying on larger llms, reducing costs and improving efficiency of the system. its practical design positions tagrouter as a promising solution for developing cost-efficient model systems.