R adial R outer: Structured Representation for Efficient and Robust Large Language Models Routing (2025)

Abstract: The rapid advancements in large language models (LLMs) have led to the emergence of routing techniques, which aim to efficiently select the optimal LLM from diverse candidates to tackle specific tasks, optimizing performance while reducing costs. Current LLM routing methods are limited in effectiveness due to insufficient exploration of the intrinsic connection between user queries and the characteristics of LLMs. To address this issue, in this paper, we present **RadialRouter**, a novel framework for LLM routing which employs a lightweight Transformer-based backbone with a radial structure named **RadialFormer** to articulate the query-LLMs relationship. The optimal LLM selection is performed based on the final states of RadialFormer. The pipeline is further refined by an objective function that combines Kullback-Leibler divergence with the query-query contrastive loss to enhance robustness. Experimental results on RouterBench show that RadialRouter significantly outperforms existing routing methods by 9.2% and 5.8% in the *Balance* and *Cost First* scenarios, respectively. Additionally, its adaptability toward different performance-cost trade-offs and the dynamic LLM pool demonstrates practical application potential.

## introduction recent advances in natural language processing, significantly driven by the development of large language models (llms), has opened new frontiers across numerous applications. llms demonstrate outstanding performance on various tasks, including mathematical problem-solving @xcite , commonsense reasoning @xcite , and code generation @xcite . the growing reliance on llms gives rise to the concept of llm ensemble @xcite , which integrates multiple llms to establish a system capable of multitasking, thereby generating more accurate and robust responses to user inputs. through this collaborative approach, practitioners can leverage the unique strengths of different llms, potentially improving the overall performance and reliability in addressing diverse requirements. however, as the size and complexity of llms increase, the challenges of deploying llm ensemble -such as computational cost, latency, and scalability -also intensify. to address these challenges, llm routing is explored to dynamically assign a specific llm in the llm ensemble to user queries. as shown in fig. 1 , early attempts @xcite employ a binary score router to decide whether to select a smaller llm or a larger one for a given query. @xcite utilizes a router to guide cascaded llms for generating responses. further research @xcite introduces similarity matching to align the query with llms and select the most appropriate llm. however, these methods are limited in the following aspects: 1) they narrow the routing sorely to identify the optimal llm, failing to capture the intrinsic connection between the query and llms, which undermines their effectiveness in achieving ideal routing results. in this paper, we propose a novel llm routing approach named radialrouter, which leverages a transformer-based architecture to enhance performance for efficient and robust llm routing. to represent the interrelationship between the query and llms, we propose radialformer as the backbone of radialrouter and incorporate the structure consisting of a relay node and n satellite nodes. during update, these nodes are processed through the multi-head attention mechanism. compared with the standard transformer, radialformer reduces the computational complexity from o(l 2 d) to o(ld) given the sequence length l and the dimension of the hidden state d. the optimal llm selection is performed based on the final states of satellite nodes. to guide the selection and enhance comprehensive representation, we employ a kullback-leibler divergence loss for supervision. additionally, we cluster the queries into groups and introduce a query-query contrastive loss, which fosters the generation of similar embeddings for semantically related queries, facilitating robust llm routing. we conduct experiments on challenging router-bench @xcite encompassing 4 task domains (commonsense reasoning, knowledge-based language understanding, math, and coding) to evaluate the proposed radialrouter in three scenarios. extensive experiments demonstrate that radial-router efficiently harnesses the interrelationship of routing and outperforms existing routing methods by a large margin. furthermore, the adaptability of radialrouter toward different performance-cost trade-offs and the dynamic llm pool is verified through extended experiments. our contributions are summarized as follows: ## method we consider a set of candidate llms {llm i : @xmath0, . . . n}, which can include local open-source llms and off-the-shelf llms hosted on cloud platforms. our goal is to learn a router to select the most suitable llm for each user query. for each round, the router receives the user query x as input and chooses the optimal llm î for response, balancing high performance with minimal cost. in this section, we propose radialrouter, a framework that leverages a transformer-based architecture for query-based llm routing. fig. 2a shows the framework of the proposed method. we present radialformer, a lightweight transformerbased architecture as the backbone of routing (sec.3.1). based on the update of radialformer, we perform optimal llm selection (sec.3.2) and introduce a contrastive loss to optimize the router (sec.3.3). the overall algorithm of radialrouter is illustrated in alg.2 of appendix b. ## conclusions in this paper, we introduce radialrouter, a novel transformer-based framework for efficient and robust llm routing. we achieve a structured representation of the routing process with radialformer, which efficiently captures the interrelationship be-