ICML 2025

A Unified Approach to Routing and Cascading for LLMs
Jasper Dekoninck 1 Maximilian Baader 1 Martin Vechev 1
Abstract
The availability of a wide range of large language
models (LLMs) embedded in various agentic
systems has significantly increased the potential
of model selection strategies to improve the cost-
performance tradeoff. Existing strategies involve
either routing, where a single model is chosen per
query, or cascading, which sequentially runs in-
creasingly larger models until a satisfactory an-
swer is found. However, current approaches face
three key limitations: they (1) lack formal proofs
of optimality, (2) fail to identify the conditions
under which these strategies are most effective to
improve the cost-performance tradeoff, and (3)
are unable to combine both paradigms for fur-
ther improvements. To address these issues, we
first derive a novel optimal strategy for cascading
and prove the optimality of an existing routing
strategy. Further, we propose cascade routing,
a unified framework that integrates routing and
cascading into a theoretically optimal strategy.
Through our analysis, we identify good quality
estimators as the critical factor for the success of
model selection paradigms. Finally, in our ex-
periments, we show that cascade routing consis-
tently outperforms the individual approaches by
a large margin and we analyze quality estimators
to determine when routing and/or cascading are
useful paradigms for model selection.1
1. Introduction
Large language models (LLMs) have found applications in
a wide range of tasks, some of which are easily handled by
small models, while others require the full capacity of state-
of-the-art LLMs. This has led to the development of many
fine-tuned models of various sizes that target specific tasks.
1Department
of
Computer
Science,
ETH
Zurich,
Switzerland.
Correspondence
to:
Jasper
Dekoninck
<jasper.dekoninck@inf.ethz.ch>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Cana...