AutoMix: Automatically Mixing Language Models
Authors: name: Aman Madaan, name: Pranjal Aggarwal, name: Ankit Anand, name: Srividya Pranavi Potharaju, name: Swaroop Mishra, name: Pei Zhou, name: Aditya Gupta, name: Dheeraj Rajagopal, name: Karthik Kappaganthu, name: Yiming Yang, name: Shyam Upadhyay, name: Mausam, name: Manaal Faruqui

Large language models (LLMs) are now available in various sizes and
configurations from cloud API providers. While this diversity offers a broad
spectrum of choices, effectively leveraging the options to optimize
computational cost and performance remains challenging. In this work, we
present AutoMix, an approach that strategically routes queries to larger LMs,
based on the approximate correctness of outputs from a smaller LM. Central to
AutoMix is a few-shot self-verification mechanism, which estimates the
reliability of its own outputs without requiring training. Given that
verifications can be noisy, we employ a meta verifier in AutoMix to refine the
accuracy of these assessments. Our experiments using LLAMA2-13/70B, on five
context-grounded reasoning datasets demonstrate that AutoMix surpasses
established baselines, improving the incremental benefit per cost by up to 89%.
Our code and data are available at https://github.com/automix-llm/automix.