Self-Steering Language Models
Authors: name: Gabriel Grand, name: Joshua B. Tenenbaum, name: Vikash K. Mansinghka, name: Alexander K. Lew, name: Jacob Andreas

While test-time reasoning enables language models to tackle complex tasks,
searching or planning in natural language can be slow, costly, and error-prone.
But even when LMs struggle to emulate the precise reasoning steps needed to
solve a problem, they often excel at describing its abstract structure--both
how to verify solutions and how to search for them. This paper introduces
DisCIPL, a method for "self-steering" LMs where a Planner model generates a
task-specific inference program that is executed by a population of Follower
models. Our approach equips LMs with the ability to write recursive search
procedures that guide LM inference, enabling new forms of verifiable and
efficient reasoning. When instantiated with a small Follower (e.g.,
Llama-3.2-1B), DisCIPL matches (and sometimes outperforms) much larger models,
including GPT-4o and o1, on challenging constrained generation tasks. In
decoupling planning from execution, our work opens up a design space of
highly-parallelized Monte Carlo inference strategies that outperform standard
best-of-N sampling, require no finetuning, and can be implemented automatically
by existing LMs.