NeurIPS 2025

Reasoning Planning for Language Models
Bao Nguyen1
Hieu Trung Nguyen1
Ruifeng She2
Xiaojin Fu2
Viet Anh Nguyen1
1 The Chinese University of Hong Kong
2 Huawei Noah’s Ark Lab
nbnguyen@se.cuhk.edu.hk, thnguyen@se.cuhk.edu.hk
she.ruifeng@huawei.com, fuxiaojin32@hotmail.com, nguyen@se.cuhk.edu.hk
Abstract
Selecting an appropriate reasoning method for a given query remains a key chal-
lenge in language model generation. Existing approaches typically generate multi-
ple candidate responses and use an aggregation strategy to select the output answer,
often assuming that more candidate answers yield higher accuracy. We revisit this
assumption through a rigorous theoretical analysis, deriving accuracy bounds for
standard aggregation methods under fixed generation distributions and candidate
sizes. Building on these insights, we introduce EPIC, an Ensemble PlannIng with
Contrastive learning framework to learn a shared representation space that captures
both model reasoning abilities and query-method compatibility. EPIC incorpo-
rates our probability bounds as a regularizer in a utility-driven optimization that
balances accuracy and computational cost. Experiments on diverse mathematical
reasoning tasks show that EPIC consistently selects optimal reasoning methods,
improving accuracy while reducing computational overhead. Our code can be
found at https://github.com/nguyenngocbaocmt02/EPIC.
1
Introduction
Large Language Models (LLMs) have demonstrated remarkable abilities to understand and reason
in human natural language. These advancements have transformed applications, including travel
planning [Xie et al., 2024a], AI teaching platforms [Jin et al., 2024], and human population simula-
tions [Park et al., 2023, Bui et al., 2025]. However, even with a pre-trained LLM, the computational
expense of serving LLM-powered systems remains a significant bottleneck due to the massive scale
of the models [Lin et al., 2024], the quadratic complexity of the attention mechanism [Dao et al....