Token-Budget-Aware LLM Reasoning (2025)

Abstract: Reasoning is critical for large language models (LLMs) to excel in a wide range of tasks. While methods like Chain-of-Thought (CoT) reasoning and enhance LLM performance by decomposing problems into intermediate steps, they also incur significant overhead in token usage, leading to increased costs. We find that the reasoning process of current LLMs is unnecessarily lengthy and it can be compressed by including a reasonable token budget in the prompt, but the choice of token budget plays a crucial role in the actual compression effectiveness. We then propose a token-budget-aware LLM reasoning framework that dynamically adjusts the number of reasoning tokens based on the reasoning complexity of each problem. Experiments show that our method effectively reduces token costs in CoT reasoning with only a slight performance reduction, offering a practical solution to balance efficiency and accuracy in LLM reasoning. Code: https://github.com/GeniusHTX/TALE.

## introduction reasoning plays a crucial role in enabling large language models (llm) to perform effectively across a wide range of tasks @xcite @xcite @xcite . a variety of methods have been proposed to enhance the reasoning capabilities of large language models @xcite @xcite . among these, chain-of-thought (cot) @xcite is the most representative and widely adopted approach. it enhances the reliability of the model's answers by guiding large language models with the prompt "let's think step by step", encouraging them to decompose the problem into intermediate steps and solve each before arriving at the final answer. figure 1a and figure 1b illustrate an intuitive example. observe that without cot, the llm produces incorrect answers to the question. with a cot-enhanced prompt, the llm systematically breaks the question into multiple steps and reasons through each step sequentially. by addressing each step incrementally, the llm eventually arrives at the correct answer. recent reasoning models, such as openai o1 (openai, 2024c) and deepseek r1 @xcite , integrate cot into their design. notably, these models can perform cot reasoning even without explicit prompting. although reasoning enhancement approaches such as cot impressively improve llm performance, they produce substantial additional overhead, specifically in the form of the increased number of tokens produced @xcite @xcite . as shown in figure 1b , the answer to prompt with cot has notably higher token costs due to the detailed intermediate reasoning steps included in the output. such high token costs can lead to significant expenses, including increased computational resource usage and longer running times during the llm inference, ultimately resulting in significant additional monetary and energy costs. this raises an important question: "is the reasoning process of current llms unnecessarily lengthy, and how can it be compressed?" @xcite demonstrate that llm has the potential to follow a length constraint in the prompt. building on this, we find that including a token budget (see q: peyton scheduled after-work activities of a onehour yoga class on monday, a cooking class that lasts 9times as long as monday' s yoga on tuesday, a half-hour cheese-tasting event on wednesday, a museum tour that takes half as long as the cooking class on thursday, and two hours of errands on friday. how many hours will all peyton's afterwork activities take? let's break down activities step by step: 1. monday: yoga class lasts 1 hour. q: peyton … how many hours will all peyton's after-work activities take? let's think step by step and use less than 10 tokens: let's break down each activity: q: peyton … how many hours will all peyton's afterwork activities take? let's think step by step and use less than 50 tokens: based on the above observations and analysis, we propose a token-budget-aware llm reasoning framework that dynamically adjusts the number of reasoning tokens based on the reasoning complexity of each problem. we call our method tale (token-budget-aware llm reasoning), which includes two implementations: token budget estimation and prompting (tale-ep) and token budget awareness internalization via post-training (tale-pt). tale-ep estimates a reasonable token budget for each problem using zero-shot prompting and incorporates it into the reasoning process, while tale-pt internalizes token-budget awareness through post-training, enabling the llm to generate more token-efficient responses without explicit token constraints in the prompt. we discuss both implementations in section 5. experiment results show that tale significantly reduces token costs in llm chain-of-thought (cot) reasoning while largely maintaining answer correctness. on average, tale-ep achieves a 67% reduction in token usage while maintaining accuracy with less than a 3% decrease. tale-pt cuts token usage by around 50% compared to vanilla cot and achieves competitive performance. ## token redundancy in llm reasoning token budget. previous research @xcite demonstrates that llm has the potential to follow a length constraint in the prompt. causes of token redundancy in llm reasoning. a possible explanation for this token redundancy is that during the post-training phase, such as the rlhf process @xcite , annotators might favor more detailed responses from llms, marking them as preferred. as a result, the model learns to associate longer, more detailed responses with alignment to human preferences and tends to produce such outputs during reasoning. however, in many scenarios, we primarily need llms to provide the correct answer and make accurate decisions, rather than elaborate extensively with detailed explanations. this motivates the need to eliminate redundant tokens in the llm reasoning process in many cases. ## searching optimal token budget as demonstrated in figure 1 , different token budgets have different effects. therefore, it is natural to investigate the following question: "how to search algorithm 1 budget search input: feasibility checking function isfeasible, a large language model m, a given question x and the ground truth label y output: searched budget β 1: function search(isfeasible,m, x, y) 2: ## evaluation in this section, we provide the experiment results to evaluate the effectiveness of two versions of tale, ## conclusion in this paper, we introduce tale, a framework that reduces token redundancy in chain-of-thought (cot) reasoning by incorporating token budget awareness. tale dynamically adjusts the number of reasoning tokens based on the reasoning complexity of each problem, balancing token efficiency and answer correctness. experiments show that tale reduces output token usage and expense significantly with acceptable accuracy loss, outperforming vanilla cot in cost-effectiveness while generalizing well across various llms.