Thinking Before You Speak: A Proactive Test-time Scaling Approach (2025)

Abstract: Large Language Models (LLMs) often exhibit deficiencies with complex reasoning tasks, such as maths, which we attribute to the discrepancy between human reasoning patterns and those presented in the LLMs’ training data. When dealing with complex problems, humans tend to think carefully before expressing solutions. However, they often do not articulate their inner thoughts, including their intentions and chosen methodologies. Consequently, critical insights essential for bridging reasoning steps may be absent in training data collected from human sources. To bridge this gap, we proposes inserting insight s between consecutive reasoning steps, which review the status and initiate the next reasoning steps. Unlike prior prompting strategies that rely on a single or a workflow of static prompts to facilitate reasoning, insight s are proactively generated to guide reasoning processes. We implement our idea as a reasoning framework, named Thinking Before You Speak (TBYS), and design a pipeline for automatically collecting and filtering in-context examples for the generation of insight s, which alleviates human labeling efforts and fine-tuning overheads. Experiments on challenging mathematical datasets verify the effectiveness of TBYS. Project website: https://gitee.com/jswrt/TBYS

## introduction openai's o1 (openai, 2024) demonstrates the potential of leveraging long chains of thought (cot) @xcite to enhance the reasoning capabilities of large language models (llms). through its generated reasoning, o1 exhibits advanced cognitive skills, such as problem decomposition, error identification, and correction -processes that continuously guide thinking toward accurate solutions. inspired by this, various test-time scaling @xcite approaches were proposed, such as using prompts ## the tbys reasoning framework tbys utilizes a library l of high-quality insights. the automatic construction of this library is detailed in section 3. during inference, examples are retrieved from l using some off-the-shelf embedding model for in-context learning. we also manually define three seed examples s, each containing a question and the complete reasoning steps for the question with the associated insights. as shown in figure 2 , tbys employs a multi-round reasoning approach. each round t consists of three steps: (1) insight generation: a preliminary insight i pre t is generated based on the current reasoning history h t-1 = (q, (i 1 , s 1 ), (i 2 , s 2 ), • • • , (i t-1 , s t-1 )), where q is the question, and i j , s j denote the insight and solution step in round j, respectively. (2) example retrieval: each insight is defined by its two components: situation (summarizing the current reasoning status) and goal (stating the intention for solution step s t ). the situation of i pre t is used to retrieve k @xmath0. using these k e high-quality insights as in-context examples, a refined insight i t is generated. (3) solution step generation: the solution step s t is generated using h t-1 and i t , then appended to h t-1 to form h t . to signal the end of reasoning, s t includes a field indicating whether a confident answer to q has been reached. ## construction of the insight library as shown in figure 2 , we build the library of insights in two stages: initialization and filtering. initialization: we use manually curated seed examples s and a dataset d s containing questions and their chain-of-thought solutions. first, an llm is prompted to split each solution in d s into 1-3 steps. the llm is then prompted again to generate an insight i t for each solution step s t , consisting of a situation, which should represents the reasoning status up to that step, and a goal, which should offers a purpose and a guideline to stimulate the llm to reproduce solution step s t . all insights and divided solution steps are collected into an initial library l 0 . filtering: to identify high-quality insights, we use a dataset d g (containing questions and groundtruth answers) and a scoring mechanism: (1) for each insight i @xmath1, maintain counters r i (correct uses) and w i (wrong uses). (2) evaluate l 0 by running tbys on each question @xmath2. for each reasoning step for q, retrieve k @xmath3. if the reasoning yields a correct answer, increment r i for each i i used; otherwise, increment w i . (3) rank insights in l 0 by the score r i r i +w i log(r i + w i ), which balances accuracy and usage coverage. select the top-k l examples to form l 1 . the insight library can be progressively improve through multiple iterations. in each iteration, the initial library l 0 is updated to include the filtered library l 1 and the newly generated insights from dataset d g , which is produced during the filtering of l 1 . in our experiments, the math-500 dataset @xcite serves as d s and the test set, e.g., math-500 or aime @xcite , serves as d g in a test-time adaptation @xcite manner, with k l as a variable parameter. ## related work extensive research has investigated prompt designs to improve llm reasoning, including chain-of-thought @xcite , least-to-most @xcite , self-consistency @xcite , and tree-of-thoughts @xcite . methods to enhance task-specific performance include question rephrasing, subtask decomposition, verification, and symbolic grounding @xcite @xcite @xcite ; factuality and faithfulness checking for reasoning chains @xcite ; and separating knowledge retrieval from reasoning @xcite . iterative prompting techniques rely on predefined, hardcoded actions to guide reasoning, such as self-refine @xcite , ircot @xcite , icap @xcite , metagpt @xcite , and chain of ideas @xcite . memory-based methods include buffer of thoughts @xcite , which distills highlevel guidelines from previously solved tasks and stores them in a buffer for future reuse. skill-based @xcite predicts skill-based labels for the questions. @xcite identifies key concepts in questions and uses inductive prompting templates to extract related concepts. and aflow @xcite , demonstrate that iterative training on reasoning histories and taskspecific workflows of correct answers enables models to tackle increasingly complex problems. ## additional comparison experiments we compare with skill-based @xcite , a prompt-guided interaction procedure that enables llms to assign skill labels to math questions and perform icl with label-specific examplars. we also conducted experiments using a kwait approach, where we append "wait, " after the model completion and let the llm to continue its generation for k times. below are the comparison results. results in figure 3 shown that tbys is slight better than skill-based cot, which is task-specific, and is much better than k-wait. ## qualitative analysis of insight quality we provide qualitative analysis of the insights using two selected examples. these problems are relatively simple, but where k-shot reasoning fails. we use these examples to illustrate how tbys's insights effectively steer its multi-step reasoning processes. the first example in figure 6 asks to convert 21 2 2 •5 7 to a terminating decimal. tbys solves the problem in two steps, with the goals of the insights being "ensure the denominator is a power of 10" and "simplify the numerator and express the fraction as a terminating decimal". the second example in figure 7 asks to solve the question: x + √ 3x + 6 + x -√ 3x + 6 = 6. tbys solves the problem in two steps, with the, with the goals of the insights being "simplify the equation by squaring both sides to remove the square roots" and "find the value of x by substitution". both examples demonstrate that tbys generates suitable insights for their respective problems. ## conclusion and future work this paper introduces a novel proactive prompting paradigm, instantiates it with the simple tbys reasoning framework, and verifies the effectiveness of tbys on challenging advanced mathematics reasoning tasks. promising directions for future improvement include: automated search for optimal insights @xcite ; integration of long-term memory mechanisms @xcite ; enhancement of programming capabilities @xcite ; enforcement of structured inference processes @xcite .