Knowing Before Saying: LLM Representations Encode Information About Chain-of-Thought Success Before Completion (2025)

Abstract: We investigate whether the success of a zero-shot Chain-of-Thought (CoT) process can be predicted before completion. Our classifier, based on LLM representations, performs well even before a single token is generated, suggesting that crucial information about the reasoning process is already present in the initial steps representations. In contrast, a strong BERT-based baseline, which relies solely on the generated tokens, performs worse—likely because it depends on shallow linguistic cues rather than deeper reasoning dynamics. Surprisingly, using later reasoning steps does not always improve classification. When additional context is unhelpful, earlier representations resemble later ones more, suggesting LLMs encode key information early. This implies reasoning can often stop early without loss. To test this, we conduct early stopping experiments, showing that truncating CoT reasoning still improves performance over not using CoT at all, though a gap remains compared to full reasoning. However, approaches like supervised learning or reinforcement learning designed to shorten CoT chains could leverage our classifier’s guidance to identify when early stopping is effective. Our findings provide insights that may support such methods, helping to optimize CoT’s efficiency while preserving its benefits.

## introduction chain-of-thought (cot) prompting @xcite enhances the capability of large language models (llms) to perform multi-step reasoning. it explicitly guides the llm in creating intermediate explanations to solve a problem, offering a sequence of reasoning steps while responding to a prompt. given its effectiveness, cot has found success in mathematical reasoning @xcite , 1 code and data is available at github.com/anum94/cotpred. medical applications @xcite , faithfulness evaluation @xcite , and multimodal models @xcite @xcite . while cot reasoning has been shown to improve performance across many tasks, it is computationally expensive, as it requires decomposing complex problems into a series of intermediate steps, each demanding its own processing. this raises two intriguing questions: a) do llms implicitly "know" whether they will arrive at a correct answer before completing their reasoning? and b) if progressing past the initial steps doesn't improve this knowledge, does this indicate that the llm has completed its calculation? given the high computational cost of cot, understanding when and how llms "know" their answer could enable more efficient and targeted reasoning strategies. developing a method to assess whether cot will lead to a correct conclusion could optimize resource allocation-stopping reasoning early when the outcome is clear or dedicating more steps when uncertainty remains. furthermore, this knowledge could inform annotation efforts to support cotspecific fine-tuning. to explore these questions, we create a cot success prediction dataset derived from popular mathsolving datasets, where zero-shot cot has been shown to significantly outperform vanilla prompting (more details are provided in section 4.3). by applying a cot prompt to these questions, we capture the llm activations across multiple cot steps and annotate the final answer as either correct or incorrect. we then train a probing classifier (belinkov, 2022) on top of the llm representations (assuming white-box access) to predict if a given prefix of the ongoing cot sequence would lead to a correct answer (see figure 1 ). our experiments show that by leveraging the llm's internal representations, our classifier can effectively predict whether a cot sequence will be successful-even before generating a single token-achieving 60% to 76.4% accuracy across different datasets and llms. notably, it outperforms bert @xcite , a strong text classifier that relies only on the input text, demonstrating that the llm's internal representations encode valuable information about intermediate calculations-information that bert, constrained to shallow linguistic features, cannot capture. further experiments with mid-cot steps reveal an intriguing pattern: in two of six cases, providing the classifier with later reasoning steps does not significantly improve its prediction accuracy. using svcca, a complementary method to probing, we find that in these scenarios, earlier steps are more similar to the final step compared to the dataset where additional cot context benefits the classifier. we conduct initial zero-shot experiments to investigate whether this similarity allows for early termination of the cot process without affecting the final answer. while zero-shot alone is insufficient, our results suggest that more targeted approaches, such as supervised learning or reinforcement learning, could effectively shorten the cot process while maintaining strong performance. to conclude, our contributions are as follows: ## related work we review studies that analyze and customize cot reasoning and those that use the internal llm representations to predict aspects of their generation in advance. given the breadth of research in these areas, we focus on the most relevant works for our setup. ## results and discussion since the test set for llama-3.1-8b was manually annotated by human experts, it is considered more reliable than the gpt-4o mini-annotated test set used for mistral-7b. as a result, in certain analyses where only one llm is presented, we focus solely on llama-3.1-8b to ensure more reliable evaluation. ## see appendix b for scores between all possible combinations of representations reasoning step early stopping in cot reasoning to examine whether the model's intermediate representations encode sufficient information for correct answers, we prompt it to generate an answer at various reasoning steps and evaluate its accuracy. as in our previous experiments, this intervention is performed in a zero-shot manner without explicitly training the model to follow such instructions. specifically, we halt the llama-3.1-8b reasoning process by providing the generated sequence up to a certain point, followed by the instruction: "stop all computation and give me the correct answer in 2-3 words, if you already know it". this allows us to assess whether the model can extract a final answer without completing the full chain of thought (see appendix b for an illustration). we conduct human annotation on 100 samples per dataset at three timesteps foot_2 , summarizing the results in ## conclusion we demonstrate that the success of the cot reasoning process can be predicted from the internal representations of the llm even before the generation of a single token. however, we also observe 7 in some cases, halting cot before completion improved answer correctness, as reflected in the corrected column in the table.