Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in Its Latent Thoughts
Authors: Hanwen Du
Yuxin Dong
Xia Ning
Venue: ICLR 2026 Poster (2026)
Keywords: Latent representation learning
scaling test-time compute
Primary area: foundation or frontier models, including LLMs
TL;DR: We demonstrate that latent thoughts of LLMs contain rich reward signals, and scaling test-time thinking with supervision can be directly performed in the latent space.

Abstract: Large Language Models (LLMs) excel at problem solving by generating chain of thoughts in natural language, but such verbal thinking is computationally costly and prone to overthinking. A recent work instead proposes a latent thinking architecture, Huginn-3.5B, which represents intermediate reasoning steps as a sequence of latent representations. However, latent thoughts lack interpretability and are difficult to supervise, raising concerns about the correctness and reliability of the model's latent thinking processes. In this paper, we provide a systematic study of how Huginn-3.5B thinks in the latent space and how external supervision signals can improve its latent thinking processes. We show that latent thoughts leading to correct versus incorrect answers exhibit highly distinguishable patterns, and that a latent classifier can reliably predict answer correctness directly from latent thoughts. Leveraging these insights, we propose Latent Thinking Optimization (LTO), a probabilistic algorithm that employs the latent classifier as a Latent Reward Model (LRM) to optimize the latent thinking processes. Extensive experiments across diverse reasoning tasks demonstrate that LRM is highly effective in detecting incorrect latent thinking patterns, and LTO can significantly improve the latent thinking processes. Furthermore, we show that LRM can generalize across diverse domains, and LTO can be seamlessly applied to general LLMs to improve their thinking processes. In contrast to verbal thinking, our method demonstrates that reward modeling and scaling test-time thinking with supervision can be performed directly in the latent space, highlighting its potential as a general, efficient, and domain-agnostic approach to improving the thinking processes of LLMs.

Published as a conference paper at ICLR 2026
LATENT THINKING OPTIMIZATION: YOUR LATENT REASONING LANGUAGE MODEL SECRETLY ENCODES REWARD SIGNALS IN ITS LATENT THOUGHTS
Hanwen m Yuxin Dong* Xia Ning
DDpparmeent of Computer Science and Engineering, The Ohio State University, USA
Department of Biomedical Informatics, The Ohio State University, USA Translational Data Analytics Institute, The Ohio State University, USA {du. (du.1128) 1128, dong. 1357, ning. 1 104]0osu.ech edu
ABSTRACT
Large Language Models (LLMs) excel at problem solving by generating chain of thoughts in natural language, but such verbal thinking is computationally costly and prone to overthinking. A recent work instead proposes a latent thinking archi- tecture, Hugimn-3.5B, which represents intermediate reasoning steps as sequence of latent representations. However, latent thoughts lack interpretability and are difficult supervise, raising concerns about the correctness and reliability of the model's latent thinking processes. In this paper, we provide a systematic study of how Huginn-3.5B thinks in the latent space and how external supervision signals can improve its latent thinking processes. We show that latent thoughts leading to correct versus incorrect answers exhibit highly distinguishable patterns, and that latent classifier can reliably predict answer correctness directly from latent thoughts. Leveraging these insights, we propose Latent Thinking Optimization (LTO). probabilistic algorithm that employs the latent classifier Latent Re- ward Model (LRM) to optimize the latent thinking processes. Extensive experi- ments across diverse reasoning tasks demonstrate that LRM is highly effective in detecting incorrect latent thinking patterns, and L.TO can significantly improve the latent thinking processes. Furthermore, diverse domains, and LTO can be seamlessly applied to general LLMs to improve their thinking processes. In contrast verbal thinking, our method demonstrates that reward modeling and scaling test-time thinking with supervision can be per- formed directly in the latent space, highlighting its potential as a general. efficient, and domain domain-agnostic approach to improving the thinking processes of LLMs.
show that LRM can generalize across
INTRODUCTION
Large Language Models (LLMs) (OpenAI, 2023; Bai et al., 2023; Touvron et al., 2023a; Gemini Team, 2023) have demonstrated impressive problem-solving abilities by generating natural language as a form of thinking and reasoning' (Wei et al., 2022; Kojima et al., 2022; Yao et al., 2023). This ability to "think" enables them to solve variety of complex tasks, such as math (Lightman et al., 2024: Gao et al., 2023), coding (Li et al., 2022: Nijkamp et al., 2023), and embodied planning (Shinn et al., 2023; Hao et al., 2023). However, generating the whole thinking process in natural language is very costly and prone to the overthinking issue where LLMS output redundant or misleading thoughts that degrade both accuracy and efficiency (Sui et al., 2025; Chen al., 2025).
In contrast, humans think largely through internal latent pnneeseenaaiins-cooppp codes that capture abstract concepts and hidden structures (Quiroga al., 2005; Mishchanchuk et al., 2024). Such a latent thinking process is highly efficient as it avoids the need to verbalize every intermediate step, and is well-suited for reasoning with abstract logic or concepts that are often difficult to convey through natural language. Motivated by this, a recent work explores modeling
abstract mental
In this paper, we use the terms "thinking" and "reasoning" interchangeahly refer to the process by which I.I.M generates intermediate steps latent thoughts toward an answer.

---

Published as a conference paper at ICLR 2026
the thinking process sequence of latent representations (i.e., latent thoughts) and proposes new latent reasoming language model Hugimn-3.58 (Geiping et al., 2025), where each latent thought corresponds to a thinking step. These latent thoughts form latent reasoning chain that enable the model to reason effectively in the latent space and achieve impressive performance across a variety of reasoning tasks.
Despite promising, such latent thinking architecture faces a major challenge: it lacks interpretability and supervision. Unlike verbal thinking, where each intermediate step can be inspected and evalu- ated (Wang et al.. 2024), latent thinking is encoded in internal hidden states that are hard to interpret. This makes it difficult to understand what the model is actually thinking about or verify its cor- rectness. Furthermore, the model is trained to generate these latent thoughts in an unsupervised manner without explicit supervision or reward signals that can indicate what a "good" latent thought is. This raises concerns on whether the model is truly learning to think in the latent space, or simply memorizing the answers using the parameters of latent representations (Wang et al., 2025b).
In this paper, we aim to understand how Huginn-3.5B thinks in the latent space and how external supervision signals can improve its latent thinking process. Specifically, we observe that latent thinking trajectories (i.e., sequences of latent thoughts) that lead to correct versus incorrect answers exhibit distinct patterns. To further investigate this, we train a latent classifier to predict answer conrectness from the latent thinking trajectories, and observe that it can reliably distinguish between correct and incorrect trajectories, even for partial trajectories with just the first few thinking steps.
Building on these insights, we formulate latent thinking improvement as reward optimization prob- lem over latent policies, and propose a Latent Thinking Optimization (LTO) algorithm that uses the latent classifier as Latent Reward Model (LRM) sample latent thinking trajectories with higher estimated likelihood of correctness. LTO is theoretically guaranteed to improve the expected cor- rectness rate and empirically yields significant gains across a range of challenging reasoning tasks.
While we use Huginn-3.5B as a starting point to understand the latent thinking processes, the pro- posed LRM and LTO extend naturally to general LLMs. Although general LLMs do not explicitly incorporate latent thinking, their latent representations across multiple layers can be interpreted as latent chain of thoughts (Wang et al., 2025c). Under this view, LRM and LTO can be readily ap- plied to general LLMs. In our experiments, we demonstrate that the latent thoughts from general LLMs also encode appropriate reward signals and LTO can significantly improve the performance of general LLMs on diverse reasoning tasks using these LRMss Furthermore, we show that LRM ex- hibits strong cross-domain generalization even with a small amount of training data, highlighting its potential as efficient and generalist reward model in the latent space. In contrast to verbal think- ing approaches that scale test-time compute through natural language generation (Guo et al., 2025; Muennighoff et al., 2025), our method demonstrates that reward modeling and scaling test-time thinking with supervision can be performed directly in the latent space, highlighting its potential as a general, efficient, and domain-agnostie approach improving the thinking processes of LLMs.
DEFINITIONS AND NOTATIONS OF REASONING LLMS
Given a question D sampled from the dataset D, a language model IT-) can directly generate an answer by sampling from y ~ π [y x)) For complex questions, however, it is often beneficial to introduce intermediate reasoning steps z represent the model's thinking process. In this case, the model first thinks by sampling from 2 ~ x(2 | x) and then generates the final answer conditioned on 2, that is. y ~ T(y 2). Empirically, this two-stage generation process often improves the answer correctness rate, as generating a allows the model to decompose a complex problem into simpler subproblems that are easier to solve and verify (Wei et al., 2022; Kojima et al., 2022).
a sequence of reasoning steps (Wei et al., 2022) in natural language, that is, = = (e1..... et, ... .r), where specific step in the reasoning process. However,
Verbal Thinking Common reasoning LLMs (Zhang al., 2026) represent
each e, is chunk of text that corresponds generating all the reasoning steps in natural language introduces significant computational overhead, and increases the risk of overthinking where the model generates unnecessarily verbose or logically inconsistent reasoning chains that lead to incorrect answers (Chen et al., 2025; Sui et al., 2025).
Latent Thinking To address the limitations of verbal thinking, inspired by the human cognitive theory, recent work proposes latent reasoning language model Huginn-3.5B (Geiping et al.,
2

---

Published as a conference paper at ICLR 2026
2025) which represents the sequence of reasoning steps sequence of internal hidden states 2 = thi..... h,, hy) (i.e., a latent thinking trajectory). Each h, E Rxd represents a latent reasoning step (i.e., a latent thought), where L is the number of tokens in the output y, d is the hidden dimensionality. The number of thinking steps 7 is set as 32 by default, and can vary according to the computation budget. The initial latent thought hg No0, T² is sampled from random Gaussian noise with the standard deviation a, and a recurrent block is introduced to generate the latent thoughts hi.T recursively conditioned on the question 2. A lightweight decoding module generates the answer y in natural language conditioned on the last latent thought hy. Because both chain-of- ohothht reasoning and recurrent architectures can be conceptualized as finite-state automata (Svete & Cotterell, 2023; Zhang et al., 2024), this approach can be viewed generating the chain of thoughts in the latent space without the need for verbose reasoning. While efficient, difficult to trace the model's logic or provide step-level supervision due to the lack of interpretable patterns in the latent space.
3 DECIPHER HOw HUGINN-3.. 3.5B THINKS IN THE LATENT SPACE
Latent thoughts hidden states and may not have an intrinsic notion of "correctness" themselves. To determine what constitutes a "good" or "bad" latent thought, in this paper, we define the cor- rectness of latent thinking trajectory (latent thinking process) terms of whether the trajectory (thinking process) leads to a correct answer. This definition provides reference point for dis- tinguishing "good" from "bad" latent thoughts and enables us to systematically investigate whether these trajectories exhibit distinct patterns in latent space. It is also consistent with the idea of process reward models (Wang et al., 2024; Lu et al., 2024), where the correctness of intermediate reasoning steps is labeled based their relation with the final answer.
To understand the latent thinking processes, we consider an interesting research question:
Research Question: Do latent thoughts that lead to correct answers exhibit different pat- terns in latent space compared to those leading to incorrect answers?
If differences exist, they would not only provide insights into how Huginn-3.5B encodes abstract concepts during its thinking process, but also provide a foundation for detecting and correcting thinking errors directly in the latent space.
3.1 VISUALIZATION OF LATENT THOUGIITS
To answer this research question, we select two datasets from different domains: SVAMP (Patel et al., 2021) (grade school math) and MBPP (Austin et al., 2021) (python programming). For each problem in these datasets, we randomly sample 100 latent thinking trajectories by sampling from initial latent thought hn with different random seeds, and generate the corresponding answers from these latent thoughts. To compare the difference between latent thoughts that lead to correct and incorrect answers, select those problems that contain both correct and incorrect answers, then visualize and compare their latent thoughts in Figure 1. We have the following observations:
Correct and incorrect latent thoughts exhibit different structures in the latent space. For the same problem, the trajectories of correct and incorrect latent thoughts diverge in both their paths and endpoints, indicating that the model is engaging in different thinking behaviors for correct and incorrect solutions. Interestingly. the distributions and tend to converge toward consistent solution paths. In contrast, incorrect latent thoughts more dispersed in the latent space, suggesting that they lack stable and consistent reasoning pattern.
correct latent thoughts are relatively compact
Both correct and incorrect latent thoughts exhibit different thinking dynamics at different steps. Latent thoughts from early steps show sharp and abrupt changes. This suggests that the model is probably doing active computation and exploratory reasoning, which might involve cogni- tive behaviors such as searching or backtracking (Gandhi et al., 2025) that are helpful for problem solving. Latent thoughts evolve more smoothly in the middle steps, suggesting that the model is probably finetuning its thinking process for iterative refinement (Madaan et al., 2023). In the last few steps, latent thoughts almost converge, indicating that the thinking process is complete and
3

---

Published as a conference paper at ICLR 2026
Correct
Incorrect
Correct
Incorrect
Start
Fod
End
Start
Endy
Start
10
10
-10
10
End
-
0
-10
10
5
5
50
2
(a) Problem #26 from SVAMP.
(b) Problem #97 from MBPP.
Correct
Incorrect
Correct
Incorrect
Fnd
Rod
End
0
Start
End
-5
20
-10
Start
-25
-40
10
-50
10
10
-20
Problem #156 from SVAMP.
(d) Problem #105 from MBPP.
Figure I: Visualization of the distribution of the correct and incorrect latent thoughts projected onto 3D space using PCA for dimension reduction. The arrows along the lines indicate the progression from the current step to the next step of the latent thought. More examples are in Appendix B.
conclusion is reached. These patterns suggest that the latent space effectively captures the progres- sion of the thinking dynamies, with distinet behaviors emerging different steps.
Distinet thinking patterns emerge for different types of problems. Latent thoughts from math problems display different thinking patterns from those observed in programming problems. Within the same dataset, the model also generates latent thoughts with different patterns for different types of problems. Notably, convergence of latent thoughts is faster on math problems, which typically require only two to three arithmetic computations (Patel et al., 2021). By comparison, the latent thoughts take more steps to converge for programming problems, which are more difficult and re- quire longer reasoning steps (Austin et al., 2021). These observations indicate that the model can flexibly adjust its thinking strategy in response different problem types and difficulty levels.
3.2 QUALITATIVE AND QUANTITATIVE ANALYSES ON LATENT THOUGHTS
The case studies in Section 3.1 demonstrate that the model is engaging in different thinking behav- iors for latent thoughts that lead to correct and incorrect answers. To gain deeper understanding of its thinking processes, we evaluate the latent thoughts at different thinking steps using four metrics that measure the quality of latent representations from the perspective of information content (En- tropy, Effective Rank) and geometric structure (Anisotropy, Intrinsic Dimension). These metrics are calculated using all the samples from each dataset.
Entropy (Skean et al., 2025) quantifies how much information content the latent representations carry. A higher entropy indicates the latent representations contain diverse, more informative features, while lower entropy suggests the existence of redundant information.
Effective Rank (Wei et al., 2024) measures how the dimensionality of the latent representation effectively shrinks under strong compression. A higher effective rank implies noisy features, while a lower effective rank indicates better noise reduction and more compact representations. Anisotropy (Razzhigaev al., 2024) measures the non-uniformity of distribution in the latent space. A higher anisotropy suggests that representations are more directed in specific orientations, while lower anisotropy indicates that the representations Intrinsic Dimension (Facco et al., 2017; Cheng et al., 2025) quantifies the minimal number of coordinates required to describe the local geometric structure of the representations without sig-
spread out more evenly.
4

---

Published as a conference paper at ICLR 2026
22.5
Intrinsic Dimension
0.85
Effeetivve Runk
0:40
15.0
Enttopy
20.0
Anisorropy
0.80
0.35
12.5
17.5
0.30
0.75
15.0
0.25
1.5
4
16
20
28
32
8
12
20
24 28
16 20 28 32
8
12
20 24
Latent Thinking Stegs
Latent Thinking Steps
Latent Thinking Steps
Latent Thinking Steps
(a) Distribution representation quality metries across 32 steps of the latent thoughts on SVAMP.
40.0
0.35
Intinsic Dimension
17.5
0.85
Effective Runk
Anisorroy
15.0
Entoopy
35.0
0.30
0.80
12.5
30.0
0.25
10.0
0.75
25.0
0.20
7.5
0
4
16
20
28
32
8
12
20
24
16 20 28
8
12
Latent Thinking Stegs
Latent Thinking Steps
Latent Thitking Steps
Latent Thinking Steps
(b) Distribution representation quality metrics across 32 steps of the latent thoughts on MBPP
Figure 2: Representation quality metrics of the latent thoughts on two datasets. The blue and red distributions represent the distributions for the correct and incorrect trajectory of latent thoughts, respectively. These metrics are calculated using all the samples from each dataset.
nificant information loss. A higher intrinsic dimension indicates a rich, complex latent structure, while a lower intrinsic dimension suggests the representation lies on simpler manifold.
The calculation details of these metrics are in Appendix C. From the visualization of the represen- tation quality metrics across all the thinking steps in Figure 2, we have the following observations:
Correct thinking processes carry richer information with less noise. The entropy of correct latent thoughts is consistently higher than that of incorrect ones, and the effective rank of correct latent thoughts is consistently lower. This suggests that correct thinking processes can preserve richer and more informative features (higher entropy), while reducing noisy components (lower effective rank). These observations are consistent with the view of language modeling as form of compression (Deletang et al., 2024), where effective thinking of LLMs can be understood as process of extracting the key concepts while discarding noisy or redundant information.
Correct thinking processes generate more expressive latent representations with structured and complex geometries. The anisotropy and the intrinsic dimension of correct latent thoughts are consistently higher than those of incorrect ones. This suggests that correct latent thoughts align well along informative directions in the latent space, with a richer and more diverse manifold structure capable of capturing task-relevant features (Valeriani et al., 2023). In contrast, incorrect thoughts collapse into flatter, less organized structures, reflecting collapse of expressiveness and represen- tational capacity (Ansuini et al., 2019; Cheng et al., 2025).
Differences in thinking patterns become more distinguishable later steps. At the beginning of the thinking processes, the representation quality metrics change rapidly and show little difference between correct and incorrect latent thoughts. This probably reflects an exploratory reasoning phase, where the model is actively processing information and has not yet formed a clear solution path. As the thinking progresses, these metrics then stabilize and the difference between correct and incorrect thoughts becomes more salient, suggesting that the thinking process has converged to solution, with the emergence of distinct reasoning patterns between correct and incorrect latent thoughts.
3.3 LATENT THOUGHTS ENCODE SIGNALS PREDICTIVE OF THEIR CORRECTNESS
Empirical results from Section 3.1 and Section 3.2 demonstrate that the latent thoughts contain rich semantic and geometric features that predictive of their correctness. If these signals indeed cap- ture the distinction between correct and incorrect thinking processes, they should be discriminative enough for a model to identify their correctness directly from the latent thoughts. To verify this hy-
5

---

Published as a conference paper at ICLR 2026
1.0
ROC-AUC
ROC-AUC
0.8
0.9
0.7
ROC-AUC
ROC-AUIC
ROC-AUC
ROC-AUC
0.8
Improvee
Platcaus
Impeoves
Platoaus
0
8
12
16
20
24
28
32
12
16
20
24
28
32
Latent Thinking Steps
Latent Thinking Steps
(a) ROC-AUC of the latent classifier on SVAMP.
(b) ROC-AUC of the latent classifier on MBPP.
Figure 3: Performance of the latent classifier trained with varying numbers of latent thinking steps on the SVAMP and MBPP datasets, Additional metrics and results are available in Appendix H.1
pothesis, we follow the widely-used probing technique (Liu et al., 2019; Hewitt & Manning, 2019), and train lightweight sequence classifier to predict the correctness of latent thoughts.
The classifier takes as input the trajectory of latent thoughts from problem, and predicts the proba- bility that the thinking process is correct. For each problem in the training set, we sample 5 different latent thoughts and answers, and train the classifier to predict the correctness of the answer via bi- nary cross-cntrops loss. More training details of the latent classifier are in Appendix E. To study how predictive the latent representations tal settings for each dataset. In the 1-th experiment (1$1$32), the classifier receives the first steps of latent thoughts 11 as input. The maximum of 32 steps is chosen to match the default number of thinking steps in Huginn-3.5DD classification metries such as ROC-AUC and Accuracy.
at different thinking steps, construct 32 experimen-
Evaluation is performed on the test set using standard binary
The results are shown in Figure 3. We observe that this latent classifier achieves strong performance on the test set. although it is trained with only a relatively small amount of data. On SVAMP, achieves an ROC-AUC score close to I.0, while on MBPP achieves an ROC-AUC score of around 0.8. These results indicate that latent thoughts encode rich signals that are highly predictive of their correctness. We also observe that classification performance improves steadily with more thinking steps included, before reaching plateau. This consistent with the observation in Section 3.2 that differences between correct and incorrect thinking patterns become more distinguishable after a few thinking steps. Furthermore, the fact that incorporating latent thoughts from multiple steps improves classification performance suggests that the correctness signal is not solely reflected in specific step of thought, but also in the evolving dynamics of the whole latent thinking trajectory.
The latent reasoning language model displays distinct thinking pat- terns between correct and incorrect thinking processes. and such difference is highly distin- guishable in the latent space, especially after few thinking steps.
Major Observation:
4 LATENT THINKING OPTIMIZATION
Motivated by our observations, we propose Latent Thinking Optimization (LTO), probabilistic optimization approach designed to improve the latent thinking processes by selectively sampling trajectories that exhibit correct patterns. LTO formulates this as an optimization problem over latent policies, and introduces probabilistic algorithm solve the optimization problem. While LTO uses Huginn-3.5B as starting point, we further demonstrate that LTO can also be applied to general LLMs, and achieves strong transferability across diverse domains with high efficiency. These results highlight the potential of LTO as an effective and scalable approach for optimizing LLM thinking by performing reward modeling and thinking correction directly in the latent space.
Objective of LTO To formalize this, we introduce a binary variable O that indicates whether the latent thinking trajectory is correct. Our goal to find an optimal latent thinking policy x² <=|xx such that it maximizes the expectation of generating a correct latent thinking trajectory 2:
(z x) arg m E (=)p()) = 1 |xx,2)
(1)
where p(O = 1 1/x, 2) is the probability of a latent thinking trajectory 2 being correct for a question x. Since the classifier introduced in Section 3.3 is trained to predict the correctness of latent thoughts,

---

Published as a conference paper at ICLR 2026
Algorithm 1 Latent Thinking Optimization
1: Input: question z, the original policy [ LRM r(z, z), sampling budget N, the number of required
samples M, weight & > 0 to control the strength of KL-regularization
2: Output: sampled set of latent thinking trajectories C
3: C - 0. Tmmx f-0 0
Initialize the output set and the maximum reward.
4: for = to do
5: 2/ - ~ (z(()
Di Sample the i-th latent thinking trajectory from the original policy. > Update the maximum reward.
6: Tiux <- max((rrma (2),
7:
while [C] < M do
>> Repeat until M samples are collected.
8:
2; ~ (2;
~ Uniform(O, 1)
9:
& f- exp((r(2., xz -mmx)/3)
D. Calculate the acceptance probability di. Reject the sample with probability - o.. Otherwise, accept the sample with probability di.
10:
ifu 2 o, then continue
11:
C' {
12:
return C
it can be used as Latent Reward Model (LRM) r(x,2) 2) estimate the probability of p(O 1 lr, 2). To ensure that the optimized policy does not deviate significantly from the original policy (z)() (the latent policy distribution of the model before LTO optimization), we introduce KL- regularization term (Jaques et al., 2017; Ziegler et al.. 2019; Rafailov et al., 2023) with the weight penalize the difference between the optimized policy x* (z|zr) and the original policy (z)r) The optimization objective then becomes:
' (z|z) = arg max (=(x( Exon(s)a) [r(æ, z)] 3Dx (
(2)
The Necessity of KI. regularization The inclusion of KL regularization with the weight B is well motivated from the Kl-regularized policy optimization theory. Without KL regularization, the optimized policy might collapse into degenerated policy that significantly deviates from the base policy. Moreover, if we remove 3 (set 3-+),, LTO will reduce to best-of-N sampling in the latent space, i.e., we sample N latent trajectories, score them with the LRM, and pick the highest- scoring one. This strategy only works well if the LRM is nearly perfect with almost 1.0 classification accuracy for correct and incorrect latent trajectories. In practice the LRM is learned and not perfect. Without any regularization, LTO may exploit the errors of the LRM and select suboptimal latent thoughts. By including a KL penalty, we constrain the optimized policy so that does not drift too far from the base policy, thereby preserving the diversity of sampled latent thoughts and mitigating overfitting to reward model noise.
Probabilistic Sampling Directly optimizing over the latent policy π (2(2²) is often difficult. In- stead, we approximate π <2)(x;) using a finite set of N sampled latent thinking trajectories { In this case, we show that Equation has a closed-form solution:
Theorem 1. Given a sampled set of {2))
approximate the policy distribution x² (2|xx, for Tm<(=]=)) exp((r(z.z.))
each i, the solution to Equation is TH (z(l)) = Σn Fin(2, exp (r(,,))))
We provide the proof in Appendix F.1. Here we use the subscript notation n, to indicate that the policy is derived from the reward function r(x,2), For simplicity, we omit the superseript *, but n, still represents the optimized policy.
While Theorem gives a closed- form solution for our optimization problem, sampling directly from the distribution (=|(=]):)) is still difficult, since it is hard to accurately estimate the probability of each [T)((2)22) (.). To address this issue, inspired by acceptance-rejcction sampling algorithms (Flury, 1990; Grover et al., 2018; Liu et al., 2024), we propose Algorithm to draw samples a without explicitly calculating the value of क, (2|2). It draws N candidate thinking trajectories { from the original policy Tee((2())) (r). Each candidate sample 2; will only be accepted with probability di, where Φ; is designed such that the latent thinking trajectories with higher reward are more likely to be accepted.
This procedure is repeated until M valid samples are collected. Theoretically, the of samples drawn in Algorithm is guaranteed to follow the distribution ,, (2|z):
Theorem 2. In Algorithm for each i, the probability of z being drawn and accepted is Pr(z,|u < &,, x) = (z)
7

---

Published as a conference paper at ICLR 2026
We provide the proof in Appendix F.2. This theorem shows that each accepted sample 2; is drawn with probability TH (2(22)). Since each sampling process is independent, repeating the procedure produces i..d. samples from exactly the same distribution (2(22)
We summarize the workflow of LTO as follows: 1) Collect latent thinking trajectories from the training data to train LRM and 2) sample multiple latent thinking trajectories and accept only high- rewarded ones that are more likely to be correct via Algorithm 1. The samples drawn from Algo- rithm 1 is theoretically guaranteed to follow the distribution in Theorem 1, which is the solution to the objective of latent thinking optimization problem as defined in Equation 2. Note that although LTO relies on probabilistic sampling instead of parameter update improve the latent policy, this does not diminish its nature as solving discrete optimization problem over the latent policy.
Application to General LLMS While our main focus is to improve the thinking process of the latent reasoning language model, the proposed LTO algorithm can also be applied to general LLMs, such as OLMo (Groeneveld et al., 2024), Llama (Touvron et al., 2023b) and Mistral (Jiang et al., 2023). Although general LLMss do not explicitly introduce latent thinking process, their latent representations across multiple layers can be interpreted as latent chain of thoughts (Wang et al., 2025c). Under this view, LRM and LTOO can be readily applied general L.Mss. In Appendix 1.1, we demonstrate that LRMs trained with the latent representations of general LLMs can also achieve strong classification performance, indicating that the latent thoughts from general LLMs also encode appropriate reward signals. In Section 6.2, we demonstrate that LTO can significantly improve the performance of general LLMs on diverse reasoning tasks using these LRMs.
Generalist Reward Modeling Natural language- based process reward models are often limited to narrow domains such as math (Wang et al., 2024; Lu et al., 2024) due to their reliance on domain- specific thinking formats and structures (Zeng al., 2025). By comparison, reward modeling in the latent space has the potential for better generalizability, since latent thoughts share unified form of latent representations and may be more transferable across diverse domains. In Section 6.3, we demonstrate that LRM achieves strong transferability across diverse domains and shows potential for building a generalist reward model in the latent space.
High Efficiency LRM only requires a modest number of training samples (Section G.2), and LTO is highly efficient at both the training and inference stage (Section H.7). This highlights the potential of LTO an efficient alternative that performs reward modeling in the latent space, in contrast to reward models that require substantial finetuning and inference costs (Wang al., 2024; Lu et al., 2024; Lightman al., 2024).
natural language-base
Guaranteed Performance Improvement While LTO does not explicitly modify the latent policy, we theoretically demonstrate in Appendix F.3 that improving the accuracy of the LRM directly translates into higher expected correctness rate. Thus, LTO enables latent thinking improvement simply by scaling and refining the LRM (e.g., with more training data) which computationally lightweight, rather than costly finetuning the base model to improve its latent policy.
5 EXPERIMENTAL SETTINGS
Datasets To study whether our approach can improve the thinking processes of Huginn-3.5B across diverse tasks with different thinking patterns, we evaluate its performance on five datasets from three domains: (1) GSM8K (Cobbe et al., 2021), SVAMP (Patel et al., 2021), GSM- Symbolic (Mirzadeh et al., 2025) for the Math domain, (2) Commonsense(A (Talmor et al., 2019) for the Commonsense Reasoning domain; and (3) MBPP (Austin et al., 2021) for the Code Gener- ation domain. The details of the datasets are in Appendix G.I.
Baselines and Implementation Details Since Huginn-3.50 generates the thinking process in the form of latent representations, many thinking correction methods with trained process verifier in the natural language space (Lu et al., 2024; Wang et al., 2024) may not be applicable to the la- tent space. Therefore, compare our approach against two types of reasoning correction and improvement methods applicable to Huginn-3.5B (1) Answer Correction. These methods cor- rect and improve the answers without requiring access the thinking process. We include three
8

---

Published as a conference paper at ICLR 2026
Table Comparison of the answer correctness rate of Huginn-3.5B using different correction meth- ods. The best performance in each column is in bold, and the performance of the best baseline in each column underlined. * indicates statistically significant improvement with p < 0.05.
Method
GSM8K GSM-Symbolic SVAMP Commonsense MBPP
Base Model
0.326
0.265
0.517
0.500
0.278
Majority Voting
0.333
0.269
0.511
0,504
0.288
Self-Correction w. Confidence Score
0.342
0.281
0,524
0,507
0,288
Self-Correction w. Verbal Evaluation
0.262
0.193
0,518
0.505
0.226
Latent Thinking Correction w. CoE-R
0.330
0.259
0.510
0,504
0.276
Latent Thinking Correction CoE-C
0.324
0.256
0,516
0,507
0,280
Weighted Majority Voting w. LRM
0.375
0.301
0.537"
0.509
0.295
Latent Thinking Optimization LRM
0.385
0.305'
0.538
0.517
0.299
representative approaches: Majority Voting (Wang et al., 2023), Self-Correction with Confidence Score (Ren et al., 2023b), Self-Correction with Verbal Evaluation (Manakul et al., 2023). (2) Latent Thinking Correction. While explicit correction of latent thoughts remains underexplored, recent work (Wang et al., 2025c) introduces two heuristic metrics (CoE-R and CoE-C) to evaluate the of latent thoughts. We adopt these scores as the correction signals, yielding two additional baselines: Latent Thinking Correction with CoE-R, and Latent Thinking Correction with CoE-C. Furthermore, we evaluate a simplified version of our approach, Weighted Majority Voting with LRM, which use the LRM reward as weighting signal. We also report the performance of the base model (directly generating latent thinking trajectory and the corresponding answer without any correction) to quantify the performance improvement achieved by our approach and competing baselines. Implementation details of baselines and our method are Section G.2.
correctness
EXPERIMENTAL RESULTS
6.1
OVERALL PERFORMANCE COMPARISON
Table presents the experimental results on all the datasets. We have the following observations:
LTO significantly improves the latent thinking processes. Across all datasets, LTO consistently outperforms both the base model and the best baseline for thinking correction. Leveraging a well- trained LRM, LTO can effectively detect and correct erroneous thinking patterns the latent space via probabilistic algorithm, bringing robust and consistent improvements to the latent thinking processes. By comparison, other thinking correction methods show suboptimal performance or worse performance than the base model, indicating that these techniques originally developed for verbal thinking are not suitable for identifying errors for latent thinking.
LRM is highly effective in detecting incorrect latent thinking patterns. Both weighted majority voting and Latent Thinking Optimization with LRR achieve consistent improvements over the base model. Notably, standard majority voting yields little benefit; however, when the LRM reward used as a weighting signal, weighted majority voting achieves substantial gains. This demonstrates that the LRM reward provides reliable estimation of the correctness of latent thoughts and serves as an effective correction signal for thinking correction algorithms in the latent space.
6.2 APPLICATION TO GENERAL LLMS
While we mainly focus on improving the thinking process of Huginn-3.5B. we also demonstrate that LTO can be applied to general LLMs, such as OLMo (Groeneveld et al., 2024), Llama (Touvron et al., 2023b) and Mistral (Jiang et al., 2023). To evaluate the performance of LTO on general LLMs, use the same LRM and LTO configurations as described in Section 5, and train LRMs using the latent representations from general LLMs. From the experimental results in Table 2, we can see that LTO achieves substantial performance gains across diverse datasets, with improvement of up to 103% over the base model, even with a modest sampling budget (N = 20). These results highlight the potential of LTO general method for improving the latent thinking processes of L.MMs.

---

Published as a conference paper at ICLR 2026
Table 2: Performance of LTO on general LLMs. The best-performing method for each model is in
bold. *
indicates the improvement over the best runner-up is statistically significant with p < 0.05.
Model
Method
GSM8K GSM-Symbolic SVAMP CommonsenseQA MBPP
Base Model
0.124
0.078
0.297
0,464
0.244
OLMo-7B
Majority Voting
0.209
0.149
0.469
0.521
0.240
Latent Thinking Optimization
0.252
0.154
0.552
0.602
0.308
Base Model
0.223
0.204
0.473
0.399
0.189
Llama-2-7B
Majority Voting
0.275
0.302
0.598
0.493
0.193
Latent Thinking Optimization
0.389
0.316
0.776
0.606
0.237
Base Model
0.306
0.273
0.521
0.398
0.247
Llama-2-13B
Majority Voting
0.417
0.379
0.612
0.501
0.263
Latent Thinking Optimization
0.534"
0.442"
0.791
0.650
0.322
Base Model
0.368
0.278
0.548
0.671
0.315
Mistral-7B
Majority Voting
0.529
0.413
0.624
0.687
0.334
Latent Thinking Optimization
0.565
0.462"
0.771
0.708
0.388
6.3 GENERALIST REWARD MODELING
Lateat Reward Roward Model
None
0.326
0.266
0.517
GSM8K
0.305
0.497
To evaluate whether LRMs trained on dataset can be applied to another dataset, we first examine the cross-dataset transferabil- ity of LRMs by evaluating the performance of LTO when paired with an LRM trained on different datasets. We extend the study by training general LRM on the combined training data from all datasets and evaluating the performance of LTO with the general LRM. From the results in Figure 4, we can see that LRMs demon- strate transferability across different domains, since LTO can im- prove the performance of the base model when paired with LRM trained on different dataset. The improvement is consistent even the gap between domains large. For example, although Com- monsenseQA primarily involves commonsense reasoning, an LRM trained on Commonsense@AA still improves performance on math- focused datasets such GSM8K, GSM-Symbolic, and SVAMP. This suggests that LRMs may capture some fundamental aspects of latent thinking patterns shareable across different domains. Fur- thermore, the performance of LTO using the general LRM is on par with the performance of LTO using domain-specific LRMs.. These results suggest that latent ward modeling can generalize across domains. While our empirical results have not yet achieved full transferability across all possible tasks, we believe that they indicate promising cross-domain potential for building a generalist reward model in the latent space for future work.
SVAMP
CQA
0.110
0.117
MBPP
0.532
0.382
0.537
GSMSK
GSM-S
SVAMP
COA
MBVP
Dataset
Figure 4: Performance of
LTO using different LRMs. "GSM-S" refers to the GSM- Symbolic dataset. "CQA" refers to the Common- senseQA dataset. "None" refers to the performance of the base model without LTO.
7 CONCLUSION
In this paper, observe that the latent thoughts of Huginn-3.55 that lead correct versus incorrect answers display distinct thinking patterns, and such difference is highly distinguishable by a latent classifier. Building on these insights, we formulate latent thinking improvement as a reward opti- mization problem over latent policies, and propose an LTO algorithm that uses the latent classifier as an LRM to optimize the latent thinking processes. Extensive experiments across diverse reason- ing tasks demonstrate LTO can significantly improve the latent thinking processes of Huginn-3.55 Furthermore, we show LRM can generalize across different domains, and LTO can be seamlessly ap- plied to general L.MMs to improve their thinking processes. In contrast to verbal thinking approaches that scale test-time compute through natural language generation (Guo et al., 2025; Muennighoff et al., 2025), our method demonstrates that reward modeling and scaling test-time thinking with supervision can be performed directly in the latent space, offering general (Section 6.2), efficient (Appendix H.7), and domain-agnostic (Section 6.3) approach to improving the thinking processes of LLMs. We discuss the related works, limitations, broader impact and reproducibility of our research in Appendix A, Appendix I, Appendix and Appendix L, respectively.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Alessio Ansuini, Alessandro Laio, Jakob H Macke, and Davide Zoccolan. Intrinsic dimension of data representations in deep neural networks. Advances Neural Information Processing Sys- tems, 32, 2019.
Jacob Austin, Augustus Odena, Maxwell I. Nye, Maarten Bosma, Henryk Michalewski, David Do- han, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. Program synthesis with large language models. arXiv preprint arXiv:2 07732, 2021.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang. Xiaodong Deng, Yang Fan, Wenbin Ge, Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu. Qwen technical report. arXiv preprini arXiv:2309. 16609, 2023.
Peter I. Bartlett and Shahar Mendelson. Rademacher and gaussian complexities Risk bounds and structural results. Journal of Machine Learning Research, 3(Nov):463-482, 2002.
Peter I. Bartlett, Dylan J Foster, and Matus Telgarsky. Spectrally-normalizeed margin bounds for neural networks. Advances in Neural Information Processing Systems, 30, 2017.
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang. Zhaopeng Tu, Haitao Mi, and Dong Yu. Do NOT think that much for 2+3=? on the overthinking of long reasoning models. In Forty-second International Conference Machine Learning, 2025.
Emily Cheng, Diego Doimo, Corentin Kervadec, Iuri Macocco, Lei Yu, Alessandro Laio, and Marco Baroni. Emergence of high-dimensional abstraction phase in language transformers. In The Thirteenth International Conference on Learning Representations, 2025.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint 100.v:2110.14168 2021.
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley-Interscience, 2006.
Gregoire Deletang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, Marcus Hut- ter, and Joel Veness. Language modeling compression. The Twelfth International Conference Learning Representations, 2024.
Elena Facco, Maria d' Errico, Alex Rodriguez, and Alessandro Laio. Estimating the intrinsic dimen- sion of datasets by minimal neighborhood information. Scientific reports, 7:12140, 2017.
Shengyu Feng, Xiang Kong, Shuang Ma, Aonan Zhang, Dong Yin, Chong Wang, Ruoming Pang. and Yiming Yang. Step-by-step reasoning for math problems via twisted sequential monte carlo. In The Thirteenth International Conference on Learning Representations, 2025.
Bernard D Flury. ceceptance-rejection sampling made easy. Siam Review, 32(3):474 1990.
Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah Goodman. Cogni- tive behaviors that enable self- ceff-improving reasoners, or, four habits of highly effective STars. In Second Conference on Language Modeling, 2025.
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. PAL: Program-aine aided language models. In Proceedings of the 40th International Conference Machine Learning, volume 202, pp. 10764-10799, 2023.
11

---

Published as a conference paper at ICLR 2026
Jonas Geiping, Sean Michael McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time com- pute with latent reasoning: A recurrent depth approach. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.
Gemini Team. Gemini: a family of highly capable multimodal models. arXiv:2312.11800 2023.
arXiv preprint
Aldo Glielmo, luri Macocco, Diego Doimo, Matteo Carli, Claudio Zeni, Romina Wild, Maria d' d'Errico, Alex Rodriguez, and Alessandro Laio. Dadapy: Distance-based analysis of data- manifolds in python. Patterns, pp. 100589, 2022.
Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. Think before you speak: Training language models with pause tokens. In The Twelfth International Conference Learning Representations, 2024.
Dirk Groeneveld, Iz Beltagy, Evan Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang. Shane Arora, David Atkinson, Russell Au- thur, Khyathi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff. Aakanksha Naik, Crys- tal Nam, Matthew Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, William Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zeulleooer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah Smith, and Hannaneh Hajishirzi. OLMo: Accelerating the science of language models. In Pro- ceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15789-15809 2024.
Aditya Grover, Ramki Gummadi, Miguel Lazaro-Gredilla, Dale Schuurmans, and Stefano Ermon. Variational rejection sampling. In International Conference on Artificial Intelligence and Statis- tics, pp. 823- 002-832,222 2018
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang. Xingkai Yu. Yu Wu, Z. F. Wu, Zhibin Gou, Zhi- hong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, Hao Zhang, Hanwei Xu, Honghui Ding, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jingchang Chen, Jingyang Yuan, Jinhao Tu, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaichao You, Kaige Gao. Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingxu Zhou, Meng Li, Miaojun Wang. Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang. R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Tao Yun, Tian Pei, Tianyu Sun, Tao Wang, Wangding Zeng, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li. Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang. Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei. Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang. Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunx- ian Ma. Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu. Zilin Li. Ziwei Xie, Ziyang Song. Zizheng Pan, Zhen Huang. Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. DeepSeek-RR R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, (55(801))6633-25
2025.
12

---

Published as a conference paper at ICLR 2026
Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. Reasoning with language model is planning with world model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 8154-81777 8173. 2023.
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason E Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. In Second Conference on Language Modeling, 2025.
Dan Hendryeks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. In Thirty-fiith Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2021.
John Hewitt and Christopher D Manning. A structural probe for finding syntax in word representa- tions. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Pa- pers), pp. 4129 4138, 2019.
Natasha Jaques, Shixiang Gu, Dzmitry Bahdanau, José Miguel Hernández-Lobatto Richard E. Turner, and Douglas Eck. Sequence tutor: Conservative fine-tuning of sequence generation mod- els with KL-control. In Proceedings of the 34th International Conference on Machine Learning, volume 70, pp. 1645-1654, 2017.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Ann Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023.
Daniel Kahneman. Thinking, fast and slow. Farrar, Straus and Giroux, 2011.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In The Third International Conference Learning Representations, 2015.
Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems, volume 35, pp. 22199-2222 22213,, 2022.
Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su. John Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, Lei M Zhang, Kay McKinney, Disha Shrivastava, Cosmin Paduraru, George Tucker, Doina Precup, Feryal Behbahani, and Aleksandra Faust. Train- ing language models to self-correct via reinforcement learning. In The Thirteenth International Conference on Learning Representations, 2025.
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d' Autumee Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Push- meet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals. Competition--evel code 2022.
generation with alphacode. Science,
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2024.
Nelson Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. Linguistic knowledge and transferability of contextual representations. In Proceedings of the 2019 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pp. 1073-1094, 2019.
Tianqi Liu, Yao Zhao, Rishabh Joshi, Misha Khalman, Mohammad Saleh, Peter J Liu, and Jialu Liu. Statistical rejection sampling improves preference optimization, In The Twelfth International Conference on Learning Representations, 2024.
13

---

Published as a conference paper at ICLR 2026
Llama Team. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. 2023.
Jianqiao Lu, Zhiyang Dou, Hongru Wang. Zeyu Cao, Jianbo Dai, Yingjia Wan, Yunlong Feng, and Zhijiang Guo. AutoPSV: Automated process-supervised verifier. In Advances in Neural Information Processing Systems, volume 37, pp. 19935-79962) 2024.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uni Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, volume 36, pp. 46534-46594, 2023.
Potsawee Manakul, Adian Liusie, and Mark Gales. SelfCheckGPT: Zero-resource black-box hallu- cination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods Natural Language Processing, pp. 9004-9017, 20233
Shen-Yur Miao, Chao-Chun Liang, and Yih Su. A diverse corpus for evaluating and developing english math word problem solvers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 975-984, 2020.
Seyed Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. GSM-Symbolic: Understanding the limitations of mathematical reasoning in large language models. The Thirteenth International Conference on Learning Representations, 2025.
Karyna Mishchanchuk, Gabrielle Gregoriou, Albert Qü, Alizée Kastler, Quentin JM Huys, Linda Wilbrecht, and Andrew F MacAskill. Hidden state inference requires abstract contextual repre- sentations in the ventral hippocampus. Science, 386(6724)96662 932, 2024.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candes, and Tatsunori Hashimoto. sl: Simple test- time scaling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 20286-20332, 2025.
Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. CodeGen: An open large language model for code with multi-turn program synthesis. In The Eleventh International Conference on Learning Representations, 2023.
OpenAI. GPT-4 technical report. arXiv preprint arXiv:2303.08774. 2023.
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2080-2094, 2021.
R Quian Quiroga, Leila Reddy, Gabriel Kreiman, Christof Koch, and Itzhak Fried. Invariant visual representation by single neurons the human brain. Nature, 13577045):1002111000 2005.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Ad- vances in Neural Information Processing Systems, volume 36, pp. 53728-53741, 2023.
Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova, Ivan Oseledets, Denis Dimitrov, and Andrey Kuznetsov. The shape of learning: Anisotropy and intrinsic dimensions in transformer- based models. In Findings of the Association for Computational Linguistics: EACL 2024, pp. 868 874,874222 2024.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. GPQA: A graduate-level google-proof Q&A benchmark. In First Conference Language Modeling, 2024.
Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Saleh, Balaji Lakshminarayanan, and Peter Liu. dittt---istritttion detection and selective generation for conditional language models. In The Eleventh International Conference on Learning Representations, 2023a.
14

---

Published as a conference paper at ICLR 2026
Jie Ren, Yao Zhao, Tu Vu, Peter J Liu, and Balaji Lakshminarayanan. Self-evaluation improves selective generation in large language models. arXiv preprint arXiv:2312.09300 2023b.
Nikhil Sardana, Jacob Portes, Sasha Doubov, and Jonathan Frankle. Beyond chinchilla-optimal: Accounting for inference in language model scaling laws. In Proceedings of the 4 International Conference on Machine Learning, volume 235, pp. 43445-43460, 2024.
Amrith Setlur, Nived Rajaraman, Sergey Levine, and Aviral Kumar. Scaling test-time compute
without verification RL is suboptimal. In Forty-second International Conference Machine Learning, 2025.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflex- ion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, pp. 8634-8652, 2023.
Oscar Skean, Md Rifat Arefin, Dan Zhao, Niket Nikul Patel, Jalal Naghiyev, Yann LeCun, and Ravid Shwartz-Ziv. Layer by layer: Uncovering hidden representations in language models. In Forty-second International Conference on Machine Learning. 2025.
Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time com- pute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth International Conference on Learning Representations, 2025.
Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, Hanjie Chen, and Xia Hu. Stop overthinking: A survey on efficient reasoning for large language models. Transactions on Machine Learning Research, 2025.
Anej Svete and Ryan Cotterell. Recurrent neural language models as probabilistic finite-state au- tomata. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 8069-8086, 2023.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: ques- tion answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Con- ference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pp. 4149-4158, 2019.
Ian Tenney, Dipanjan Das, and Ellie Pavlick. BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp- 4593 4601, 2019.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. arXiv preprint ar0iv:2302. 13971, 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami. Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux. Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint ar3iv: 00200009088... 2023b.
Lucrezia Valeriani, Diego Doimo, Francesca Cuturello, Alessandro Laio, Alessio Ansuini, and Al- berto Cazzaniga. The geometry of hidden representations of large transformer models. Advances in Neural Information Processing Systems, 36:51 234-51252, 2023.
15

---

Published as a conference paper at ICLR 2026
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems, volume 30, 2017.
Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-Shepherd: Verify and reinforce LLMs step-by-step without human annotations. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 9029-9469999 9439, 2024.
Xinran Wang, Qi Le, Ammar Ahmed, Enmao Diao, Yi Zhou, Nathalie Baracaldo, Jie Ding, and Ali Anwar. MAP: Multi haatt-humnn-valll alignment palette. In The Thirteenth International Conference on Learning Representations. 2025a.
Xinyi Wang, Antonis Antoniades, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, and William Yang Wang. Generalization v.s. memorization: Tracing language models' capabili- ties back to pretraining data. In The Thirteenth International Conference on Learning Represen- fations, 2025b.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representatioss, 2023.
Yiming Wang. Pei Zhang, Baosong Yang. Derek F. Wong, and Rui Wang. Latent space chain-of embedding enables output-free LLM self-evaluation. In The Thirteenth International Conference on Learning Representations, 2025c.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pp.
2022.
Lai Wei, Zhiquan Tan, Chenghai Li, Jindong Wang, and Weiran Huang. Diff-eRank: A novel rank- based metric for evaluating large language models. In Advances in Neural Information Processing Systems, volume 37, pp. 39501-39521, 2024.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou Fei Huang, Feng Hu, Hao Ge. Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jian Yang, Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang. Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report. arXiv preprini arXiv:2505.099 093888 2025.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Ad- vances in Neural Information Processing Systems, volume 36, pp. 11809-11822, 2023.
Eric Zelikman, Georges Raif Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah Goodman. Quiet-STar: Language models can teach themselves to think before speaking. In First Conference on Language Modeling, 2024.
Thomas Zeng, Shuibai Zhang, Shutong Wu, Christian Classen, Daewon Chae, Ethan Ewer, Minjae Lee, Heeju Kim, Wonjun Kang, Jackson Kunde, Ying Fan, Jungtaek Kim, Hyung Il Koo, Kannan Ramchandran, Dimitris Papailiopoulos. and Kangwook Lee. VersaPRM: Multi-domain process reward model via synthetic reasoning data. In Proceedings of the 42nd International Conference on Machine Learning. volume 267, pp. 74197-74239, 2025.
Duzhen Zhang, Zhong-Zhhi Zhi Li, Ming- Laing Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Xiuyi Chen, Yingying Zhang, Fei Yin, Jiahua Dong, Zhijiang Guo, Le Song, and Cheng- Lin Liu. From system to system 2: A survey of reasoning large language models. IEEE Transactions Pattern Analysis and Machine Intelligence, 48(3):333-3333 2026.
16

---

Published as a conference paper at ICLR 2026
Xiang Zhang, Muhammad Abdul- Madul-Mageed, and Laks VS Lakshmanan. Autoregresssivetchan of thought-recurrent Recurrence's s role in language models' computability and revisit of recurrent transformer. arXiv preprint wwtp:::::20000922999. 2024.
Yizhe Zhang, Jiatao Gu, Zhuofeng Wu, Shuangfei Zhai, Joshua Susskind, and Navdeep Jaitly. Plan- ner: Generating diversified paragraph via latent language diffusion model. In Advances in Neural Information Processing Systems, volume 36, pp. 80178-80190 2023.
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv: 08593,, 2019.
17

---

Published as a conference paper at ICLR 2026
Appendices
A
RELATED WORKS
A.1 VERBAL AND LATENT THINKING FOR LLMS
Human cognition often involves thinking through intermediate steps rather than directly answering the question (Kahneman, 2011; Zelikman et al., 2024). Inspired by this, growing line of research focuses on guiding LLMs to generate intermediate reasoning steps as the thinking process before generating the answers. Most approaches represent the thinking process in natural language, such as step-by-step chain-of-thought prompting (Wei et al., 2022; Kojima et al., 2022; Wang et al., 2023), self-correction via iterative feedback (Shinn al., 2023; Madaan et al., 2023; Kumar et al., 2025), or building reasoning trees to explore diverse solutions (Yao et al., 2023; Hao et al., 2023). While effective, such verbal thinking incurs significant computational cost, and is also prone the over- thinking issue (Chen et al., 2025; Sui et al., 2025). In contrast, latent thinking offers an alternative approach, where the model represents its thinking process as compact latent representations rather than natural language. This approach is more computationally efficient and better suited for rea- soning with abstract concepts that are difficult to verbalize. Among various approaches for latent thinking (Zhang et al., 2023; Goyal et al., 2024; Hao et al., 2025; Geiping et al., 2025), represen- tative one is the latent reasoning language model (Geiping et al., 2025), which is pretrained from scratch as new language model architecture. It introduces recurrent unit to generate sequences of latent thoughts and supports test-time scaling with flexible computation budgets. Despite promis- ing, the lack of interpretability in the latent representations makes it difficult understand what the model is actually thinking about or to verify the correctness of its thinking process. In this paper, we aim bridge this gap by investigating how the latent reasoning language model thinks in the latent space and how external supervision can guide and improve the latent thinking processes.
A.2 SCALING UP TEST- TIME COMPUTE
As LLMs are tasked with increasingly difficult problems, directly prompting the LLM to generate the answers is often insufficient. To address this, recent works emphasize scaling up test-time com- pute as an effective approach to enhance the problem-son sooleig capability of LLMs (Sardana et al., 2024: Snell et al., 2025). Existing approaches scale up test time compute from different perspec- tives, such as sequential scaling with revisions to refine the answer (Shinn et al., 2023; Madaan al., 2023; Muennighoft et al., 2025), parallel scaling by generating multiple answers search for diverse solutions (Wang et al., 2023; Yao et al., 2023; Hao et al., 2023), or scaling with a verifier or reward model ensure the correctness of solutions (Wang al., 2024; Lu et al., 2024; Feng al., 2025; Setlur et al., 2025). However, most of these approaches focus on scaling up test-time compute using natural language, and how to scale up test-time compute in the latent space (Geiping et al., 2025) remains underexplored. In this paper, we introduce a probabilistic sampling approach with latent reward model that can improve the latent thinking processes and enable efficient and effective test-time scaling in the latent space.
B ADDITIONAL VISUALIZATION OF LATENT THOUGHTS
Additional examples on the visualization of correct and incorrect latent thoughts are in Figure A1.
C CALCULATION OF REPRESENTATION QUALITY METRICS
In this section, we provide the details on how to calculate the representation quality metrics. For question x, Huginn-3.5B generates steps of latent thoughts h recursively. Each latent thought h, E _4 is internal hidden state generated by Huginn-3.5B. where L is the number of tokens, is the hidden dimensionality. The representation quality metrics are calculated over the latent thoughts across all the thinking steps to capture the evolving dynamics of latent thinking processes. Note that in the Huginn-3.5D architecture, the same RMSNorm module with the same rescaling weight is applied to each step of latent thought. Therefore, all the latent thoughts from different
18

---

Published as a conference paper at ICLR 2026
Correct
Incorrect
Correct
Incorrect
Pnn
20
Start
20
Start
0
End
-10
* Start
End
-
Bod
-20
-10
-20
10
10
40
20
20
20
20
(a) Problem #171 from SVAMP.
(h) Problem #70 from MBPP.
Correct
Incorrect
Correct
Incorrect
Start
Start
End
20
5
Start
End
End
20
-20
20
Start
10
0
-2
-2
20
-00
0
10
-40
20
(c) Problem #257 from SVAMP.
(d) Problem #102 from MBPP.
Correct
Incorrect
Correct
Incorrect
End
10
End
Start
10
End
Start
-EEd
Start
-10
20
0
Start
20
20
25
-25
0
20
20
-50
20
-50
10
0
20
(e) Problem #276 from SVAMP.
(f) Problem #159 from MBPP
Correct
Incorrect
Correct
Incorrect
Start
10
Fndwa
Start
Ebd
Start
20
Find
Start
0
bod
-10
-20
10
0
0
-20
10
-20
25
-20
(g) Problem #365 from SVAMP.
(h) Problem #191 from MBPP.
Correct
Incorrect
Correct
Incorrect
Fnd
Start
Start
Start
0
-20
-20
-20
-10
-0
0
-20
-25
-40
20
-50
20
Problem #624 from SVAMP.
(j) Problem #215 from MBPP.
Figure A1: Visualization of the distribution of the correct and incorrect latent thoughts projected onto 3D space demonstrate that correct and incorrect latent thoughts exhibit different patterns in the latent space. Note that this phenomenon is not limited to these cases. On the SVAMP dataset, we identify 1,654 problems with both correct and incorrect answers, and on the MBPP dataset, we identify 179 problems with both correct and incorrect answers. In all of these cases, the latent thoughts leading to correct versus incorrect answers show different patterns in the latent space.
19

---

Published as a conference paper at ICLR 2026
steps already normalized to the same scale before caleulate the representation quality met- rics, making these latent representations scale-invariant. For each latent thought h, (1StTTT, we calculate the Entropy, Effective Rank, Anisotropy and Intrinsic Dimension of h, as follows.
C.I ENTROPY
Entropy (Skean et al., 2025) quantifies how much information content the latent representations carry. A higher entropy indicates richer spread of information across many dimensions, reflecting diverse, less redundant features and better information preservation. Conversely, a lower entropy re- flects concentrated eigenvalue spectra, suggesting that the latent representations may contain redun- dant information. We compute the entropy over the Gram matrix K h,h," using the matrix-based Rényi entropy. For any a this is defined as:
Entropy (h.)
log Σ
(:K (x(
where A₂ (K) denotes the i-th eigenvalue of the Gram matrix K, rank(K) denotes its rank. While we can vary a to get different formulations of matrix entropy, we follow the approach of Skean al. (2025) and choose a-11, which is equivalent the standard von Neumann entropy.
C.2 EFFECTIVE RANK
Effective Rank (Wei et al., 2024) measures how effectively the model extracts key concepts and reduces noisy features in its latent representations. A higher effective rank implies that the repre- sentations contain noisy features, while a lower effective rank indicates better noise reduction. It is defined as follows:
EffoctiveRank(())) exp
Σ
log oi
(4)
Σx
2i=1
02
where K = min(L,d)) and 01 " the singular values of the matrix h,-
C.3 ANISOTROPY
Anisotropy (Razzhigaev et al., 2024) measures the non-uniformity of distribution in the latent space. A higher anisotropy suggests that representations are more directed specific orientations, while lower anisotropy indicates that the representations are spread out more evenly in all direc- tions. is defined as follows:
Anisotropy(h:)
(5)
Σ o?
where = min(L, d}. and 01, ..,
. OK are the singular values the matrix h,,
C.4 INTRINSIC DIMENSION
Intrinsic Dimension (Facco al., 2017; Cheng et al., 2025) quantifies the minimal number of coor- dinates required to describe the local geometric structure of the representations without significant information loss. higher intrinsic dimension indicates rich, complex latent structure, while lower intrinsic dimension suggests the representation lies on simpler manifold. Specifically, for the matrix h, TR. Lxd we can view as a collection of points h in the d-dimensional space, es-
i... h, [hi To calculate the intrinsic dimension, we use the timator (Facco et al., 2017): for each point hj, we compute its nearest-neighbor distance T1,; and distance 72,5, and form the ratio 14; = r2.//11,. Sorting {u, [ ascend- ing order yields p())....: P(L)) and the empirical cumulative distribution is given by F, j/L. Each (G) is then mapped to transformed data point (x, log (G)) yj = - log(1 1 - Fj)). Un- der mild assumptions, the points {zz,, y are theoretically expected align through the origin, and the slope of this line provides an estimation the intrinsic dimension. Fol- lowing Glielmo al. (2022),
straight line
the standard Fuclidean distance the distance metric, and
20

---

Published as a conference paper at ICLR 2026
introduce trimming factor/ 0.1 discard extremely large values of μr /2.//1... ensuring robustness against outlier data points that may violate the estimator's assumptions. The detailed algorithm is summarized in Algorithm 2.
Algorithm 2 Calculation of Intrinsic Dimension with Two-Nearest--eigghhor Estimation
1: Input: matrix h, [b/)/=1, distance metric dist(:,)), trimming fraction f E E0,,1
2: Output: estimated intrinsic dimension d
3: for i= to L do
4: Compute the pairwise distances (dist(hj, h:)
rı, 4- smallest distance (nearest neighbor)
6: P2,1 f second smallest distance (second nearest neighbor)
7:
14 + -r2.//T1.
8:
Sort [/ 1 in ascending order to obtain (M)))-.. ((())
9:
for = 1 to do
10:
zj < log(p())
12:
Mj log F2)
f then
14: Trim the largest [f L] values of P(y) setting L' L(1 L]
else
16:
Keep all the P( by setting L' + L
Fit the points the plane given by coordinates ((T,, v;)) with straight line y = d.w passing through
the origin
return the slope d as estimated intrinsic dimension
D INTERPRETABILITY ANALYSIS OF LATENT THOUGHTS
To better understand the observed patterns in the correct and incorrect latent thoughts, we provide an interpretability analysis by decoding the latent thoughts different reasoning steps and examining how they evolve toward (or away from) the correct answer. To make this analysis clear, we use one-digit arithmetic dataset, where the model must output single digit answer to an arithmetic question. This setting is ideal for interpretability because the operations are simple and easy to ver- ify, and we can directly inspect how the decoded latent thoughts evolve at the specific token position corresponding to the answer digit. Using the coda module (decoder) from the latent reasoning lan- guage model, we decode the latent thought each thinking step into its top-5 most probable tokens and analyze the progression of latent thoughts and show representative examples with correct and incorrect thinking patterns in Figure A2.
For correct examples, observe that the correct digit token emerges among the top-k candidates at middle thinking steps, then rises to rank-1 and stabilizes in later steps. In contrast, for incorrect examples, the correct token does not consistently rise in rank, the latent thoughts show fluctuation or drifting behavior and the final step fails to converge to the correct answer. The pattern differences between correct and incorrect latent thoughts demonstrate that the latent reasoning language model encodes meaningful reasoning patterns in its latent thoughts that reflect its thinking processes.
E TRAINING DETAILS OF THE LATENT CLASSIFIER
To capture the thinking dynamics of the latent thoughts across different thinking steps. we de- sign a latent classifier that can operate over the sequence of latent representations. Specifically, we adopt a 2-layer Transformer (Vaswani et al., 2017) with Sinusoidal positional encoding to en- code the sequence latent thoughts. The configuration the latent classifier (hidden dimension- ality 5280, number of attention heads 55, and MLP hidden size 17920) follows the configuration of Huginn-3.5B. While we observe in our experiments that alternative configurations also bring comparable performance, of the Transformer are aggregated with mean pooling over the dimension (number of thinking steps), followed by two-layer MLP with ReLU activation function to produce logits for binary
use this configuration
the default setting. The output sequences
21

---

Published as a conference paper at ICLR 2026
Example with Correct Thinking Patterns (Answer: 7)
Question: What is (1 * 1) 6? Answer:
Top-5 ranked tokens decoded from latent thoughts at different thinking steps:
Step 16: [-,
3, 2,
Step 32: [1, 7, 6, 2, 8]
Step 48: [7, 6, 1, 8, 2]
Step 64: [7, 6, 1, 8, 2]
Example 2 with Correct Thinking Patterns (Answer: 5)
Question: what (7 + 2) 42 Answer:
Top-5 ranked takens decoded from latent thoughts at different
thinking steps:
Step 16: [-, 1, 3, 2, 4]
Step 32: [1, 9, 2,
Step 48: 15, 1, - 7, 4]
Step 64: [5, 7, 9, 1, -]
Example with Correct Thinking Patterns (Answer: 7)
Question: What is (3
2) + 1? Answer:
Step 16: [-, 3, 2,
Step 32: 16, 1, 5, 7,
Step 48: [6, 1, 7, 5, 9]
Step 64: [7, 1, 6, 5, 9]
Example with Incorrect Thinking Patterns (Answer: 8)
Question: What is (8 2) 2? Answer:
Top-5 ranked tckens decoded from latent thoughts different thinking steps:
Step 16: [-, 1, 2, 3, 4]
Step 32: [6, 4, 1, -1 5]
Step 48: 16, -, 5,
Step 64: 16, 4, 8, -,
Example with Incorrect Thinking Patterns (Answer: 7)
Question: What is (3 - 2) + 62 Answer:
Top-5 ranked tckens decoded from latent thoughts at different thinking steps:
Step 16: [-, 1, 3, 2,
Step 32: [1, 5, 2, 3]
Step 48: [3, 4, 5, 2, 1]
Step 64: [4, 5, 6, 3, 2]
Example with Incorrect Thinking Patterns (Answer: 7)
Question: What (6 4) 52 Answer:
Top-5 ranked takens decoded from latent thoughts at different thinking steps:
Step 16: [-, 1, 3, 2, 4]
Step 32: [1, -, 2, 4,
step 48: [2, 1, 3, 4,
Step 64: [1, 2, 3, 5, 4]
Figure A2: Tokens decoded from correct and incorrect latent thoughts different thinking steps.
22

---

Published as a conference paper at ICLR 2026
Table AI: Performance comparison the latent classifier with different aggregation strategies. The best performance in each column is in bold.
Aggregation Strategy
GSM8K
SVAMP
CommonsenseQA
MBPP
Accuracy ROC-AUC
Accuracy
ROC-AUC
Accuracy ROC-AUC
Accuracy
ROC-AUC
first 10 tokens
0.708
0,742
0.924
0.980
0.574
0.595
0.732
0.742
last tokens
0.790
0,863
0.957
0.987
0.610
0.644
0.749
0.773
all the tokens
0.820
0.884
0.960
0.987
0.623
0.671
0.790
0.807
classification. Training is performed with binary cross-entropy loss for 10 epochs using the Adam optimizer (Kingma & Ba, 2015) with a learning rate of 6.
However, challenge is that each latent thought h, E R is matrix rather than vector, and this requires an aggregation over the dimension I. before can be processed by the Transformer. To address this challenge, empirically we observe that apply mean pooling over the hidden states corresponding to all the L tokens yields better performance than mean pooling over the hidden states corresponding the first 10 or the last 10 tokens. Therefore, we choose to apply mean pooling over the dimension for each latent thought h, This design choice is also motivated by the common practices in probing methods, where mean pooling over the sequence dimension is widely adopted as a standard approach for deriving fixed-length representations from variable-length sequences (Hewitt & Manning. 2019; Tenney et al., 2019; Ren et al., 2023a).
experiment with different aggregation strategies in Table AI, and
F ADDITIONAL THEORETICAL RESULTS
F.1 PROOF FOR THEOREM 1
Theorem 1. Given a sampled set of N approximate the policy distribution x (2|x), for sxdes fr(zx))
each 24 the solution to Equation 2 is (z(2)z) E (n((,]10) (r(2,,)))
Proof. Since we are sampling from a discrete set of { we represent the policy distribution (2)xx) as a vector over the set of latent thoughts { To ensure that T<22z) forms a valid policy distribution, (22(x) should satisfy the constraint Σ [21[2> = 1. To solve the optimization prob- lem from Equation subject this constraint, we introduce a Lagrange multiplier À and construct the Lagrangian:
A() N (((l)))((, (2. log
T (x)
Σ
(π(*(x)- 1)
meer (2 [x:)
To find the solution to this problem, since we are optimizing over probability distribution क| (z), can compute the partial derivative of the objective [(T(=)=), A) with respect to each coordinate n (z.(z).. Setting the partial derivative to zero, for each z;, we have:
(C(T()))),) A) (x(()))
r(x, zi) - 3 log (2)(F)
Tee Ter( (2/(z) +1 + +\=
By rearranging this equation, we can get:
(z+lz)
exp( 7xx, zi) + B, <<2(2r) x )er((z)())) exp( r(x, 2i)
Teen( ())rr
Plugging in the constraint that Σn T(2((T)= = 1. for each we obtain the solution:
Teer(2: x:) exp (1)(2,22))
(z(2/))
Σn Ther(23 x)) exp (2,
23

---

Published as a conference paper at ICLR 2026
Here the subscript notation 7, to indicate that the policy is derived from the reward function r(z, 2). For simplicity, we omit the superscript *, but ny still represents the optimized policy.
Intuitively, the optimized policy тH reweights the original policy Tee with the exponential reward term exp( 3 r(x,22) 2)): latent thinking trajectories with higher reward r(z, z) will have higher prob- ability of being selected, while trajectories with lower reward will have lower probability of being selected. The weight /3 controls how strong this adjustment is: when 3 is small, the policy becomes more "greedy" and focuses heavily on the high-rewarded latent thinking trajectories; when B is large, stays closer to the original policy Tie--
F.2 PROOF FOR THEOREM 2
Theorem 2. In Algorithm for each the probability of being drawn and accepted is Pr(zi|us < 6s.07 x) = (z)
Proof. In Algorithm 1, since the distribution (2(22) is difficult to directly sample from, would like to draw candidate samples = from the distribution [==(= and only accept those samples that follow the distribution T, (2|zz) with probability M- Here M is constant, and for the (T(=((z
x_(=[²)
acceptance probability to be valid, must satisfy M=Fer(=(2) S that is, M xr(s,lz)
<a((=a/)) for each
24. We choose the smallest possible M so that each 2; has the highest chance of being accepted, because a tight M avoids unnecessary rejections and makes the algorithm more efficient. Therefore, the value of M can be calculated as:
exp (2)( 23)
Th (r(22/z) max
M = max
IS$SN
Teef Tee (2/
IS1SN
N j=1 mer x)) exp (22(z,
exp [ rimxx
Σj=1 (rer (z) exp (gr(0,2g))
where Vmmx is the maximum reward calculated in Algorithm 1. Then we can get the acceptance probability o; for each zy:
exp (r(2, 2i)
TT (2.(2)
Φ M (eel(2z (2//z)
exp((r(x,, 2:) /imaa//3
M EE Mee(2 x)) exp (f)(2,2z))
For each candidate 2; have:
n. (2)lz)) (2((2)) mer(Zi(())
Pr(z,, u, < のi x)) = teत(( 2). = कातl(Z x)
M
The total probability of acceptance is:
N
N
ग (2, x)
1
Pr(u < 0: [x) = Σ Pr(zy, < di xz)
Σ
T. (2)l2z) M
M
M
Therefore, by Bayes rule, the probability of being drawn and accepted is:
Pr(2; u, < x),x)= Pr(z,, u, < Φi, 22)
a.(a.l)
ग. (2)
Pr(u, < di x))
M
F.3 THEORETICAL ANALYSIS ON CORRECTNESS RATE
To analyze the expected correctness rate of the LTO algorithm using the trained latent classifier LRM, first introduce the notion of perfect reward model, which serves oracle for evaluating the correctness of latent thinking trajectories. This formalization provides a reference point for quantifying the performance of the latent policy derived from the trained LRM::
24

---

Published as a conference paper at ICLR 2026
Definition 1 (Perfect reward model). A perfect reward model (", 2) is function that always assigns a value of 1.0 if the latent thinking trajectory& z is correct question x, and 0.0 if the latent thinking trajectory 2 is incorrect for question Using this definition, for question z, the expected correctness rate of a latent policy π can be represented as Ezzar" (I,
Next, we introduce the following theorem to measure how the expected conrectness rate of T. (2(zz) (the policy derived from the trained LRM) relates to that of z ~ TT- (the policy derived from the perfect reward model):
Theorem 3. For a question x. for each sample 2, if the error between the trained reward model
r(a, 24) and the perfect reward model r" (xx zi) is bounded by e, that is, |r(x, 21) -r" zi]l S then the performance gap of using an imperfect reward model is upper bounded by
[Ezwxls w)r" uyr 2) - -E (x, z)
Proof. The expectation of the performance gap A between using the trained reward model and using the perfect reward model is:
A
E (slo) (x,, =) - E Esra, (slx)) (z, =)1
Σ (z][0)) -T- (2(2z=)) r" (x)
| -ep (z/|z²) |z) 1
Using Pinsker's inequality (Cover & Thomas, 2006), we have:
mr (z]lat) - n. (2:(=r) < V 2T. (Tr (=|x=))] (=(xr)
Teal (2) exp( )((,,))) Em- Tル(2z||2 exp( ao2 )(
Recall that in Theorem 1, we can get the solution T. (2((2))
(m) (xp(3r (rr()))))))))) (a,a.) Tye (2/(z) Σg ma((sy (x) exp( (2,,)))) Therefore, the KL divergence between the policy dis- tributions can be written as:
Dxx (#| (=|x²)
N
(=(zz> log T (z((z) n (z)
nutert exp( 1 (()))))
Σ
E my (=/[10) log En mar(a,) (z) exp( }r((2,5))
nuulerttf exp( 3rr (c.
Σn Ferl (aa exp( J rm (x,,2))
Σj=1 Trar(ej x)) exp (3r(z, 23)
N
Σx
1
N
(2](12)))
log log exp( cxpppp(((0 (r(2, 21) - (x, 21))
log
1i M,r(ze x)) exp (1rr(a (x, 23))
Te (2/1z)
(g) (1(,,) - -r (I, 21))
Σj= mer x)) exp ( (I. =3) exp ( (r(2, 25) - (x, (1,))
log
En Mee((e) x) exp (1,)
En i=1
N
Ty (2)lz)) (g(r,,,))) - r" (x,
Teee(ez I x)) exp 1 (I. (I,
exp l1 (1rless)
log
Ene Teer(e) x)) exp (1 (a, 23))
N
T (2x
[ (1(00, 21) - (x, - loog
T (25l2z) exp (1(00 25) - r (z, (1.,
(3
j=1
25

---

Published as a conference paper at ICLR 2026
Using Jensen's 's inequality, we have:
(zgl2z) exp 1
log
zz) - -x)(z, )
N
N
T (zg log exp ( ((((2)))) r- (0, (1x =
Σ
- (3)122) (3(te))))))))
r
1=1
==1
Therefore, we have:
ExL (Tr (2/2 (2(x))
N
N
Rr (2//2)
(r(2, zi) (2, zi) -
Σ
Tr- (z)
-r -r (v.2))))
1
B
j=1
N
1
Σ
mo. (2/|20)
|r(x, 25) - -r" (1, Σ
Tr (z/ (z)
(
(x, -
3
N [e 6 E
Tr (z(21(22) x
+
1=ر T. (2)1(z)
B B
N In Theorem 1, we have the constraint that Σ/= म. (2y (z) 1, and Σj=1 T.. (2ylz) 1. Therefore, the KL divergence between the policy distributions can be written as:
DeL. (n, (=|x))]) (=)1a)) - E (2/l0))
6
Tr (zg (r)
3
3
j=1
6
2c
2 3
(2)l(r)
+
=
3
13_
3
Putting all the results together, we get:
4e
[ [e (x, (slar
(r, S
3
This theorem establishes a bound on the expected correctness rate of trajectories generated using the trained LRM in comparison to the perfect reward model. As the performance of the classifier improves, the егrоr will drop, leading to tighter bound and higher expected correctness rate. Notably, even if the latent policy of the base model is not explicitly optimized, more accurate LRM with smaller enables L.TO to more accurately select only the correct latent thinking trajectories, thereby improving the expected correctness rate. Empirically, as shown in Section 3.3, the classifier achieves very high AUC-ROC, implying that is small in practice. From theoretical perspective, standard generalization bounds for binary classifiers guarantee that the reward error is controlled by the classification error on the training set plus a complexity term of order O I//S) with S being the number of training samples (Bartlett & Mendelson, 2002; Bartlett et al., 2017). Consequently, with a well-trained classifier as the reward model, this bound guarantees that the expected correctness rate under the trained reward model closely matches that of the perfect reward model.
G EXPERIMENTAL DETAILS
G.1 DATASET DETAILS
We select five datasets from three domains for a comprehensive evaluation. The details of datasets are described follows:
Math Problems
GSMSK (Cobbe et al., 2021) a collection of grade-school math word problems written by human annotators. The dataset is designed to evaluate arithmetic and reasoning skills at the
26

---

Published as a conference paper at ICLR 2026
grade-school level and serves benchmark for testing the multi-step reasoning capability of LLMs. It divided into 7,473 training problems and 1,318 test problems, and each problem is paired with detailed teep-by-step solution based on basic arithmetic operations.
GSM-Symbolic (Mirzadeh et al., 2025) is more challenging extension of GSM8K that gen- erates diverse math problem variants using symbolic templates. It includes 5,000 test problems but does not provide training split.
SVAMP (Patel et al., 2021) is also collection of grade-school math word problems. It is constructed by applying systematic variations to seed examples from the ASDiv dataset (Miao et al., 2020) to discourage shortcut reasoning patterns. The dataset is split into 35,381 training problems and 1,000 test problems.
Commonsense Reasoning
CommonsenseeQ (Talmor et al., 2019) is a multiple-choice question answering benchmark dataset designed to evaluate the capability of LLMs to perform commonsense reasoning. It consists of 9,741 training problems and 1,221 1 test problems.
Code Generation
MBPP (Austin et al., 2021) is benchmark dataset for evaluating the capability of LLMs to generate programming codes. It consists of Python programming problems covering basic algorithmic and data-processing tasks. Each problem is paired with natural language descrip- tion, reference implementation, and multiple test cases. The generated code is considered correct only if successful passes all the test cases. The dataset is divided into 374 training problems and 483 test problems.
G.2 IMPLEMENTATION DETAILS
To train the latent classifier as the LRM, we generate multiple latent thinking trajectory-answer pairs for each dataset. For GSM8K, SVAMP, and Commonsense@AA, sample 5 different latent thinking trajectories and answers per problem from the training split. For MBPP, which contains only 373 ensure
training problems, we sample 50 latent thinking trajectories and answers per problem sufficient training data. The latent classifier is trained to predict the correctness of the answer from the latent thoughts on each dataset. For GSM- MSMM-yymbolic,, which does not include a training split, we use the classifier trained on GSM8K. We evaluate the performance of baselines and our approach on the test split of each dataset. For LTO and the baselines, we allocate a sampling budget of N-20 per problem. Each method selects single final solution from these candidates (i.e., the number of required samples M=1). For baselines, the solution with the highest evaluation score (e.g., verbal evaluation score, confidence score CoE score) will be selected. We adopt the default setting with sampling budget 20 and latent thinking steps T-32, and the performances with different sampling budget, different Bs and different number of thinking steps are studied in Section H.2, Section H.3 and Section H.4, respectively.
For LRMs general LLMs, follow the same training configuration in Appendix E. For each LLM, we configure the corresponding LRM with the same hidden dimensionality, number of attention heads, and MLP hidden size that LLM, following the training setup in Appendix E. For example, an LLM has hidden dimensionality 4096, number of attention heads 32, and MLP hidden size 14336, then the LRM also has the same hidden dimensionality 4096, attention heads 32 and MLP hidden size 14336. The hidden states from all layers are stacked together to form a sequence of latent thoughts. For example, the hidden representation from layer is treated as latent thought step 1, the representation from layer 2 as step 2, and so on. This stacked sequence of latent thoughts from general LLMS serves as the input the LRM, and has exactly the same format as the sequences of latent thoughts from Huginn-3.5B. To ensure that general LLMs will generate different latent representations for multiple samples of latent representations, we randomly sample one example problem--nswed pair) from the training split of each dataset, and append this example as an in-context demonstration to the input question. For Symbolic,,,, which lacks a training set, instead draw examples from the training of GSM8K. Because new example is drawn at each iteration, the input tokens and consequently the latent representations will be different across multiple samples.
27

---

Published as a conference paper at ICLR 2026
0.9
Accurncy
0.8
ROC-AIX
0.8
0.7
ROC-AUC Improves
ROC-AUC
Improves
Plateous
Platoous
8
16
24
28
4
12
16
20
24
28
32
Thinking Steps
Latent Thinking Steps
(a) Accuracy on GSM8K.
(b) ROC-AUC on GSM8K.
1.0
1.0
AUCC
Accuracy
0.9
0.9
ROC-ADCI
ROC-ALIC Plaseuus
0.7
Impeoves
Plateous
0.8
16
20
24
28
4
12
16
20
24
28
32
Thinking Steps
Latent Thinking Steps
(c) Accuracy on SVAMP.
(d) ROC-AUC on SVAMP.
0.8
ROC- ROC-ADiC
0.8
Accurncy
0.7
0.7
ROC-AUI
ROC-AIIC
improves
Plateaus
Plazeous
0
R
16
20
24
28
12
16
20
24
28
32
Latent Thinking Steps
Latent Thinking Steps
(c) Accuracy on MBPP
(f) ROC-AUC on MBPP.
0.7
Accurncy
0.6
ROC-AUC AtG
0.6
ROC-AUI
ROC-AUI
0.5
Improves
Plateous
0.5
Platesus
8
16
24
28
12
16
20
24
28
32
Lalent Thinking Steps
Latent Thinking Steps
(g) Accuracy on CommonsenscQA.
(h) ROC-AUC on CommonsenseQA.
Figure A3: Test-set performance of the latent classifier (measured by Accuracy and ROC-AUC) on
the test set trained with varying numbers of latent thinking steps on the SVAMP and MBPP datasets.
Correctness Rate
Correctness Rate
Comeetnese Rate
0.54
Currectnets Rate
0.52
Coneetness Rate
0.30
0.38
0.30
poood pocooed
0.53
0.51
0.29
0.35
0.28
Buse: 0 326
Base: 0.265
0.52
Hase: 01 0.517
0.50
Hase: O 500
0.28
Base:
2.2%
0.26
12 16 20
8 12 16 20 Number of Samples
16
Number 'SSmples
Samples
Number Samples
Number Samples
(a) GSM8K
(b) GSM-Symbolic
(c) SVAMP
(d) CQA
(e) MBPP
Figure A4: Performance of LTO with different numbers of samples. "CQA" refers the Common- senseQA dataset. "Base" refers to the performance of the base model.
Correctreass Rate
0.38
Corfectresss Rate
0.30
Correcteess Rtae
0.54
Corretness Rate
0.52
Correctress Rate
0.30
0.35
0.28
0.53
0.51
0.29
0.32
0,326
0.26
2665
0.50
Base: 500
0.28
0.278
(a) GSM8K
(b) GSM-SSMmbblloa
(c) SVAMP
(d) COA
(e) MBPP
Figure A5: Performance of LTO with different betas. "CQA" refers to the Commonsense(A dataset.
"Base" refers to the performance of the base model.
28

---

Published as a conference paper at ICLR 2026
Table A2: Performance of the latent classifier on the test for general LLMs on different datasets.
Model
Metric
GSM8K
SVAMP
CommonsenseQA
MBPP
Accuracy ROC-AUC
0,896
0,854
0,652
0,858
OLMo-7B
0.851
0.899
0.708
0.882
Accuracy
0,836
0,919
0,681
0,834
Llama-2-1 2-7B
ROC-AUC
0.858
0.970
0.738
0.822
Accuracy
0.805
0,925
0.729
0,805
Llama-2 2-13B
ROC-AUC
0.868
0.974
0.773
0.839
Accuracy
0.793
0,968
0.736
0.741
Mistral-7B
ROC-AUC
0.868
0.992
0.765
0.794
H ADDITIONAL EXPERIMENTAL RESULTS
H.1 ADDITIONAL RESULTS ON THE PERFORMANCE OF THE LATENT CLASSIFIER
Additional experimental results on the performance of latent classifier for Huginn-3.. using dif- ferent thinking steps on different datasets are shown in Figure A3.
Additional experimental results on the performance of latent classifier for general LLMs on different datasets are shown in Table A2. In this setting, each LRM is trained with the latent representations from all the layers of each general LLM.
We can that the latent classifier achieves strong performance the test set for Huginn-3.5D and general LLMs across diverse datasets. These results demonstrate that latent thoughts encode appropriate reward signals that can indicate whether they will lead to the correct answer.
H.2 PERFORMANCE WITH DIFFERENT SAMPLING BUDGET
To investigate the performance of LTT with different sampling budget N, we vary N from 1 to 20 and report the performance of LTO in Figure A4. Performance steadily improves N increases, as a larger N enhances the diversity of sampled latent thoughts and increases the likelihood that least one sampled latent thinking trajectory is correct. Moreover, even with very small budget (e.g., N = 2). LTO can still achieve substantial performance improvement compared with the base model, demonstrating that LTO is sample-efficient without the need for large sampling budget.
H.3 PERFORMANCE WITH DIFFERENT BETAS
To investigate the performance of LTO with different B. vary 3 from
and report the performance of LTO in Figure A5. Across different values of B, LTO consistently outperforms the base model, demonstrating that it can reliably improve the latent thinking processes with different choices of the hyperparameter.
H.4 PERFORMANCE WITH DIFFERENT NUMBERS OF THINKING STEPS
a fixed number of latent thinking steps, we also investigate the adaptability of LTO to latent thinking trajectories of varying thinking steps. Specifically, for each dataset, train the LRM with the sampled latent thinking trajectories with varying number of thinking steps. We then test the performance of LTO using this LRM trained with varying number of thinking steps.
While most of our evaluation
From the experimental results in Table A3, we can see that LTO achieves a consistent improvement over the base model in all the cases, indicating that LTO can be flexibly applied to latent thinking trajectories of varying numbers of thinking steps. Interestingly, performance slightly declines as the number of steps increases. This attributed to the reduced diversity the sampled latent thoughts and answers when longer thinking steps used. For example, SVAMP, when using 16 thinking steps, 427 problems have sampled answers that are all incorrect, 437 problems have sampled answers that all correct, and 136 problems have both correct and incorrect answers.
29

---

Published as a conference paper at ICLR 2026
Table A3: Performance of LTO with different numbers of thinking steps. For each thinking step, the best- pest-performing method is highlighted in bold. indicates the improvement over the best runner-up statistically significant with p 0.05.
Thinking Steps Method
GSM8K GSM-Symboliic SVAMP Commonsense(A MBPP
Base Model
0.333
0.269
0.503
0.498
0.276
16 Steps
Majority Voting
0.345
0.279
0.501
0.498
0.274
Latent Thinking Optimization
0.434'
0.335'
0.560
0.523'
0.295
Base Model
0.326
0.265
0,515
0.507
0.282
24 Steps
Majority Voting
0.334
0.274
0,513
0.509
0.293
Latent Thinking Optimization
0.398
0.312*
0.549
0.523*
0.293
Base Model
0.326
0.265
0,517
0,500
0.278
32 Steps
Majority Voting
0.333
0.269
0,511
0.504
0.288
Latent Thinking Optimization
0.378
0.303"
0.539
0.520*
0.295
Table A4: Performance comparison of the Llama model family. The best-performing method for each model is in bold. indicates the improvement the best runner-up is statistically significant with < 0.05.
Model
Method
GSM8K GSM-Symbolie
CommonsenseQA
MBPP
Base Model
0.223
0.204
0.399
0.189
-lama-2-7B
Majority Voting
0.275
0.302
0,493
0.193
Latent Thinking Optimization
0.389*
0.316
0.606
0.237
Base Model
0.306
0.273
0,398
0.247
Llama-2-13B
Majority Voting
0.417
0.379
0,501
0.263
Latent Thinking Optimization
0.534
0.442
0.650
0.322
Base Model
0.784
0.736
0.742
0.560
Llama-3-8B
Majority Voting
0.801
0.796
0.786
0.570
Latent Thinking Optimization
0.859
0.821
0.790
0.600
Therefore, the performance upper bound is (437 136) 0.573. By comparison, when using 24 thinking steps, the split becomes 446 06////8/8 86 with the performance upper bound calculated as (468 85) /1000 0.554; when using 32 thinking steps, the split becomes 454/486/ 486/60 with the performance upper bound calculated as (486 + 60)/1000 = 0.546.
While increasing the number of thinking steps slightly improves the expected correctness rate of the base model, it substantially reduces the diversity of sampled latent thoughts and answers, probably due overthinking (Sui al., 2025). As result, fewer problems contain both correct and incorrect answers (i.e., diverse sets), leaving less room for improvement with LTO. is possible that there the base
exists an optimal number of thinking steps that balances the expected correctness model with the diversity of the latent thoughts and answers, and future work may design adaptive mechanisms identify such optimal thinking steps and further improve the performance of L.TO.
H.5 PERFORMANCE COMPARISON ON LLM MODEL FAMILY
To further validate the effectiveness of LTO on general LLMs, we conduct an additional experi- ment evaluating LTO the widely-used Llama model family (Touvron et al., 2023b; Llama Team, 2023). The experimental results in Table A4 show that LTO consistently enhances the reasoning performance across all models, demonstrating its effectiveness in improving the latent thinking pro- cesses of the LLM model family.
H.6 PERFORMANCE ON MORE CHALLENGING BENCHMARKS
To further validate the effectiveness of LTO general LLMs, provide additional analysis ing two more recent LLMs (Llama-3-8B (Llama Team, 2023) and Qwen-3-4B (Yang et al., 2025)) on two broader, frontier benchmarks (MATH (Hendryeks al., 2021) and GPQA (Rein et al., 2024)),
30

---

Published as a conference paper at ICLR 2026
Table A5: Performance of LTO on more challenging benchmarks. The best-performing method for each model is in bold. * indicates statistically significant improvement with < 0.05.
Model
Method
MATH
GPQA
Base Model
0.267
0.268
Llama-3-8B
Majority Voting
0.335
0.276
Latent Thinking Optimization
0.375
0.310
Base Model
0,552
0.347
Qwen-3-4B
Majority Voting
0.555
0.368
Latent Thinking Optimization
0.619
0.490'
Table A6: Comparison of the total training time and GPU memory usage of LRM across different datasets and settings. "General" denotes the general reward model from Section 6.3.
GSM8K
SVAMP
CommonsenseeQA
MBPP
General
Total Training Time (h)
0.85
4.19
1.05
0.92
6.12
GPU Memory Usage (GB)
10.39
10.40
10.39
10.40
10.39
which are known be relatively noisy and pose more challenging reasoning conditions for LLMs. The results in Table A5 show that LTO consistently enhances the reasoning performance across all models and datasets, demonstrating its effectiveness and robustness in improving the latent thinking processes of the general LLMs on broader benchmarks under noisy and challenging conditions.
H.7 EFFICIENCY ANALYSIS
We evaluate the efficiency of our framework from two perspectives: the training efficiency of LRM and the sampling efficiency of LTO. Our results demonstrate that LRM requires only modest re- sources to train, and sampling answers with LTO brings negligible additional cost during inference.
Training Efficiency of LRM We analyze the training efficiency of the LRM on Huginn-3.5B by measuring the total training time and GPU memory usage of LRM across different datasets and settings. All the experiments are conducted on a single A100 GPU using the default 32 thinking steps. From the experimental results in Table A6, we can see that the training of LRM can be completed within reasonable time and modest memory budgets in all the settings. Such resource cost is significantly lower than that of language-based reward models (Wang et al., 2024; Lu et al., 2024). These results demonstrate that reward modeling the latent space offers more efficient alternative to reward modeling in the natural language space.
Sampling Efficiency of LTO Compared to standard inference procedure, which directly samples latent thoughts and responses from the base model, LTO introduces an additional step for latent reward computation. To evaluate the efficiency of this step on Huginn-3.5B. we compare the av- erage computation time the base model inference and the latent reward computation per sample across five datasets. All the experiments are conducted on single A100 GPU using the default 32 thinking steps. From the experimental results in Table A7, we can see that the computation time of LRM is orders of magnitude lower than the inference time of the base model, indicating that LRM highly efficient and incurs little computation cost. Moreover, since there are not sequential depen- dencies between the sampled latent thinking trajectories, the sampling process in LTO can be fully parallelized. Therefore, LTO incurs only negligible additional inference cost, and its total inference time be almost the same with direct sampling from the base model when parallel sampling is introduced.
Efficiency Analysis on General LLMs LRMs are highly efficient both for general LLMs and Huginn-3.5B due to its lightweight architecture with only 2 layers of Transformer encoder. Its training time is small because it only requires small amount training data and its inference time is orders of magnitude lower than the inference time of the base LLM. For example, as shown in
31

---

Published as a conference paper at ICLR 2026
Table A7: Comparison of the average computation time (seconds) of the base model inference and the latent reward computation per sample across five datasets.
GSM8K
GSM-Symbolic
SVAMP
CommonsenseQA
MBPP
Based Model Inference Latent Reward Computation
39.5
43.0
6.0
7.3
20.4
7.6e-02
7.6c-02
7.5e-02
7.5e-02
7.6e-02
Table A8:
Comparison of the total training time and GPU memory usage of L.R across different datasets for Llama-3-8B.
GSM8K
CommonsenseQA
MBPP
MATH
GPQA
Total Training Time (h)
1.21
1.53
0.66
1.69
0.42
GPU Memory Usage (GB)
6.39
6.40
6.39
6.39
6.39
Table A6 and Table A9, for Llama-3-8B, on a single A100 GPU, LRM only requires about 1.5 hrs to train and 6.4GB of memory usage, and its reward computation can be completed within about 0.02s which is negligible compared with the inference time of the base LLM.
I LIMITATION STATEMENT
Direct Optimization over the Latent Thinking Processes Although LTO is formulated as an optimization problem, it achieves the optimization objective by selectively sampling correct latent thinking trajectories that follow the optimized distribution, rather than directly modifying or updat- ing the latent thinking policy of the base model. As with the common limitation of test-time scaling methods (Gandhi al., 2025; Setlur et al., 2025), when the latent thinking policy of the base model diverges substantially from the optimized distribution (e.g., when the model lacks the problem- solving ability and generates latent thoughts and answers that all incorrect), then LTO cannot improve the latent thinking processes, since every generated latent thinking trajectory remains in- correct. To address this limitation, future works may integrate the reward signals from LRM into reinforcement learning-basse preference optimization framework (Rafailov et al., 2023), enabling direct optimization and refinement of the latent thinking processes.
Optimization with Multiple Reward Signals The reward signals derived from LTO are binary and can only indicate if the latent thinking processes will lead the correct answer. This restricts reward modeling to the correctness of the answer but may not capture other important dimensions, such as safety or helpfulness. An interesting direction for future work is investigate whether latent thoughts are separable along these additional dimensions, and to extend the latent classifier to incorporate these criteria for latent reward modeling. Another interesting direction to extend LTO into multi-objective optimization framework (Wang al., 2025a). This will enable simulta- neous optimization latent thinking processes across multiple reward dimensions and broaden its applicability more general settings for reward optimization and preference alignment.
J IMPACT STATEMENT
Existing methods for reward modeling and thinking optimization in LLMs are primarily performed in the natural language space (Wang et al., 2024: Lu et al., 2024), but may be costly and prone to the overthinking issue (Sui al., 2025). Our research demonstrates that the latent representa- tions of both Huginn-3.5B and general LLMs encode appropriate reward signals that can be directly leveraged to optimize the latent thinking processes. Furthermore, we show that reward modeling in the latent space can generalize across domains and shows strong potential for building gener- alist reward model in the latent space. Our results demonstrate that reward modeling and scaling test-time thinking with supervision (Muennighoff al., 2025; Guo et al., 2025; Setlur et al., 2025) can be performed directly in the latent space. highlighting its potential as a general, efficient, and domain-agnostic approach improving the thinking processes of L.MMs.
32

---

Published as a conference paper at ICLR 2026
Table A9: Comparison of the average computation time (seconds) of the base model inference and the latent reward computation per sample across different datasets for Llama-3-8B.
GSM8K
CommonsenseQA
MBPP
MATH
GPQA
Based Model Inference Latent Reward Computation
8.2
12.5
21.8
1.3e-2
1.6e-2
5.0e-3
5.0e-3
5.0e-3
We do not aim to claim that latent reward modeling and latent thinking optimization are better than natural language- based reward modeling and verbal thinking optimization. Instead, we show that they offer efficient and effective alternatives in specific settings and open up promising directions for future works. For example, in resource-constrainee and inference efficiency is imperative, L.TO effectively optimize the thinking processes of LLMs with low computation cost. We hope that our research motivates further exploration of reward mod- cling and thinking optimization in the latent space-a a largely underexplored but highly promising direction for advancing scalable, efficient, and generalist LLM thinking and reasoning.
settings where training computation is limited
K ETHICS STATEMENT
All the datasets used in this research are from public open-access benchmark datasets, which are fully anonymized and do contain sensitive or private information.
L REPRODUCIBILITY STATEMENT
Calculation methods for the representation quality methods are provided in Appendix complete proof of the theorems is provided in Appendix F. The implementation details and the computational cost of the LRMs are provided in Appendix and Appendix H.7, respectively. The implementation details of the baseline methods are provided in Appendix G.2. Our code and datasets are available at this link.
M USAGE OF LARGE LANGUAGE MODEL
We do not use large language models as contributors to generate any part content or write paper. Large language models are only used as the investigation object of our study (e.g., observe how large language models think in the latent space).
33


Reviews:
[{"review_id": "PCKfJwpdzy", "reviewer": "Reviewer_aTJi", "summary": "This paper investigates latent thinking in reasoning language models, focusing on the Huginn-3.5B model which generates intermediate reasoning steps as latent representations rather than natural language. The authors analyze differences in latent thought trajectories between correct and incorrect answers using visualizations and representation quality metrics, revealing distinguishable patterns. They train a latent classifier as a Latent Reward Model (LRM) to predict correctness from these trajectories and propose Latent Thinking Optimization (LTO), a probabilistic sampling algorithm that optimizes latent policies to favor trajectories likely to yield correct answers. Experiments on math and programming tasks show improvements, and the approach is extended to general LLMs with cross-domain generalization.", "questions": "* How sensitive is LTO to the number of sampled trajectories N or the KL regularization weight β? Did you observe trade-offs between optimization gains and deviation from the reference policy?\n\n* The paper mentions cross-domain generalization of LRM with small training data; what specific domains were tested for transfer (e.g., beyond math/programming to commonsense or scientific reasoning), and how much data was used for fine-tuning in those cases?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The exploration of latent thinking patterns provides novel insights into how LLMs encode reasoning internally, bridging cognitive science inspirations with practical LLM analysis, which could inspire future work on interpretable latent spaces.\n\n* LTO is a computationally efficient alternative to verbal chain-of-thought methods, avoiding overthinking and verbosity, with theoretical grounding in reward optimization and empirical gains on benchmarks like SVAMP and MBPP.\n\n* The generalization of LRM and LTO to standard LLMs (beyond Huginn-3.5B) and across domains is promising, suggesting a scalable, domain-agnostic way to enhance LLM reasoning without heavy natural language generation.", "weaknesses": "* The analysis heavily relies on Huginn-3.5B, a specific latent reasoning model; while extensions to general LLMs are claimed, the paper lacks detailed comparisons or ablation on how well LTO performs on diverse LLM model families.\n\n* Evaluation metrics for latent representations (e.g., entropy, anisotropy) are insightful but somewhat indirect; the paper could benefit from more direct interpretability probes or causal interventions to confirm that observed patterns truly reflect \"thinking\" rather than spurious correlations or memorization artifacts.\n\n* Experimental details on training the LRM (e.g., data scale, hyperparameters) are sparse in the provided sections, and results focus on two datasets; without broader benchmarks or robustness tests (e.g., against adversarial inputs), it's unclear how LTO scales to real-world, noisy reasoning scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hfB73Xe7OQ", "reviewer": "Reviewer_Pyxq", "summary": "This paper first empirically investigates the property of latent in latent thinking LLMs, such as Huggin-3.5B to understand their internal reasoning processes. They discover that the latent trajectories leading to correct versus incorrect answers are highly distinguishable and provide both visual and quantitative measurements to support this claim. Based on this finding, they successfully train a Latent Reward Model (LRM) that predicts the correctness of the final answer directly based on the sequence of latent thoughts. Finally, the paper proposes Latent Thinking Optimization (LTO), a probabilistic sampling algorithm that leverages the LRM's reward signal to guide the model's generation at test-time. This LTO algorithm is supported by both theoretical justification and strong empirical results, demonstrating performance improvements on math, coding, and general QA tasks.\n\nIn general, this paper validates the separation of correct/incorrect latent trajectories and train a classifier LRM that predicts the trajectory correctness sorely based on the latent. Using LRM as reward model to guide the sampling with LTO yields better results than other test-time scaling method.", "questions": "1. Can the authors specify how multi-layer latents in general LLM are composed for the LRM as well as the efficiency analysis of LRM on general LLMs?\n2. Can the authors provide more experiment results for general LLM on recent benchmarks such as MATH/GPQA-Diamond/AIME?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Clear motivation and empirical insight: convincingly shows correct vs. incorrect latent trajectories are separable, supported by PCA visualizations and four representation metrics (entropy, effective rank, anisotropy, intrinsic dimension). A lightweight sequence classifier (LRM) can achieve strong predictive power.\n2. The sampling algorithm LTO is principled and theoretically sound.\n3. Consistent empirical gains on math/coding/QA vs. majority voting, self-correction, and latent heuristics (CoE-R/CoE-C).", "weaknesses": "1. Primary Application Targets a Niche Architecture: The paper's methodology is heavily anchored to the Huggin-3.5B model, which features a specific, recurrent \"latent thinking\" architecture. This model is not a widely adopted or standard foundation model, making it a niche target. \n2. Ambiguous Application to General LLMs. The paper's method for applying LTO to standard transformers (like Llama 2 or Mistral) is conceptually ambiguous. In section 6, the author mentioned that \"train LRMs using the latent representations from general LLMs\". In the appendix D, the authors state \"The latent representations of general LLMs from all the layers are regarded as latent chain of thoughts.\". Does that implies that hidden states in all layers are concatenated together as input to LRM? Meanwhile, efficiency of such methods on the application to general LLM is not sufficiently discussed.\n3. Limited Evaluation Scope for General LLMs: The experiments on general LLMs are constrained to relatively weak benchmarks such as GSM8k, it is unclear if these gains would translate to more capable models on frontier benchmarks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Dcp8et1am4", "reviewer": "Reviewer_jnr6", "summary": "This paper studies how reasoning language models internally represent their “thoughts” in latent space. Using the Huggin-3.5B model, the authors show that hidden-state trajectories corresponding to correct and incorrect answers form clearly separable patterns. They train a small latent classifier to detect these patterns and use it as a Latent Reward Model (LRM) in a new method called Latent Thinking Optimization (LTO)—a test-time sampling procedure that reweights latent trajectories toward those predicted to be correct. Without retraining the base model, LTO consistently improves reasoning accuracy across five benchmarks and generalizes to other LLMs (e.g., Llama-2, Mistral), demonstrating that latent representations implicitly encode reward-like signals useful for optimizing reasoning.", "questions": "Q1: can author explain the KL divergence term. This is is common in LLM post-training, but LTO is essentially not a optimization method, the distribution won't deviate from base policy a lot.\nQ2: can author provide more insight about why correct and incorrect latent trajectories lead to different patterns? For example, why correct latent trajectories tend to be more compact dispersed?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "* The author conduct in-depth analysis of the latent reasoning trajectories, both qualitatively and quantitatively through metrics including entropy, effective rank, anisotropy, and intrinsic dimension.\n* The proposed method is simple that only involves computing evaluation metrics as input features to a small LRM, which is later used to guide the selection of answers through LTO.\n* The paper is well organized, balancing theoretical exposition with extensive experiments and visualizations.", "weaknesses": "* Optimization vs. resampling: LTO is effectively a weighted rejection sampling procedure, not true optimization of the model’s latent policy; the optimization problem formulation is confusing. Also, it not clear if the KL divergence constraint is actually needed. \n* Limited interpretability: Despite excellent quantitative separation, there is little qualitative analysis linking specific latent dimensions or manifolds to interpretable reasoning concepts.\n* Minor overclaiming: Phrases like “generalist reward model” or “domain-agnostic optimization” are ambitious relative to the experimental evidence.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bxA5kWLGBn", "reviewer": "Reviewer_qGuA", "summary": "The paper studies latent thinking in the latent-reasoning LM Huggin-3.5B and claims that correctness signals are encoded in latent trajectories. The authors (i) empirically show separable patterns between correct vs. incorrect latent thoughts via visualization and four representation metrics (entropy, effective rank, anisotropy, intrinsic dimension); (ii) train a latent classifier that predicts answer correctness from partial latent trajectories with high ROC-AUC; and (iii) propose Latent Thinking Optimization (LTO): use the classifier as a Latent Reward Model (LRM) and perform KL-regularized probabilistic selection of latent trajectories via an acceptance–rejection sampler with proofs (Thm. 1/2). LTO improves correctness over voting/self-correction and latent heuristics and is further applied to general LLMs by treating intermediate layer states as “latent thoughts,” showing sizable gains and cross-domain transfer of the LRM.", "questions": "1. As mentioned in Author's guidence, you should disclose the use of Large Language Models. I don't find it in your paper. Could you disclose it here? or you may violate the guidence and lead to desk rejection.\n\n2. Please see weaknesses and try to answer 1-2. For 3, I am curious about methods you have tried.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. neat empirical evidence that correct trajectories differ from incorrect ones, with stepwise analyses and a probing classifier that improves with longer prefixes. \n\n2. LTO formulates a KL-regularized objective and provides a closed-form reweighting plus an accept-reject sampler with correctness guarantees (Thm. 1/2). \n\n3. Broad task coverage & signal utility: improvements across math/commonsense/code; LRM used as weights already helps; LTO helps more. \n\n4. Generality claims with initial evidence: application to general LLMs (OLMo/Llama/Mistral) and cross-dataset LRM transfer; training-data footprint noted as modest.", "weaknesses": "1. Entropy/effective-rank/anisotropy/intrinsic-dimension are scale- and whitening-sensitive; without controlling for per-step activation scaling or layer-norm statistics, a change in metric may reflect variance rescaling rather than “better thinking.” Please specify invariances and show robustness across metric definitions and layers.\n\n2. The LRM is trained from the same model’s latent trajectories using final-answer correctness as labels; if the generator exhibits systematic artifacts (e.g., decoding shortcuts at later steps), the classifier may learn spurious correlates. The strong AUC near the end steps could partially capture answer-formation traces rather than “process quality.” A control where latents are early-stopped and the answer is masked/not decoded would help.\n\n3.  The reward signa is derived from LTO are binary and can only indicate if the latent thinking processes will lead to the correct answer, as you mentioned in appendix", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)