Continuous Chain of Thought Enables Parallel Exploration and Reasoning
Authors: Halil Alperen Gozeten
Muhammed Emrullah Ildiz
Xuechen Zhang
Hrayr Harutyunyan
Ankit Singh Rawat
Samet Oymak
Venue: ICLR 2026 Poster (2026)
Keywords: chain-of-thought
latent space reasoning
parallel exploration
transformers
policy optimization
multi token sampling
Primary area: foundation or frontier models, including LLMs
TL;DR: We establish theoretical benefits of chain-of-thought with continuous tokens and introduce new supervision and policy optimization strategies.

Abstract: Modern language models generate chain-of-thought traces by autoregressively sampling tokens from a finite vocabulary. While this discrete sampling has achieved remarkable success, conducting chain-of-thought with continuously-valued tokens (CoT2) offers a richer and more expressive alternative. Our work provides new theoretical guarantees and algorithms for CoT2, motivated by logical reasoning tasks that inherently require search capabilities. Theoretically, we establish how CoT2 facilitates the model to track multiple discrete traces in parallel; and quantify the level of achievable parallelism and its benefits for inference efficiency. We also provide a CoT2-based one-layer transformer construction that solves the combinatorial ``subset sum problem'' given a sufficient embedding dimension. These insights arise from a novel and effective supervision strategy where we match the language model outputs to the empirical token distributions of a set of target traces. Complementing this, we introduce sampling strategies that unlock policy optimization methods for CoT2. Our primary strategy samples and composes $K$ discrete tokens at each decoding step to control the level of parallelism. 
Experiments confirm that (i) the optimal level of parallelism is governed by the embedding dimension, (ii) our continuous supervision strategy can outperform alternative methods, and (iii) policy optimization with CoT2 indeed improves the performance of the model beyond its initial discrete or continuous supervision.

Published as a conference paper at ICLR 2026
CONTINUOUS CHAIN OF THOUGHT ENABLES PARALLEL EXPLORATION AND REASONING
Halil Alperen Gozeten
M. Emrullah Ildiz"
Xuechen Zhang
Hrayr Harutyunyan'
Ankit Singh Rawat"
Samet Oymak
μ: University of Michigan Ann Arbor falperen, www.eeeeeliiiizzzeeieeeeeee zxuechen, oymak Cumi ch, edu
y: Google Research
(hrayrh, anki (tssttaaaatttoolseeset
ABSTRACT
Modern language models generate chain-of-thought traces by autoregressively sam- pling tokens from finite vocabulary. While this discrete sampling has achieved remarkable success, conducting chain-of-thoupe with contimuusshy-valee tokens (CoT2) offers a richer and more expressive alternative. Our work provides new the- oretical guarantees and algorithms for CoT2, motivated by logical reasoning tasks that inherently require search capabilities. Theoretically, we establish how CoT2 facilitates the model track multiple discrete traces in parallel; and quantify the level of achievable parallelism and its benefits for inference efficiency. vide a CoT2-based one-layer transformer construction that solves the combinatorial "subset sum problem" given a sufficient embedding dimension. These insights arise from novel and effective supervision strategy where match the language model outputs to the empirical token distributions of set of target traces. Complementing this, introduce sampling strategies that unlock policy optimization methods for CoT2. Our primary strategy samples and composes K discrete tokens at each decoding step to control the level of parallelism. Experiments confirm that the optimal level of parallelism is governed by the embedding dimension, (ii) our continuous supervision strategy can outperform alternative methods, and (iii) policy optimization with CoT2 indeed improves the performance of the model beyond its initial discrete or continuous supervision.
also pro-
INTRODUCTION
slain-of-thought (CoT) strategies (Wei et al. 2022), when paired with strong base models, have achieved immense success and facilitated progress in remarkably challenging tasks, such as solving AIME IOI problems (Guo et al., 2025 Jaech al. 2024). Despite these advances, modern language model architectures may fail to utilize their full potential for a few reasons. First their discrete sampling tokens-sslecting tokens. This limits the model to emitting at most logz (v) bits per sample, or more specifically, the Shannon entropy of the softmax output. This contrasts with the O(d) bits each token embedding can store, where d is the embedding dimension. Secondly, discrete sampling can cause the model to commil to certain solutions and avoid exploring alternatives (Yao et al. (023). A practical method to address this is sampling multiple CoT traces and aggregating them, either through consistency (Wang ctt al. 2022) or best-of-N decoding (Ouyang et al. 2022) through more test-time computation.
single token each decoding step from a vocabulary of
In this work, propose and investigate CoT with Continuous Tokens (CoT2) to address these challenges, building on COCONUT (Hao et al.) 2024). The fundamental idea in our CoT2 proposal is that rather than the model sampling single token from the vocabulary, it samples or deterministically
"Equal contribution.
Code: http:///ggthhuuboooo/pa/eeeegr/e

---

Published as a conference paper at ICLR 2026
Supervision
Input
Reasoning Steps
Input
Reasoning Steps
1=2
-
Discrete Col
with Continuous Tokems (Col2)
B=2
Figure 1: Illustration of CoT2 with varying budgets for Minimum Non-Negative Sum (MNNS) task with m = 3 and input numbers 2, 1,4. CoT2 supervision with budget B at steps r (l...., m 1| is the average of embeddings of states visited by B selected trajectories among the 8 possible, and for / = m is the embedding corresponding to the single correct final answer. here for the minimal non-negative sum 1. For B I (discrete CoT), the correct trajectory (-2, -3, 1) highlighted with yellow is used; for = 2, the red and yellow trajectories are used; for 8, all trajectorics are included in supervision.
selects continuous superposition of tokens according to the softmax output. Intuitively, this capabil- ity efffectievvvl selecting multiple tokens simultaneously through continuous superposition wouudd allow the model pack more information within each token embedding and also enable it to track multiple reasoning paths in parallel- potentilly emulating self-consistency or best-of-N decoding with single trace. Toward this vision, we make the following technical contributions:
Budget-constrained supervision: We introduce the continuous supervision strategy (CSFT) for CoT2 models to explicitly track multiple teacher traces in parallel, constrained by a budget. This is done by fitting the model to the empirical distribution of the tokens within the expert traces as visualized in Figure I Through budget choice, CoT2 can interpolate from discrete CoT to track all reasoning traces. Our method reveals fundamental trade-offs between the budget and embedding dimension in terms of accuracy, validated by an information-pact packonn bound in App. E Experiments highlight sweet spot for the level of parallelism (see Figure 2 in line with theory.
Expressivity and statistical benefits: We introduce the problem of Minimum Non-Negative Sum (MNNS) as generalization of the classical Subset Sum problem. These problems, well as related tasks like ProntoQA (Saparov & He. 2022), inherently benefit from parallel search capability. We prove that a single- layer transformer can solve MNNS using CoT2, showcasing the capability of the transformer block to track and expand multiple reasoning traces in latent space.
On the statistical side, we study CoT2 decoding methods: (i) Base CoT2: deterministic inference which creates and feeds continuous tokens using raw softmax output at each step (Sec. 2 and CoT2-MTS (multi-token sampling): conget--on method which samples and averages K discrete tokens to form a continuous token (Sec. D); and standard COT. which is a special case of MTS with K 1. Under suitable conditions, we first prove that base CoT2 tracks the "ideal state" by aggregating all reasoning paths, whereas MTS provides unbiased but noisy estimate of this "ideal state". Finally, Prop. 1.pp.3este establishes that the MTS estimate is as powerful as aggregating the outputs of K standard CoT trajectories. In plain language, this formalizes that CoT2 with budget K can be just as expressive as self- consistent prompting with K traces.
We introduce policy optimization methods for CoT2 (Sec. 5). Our primary strategy MTS samples and composes K discrete tokens at each forward pass to control the level of parallelism. We also introduce purely continuous sampling scheme over the probability simplex via Dirichlet sampling. Experiments on the MNNS, ProntoQA, and ProsQA tasks show that GRPO-based RL with CoT2 further improves the accuracy over SFT CSFT (Sec. (.3). This demonstrates that the RL phase helps the model better prioritize relevant reasoning traces and offers promising strategy for training Coaed language models.
Reinforcement learning for CoT2:
The paper is organized as follows: Section introduces the technical setup, Section 3 presents our continuous supervision strategy and the MNNS, ProntoQA, and ProsQA tasks. Section provides constructive results and strategies and GRPO-based policy optimization methods, and Section 6 concludes with discussion.
guarantees for CoT2. Section 5 describs our sampling
2

---

Published as a conference paper at ICLR 2026
toumber of samples)
C912
COCOACT ERCTe Oiccro
(a) Pass@k accuracies for CoT2 and discrete CoT across different temperatures on the MNNS task.
(b) Accuracy of CoT2, CO-
(c) Supervision trajectory budget vs. different embedding dimen- sions for CoT2 on the MNNS task.
CONUT, discrete CoT, and no- CoT on MNNS and ProsQA tasks.
Figure 2: (a): Discrete CoT model requires multiple samplings (Pass@k) to match single-shot performance of CoT2 model on MNNS (10-run avgs). (b): CoT2 model outperforms COCONUT, discrete CoT, and no-CoTT in tasks involving search, like MNNS and ProsQA (5-run avgs). (c): Tradeoff hetween the number of trajectories superposed and the embedding dimension (5-run avgs). Setting: MNNS with 4 input digits in 1-9. In (a-b), is the full budget for CoT2, and B for discrete CoT. (a): I-layer, 1-head GPT2 with = 24. For CoT2, the intermediate continuous steps are deterministic. so sampling is performed only for the final discrete answer token. (b): MNNS: 2-layer 2-head GPT2, d = 32; ProsQA: 4-layer 4-head GPT2, d = 32. (c): 2-layer, 2-head GPT2 with d E 116, 24, 32).
PROBLEM SETUP
Notation. For an integer n 2 I, we use the shorthand [m] = (1.... n), and denote vectors by bold lowercase letters (e.g. xt) and matrices by bold uppercase letters (e.g. X). For vector R", the component x, is its i-th entry. The zero vector in R" is O,,, and the zero matrix Ruxxe is OWXO. Finally, we let 1 denote the standard v simplex in R".
Assume we have an input context X E Rnd where each row is a d-dimensional embedding vector. Our goal is to output tokens given the context X with mth output token being the final answer that evaluated under a performance metric (e.g. accuracy or reward). For the first m - steps, the model outputs continuous thought tokens (Zche[m-11] that enable reasoning process. In the final step m, the model outputs a discrele token Zm from a vocabulary of size v. In the remainder of this paper, we investigate strategies for training this system in a way that improves final performance over standard discrete next-token prediction.
Formally, let [e,,...,e,J' e, E Rexd be the embedding matrix for the vocabulary of tokens, where E Rdd represents the embedding of the ith token. We define the next-token prediction model LM,, parameterized by 0 that assigns, at each step r, probability distribution over possible next tokens given the prefix Zer and context X. Concretely, for 1 S r m 1, the model outputs the following distribution over the v vocabulary entries via softmax operation:
LMØ(- | Zet. X) == where a; = ar. [
E EAT!
i.e. au 2 and Σx au = 1. We then form the continuous token as convex combination of all tokens in the vocabulary:
2, ==Ea,, VI
At the final step r m. the model samples a discrete token Zm E (ee,....,,, eyt from policy distribution LM, ( Zem- X) am- We note that we assume that the answer depends only on the final discrete token Zn, merely for simplicity; the same framework naturally extends to decoding multiple final dis- crete tokens after continuous ones. We refer to this decoding strategy as base CoT2 and observe that it results in deterministic reasoning chain because the continuous tokens are precisely determined by the softmax map (see Remark in Appendix for a brief discussion of its scalability). In Section 5 will introduce stochastic alternatives, such as CoT2-MTS, to facilitate generative reasoning.
3 CSFT: A SUPERVISED TRAINING METHOD FOR CoT2
In this section, present method of continuous supervised training to learn intermediate thought tokens as "soft" targets rather than "hard" target tokens. Specifically, we provide the model with convex combinations of vocabulary emheddings, which allows the model flexibility in those reasoning
3

---

Published as a conference paper at ICLR 2026
steps. Such an approach is particularly suitable when the task accuracy depends only on the final token or token distribution. Formally, at each reasoning step r = l..... n! - I, the supervision specifies target probability distribution
0; = =
Ri EΛ
where o 2 0 and Σx=1 o 1. supervision distribution o; rather than one-hot labels by using a divergence-hased loss:
We train the model to align its predicted distribution a, to the
(0: x, = D D(a; ar)).
where D (-I1-) is the cross-entropy (or equivalently KL divergence) between two distributions. This approach can also viewed as token-level crowledge-distillttiee
where the teacher distribution a is obtained through logic/search algorithm. At the final step r m, have discrete target Zm E (el....,e,, e. so that a is one-hot distribution placing probability on target token and elsewhere. This is equivalent to using a standard cross-entropy loss log LMa (zjn Zem X) the final step. Hence, for each training example, the total loss for continuous supervised training is the sum of the continuous-toke divergence losses:
Σ Lcom (0; X,
LeS X)
By minimizing LCSFT(0), we teach the model the soft targets or at each step and to predict the correct final discrete token. Inspired by the discussions in Bachmann & Nagarajan (2024); Bengio et al. (201)), we consider two ways of providing prefixes to the language model:
1. Teacher forcing:
Each step is conditioned the ground-truth prefix z meaning the model has access to all previous ground-truth tokens during prediction. Formally, for each step r' < f, the corresponding input z E a is convex combination of all vocabulary tokens.
Self-feeding: Each step autoregressively uses the model's previously generated outputs, Zzr- during training. In particular the continuous output token ž₁ = E a, is convex combination of vocabulary embeddings, which is then fed back the model part of the prefix.
It is also worth noting that one may apply temperature scaling or thresholding to a, before forming & in order to filter the model's 's predictions. In our experiments, we find that teacher forcing leads to superior performance for CSFT, even though at inference time, the model runs in an autoregressive manner, as discussed below. See Appendix for further discussion.
Inference. At inference, the model does not rely the ground-truth distributions aj Instead, at each continuous step / m, autoregressively produces the output distribution a,, converts it to continuous token 2, E ar and appends z, the prefix for the next prediction. In the final step, the model samples a discrete token from a,,, LM5( Zem. X).
Baselines. Our first baseline is discrete CoT, which uses teacher-forced training where the next token prediction is performed conditioned on the previous ground-truth tokens with standard cross-entropy loss. Discrete baseline enforces z to be a token in vocabulary (ei,.... e,l. which means that it is a special case of CSFT where the ar are one-hot vectors rather than an arbitrary element of --1 The model minimizes the following objective, which is obtained by summing over all steps of teacher-forced next-token prediction:
LS X)= Σ log LMa (z, zzz X)
Here, setting m 1 gives the discrete no-CoT baseline, where the model directly predicts the final answer. Another baseline we compare against is COCONUT, which replaces discrete tokens sequentially with the last hidden state of the LIM following a left-to-right curriculum learning strategy. In inference, the COCONUT model directly outputs the answer the final step m), following intermediate continuous thought tokens produced by the hidden state output.
3.1 TASKS REQUIRING EXPLORATION OVER STATES
In this subsection, we illustrate CSFT training described in ( on tasks that require exploration over multiple states but have single final correct state. We consider directed graph exploration problem
4

---

Published as a conference paper at ICLR 2026
where the aim is to follow m-step trajectory through reachable states starting from go and arriving at a desired state 8m: Suppose that the vocabulary is sufficiently large that each state g of the task can be assigned a unique embedding. Let I denote the set of states reachable at step building upon step (1 - 1) with To = (gol. and let 'T be the set of complete trajectories æ of length m, each inducing state sequence (g) (7),.. ................................
-......-T)))) (m)))) with gr(T) E r, Next, we describe how to obtain CSFT targets [a)) by curating a set of complete trajectories and projecting them back to intermediate steps.
CSFT Targets Through Hindsight Top-B Superposition. Let : - Rgo be task-specific final score function with lower is better, and fix a budget B 2 for the number of trajectories. We form the set HIg by filtering trajectories using F and taking the top B:
Σ F(T). nsT MM TEll
1gg
arg min
At step 1 < m, we form the supervision of by superposing the states visited by Π. and at step = m, we select one correct final state from m. so that or is one-hot vector:
1 aig B
(ig(0T)) - gl. gET, 1 my nella
1. if is the correct final state Km
Amg
(3)
0, otherwise.
This rule is retrospective in the sense that it ranks trajectories and then supervises the previous steps accordingly. If B = 1. the supervision reduces to discrete CoT using ground-trum trajectory in (2) by simply choosing F to penalize any trajectory with incorrect final answer. If = |T| is the full trajectory budget, then 1I = T, and the supervision (a becomes the superposition of all reachable states. As a result, the described methodology provides flexibility to select the number of superposed states based on model capacity and the task structure.
3.1.1 MINIMUM NON-NEGATIVE SUM TASK
We now introduce the Minimum Non-Negative Sum (MNNS) task, where the goal to assign signs to list of numbers so that their sum is small as possible while being nonnegative. The MNNS task can also be viewed as partitioning a set of numbers into two subsets with a minimal difference. which makes it closely related to the subset-sum problems explored in Dziri al. (2023) Thomm et al. (2024). Formally, given in integers di....,dm: the task to assign signs Of E [+1,-1] such that s Ji dj + .... + Fm dm 2 O and s is minimized. Let 8 = (o ..... opt denote the optimal assignment that achieves the minimal nonnegative sum s out of 2" possible sign assignments. Here, every possible partial sum ojdj= + o,d, [, is treated as a state and assigned a unique embedding (g(r)d))---dddd) where 6(-) maps each sum to a distinct id in [v]. At step 1, the state oid +..+ Or,d, is reachable from the state [dd +... 0 step 1.
We supervise the model with CSFT targets oi defined in (3) with different budgets B between 1 and [T] = 2" For MNNS, we rank trajectories by the absolute value of the final sum for budgets 1 << B x 2"", while for B = we select the ground-truth trajectory. We split the training and validation datasets by ensuring that any permutation of numbers appears in exactly one split in order to prevent memorization and make a fair evaluation. We encode input and output numbers with separate tokens in our vocabulary. As an example, an input appears as (BOS) di d2 ... -, and the corresponding output as 52 SuP (EOS), where 5d is the minimal nonnegative sum for (di...., dm). For the model, we use the GPT2 architecture (Radford et al. 2019 with different head, layer, and embedding dimension configurations, and train it from scratch. During evaluations, we only assess the final answers of both approaches. For more details on the experiments, see Appendix
3.1.2 PRONTOQA AND PROSQA DATASETS
Other datasets we explore in our investigation of the CSFT approach are the ProntoQA (Saparov & He| 2022) and ProsQA (Hao et al. 2024), which are logical reasoning tasks that require exploration over multiple paths. Each question in ProntoQA asks whether certain target word (node) 7 is reachable from a root word (node) R within a fixed number of hops, while for ProsQA it asks which of the target words T, or T2 is reachable. We use 5-hop questions and present the graph in structured format by representing nodes and edges using embeddings inside the context for each question, and use these the model input rather than text input.
5

---

Published as a conference paper at ICLR 2026
The graph structure of the ProntoQA and ProsQA tasks naturally obeys the supervision in (3). For CoT2 model, we use the full trajectory budget B = 177, so at step the target a is the weighted distribution over all nodes reachable from R in hops. thus tracking all possible trajectories without filtering In final step m, the supervision assigns probability to the correct label: yes or no for ProntoQA, and I₁ or T2 for ProsQA. For discrete CoT model (B 1), we provide the ground-truth path from A to the target node (Ti or T2) as the supervision. Please refer to Appendix C22 for additional details on supervision and data format.
3.2 RESULTS AND DISCUSSION OF CoT2 SUPERVISION
80
Vallduatoon Accuracy
CoT2 with full trajectory budget B = 17 As demon- strated in Figure [ training with full budget B [T] corresponds to forming supervision by superposing all reachable states at each step. In experiments on MNNS, ProsQA, and ProntoQA tasks, observe that the CoT2 model trained with CSFT using full trajectory budget B significantly outperforms other baselines, as shown in Figures 2bb 3 Moreover, this enables faster con- vergence as illustrated in Figure 3] In particular, when trained with full trajectory budget B, the model does not make intermediate decisions but instead conducts search over states through continuous tokens and defers the decision to the last step. This mitigates error accumulation by avoiding early commitments. Supporting this, Figure 2a applies Pass @K to both methods and shows that CoT2 consistently outperforms discrete Co'T at every k; importantly, the discrete CoT requires multiple attempts to approach the single-attempt (k 1) performance of CoT2 model, aligning with the "snowballing errors" phenomenon observed in discrete autoregressive generation (Bachmann & Nagarajan 2024). Our results also indicate that when supervision from search algorithm is available, leveraging this denser signal via CSFT is preferable to approaches that internalize discrete CoT with continuous tokens, such as COCONUT. Furthermore, once the CoT2 model is given moderate threshold embedding capacity (i.e., d 24 in Figure B to represent all capprciiggaiiiig possible states at each step, it solves the search-based tasks with near-perfect accuracy and achieves superior performance with fewer layers/heads compared to the discrete CoT model (Appendix [D.I). We also provide additional experiments on ProntoQA and ProsQA in Appendix D.I that confirm similar findings the MNNS task in Figures 2a and
60
Coff,
20-
Coff, Dra
Dierec OCC Dra
200
4.00
BS0
1000
Epurch
Figure 3: Training performance vs. embedding dimension for CoT2 (B = 16) and discrete CoT (B = 1) on MNNS with 4 input digits from 1-9 and 2-layer. 2-head GPT2 with d' (16, 24, 32).
Budget-embedding dimension tradcoff. Fig. 2c varies supervision trajectory budget B E (1, 4, 8, 16) (with [T] 16) across embeddings d E 116. 24, 32). With small embedding (d 16), the model capacity is not enough to learn large budget B 16 superposing all reachable states, whereas a moderate budget B performs best by reducing representational load relative to 16 and avoiding the per-step harder commitments of B 1.4. Note that B 16 imitates search by representing all possible states with decision taken only in the last step, whereas earlier decisions but reduces the representational load. Accordingly, performance improves monotonically with budget B and reaches near-ceiling performance when B = 16. This indicates capacity-paralle parallelism tradeoff in which larger B helps only when d is sufficiently large to represent the superposed traces.
4, forces d grows (e.g.. 24 32), the
Theoretical Perspective on Embedding Capacity. The embedding cappnciy threshold behavior in Fig. 2c aligns with the information paacaiinnnn bound we discuss in Appendix [ showing that robust decoding of a budget-B superposition over v candidate states requires embedding d Q log(v/B) in the worst case. This implies, once d > log, v, discrete CoT (B = 1) becomes iiformation-ineffiiiie as each discrete token carries at most logz bits, whereas CoT2 can pack B O (d/ log(v/ states per step, which is intuitively consistent with the entropy being z loga (B) z Blog, s d.
THEORETICAL ANALYSIS OF CoT2
In this section, we construct one-layer transformer solving the MNNS task using an attention layer followed by an MLP layer, inspired by the capability of layer, one-head transformer under CSFT supervision to solve this task with sufficient embedding dimension. We then provide a theoretical comparison the sample complexities of base CoT2, CoT2-MTS, and discrete CoT models.

---

Published as a conference paper at ICLR 2026
4.1 SOLVING THE MINIMUM NON-NEGATIO SUM TASK
Proposition 1 (Solving MNNS). There exists a 1 -layer transformer architecture with embedding dimension d de dp. where d, 2 is the state-encoding dimension and d, + 2 the posiiinnalenccoding dimension, that solves the MNNS task using CoT2 by storing sine-cosine embeddings of all 2 states at the k-th iteration in a non-overlapping manner.
The above construction utilizes trigonometric embeddings, inspired by the mechanistic insights given by Nanda et al. (2023) Our approach leverages these trigonometric embeddings to provide a theoretical guarantee that the transformer can track and add/subtract multiple numbers in negative
parallel by benefiting from the embedding capacity and reading off the minimum number at the final step. An important observation regarding our construction is that the trajectories each intermediate reasoning step are truly decoupled as it stores each state using non-overlapping (sine, cosine) representations. This closely parallels the left side of Figure [ but we also utilize rotations/shifts ensure distinct states are orthogonal and are easy to read out. Finally, as empirical validation of this theoretical result, we observe that training according to this construction with trigonometric embeddings yields perfect accuracy.
4.2 UNDERSTANDING FORMALIZING BENEFITS CoT2 COMPARISON TO CoT
In the rest of this section, we argue that CoT2 equips the model with the ability to track multiple paths in parallel, which is formalized through Assumption below. Building on this condition, we will provide a formal comparison of CoT2 and discrete CoT models in the remainder of this section. To improve expressivity, we allow a step-indexed (t) policy LM throughout, while remaining consistent with the problem setup.
Assumption 1. Recall the model LMg in Section 2 For any step and prefix lokens Zgn we assume (i) the next loken probabilities depend only on the last token 2, and the query X and (ii) if the last token is Z Σj=1 j=1 atj e so that Σ)=1 Dr.j = 1, the output distribution 0x11 decouples as follows:
L Rgr. X) LM (- Z, X) CO Σ
Orr (.
Jul
This assumption holds for inherently serial tasks, such as MNNS construction in Proposition I where previous reasoning history is summarized in the final token and different trajectories decoupled at intermediate reasoning steps. Under Assumption the token distribution a,-1 LMM z,, X)
evolves with the equation dx+1 = a, (z,; X), starting from a = LM X) until am-- Here, M,,,; X) Ruxe is Markov transition matrix that depends on the input X and the last token z, (see (Iddiz et al. (024) for related discussion). To keep exposition cleaner, we omit ž, and X in the notation of M,(2; X), and use M, instead. We start by defining the three inference strategies.
Base CoT2: At each step / = l....,m, m, the model outputs the continuous token z, = EE a, and uses it the query for the next step.
Interpretation: Base CoT2 simultancously tracks and aggregates all possible r" traces over m steps; where the trace (T has a weight of [1 aus.
Discrete CoT: At each step $ 1 m, the model samples exactly one token 2, = e, from a, and uses it as the query for the next step.
Interpretation: Discrete CoT samples a single trace out of v" traces with likelihood of [ au for trace T-)=
CoT2-MTS (multi-token sampling): At each step 1 S 1 S m, i.i.d. sample tokens ei....., eix from a, average these tokens to form z, 1 Σx e,, e., which it uses as query for the next step. Interpretation: CoT2-MTS tracks K traces in parallel according to their discrete CoT likelihoods. However, these traces are not statistically independent.
With these methods defined, we now present a result on the statistical consistency of their outputs. Proposition 2 (Consistency of CoT and CoT2 inference). Under Assumption 7 and given X, the output of base CoT2 is Zm Σj=1 (m,je) where have the same output once we take the expectation over their stochastic sampling.
a Im=- M, Discrete CoT and CoT2-MTS
Remark: The above proposition states that as the number of samples approaches infinity, the empirical distribution over the vocabulary ån obtained from CoT2-MTS or discrete CoT traces
7

---

Published as a conference paper at ICLR 2026
converges in probability am- Here, Om is the deterministic output of the base CoT2 model, which is computed without sampling and is not a random variable.
Proposition 2 establishes the statistical consistency of all three methods they estimate the same distribution 0mn- However, they differ in the samples needed to approximate this distribution. In particular, the base CoT2 model outputs the entire probability distribution over tokens at every intermediate step, implicitly tracking all possible trajectories in parallel as continuous embeddings. Consequently, it computes the exact final token distribution in one forward pass without repeated sampling. In contrast, due to stochasticity, discrete CoT or CoT2-MTS require multiple i.i.d. samples to approximate this distribution. This motivates us to study and contrast their sample complexities. The next proposition provides a distribution approximation guarantee in la distance and shows that CoT2-MTSS reduces the sample complexity of estimation compared to discrete CoT by factor of K.
Proposition 3. (i) Let am be the expected output distribution after m steps of CoT according to Proposition 2 Let @ (MTS)) be the distribution resulting from averaging the outputs of N i.i.d. Co'22- MTS traces with parallelism K. Then, to guarantee [| (MTS))
-a amlll S e with high probability, the total number of samples (traces) required scales as O(K-l ) (ii) Defining &m addise) to be the distribution
obtained by averaging the outputs of NK i..dd discrete CoT traces, we have the following upper bound on the expected error of MTS:
B BI1a (MTS)
B [|addiss) -am11]]
K
The above proposition implies that one MTS rollout tracks traces in parallel. behaving in terms of estimation error like K independent discrete CoT traces. Recall that CoT2-MTS generalizes discrete CoT, which corresponds to K = 1. For this case, the proposition reduces to the known ( sample
complexity of approximating v-category distribution in l distance (Kamath et al. 2015). Note that as K - co, CoT2-MTS converges to the base CoT2, and the proposition recovers the one-shot performance of base CoT2. Thus, although the three models yield the same final distribution, discrete CoT requires O(K) times more rollouts than CoT2-MTS with parallelism K to achieve similarly accurate approximation, due to inherent noise from single-token sampling. contrast, the base CoT2 model carries the entire mixture of partial expansions at each step and computes the distribution in one shot. This theoretical intuition aligns with empirical findings the Pass@k experiments, where CoT2 achieves comparable performance to discrete CoT with substantially fewer samples.
5 REINFORCEMENT LEARNING METHODS FOR CoT2
In this section, we present two sampling methods (Multi-token and Dirichlet sampling) to apply reinforcement learning (RL) with continuous output tokens. Specifically, we explore Group Relative Policy Optimization (GRPO) training on top of (i) discrete models and (ii) continuous models that supervised trained following the previous section for the MNNS, ProntoQA, and ProsQA tasks. Concretely, we demonstrate that RL (a) adapts discrete SFT models to produce continuous outputs via MTS/Dirichlet rollouts, and (b) sharpens CoT2 models by prioritizing relevant reasoning traces through reducing entropy of continuous token representations rather than weighting them equally. We note that the existing literature in RL operates in the model's native discrete action space over finite vocabulary to maximize a scalar reward on the generated sequence (Ouyang et al. 2022 Shao et al. 2024). In contrast, we perform RL in a continuous action space, which is the linear combination of token embeddings. to maximize the same scalar reward.
In our setup, a language model LM, acts as a policy over tokens. Let 12 be a group of G trajectories sampled from old policy LM where each trajectory z = (z() ..... Zm (0) contains m output tokens for fixed input X. We assume sparse reward setting where the reward is for correct final answer and 0 otherwise. Let A,, denote the advantage estimate at step / in trajectory and under sparse reward setting, A. = Å, identical across all steps. To quantify how the new policy LM,
LM,(4/ LM [g,x
differs from the old one on token z! in ith trajectory, we define the policy ratio 7()(0)
We update the model by minimizing the objective (Shao al.
2024 Yu et al.
2025):
(0)
min (,,0 ALo clip dip(? -- e, Au) B DKL [LM LM
8

---

Published as a conference paper at ICLR 2026
As the output length is fixed in our setting, we have | for each trajectory. Here, clips the ratio r0), and / controls the KL-divergence from a SFT-initialized reference policy LM We set GRPO iterations μ and estimate KL divergence with Schulman Approximator (Shao et al. 2024).
5.1 MULTI-TORE SAMPLING
We emulate the rollout of a continuous token by sam- pling a fixed number of K discrete tokens and averaging them at steps I.....m 1. We refer to this hybrid method as Co12-MTS (multi-token sampling). For the GRPO objective, we propose calculating the policy ratio for continuous tokens as follows. Assume that at step I, the old policy samples K discrete tokens ei....., Ci, from apld Let a......., Gi denote the probabilities assigned to these same sampled tokens by the current policy. We define the policy ratio for continuous steps by dividing geometric means:
ProsQA
ProntoQA
SET
SET-
SET
SPT+
CoT2
93.37
93.83
75.36
76.15
Discrete CoT
68.50
68.24
59.58
62.28
CaT2
93.37
94,09
75.36
76.66
Discrete CoT
59.58
71.53
CaT2
93.37
94.21
75.36
77 64
Discrete CoT 68.50
72.76
59.58
74.03
Table 1: Validation accuracics on ProsQA and ProntoQA for CoT2 and Discrete COT. evaluated at K = 6. 8, 12 with CoT2-MTS sampling GRPO. All models
4-layer, 4-head GPT2 with embedding dimension 32. Remarkably, GRPO with MTS sampling scheme results in consistent improvements.
1/K
LMa (Ze Zeci X)
11 old old
(#)
MM (Z, Zetz X)
for I,...,,. 1. The geometrie mean ensures the ratio for each continuous step remains on the same scale as the final discrete token's 's ratio and, thus, helps avoid overly large or small updates and stabilizes GRPO training compared the direct multiplication of probabilities. Once this ratio is computed, we average the K sampled tokens to form z, which is fed to the model as the query for the next prediction step. At the final step = m, where the token um = ej is discrete with j e [v] denoting its index, the policy ratio is simply the probability ratio of selecting that token:
L.Mg (Im Zem. X)
mm (0) LM (Im Zamss X)
(5)
oold m.j
Inference. After GRPO training, we apply the multi-token sampling procedure at each of the first m 1 steps to form the continuous token via the average of K sampled embeddings.
5.2 DIRICHLET SAMPLING
In this section, we present another method for generat- ing continuous tokens each step by interpreting the
Val.Ase: (9))
Enasgy (SF7 SFT-GRPO)
SPT+GRPO
Jcken)
token:
tnkets
tokee,
model's output distribution a, E A as concentration parameters of Dirichlet distribution. We introduce 0 and define the Dirichlet distribution with parameters y a = (ya ya,,). Without this scaling, directly using a, as parameters often causes training instability, particularly when many a, values are small. We then sample a point à E - from the resulting distribution Dir (yar)) After sampling, we form the continuous token by mapping 2 = E à E Rd, which becomes the query for the next step. We denote the Dirichlet densities induced by current and old policies as folã;; ya,) and define the policy ratio continuous step m as:
0.32
0,6
0.55
39.76
52
0.00-
0.4
scaling hyperparameter
51
55.66
0.00
0.04
0.35
0.4
0.1
0.00
0.15
Table 2: Validation accuracy and token-level entropy of CoT2-MTS GRPO discrete CoT model across rollout sizes K for MNNS. We use 4 input digits in 1-9; 1 1-layer, 1-head GPT2 with embedding 24 and 32; SFT accu- racies are 39.76% and 43.50% respectively.
(āu ya,"!) respectively. Accordingly, we
LMd(z, Ze. X) San (d;;; plo)
folit;
(0)
LM (z, Zec. X)
The above definition parallels the probability ratio for discrete actions, but replaces the categorical pmf with continuous Dirichlet pdf. At the final step m, we sample discrete token 2m (ei....,e,, e from ar and use the standard policy ratio in (5). At inference, we follow the aforementioned autoregressive procedure by forming z, E a.

---

Published as a conference paper at ICLR 2026
5.3 RESULTS AND DISCUSSION POLICY OPTIMIZATION CoT2
MNNS evaluation: Table demonstrates that, for each K E (1,3,6), CoT2-MTS significantly improves validation accuracy relative to the discrete SFT baseline (39.76%), with moderate K yielding the best performance. We also observe that smaller K-values correspond to larger reductions in token-level entropies, suggesting that the model becomes more confident in each intermediate step by learning to commit to fewer tokens. This suggests curriculum on K K-starting small and gradually increasing- could potentially further improve the training on the MNNS task. Interestingly, the third token's entropy remains relatively high, which might indicate that the model hedges among partial expansions at this step that preserves useful diversity. Therefore, CoT2-MTS enables a discrete CoT model to produce continuous outputs and improves its final performance.
Pr and ProntoQA evaluation: Table T shows the benefits of GRPO with CoT2-MTS on models trained with discrete or continuous SFT. Remarkably, both CoT2 and discrete CoT models consistently improve across all rollout sizes (K = 6, 8, 12), with larger K values yielding better final accuracies by promoting more exploration. Notably, the discrete CoT benefits more from RL training compared to CoT2, likely because the CoT2 already internalizes exploration through CSFT training. Aligning with this, for ProntoQA task, the final performance of discrete CoT approaches that of the CoT2 model. Finally, gains on MNNS are smaller than on highly structured and closely aligned with the CSFT targets, leaving limited headroom for RL.
likely because MNNS
In Appendices Cnd D.2 we provide additional results showing that GRPO with Dirichlet sampling further improves CSFT-trained CoT2. We also report preliminary GRPO results on GSM8K with Qwen3-0.6B 0.6B,, where CoT2-MTS consistently improves over discrete CoT while reducing response length, suggesting that exploration in the continuous action space via MTS a promising direction.
5.4 RELATED WORK
The efficacy of eliciting reasoning in LLMs through chain-of-thoughh (CoT) prompting has been
well-established (Nye et al.) 2021; | |Wei al. 2022] Kojima al.] 2022 Suzgun al. [023 Guo
et al.] (2025). ). CoT prompting conveniently increases inference-tinge time compute and computational depth. both are independently useful (Płau et al. 2024: Goyal et al. 2024 Feng et al. 2023; Merrill & Sabharwal 2024). However, the discrete nature of CoTl tokens forces sequential exploration of rea- soning paths, resulting in longer reasoning paths and consequently increased inference-time compute. Furthermore, restricting reasoning to natural language can be inefficient, groups of tokens can often be more effectively represented by single continuous token. Thus, CoT2 offers an alternative strategy for compute-efficies reasoning and complements methods that aim to shorten/control the trace length of CoT (Aggarwal & Welleck 2025 Zhang et al. 2025a Sui et al. (025).
Our work is most related to recent body of work introducing L.MSs capable of reasoning with explicit continuous tokens decoded autoregressively. In particular, recently proposed COCONUT (Hao et al. 2024) autoregressively feeds the last token's final-layer representation input to the next step. Given labeled CoT data, COCONUT is trained to progressively replace discrete tokens with continuous tokens (from left to right). While COCONUT and our CoT2 aim to reason in continuous space, we propose distinct algorithmic approaches that also address the exploration challenge. Key differences include: (1) Our continuous tokens are -weihtee compositions of vocabulary tokens. (2) Our supervision method is novel and explicitly targets implicit parallelism. (3) CoT2 does not initialize from, nor attempt to mimic, discrete COT. (4) By introducing sampling strategies and associated GRPO variations, we realize the "Supervised Training - Reinforcement Learning" paradigm in the context of CoT2. A concurrent work Zhu et al. (2025) provides a theorctical construction for continuous chain of thought graph reachability problems, which is at a high level similar to our supervision CSFT. We provide further discussion of the literature on multi-token prediction and reinforcement learning in Appendix B
CONCLUSION
In this paper, we provided thorough theoretical characterization of CoT2 spanning the achievable level of parallelism and dimension-budeee tradeoffs, CoT2-specific capabilities of transformer, and the statistical benefits of CoT2. Our theory is developed in tandem with novel continuous supervision (CSFT) and reinforcement learning strategies. As future work, applying CSFT selectively on segments of the L.L trajectories represents a promising direction for equipping LLMSs with CoT2 capabilities.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work is supported by the National Science Foundation grants CCF-2046816, CCF-2403075. CCF-2212426, the Office of Naval Research grant N000142412289, gift from Open Philanthropy. and an Adobe Data Science Research Award. The computational aspects of the research are generously supported by computational resources provided by the Amazon Research Award on Foundation Model Development and by the Google Cloud Research Credits program with the award GCP440569534 on Continuous Chain-of Chain-of-Thougeee of-Thought Models.
REFERENCES
Pranjal Aggarwal and Sean Welleck. L1: Controlling how long reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2503.0400 2025.
Gregor Bachmann and Vaishnavh Nagarajan. The pitfalls of next-token prediction. In Forty-first International Conference Machine Learning, 2024. UtLL ferrrmmmmm
id=76zg8Wk16Z
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks, 2015. B.RLittps//arxvvvorr////
03099
David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent dirichlet allocation. J. Mach. Learn. Res,, 30mu1))9933110022 March 2003. ISSN 1532-4435.
Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through 2024.
dense representations. arXiv preprint
Yuntian Deng, Kiran Prasad, Roland Fernande7, Paul Smolensky, Vishrav Chaudhary, and Stu- art Shieber. Implicit chain of thought reasoning via knowledge distillation. arXiv preprint arXiv:2311.01460, 2023.
Yuntian Deng, Yejin Choi, and Stuart Shieber. From explicit cot to implicit cot: Learning to internalize cot step by step. arXiv preprint 2rXiv:2405.14883 2024.
Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D. Hwang, Soumya Sanyal, Sean Welleck, Xiang Ren, Allyson Ettinger, Zaid Harchaoui, and Yejin Choi. Faith and fate: Limits of transformers
compositionality, 2023. uttp::ts::// httttp:://axxi.oor//baass hts//b0ss230555 18654
Guhao Feng. Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, and Liwei Wang. Towards revealing the mystery behind chain of thought: A theoretical perspective. In Thirty-sevnnth Conference on Neural Information Processing Systems, 2023. URL htt.ttts////oeeeeiiieeee wttp://uutteeoemiieeiea qHrADgAdYu
Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach. arXiv preprint arXiv:2502.05/71, 2025.
Angeliki Giannou, Shashank Rajput, Yong Sohn, Kangwook Lee, Jason D. Lee, and Dim- itris Papailiopoulos. Looped transformers as programmable computers. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 11992-11442. PMLR, 23-29 Jul 2023. URL www.sames.....eeee.eeeeeee
https://proceediins. mlr. www.prresssvee siieess/vv2///e /ttp:///22/////2222aaaa html
Fabian Gloeckle. Badr Youbi Idrissi, Baptiste Rozière, David Lopez-Paz, and Gabriel Synnaeve. Better & faster large language models via multi-token prediction. In Proceedings of the 41st International Conference on Machine Learning, ICML''24. JMI.Rorrg 2024.
Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh
Nagarajan. Think before you speak: Training language models with pause tokens. In The Twelfth International Conference on Learning Representations, 2024, URL pettps://openeeet net/ forumPiid-hoo-cRRPac
11

---

Published as a conference paper at ICLR 2026
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXi::2501.12945 2025.
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprint (7Xi::2412.06669, 2024.
M Emrullah Ildiz, Yixiao Huang, Yingcong Li, Ankit Singh Rawat, and Samet Oymak. From self-attention to markov models: Unveiling the dynamics of generative transformers. International Conference on Machine Learning, 2024.
Aaron Jaech, Adam Kalai, Adam Lever, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint arXiv:2412.16720, 2024.
Sudeep Kamath, Alon Orlitsky, Dheeraj Pichapati, and Ananda Theertha Suresh. On learning distributions from their samples. In Peter Grünwald, Elad Hazan, and Satyen Kale (eds.), Proceedings of The 28th Conference on Learning Theory, volume 40 of Proceedings of Ma- chine Learming Research. pp. 1066-1100, Paris, France, 03-06 Jul 2015. PMLR. URL
https://proceeedings mlr www.erppeesseeeeeaeeee th15 .html
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35: 22293, 2022.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng. Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.
William Merrill and Ashish Sabharwal. The expressive power of transformers with chain of thought In The Twelfth International Conference on Learning Representations, 2024. URL /ttps://
openreview httpviiwww.eeeeeeeeee wttt/ffoummiicem J
Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretabiliiyy 2023. URL https::/aaxii..ooo wwtp:////raabbb//330// httpppss//axii.oorr///////...... 05217
Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowyez, Maarten Bosma, David Luan, et al. Show your work: Scratchpads for intermediate computation with language models. arXiv:21 /2t::2//2.01110 14, 2021.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
Jacob Pfau, William Merrill, and Samuel R. Bowman. Let's think dot by dot: Hidden computation in transformer language models. In First Conference Language Modeling, 2024. URL https //openreview. net/ BadrummiidNNibbbdtWWe
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models unsupervised multitask learners. OpenAl Technical Report, 2019. URL hattps://cdn.do owwnai.eeii.....o coteemai.com/aae www.wootteeteeeeeeeeeeeeeeeee wwweeeeeee-taggggeeeeeeeeee www.llaangggudlleeo modell//anguage_moott are_ sespuperriseed_mlltttsstes. www
pdf
Abulhair Saparov and He He. Language models are greedy reasoners: A systematic formal analysis of chain-of- thhtuughh.. arXiv preprint arXiv:2210.0/2400 2022.
Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, and Sashank J. Reddi. Reasoning with latent thoughts: On the power of looped transformers. In The Thirteenth International Conference on Learning Representations, 2025. URL
Yuval Shalev, Amir Feder, and Ariel Goldstein. Distributional reasoning in 1lms: Parallel reasoning processes in multi-hop reasoning. arXiv preprint arXiv:2406. 13858, 2024.
12

---

Published as a conference paper at ICLR 2026
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits mathemat- ical reasoning in open language models, 2024.
Noam Shazeer. Glu variants improve transformer. arXiv preprint arXiv:2002.05202. 2020.
Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. Codi: Compressing chain-of-thouud -hainhhf-thought into continuous space via self-distillation. arXiv preprint arXiv:2502.21074, 2025.
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, and al. Mastering the game of go without human knowledge. Nature, 550(7676):354 359, Oct 2017. doi: 10. 00.103//atture24270.
Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. Learning to summarize from human feedback, 2022. URL
https:// http:///axivv..rrogg org/ abagg//bs/200 http//bbb/200.01333
Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503. 16419, 2025.
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoe Le, Ed Chi, Denny Zhou, and Jason Wei. Challenging BIG-bench tasks and whether chain-of-thought can solve them. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 13003-13051, Toronto, Canada, July 2023, Association for Computational Linguistics. doi: findings acl
10. 18633/v1/2023 1/2.5///////222..nnngg-----e 8224. httpttpp///aclaatooooooo.ooo//// 824/
Jonathan Thomm, Giacomo Camposampiero, Aleksandar Terzic, Michael Hersche, Bernhard Schölkopf, and Abbas Rahimi. Limits of transformer language models on learning compose algorithms, 2024. URL httpp::/aaxx...r///bbbssss/
05785
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11172 2022.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837 2022.
Zheyang Xiong, Ziyang Cai, John Cooper, Albert Ge, Vasilis Papageorgiou, Zack Sifakis, Angeliki Giannou, Ziqian Lin, Liu Yang, Saurabh Agarwal, et al. Everything everywhere all once: Lims can in-context learn multiple tasks in superposition. International Conference on Machine Learning, 2025.
Sohce Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel. Do large language models latently perform multi-hop reasoning? In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp. 10210-10229, Bangkok, Thailand, August 2024, Association for Computational Linguistics. doi: 10. 10.18663/1//2224..ddooo.....
URL Rtttsttss////catttin
org 201 550/
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:1 1809-11822, 2023.
Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. Distilling system 2 into system 1. arXiv preprint arXiv:2407 06023,, 2024.
13

---

Published as a conference paper at ICLR 2026
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue. Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin. Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong. Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Weinan Dai, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei- Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang. Dapo: An open-source Ilm reinforcement learning system at scale, 2025. URL htt.tttss//arri...rr////////33 www../orraabbb/23...22 14476
Zhenrui Yue, Bowen Jin, Huimin Zeng, Honglei Zhuang. Zhen Qin, Jinsung Yoon, Lanyu Shang, Jiawei Han, and Dong Wang. Hybrid latent reasoning via reinforcement learning, 2025. URL.
https://arxiv.org/ wtwp:////.rgg///o///aooooo 1845.
Xuechen Zhang, Zijian Huang, Chenchun Ni. Ziyang Xiong, Jiasi Chen, and Samet Oymak. Making small language models efficient reasoners: Intervention, supervision, reinforcement. arXiv preprint arXiv:2505.07991, 2025a.
Zhen Zhang, Xuehai He, Weixiang Yan, Ao Shen, Chenyang Zhao, Shuohang Wang. Yelong Shen, and Xin Eric Wang, Soft thinking: Unlocking the reasoning potential of Ilms in continuous concept space, 2025b. URL https://arxiv org/ abs/2505.
Hanlin Zhu, Shibo Hao, Zhiting Hu, Jiantao Jiao, Stuart Russell, and Yuandong Tian. Reasoning by superposition: A theoretical perspective chain of continuous thought, 2025, URL https; / warxiv.org/abs/2253 12514
14

---

Published as a conference paper at ICLR 2026
APPENDIX
We discuss additional related work in Appendix B We provide further implementation details in Appendix [C] including those for the MNNS task (Appendix[C.) the ProntoQA/Pross datasets
(Appendix C.2p and GRPO training (Appendix C.4). We present additional experimental results in
Appendix D and we offer details on continuous supervised training and GRPO in Appendix [D. and
Appendix D.2 respectively. Finally, we include the proofs of Propositions and in Appendix
A
LLM USAGE
We used large language models solely as assistive tools. They provided support for text clar- ity/grammar, coding utilities, ETEX edits, and literature search. The LLM did not design algorithms. implement core methods, choose hyperparameters, or run experiments. All scientific claims, figures, math, and code were authored and verified by the authors.
B FURTHER RELATED WORK
One way to address the limitation of discrete tokens is by leveraging the implicit reasoning capabilities
of transformers (Yang et al. 2024] [Shalev et al., 2024). Works such (Deng et al., 2023 (224] Yu
et al. 2024) use various techniques to obtain models that can perform reasoning internally without emitting CoT tokens. Another line of work has found looped transformers to be effective on reasoning
problems Giannou et al., 2023, Geiping et al. 2025), notably being able to mimic CoT (Saunshi ct al. 2025) with a sufficient number iterations. Another way to address these challenges is to incorporate explicit continuous tokens similar to COCONUT (Hao et al. 2024). Specifically, (Shen et al. (2025) proposes CODI, where an LLM with continuous CoT is supervised to produce the correct answer, while also aligning its hidden representation on the last reasoning token to that of a discrete CoT model that shares the same backbone. Cheng & Van Durme (2024) propose CCOT, where an auxiliary module is first trained to decode autoregressively a compressed representation of discrete CoT trace, and later the main LLM is fine-tuned to produce correct answers by additionally conditioning on the generated continuous tokens. Our work is similar to this line of work in that continuous representations are used to perform reasoning. In parallel with our work, /haag et al. (20256) propose Soft Thinking which is training-free inference scheme that replaces intermediate discrete tokens with seftmax-weighted concept tokens and uses an entropy- based early stop: unlike CoT2, we introduce explicit continuous supervision (CSFT) and RL with multi-token sampling (MTS/GRPO) to track multiple trajectories with controllable parallelism.
The proposed CoT2 approach simultaneously tracks all possible trajectories and superposes them within continuous tokens. This approach is similar to that of Xiong et al. (2025), who superpose multiple candidate outputs into a single final token. Our approach also shares similarities with decoding algorithms like seff-consistency (Wang et al., 2022) and Best-of-N-Se Best-of-N-Sampling -Samling (Stiennon et al., 2022), which generate multiple trajectories by running inference multiple times and then select final answer based on the aggregate statistics. In contrast, our algorithm performs a single inference, superposing different trajectories all at once and determining the final answer in one pass. concurrent work by Yue al. (2025) explores hybrid latent RL using learnable gating mechanism, where continuous tokens are directly linked to model parameters and thus each trajectory can only be used for single gradient update due on-policy constraints. Whereas, our MTS strategy integrates stochasticity within continuous tokens and makes exploration possible through continuous tokens. Furthermore, our Dirichlet sampling approach for generating multiple rollouts in GRPO training draws connections previous works such Latent Dirichlet Allocation (LDA) (Blei et al. 2003). which introduces Diríchlet priors within a hierarchical Bayesian framework, and AlphaGo (iilerr et al. 2017), which injects Dirichlet noise encourage exploration.
Our work also tangentially relates to research on multi-toker tokti-tooenn prediction (Bachmann & Nagarajan 2024; Liu et al. 2024: Gloeckle al. 2024), which aims to improve the efficiency and quality of generation by predicting multiple tokens at once. It is hypothesized that effective future prediction necessitates the exploration of many possible continuations, which is similar to our CoT2 approach.
15

---

Published as a conference paper at ICLR 2026
C IMPLEMENTATION DETAILS
Computational Resources: All experiments were run on Slurm-managed cluster using L40S GPUs with 48GB of memory. Each experiment fits on a single GPU. In the case of 4 input digits, SFT or CSFT training takes approximately 3 hours on a single GPU. For 5-digit inputs, the dataset size increases by roughly a factor of 10, and the training time increases proportionally. The entire codebase implemented in PyTorch.
Remark (Scalability of base CoT2 decoding.). Computing the continuous token E'aa, adds a single additional O(yd) matrix-vecte vector multiply per step. This matches the O(vd) cost already incurred by the standard output projection that produces the logits l, Wouth, and distribution 0, = (oftmax(()). Moreover, this additional cost is independent of the context length N and is dominated by the transformer stack's 's per-token compute, so in practice the overhead is minor even for larger language models.
C.I IMPLEMENTATION DETAILS OF EXPERIMENTS ON MNNS TASK
Dataset Details: For the MNNS task, the vocabulary consists of a range of numbers from [-S.S] for some positive integer 5, together with <BOS>, <EOS>, and - special tokens. The integer S is chosen that all possible partial sums of the selected input digits lie within [-S,S]. For example, when the input digits lie in the range 1-10, we set 36, whereas for digits in 5-14, we set S = 40. We performed experiments the 4 and 5 input digit scenarios. A sample input line with numbers is:
<BOS> D1 D2 Dm +
Accordingly, the output will be sum tokens, where the final token corresponds to the answer, followed by <EOS> token:
S Sm <EOS>
As a concrete example, consider the input 2, 1.4 (m = 3). following Figure | In this case, the solution for the MNNS task is -2 input is <BOS> D2 D, D4 - and we supervise it along the trajectory of correct output tokens S S <EOS>, as illustrated Figure On the other hand, the continuous supervision at the first step holds 22 and S possibilities. Then, for the next step, add -1 to these numbers, and the resulting possibilitics are 5. S. S., Finally, at the last step, the model is supervised to pick the correct answer S the token.
Therefore, for the discrete model, the
We split the datasets by ensuring that each permutation of a set of numbers is exactly in one of the train and validation datasets, the answer to the question is cermutation-invariin This way, prevent the models from memorizing the answer and make a fair comparison. We also use 0.8-0.2 split for train-val datasets.
Model and Hyperparameters: We use the GPT2 model, with layer head, layer 2 head, and experiment with embedding dimensions
layer 4 head the configurations. For each configuration, of 16, 24, or 32, We train with a learning rate of 1r = 10-4 and use AdamW (no weight decay). The batch size is 16 for 4-digit inputs and 64 for 5-digit inputs.
Evaluation of the models:
To make a proper comparison, we only check the final answer of the models, as checking the correctness of the full path of the discrete model would be unfair.
Pass @ Experiment: We perform our experiments for temperatures 0, 0.4, 0.8, and I by repeating the evaluation 10 times for each k value where k changes from to 14.
C.2 IMPLEMENTATION DETAILS OF EXPERIMENTS ON PRONTOQA/PROSQA DATASETS
Dataset Details: Different from the original ProntoQA/ProSQA datasets which described the struc- tured ontology in natural language as a of known conditions, through a token-level representation. An example prompt is shown below.
structured format
Description of the structured ontology: Each component of the ontology and associated questions is represented through discrete tokens with their own learned embeddings, rather than as raw textual input. Specifically, the GPT2 architecture and encode the ontology's structural components.
16

---

Published as a conference paper at ICLR 2026
Below examples demonstrating how natural-languat assertions mapped to tokenized format:
Brimpuses are not luminous "A" 'not in' 'B'
Shumpuses are amenable; Each yumpus is a lorpu; Every lorpus is floral -
'C' 'in' 'D' '.'.
Below, we have the ProntoQA and ProsQA datasets' input-output format.
The structure of ProntoQA:
Input: 'eeccipptio'' '{' 'A' 'not in' 'B' 'Question' '(' 'C' 'not in' 'F' '.' '3'
'C' 'in' 'D' '.' '
Output: Steps' { 'C' 'in' 'D' 'False' '}'
'D' 'in' 'E" " 'Answer' '{'
The structure of ProsQA:
Input: 'Deccription' '{' 'A" 'in' 'B'
'C' 'in' 'D' '.' '3'
"Question' '(' 'C' 'in' 'F" 'or' 'E' "
Output: Steep'' { CC' 'in' 'D' '.' '3'
'D' in 'E' '.' ' 'Answer' '{' 'F'
Each distinct component relation (e.g.. 'A', 'in', 'not in' is treated unique token, and singular/plural variants (such as 'lempus' and 'lempuses' ") are collapsed into a single token to simplify the vocabulary. Alongside these concept tokens, special structural tokens (" ('Description' ", ", ...", 'or', etc.) are also included, which results in a vocabulary size of 31 tokens. To avoid biases, balance the dataset. In ProntoQA, "yes" and "no" each appear with 50% probability, and in ProsQA, the correct answer is randomly permuted at the first or second position. For all the other experimental and training settings, follow [Hao al. (2024).
Model and Hyperparameters: We use the GPT2 model, with layer 2 head, and 4 layer 4 head as the configurations. We tested embedding dimensions 24, 32, 40 with these configurations. We set batch size 64. We train with a learning rate of 10 and use AdamW (no weight decay).
Maj @k Experiment: We majority voting for evaluation instead of Pass@k, because both ProntoQA and ProsQA are binary questions. We perform our experiments for temperatures 0, 0.4, 0.8, and by repeating the evaluation 10 times for each & value where changes from 1 21. If two or more answers end up with the same top vote, we pick one randomly.
C.3 IMPLEMENTATION DETAILS BASELINES
In Figure we evaluate the following baselines under matched total number of epochs:
Discrete no-CoT. The model is trained to predict the final answer directly, without intermediate thought tokens.
Discrete CoT, The model is supervised the full, correct path during SFT training.
COCONUT, Using left-to-right curriculum training, the thought token at step is replaced by the model's 's last hidden state sequentially. During inference, the model outputs the answer at the final step (/ = m) after m continuous thought tokens. To keep the evaluation fair, we use the same total number of epochs across all baselines. For COCONUT's curriculum stages, we divide the total epochs evenly across curriculum stages, e.g., with 1000 epochs and m = 5 output tokens, we train 200 epochs for each stage.
C.4 IMPLEMENTATION DETAILS OF GRPO TRAINING
In Hao et al. (2024), the reference model is updated by LM... LM, in each iteration (epoch). This approach is reasonable for their setting with a large dataset and small number epochs over For our setting, however, the reference model the initial model and never update it through
17

---

Published as a conference paper at ICLR 2026
Algorithm 1 Multi-Token Sampling GRPO for Continuous Token Generation
Input: Initial policy LM hyperparametters K. C.m,
1: LM,,, - AM
2: for iteration = 1,2, .... and for step 1, do
Sample batch inputs (A'")) 10-1
4: Update a. L.M,
5: for each input X in the batch and for cach trajectory == I..... G from that X do
6:
for each token step I...,, m de
7:
if then Sample tokens (e,,.................... e from at (h.old create continuous token a, - Σx e,
Continuous token
Policy ratio for continuous token r,(0) - (I1,-1 d / /1T== oh jfl.oiikk else
9:
Final discrete token
Sample Z.. = e from a'
(3,odd
12:
Policy ratio for discrete token mm) / Ghoodd Obtain advantage estimates Å, for each token in each trajectory n and calculate objective.
13:
14:
Update 0 to minimize LoRro(0).
Output L.M,
iterations as we have a smaller dataset. Mcanwhile, we update the old model before every batch LM - LMg.
In our experiments, use G trajectories per input data point, use clipping parameter e 0.1, and set the KL-divergence coefficient B = O in most cases (with / = 0.1 in a few). For the CoT2 model with MTS sampling, we change the number of tokens sample K from the 5-digit case has about ten times more data than the 4-digit case, so we typically focus 4-digit MNNS because of computational considerations and use a batch size 16 in those experiments.
12. In the MNNS task,
Learning rates differ by model and setting. We use 1r = 5x 10 for CoT2-MTS sampling (figures in the main text). 1r = x 10-5 for discrete CoT with Dirichlet sampling, and 1r = x 10 for CoT2 with Dirichlet sampling. For ProntoQA and ProsQA experiments,
perform grid search over x 10 and select and report results using the best-performing find Ir x 10-5 optimal; however, for CoT2 and discrete CoT
learning rates ranging from x 1 configuration. For most settings, models with K = 6, we set 1r x 10-6, We also use Adam with a weight-decay of 0.01. For Dirichlet experiments on the MNNS task, try various scale parameters y, but find y 20 to work best in most settings. Unless stated otherwise, we report the best validation accuracy found during training for each setting.
C.4.1 FURTHER DETAILS oN MULTI-TOEN SAMPLING
The full algorithm describing MTS sampling is provided in Algorithm T
Remark. An alternative the normalization in (4) is scaling the logits by 1 1/K before applying softmax. However, this leads to a distribution shift from the SFT-trained model at inference, and ultimately degrades performance.
D EXPERIMENTAL RESULTS
D.1 CONTINUOUS SUPERVISED TRAINING RESULTS
Teacher Forcing and Self-feeding Comparison: As described in CSFT section, we tested two approaches of providing prefixes during training the CoT2 model with CSFT. Although the model autoregressively generates at inference time, teacher forcing yields better performance than self- feeding during CSFT training. We also tested curriculum settings, where we switch to self-feeding after drtemmneddd number of epochs in the training. Still, the accuracies didn't 't improve beyond pure teacher- teacher-forcing training. The results are illustrated in Figure where we refer to teacher-forcing and refer to self-feeding "soft-teacher".
Sparse Supervision for Discrete Baseline: We also tested providing a subset of the correct path to the discrete model. We observed that sparsely supervised discrete model can achieve better
18

---

Published as a conference paper at ICLR 2026
Method
MNNS
ProsQA
ProntoQA
CoT2
98.94
93.37
98.01
COCONUT
92.58
90.03
96,94
Discrete CoT
84.92
68.50
82.47
Discrete no-CoT
68.35
54.91
73.65
Table 3: Validation accuracies on MNNS, ProsQA, and ProntoQA for CoT2, COCONUT, Discrete CoT, and Discrete no-CoT. Setting: MNNS with 4 imput digits in 1-9; CoT2 full budget and Discrete CoT uses ProntoQA: 4-layer, 4-head GPT2 (d=32).
B=1. MNNS: 2-layer, 2-head GPT2 (d=32); ProsQA
performance than the fully supervised discrete model when the distribution is "easier" to handle by the model. As an example, we tested the case when we have 5 input digits from the range 11 to 19. In this case, in nearly all of the cases, the answer to MNNS game is (sum of minimum numbers) (sum of maximum 2 numbers) out of the 5 input numbers. In this case, when only token from the correct path is provided to the discrete model, it's better than 3 and 5 token cases. However, when we change the distribution to a range of numbers from 5 to 13, which makes the question reasonably harder, the discrete model with token supervision performs worse than the other two, and the discrete model with full supervision performs best. The results are demonstrated in Figure 4
The results Figure also indicate that above an embedding dimension threshold, the CoT2 model has superior performance and trains significantly faster than the discrete CoT model. Moreover, combining the results Figure 7 with Figure 3 we see that the CoT2 model with one layer and one head GPT2 model performs better than discrete CoT model with two layers and two heads at embeddings 24 and 32. While the continuous approach requires greater embedding capacity support its distributional representations at cach step, it can outperform the discrete model using fewer layers and attention heads. Supporting the findings in Figure [ Figure illustrates Jillstaatlsss that on the ProntoQA task, CoT2 consistently outperforms the discrete CoT baseline when the embedding dimension above a threshold. Likewise, as depicted in Figure 9 and Figure 10 the discrete CoT model requires multiple samplings (Maj @k) match the single-shot performance of CoT22 on both ProntoQA and ProsQA, which indicates that CoT2 model is more sample-efficient.
Further Results on CoT2 vs Discrete CoT:
Empirical Evidence of CoT2 Alignment with Supervision During Inference: We provide direct empirical evidence supporting that well-trained CoT2 models indeed follow the intended search patterns prescribed by continuous supervision (CSFT). Specifically, we analyze token-level entropy as probe for model behavior during inference on the MNNS validation set. At each intermediate reasoning step CSFT ideally prescribes uniform distribution over 2' partial sums, yielding an entropy H, = / 2. On the MNNS validation set with four digits we measure token-level entropies obtained from inference with 4 input digits (1-9) using l-layer, 1-head GPT2 architecture at embedding dimension 32. The measured entropies at each intermediate step are:
H1 0.6896, H2 1.3682, H; 1.9588, H4 0.2461.
Observe that the token-level entropies closely match expected theoretical values: H1 0.6896 In H₂ 1.3682 as 4n4, H; = 1.9588 = In 8, and a sharp drop at H4 0.2461. The alignment for steps 3 indicates that the model maintains approximately uniform superposition over roughly 2, 14.8 8 trajectories, rather than collapsing into single path or averaging them. The steep entropy reduction at the final step 4) clearly reflects the one-hot supervision entropy pattern provides direct empirical evidence that CoT2 inference aligns precisely with the CSFT training objective: the model explores multiple reasoning branches in parallel until committing decisively to the correct outcome at the final step.
the final answer. This
19

---

Published as a conference paper at ICLR 2026
Validation Accursr
00
30
Raizees 1 1ilen
Tpach
Figure 4: The figure illustrates that when the range of digits makes the question non-trivial an MNNS task, the discrete CoT model trained with full token supervision outperforms sparse supervisions; in particular, single token supervision yields the worst performance. Setting: 5 input digits 13; 2-layer, 2-head GPT2 with 32.
100
Validation Accuracy (%)
4.
Discrwta SPT, 8mh 24
Coetinuous Lmb
Continuous SFT, Emb 40
Discrete Si"l tim
200
4.00
600
800
1000
1200
1400
Epoch
Figure 5: The figure reveals that CoT2 model superior to discrete CoT in ProsQA, while also exhibiting faster convergence. Setting: 4-layer, 4-head GPT2 with d (24,32, (4,32,40) 40).
Validaatinn Accuracy (N))
30
00
70
1.0
Hux Teadher: Embl1
40-
Dacche Bab
Taadurt
30
Teadoet Eaab
Sab Dincher End
229
590
750
1000
1239
1550
1750
2000
Figure 6: The comparison between the hard and soft teachers for different embedding dimensions on MNNS task. The figure illustrates that the hard teacher is superior to the soft teacher. Setting: input digits in 1 9; 4-layer, 4-head GPT2 with E (16, 22)
20

---

Published as a conference paper at ICLR 2026
Valldrtiin Accurayy (N
60-
Coolisanes Eua 15
20
Mtarstt, Bich
Coedivuess Esd 22 Mtaritt, Bibb
Figure 7: Comparison between CoT2 and discrete CoT model for different embedding dimensions. The figure demonstrates that above certain embedding dimension threshold, the CoT2 model outperforms the discrete CoT model in the MNNS task. Setting: 4 input digits in 9; l-layer, 1-head GPT2 with E (16,24,32) 22)
Validalto Accurary
40
20
Chetttaaaa tuz
Bacont Dalk
Condmavus, Fas
Figure 8: Comparison between CoT2 and discrete CoT model for different embedding dimensions in ProntoQA task. The figure shows that above an embedding dimension threshold, the CoT2 model outperforms the discrete CoT model. Setting: 4-layer, 4-head GPT2 with d E (24.32,4)) 40).
75
Maj@kk accuracy
70
Temp=0.0
60
Temp=0.4
Temp-0.8
Temp=1.0
Continueus
50
(number of samples)
Figure 9: The figure illustrates that the discrete CoT model requires multiple samplings (Maj@k) to match the single-shot performance of the CoT2 model on ProntoQA (10-run average). Setting: 4-layer, 4-head GPT2 with d = 32.
21

---

Published as a conference paper at ICLR 2026
Maj@k accuracy (%)
85
Temp=0.0
75-
Temp=0.4
Temp-0.8
70
Temp=1.0
Continuous SFT
65
2.5
12.5 15.0
17.5
(number samples)
Figure 10: The figure illustrates that the discrete CoT model requires multiple samplings (Maj@k) to match the single-shot performance of the CoT2 model ProsQA (10-run average). Setting: 4-layer, 4-head GPT2 with 32.
90
Pars@kk accuracy
70
60
Temp=0.0
Temp-0.4 4
Temp=0.8
Temp=110
Continuous
30:
(number of samples)
Figure 11: Performance of single-shot CoT2 vs. discrete CoT with Pass K across different temperatures the MNNS task. Setting: 4 input digits 1-9; 1-layer, 1-head GPT-2 with 24; CoT2 uses full trajectory budget B=[7] and discrete CoT uses B=1.
22

---

Published as a conference paper at ICLR 2026
D.2 GRPO RESULTS
Discussion on ProntoQAA/ProQQA Datasets: Table Iillustrates that GRPO training using CoT2- MTS sampling consistently improves discrete CoT and Co'22 models over their initial SFT accuracy. Moreover, we observe that the improvement in the discrete CoT model is greater, which might indicate that the CoT2 model already gains an RL-like exploration mechanism through CSFT training. We observe that while increasing initially increases the accuracy by sampling more tokens at each step, beyond some K, the improvements diminish. This observation is consistent with Table where we see that a moderate K value offers the best final performance. One possible explanation that while higher K promotes better exploration, it also raises the chance of sampling unhelpful tokens that disrupt the averaged token representation. Indeed, for larger K, we observe that the RL. objective saturates to near zero which suggests that most rollouts fail once the averaged token contains too many distracting tokens.
Discussion Dirichlet Sampling: We also investigate the effects of Dirichlet sampling in GRPO training discrete CoT and CoT2 models. The results in Table 4 indicate that applying Dirichlet sampling (y 20) in GRPO training of discrete CoT model consistently improves over the initial SFT training accuracies. Similar to the CoT2 + MTS sampling results in Table 2 we observe that the entropy the third token remains relatively high, which suggests beneficial diversity in model's predictions for that token. Moreover, the Table debonnseemtst that Dirichlet sampling also improves the CoT2 model's SFT accuracy, even though has high initial SFT accuracy. As illustrated in Table [ we find there is an optimal value for the scale parameter y. since larger y typically yields more uniform sampling distributions, whereas smaller y concentrates the distribution more sharply. Thus, adjusting y provides a balance between exploration and stability in GRPO training.
Val. Accuracy (%)
Val. Entropy (SFT - SFT+GRPO)
Layers
Heads
SFT
SFT+GRPO
token)
token:
tokens
token,
1
1
39.76
46.25
0.4851
0.1701
0.5165 - 0.6380
0.3243
0.6590
0.1597
0.4878
2
2
70.26
75.84
0.4851
0.4027
0,5165 - 0,4413 0.3243
0.2907 0.1597
0.1386
Table 4: Discrete CoT models trained with GRPO after SFT using Dirichlet sampling (y 20) and a learning rate of x 10-5. We show validation accuracy (%) and token-level entropy (SFT - SFT+GRPO) for each (Layers, Heads) setting, with embedding dimension of 24 for GPT2 model.
Dirichlet Scale (γ)
SFT Val. Acc (%)
SFT GRPO Val. Ace (%)
10
87.84
89.76
20
87.84
90.75
40
87.84
90.37
Table 5: Validation accuracies GRPO training with CoT2 models using different Dirichlet sampling scales (7) with learning rate of x 10-6, We show the baseline SFT accuracy (87.84%) and final performance after GRPO.
CoT2-MTS GRPO experiments on GSM8K. According to raised points to evaluate our CoT2- MTS method on reasoning benchmarks beyond logical multi-hop reasoning, we ran Qwen3-0.6B on GSM8K using our CoT2-MTS continuous-token sampling procedure. We implemented the continuous rollout/training code from scratch, and keep generation and training computationally feasible, we conducted our experiments on modest model sizes and decoding budgets. For CoT2- MTS. we used a curriculum learning approach where continuous tokens are introduced from 50% of training, ramped to full by 80%, and used thereafter. The table below reports single-run results for different maximum response lengths (160, 128, 96, 80 tokens). We use continuous tokens for the 96/80 settings and 32 continuous tokens for the 1 128/160 settings; for example, in the 160-token configuration the effective budget grows from 128 discrete tokens at the start of training to 128 discrete + 32 continuous tokens (i.e., 128 118~1600 160) as the curriculum completes.
For these experiments we adopt a reward setting by using only the final answer in the 0/1 reward, and train with GRPO for 2000 steps using learning rate 3 x 10-6, batch size 6, and gradient accumulation of 4. The GRPO group size is set to G = 4, the clipping parameter to e = 0.1, the KL coefficient to 0, and the number of sampled tokens in MTS to K 4.
23

---

Published as a conference paper at ICLR 2026
Max Resp. Len
Initial Acc (%)
Disc. CoT Acc (%)
CoT2-MIS Ace (%)
Avg. Len (Disc CoT)
Avg. Len (CoT2-MTS)
160
34.6
37.0
41.3
97.3
114.8
128
27.6
34.3
36.2
104.8
96.6
96
17.2
27.2
27.1
84.0
83.7
9.5
21.5
27.0
73.9
67.0
Table 6: Validation accuracies and response lengths for GRPO training with CoT2-MTS (K 4) on GSM8K using Qwen3-0.6B under different maximum response lengths.
Across GSM8K with Qwen3-0.6B, CoT2-MTS improves over discrete CoT in three out of four maximum response-length settings and essentially matches it in the remaining one, while often shortening the average response length. Overall, these preliminary results indicate that performing exploration in the continuous action space via CoT2-MTS is promising direction.
E
THEORETICAL DETAILS
This section of the appendix gathers theoretical results that support our continuous chain-of-thouge framework. Appendix DreenddiEE..preeentn simple packing-style argument that makes the tradeoff between the trajectory budget (i.e., the number of superposed states) and the embedding dimension needed for robust representation explicit. Appendix F.2 discusses when Assumption Tis reasonable modeling assumption for structured reasoning tasks. In Appendix [E.3] provide proofs for consistency and sample-complexin comparisons between CoT2-MTS and discrete COT. Finally, Appendix E.4
gives an explicit transformer construction that solves the MNNS task.
E.1 DISCUSSION ON BUDGET-EMBEDDING DIMENSION TRADEOFF
Without loss of generality, assume that the token embeddings lie inside unit-circle lle:llz s 1. Let [v] be a set superposed states at step with size |S| = By triangle-inequality, we know that the convex superposition = Eies a; e; with a; 2 0 and Ens ar satisfies ||zs|12 1.
Suppose there exists downstream module to recover the superposed states robustly from the single d d-dimensional vector Zs. Formally, assume fixed margin 5 > 0 such that even if zy is perturbed by any with ||7]| I 5, the decoding does not change: there is decoder Dec : Rd + (S C [v] B) with Dec(zs + 11) S for all S and all such 17.
For each Zs, consider the closed ball B(zs. 6) = (x ||-=z x 6). The robustness assumption trivially implies these balls are pairwise disjoint; moreover, by the triangle inequality, for any x B(zs,
||x||| < ||x zs |I || s 1,
so, each 8-ball sits inside larger ball B(zs C B(0, + 6). Therefore, the union of all disjoint 5-balls lies inside B(O, + 6):
U B(as, 5) B(0, 1 6).
SCI0]
18|=8
The number of different sets of superposed states of size is (2) The packing constraint says the union of the (t) disjoint 6-balls must fit inside B(0, 5). In Rd the volume of a radius-r Euclidean ball is cardd where ca 1(1+//2)
Comparing volumes gives
(")
vol 1 11++)) 1 8
B
vol(B(O, 5))
In the regime > B. taking logs and using the Stirling approximation for B! gives the inequality log (:) 2 Blog(v/B). Using this yields:
B Blog V s d log
1td (
<d log
B
B Blog(v/B) 2 B log(v/ B)) when d is a fixed constant.
for 8 S 1. Equivalently, d2
log(2/0)
24

---

Published as a conference paper at ICLR 2026
E.2 JUSTIFICATION ASSUMPTION I
Assumption I holds for tasks where the next-token distribution depends solely on the current token and the input tokens rather than the full history output tokens. This is satisfied by many reasoning tasks, where the aim is to keep track of an intermediate state (c.g., the current sum) and update this state based only on the current state and the input, independently of the earlier trajectory.
For example, in the MNNS task, the model generates a token representing the current partial sum each step. To compute the distribution over the next possible sums, the model adds subtracts the selected number from the input context X to the current sum, without needing to remember the sequence of previous sums explicitly. Thus, the next-state distribution each step is only determined by the current state and it naturally satisfies the Assumption I
E.3 PROOFS FOR CONSISTENCY AND SAMPLE COMPLEXITY RESULTS FOR C0T2-MTS vs. DISCRETE CoT
Proposition (Consistency of CoT and CoT2 inference). Under Assumption and given X, the output of base CoT2 Zm. Σj=1 Cm jj where am on Π M, Discrete CoT and CoT2-MTS have the same oulput once we take the expectation over their stochastic sampling.
Proof. Let á. ((diss)) a (MTTS)) denote the empirical output token distributions at step r under one trajectory obtained by the discrete CoT, and CoT2 with MTS models, respectively. We define a; a.diis) B [ (dise)
and (MTS)) = E [ to be corresponding expected distributions. The discrete CoT model at each step picks exactly token from a;, On the other hand, CoT2-MTS samples K i.i.d. tokens at every step independently according their probabilities from å, We denote them i,.... Ik. and average their embeddings to produce single query.
We will use induction in our argument. For the base case, all models start with the same initial distribution, so we trivially have ar (disc)
(MTS) a (CoT2) For the inductive step, assume that we = a;
(dise) = a a-11 (MTS) (CoT2) We will show that amiino) a (MTS) a (CaT2) On the other, for the 1-1 discrete CoT model, the samples one token e.., from the row of M, for a token f,+1. Therefore, need to condition the token at step We have:
have 1-1
B
(dise)
=
P(z, e,) Ba) z, =e]
(6)
F=1
Σ
P(z, ej) LM, (ee,,
F=1
(disc) LM le; X)
(CT22)
LM
(p) Co12) (-]
(dise) where (a) follows from the induction argument and (b) follows from Assumption |
Therefore, we
obtain a à c-1 (CoT2) For the CoT2-MTS model, the argument will be similar. Using the decoupling of trajectories by Assumption I the next distribution is:
[dise)]
LM
.1
ei.
X
LM (-le,, xe.
r=1
25

---

Published as a conference paper at ICLR 2026
Therefore, write:
MTSS)
B
Σ
P(e: e #+1
(MIS))
Cip eix....,e
Σ
P LM
c
Σ
P(e: lic Σ LM (- X)
K
K
Σ
[((.)))
P(e.)
1
LM (:,,;,
K
G [v]
rml
Σ
K
(MTS)
1
LM (-1 X)
K
K
LM le,
Σ
(MTS)
(MTS)
a
1.i.
sm
syhr
K
(MTS) (.j
LM
Σ
(MTS)
K
s=1
spr
(MTS)
LM (-1e,,)==
Σ
(MSS)
LM (-1ej.
F=1
Σ (Co12) dij
LM ( (1)e,.)= - (Co12)
(8)
Thus, combining and completes the induction and our argument:
(disc) f+1
= E (cliic)]
-a
(CoT2)
[+1
a (MTS)
= aM
(MTS)
Proposition 3. (i) Let am be the expected output distribution after steps of CoT according to Proposition 2 Let @ (MTS) be the distribution resulting from averaging the outputs of N i.i.d. CoT2- MTS traces with parallelism K. Then, to guarantee [| (MIS)) -amllz S e with high probability, the total number of samples (traces) required scales as (KK-) c (ii) Defining (dise) to be the distribution obtained by averaging the outputs of NK i.i.d. discrete CoTí' traces, we have the following upper bound on the expected error of MTS:
|Ia m
B [| am111]]
K
Proof. Our notation in the proof will deviate from that in the proposition statement. As in the proof of the previous proposition, we denote by (MMTS) a single realization of the distribution obtained from the last K tokens of CoT2-MTS. On the other hand, the random variable a (ddise) represents the distribution obtained by averaging K i..d. discrete CoT traces. We will compare the variances of these two approaches. The resulting inequality will similarly hold for the MTS and discrete CoT estimators given in the proposition statement, which N and NK samples, respectively. Initially, we'll compare the variances of the two approaches at step / = (empirical distributions formed at the end of the second sampling step), and parallel argument will hold for any
CoT2-MTS: In the beginning, have deterministic distribution ai
LM ( X)
Categorical(a1). The (oLb......)) A-1 At step 1, draw tokens ig. iid.
26

---

Published as a conference paper at ICLR 2026
empirical frequencies over the vocabulary
K
(MIS))
K 1(i, f)
je [v].
This means, the query fed into the model is 21 Enn L ey. By Assumption is:
the output distribution
@ LMM Z1, X)
a a L. (-le,,
Thus 2 is a convex combination of the fixed distributions LM ( le, X) for je [v]. step 2, we
will draw tokens from this common distribution dz Let
Y(MTS) E [v], I...,,K K, YONTS) &z (MTS) (at
be the tokens drawn. Now, we'll calculate the variance along any fixed vocabulary coordinate E [v] and define
p(MTS) == 1[Y(MTS)) = =).
FIMTS)
(MTS)
K
r=1
Denote the distributions of the drawn tokens:
U == LLM, ei,
X) (-le,
EA
(MTS))
(U
U.MIS)
(MSS)
Then (MMTT)) depends only on the first-step choice i, and therefore i.i.d. across r. U â2 the random success parameter for the Bernoulli variables y(MTS) By using the unbiasedness given by Proposition 2 we know that B
r
(CoT2) Define the shorthand notations:
az22
#4 == B[UMSST) B(U = a22x
(Co12)
of == V (MTS)
We calculate the variance of the MTS estimator Varl Car[P(MTS) by using the law of total variance:
Varl 4 J B VarlY
[(NTT))] + Var (ELF(TTS) U ((AMSS))
Given U(MTS) the M(MTS) are i.i.d. B Thus:
(MTS)) (1 (MTS))
Var POMTS) (
Taking expectation and using Var(UAMTSS
o./K:
(AMIS) (1 - J(MIS))
(BIU (2((MTT))
B
K
k (u -0ž/ HF)
(A)
1x m4)
o7
K
K2¹
Also, the other term is:
0
Var (BIT ((MTT)))
Varl
(B)
K
27

---

Published as a conference paper at ICLR 2026
Combining (A) and (B) yields:
Varl ((MIT)) k
M4(11 m4))
K
K
K²
For independent traces, each trace r draws at step 2 from its own distribution U. (instead of a common mixture). Let yídise) Bernoulli( law of total variance:
Co'::
( and Fidise) := Σ, p Apply the
r
Varl Varly [uddidd =
Varl (ddise)
[ndiso) rk 1]
Var (B(ridiso) (ddiso)
rx
(A)
(B)
Given U(MTS) the variance of yídise) is: ik
VarlY<dise) [ddisc)] = (ndisc) (1-
(diss))
n,
Taking expectation over (ddise)
B
(diis)
(ddise))
) 1B(Uidiioo 2.cdise)
rk
1,t
Mk - (Varr(V (M + 11
rit
p m4: -
where of Var(U Note that the variances the same V VarlU of since the initial distribution a₁ shared between MTS and discrete CoT approaches. For the conditionallexpecaaiion variance term
Var ([rdd (1dheo) Var[U ]= o
Combining (A) and (B) yields:
Var[r/diio] (11(1 14) - -0) = 14(1 1k).
rd
Mk(1 p1x)
Var[Y(dlvo]
(10)
K
Using 99) and I0
we have:
Varl Var[P(MTS)) P(MTS)
Varl p(dise) k
]
+
K -
o
K²
Var F(MTT)) K-1
of
Varl V(iss
K
ux(1 p4a))
Because every ((MTS) E [0, 11. the known inequality Var[Z] s B[Z] B[Z]) for any random
variable E [0, 1] implies of s 1u(1 -4). As a result:
Varl (AMTS
K-1
1
$ I1
=2 K
Thus, summing this over all & e [v] gives:
amll? -m s 5(2
))
am1
Proof for any 1. We will now show this at any step Consider the discrete CoT case with K i.i.d. traces. Different from the 2-step case, where the tokens for MTS and discrete CoT at step are drawn from the same distribution a, they will be drawn from the empirical distributions a (MSS) and (ddis)) Similarly define ((diis)) to be the distribution LM (- e,,- X) E A obtained by sampling the token i, For each vocabulary index [v]
2.(disc)
(diss) 0
= Var ((dise)) ). Var p!dice)
K
28

---

Published as a conference paper at ICLR 2026
The equality follows because the ((dis)) are i.id. across Now, calculate the variance at step ++ 1. Each trace now draws Bemoulli variable y(dise) Bernoulli(U ((a Parallel to the previous case, using the law of total variance cancels out the variance terms o 2(diie) and yields:
Fidise]
Me-1,1 - Mst1a)
Var
(1I)
K
CoT2-MTS: This time, all tokens at step + are drawn from a shared p (MTS) which is resulted after drawing K tokens from dTTSS Using y(MSS) of total variance gives:
(MTS) Bernoulli( (
(MTS) and applying the law
Var
POMTS) #+1
=
(MTS)
(1
Mik
(MTS)
Var p
(MTS))
K
2.1TS
2(MTS)
M1 M2+1.k)
1
K
K2
K
K K-1 2(TTS)
(12)
K
K²
18
Take the ratio of (12 to
Var ((MTS))
O 2(MSS)
K-
Varl pidiso
K
Mr M
Again, every (MMTS)
is 0 S UMMIS)
I For any random variable Z bounded in [0. 1J, we know Therefore we always have o
r
that Var[Z]
B[Z] (1 B[Z)).
24MTS) Pi+LA (1 - M411)) and thus,
1, (1 (4+11.)
(MT))) Var 1
Using this in the previous equality yields:
K
Varl P(MTS) 1+L
K-1
2 1
S
1
Varl p(disc)
K
K
1+1
Thus, defining of == |10 (dinc)
mm we have shown that:
mmml s x od
B
(13)
We also know that (disc)
am
is the average of K i.i.d. discrete rollouts, and thus:
o disc
= =B -Blladiioo
aml
Σ Mmj (1 Mm..) S K
Define the average of N independent MTS roll-outs & := & ΣN a add for independent runs, we have Var[ām]] Varla (MTS)/ J/N. Applying Chebyshev's incquality yields:
(TTS), and because variances
(2 (2- k) disc
(2 + KN 2
@| amell > el 5
2
In order to make this probability s 6, is enough to take
2 2
N
- (
K 2
To finish our argument by finding a lower bound on N, we leverage the standard result in multinomial estimation that @ (+) i.i.d. samples are necessary and sufficient learn a v-category distribution in || dii-tnst S e (Kamath et al. 2015). We know by Proposition 2 that the distribution MTS to be recovered is shared between MTS and discrete CoT estimators. Additionally, from (13),
29

---

attend and select the input tokens Published as a conference paper at ICLR 2026
(disc) Note that the estimator know the variance of the estimator (MTS) is least large as that of im (diis)) is the average of K i.i.d. discrete CoT draws, which guarantees estimation emror of at most
with @ K aggregated samples. Hence, the same lower bound applies to MTS samplings, yielding
sample complexity of Ω (x=) Combining with the upper bound, have O Ke as claimed. This completes the argument.
E,4 CONSTRUCTION FOR MINIMUM NON-NEGATIVE SUM (MNNS) TASK
We describe single-layer transformer with an attention block followed by a mixture-of-experts (MoE) feed-forward block. Let be the length of the input sequence of integer tokens. Denote the tokenized input numbers as Zt, .2..... Zni and let the arrow (-) token be denoted as Zn+1. We also have a dummy input token Zn+2, which is the embedding corresponding the number 0, so that have πI + tokens initially. We will construct the transformer with n MLPs in the mixture of experts layer, where the first п partial-sun MLPs and the last one is the MLP that reads off the answer from among all the stored partial sums after m steps. We start with the following assumption on the structure of the tokens.
Remark 2. As an empirical validation of Proposition 7 we observe that training according to this construction with trigonometric embeddings yields perfect accuracy.
Assumption 2. Let d de + dp be the embedding size where de 2n and dp = n + 2. The token embeddings are on the first de coordinates, while the positional encodings are on the last
dp coordinates and are one-hot encoded. where each z;
E Edd+d, is formed by vertically concatenating a content embedding e; RR4. and a positional encoding pr Rd, We assume each pt is a one-hot vector in Rdd,, so that P) = f i # j. and lipill =
pi
We now state the following proposition, which helps 2 one by one by the attention block.
Proposition 4. Suppose we have n + tokens (21, Z2. 22....,Zn=2) in Rd, each of the form zi =
pi)
where e; RRd.ptt Pi R de + dg- Let Pt, ......... Pe-2 E Rd be orthonormal set of positional vectors according to Assumption
Then, there exists a rotation matrix E Rd,xd, satisfying Rp, P 1-1 mod (n+2) for all j In 21. and the block matrices
10 d.xde
dd.aa
W, = (1le
ld. (0
0d.xd, Badd Id,
W
Raxd and
d,xd,
with oo, ensure that the attention block
Attn(z, Z) = S (z WZ ZW,.
performs a cyclic next-inder selection: if the query is 2, it selects column J = (i 1) (mod 2) from Z and returns Zp.
Proof.
Definition of Matrix W. We will first construct a rotation matrix. We have n + orthonormal position vectors Pı...., Pat2 E R4, Then, R is the following (n 2) (n 2) permutation matrix
0
0
0
0
R
00
O
which eyclically shifts the basis vectors Pj backward by one index, i.e., Rp; Pi-1 1 mod (r+2)- Then, we specify
04,xd
O 0axaa
W
E Raxd
0 d,yaa
c. R
30

---

Published as a conference paper at ICLR 2026
Hence for 2, pr), we have (e) p.) w (0.p.
Thus the dot Not-product with 2, is
(o p R) ej p,)
p Rp,. Rp
Since positional encodings orthogonal, we know that:
I. j 1(mod(n 2)),
p. Rpj
0.
clse.
So row-wise softmax 8 (x WXT) places all probability mass at column jr = 1(mod(n 2)) by saturating softmax at position j as c co.
Definition of Matrix Wy. In this case, simply set W, las and thus, once the row-wise softmax selects column j' with probability 1, we have
W Zy,
so the final output is precisely the chosen Zp. This completes the construction.
Having defined the attention block, we state the following proposition that helps selecting different MLPs for the tokens Zl,... In+1 outputted by the attention block.
Having defined the attention block, we now show how mixture-of exxeers layer can exclusively select MLP; for each token ž;, l...,, + outputted by the attention block.
be + experts in mixture-of-experts (MoE) module. Suppose we have fixed token embeddings (21, Z2..... ža=1) Rd, where each token is formed according to Assumption 2 Given routing parameters W = [w] Wnt1l", define the MoE feed- forward block
Proposition 5. Let MLPI,...,MII
n+1
MoEBlock(2) =
Σ
Softmax(Wz), (2)), MLP,(2).
where
exp (w]22
Softmax(Wa))
j= 1, 1=...... 1.
Σnti exp ww
There exist routing matrix W Rli+lxdd such that the distribution Softmax(c Wz,) as c - oo assigns a weight of on MLP, when zı is given as input.
Proof. We partition w, to ignore the content embedding and match the positional block pj- Concretely, write w, p,
(4,
Then, for each token z; = P.).
w] zi (0% pj
D.P
Since P, =.j. we have wTzz, 6.y. Therefore, the softmax evaluates to
exp(c ōiz) dij-
Softmax(c Wz,); C-000
Σ exp(c 5x)
In other words. Softmax(c- Wz,) places all mass expert j i. Thus each token z; (for 1,..., n+ 1) deterministically selects the i-th expert MIP,
In the next proposition, show how to iteratively expand the partial sums by adding and subtracting the digit obtained from the attention block and write each resulting sum to a distinct spot in the output vector.
31

---

Published as a conference paper at ICLR 2026
Proposition 6 (Partial-Sune Suriial-Summ MLPs). Suppose that the embedding dimension d satisfies d 2 2 dp. Let Eprey contain the 2/-1 partial sums each encoded by pair (cos(ws)), sin(ws)) of coordinates such that:
Zprev coo((ss)) sin(wss) cos(wsyi 1) sin(wsy; .).....
of' R
and let Zour contain the input digit d encoded in the first two coordinates:
Zeurr (cos(edd) sin(cod,) O ol' E R."
Then, for any 1 J there exist MLP; R x Rd - Rd such that when (Zprevs Zearr) is given as imput, it outputs the vector Zout E Rd so that its first 2 coordinate-paiio store the trigonometric encodings of (5x + dd), and the next 2 coordinate-pen paiirs store those of (5y (s-dd, dj). Formally, first 2 coordinates are [cos(u[ss + d,)). sin(co(ss +dj)ffo for all partial sums Sk. and the next 2 coordinates are [cos(w(ss sin(wAss dor all partial sums with any remaining coordinates set to zero.
Proof. Each expert MLP, (for S j n) adds j-th integer d, in both its positive and negative form to all previously computed partial sums. For simplicity, let's say that j-th integer to add is dj. By trigonometric identities, we know that
cos(w(sk dj)) cos(wss) cos(w,)) sin(asss) sin(wd,)), sin(w(s2 + dj)) sin(wss) ((((os))css(odd))))) cos(wdd)) + cos(wss) sin(wdd))
and similarly,
cos(w(ss d,)) cos(wud)) sin(ws;)) sin(wdd),, sin(w(s; - dj)) sin(wss) (in(wss))os(udd))) cos(ws;) sin(wd,). Using the above identities, we will obtain the sum by introducing matrices that do shift/swap
cos((sss) in Zprev- We define:
operations. Concretely, for fork= 1,...,2" 2", the k-th x block acts sin(wss))
0
0
W sin
diag
(
blocks
W = diag
0. o.....
2 blocks
The above constructions of W sin sin satisfy,
and W
W Sprov
sin(ws)) cos(ws)) sin(w182-11 cos(wus2-1100.. O.... .0] E
and
W sin Sprev (in((s))) cos(wsi) sin(ws))) cos(wsy::) oJ' ERC.
Each of these acts blockwise the first 2 coordinates of Zprev and zeroes out everything else in dimension d. We also have Zeur Rd with two designated coordinates Zeur.1 cos(wd,) and Zeum.1 = sin(wd,,), with all other coordinates being zero. We multiply Zprev by cos(wd,) and sin(wd,)) elementwise. Formally, the sum
Zeurr
Zprev Zeumr
(W+ Zprev)
gives the 2/-1 partial sums (st d d stored in the coordinates from I to 2/. We define W R in a block form with three row blocks and two column blocks:
02/x2/
02/id-2/)
W stan =
h
22xidd211
0
(1-22
(d-22))xd-211)
32

---

Published as a conference paper at ICLR 2026
When applied, the above matrix shifts the first 2 entries of Zprev by 2/ coordinates. Now, also define
2 W shift sin 2prev) + Zemrr. (W (Whhin
This way, the above sum gives us the 2 partial sums (sx -a--da d,j-ii stored in the coordinates from 2 2 encoded in trigonometric format. Then, we normalize this output of the model by 1/2 and obtain the following output:
Zeurr.1 Zeurr.2 (Main Zola) + Zeurn.2 (Wahin W 2prev) ) Zear.11
(W (W shift 2preev
((s)( + +d)) sin (w) d,)) ..... cos (os(c(ss))) (sz- + dj)) sin (1 + dj) cos (w di)) sin (co - dd)) (.)))......oo) (822-1111111 0, O,....OJ" Rd.
(82 - dj)) sin (1 - dj)
Thus, this is exactly the representation of 2' partial sums. This completes the argument. We should remark that, the above argument utilizes a gated MLP which explicitly multiplies the clements of the input features, namely, Zeun with the partial sums Zprev- On the other hand, we don't 't' require any nonlinear activation function,
MLP constructions have the form MLP(z) Wa (WW O W2z) for suitable choices of W, W2, W3 where O denotes the Hadamard product. The use of gated MLPs a standard practice transformer architectures (Shazeer 2020).
Proposition 7 (Read-Off MLP). Suppose that every partial sum Sy is in the range [-S,S] and let < n/2S. Assume that the vector
= [cos(ws1), sin(ws)), .... cos(wss)) oin..s)-).o.....00' 00J Rd
contains 2" partial sums (S1...., )z) encoded in trigonometric form, where d = 2n+1 + n +n++ 2. Then there exists a single feed-forward network MLP u+1 Fe Rd such that, given input it selects the smallest nonnegative st from (S)...., S and oulputs the embedding es, E 4, where St is that minimal nonnegative partial sum.
Remark: Our construction relies on gated MLP, rather than standard MLP, as in Proposition 6
Proof. We know that the input embedding z represents 2" pairs, each pair (cos(ws))) sin(ws;)) stored consecutively. That is,
[cos(ws)), sin(ws)),, .... cos(ws2)) sin(wox:).
R
We will identify the smallest & 2 and output an embedding e,, denoting that integer. We are given that wo is small enough such that when and only if s 2 First, we wish collapse z into single vector of size 2", keeping cos(ws)) only when sin(ws))) 0 and zeroing out otherwise. We define two matrices Woos Wsin E Rexi by
[0,5], J. we ensure Sw x/2. This guarantees sin((s); 2 0
(Waos), (2--1) (Wens)., = for j
(W an 1, (Wsin), for j # 2i.
for 1 i < 2" and all other rows/columns of Wsin- Woos are zero. Hence each matrix picks out alternate coordinates:
cos(wss) )
sin(wsi)) sin(ws)))
cos(w)))
Zens Won: z cos(us2))
E Rd. Zsin Wan z sin(wS
e Rd.
0
In order to find the minimum non non-negane non-negative number, we need to find the number such that maximizes cos(ws) and satisfies sin(ws) 0. For this, we utilize sigmoid activation function in the following way:
Züiter Zeos (C Zsin).
33

---

Published as a conference paper at ICLR 2026
where (xx) T-exp(-xj is element- wise sigmoid function, and c oo is large constant. With this choice of c, the sigmoid output will be when St 2 0 and 0 otherwise. Therefore, the resulting vector Zülter contains cos(ws) values at indices where sin(ws) is positive. Now, for 0 S 8 S S with Sw s r 2' the ordering of st from smallest to largest is the same as the ordering of cos(ws)) from largest to smallest. Thus, to find the minimum nonnegative sum, we find the partial sum '" that maximizes cos(ws;). Utilizing another gating, we calculate
Softmax (c Zfheer Ziitter
co. The softmax vector will be one-hot with I at index 6" that has the largest cos(wss)) second multiplication with Züter will return this cos(wse)) Therefore, Softmax (c Zitter' Züiterr cos(ws))) Next, we retrieve the corresponding sine entry of sc- by applying the same one-hot selection to Zsin- Formally,
Softmax (c Znherr Zain sin (wss)),
as - co. Hence, from these two selected coordinates, [cos(wxr)), ) sin(wsr)) ), we produce the final embedding in Rd by placing them in the first two coordinates and zeros elsewhere:
Sy- (cos(wse)) ), sin(wse)). O,.....J",
where 5e- is the minimal nonnegative sum. This completes the argument.
Proposition (Solving MNNS). There exists a I llayyer trunsformer architecture with embedding dimension d de dp. where de 2=r1 is the state-encoding dimension and dp + 2 is the ansitional-encoding dimension, that solves the MNNS task using CoT2 by storing sine-cosine embeddings of all 2 states at the k-th iteration in a non-overlapping manner.
Proof. We will argue that by combining Propositions to 7 we obtain a single-layer transformer that formed by an attention block followed by an MoE feed forward block, which solves the Minimum Non-Negative Sum (MNNS) task.
Suppose that we have n input integers di...., dn, encoded as Zi. Za. plus an arrow (-) token Ze+1 and a dummy token Za+2 corresponding to the integer O. In this case, we will output the tokens representing the ground-truth sums Si. Sn. therefore, the number of output tokens is m = n in the MNNS setting. We assume that the inputs are encoded according to Assumption 2 By Proposition 6 there exist MLPI...., MLP,, that perform the following: whenever MLP, is selected with input (Zprevs Zeurr) such that Z prev stores 2 partial sums and Σearr stores the digit dj. it adds and subtracts d, all previously stored partial sums and stores the resulting 2 partial sums in Zou- The dummy token Zn+2 that corresponds to the integer O allows us to initialize the partial sums from zero. If the query token is 2 we produce the first partial sums by combining this dummy with di, which are (+d1) and (-d1) encoded in an output token.
We assign positional encodings cyclically to output tokens. That means, the first n + 2 input tokens have positional encodings from Pi to pm+2. and the output tokens have pt. P...... as their positional encodings, in this exact order. This way, by Proposition 4 Attn(z, Z) attends and selects the input digit tokens Z1, .2, Zn and finally arrow Za+1 one by one and feeds to MoEBlock())
By Proposition 3 there's a MoEBlock(z) such that if the input is 2 (for j S n), MLP, is selected with probability I, and if the imput is arrow token Z MLP,.1 is selected with probability 1, which is the MI.P to read-off the final answer. In the input tokens Zı,..., Zu. the first two coordinates store the trigonometric representation of d,.....de d, To allow outputting the final answer by MLP, p-11 the partial sums obtained in the intermediate steps need to be written to separate coordinates. Therefore, MLP, takes vector filled in the first 2/ coordinates, adds d, and writes the first 2 coordinates. subtracts dj and writes to the next 2 coordinates, and finally divides the entire representation by 2 to maintain consistent scaling since the number of partial sums is doubled. In other words, the first n MLPs have some repeated behavior. Finally, by Proposition] MLP,,t n+1 receives a vector that encodes all 2" possible partial sums in cos/sin form in 2 coordinates and extracts the embedding of the smallest nonnegative number among them.
Altogether, this single-layer transformer with an attention module to pass the tokens to the mixture- ex-extssss MLP solves the Minimum Non-Negative Sum task by following CSFT described in
34


Reviews:
[{"review_id": "SlCLWFfjjq", "reviewer": "Reviewer_oWRB", "summary": "The paper proposes CoT2, a new method for continuous chain of thought, which constructs continuous tokens as a linear superposition of tokens. Such models can be trained via SFT on a combination of top-K teacher traces. It is shown theoretically that a one-layer transformer construction can solve a version of subset-sum using CoT2 by tracking multiple computations in parallel. Moreover, it is shown that decoding with $N$ chains of CoT2 by averaging top $K$ sampled tokens, is upper bounded by the expected error of $NK$ ordinary CoT chains, showing the statistical benefits of parallelism. Finally, a GRPO objective is proposed for continuous CoT.", "questions": "* As mentioned by the authors, the CSFT training scheme can be seen as token level distillation. Can the authors give a comparison with related methods in distillation?\n* See also the questions in Weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The proposed CoT2 method is novel and simple to implement on top of existing models.\n* Since each continuous token is a superposition of discrete tokens, the CoT should be more interpretable compared to methods which simply append the hidden representations.\n* Theoretical analysis supports the intuitive benefits of tracking multiple traces. In particular, the increased information content is reflected in improved sample complexity.\n* It is demonstrated that GRPO with MTS can adapt models to continuous rollouts and improve performance on synthetic tasks.", "weaknesses": "* While each continuous token should carry $K$ times more information than a discrete token with MTS as demonstrated by the sample complexity results, we also need to sample $K$ times more tokens when decoding. What is the tradeoff in terms of actual runtime (during SFT, decoding, RL, etc)?\n* While a transformer construction solving MNNS with CoT2 is provided in Proposition 2, there is no lower bound for ordinary CoT, os it is unclear whether this constitutes an actual improvement over discrete CoT.\n* The GRPO policy ratio proposed in Eq.(4) is not an unbiased estimate via policy gradient theorem.\n* Since GRPO with MTS can be used to adapt existing discrete models to output continuous CoT, I would expect to see performance on larger models than GPT-2 and more challenging reasoning benchmarks. Can the authors add any larger-scale experiments?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VyhDI11OPR", "reviewer": "Reviewer_JDh5", "summary": "This paper proposes an interesting framework termed _Continuous Chain of Thought_ (CoT2). Inspired by a previous work titled chain of continuous thought (COCONUT), this work trains transformer models to reason in a continuous token space rather than via discrete linguistic steps. Each reasoning token is represented as a convex combination of vocabulary embeddings, allowing multiple reasoning trajectories to coexist in superposition. The model is trained from scratch using a Continuous Supervised Fine-Tuning (CSFT) objective that aligns predicted token distributions with soft targets derived from the top-BBB teacher trajectories, followed by a Generalized Reinforcement Policy Optimization (GRPO) phase to reinforce correct reasoning paths. Experiments on synthetic reasoning benchmarks such as MNNS and ProsQA show that this approach achieves higher task accuracy compared with standard discrete CoT baselines. In addition, the authors revealed a link between embedding dimensionality and the number of representable reasoning branches via theoretical analysis.", "questions": "- Can the proposed continuous-token formalism scale to full-vocabulary pretrained LMs, or is it confined to small synthetic domains? No experiments needed if the authors are pressed for time. Simple speculation suffices.\n- How sensitive are results to the trajectory budget B and to the number of reasoning steps? Can one set these hyper-parameters automatically?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The framework offers a theoretically principled formulation of parallel reasoning within neural sequence models, on a scale that is testable with single GPUs.\n- The CSFT objective is mathematically clean, converting multi-trajectory supervision into a single convex target, and the accompanying proofs clarify the representational capacity of continuous token mixtures. \n- The paper establishes a precise connection between continuous probabilistic reasoning and information-theoretic efficiency, supported by consistent empirical trends in accuracy and entropy reduction.", "weaknesses": "- The study’s experimental scope and scale are limited. All models are toy-sized and trained from scratch with pruned vocabularies on synthetic tasks and symbolized formulations that departure significantly from natural languages, making it unclear whether the proposed principles extend to large, pretrained language models operating over natural text. \n- For reasons explained above, I am not convinced that the comparisons against conventional CoT and COCONUT are fair. The uncontrolled and unfair nature of the benchmarking obscures the contribution and significance of the results.\n- Moreover, the self-designation “CoT2” implies direct lineage from the original chain of thoughts. I find this misleading since the proposed method differs fundamentally from both discrete CoT prompting and latent-space reasoning approaches like COCONUT.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wT9JlsmXrD", "reviewer": "Reviewer_sQNo", "summary": "The paper presents a technique of chain-of-thought with continuous valued tokens (termed CoT2) for transformer models, and aims to study the effectiveness of this approach over standard CoT and COCONUT baselines. In particular, the authors use the setup with minimum non-negative sum,  ProsQA, and ProntoQA tasks to show how a shallow transformer can utilize CoT2 for parallel search over possible trajectories and avoid error propagation with intermediate step mistakes.\n\nThe main idea of CoT2 is to use the softmax outputs (probs) during auto-regressive token generation and consider the probability-weighted sum of embeddings as the continuous-valued token for the subsequent generations. Thus, it captures information from the whole vocabulary instead of committing to a single token. Overall, this work deals with small-scale (2-layer) transformers to study the role of this approach for parallel exploration of trajectories.", "questions": "1. In Figure 2(a), the majority@K performance of standard CoT with sufficient K surpasses the CoT2 results, and I also noticed that CoT2 does not show any scaling behaviour in the results. Is it because the continuous tokens were always limited to \"m\" steps?\n\n2. How to decide the \"m\" value for tasks beyond the toy setup? If a standard CoT approach can solve the task with say A CoT tokens, then can CoT2 solve it with B < A continuous valued tokens? A discussion on how to think about these situations can be helpful.\n\n3. Can a model trained on MNNS with 4 input digits and CoT2 transfer the latent reasoning capability to tasks with 5 inputs? Changing the range of values seems to affect the performance, so how well can CoT2 be transferred to out-of-distribution inputs?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The work is well presented with small-scale experiments, toy setups, and theoretical results to convey the CoT2 idea (especially the continuous token construction and fine-tuning/RL) and to build intuition about its effectiveness. The authors also discuss potential RL-based extensions for sampling and policy optimization with continuous-valued tokens, which can also inspire future efforts.", "weaknesses": "The extension of this approach to practical scenarios is not clear/discussed. Since CSFT requires ground truth $\\alpha^*$ for each step, it is unclear how one can compute them for real-world data. There is also no evidence of the benefit of scaling horizontally with more CoT2 tokens in the paper, whereas standard CoT + majority@K (and increasing K) can indeed surpass CoT2.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E4gY2p0rCt", "reviewer": "Reviewer_rbV9", "summary": "The authors propose an approach for continuous chain of thought, where at each step, the model stores a weighted average of all token embeddings weighted by their logit weights. They argue that this allows models to explore several possible reasoning traces in parallel. To perform this technique in practice, the authors propose a supervision approach based on selecting a number of high-performing trajectories in search problems. They show that this approach performs better than standard CoT in the minimum non-negative sum (MNNS) task. They also show theoretically how models with CoT2 are capable of solving this problem. The authors conclude by adapting GRPO for CoT2, which improves accuracy on MNNS.", "questions": "### Suggestions/comments\n- \"AIME or IOI problems\" acronyms should be defined, or avoided (e.g., \"challenging math problems\")\n- \"simplex-weighted compositions\": isn't \"weighted average\" a simpler way of saying the same thing?\n- The theoretical perspective on embedding capacity is nice\n\n\n### Questions\n1. Figure 1, $B=8$, why in $t=3$ is there only 1 embedding $e_1$? Why isn't it the mean of all 8 traces? (similar question for $B=2$, $t=3$)\n2. For teacher forcing, how is a ground truth prefix $z^*_{<t}$ converted into the continuous tokens $z_{t'}^*$? Where do the weights $\\alpha^*_{t'}$ come from? Do you feed the prefix at each step into the current transformer and use its output logits to weight the next token?\n3. What model is generating the supervision distribution $\\alpha_t^*$? Ah, I see in 3.1 that there are problem-specific approaches to generating $\\alpha_t^*$. It would be helpful to specify at the beginning of Section 3 where these $\\alpha$ will come from.\n4. What is the complexity of MNNS?\n5. In proposition 1, what embedding dimension is required for this to work?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The approach is natural and intuitive, and backed up by experiments and theory\n2. The presentation is fairly clear overall\n3. The topic is very relevant for the conference and of high impact", "weaknesses": "1. Some points could be described more clearly (see questions)\n\n### Overall evaluation\nThis seems like a strong paper with a good blend of theory and experiments, proposing a continuous CoT approach that is reasonable and seems to perform well. I'm not familiar enough with the continuous CoT literature to know how novel this method is (it seems like using logits to weight all token embeddings is a very natural thing to do; but perhaps the challenge is figuring out the right way to supervise this method in training).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VwK8wJY6Zl", "reviewer": "Reviewer_EUvY", "summary": "This paper introduces Continuous Chain of Thought (CoT2), a novel latent chain-of-thought method in which the model outputs a convex combination of vocabulary embeddings. This allows the model to explore multiple reasoning paths in parallel within a single forward pass. The authors also propose a method for supervised training, called Continuous Supervised Fine-Tuning (CSFT), and a GRPO based reinforcement learning objective for policy refinement. Empirically, they show that CoT2 outperforms prior approaches such as COCONUT, discrete CoT, and no-CoT on the MNNS, ProsQA, and ProntoQA benchmarks, while converging faster. The experiments also show that the embedding dimension determines the model’s capacity for parallel reasoning, and that reinforcement learning further boosts accuracy and confidence.", "questions": "See the comments from the weaknesses section. Additionally, have you tested CoT2 on other benchmarks such as GSM8K? What would training look like in this setting? It would be quite interesting to see how it compares to COCONUT, since the authors observed a performance decrease compared to discrete CoT.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-motivated and introduces a clear novelty over previous methods. CoT2 itself, as well as the supervised training and reinforcement learning methods are original.\n\n2. The paper provides a theoretical analysis of CoT2, including a proof showing that a single-layer transformer can solve the MNNS problem, and bounds demonstrating improved sample efficiency compared to discrete CoT.\n\n3. The theoretical claims are backed by empirical results showing that CoT2 outperforms prior approaches such as COCONUT, discrete CoT, and no-CoT on the MNNS, ProsQA, and ProntoQA benchmarks, while converging faster. Additionally, the experiments show that the prosed RL-based method further boosts the accuracy.", "weaknesses": "1. A key limitation of CSFT is the reliance of some ground-truth distribution over all intermediate reasoning steps. This is only feasible for the toy tasks like the ones considered in this paper. This dependence seems to hinder the method's applicability to more open-ended reasoning tasks, where such an oracle not available. \n\n2. The need for a new task like the MNNS is not discussed. It seems like the existing datasets are perfectly suited for the methods introduced in the paper, therefore in the current form it seems like its main purpose is to be used in the proof from Section 4, showing that it can be solved by a transformer. \n\n3. Computing each dense vector involves performing a matrix-vector multiplication, where the matrix contains the embeddings of all tokens in the vocabulary. It seems like for large vocabularies, this could be a bottleneck that would reduce the practical gains from parallel explorations of reasoning trajectories.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)