Title: Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought
Venue: NeurIPS 2025
Accepted: yes
Keywords: reasoning

chain of continuous thought

superposition

transformer
Abstract: Large Language Models (LLMs) have demonstrated remarkable performance in many applications, including challenging reasoning problems via chain-of-thought (CoT) techniques that generate ``thinking tokens'' before answering the questions. While existing theoretical works demonstrate that CoT with discrete tokens boosts the capability of LLMs, recent work on continuous CoT lacks a theoretical understanding of why it outperforms discrete counterparts in various reasoning tasks, such as directed graph reachability, a fundamental graph reasoning problem that includes many practical domain applications as special cases. In this paper, we prove that a two-layer transformer with $D$ steps of continuous CoT can solve the directed graph reachability problem, where $D$ is the diameter of the graph, while the best known result of constant-depth transformers with discrete CoT requires $O(n^2)$ decoding steps where $n$ is the number of vertices ($D<n$). 
In our construction, each continuous thought vector is a superposition state that encodes multiple search frontiers simultaneously (i.e., parallel breadth-first search (BFS)), while discrete CoT must choose a single path sampled from the superposition state, which leads to a sequential search that requires many more steps and may be trapped in local solutions.
We also performed extensive experiments to verify that our theoretical construction aligns well with the empirical solution obtained via training dynamics. Notably, encoding of multiple search frontiers as a superposition state automatically emerges in training continuous CoT, without explicit supervision to guide the model to explore multiple paths simultaneously.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides a theoretical analysis and experiments on directed graph reachability to explain why Chain of Continuous (Coconut) Thought outperforms Chain of Thought (CoT, the discrete counterpart). The core idea is that CoT is working in a step-by-step manner; the thinking process can only contain one single reasoning path at a time, while Coconut implicitly traverses multiple reasoning paths in parallel within the continuous thought vector. The paper proves that a two-layer transformer with $D$ steps of continuous CoTs can solve the directed graph reachability problem of a graph with diameter $D$. The paper also provides experiments that show that a 2-layer transformer outperforms a 12-layer Coconut transformer with CoT.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**:\n1. The paper is well-written and well-motivated -- trying to understand why Coconut outperforms discrete CoT. \n2. The paper proposes an interesting perspective: Coconut performs multiple reasoning paths in parallel, and tries to prove it on the graph reachability task.\n3. The experimental results, including ablation studies, support the theoretical analysis.\n\n**Weakness**:\n1. The paper focuses solely on synthetic graph reachability tasks; it is unclear how well these proofs and findings can be generalized to most other natural language problems that LLMs face. While the paper aims to answer why Coconut outperforms its discrete counterparts in various reasoning tasks, however, this paper solely focuses on the graph problem and how the advantage can be generalized to other tasks are not discussed.\n2. The continuous thought vectors may saturate in more complex tasks. The graph reachability task is a relatively easy task where a simple reasoning path can be easily represented, and the inference of the next step is also easy (just traverse). However, when the task gets complicated, for example, the math reasoning problem. The continuous thought vectors may not be able to carry many more reasoning paths than the written thoughts in CoT.\n3. The theory part of this paper assumes that the token embeddings are orthonormal; however, this is not true in general LLM's embedding space. [1]\n\n[1] Mikolov, Tomas, et al. \"Efficient estimation of word representations in vector space.\" arXiv preprint arXiv:1301.3781 (2013).\n\nQUESTIONS:\n1. As described in the second weakness, can the viewpoint of this paper be generalized to more complex tasks? \n2. Do you need to increase the capacity of the continuous thought when the task gets more complex?\n3. In the graph reachability problem, can the capacity of the continuous thought reduce the steps needed to get the answer?\n4. As described the the third weakness, how would that affect your conclusions?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-23"}

comments: SUMMARY:
This paper provides theoretical guarantees for this paper: *Training Large Language Models to Reason in a Continuous Latent Space.*

STRENGTHS AND WEAKNESSES:
Strengths: This paper offers rigorous theoretical proofs and simulation experiments that demonstrate its effectiveness on reasoning tasks such as graph reachability. Although the article focuses on graph reachability as a case study, its theoretical analysis and constructions—enabling “relative‐offset” access—apply to any task, with the potential to extend to path search, planning, theorem proving, and other complex reasoning scenarios.

I didn't find any obvious shortcomings in this paper.

QUESTIONS:
According to Lemma 2, it seems the model must extend its reasoning one more “hop” in the graph at each step. To keep training stable, if we move a portion of the language-based reasoning into the latent space at every stage, doesn’t that mean that what used to be a single fine-tuning pass now requires many rounds? If I’ve understood correctly, wouldn’t this greatly increase the overall training cost?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nI am not very familiar with the underlying theoretical framework used in this paper. In my view, this paper develops a theoretical construction showing that two-layer transformers with continuous latent chains can solve directed graph reachability in a number of steps equal to the graph’s diameter, while discrete chains require quadratically more steps. The proofs use practical positional encodings, and experiments on synthetic and math tasks dshow similar behaviors emerge during training.\n\nSTRENGTHS AND WEAKNESSES:\n---\n**Quality (3)**  \n- Strengths:  \n    - Clear separation result between continuous and discrete chain-of-thought expressivity.  \n    - Proofs account for realistic positional encodings (sinusoidal, RoPE).  \n    - Empirical results align tightly with theory, with useful visualizations of latent states.  \n- Weaknesses:\n    - Relies on idealized assumptions (exact orthonormal embeddings, large embedding dimension).  \n    - Requires embedding size proportional to vocabulary, which may be impractical in real LLMs.\n\n---\n\n**Clarity (3)**  \n- Strengths: Overall, the paper is well formatted and has a clear introduction and literature review. \n- Weaknesses:\n    - Heavy notation and many indices make the paper hard to parse for someone new to transformer expressivity.  \n    - Key mechanisms (attention chooser, buffer/MLP filter) could be summarized more succinctly, with full details in the appendix.\n\n---\n\n**Significance (2)**  \n- Strengths: in general, understanding why (continuous) latent reasoning helps with theoretical insights is an important topic and can inspire many future works. \n- Weaknesses:\n    - The paper focused on a toy reachability problem. It is slightly unclear on how the proposed analysis can transfer to real-world reasoning tasks or larger LLMs.  \n    - Practical impact on real world benchmarks remains to be validated.\n\n--- \n\n**Originality (3)**  \n- Strengths: I believe the  theoretical link between continuous superposition and expressivity speedup is fresh. \n\n---\n\nQUESTIONS:\n- How practically useful is this theory for large-scale LLMs?  \n- Can the authors discuss in more details on how the proposed construction transfer to other domains beyond graph reachability?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 1, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper gives theoretical investigation of the expressivity and reasoning capability of LLMs when using continuous chains of thought (CoT), specifically, the COCONUT framework. The authors focus on the graph reachability problem and provide a theoretical construction showing that a two-layer transformer with D continuous latent steps (where D is the diameter of the graph) can solve the problem efficiently. This contrasts with discrete CoT methods, which require O(n^2) steps for graphs with n nodes. The key finding is that continuous latent thoughts act as \"superposition states\", representing multiple search frontiers simultaneously—akin to a parallel BFS—while discrete CoTs sequentially explore paths, leading to inefficiency and possible local minima.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nNovel theoretical contribution: The work formalizes the intuition behind continuous CoT's advantage via a compelling superposition-based analogy and mathematical proof, which is a very innovative way of analyzing reasoning behaviors and has deep implications.\n\nWeaknesses\nLimited scope: The paper focuses specifically on the graph reachability problem, and While the problem is a representative benchmark, the theoretical findings are not yet generalized to broader reasoning tasks (e.g. arithmetic, planning, theorem proving).\n\nQUESTIONS:\n1. Generalization beyond graph reachability\nWhile the theoretical framework is very interesting, it focuses on the setup of graph reachability. Do you anticipate similar superposition-based representations to emerge for reasoning tasks that are not inherently graph-structured (e.g., algebraic reasoning, program synthesis)? Can your theoretical framework be extended or adapted to such domains?\n\n2. Superposition representation robustness\nHow robust are the emergent superposition representations to noise, adversarial graph perturbations, or imperfect supervision? Have you evaluated how the representations evolve when the graph has spurious or misleading edges?\n\n3. Interpretability of the continuous reasoning path\nWhile the superposition representation appears to be powerful, how interpretable is it in practice? Can individual paths or reasoning traces be extracted or disentangled from the latent vectors?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}