Title: CoT Information: Improved Sample Complexity under Chain-of-Thought Supervision
Venue: NeurIPS 2025
Accepted: yes
Keywords: chain-of-thought

learning theory

statistical learning theory

PAC learning

sample complexity
Abstract: Learning complex functions that involve multi-step reasoning poses a significant challenge for standard supervised learning from input-output examples. Chain-of-thought (CoT) supervision, which augments training data with intermediate reasoning steps to provide a richer learning signal, has driven recent advances in large language model reasoning. This paper develops a statistical theory of learning under CoT supervision. Central to the theory is the *CoT information*, which measures the additional discriminative power offered by the chain-of-thought for distinguishing hypotheses with different end-to-end behaviors. The main theoretical results demonstrate how CoT supervision can yield significantly faster learning rates compared to standard end-to-end supervision, with both upper bounds and information-theoretic lower bounds characterized by the CoT information.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the statistical theory of learning under CoT supervision. It introduces an abstract model of CoT supervised learning, a CoT information measure, covering both finite-cardinality hypothesis classes and infinite hypothesis classes, as well as the agnostic setting. Two types of information-theoretic lower bounds are established, along with their corresponding upper bounds.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- S1: The paper addresses a very important problem in CoT theoretical research, providing a detailed analysis of sample complexity under specific CoT problems, which contributes to understanding the underlying mechanisms of CoT.\n- S2: The paper provides detailed derivation proofs, covering cases from simple to complex.\nWeaknesses\n- W1: The paper lacks a specific analysis of its differences from related work, making it difficult to effectively gauge its novelty. It's recommended to add a detailed comparative analysis with existing work and a specific description of the paper's innovative points.\n- W2: There's a lack of discussion on the practical applications of the paper's theory, despite some application-oriented theoretical analysis being added in Section 6. Adding insights on how the theory guides practical applications or points of application for solving real-world problems would significantly increase the paper's utility.\n- W3: Possibly due to it being a theoretical paper, it contains a large number of symbols, many of which lack detailed explanations (e.g., line 31). This makes it less user-friendly, especially for readers unfamiliar with the relevant theories.\n\nQUESTIONS:\n- The paper models CoT supervision z as an intermediate variable between input x and output y, which is a relatively general modeling approach. Are the derived conclusions applicable to general problems with intermediate supervision? Are there more specific modeling methods for CoT itself?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper develops a statistical learning theory framework to analyse the benefits of CoT supervision, introducing a quantity called **CoT information** that quantifies how much additional signal CoT annotations provide for distinguishing correct from incorrect models. The authors show that, unlike standard supervision, which minimises end-to-end loss, CoT-supervised training minimises a richer loss that includes both intermediate reasoning steps and final answers, leading to potentially much lower sample complexity. Through both upper and lower bounds, the paper demonstrates that informative CoT traces can significantly accelerate learning and provide theoretical guarantees in both realisable and agnostic settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is exceptionally clear and well-organised.\n- CoT prompting and fine-tuning can enhance the LLM performance significantly through empirical results [1,2]; however, it is still unknown why CoT can lead to the performance gain. This paper tried to theoretically understand an important but challenging problem, which is significant.\n\nWeaknesses\n- The primary weakness is that the empirical validation, while effective for illustrating the theory, is confined to symbolic tasks (e.g., learning finite automata). While this is a standard practice for theoretical papers, the work would have been even more compelling if it included a small-scale experiment with a neural model (e.g., a small transformer). Such an experiment would help bridge the gap between the theoretical framework and current LLM practices, showcasing how these insights apply to the very models that motivated the research.\n\nNote: I can't validate the theoretical correctness as my primary expertise is not in statistical learning theory. :)\n\n[1] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., ... & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35, 24824-24837.\n[2] Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., ... & Guo, D. (2024). Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.\n\nQUESTIONS:\n1. The paper shows that an \"informative\" CoT is beneficial. This raises a practical question: what structural properties can make a CoT informative? Have the authors investigate this?\n2. In the agnostic setting, the authors admit that CoT supervision can hurt if the hypothesis class poorly fits the CoT traces. How practical is it to design hypothesis classes that jointly fit outputs and reasoning traces?\n3. Do the bounds extend to stochastic CoT annotations, such as those from RLHF-annotated intermediate steps or from uncertain model generations?\n\nLIMITATIONS:\nSee Weakness.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a theoretical framework for understanding the statistical benefits of chain-of-thought (cot) supervision. The authors introduce a the concept of cot Information, which quantifies how cot annotations can help distinguish between different hypothese classes more effectively than standard input-output supervision. They show that cot supervision can significantly reduce sample complexity, sometimes requiring far fewer examples to reach the same accuracy. The paper provides both tight upper and lower bounds using this newly proposed information measure. Theoretical results are extended to agnostic settings and infinite hypothesis classes. The authors also include case studies, such as regular languages with state trajectories as cot , where cot supervision yields up to 600x gains in sample efficiency. The work positions itself as going beyond prior studies that focused on VC dimension or structural assumptions without explicitly modeling the information gain from chain-of-thought.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: To my eyes, the core idea of cot information proposed in the paper is novel and interesting. The paper presents a new perspective to measure how helpful chain-of-thought supervision can be for machine learning. The idea of cot information captures how much more easily a model can learn when given reasoning steps, not just final answers. To my eyes, the theoretical results are also correctly constructed to show cot can need far fewer examples, and this is true in both clean and noisy data settings. The idea that cot supervision helps learning because it adds information per example is well motivated and explained. The paper also gives clear examples and simulations that match the theory and make the results easier to understand. The overall presentation is clear and careful, and I think the community will benefit from this paper a lot.\n\nWeakness: Primarily this paper can be deemed lack of experimental diversity: Simulations are instructive but limited in scope; broader tasks (e.g., toy tasks in NLP or vision) would strengthen the case. However, this is not crucial for the topic of this particular conference submission, which focuses on theoretical foundation for LLM prompting techniques.\n\nQUESTIONS:\nI thank the authors for their novel contribution. Cot information measure is central to the paper, but for much of the non-theory community, it’s unclear how one could estimate or approximate it in practice. If we have a dataset with cot annotations, is there a way to tell whether the cots are informative under your framework? Even a heuristic or proxy would help bridge the gap between the theory and practical applications.\n\nLIMITATIONS:\nThe authors addressed some of the work's limitation in the conclusion section.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}