Rethinking Chain-of-Thought from the Perspective of Self-Training
Authors: Zongqian Wu
Baoduo Xu
Ruochen Cui
Mengmeng Zhan
Xiaofeng Zhu
Lei Feng
Venue: ICML 2025 poster (2025)
Keywords: Chain-of-Thought
Self-Training
Reasoning
LLM
Primary area: deep_learning->large_language_models

Abstract: Chain-of-thought (CoT) reasoning has emerged as an effective approach for activating latent capabilities in LLMs. Interestingly, we observe that both CoT reasoning and self-training share the core objective: iteratively leveraging model-generated information to progressively reduce prediction uncertainty. Building on this insight, we propose a novel CoT framework to improve reasoning performance. Our framework integrates two key components: (i) a task-specific prompt module that optimizes the initial reasoning process, and (ii) an adaptive reasoning iteration module that dynamically refines the reasoning process and addresses the limitations of previous CoT approaches, i.e., over-reasoning and high similarity between consecutive reasoning iterations. Extensive experiments show that the proposed method achieves significant advantages in both performance and computational efficiency. Our code is available at: https://github.com/zongqianwu/ST-COT.

Rethinking Chain-of- Thoughh from the Perspective of Self-Training
Zongqian Wu
Baoduo Xu
Ruochen Cui Mengmeng Zhan
Xiaofeng Zhu
Lei Feng"
Abstract
Self-Training Process
Chain-of-thoughtt (CoT) reasoning has emerged as effective approach for activating latent capabil- ities in LLMs. Interestingly, observe that both CoT reasoning and self-training share the core objective: iteratively leveraging -ener-egee information to progressively reduce prediction un- certainty, Building on this insight, we propose a novel CoT framework to improve reasoning per- formance. Our framework integrates two key com-- ponents: (i) task-specific prompt module that optimizes the initial reasoning process, and (ii) an adaptive reasoning iteration module that dynami- cally refines the reasoning process and addresses the limitations of previous CoT approaches, i.e., over-reasoning and high similarity between con- secutive reasoning iterations. Extensive experi- ments show that the proposed method achieves significant advantages in both performance and computational efficiency. Our code is available at: hutp:://ittabbcoom//ogiiiaa////a
Ieraaioo 0
Pseudo -labels
Iteration 1
Pseudo -labels
Iteration
Model
Model
Model
Dog Cat Pig
Dog Cat Pig
Dog Cat
Pig
Chain-of Chain-of-Thuuje Thought Process
Iteration 0
Iteration
Iteration
travels km south, 5 km east, north, and 2 km west. How far from home?
CoT
CoT
CoT
km
3 km
3 km 2 km
km
km
2 km
Figure I. Both self-training and CoT reasoning iteratively lever- age model-generated information (pseudo-labels or reasoning processes) to gradually reduce the uncertainty of predictions.
on prompts (e.g., "Let's think step by step") to guide LLMs to generate intermediate reasoning processes relevant to the given question, thereby facilitating logical inference. In contrast, few-shot CoT methods provide some examples that include intermediate reasoning processes from the dataset, allowing LLMs to reference these examples during testing to construct reasoning processes. Both zero-shot and few- shot CoT methods leverage the generative capabilities of LLMs augment question-relevant information, which effectively improves the reliability of inferences. Moreover, this process of information augmentation be applied iteratively, with deeper iterations enabling LLMs to tackle more complex reasoning tasks (Zhong et al., 2024).
1. Introduction
Chain-of-thougle (CoT) reasoning has attracted significant attention in recent years due its capacity to unlock the latent potential of large language models (LLMs) (Wei et al., 2022). By requiring LLMs to explicitly outline intermediate reasoning processes before generating final outputs, CoT effectively improves the reliability of inferences, particularly when tackling complex reasoning tasks.
Previous CoT methods in LLMs can be divided into two categories, i.e., zero-shot CoT (Kojima et al., 2022) and few- shot CoT (Wei et al., 2022). Zero-shot CoT methods rely
Interestingly, we observe that CoT methods share many con- ceptual similarities with self-training (a well-established semi- supersupe framework), regarding leveraging itera- tively model-generated information enhance task perfor- mance (as shown in Figure 1). Concretely, in self- tiffnirainnng, pseudo-labels are iteratively generated for unlabeled data and used to retrain the model, thereby progressively enhanc- ing the generalization capabilities of the model. Inspired by this parallel, this paper aims to improve CoT reasoning performance from the perspective of self-training.
"Equal contribution School of Computer Science and Engi- neering, University of Electronic Science and Technology of China, Chengdu, China Schooo of Computer Science and Technology, Hainan University. Haikou, China "School of Computer Science and Engineering. Southeast University. Nanjing, China. Corre- spondence Xiaofeng Zhu <seanzhuxf@ aaairfaggmaaaammaaa... gmail.com>> Lei Feng <fenglei ses..cdu.c...
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Rethinking Chain-of-Thought from the Perspective of Self- -raining
To this end, we conduct a theoretical analysis of entropy vari- ation in self-training, which demonstrates that the overall uncertainty of predicted samples exhibits decreasing trend. Furthermore, our experimental results validate that CoT rea- soning follows a similar principle, as it iteratively leverages intermediate reasoning processes to gradually reduce the uncertainty on prediction questions. Based on the insight that CoT reasoning is an uncertainty minimization process, propose a novel CoT framework improve reasoning performance, which comprises two main components, i.e., task-specific prompt and adaptive reasoning iteration. Specifically, the task-specific prompt module is designed to search for optimal prompts with the minimum uncertainty. Unlike generic prompts (e.g., "Let's think step by step"), our tailored prompt effectively guides LLMs to generate initial reasoning processes aligned with the intrinsic characteristics of the task, significantly reducing the number of iterations required for L.MSs to obtain correct answers.
soning processes, thereby significantly reducing the number of iterations required for LLMSs to obtain cor- rect answers in CoT reasoning.
we propose an adaptive reasoning iteration module to dynamically refine the CoT reasoning process and address issues of over-reasoning and high similarity between consecutive reasoning iterations.
2. Related Work
2.1. Self-Training
Self-training (Scudder, 1965) has emerged as a widely used approach in semi-supervised learning, aiming expand labeled datasets through pseudo- techniques (Yang et al., 2022). An effective pseudo-labeling strategy ensures that the pscudo-labels assigned to unlabeled data align well with the distribution of labeled samples. Recent research has focused on two main directions: selecting high- high-confidence pseudo-labels and optimizing multi-classifier training frame- works (Amini et al., 2024). For pseudo-label selection, some methods directly incorporate unlabeled samples with high prediction confidence in iterative training (Lee et al., 2013; Zou et al., 2018), while others explore more robust strate- gies such as majority voting (Bartlett et al., 1998), entropy minimization (Grandvalet & Bengio, 2004), uncertainty es- timation (Mukherjee & Awadallah, 2020), noise injection (Miyato et al., 2018), confidence regularization (Zou et al., 2019), and curriculum-based pseudo-labeling (Zhang et al., 2021). However, the presence of incorrect pseudo-labels can mislead model optimization, motivating the development of debiasing techniques and other corrective mechanisms to mitigate their negative impact (Chen et al., 2022).
After establishing initial CoT reasoning processes, further iterative refinement can help improve the reasoning process. straightforward method involves integrating the question, reasoning processes, and output into a new input and reusing the prompt for another reasoning round. However, this intu- itive method encounters two key challenges: (i) the correct predictions in earlier iterations may turn incorrect after mul- tiple rounds, a phenomenon we term over-reasoning, and (ii) the reasoning generated in new iterations often closely re- sembles the previous reasoning. To address these challenges, our adaptive reasoning iteration module is implemented to assess the uncertainty of prediction questions. When the uncertainty is low, the current prediction is adopted as the final output, effectively mitigating challenge (i). However, if uncertainty remains high, the reasoning process continues into the next iteration. In these subsequent iterations, we introduce a new prompt and employ reasoning similarity metrics (e.g., the Jaccard index) to guide LLMs in explor- ing alternative reasoning pathways. By fostering greater diversity across reasoning iterations, address the chal- lenge (ii). thereby enhancing the ability of LLMs to tackle complex reasoning tasks effectively.
2.2. Chain-of-Thought Reasoning
Trustworthy reasoning is essential for large language mod- cls (LLMs), and thain-of- (CoT) prompting has emerged as a key technique to improve reliability and trans- parency by guiding LLMs to generate intermediate reason- ing steps (Chu et al., 2024). Early CoT methods relied implicit prompting to help LLMs solve complex tasks such as arithmetic and commonsense reasoning by decom- posing them into simpler subproblems (Chia et al., 2023), demonstrating strong generalization from limited examples and carefully designed prompts (Sun et al., 2023). Recent advances in CoT focus on improving flexibility by gener- ating diverse reasoning paths, particularly for unfamiliar tasks (Kojima et al., 2022; Ling et al., 2024). Methods like PSPrompting (Wang et al., 2023) and Concise-CoT (Nayab et al., 2024) selectively activate pretrained knowledge ac- cording to task-specific needs, while approaches such as VerifyCoT (Ling et al., 2024) enhance trustworthiness by generating additional question- answer pairs for validation.
Motivated by the conceptual connection between self- training and chain-of-thought reasoning, our key contri- butions can be summarized as follows:
Through a combination of theoretical analysis and ex- perimental validation, we reveal that both self- self-training and CoT reasoning share the core objective of itera- tively leveraging model-generated information to grad- ually reduce the uncertainty of predictions (i.e., entropy minimization), thereby turning some early incorrect predictions into correct ones.
We design a task-specific prompt module to search for optimal prompts that generate high-quality initial rea-
2

---

Rethinking Chain-of-Thought from the Perspective of Self- Training
3. Understanding Uncertainty in and Chain-of-Thought Reasoning
Building on Lemma 3.1, assuming that pseudo-labels follow Bernoulli distribution, and leveraging the relationship be- tween the dot product of the classifier and the samples with 8, we can readily derive the entropy variations for different samples and identify the regions to which different types of samples belong, as stated in the following theorem.
This section investigates the uncertainty in self-training and CoT reasoning by analyzing variations in information en- tropy and semantic entropy. We observe that while both methods exhibit a general trend of entropy reduction, this pattern does not hold for every data point. Performance improvements are primarily driven by data points where entropy decreases, enabling transition from incorrect to correct predictions. Specifically, Section 3.1 presents theoretical analysis of entropy variations in self-training. Building on these insights, Section 3.2 extends the discus- sion to CoT reasoning, deriving key conclusions that help further enhance the performance of CoT reasoning.
Theorem 3.2. Under the assumptions of Lerma 3.1, let d = 2 and suppose (/0) pr ~ (er x)) is the pseudo- label of x, where ((2) = IIte- - Define I(x)) as the line xfa- =0, 0, where at is perpendicular to a. Let A(n, denote the region swept by ((a1) rotating to l(a2) along the trajectory of Binit towards ju during self-training. Denote
Ho(x) = |ntlgf///z]] be the entropy of g0 For t < T T-1, 1, with probability at least 1 - 5, the entropy changes as follows: (i) Hy(x)) first decreases and then increases if 2 E A(Bo, p): (ii) Hz(z) decreases if x E A(p, P ); (iii) H₂(x) forst increases and then decreases if E A(30 -t)); and (iv) (z)x) increases if x E &A('*, ,80)-
3.1. Information Entropy Variation in Self-Training
The primary objective of self-training is to reduce predic- tion uncertainty by leveraging pseudo-labels generated by the model, where the uncertainty can be quantified using information entropy. As empirically shown in Figure 8(a) in Appendix, the average entropy of predictions across sam- ples progressively declines throughout the iterative training process. This reduction facilitates some samples being cor- rected from initial mispredictions to accurate predictions.
Based on Theorem 3.21, entropy variation during the it- erative process of self-training can be classified into four patterns: (i) a decrease followed by an increase; (ii) con- sistent decrease; (iii) a decrease followed by increase; and (iv) consistent increase. These patterns are visualized in Figure 2(a), where the light blue, gray, dark blue, and yellow regions correspond to each respective entropy vari- ation pattern. Specifically, we partition I² using vectors Se ub. 86 ,1r² and delineate the positions of the initial classi- fier Piii and optimal μt. The updates the classifier during self-training can be interpreted as the gradual rotation of the initial classifier Siit toward the Bayes-optimal classifier μ.. Moreover, due to the small initial error of O(1) in the classifier, the angle 0g between 30 and y is relatively small. As a result, the regions A(So. (u) and A(3t , occupy an relatively small proportion of R indicating that most samples exhibit monotonic entropy changes.
However, not all samples exhibit the expected reduction in information entropy. This deviation can be attributed to incorrect annotations within pseudo-labels, which may misdirect the model's training direction. To provide deeper insights into the entropy variation of self-training, we con- ducted a theoretical analysis using a Gaussian mixture model. Specifically, an initial classifier with a small yet constant error was constructed and iteratively updated with pseudo-labels based on model predictions. Through this approach, we examined the classifier's progression from its initial state toward the optimal state, capturing the un- derlying mechanism of entropy variation. Based on the conclusions of (Frei et al., 2022) regarding the sample com- plexity of unlabeled samples in self-training, we discover that the intermediate classifier update process is a rotation of the initial classifier towards the Bayes optimal classifier. This conclusion is stated in the following lemma.
From experimental and theoretical analyses of self-training, we derive five key insights regarding entropy variation: (i) while the overall entropy of samples typically decreases, this trend does not hold for every individual sample; (ii) most samples exhibit monotonic entropy changes; (iii) the ef- fectiveness of self self-training arises from samples undergoing entropy reduction, which enables their transition from incor- rect to correct predictions; (iv) in some cases, increase in entropy could lead to reversal of predictions, causing pre- viously correct samples to become incorrectly classified; (v) these entropy variations are intrinsically linked to the spatial relationships between the samples and the classifiers Bmit and J.. These insights will help deepen our understanding of CoT from the perspective of entropy variation.
Lemma 3.1. Suppose (s, y) D where D is a Gaussian mixture models in Rd x {+1} with mean μ satisfying ||||| @(1), iee., y ~ 1n) and xly ~ N I). Let {(=)= = log(1 + eexp(=xp>--- ex)(-z)), and assume a 2 max(1, [ld()). Assume we can access a initial classifter Piiit. which satisfies Pr(xynDD(y (y # sgu(B(n = O(1). Let 5, 8 E (0, 1). and assume that B- = ((E suppose O2 is the angle between 3r and p. then by running algorithm with step size 7 and batch size B, when t < T- 1, 0 2 d.41 holds with probability at least - 8, and with probability at least 5, 0r- S O()).
(dde-11), 17 = (dd=1)),
Prooff of Lemma 3.1 and Theorem 3.2 are in Appendix A.1.
3

---

Rethinking Chain-of-Thougle from the Perspective of Self- SelfnTraining
soning path is a specific path within T that begins the root node corresponding to q and terminates at a leaf node representing an answer
Punt
Pet
Pa
0g
Building on the formalization of reasoning structures, we in- troduce the definition and computation of semantic entropy. For clarity, we present the formulation where the LLM input consists only of the question g and the prompt p.
Pb-
Answers with the some semantics (Incorrec)
Correct
Definition 3.4 (Semantic Entropy). Let A Answer 7) be the set of answers generated by an LLM for a given question q and prompt p. Assume that this answer set can be partitioned into several disjoint semantic clusters C [Ci)ke(a), such that A -ie(n) C. where exactly one cluster contains the correct answer. Let g(C q.P; denote the probability distribution over C. The semantic entropy is then defined as Ecl- log g(C' | 4, 7)]. In practical use, let LLMs generate N distinct answers Á = fa; HEENY which are then divided into k semantic clusters é = {Cz) By normal- izing, obtain discrete probability distribution {ø)) se()) where = |Cyl /N, J 93 = 1. Consequently, we can use ē = - - 9, log 9j as an approximation of e.
(a) Seif-Training
(b) Chain-of Dhain-of-Thoor oh-Tuhtght
Figure 2. Visualizations of entropy variations in the iterative pro- cess of self-training and CoTT' reasoning. In the self-training diagram, the iterative process represents the gradual convergence of the initial classifier S.i toward the Bayes optimal classifier μ. At each iteration, changes in the angle between the classifier and the samples in different regions correspond to entropy variations within those samples. In the CoT reasoning diagram, each node in the directed acyclic graph represents computation, with red nodes indicating erroneous computations. Each ellipse denotes set of leaf nodes, corresponding to semantically equivalent an- swers, and the numbers indicate their respective iteration rounds. Bold arrows used to represent complete reasoning paths. As the iterations proceed, these paths are gradually corrected. The semantic entropy each iteration is determined by the distribution of generated answers across different semantic categories.
In self-training, pscudo-labels guide the initial classifier to- ward the Bayes-optimal classifier. Similarly, in CoT, the reasoning process aims to steer the LLM's predictions to- wards the correct path, and thus, getting the correct answer. Building on this analogy and the discussion in Section 3.1, we propose the following definitions for CoT reasoning.
3.2. Semantic Entropy Variation in Chain-of- ofTThughht
Chain-of-thought (CoT) reasoning, akin to self-training, relies on model-generated information to enhance task per- formance. Specifically, CoT aims to reduce prediction un- certainty in LLMs by leveraging intermediate reasoning processes generated by the models themselves. This uncer- tainty can be effecttiveyy quantified using semantic entropy (Farquhar et al., 2024). To formalize this concept, we begin by defining the LLM generation process and the mechanism of iterative CoT reasoning, then introduce the definition of semantic entropy, and finally discuss its parallels with self soff-training, drawing conclusions from this analogy.
Definition 3.5 (Initial and Optimal Paths). The initial reasoning path, To. is the reasoning path generated by LLM for a given question q using an initial prompt po. The set of optimal reasoning paths, Ropt. comprises those reasoning paths r E R(g, T) for which the probability of deriving the correct answer, Acorrect, Jair (Acorrect |r), greater than or equal a predefined high confidence threshold vh.
In Section 3. the classifier 3 is updated through informa- tion augmentation using unlabeled data. In contrast, the update of reasoning paths in CoT reasoning is achieved by in in-depth problem analysis, transforming the problem into more tractable sub-problems, and awakening unused inter- nal knowledge within LLM directly aiding problem-solving Based on this, existing reasoning paths are updated through error correction or by considering different perspectives.
Let LLM(.) denote a probabilistic language model. We write s' ~ LLM(s;7) r) to indicate that the output s' is sam- pled from the model given input s and temperature param- eter T. Based on this formulation, we next define how an LLM performs reasoning to solve complex problems.
Definition 3.3 (Reasoning Structures). Given a question q. a prompt p, and a temperature parameter т, the set of crasoning-answe pairs (r, a) generated by LLM is de- noted by [(q,p; pt T). with its associated joint probability density denoted by J. A reasoning tree T (V, E) is directed acyclic graph (DAG) constructed by the LLM to represent the computational steps from the question to potential answers. Each node U E V is associated with a state s(u) E {0, 1} indicating whether the reasoning at that node is correct (s(r) = 1) or incorrect (s(v) = 0). rea-
Definition 3.6 (Iterative CoT Reasoning). Iterative CoT is process that refines reasoning paths over successive iter- ations. A new reasoning path (re+1, a++1) is generated by LLM based on the original question g, an adaptive prompt p', and the history of previously generated paths ro.... Tz- The update from r, to T typically involves identifying the first erroncous node o, in r, (where s(v;) = 0), back- tracking, and generating a corrected sub-path.
4

---

Rethinking Chain-of-Thought from the Perspective of Self- Training
To further investigate how semantic entropy evolves within the CoT framework, we adopt the following assumption. Assumption 3.7. As the semantic entropy e decreases, the likelihood of the LLM with CoT reasoning producing a correct answer to question q increases.
74
AQUA Dataset
76-
Aosura (N)
Accurccy (%)
-2
74
70
LastLetter Dataset
Building on the preceding definitions and Assumption 3.7, the iterative process of applying CoT reasoning to tackle complex questions can be interpreted as the L.M progres- sively producing and correcting to search for an optimal reasoning path within Ropt, starting from an initial reason- ing path Vo- This process is often marked by gradual reduction in semantic entropy, supported by empirical evi- dence presented in Figure 3. The decreasing entropy reflects the refinement of predictions, enabling the correction of initial errors and the generation of accurate answers.
68
0.94
1,01
1.17 Semantic Entropy
1.07
1.29
1.26
1.29
1.35 1.57 1.39
Semanric Entropy
Figure 3. Accuracy under varying levels of semantic entropy on the AQuA and LastLetters datasets (based on 100 sampled instances).
rive five important conclusions regarding semantic entropy variations in CoT reasoning: (i) although the overall seman- tic entropy of questions generally decreases, this trend does not apply to every individual question; (ii) when the reason- ing processes of the new iteration are excessively similar to those in the previous iteration, the semantic entropy of certain questions may exhibit no changes; (iii) the effective- ness of CoT reasoning stems from questions undergoing semantic entropy reduction, facilitating a transition from incorrect correct predictions; (iv) for some questions, an increase in entropy could lead to prediction reversals, where previously correct predictions become incorrect; (v) these variations in semantic entropy are tied to the spatial rela- tionships between the initial state and the optimal reasoning paths of the LLMs with CoT reasoning on a given question.
Similar to self-training, not all questions show the expected reduction in semantic entropy during CoT reasoning. This phenomenon may arise from the presence of noisy infor- mation in reasoning processes, which could misdirect the reasoning trajectory of LLMs. To verify this, we conducted a simple experiment involving three rounds of iterative CoT reasoning on the AQuA arithmetic dataset, analyzing the semantic entropy variation of LLMs. The results, presented in Figure 8(b) in Appendix, reveal three distinct patterns of semantic entropy variation: (i) monotonic increase or decrease; (ii) increase followed by decrease or decrease followed by increase; (iii) no change. These patterns are visualized in Figure 2(b). For example, with a sampling number N = 3 for reasoning processes, if the first iteration yields three semantically distinct answers, this reflects high semantic uncertainty. If the second iteration converges to two distinct answers, uncertainty is reduced. By the third iteration, if all processes yield the same semantic answer, uncertainty may reduce to zero, indicating the LLM has identified an optimal reasoning path leading to the correct answer. Moreover, the other patterns of semantic entropy variation can also be similarly derived.
We will demonstrate in the next section how these findings can be utilized to further enhance CoT performance.
4. Methodology
Building on the conclusions from the analysis of semantic entropy variation in CoT reasoning presented in Section 3.2, we propose a novel CoT framework to improve reasoning performance. Specifically, we design a task-specific prompt module in Section 4.1 to guide the LLMs in generating high-quality initial reasoning processes. Next, in Section 4.2, we introduce an adaptive reasoning iteration module to refine the reasoming process and address the limitations of traditional CoT approaches, i.e., over-reasoning and ex- cessive similarity between consecutive reasoning iterations. An overview of our proposed CoT framework is provided in Figure 4. Detailed information about the task-specific prompt module is provided in Figure 6 in Appendix.
Although iterative CoT reasoning does not update model parameters, it dynamically adjusts the context (the set of reasoning paths). This process essentially achieves a tight- ening of the distribution in the generation space. That is, the longer correct sub-path r' within reasoning path r becomes, the smaller the scope of the answer set at the ter- minus of r' (i.e., the leaf nodes of the reasoning tree) (see Figure 2(b)). Consequently, the proportion of the cluster corresponding to the correct answer increases, leading decrease in semantic entropy. This reduction in semantic entropy, equivalent to an increased probability of the correct answer cluster, aligns with the mathematical objective of label distribution entropy reduction in self-training.
4.1. Task-Specific Prompt
The conclusion (v) presented in Section 3.2 underscores the pivotal role of CoT reasoning quality in the initial iteration. A high-quality initial CoT reasoning process effectively reduces the distance between the initial state and the optimal reasoning paths, thereby decreasing the number of iterations required for the LLMs to converge to the correct answers. Given that the initial CoT reasoning process generated by
Based on the insights gained from self- trainini in Section 3.1 and the experimental analysis of CoT reasoning, we de-
5

---

Rethinking Chain-of-Thoughe from the Perspective of Self- Training
Iteration
Iteration 2
P1: Let's think step by step. how to calculate specific quantities based on the given scenarios and conditions in each situation, (Semantic Entropy: 0.64)
Input Question 4:
Input Question o Prompt.
Prompt
P2: Let's think step by step. how to calculate the total quantity of objects or items mentioned in the sentences.
Temporary New Input
Temporary New Inpul
Reasoning qu P. Rj
Reasoning q..15. R2
(A....)
(Semantic Entropy: 1.13)
LLMS
Prompt
Prompt."
Pm: Let's think step by step how to break down the mathematical operations and identify the key concepts to solve them. (Semantic Entropy: 0.91)
Prediction Ai
Else Merge as New Input
Prediction Ae'
Else Merge as New Input
Ifej 8:
If <
Final Output
Final Oulput
Figure 4. The flowchart of the proposed CoT framework consists of two key modules, i.e. Task-Specific Prompt (light purple block) and Adaptive Reasoning Iteration (light blue block). Specifically, the task-specific prompt module first utilizes LLMs generate candidate prompts and evaluates their semantic entropy the given dataset. The prompt with the lowest entropy is selected as the optimal prompt p. providing guidance for the subsequent adaptive reasoning iteration module to produce high-quality initial reasoning. In the adaptive reasoning iteration module, the uncertainty is caleulated each iteration and compared a predefined threshold 6. This evaluation determines whether to accept the current prediction as the final output or to proceed to another iteration. If the uncertainty remains high, new reasoning round is initiated with new prompt v", designed to introduce diversity compared previous reasoning steps. This iterative process continues until the uncertainty is substantially reduced or the maximum number of iterations is reached.
the LLMs based on the provided prompts, this highlights the critical necessity of optimizing prompts to enhance the quality of reasoning processes from the outset.
Next, we sample another disjoint question set Q" {d' .'.... .4') from Q, ensuring Q' n Q" = 0. Each candidate prompt from P concatenated with the questions in Q" and used for zero-shot CoT inferences (Kojima et al., 2022). During inference, the mean semantic entropy for all questions in Q" is calculated under each candidate prompt, yielding in the set F {el, e2, .... Cm }- For clarity, we define a simple mapping function f : - E to repre- sent the relationship between candidate prompts and their corresponding mean semantic entropy values.
Previous CoT methods (Wei et al., 2022: Ling et al., 2024) often rely on general prompts (e.g.. "Let's think step by step") to guide the reasoning process. Although such prompts are effective for generic tasks, they frequently fall short of capturing the nuances of domain-specific or fine- grained tasks, resulting in inadequate reasoning processes. To address this limitation, we propose a task-specific prompt module that automatically searches for the optimal prompt tailored to the task's ss characteristics. Specifically, our ap- proach begins by constructing an instruction: # Instruction: "Let's 's think step by step" is a general prompt that can guide the LLMs to produce reasoning processes. However, in spe- cialized domains, this prompt may lack accuracy and clarity. Below is dataset sample. Please enhance the "Let's 's think step by step. 4s" prompt by adding a sentence in the %s section to better fit the dataset's 's characteristics.
(2,,
Based on conclusions (i) and (iii) from the analysis of seman- tic entropy variation in Section 3.2, we can infer that lower semantic entropy corresponds to better CoT performance. Consequently, the prompt with the minimal semantic en- tropy can be regarded as the optimal prompt:
p D= f- (Carg (Carg mirr (eslece
(I)
where f 1-) function maps the minimum mean semantic entropy value back to its corresponding prompt p. This process ensures that the selected prompt ÿ minimizes the se- mantic uncertainty associated with the specific task, thereby improving the quality of initial reasoning processes.
We then sample question set Q' = [ & ......... . from the dataset Q, representing the task distribution. This set Q' is concatenated with the instruction and fed into the LLMs, which performs rounds of sampling to generate candidate task-specific prompt set P = {pI, P2, PPm)
During the testing phase, we concatenate the optimal prompt p derived from Eq. (1) with the question, and input the com-
6

---

Rethinking Chain-of-Thought from the Perspective of Self- Training
bined text into the LLMs. Following the self-consistency CoT approach (Wang al., 2022), we perform N rounds of sampling to generate diverse reasoning processes;
greater divergence between reasoning iterations, enabling exploration of alternative paths and enhancing the ability of LLMs to solve complex questions. Specifically, we design a new prompt p" to replace the p. fostering a departure from prior reasoning steps: # p": Based on the above thoughts, reevaluate from alternative perspectives to produce deeper, solution-orienter insights that go beyond prior inferences. Focus on identifying unexplored assumptions or challenges in the question context, and propose new processes.
R²= = (LLM(Concat((::))) D)); j 1,2, ...., ...,..... N
(2)
where LLM(.), denotes the j-th sampling result produced by the LLMs. R. = [r',1, .'...,, v represents the set of reasoning processes generated, qi corresponds to the i- th question in the dataset 2, and the Concat(-) function refers to the sequential concatenation of the specified texts. Next, each of the N reasoning processes obtained from Eq. (2) is individually concatenated with the question qi and the prompt P. The resulting concatenated texts are then fed into the LLMs to generate predictions:
Then, we concatenate the new prompt p². ,the original ques- tion qi the previous reasoning processes RV and the previ- ous predictions A to generate a new round of reasoning:
R ULLM(Concatti 9. rl.g, oh.3. p')) 1 j= NN,
(4)
A =
P. (. j j= I,..., N}. (3)
the prompt p' encourages LLMs to critically reflect on prior information, producing reasoning steps R. that surpass and differ from the previous I2,- To ensure sufficient divergence between new and previous reasoning, we measure their sim- ilarity using the Jaccard index (Jadhao & Agrawal, 2016):
where A/ = f ./.2 (..g...,0'',N represents the set of pre- dictions generated by LLMs. At this stage, two options are available: (i) select the most frequent class from the N answers in A as the final output (iteration terminates); (ii) concatenate the question qi, the reasoning processes R', and the predictions A for a new round of reasoning and prediction. If option (ii) is chosen, when should the iteration be stopped? Meanwhile, how can we ensure that the newly generated reasoning and predictions surpass the previous ones? These issues will be discussed in depth in Section 4.2, where corresponding solutions will also be proposed.
[R2 n RA R U R.
o = ==mmm R2, R2)
8 E R,,
(5)
if sy is greater than a predefined threshold т, Eq. (4) is reapplied for resampling until the condition is met. Finally, the LLMss discard the earlier reasoning R. and the prediction A₁, generating new predictions based solely on R.:
A"" (LLM(Connat(q... 1 = 1....,, N}, (6)
4.2. A Adaptive Reasoning Iteration
The conclusion (iv) presented in Section 3.2 highlights that once the reasoning processes guide the LLMs to predictions with low uncertainty, deeper iterations do not further reduce semantic entropy. Instead, such iterations often introduce noisy information, undermining predictive accuracy. This phenomenon, which we term over-reasoning, risks altering correct early predictions during subsequent iterations. To address this, we calculate the semantic entropy ef of the predictions AA and compare it against a predefined threshold 6. If ef S d, the predictions A/. derived from reasoning processes R,, is accepted as the final output. By leveraging semantic entropy to quantify the uncertainty of LLMs, we can stop iterations at the right moment to output predictions, effectively mitigating the risk of over-neas.ning
After obtaining A/ via Eq. (6), its semantic entropy el is computed and compared to ô again. ee > 8, the process repeats (Eqs. (4) (6)) until the uncertainty drops to 5 the maximum iteration count T is reached.
When semantic entropy remains above d after T iterations, three scenarios may explain this outcome: (i) previous iter- ations contained valid reasoning steps, but randomness in LLMs sampling introduced biases in calculating semantic entropy. Increasing N may mitigate this issue; (ii) effective reasoning paths exist but have yet to be discovered by LLMs. Extending T could allow LLMs to identify such paths; (iii) the limitations of LLMs make the question inherently un- solvable, rendering further iterations unproductive.
Conversely, if e/ >> 5, the reasoning process proceeds to the next iteration to further reduce uncertainty. naive solution involves concatenating the reasoning processes Rt predic- tions AY, and question qi from the previous iteration while reusing the prompt p. However, this approach often results in new reasoning processes that closely resemble previous iterations. As demonstrated by the conclusion (ii) presented in Section 3.2, such high similarity could hinder further en- tropy reduction. To overcome this, we propose introducing
A Although increasing T can enhance performance in the sec- ond scenario. also escalates time and computational costs. To strike a balance between performance and efficiency, the strategy proposed in this paper is to stop further iterations when the entropy remains above ó after T iterations. Instead, we apply majority voting across the predictions from all it- erations, selecting the most frequent prediction as the final output. This approach enhances overall reliability while effectively reducing computational overhead.
7

---

Rethinking Shain-of-Thoughe from the Perspective of Self- Trainini
Method
Arithmetic
Commonsense
Symbolic
Overall
MultiArith
GSM8K
SingleEg
AddSub
AQuA
SVAMP
STQA
CSQA
Letter
Coin
Avg. (%)
RE2
91.7
76.3
81.5
72.9
54.3
78.3
66.2
74.9
57.0
91.2
74.4
RE2 + SC
96.2
77.3
87.0
80.0
60.6
83.2
68.6
77.5
63.8
98.0
79.2
Zero-Shot
51.2
10.8
62.4
56.7
38.6
56.3
66.2
74.5
1.4
50.2
46.8
Zero-Shot-CoT
92.8
74.7
84.4
74.7
55.5
77.0
63.5
73.6
55.0
93.4
74.5
Paro-Shot-CoT- + SC
95.7
79.2
88.8
81.3
63.0
82,2
65.9
75.3
66.2
97.2
79.5
97.0
81.1
90.0
84.8
65.7
85.5
66.7
76.7
68.4
97.6
81.4
TSP
(+1.3)
(+1.8)
(+1.2)
(+3.5)
(+2.7)
(+3.3)
(+0.8)
(+1.3)
(+2.2)
(+0.4)
(+1.9)
96.7
82.6
92.1
87.1
69.3
87.1
67.5
77.5
75.8
97.2
83.3
ARI
(+1.0)
(+3.4)
(+3.3)
(+5.8)
(+6.3)
(+4.9)
(+1.6)
(+2.2)
(+9.6)
(+0.0)
(+3.8)
98.2
83.0
92.9
88.4
70.1
87.5
66.7
76.7
77.2
96.4
83.7
+SP ARI
(+2.5)
(+3.8)
(+4.1)
(+7.1)
(+7.1)
(+5.3)
(+0.8)
(+1.4)
(+11.0)
(-0.8)
(+4.2)
Table Accuracy (%) across ten reasoning datasets from three categories of zero-shot reasoning tasks. The number of self-consistency (SC) sampling is fixed at 3 for all cases. Blue and red fonts indicate increases and decreases in task performance compared to the "Zoro-Shot-CoT SC" method, respectively, while bold font highlights the best performance in each column.
5. Experiments
Method
MultiArith
GSM8K
Avg. (%)
Contrastive-
89.7
69.4
79.6
5.1. Experimental Settings
Contrastive-CoTT SC
93.0
71.9
82.5
Few-Shot
78.3
53.8
66.1
We evaluate our CoT framework on 10 reasoning datasets, including six arithmetic datasets (i.e., MultiArith (Roy & Roth, 2016), GSM8K (Cobbe et al., 2021), SingleEq Kedzios-Ki et al., 2015), AddSub (Hosseini al., 2014), AQuA (Ling et al., 2017), and SVAMP (Patel al., 2021)), two commonsense reasoning datasets (i.e., Strate- gyQA (Geva et al., 2021) and CommonsenseQQ (Talmor et al., 2018)), and two symbolic reasoning datasets (i.e., LastLetter and CoinFlip (Wei et al., 2022)). We utilize GPT- 3.5-turbo-0125 as the foundation model for all experiments, given its accessibility and cost-effeetiveeesss.
Few-Shot-Col
94.3
69.1
81.7
New-Shot-CoTs SC
97.2
73.7
85.5
97.3
77.4
87.4
+ARI
(+0.1)
(+3.7)
(+1.9)
Table 2. Accuracy across the MultiArith and GSM8K datasets from the arithmetic category of few-shot reasoning tasks. The number of self-consistency (SC) samplings is fixed at 3 for all cases.
GSM8K datasets. The results of these experiments are presented in Table I and Table 2, respectively.
For the zero-shot task, the baseline zero-shot method achieves an overall average accuracy of 46.8%. Incor- porating CoT reasoning (Zero-Shot-CoT)) significantly en- hances performance, raising the accuracy to 74.5%. Fur- ther improvement is observed with the integration of self- consistency (SC), which increases the average accuracy to 79.5%. Building on this foundation, the addition of task task-specific prompt (TSP) and adaptive reasoning itera- tion (ARI) modules further elevates the average accuracy to 83.7%. This represents 4.2% improvement over the SC ap- proach, demonstrating the advantages of our method across various task categories. Furthermore, our method showed the most significant performance on arithmetic datasets. Specifically, compared to the SC approach, the average performance across six datasets increased from 81.7% to 86.7%. This improvement can attributed to the fact that deep reasoning enables the LLMs to systematically identify solution pathways. In contrast, the performance gains on commonsense datasets are relatively weak, even approach- ing the level of the zero-shot method. This limitation may arise from the dependency of these tasks on the prior knowl- edge of the LLMs. If the relevant commonsense knowledge was not encountered during pre-training, deep reasoning alone is insufficient to address the question.
Our evaluation adopts progressive comparative approach. Specifically, first test zero-shot reasoning by directly inputting questions into the LLMs without prompts. Next, we employ zero-shot CoT (Kojima et al., 2022), utilizing general prompts with greedy decoding to generate answers. Finally, we implement zero-shot CoT with self-connistency (Wang et al., 2022), which enhances accuracy through mul- tiple decoding attempts and voting mechanism. Building on these baselines, we propose novel CoTT framework comprising two modules, i.e., task-specifie prompt (TSP) and adaptive reasoning iteration (ARI). The TSP module improves the initial CoT reasoning process by replacing generic prompts with task-specific ones, while the ARI module further enhances task performance through iterative refinement of the reasoning process. Moreover, we also compared two popular CoT methods, i.e., RE2 (Xu al., 2024) and Contrastive-CoT (Chia et al., 2023).
5.2. Main Results
We followed literature (Kojima al., 2022) construct zero-shot reasoning tasks across all 10 datasets, and per- formed few-shot reasoning tasks on the MultiArith and
8

---

Rethinking Chain-of-Thought from the Perspective of Self- Training
In the few-shot task, the SC approach achieves an average accuracy of 85.4%. The incorporation of the ARI module results in a significant performance improvement, raising the accuracy to 87.4%. However, the TSP module is not applicable to few-shot tasks, as the LLMs have already utilized the limited examples provided to generate optimal reasoning the initial iteration, rendering the reconstruction of a new reasoning process through TSP unnecessary.
708
70.1
71,7
71.3
Our Method
Accmey (%)
70
Time (hous)
Fixed Iteration
Bon
4h 18
63
3b 29m
2h 50m
2h 20m
60.2
62.6
Sm
00.2
39.8
3
Number of lieration
Number of Iteration
Based on the above analysis, it is evident that the proposed ARI module enhances performance in both zero-shot and few-shot tasks, demonstrating its potential as a plug-and- play solution applicable across all CoT methods. Similarly, the TSP module can be applied to zero-shot methods to improve the quality of reasoning in the initial iteration. Ab- lation experiments presented in the last four rows of Table 2 confirm that, compared to the ARI module alone, the TSP module contributes an average performance improvement of 0.4%, thereby validating its effectiveness.
(a) Accuracy
(b) Time costs
Figure 5. Accuracy and time costs of adaptive reasoning iteration compared to the fixed reasoning iteration the AQuA dataset.
entropy. The iteration process is terminated as soon as a low- uncertainty state is reached, ensuring predictions are made at the optimal stage. For example, as illustrated in Table 7, the semantic entropy after the first iteration falls below the predefined threshold of 0.95 (indicating that at least two out of three elements in the prediction set are consistent). At this point, further iterations are deemed unnecessary, and the current prediction is finalized as the output.
5.3. Adaptive Versus Fixed Iterative Reasoning
To further verify the effectiveness of our proposed method, we compared its accuracy and time costs with the fixed iterative reasoning approach on the AQuA dataset.
When the uncertainty is high and further iterations are re- quired, the issue of high similarity between consecutive iterations may arise. To address this, we propose a novel prompt p" to encourage greater divergence in subsequent reasoning iterations, with the Jaccard index introduced to quantify this diversity. If insufficient diversity is detected, the reasoning process is resampled until predefined condi- tions are met. This strategy effectively ensures the diversity and independence of consecutive reasoning iterations. For example, as detailed in Table 7, the prompt p* and resam- pling strategy successfully guided the L.MMs to generate more distinct reasoning paths, transforming an initial incor- rect prediction into a correct one. Furthermore, as shown in Table 5, our method achieves lower similarity between consecutive iterations compared to existing approaches.
5.3.1. COMPARISON OF ACCURACY AND TIME COSTS
In terms of accuracy (see Figure 5(a)), our method demon- strates clear advantage. It achieves 70.8% at the second iteration and maintains stability, reaching 71.3% by the fifth iteration. In contrast, the fixed iteration method shows slower improvement, peaking at 67.3% and then dropping to 62.6% by the fifth iteration. Regarding time cost (see Figure 5(b)), both methods exhibit a linear growth trend, but our method is significantly more time-efficient. The time cost of our approach increases gradually from hour and minutes at the first iteration to 4 hours and 18 minutes the fifth iteration, reflecting moderate growth rate. In comparison, the fixed iteration method follows steeper trajectory, with the time cost rising from I hour and 5 minutes to 6 hours and 29 minutes by the fifth iteration.
By dynamically adjusting reasoning iterations and promot- ing diversity in reasoning paths, our approach enhances accuracy while significantly reducing computational costs.
6. Conclusion
Overall, our adaptive iteration outperforms fixed iteration in both effectiveness and efficiency, achieving an optimal balance between the two as early as the second iteration, highlighting its practicality and strong performance.
This paper explores the conceptual parallels between chain- of of-thought (CoT) reasoning and self-training, identifying their shared objective of iteratively leveraging information augmentation to minimize prediction uncertainty. Through theoretical analysis and experimental validation, we reveal semantic entropy dynamies in CoT reasoning and propose improvements, including task task-specific prompt module to optimize initial reasoning processes and an adaptive reason- ing iteration module to mitigate over-reasoning and enhance reasoning diversity in consecutive iterations. Collectively, these innovations significantly improve the performance of CoT reasoning in addressing complex tasks.
5.3.2. WHY OUR METHOD WORKS?
Our proposed method significantly surpasses fixed reason- ing iteration by effectively addressing two critical chal- lenges: (i) over-reas,ning, and (ii) high similarity between consecutive reasoning iterations. To resolve the issue of over-reasoning, we incorporate mechanism that quanti- fies the prediction uncertainty of the LLMs using semantic

---

Rethinking Chain-of-Thought from the Perspective of Self- Sef-Training
Acknowledgements
benchmark with implicit reasoning strategies. Transac- tions of the Association for Computational Linguisties, 9: 346-361, 2021.
This work was partially supported by the National Key Re- search and Development Program of China under Grant No. 2022YFA1004100, the Natural Science Founda- tion of Guangdong Province of China under Grant No. 2024A 1515011388 and the National Natural Science Foun- dation of China under Grant No. 62476048.
Grandvalet, Y. and Bengio, Y. Semi-supervised learning by entropy minimization. In NeurIPS, volume 17, 2004.
Hosseini, M. J., Hajishirzi, H., Etzioni, O., and Kushman, N. Learning to solve arithmetic word problems with verb categorization. In EMNLP, pp. 523-533, 2014.
Impact Statement
Jadhao, A. and Agrawal, A. Text categorization using jac- card coefficient for text messages. International Journal of Science and Research, 5(5):2047-20500 2016.
This paper presents work whose goal advance the field of Machine Learning. There are many potential societal consequences of our work, none which feel must be specifically highlighted here.
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y. Large language models are zero-shot reasoners, 2022.
References
Koncel-Kedziorski R., Hajishirzi, H.,, Sabharwal, A., Et- zioni, O., and Ang, D. Parsing algebraic word prob- lems into equations. Transactions of the Association for Computational Linguistics, 3:585-597, 2015.
Amini, M.-R.,, Feofanov, V., Pauletto, L., Hadjadj, L., De- vijver, E., and Maximov, Y. Self-training: A survey. Neurocomputing, pp. 128904, 2024.
Bartlett, P., Freund, Y., Lee, W. S., and Schapire, R. E. Boosting the margin: A new explanation for the effective- ness of voting methods. The annals of statistics, 26(5): 1651-1686, 1998.
Lee, D.-H. et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural net- works. In ICML. pp. 896, 2013.
Ling, W. Yogatama, D., Dyer, C., and Blunsom, P. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In ACL, pp. 158-167, 2017.
Chen, B., Jiang. J., Wang, X., Wan, P., Wang, J., and Long, Debiased self-training for semi-supervised learning. In NeurlPS, volume 35, pp. 32424-32437, 2022.
Ling, Z., Fang, Y., Li, X., Huang, Z., Lee, M., Memisevic, R., and Su, H. Deductive verification of chain-of-thougeee reasoning. In NeurlPS, volume 36, 2024.
Chia, Y. K., Chen, G., Tuan, L. A., Poria, S., and Bing, L. Contrastive sfain-of-tthugeee prompting. arXiv preprint arXiv:231 (tti::331..02277) 2023.
Liu, J., Lin, J., and Liu, Y. How much can rag help the reasoning Ilm? arXiv preprint arXiv:2410.02338, 2024.
Chu, Z., Chen, J., Chen, Q. Yu, W., He, T., Wang, H., Peng, W., Liu, M., Qin, B., and Liu, T. Navigate through enig- matic labyrinth survey of chain of thought reasoning: Advances, frontiers and future. In ACL. pp. 1173-1203, 2024.
Miyato, T., Maeda, S.-i., Koyama, M., and Ishii, S. Virtual adversarial training: regularization method for super- vised and semi-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 11(9))1979 1993, 2018.
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint
2021.
Mukherjee, S. and Awadallah, A. Uncertainty- self- training for few-shot text classification. In NeurIPS, vol- ume 33, pp. 21199-21212, 2020.
Farquhar, S., Kossen, J., Kuhn, I.., and Gal, Y. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625-630, 2024.
Nayab, S., Rossolini, G., Buttazzo, G., Manes, N., and Gia- comelli, F. Concise thoughts: Impact output length on Ilm reasoning and cost. arXiv preprint arXiv:2407.19825, 2024.
Frei, S., Zou, D., Chen, 7., and Gu, Q. Self-training converts weak learners to strong learners in mixture models. In AISTATS, pp. 8003-8021. 2022.
Patel, A., Bhattamishra, S., and Goyal, N. Are nlp models really able to solve simple math word problems? arXiv preprint arXiv:2 103.07/99 2021.
Geva, M., Khashabi, D., Segal, E., Khot, T., Roth, D., and Berant, Did aristotle use a laptop? question answering
10

---

Rethinking Chain-of-Thought from the Perspective of Self- Training
Roy, S. and Roth, D. Solving general arithmetic word problems. arXiv preprint arXiv: /608001403 2016.
Scudder, H. Adaptive communication receivers. IEEE Transactions Information Theory, 1 11(2):167-174, 1965.
Sun, J., Zheng, C., Xie, Liu, Z., Chu, R., Qiu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. A survey of reasoning with foundation models. arXiv preprint arXiv:2312. 1566, 2023.
Talmor, A., Herzig, Lourie, N., and Berant, J. Common- senseqa: A question answering challenge targeting com- monsense knowledge. arXiv preprint arXiv: 00111111100077 2018.
Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. K.-W,, and Lim, E.-P. Plan-an -nd--ad--olve prompting: Improving zero-shot chain-of-thougie reasoning by large language models, arXiv preprint arXiv:2305. 04091, 2023.
Wang, X., Wei, J., Schuurmans, D., Le, Q. Chi, E., Narang, S., Chowdhery, A., and Zhou, D. Self-consistency im- proves chain of thought reasoning in language models. arXiv preprint arXiv:2203. 1171, 2022.
Wei, J., Wang, X., Schuurmans, D.. Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., al. Chain-of- -f-thogghhht prompting elicits reasoning in large language models. In NeurlPS, 2022.
volume 35, pp. 24824-2493
Xu, X., Tao, C., Shen, T., Xu, C., Xu, H., Long, G., Lou, J.-G. and Ma. S. Re-reading improves reasoming in large language models. In EMNLP, pp. 15549-15575, 2024.
Yang. X., Song, Z., King, I., and Xu, survey on deep semi-supervised learning. IEEE Transactions on Knowl- edge and Data Engineering, 35(9):8934 8954, 2022.
Zhang, B., Wang, Hou, W. Wu, H., Wang, J., Oku- mura, M., and Shinozaki, T. Flexmatch: Boosting semi- supervised learning with curriculum pseudo labeling. Ad- vances in Neural Information Processing Systems, 34: 18408-18419, 2021.
Zhong, T., Liu, Z., Pan, Y., Zhang, Y., Zhou, Y., Liang, S., Wu, Z., Lyu, Y., Shu, P., Yu, X., et al. Evaluation of openai ol: Opportunities and challenges of agi. arXiv preprint arxiv22409. 18486, 2024.
Zou, Y.. Yu, Z., Kumar, B., and Wang. J. Unsupervised domain adaptation for semantic segmentation via class- balanced self-training. In Proceedings of the European conference on computer vision (ECCV), pp. 289-305, 2018.
Zou, Y., Yu, Z., Liu, X., Kumar, B., and Wang. J. Confidence regularized self- self-training. In CVPR, pp. 5982-5991, 2019.

---

Rethinking Chain-of-Thought from the Perspective of Self- Trainingg
A. Appendix
A.I. Omitted Definitions and Proofs in Section 3.1
In this section, we supplement the definitions and detailed algorithm of self-training that were not claborated in Scction 3.1 and provide the proofs of Lemma 3.1 and Theorem 3.2. Our results primarily reference (Frei al., 2022), and most definitions and algorithms will adopt the settings from their work. Here, we consider very simple mixture Gaussian model (GMM) rather than adopting the more general but relatively more complex sub-exponential distribution with parameters K, U, U", R as defined in (Frei et al., 2022).
Definition A.1 (Gaussian Mixture Model). A joint distribution (z. y) D over I {=1} is called Gaussian mixture model, if y Uni[((+1)), and sly Nyy,, T), where pa IRd is the mean of 'D. We D, to denote the marginal distribution of D.
The information entropy of a discrete random variable X is defined as Ent[X] = BI- log X] = --Exx p(z) log p(r), where X' is the set of values that the random variable X can take. A classifier for the Gaussian mixture model given by - sgn(3" 3t), where B Rd is an arbitrary vector. So we use 3 to denote classifier without saying x sgn(B" According to Fact 3.4 in (Frei et al., 2022), the Bayes-optimal classifier of the Guassian mixture model defined in Definition A.I is μ.. Assume we can access a initial classifier pseudo-labeler Smit. which is also called a pseudo-labeler, and the population error of Sinit is sufficiently small but constant. We then use a weight-normalizze logistic regression method to train start from Sinit using only unlabeled samples. The loss function is {2) = log(1 + exp((=)) ()). Let o > O be temperature. The training dataset S is partitioned into T batches of size B. The general process of the self-training algorithm involves multiple iterations, where in each iteration, a batch of data is assigned pseudo-labels. The data labeled in this iteration, along with the existing labeled data and data pseudo-labeles in previous iterations, is used to update the model. This process continues until all samples have been assigned pseudo-labels, The detailed and formal algorithm description of self training using the pseudo-label strategy is presented in Algorithm 1.
Algorithm 1 Self-Training
Require: Training dataset 5 = (0 ogesT-1
15i5B
step size 17, temperature a > 0, initial pseudo-labeles Biiit
1: fo + Pinit/ [|
2: for 0, 1, T 1 do
3: Generate pseudo- labels (1)) sgn (oF
(1) for batch {a/TsisB
4: Be+1 Be
Ei
B ve
1 0/¹)
5: Be+1 B |c+1
6: end for
7: return 31-1
A. 1.1. PROOF OF LEMMA 3.1
We first restate Lemma 3.1.
Lemma A.2 (Lemma 3.1, restate). Suppose (I,y) ~ DD follows a mixture Gaussian models in Rd x {=1} with mean y satisfying || = @(1), i.e., y ~ Unif({+1}) and zly ~ N(w,, 1). Let f(z) = log(1 + exp(-2)), and assume 2 max(1 1, [|x]]. Assume we can access a initial pseudo-labeler Biit which satisfies Pi(z.o)~Dly & ((d-Le), suppose Or is the angle between 3e and then by running algorithm with step size 7 and batch size B, when < 1, 0g 2 0ct1 holds with probability at least
= O())
Let e, 8 E (0, 1), and assume that B = )1) T = (de
1 5, and with probability at least 1 - 5, 0 S O(s).
Proof. To prove this lemma, we first present the results on sample complexity for labeled and unlabeled data obtained in (Frei et al., 2022) for selfntraining Theorem 4.1 in (Frei et al., 2022) ensures that we can obtain a classifier with sufficiently small but constant error. More precisely, a standard logistic regression procedure produces a pseudolabler that can achieve the desired constant accuracy by using (dd) labeled samples, which essentially represents the sample complexity of labeled data. As for the unlabeled data, the sample complexity expressed in the lemma below.
12

---

1| - 2= = 2(1 cos 0) = 4snn2 0₂ 2 Rethinking Chain-of-Thought from the Perspective of Self- Trainingg
Lemma A.3 (The Sample Complexity for Unlabeled Data, Theorem 3.6 in (Frei al., 2022)). Suppose that (a, y) D follows mixture distribution with mean I satisfying || @(1) and parameters K, K,U,U', R =(1). Let & be well- max(R, [[a])). Assume access to a pseudo-labeler Smit which satisfies PT(a.g)~DD * sgn (Smit T x))) S Car. where Cerr = R /F/C(UUUU) Let Let E (0.1), and assume that Then with probability least 8, by running Algorithm with step size and batch size B, the last iterate satisfies err (3r-1) & err(j) + e. In particular, T = (d//s) iterations using at most TB = =(d/=²) unlabeled samples suffices to within error of the Bayes-optimal classifier.
behaved for some Ce 2 1, and assume the temperature satisfies
B ()= ,T 52 (de -
Some definitions mentioned in Lemma A.3 can be found in the original paper and are omitted here due space constraints. We use the function (=) log(1 + exp(-z) as our loss function and it's well behaved. For GMM, it is mixture distribution with parameters K, U, U', R = (1). Under the conditions of Lemma A.3, let īa /1/ ||œ|| We can derive the following lemma regarding || - 74l, which represents the distance between the classifier produced at each iteration the algorithm and the normalized optimal classifier.
Lemma A.4 (Recursion of A Lemma D.2 in (Frei et al., 2022)). 1 SttsT
Suppose A
11,ee Tal then A satisfies that for
A s (1 (1-1/22,) +
nec
+
2Ca
8C
r²
where Cg. Cd. are all some positive constants such that K = CoC3o² 1, 4o 2
Note that there is close relationship between 8g and 4, we can use the changes in A, described Lemma A.4 to characterize the changes O,, leading to the following lemma.
Lemma A.5. Let Os denote the angle between 8, and μ. 0₁ 702) and A =113, T.l.. Then for <t< 0, 2 O,41, and Or S c.
Proof. Since B, and both have unit norm, it's easy to verify that
It's sufficient to assume that 0, (0, x x/2) for any t, the error rate of 8iit is sufficiently small. Therefore A, sin 2 i.e., 9, arcsin A. This implies that 0, and A, share the same monotonicity, so suffices to show that for < T, =Cg A, < A,-1- By Lemma A.4, when 7) I&K and T 2 32 log it's easy to verify that Ar S E, which means that A, > E fort < T. Hence, with the recursion of A? in Lemma A.4, we have
7E
2CaT²
A -- 1-1
<
A
2C,
8C,
9
o2
1
1
A
+
8)
2C,
4
40
(A,-1 8)
2C,
0
At the same time, 07 arcsin Ar 0(Ar) O(:). This concludes the proof.
Finally, through the conditions and assumptions of Lemma A.3, along with the conclusion of Lemma A.5, we complete the proof of Lemma 1.
13

---

Rethinking Chain-of-Thought from the Perspective of Self- Training
A.1.2 PROOF OF THEOREM 3.2
Theorem A.6 (Theorem 3.2, restate). Under the assumptions of Lemma 3.1, let d = 2 and suppose g (er x)) is the pseudo-label of z, where d(2) I+e-z. Define I(a) as the line x²a 1 O, where a is perpendicular to Let A(a1, (22) denote the region swept by ((11) rotating to ((22) along the trajectory of fmit towards μ during self-training. Denote Hz = Ent(i((1p) be the entropy of g [z. For t < T, with probability at least 1 - 5, the entropy changes as follows: (i) Ha)x) first decreases and then increases if E A(3o, p); (ii) He(x) decreases if x E A(u,8)) (iii) H,(x) first increases and then decreases if x E A(33 ,²) ); and (iv) Hz) increases if x E AAlu (8))
Proof. Lemma 3.1 describes the trend in the changes of the vector corresponding to the classifier, which is particularly useful for analyzing entropy changes in TR22, as the entropy of the pseudo-labes distribution is directly related to the angle between the classifier and the samples. Since we define the pseudo-label distribution as Bernoulli distribution using the sigmoid function, we first present the following lemma the entropy of Bernoulli distribution.
Lemma A.7. Let X ~ Ber(p). P E [0, 1]. Then Ent [X] is decreasing function of lp 1/2,, ie., if X1 ~ Ber(pi). X2 ~ Ber(p2), then Eut[X 2 Ent if and only if (Pi - 1/2| S |p2 1/2/.
Proof. For X ~ Ber(p), Ent[x] = f(p) = -p pogp- - (1 - p) log(1 - p)- We define Olog 0 = 0. It is easy to verify that f(p) is symmetric about the line x = 1/2. On (0,1/2), f()) is monotonically increasing, and on (1/2, 1). [()) is monotonically decreasing. Thus, for X1 and X2, max(p1, -p1) E (1/2,1) and max(p2, 1 - p2) E (1/2, 1), by the monotonicity and symmetry of f()), it is straightforward to conclude that Ent(X;) 2 (X2 if and only if max(p1, 1 11ppp p1) S max(p2, 1 p2). Moreover, max(p1, 1 pı) S max(p2, - -p2) if and only if [p1 1/2] I Ip2 1/2 which completes the proof.
Now, we analyze the monotonic properties of Hz) From Lemma A.7, we know that for X ~ Ber(p), Ent depends on the magnitude of Ip- 1/2]. In the subsequent proof, we establish the conclusion that during the iteration process, if ((3T Φ) < 1/2, it will not happen that ((T ) > 1/2 unless (.85 x) is very close to 1/2, causing ((8[ > 1/2 in the next iteration. This essentially states that in the self-training process, pseudo-labels do not fluctuate across iterations. Therefore, we only need to consider the relationship between (65 2) and (8T,12).
Since {()) T+e is monotonically increasing, it suffices to compare (8 x and 8 Jr. Let the angle between 3, and be (x) (0, r). Then, ff'x = ||z||| cos ((0) (x) = sgn([>>] cos (())())) ||=||| cos ((t) For the same sample, only need to consider the changes in g(*) (x)) as t varies and the monotonicity of the function f(z) = | cos(z)) on (0, r)
(f(=) is decreasing on E (0, 7/2) and increasing on E (#/2, Define the angle between æ and as g(o))(r).
We now discuss the four regions defined in the theorem respectively.
1. x E A(Bo. ui). When ((0) E 2/2) and ((0) (z) S 6. x can be considered to lie between vector Bu and vector μ. If 0, 2 (x),, then (x) = O. - (0n - (x)) E 7/); if 0, I (x), then (x) = (Op (x)) - 0, E
(0, x 7/2). Therefore, (2x( first increases and then decreases, remaining within the interval (0, . /2), which implics that (x) first decreases and then increases. The proof for the other case is similar.
2. x E A(,, When (x) E (0, 2/2) and ((0) (a:) > Oo. x can be considered to lie between vector μ and S with the angle between & and p lying in (0, x/2). In this case, (z) = 0, + (x) E (0, x/2), so (z) decreases monotonically and remains in (0, 7/2), implying that H2 decreases monotonically. The other case is similar.
3. x E A(B ). When (z) E (z/2, T) and (xx) E (0, x/2) can be considered to lie between vector Bd and ,ut, where the angle between S61 and μt as well as the angle between 1r' and 8/ both lie in (0, x/2). In this case, (())(x)) = 0, + (x)) decreases monotonically, but there exists some I' such that & becomes orthogonal to x, leading to (x) first increasing and then decreasing. The proof for the other case is similar.
4. x E A(w,, 9g). When (10) (x)
(x)) E (2/ x) and 0100) E (T/2, x), x can be considered to lie between 1rr and -9g. In this case, g(e) (c)) = 0. + (l0=) (xr) E (T/ (m/2,, x) decreases monotonically, implying that Hx(x)) increases monotonically. The proof for the other case similar.
Through the above analysis, we complete the proof of Theorem 3.2.
14

---

Rethinking Chain-of-Thought from the Perspective of Self- Training
A.2. Rigorous Formalization of Reasoning Concepts Introduced in Section 3.2
The definitions that were not rigorously stated in Section 3.2 are presented here in detail.
Definition A.8 (LLM Generation Model). Let Σ be an alphabet, and >' be the set of all strings over >. For input >", and temperature coefficient E (0, 1), LLM generation model is defined as function LLM: E" x (0, 1) + D(E"). where '(E*) is the set of all probability distributions over E". We denote s' ~ LLM (8; as an output 8' E E* sampled from this LLM given the input and temperature 7.
Definition A.9 RReasoning-Answe Answer Pairs and Sets). Given question g, prompt p, and a temperature coefficient T; The set of all (Reasoning, Answer) pairs generated by the LLM under these conditions is denoted as
[g. p: {(r (r, LLM(4.P:7)) p;
Its probability density function is denoted concisely as J. The set of reasoning paths generated by the LLM under these conditions is R(q, p: 7) = fr (r ~ LLM p: 7)}, with the probability density being the marginal distribution Jy of J. The set of answers generated by the LLM under these conditions is Answer (4, p) {a (r,a) a) ~ LLM(
with the probability density being the marginal distribution Ja of J. Furthermore, we define
Answer(g, P, r; {a' (r',a') LLM().).r;+))
as the set of all answers under given reasoning path r, with its probability density being the conditional distribution Jair- The answer set A can be partitioned into several semantic clusters: A Ucec C, where is the set of these clusters, and exactly one cluster signifies the correct answer.
Definition A.10 (Initial Reasoning Path). Given question q. zero-shot prompt pu, and temperature coefficient the initial reasoning path ro is defined such that (ro, 0o) ~ LLM (q, Po;
Definition A.11 (Reasoning Tree Constructed by LLM, inspired by the definition in (Liu et al., 2024)). Given question g. a prompt p, and a temperature coefficient т, the reasoning tree T = (V, constructed by the LLM under these conditions a directed acyelic graph, defined as follows:
Each v V represents a computational node. Computational nodes generate content, including text, symbols, numerical values, or logical expressions. The computation at each node is defined any combination the following:
1. Symbols and text (e.g., variable definitions, natural language descriptions).
Computational expressions (e.g,, arithmetic, algebraic operations).
3. Logical assertions (e.g., implication relations, hypothetical reasoning).
4. External knowledge references (e.g., theorems, formulas, common sense).
Each (u, v) E represents a dependency relationship in reasoning, i.e., the computation of u depends on the content generated by u.
The root node corresponds to
T has multiple leaf nodes, some of which correspond to the correct answer Gcorrect
Each node V has state function - {0, 1), indicating whether the node is erroneous. Errors include, but are not limited to:
Symbolic errors.
2. Computational errors.
Logical errors (e.g., using external knowledge that does not meet requirements or is false, using incorrect logical proof methods).
Definition A.12 (Reasoning Path). The reasoning path (or reasoning process) generated by LLM for a question g. prompt p. and temperature coefficient 7 is defined as path (g - - -> a) on the reasoning tree T constructed by the LLM under these conditions.
Definition A.13 (Optimal Reasoning Path). The set of optimal reasoning paths Ropt generated by an LLM for a question q. prompt p. and temperature coefficient т is defined as Ropt = fr E R(q.p:T) | (Acorreet (r) 2 8). where e is a confidence threshold, close 1.
15

---

Rethinking Chain-of-Thoughe from the Perspective of Self- Training
# Instruction:
Semantic Entropy
Pi (ō): Let's think step by step, how calculate specific quantities based on given scenarios and conditions each situation.
The prompt "Let's think step by step" helps guide large language models to reason more deeply, but it may be less effective in specialized domains. Below are examples from a specific dataset. B Based on these, please modify the "Let's think step by step, 1%s" prompt adding sentence in the 1%s section to better suit the task's characteristics.
Question set Q"
e 0.05
Pz: Let's think step by step, how to calculate the total quantity of objects or items mentioned in the sentences.
e2: 0.13
Q Question set Q':
LLMs
LLMs
q1: car being driven, a straight line and at uniform speed, towards the base of...
Pm Let's think step by step, how to break down the mathematical operations and identify the key concepts to solve them.
e 0.44
d A train running at a speed of 100 miles/hour, takes 10 hours to reach its destination. After..
Figure 6. The proposed Task-Specific Prompt module tailors the prompt the characteristics of a given task. Initially, set of candidate prompts (pr, (p1,P,.." -. Pm.) generated by incorporating tailored instruction and a question set, Q' sampled from the dataset. Next, another disjoint question set Q" is sampled, distinct from Q' The candidate prompts {pI. P2,.,.... Po are then evaluated based on their semantic entropy values {ei, 62,""" ,mm, and the prompt á with the lowest entropy selected as the optimal one for the task.
Due to the LLM's knowledge gaps and generation limitations, "optimal" needs to be defined as the most reliable path within the model's capabilities.
Definition A.14 (Iterative CoT Reasoning). Let ro be defined as in Definition A.10. Then, in an iterative CoT process, (1++1,0++1) ~ LLM(g,p "o....., (15T,,, where p' may be an adaptive prompt.
Definition A.15 (Update Reasoning Path in Iterative CoT). For the current reasoning path Tt. a new path rt+1 generated follows: the LLM identifies the node 2 in r, closest to the root that satisfies s(v;) (i.e., is erroneous), reverts node u. corrects the error, and generates new sub-path, thereby forming Tr+1-
The conceptual parallels between self-training and chain-of- thouthtttth are summarized in Table 3.
Self-Training
Chain-of-Though
Distribution DD over feature-labe label space X' x y
LLM's generation distribution J of (Reasoning, An- swer) pairs for given (g.
Classifier 3 mapping an input æ E X' to a label E y
LLM generating an answer E Answer(g. (g.serr con-
ditioned on a reasoning path r
Initial classifier 3p
Initial reasoning path rn generated from (g.Po.T)
Bayes-optimal classifier fa
Set of optimal reasoning paths Ropt that yield Ocorrect with high probability
Iterative classifier update: 3e > B141
Iterative reasoning path update: Tt - Tt+1 through error correction and refinement
Information augmentation for updates (e.g., pseudo- labels from unlabeled data)
Information augmentation for updates (e.g., deeper anal- ysis of q. problem decomposition, prompting strategies
to elicit new reasoning steps from LLM's internal knowl- edge)
Table 3. Analogy between self-training and chain-of-thought reasoning.
A.3. Task-Specific Prompt and Experimental Dataset Details
The specific details the task-specific prompt module elaborated Figure 6, and the particulars of the experimental datasets described in Table 4.
16

---

Rethinking Chain-of-Thought from the Perspective of Self- Trainingg
Dataset
Answer Format (*1)
# of samples
Avg # words (*2)
Data split (filename)
License
SingleEq
508
27.4
questions.json
No License
AddSub
395
31.5
AddSub.json
Unspecified
MultiArith
600
31.8
MultiArith,
Unspecified
GSM8K
N
1319
46.9
test.
MIT License
AQUA
M
254
51.9
test.jsonl
Apache-2.0
SVAMP
1000
31.8
SVAMP.json
MIT License
CommonsenseQA
1221
27.8
devdrand.co
pew.randd.splt.json split.jsonl
Unspecified
StrategyQA
2290
9.6
task.json
Apache-2.0
LastLetters
500
15.0
CoinFlip
Y
500
37.0
Table Detailed description the datasets used our experiments, highlighting their diversity and structure. (1) The "Answer Format" column indicates the type of responses expected for each dataset: N represents numerical answer, M corresponds to selecting option from multiple choices, Y indicates a binary answer (Yes or No), stands for free-form answers. (2) The "Avg words" column represents the average number of words in the question texts, providing estimate of their complexity.
A.4. Parameter Sensitivity Analysis
Our proposed method involves two important reasoning paths N. We investigate the sensitivity our method to these hyper-parameters on the AQuA dataset and report the results in Figure and Figure 7. First, we fixed N 3 and varied 7 across the range of 2,,2, 5). The results indicate that our method is highly sensitive the value of T. Accuracy increased sharply from 60.2% the first iteration 70.8% at the second iteration, after which the improvement plateaued. By the fourth iteration, accuracy reached 71.7%, with negligible changes in subsequent iterations. This trend suggests that with three reasoning paths, four iterations are sufficient to explore nearly all plausible solutions. Additional iterations yielded diminishing returns, likely constrained by the inherent reasoning capabilities of the employed LLMs. Moreover, the linear growth in computation time with increasing highlights the practical need to limit iterations for efficiency.
i.e., the number iterations' T, and the number of sampled
Next, T and varied N from (1,2..... , The method exhibited significant sensitivity N, with accuracy rising from 55.5% when using a single path 70.9% with paths. This underscores the method's capacity to aggregate diverse reasoning paths effectively, Additionally, increasing N enhanced the accuracy of semantic entropy calculations, allowing more precise identification of samples prone to over-reasoning. However, this improvement came at the cost of steeply rising computation times, indicating a trade-off between accuracy and efficiency as increases.
A.5. Additional Experimental Results
Prompt Type
Iterations 1 &
Iterations 2 & 3
Avg.
General Prompt
0.44
0.32
0.38
Our Prompt p
0.28
0.29
0.29
A
-0.16
-0.03
-0.10
4%
36.4%
-9.4%
-26.3%
Table 5. The reasoning similarity between new and previous iterations guided by general prompt and our p² on the AQuA dataset.
17

---

Rethinking Chain-of-Thougee from the Perspective of Self- Training
80
74.8
75.2
9m
75
4h 30m
72.4
70.9
71.3
70.1
Accuracy (%6
70-
Time (hours)
3h 1m
2h 30m
65
26m
60
Ih 30m
55
26m
2
3
7
Sampled Reasoning Paths
Sampled Reasoning Paths
(a) Accuracy
(b) Time costs
Figure 7. Impact of the number of sampled reasoning paths on accuracy and time costs in our proposed method.
0.7
Regiors 1. Cagréiate: (22-6.1
Quastion Exumple for Patter
Begien Controte (12
Quation Eumpl: Patien
0.6
CoCoiiatte [55-4.]
Comttrate [43-11]
1.4
Quattion Exumple for Pattena
Average Entoopy Samplex
Ameraga eemapyof capple
1.2
0.5
Sentanbic Entropy
1.0
Estroyy
0.8
0.3
0.6
0.2
0.4
0.2
0.0
0.0
0
50
100
150
300
350
400
Number Iteration
Number of Iteration
(a) Information entropy variation in seff-rrainin
(b) Semantic entropy variation in CoT reasoning
Figure 8. Subfigure (a): entropy variation over iterations of self-training for four test samples in different regions. Sample 1 is in region A p). and its entropy first decreases and then increases. Sample 2 in region A(u, 30 ), and its entropy decreases stably. Sample 3 in region A(35 ), and its entropy first increases then decreases steadily. Sample is in region A(u- (B0), and its entropy increases stably. During the iteration, the average entropy decreases until stabilizes. Subfigure (b): semantic entropy variation during iterations of CoT reasoning for questions across three patterns. In pattern the semantic entropy of the question decreases gradually. pattern 2. the semantic entropy remains constant. In patter 3, the semantic entropy initially decreases before increasing.
18

---

Rethinking Chain-of- -Thoughh from the Perspective of Self- Training
Question
Iteration 1
lteration 2
Output
186,624
Suppose you can travel from a place M to place N by 3. buses, from place N. place by buses, from place O place P by buses and from place P to place by 3 buses. In how many ways can you travel
# Path I: We multiply the number of ways for each seg- ment: from M to N (3 ways), N 0 (4 ways), O to way), and to Q (3 ways). This gives us: 3 x*4 4
hope that explanation was helpful to you. If questions need further assistance, foel free to ask! The answer is 186,624. Path 2: That's correct The
# Path I:
(Answer: 36)
you have any
N.4 options from N O, I optioon from O to P. and 3 options from P Q. The answer is 3 4
ways. Path We have options from
number ways to travel from M Q is 3 options for M to N mulliplied by 4 options for N to O multiptied by I option for O multiplied by 3 options for to which equils 36 ways. # Path 3: To travel from M to Q. there are 27 ways from M to N, 256 ways from N O, I way from O to P. and ways from Pro Q. Therefore, the answer is 186,624 ways.
= 36 ways. # Path 3: We calculate the number of ways for each segment: ways from M to N, 256 ways from N to O, way from O to P. and ways from P Multiplying these together gives 27 256 1 27 = 186,624 ways.
from M Q?
A storts travel towards south 3km, then travel km towards east, and again travels 3 km to north, and finally travels 2km towards west. In the end haw for from is A from home?
# Path I: A starts at home, travels km south, 5 km cast,
# Path 1: A travels south 3 km, cast 5 km, north km (canceling the southward distance). and west km. Using the Pythugurean theorem, the linal distance from home is 5.39km, The closest answer is 5 km. Path 2: A travels southward distance), and 2 km west. This leaves A km east
3 km north, and km west. After these movements, A km. # Path 2: A travels km south, 5 km cast, 3 km north, and then west.
km
is about 5.39 km from home. rounding
(Answor: 3 km)
movements north and south cancel each other out, as do the movements east and west, leaving A 2 east of the starting point, # Path
south, 5 east,
north (canceling
travels km south, 5 km cast, 3 km north, and 2 km west. The north and south distances cancel out, leaving A 3 km east, and the east west difference results linal position km cast of the starting point.
starting point. Path 3: Great job analyzing A's journey step by step! By considering how opposite
movements cancel out, we can accurately determine the final position. Therefore, the answer km.
Table 6. Over-reasoning (row 2) and high similarity between consecutive reasoning iterations (row 3). In Iteration 2, Paths 1, 2, & 3 updated based on their paths in Iteration 1. Over-reasoning occurs when the LLMS can already provide correct answers to the question in Iteration but continue reasoning unnecessarily, introducing noise that leads to incorrect answers in Iteration 2. High similarity refers to cases where the corresponding paths in the two iterations are overly similar, failing to correct errors from the previous iteration.
Question
lteration
lteratoon 2
Oulput
36 (Answer: 36)
Suppose you con travel from a place to a place N by buses. from place N to place O by buses, from place O place P by 1 buses and from place to place by 3 buses. In how mony ways can you travel from M to Q?
# Path I: We multiply the number of ways for each seg-
This iteration is skipped becouse the somontic entropy of the previous ieraatio is below the predefined threshold.
ment: from M to N (3 ways), N 0 (4 ways). O (1 way). and to Q (3 ways). This gives
x 4 3 = 36 ways. Path 2: We have 3 options from M to N. options from N O, optinn from to P. and options from P Q The answer is 3 44 I = ways. # Path
3: We calculate the number ways for each segment: ways from M to N, 256 ways from N to O, way from O to P. and 27 ways from P to Q. Multiplying these together gives + 256 1 + 27 186.624 ways.
starts travel towards south 3lmm then travel 5 towards east, and again travels 3 km to north, and finally travels 2km towards In the end how far from is A from home?
# Path 1: starts home. travels km south, km cast. 3 km north, and 2 km west. After these movements, A Path
# Path 1: We reconsider A's movements vectors: south, 5 km east, 3 km north, and km west. The north and south movements cancel each other out, leaving a displacement of 3 km east, A Path 2: A's total displace- meni from home can be calculated using the Pytlugorean theorem by summarizing the joumey. The net displace- ment is (5 - 2) km cast and (3 - 3) km north, resulting (3, 0). Path 3: Excellent woek breaking down A's journey step by step! By accouming for the cancellation of opposite movements, we can precisely calculate final position. As result, the answer is 4 km.
home, runding 2: A travels 3 km south, 5 km cast, km north, and then 2 km west. The movements north and south cancel each other out, as do the movements east and west, leaving A 2 km cast of the starting point, # Path 3: A travels 3 km south, 5 cast.
is about 5.39 km
(Answer: km)
north, and km west. The north and south distances cancel out, leaving A 3 km east, and west difference results final position km cast of the starting point.
Table 7. Examples of how our method addresses the issues of over-reasoning and high similarity between consecutive reasoning iterations.
19

---

Rethinking Chain-of-Though from the Perspective of Self- -rainiagn
Question
Iteration
Iteration 2
Output
(E) 32.51%
John likes to have lightly Bavored tea every evening. In 50% strong milk tea, replaces 15% of it with milk twice. Then, he re- places 10 percent of the m-- sulsont sobution with more milk. What is the final concentration of tea John
# Path I: First, lot's start with the 50% strong milk tea,
This tteration is skipped because the semontic entropy of the previous iteration is below the presiefined threshold.
(Answer: (E))
John replaces 15% of with milk twice. After the first replacement. the tea concentration becomes 50% + (1
concentration hecomes 50% 15%= = 7.5%. After the second replacement, the tea con- centration becomes 42.5% * (1 - 1.5%) 36. 125%, and the milk concentration becomes 7.5% 42.5% 15% 14%. Next, John replaces 10% of this solution with more
15%) 42.5%, and
milk The concentration becomes 36. 125% (1 10%)
drinks? Answer Choices: (A) 15.38% (B) 42% (C)
= 32.5122%, and the milk concentration hecomes 14% + 125% 10% 17.6125% Therefore, the final con- centration of tea John drinks is approximately 32.51%. # Path 2: John starts with 50% strong milk tea and replaces
39, 86% (D) 22.35% (E) 32.51%
15% of with milk twice. After first replacement. 7.5% of tea is replaced, resulttnng in 42.5% tea. After the second replacement, 6.375% of replaced, resulting in 36. 125% tea. Then, John replaces 10% of this solution with more milk, which removes 3.6125% tea, leaving 32.5125% tea. Therefore, the final concentration of tea is 32.51%. 9.. Path 3: John starts with 50% strong milk and replaces 15% with milk twice. After the first replacement, 85% of the original tea remuins, resulting in 42.5% tea. Afeer the second replacement, 85% of the new mixture remains, resulting in 36 125% tea. The final con- centration of tea 36.125% which is closest to answer choice (C) 39.8
(A) 15 (Answer: (A)
Alex and Jacoh works at a toy shop that make toys. Alex takes 7 hours to make a toy, and Jacob takes hours to make toy. Dur: ing month, both of them makes 35 toys total. If both of them have worked almost similar number of hours how many toys have been prepared by Ju- cob? Choices: (A) 15 (B)
# Path I: Alex takes hours to make a toy, while Jacob takes hours make toy, Together, they made total of 35 toys, and both worked for almost the same number of bours. determine how many loys Jacub made, can calculate their individuul production rates: Alex makes a toy per hour, and Jacob makes of toy per hour. Let a represent the oumber of toys Jacob made. Since Alex made 35- x toys, the total hours, Alex worked would 7(35 - 2), and the total hours Jacob worked would be 9x. Given that their working hours are nearly equal, we
# Path I: I Lagree with your approach. Let's solve the equation step by step to find the number toys made by Jacob. Let x represent the number of toys made by Jacoh. Since Ales and Jacob together made 35 toys, the number of toys made by Alex is 35 x. Given that Alex takes 7 hours to make toy and Jacob takes hours, the total number of bours spent both making toys should he equal, leading to the equation 7 (35 x) 9 x. the left side we get 245 7x 9x, Rearrnging the equation gives 245- = 16x Dividing both sides by 16. 15.3125. Since represents
set the equation 7(35 x) = yu.. Solving this, we
we find number of toys made by Jacob and it cannot he fractional, we round down to the nearest whole number, so Jacob made 15 toys. Therefore, the Path 2: Let's consider the total amount of work done by Alex and Jaceb terms of the number of toys they each made. If Alex takes 7 hours to make a toy, then in x bours, would make to make a toy, then in x hours, he would make x/9 toys. Given that both them logether made 35 toys in total, we can up the equation: x/7 x/9 35. To salve for need to find common donominator, which in this case would
get 245- 7x 9ar, which simplifies to 245 16r. yielding x 15.3125 Since the number of toys must be a whole number, we round down to conclude that Jacob made *=15 tnys* Path To determine how many toys Jacob made, we first calculate the combined work rate of Alex Jaceb. Alex makes of a toy per hour. and Jacob makes $ toy per bour, giving combined rate of 4 + 3 16 toys per hour. Since both Alex and Jacob worked for approximately the same number of hours, let represent the hours each worked. The total hours worked by both is 2x. and the total number of toys made is 35. Using the combined work rate, we up the equation 10 x 2z = 35. Solving for ar, get 32x = 35, which simplifies 32x 2205 and a zz 68, 90625 hours. To find the number of toys Jacob made, multiply his work rate by the hours be worked: a x 68. 38.996655 RJ 7. 7.65625. Since the number of toys must be whole number, Jacob made approximately +og toys**, However, this answer does not match any of the provided choices, indicaling a discrepancy in the problem setup calculations. Path 3: Alex takes hours make a toy, so in one hour, he makes 1/7 of toy. while Jacob takes hours to make toy, making one hour. Lei A and represent the number of tuys Alex and Jacob made, respectively. Together, they made tays month, which gives the equation: A/7 + 1/9 = 35. Since they worked almost the same number of hours, the ratio of toys they made should be close to the ratio of their working speeds, which is 9.7, since Alex takes 7 hours to hours. Therefore, the number
16 17 (D) (8 19
answer is (A) 15. a
toys. Jacob takes hours
equation bacomes (9x + 7x)/ 63 35. + inng this gives 16x 63 = 35, 16x 2205. Solving which lends to 16x x gives x / 16 137.8 8 hours. Since we are looking for the number of toys made by Jacoh, we substitute x back into Jacob's 's rate, so the number of toys made by Jacob x/9 137.8/ 9 = 15.3. Therefore, Jacob made approximatcly 15 toys. and since we are looking for the nearest whole number. Jacob made 15 toys. # Path 3: Alex takes 7 hours to make a toy, so one hour, makes 1/7 of a toy. Jacob takes 9 hours to make toy, so in one hour, he makes 1/9 of toy. Together, they made 35 toys, so we have the equation: A/7 + 1/9 = 35. Since their working speods are in the ratio of 9.7, we assume Alex made 9x toys and Jacob made 7x toys, Solving fove x, we gel = 35/16 2.1575. Rounding to the nearest whole number, Alex made 18 toys and Jacub made 14 toys. Therefore, the answer is toys (D).
Simplitying
toy
muke a loy and Jacob of toys Alex made should he 9x, and the number of toys Jacob made should he 7x. Given that total number of toys is 35, we have 9x 7x = 35, which simplifies to 16x = 35. Solving for x gives x= 35/16 = 2. 1875. Since the number of toys must be whole, we round to the nearest whole numbers: Alex made 9 2 = 18 toys, and Jacob made + 2 loys. Therefore, the answer is 18 toys. Hence, the correct answer is (D) 18.
Table 8, Examples of how our method addresses the issues of over-reasoning and high similarity between consecutive reasoning iterations.
20

---

Rethinking Chain-of-Thought from the Perspective of Self- Training
Question
Iteration
Iteration 2
Output
The ratio of the volumes of a cube that of shere which will füt inside the cube Answer Choices: (A) 2: a (8) 7:2 (C) 8:2 (D) 6: x (E) 8:3
# Path 1: The volume of a aube is given by Vaes s", where the side length, while the volume of sphere is calculated as Vitare = $xr", with r being the radius.
This teration is skipped becouse the semontic entropy of the previous iteration is below the predefined threshold.
(D) 6 x (Answer: (D)
sphere fits perfectly inside cube, the sphere's v diameter equals the cuhe's ' side length, making radius r 2.. Substituting this into the sphere's s volume formula yields V Vybee
2Ta" " The ratio of the cube's volume the sphere's volume therefore Vade Vanue = 2 g 6 sT. Thus, the correct ratio
x,, comesponding option (D). Puth 2: First, consider the volume of cube and sphere. The volume of a cube Vatte = z², where the side longth. and the volume of sphere Kyoum I fxy", where is the radius. If the sphere fits perfectly insäde the cube, its diameter equals the side length of the cube, 2 which gives = 1. Substituting this into the sphere's
volume formula, we get Vester $x ( ) $ g Compuring the volumes, the ratio the cube's volume to the 's volume is Vate Vighere
4 2 G: a. Thus, the ratio of the volumes is 6 n. and the correct answer is (D). Path Let's begin by considering the cuhe. The volume of cube is Vante i 2 where the side length. Next, let's think about the largest sphere that can fit inside the cube. The diameter of this sphere equals the side length of the cube. The volume of a sphere is Vistere far where the radius. Since sphere's 's diameter the same as the cube's side length. the radius Substituting this into sphere's 's g x..
the sphere is
x ( The ratio the volumes of the cube and the sphere Vatee : Vegter = 6² 8 = 6: x. Thus, the correct answer is (D)
valume formula, we get Vyteer
(E) 0.9 (Answer: (E)
The events A and B are in- dependent. The probobility that event A occurs is 0.6,
# Path 1: Given that events A and B are independent, we can calculate P(A n B) P(A) x P(B). With P(A) 0.6 and P(A B) 0.96, use the for- mula for the union of independent events: P(A U B)
# Path 1: Given: P(A) = 0.6 P(A U B) = 0.96 We can
use fommula for union of events: P(A B) P(A)
P(B)-P(A B) Since events A and B are independent,
and the probubility that at least one of the events A
P(A n BB P(A) P(B). Substitute the values: 0,96 = 0.6
P(4)+ P(B)- P(A) P(B). Subuituting, we get
PIB) -.. P(B) Solving the equation: 0.36 0.4 P(B)
B occurs is 0. 96, What the probability occuss? Answer Choices:
0.6 P(B) - 0.6 x P(B) 0.96, which simplifies
P/B) = 0.36 /0.4 P(B) - 0.9 Therefore, the probahility that which matches answe chuice (E).
event
to + (B)
0.96, yielding P(B) 0.9.
event accurs
Therefore, the probahility that event B necurs is 0.9 (FF) a
# Path 2: 1. Given that events A and B are independent, we know that the probability of both A and B occurring is equal to the product of their individual probsbilitio: P(A and B) = P(A) P(B). 2. The probability that event A 0.6, P(A) = 0.6 3. We al given that the probability that at least one of the events A or B occurs 0.96. This can represented as: P(A B) P(A) P(B) PLA and B) 0.96. 4. Since events A and are independent. P(A and B)= = P(A) P(B) = 0.6 P(B). 5. Plugging this into the equation from step 3, get: 0.6 + P(B). 0.6 P(B) = 0.96. Solving this equation will give the prohability that event B occurs. Let's continue from here. Therefore, among A through E, the answer is caleulation:
(A) 0.5 (B) (C) 0.8 (E) 0.9
Path 2: Given that events A and B are independent. we
know that P(A and P(A) P(B). With PIA) = 0.6 and
PIA or B) = 0.96, the equation becomes: P(A)+ + P(B)-
P(A and B) = 0.96. Since P(A and B) 0.6 P(B), this simplifies to 0.6 + P(B)- 0.6 P(B) - 0.96. Solving this equation, we find
Therefore, the answer is (D) 0.8. # Path 3: Given that the probability of event A is P(A)= = 0.6 and the probability ovent B is P(B), and that A and B are independent. P(A B)= P(A)* P(B). The probability of at leust one of the events occurring is 0.96. expressed P(A
P(B)
0.96. Substinutnng the known values, we get 0.6 + P(B) 0.6 PCB) = 0.96, which sim- plifies to 0.4 0.4P(B) 0.96. Solving P(B) gives which is impossible since probabilities cannot excced I. Thus, the correct answer (C) 0.7.
Lat's continue = 0.96. 0.6 P(B) -..6(BB 0.96 0.4 + 0,4P(B) 0.96 0.4P(B) 0.56 P(B) = 0.56/ 0.4 P(B) 0.7 Therefore, the probubility that event B occurs 0.7. which corresponds to answer choice (C). # Path 3: We have: P(A) 0.6
P(R)
P(B)
independent) Given that P(A B) = 0.96, we can express this as: P(A
PLA B) P(A) * P(B) (since and
n 96 0.6 P(R) - (0.6 P(B) 0.96 = 0.6 P(B) 0.6P(B) 0.36 0.4P(B) P(B) -0.36/ / 0.4 P(B) = 0.9 Therefore, the correct probabilit that event B occurs 0.9, which corresponds to answer choice F. Therefore, the answer is (F) 0.9
R) P(A) + P(B)
Table 9. Examples of how our method addresses the issues of over-reasoning and high similarity between consecutive reasoning iterations.
21


Reviews:
[{"review_id": "VlpMs3wDqU", "reviewer": "Reviewer_Yf9H", "summary": "This paper discusses the similarities between Chain-of-Thought (CoT) reasoning and self-training, and proposes how to reduce prediction uncertainty for both iteratively leveraging on model-generated information. In particular, this paper introduces a novel CoT framework with two main ingredients: (i) a task-specific prompt module designed to optimize the initial reasoning process by searching for prompts that minimize uncertainty, and (ii) an adaptive reasoning iteration module that dynamically refines the reasoning process to address over-reasoning and similarity between consecutive iterations. Based on theoretical results from self-training, the authors link entropy variations and semantic entropy to CoT. Experimental results to demonstrate the effectiveness of their proposed framework in improving reasoning performance and computational efficiency across several datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the valuable feedback and encouraging comments. We are motivated by the suggestions and have addressed each concern in detail as follows:\n\n> **Q1.** The paper lacks sensitivity analysis for the sampling number $N$.\n\n**A1.** We evaluated the impact of $N$ on four datasets. Accuracy improves with larger $N$, especially on harder tasks, but saturates after $N=3$, suggesting small $N$ is sufficient for stable semantic entropy estimation.\n\n|Dataset|$N=1$|$N=2$|$N=3$|$N=4$|$N=5$|\n|-|-|-|-|-|-|\n|AQuA|53.9%|70.9%|70.1%|71.3%|72.4%|\n|AddSub|73.9%|85.1%|88.4%|88.9%|90.4%|\n|AMC2023|12.5%|20.0%|25.0%|27.5%|30.0%|\n|Gaokao|32.8%|41.3%|44.2%|44.7%|45.3%|\n\n> **Q2.** TSP incurs extra cost. Can the authors empirically show its advantage in both performance and efficiency over traditional prompt selection?\n\n**A2.** We compared prompt selection methods, including traditional ensemble prompting (generates multiple prompts and aggregates outputs) and iterative search (automatically generates and filters prompts using Monte Carlo search). These were compared with the TSP module based on semantic entropy. TSP achieves the highest accuracy across all datasets while significantly reducing total token consumption (prompt search $S_{\\text{Token}}$ + inference $T_{\\text{Token}}$). Compared to the search method, TSP balances performance and efficiency better.\n\n|Method|AQuA|$S_{\\text{Token}}$+$T_{\\text{Token}}$|AddSub|$S_{\\text{Token}}$+$T_{\\text{Token}}$|AMC2023|$S_{\\text{Token}}$+$T_{\\text{Token}}$|Gaokao|$S_{\\text{Token}}$+$T_{\\text{Token}}$|\n|-|-|-|-|-|-|-|-|-|\n|Ensemble|59.7%|1,703+5,957|80.4%|790+2,196|12.5%|2,089+7,749|33.6%|2,234+7,310|\n|Search|64.2%|1,932+53,581|84.4%|918+30,732|12.5%|2,672+67,642|37.6%|2,906+81,109|\n|TSP|66.1%|2,050+21,570|85.8%|1,005+10,281|17.5%|2,261+24,506|41.9%|2,554+23,716|\n\n> **Q3.** How can the entropy principle from self-training, which involves parameter updates, be applied to CoT that only modifies inputs?\n\n**A3.** We attempt to draw an analogy between the principle of self-training and CoT. Although they differ in operational mechanisms, with self-training relying on parameter updates and CoT guiding reasoning through input manipulation, we argue that both exhibit structural consistency in terms of information compression and entropy convergence. In CoT, while model parameters remain unchanged, the iterative optimization of the reasoning path set $R_t$ gradually leads to semantic-level distributional convergence over the reasoning space $\\mathcal{R}$. As iterations progress, the proportion of reasoning paths containing correct subpaths $R'$ increases, and these paths tend to converge toward the correct answer cluster. This process results in a gradual increase in the posterior probability of the correct answer, reflecting a reduction in semantic entropy that parallels the entropy minimization objective in self-training.\n\n> **Q4.** How does \"re-evaluate from another perspective\" reduce reasoning similarity and entropy?\n\n**A4.** The exploratory prompt encourages LLM to break away from the current local reasoning trajectory and actively explore subspaces within the generation space $\\mathcal{R} \\times \\mathcal{A}$ that are semantically orthogonal to the original path. Such orthogonality is reinforced through key semantic constraints embedded within the prompt (\"alternative perspectives\"), prompting substantial strategic changes in the reasoning process. This process can further converge the answer space, thereby compressing the entropy of the overall candidate answer set.\n\n> **Q5.** Include stronger CoT baselines, harder datasets, and overhead comparison.\n\n**A5.** We have newly incorporated two recent and stronger CoT methods, Active[1] and Nash[2], along with more challenging datasets, AMC2023 and Gaokao. Our method achieves higher accuracy while maintaining reasonable token consumption.\n\n|Method|AQuA|Token|AddSub|Token|AMC2023|Token|Gaokao|Token|\n|-|-|-|-|-|-|-|-|-|\n|Zero|39.8%|614|57.5%|117|2.5%|841|31.3%|508|\n|SC|63.0%|1,875|81.5%|743|15.0%|2,950|35.9%|3,357|\n|Active|66.1%|1,649|86.3%|542|12.5%|2,421|40.5%|3,409|\n|Nash|68.9%|6,598|88.9%|4,431|25.0%|12,940|43.0%|8,705|\n|Our|70.1%|2,884|89.1%|1,028|27.5%|7,604|44.2%|4,069|\n\n[1] Diao S,et al. Active Prompting with Chain-of-Thought for LLMs. \\\n[2] Zhang Z,et al. Multi-Path Inference with Preference Equilibrium.\n\n> **Q6.** Adaptive iteration improves efficiency, but the underlying mechanism remains unclear.\n\n**A6.** Adaptive iteration enhances efficiency via early stopping based on semantic entropy. When LLM shows high confidence, it dynamically halts to save computation. As shown in Table, our method averages 2.1 iterations (vs. fixed 3), with 55.9% of samples stopped early, reducing token while maintaining performance.\n\n|Maximum Iterations|1-1(0.0%)|2-1.6(40.2%)|3-2.1(55.9%)|4-2.5(60.1%)|5-3.1(66.5%)|\n|-|-|-|-|-|-|\n|Fixed Iteration|2,154|4,629|6,509|8,717|11,232|\n|Adaptive Iteration|2,154|3,779|4,591|5,458|6,609|", "claims_and_evidence": "The paper claims that CoT reasoning shares the core objective of uncertainty reduction with self-training and that their proposed framework, inspired by this analogy, improves CoT performance. While the empirical evidence presented in the experiments generally supports the performance improvements of the proposed framework, the theoretical link between self-training's entropy variation and CoT reasoning is more conceptual and analogical than rigorously derived. The claim that semantic entropy is an effective metric for guiding CoT iterations and prompt selection is supported by experimental results, but lacks direct comparative evidence against more conventional prompt selection methods. The justification for the specific design choices, such as the exploratory prompt and the adaptive iteration strategy, relies more on intuitive reasoning and analogy to self-training principles than on strong theoretical backing specific to CoT.", "methods_and_evaluation_criteria": "The proposed methods, including the task-specific prompt module using semantic entropy for prompt selection and the adaptive reasoning iteration module with entropy-based stopping and exploratory prompts, are novel and relevant to addressing limitations in traditional CoT approaches. The use of semantic entropy as a metric for uncertainty in CoT reasoning is an interesting approach. The evaluation criteria, using standard benchmark datasets across arithmetic, commonsense, and symbolic reasoning, are generally appropriate for evaluating CoT methods. However, the benchmarks used are not the most challenging and cutting-edge datasets in these domains.", "theoretical_claims": "The paper includes theoretical claims regarding entropy variation in self-training (Lemma 2.1 and Theorem 2.2). While I did not thoroughly check the correctness of the proofs, the paper primarily relies on referencing existing theoretical results in self-training. It's important to note that these theoretical claims directly apply to self-training and are used to motivate and inspire the design of the CoT framework, rather than providing a rigorous theoretical foundation for the proposed CoT method itself. The link between self-training's entropy theorems and the proposed CoT framework is thus analogical and motivational, not deductively derived.", "experimental_designs_or_analyses": "The experimental designs and analyses are generally sound and follow standard practices for evaluating CoT methods. The ablation studies effectively demonstrate the contribution of the task-specific prompt and adaptive reasoning iteration modules. \n\nHowever, a notable weakness is the lack of ablation studies on the number of self-consistency samples (N), which is a crucial parameter influencing semantic entropy calculation and the overall framework's effectiveness. \n\nFurthermore, the experimental section would be strengthened by including a direct comparison of the proposed semantic entropy-based prompt selection method with more conventional prompt selection techniques to justify its added complexity and computational cost.", "relation_to_broader_scientific_literature": "The paper builds upon the established literature of chain-of-thought reasoning and connects it to the well-studied field of self-training. The key contribution lies in applying the concept of uncertainty reduction, inspired by self-training, to enhance CoT reasoning through semantic entropy-guided prompt selection and adaptive iteration. The use of semantic entropy as a metric for uncertainty in CoT and the proposed adaptive iteration strategy are novel contributions to the CoT literature.", "essential_references_not_discussed": "To my best knowledge, this paper includes most of the essential references in the Introduction section.\n\nHowever, in order to improve the readability, the paper should improve the discussion of essential references. The lack of \"Related Work\" section leads to insufficient discussions of the proposed work in the context of existing related ones.\n\nFor example, the paper should discuss and ideally compare against more conventional prompt selection methods used in the CoT literature, such as those relying on simpler heuristics or ensemble-based prompt selection. Additionally, while the paper cites some popular CoT methods as baselines, including more recent and stronger CoT baselines in the experimental comparison would provide a more comprehensive evaluation of the proposed framework's advancements.", "other_strengths_and_weaknesses": "Strengths:\n* Originality: The paper presents a novel perspective on CoT reasoning by drawing inspiration from self-training and introducing semantic entropy as a guiding metric.\n* Significance: The proposed framework offers a potentially effective approach to improve CoT performance and address limitations like over-reasoning and lack of diversity in reasoning paths.\n* Clarity: The paper is generally well-written and clearly explains the proposed framework and experimental setup.\n* Empirical Validation: The experimental results demonstrate promising performance improvements across various datasets.\n\nWeaknesses:\n* Conceptual Link Strength: The analogy between self-training and CoT, while interesting, could be theoretically strengthened. The justification for transferring entropy-based principles from self-training to CoT reasoning is not fully rigorous.\n* Computational Cost: The computational cost of task-specific prompt selection using semantic entropy and the per-iteration cost of adaptive reasoning could be a concern, and needs more thorough justification and comparison to simpler alternatives.\n* Limited Baselines/Benchmarks: The choice of baselines and benchmarks could be more comprehensive and challenging to fully showcase the advantages of the proposed framework.\n* Theoretical Justification for Exploratory Prompt: The theoretical grounding for the specific exploratory prompt design in adaptive iteration could be more explicitly linked to the identified limitations of CoT.\n* Unclear Time Efficiency Benefit: The claimed time efficiency gains of adaptive iteration compared to fixed iteration are not entirely clear from the provided analysis.", "other_comments_or_suggestions": "* Consider strengthening the theoretical justification for applying self-training-inspired entropy principles to CoT reasoning.\n* Include experiments comparing the semantic entropy-based prompt selection with simpler prompt selection methods.\n* Perform ablation studies on the number of self-consistency samples (N) to assess its impact.\n* Evaluate the framework on more challenging and cutting-edge benchmarks, especially in mathematical reasoning.\n* Expand the baseline comparison to include more recent and stronger CoT methods.\n* Clarify the time cost analysis and the specific mechanisms through which adaptive iteration achieves time efficiency gains.", "questions_for_authors": "1. Theoretical Justification of Self-Training Analogy: While the analogy to self-training is interesting, could the authors elaborate on the theoretical justification for directly applying entropy-based principles from self-training (which involves parameter updates) to CoT reasoning (which manipulates inputs without changing model weights)? How can Theorem 2.2, derived for self-training, be rigorously argued to motivate the design choices in the proposed CoT framework? A more detailed explanation of this theoretical bridge would strengthen the paper.\n2. Comparison to Conventional Prompt Selection: The task-specific prompt module relies on computationally expensive semantic entropy calculations. Could the authors include experiments comparing this approach to more conventional and computationally cheaper prompt selection methods (e.g., simpler heuristics, frequency-based methods) in terms of both performance and computational efficiency? Demonstrating a clear advantage over simpler methods would better justify the added complexity.\n3. Theoretical Basis for Exploratory Prompt: The exploratory prompt \"reevaluate from alternative perspectives\" is designed to encourage diversity. Could the authors provide a more explicit theoretical justification for why this specific prompt design is expected to effectively promote exploration and reduce semantic entropy in subsequent CoT iterations, especially in relation to the issue of high similarity between consecutive iterations identified in Section 2.2?\n4. Ablation Study on Self-Consistency Sampling (N): The number of self-consistency samples (N) is a crucial parameter for semantic entropy estimation. Could the authors include ablation studies varying N to analyze its impact on the framework's performance and robustness? Understanding the sensitivity to N is important for practical application and evaluating the reliability of the semantic entropy metric.\n5. Rationale for Baseline Choices and Benchmark Complexity: What was the rationale behind choosing Contrastive-CoT and RE2 as the primary baselines? Were more recent and potentially stronger CoT methods considered? Furthermore, while the benchmarks are standard, could the authors discuss why these benchmarks were chosen and whether evaluating on more challenging, cutting-edge datasets, particularly in mathematical reasoning, could further validate the framework's capabilities and significance?\n6. Clarification of Time Efficiency in Adaptive Iteration: Figure 5b suggests time efficiency gains with adaptive iteration. Could the authors clarify how the adaptive iteration method achieves this efficiency gain, given that each iteration involves two LLM calls? Is the time reduction primarily due to the early stopping mechanism when semantic entropy is low, or are there other factors contributing to the improved time efficiency compared to fixed iteration? A clearer explanation of the trade-offs between per-iteration cost and total iterations in adaptive reasoning would be valuable.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QaOIfS7yGq", "reviewer": "Reviewer_wWvP", "summary": "This paper explores the conceptual similarity between CoT reasoning and self-training, highlighting their shared goal of minimizing predictive uncertainty by iteratively leveraging model-generated information. Based on this insight, the authors propose a novel CoT framework integrating a Task-Specific Prompt module and an Adaptive Reasoning Iteration module. Experiments on 10 reasoning datasets demonstrate that the proposed approach significantly outperforms baseline CoT methods in both reasoning performance and computational efficiency, with particularly strong results on arithmetic datasets. The main contributions include establishing a connection between CoT reasoning and self-training through entropy minimization, introducing TSP and ARI modules, and achieving notable improvements in complex reasoning tasks with strong generalization and efficiency.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the valuable feedback and encouraging comments. We are motivated by the suggestions and have addressed each concern in detail as follows:\n\n> **Q1.** Although the authors propose a method for automatically searching for the \"optimal prompt,\" they do not provide specific examples in the experiments, nor do they analyze why it outperforms general prompts (e.g., \"let's think step by step\").\n\n**A1.** Our proposed TSP aims to automatically generate initial reasoning prompts that better align with the specific task’s semantic distribution, serving as a replacement for generic prompts. Although generic prompts exhibit a certain degree of adaptability across most tasks, their expressions are relatively generalized and often fail to sufficiently guide LLMs to focus on the critical reasoning paths required by the task. The TSP module leverages real task samples to guide the LLM in generating multiple candidate prompts, then selects the optimal one based on the evaluation metric of average semantic entropy. This enhances the quality of initial reasoning and reduces the number of iterations. Below are examples of optimal prompts automatically searched on different datasets, showing how they further integrate task-specific features on top of the generic template:\n\n- AQuA: \"Let's think step by step, how to break down the mathematical operations involved in each problem and identify the key concepts to solve them accurately.\"  \n- AddSub: \"Let's think step by step, how to efficiently solve these word problems involving basic arithmetic operations and simple logic.\"\n\n> **Q2.** Some technical details lack contextual background. For instance, the \"Jaccard Index\" is mentioned directly in the method section, but its role or the rationale for choosing this metric is not explained. \n\n**A2.** We introduce the Jaccard Index in the method section to measure the lexical-level similarity between reasoning results of adjacent iterations, aiming to determine whether the current iteration has generated sufficiently diverse reasoning paths, thereby avoiding repetitive or redundant reasoning by the model. The Jaccard Index is chosen for its simplicity and efficiency in evaluating the overlap between two sets, making it particularly suitable for quickly assessing word-level differences between texts and effectively capturing trends in reasoning diversity. We will supplement the relevant background explanation in the paper to enhance readability and completeness.\n\n> **Q3.** There is no sensitivity analysis of the maximum number of iterations.\n\n**A3.** We have added a sensitivity analysis of the maximum number of iterations $T$, with the experimental results shown in Table. It can be observed that the model achieves a significant accuracy improvement at $T=3$, and the performance tends to saturate or even fluctuate slightly after $T=4$, indicating that a relatively small number of iterations is sufficient to achieve near-optimal reasoning performance. Taking the AQuA and AddSub datasets as examples, the accuracy improvement slows down notably after $T=3$, suggesting limited gains from further iterations. Combined with our proposed semantic entropy-based early stopping mechanism, the system can dynamically decide whether to continue reasoning for each sample, effectively reducing unnecessary computational overhead while maintaining performance.\n\n| Dataset   | $T=1$      | $T=2$      | $T=3$      | $T=4$      | $T=5$      |\n|----------|--------|--------|--------|--------|--------|\n| AQuA     | 59.5% | 70.8% | 70.1% | 71.7% | 71.3% |\n| AddSub   | 80.5% | 86.1% | 88.4% | 88.8% | 88.6% |\n\n\n\n> **Q4.** The symbol definitions in the formulas are not clear.\n\n**A4.** Thank you for the comment. We have carefully double-checked the paper and revised the notations. In the final version, we will ensure that all symbols are clearly defined at first use.\n\n> **Q5.** Why does the proposed method perform better on arithmetic datasets than on commonsense datasets?\n\n**A5.** We believe this difference primarily stems from the varying degrees to which different task types rely on the capabilities of language models. Arithmetic datasets emphasize strict logical steps and step-by-step calculation processes, which closely align with our method’s design that involves multi-turn reasoning and explicit intermediate steps. The model can iteratively approach the correct answer, thus benefiting significantly. In contrast, commonsense tasks rely more on the world knowledge and language co-occurrence patterns acquired during pretraining. These tasks have a more open reasoning space, more diverse answer forms, and often lack a clear logical chain. As a result, multi-turn reasoning offers limited gains for such tasks and may even introduce unnecessary redundant information that affects the final judgment.", "claims_and_evidence": "The proposed method is generally clear and compelling. The authors support their main contributions with experimental results on three types of reasoning tasks and corresponding theoretical analysis.", "methods_and_evaluation_criteria": "The proposed method is well-founded and suitable for the research problem. The evaluation criteria align with existing benchmarks, and the experimental setup is reasonable.", "theoretical_claims": "The theoretical arguments presented in the paper have been carefully examined and are generally correct.", "experimental_designs_or_analyses": "The experimental design effectively evaluates the proposed method. The results show that the method achieves strong performance on multiple benchmark datasets, and both experiments with fixed iteration counts and comparison experiments support the authors' main conclusions.", "relation_to_broader_scientific_literature": "The paper is consistent with research in the related field and provides new insights into the essence of chain-of-thought.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "Strengths:\n\n1.\tThe paper presents an interesting and insightful perspective by drawing an analogy between CoT reasoning and self-training, especially the pseudo-labeling strategy, highlighting their common goal of \"iteratively reducing predictive uncertainty.\"\n\n2.\tThe two modules proposed in the paper (TSP and ARI) can serve as plug-and-play enhancement components. \n\n3.\tThe writing is clear. The authors provide theoretical proofs, algorithm pseudocode, and examples of reasoning processes in the appendix.\n\n4.\tThe paper selects datasets that cover various types of reasoning, demonstrating the method's versatility. \n\n\nWeaknesses:\n\n1. Although the authors propose a method for automatically searching for the \"optimal prompt,\" they do not provide specific examples in the experiments, nor do they analyze why it outperforms general prompts (e.g., \"let's think step by step\").\n\n2. Some technical details lack contextual background. For instance, the \"Jaccard Index\" is mentioned directly in the method section, but its role or the rationale for choosing this metric is not explained.\n\n3. There is no sensitivity analysis of the maximum number of iterations.", "other_comments_or_suggestions": "The symbol definitions in the formulas are not clear.", "questions_for_authors": "Why does the proposed method perform better on arithmetic datasets than on commonsense datasets?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AhrabZZgaw", "reviewer": "Reviewer_XjxR", "summary": "Inspired by self-training, this paper designs the CoT framework to improve reasoning performance. It contains two core elements: a task-specific prompt and an adaptive reasoning iteration. This paper finally conducted experiments on 10 reasoning datasets and achieved improved results.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the valuable feedback and encouraging comments. We are motivated by the suggestions and have addressed each concern in detail as follows:\n\n> **Q1.** Main experiments should compare both effectiveness and token overhead of each method.\n\n**A1.** We have added token consumption to the main experiments to assess performance-cost trade-offs. Our method outperforms others while using fewer tokens than the SOTA method (e.g., Nash), especially on challenging tasks like AMC2023 and Gaokao, showing better efficiency in complex reasoning.\n\n| Method|AQuA|Token|AddSub|Token|AMC2023|Token|Gaokao|Token|\n|-|-|-|-|-|-|-|-|-|\n|Zero|39.8%|614|57.5%|117|2.5%|841|31.3%|508|\n|SC|63.0%|1,875|81.5%|743|15.0%|2,950|35.9%|3,357|\n|Active|66.1%|1,649|86.3%|542|12.5%|2,421|40.5%|3,409|\n|Nash|68.9%|6,598|88.9%|4,431|25.0%|12,940|43.0%|8,705|\n|Our|70.1%|2,884| 89.1%|1,028|27.5%|7,604|44.2%|4,069|\n\n> **Q2.** Lack of sufficient theoretical claims on LLM Semantic Entropy.\n\n**A2.** The highly nonlinear nature of LLMs makes it difficult to model their output distribution using traditional probabilistic methods (e.g., GMM), posing challenges for establishing a rigorous theoretical framework for semantic entropy. This paper focuses on proposing a practical, scenario-driven CoT optimization framework, using semantic entropy as a heuristic metric to capture answer distribution dynamics during reasoning. While its theoretical foundation is not fully established, experiments demonstrate its effectiveness, and we will further investigate its theoretical basis in future work.\n\n> **Q3.** I have concerns about its adaptability and generalization ability in more scenarios.\n\n**A3.** To evaluate the generalization of the semantic entropy mechanism across tasks, we added experiments on two distinct datasets: Race (English reading comprehension task) and Anli (adversarial natural language inference task), each with 300 test samples. As shown in Table, our method achieved clear performance gains on both, demonstrating strong robustness across language understanding tasks. We plan to explore its application to Vision and Cross-Modal tasks in future work.\n\n| Method|Race|Token|Anli|Token|\n|-|-|-|-|-|\n|Zero|80.0%|508|45.0%|268|\n|SC|84.5%|2,530|56.0%|1,456|\n|Our|88.5%|3,467|64.5%|2,245|\n\n> **Q4.** The assumption of a mixture of Gaussians in the proof seems overly ideal.\n\n**A4.** We adopt GMM as the modeling assumption based on two reasons: (1) GMM is a widely used and interpretable model that helps illustrate how pseudo-labels reduce entropy; (2) our results are a direct corollary of [1], which provides a general derivation under the sub-exponential family, of which GMM is a special case. To balance rigor and clarity, we use GMM as a heuristic example, while the conclusions also hold under broader distributions given certain conditions.\n\n[1] Frei S,et al. self-training converts weak learners to strong learners in mixture models.\n\n> **Q5.** How to determine the optimal predefined threshold $\\delta$?\n\n**A5.** The predefined threshold $\\delta$ is empirically determined based on the task type and the number of samples $N$ in self-consistency (SC). Specifically, we allow up to $k$ predictions to deviate from the majority class and compute the corresponding entropy threshold using the ratio $(N - k)/N$. For example, in the more challenging reasoning task AQuA, when $N = 4$ and $k = 1$ (i.e., allowing at most 1 out of 4 predictions to differ from the majority), the corresponding threshold is calculated as:\n$\\delta = -\\left( \\frac{N-k}{N} \\log \\frac{N-k}{N} + \\frac{k}{N} \\log \\frac{k}{N} \\right) \\approx 0.811$.\n\nAs shown in Table, setting the threshold too high may cause LLM to terminate early before identifying the correct answer, thus harming accuracy. Conversely, setting it too low may lead to excessive reasoning and the introduction of noise, which can also degrade performance. Therefore, we typically set $k$ between 0 and 2, and use it to compute semantic entropy as the decision threshold $\\delta$. As $N$ increases, we appropriately raise $k$ to enhance the robustness of the decision-making process.\n\n|Allowed Inconsis / SC|$N=2$|$N=3$|$N=4$|$N=5$|\n|-|-|-|-|-|\n|$k=0$|70.9%|70.1%|71.3%|72.4%|\n|$k=1$|--|68.2%|72.8%|73.6%|\n|$k=2$|--|--|68.7%|71.7%|\n|$k=3$|--|--|--|69.3%|\n\n> **Q6.** Does the pipeline generalize across tasks, or must the optimized CoT be re-tuned for each one?\n\n**A6.** Our framework includes a reasoning pipeline composed of the TSP and ARI modules. While ARI is task-agnostic and requires no adaptation, TSP involves optimization on a single source task. Once optimized, the entire pipeline can be directly applied to other tasks of a similar nature without re-tuning. This demonstrates the generalizability of our pipeline across tasks.\n\n|Source / Target|AQuA|AddSub|AMC2023|Gaokao|\n|-|-|-|-|-|\n|AQuA|70.1%|85.6%|20.0%|40.2%|\n|AddSub|67.7%|88.4%|20.0%|41.3%|\n|AMC2023|68.1%|86.3%|25.0%|41.9%|\n|Gaokao|66.9%| 84.3%|17.5%|44.2%|", "claims_and_evidence": "I think the discussion on Semantic Entropy for LLM reasoning based on self-training is interesting, but it lacks theoretical proof and experimental verification in more scenarios.", "methods_and_evaluation_criteria": "I think the main experiment needs to not only compare the effects, but also the token costs brought by the comparison methods.", "theoretical_claims": "Lack of sufficient theoretical claims on LLM Semantic Entropy.", "experimental_designs_or_analyses": "See details in Methods And Evaluation Criteria.", "relation_to_broader_scientific_literature": "The optimization and design of chain of thought may be helpful for the automated design of agents and the generalization research of ML.", "essential_references_not_discussed": "Overall complete.", "other_strengths_and_weaknesses": "I think the design of the idea from self-training to semantic entropy is interesting. However, I doubt whether this method will bring additional token overhead, and the effect in the experiment does not seem to be very significant. In addition, the effectiveness of semantic entropy is mainly verified from experimental phenomena, and I have concerns about its adaptability and generalization ability in more scenarios. Furthermore, the assumption of a mixture of Gaussians in the proof seems overly ideal.", "other_comments_or_suggestions": "There are some minor typos in the writing of the paper, for example, the first letter of \"we\" in line 105 should be capitalized, etc.", "questions_for_authors": "1. How to determine the optimal predefined threshold  $\\delta$? 2. Does this pipeline have a certain degree of generalization ability? For example, is the optimized COT adaptable to new tasks? Or does it mean that each task needs to be optimized in a specific way?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "16ezFbg0kv", "reviewer": "Reviewer_Pmva", "summary": "This paper investigates CoT approach and proposes a novel CoT framework inspired by its similarity to self-training to enhance reasoning performance. The framework consists of two core modules: a task-specific prompt module that optimizes the initial reasoning process and an adaptive reasoning iteration module that dynamically adjusts the reasoning procedure to address issues in existing CoT methods, such as over-reasoning and high similarity across consecutive reasoning iterations. Experimental results demonstrate that the proposed approach achieves significant improvements in both performance and computational efficiency.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the valuable feedback and encouraging comments. We are motivated by the suggestions and have addressed each concern in detail as follows:\n\n> **Q1.** There is a lack of discussion on negative results. While the paper primarily demonstrates the advantages of the method, it does not provide an explanation for the model's poor performance on certain datasets, such as why the improvement in the \"commonsense\" task is limited.\n\n **A1.** We acknowledge that the performance gain is relatively limited on commonsense tasks. This is mainly because such tasks rely more on pretrained world knowledge and co-occurrence patterns, rather than explicit logical steps. In contrast, arithmetic tasks better align with our iterative reasoning framework. For open-ended tasks, additional reasoning may introduce noise rather than benefit. We will explore better adaptations to such tasks in future work.\n\n> **Q2.** The main text lacks a Related Work section, and the Task-Specific Prompt part of Framework Figure 4 does not clearly describe the specific details, making it difficult for readers to understand how to obtain the candidate prompts and select the optimal one.\n\n **A2.** In the final version, we will add a “Related Work” section to cover relevant studies on Self-Training, Chain-of-Thought reasoning, and semantic entropy. We will also refine Figure 4 and clarify that the task-specific prompt module generates candidate prompts based on several example questions, evaluates them using average semantic entropy on a validation set, and selects the one with the lowest entropy as the final prompt.\n\n> **Q3.** Is it better to have a larger maximum number of iterations in the proposed ARI module?\n\n **A3.** We have added a sensitivity analysis of the maximum number of iterations $T$, with the experimental results shown in Table. Overall, increasing the maximum number of iterations does not always lead to better performance. Taking AQuA and AddSub as examples, the model already achieves a relatively high accuracy at $T=3$, while further increasing $T$ to 4 or 5 leads to saturated or slightly fluctuating performance, along with a significant increase in computational cost. At the same time, for more challenging tasks, a larger maximum iteration count may offer a broader reasoning space and thus yield further performance improvements. Therefore, the optimal value of $T$ should be determined based on task difficulty: for simpler tasks, a smaller $T$ is more efficient; whereas for more complex tasks, allowing a larger $T$ may better support sufficient reasoning.\n\n| Dataset| $T=1$| $T=2$| $T=3$| $T=4$| $T=5$|\n|-|-|-|-|-|-|\n| AQuA     | 59.5%| 70.8%| 70.1%| 71.7%| 71.3%|\n| AddSub   | 80.5%| 86.1%| 88.4%| 88.8%| 88.6%|\n\n> **Q4.** This paper should include an intuitive analysis of the prompts selected by the TSP, such as whether they provide stronger guidance or how they structurally differ from manually crafted prompts.\n\n **A4.** Our proposed TSP module is designed to automatically generate initial reasoning prompts that better align with the task’s semantic distribution, serving as a replacement for generic prompts. While generic prompts possess a certain degree of generality, their expressions tend to be overly abstract and fail to effectively guide the model to focus on the key reasoning elements within a task. In contrast, TSP leverages actual task samples to generate multiple candidate prompts and selects the most effective one based on average semantic entropy, thereby improving the quality of initial reasoning and reducing the number of iterations. Structurally, prompts generated by TSP typically extend the generic prompt template with additional task-specific semantic cues. For example, for the AQuA dataset, TSP produces a prompt: \"Let's think step by step, how to break down the mathematical operations involved in each problem and identify the key concepts to solve them accurately\".\n\n> **Q5.** Does the number of self-consistency samples have any particular impact on the proposed method?\n\n **A5.** We conducted additional experiments on four datasets to evaluate the impact of the sampling number $N$ on model performance and robustness, as shown in Table below. The results show that as $N$ increases, the overall model accuracy improves, with more pronounced gains on challenging tasks such as AMC2023 and Gaokao. Meanwhile, performance tends to saturate after $N=3$, indicating that even a relatively small $N$ can yield stable semantic entropy estimates. This balances performance with computational efficiency and demonstrates the method’s strong robustness with respect to the parameter $N$.\n\n| Dataset| $N=1$| $N=2$| $N=3$| $N=4$| $N=5$|\n|-|-|-|---------|---------|---------|\n| AQuA      | 53.9%  | 70.9%  | 70.1%  | 71.3%  | 72.4%|\n| AddSub    | 73.9%  | 85.1% | 88.4%  | 88.9%  | 90.4%|\n| AMC2023   | 12.5%  | 20.0%  | 25.0%  | 27.5%  | 30.0%|\n| Gaokao    | 32.8%  | 41.3%  | 44.2%  | 44.7%  | 45.3%|", "claims_and_evidence": "The key claims of this paper include: (1) CoT reasoning and self-training share the same core objective; (2) reasoning performance can be improved through task-specific prompting and adaptive reasoning iteration; (3) the proposed approach outperforms traditional CoT methods in computational efficiency. Experimental results support these claims, demonstrating improvements in both performance and computational efficiency.", "methods_and_evaluation_criteria": "The proposed approach effectively integrates task-specific prompt optimization and adaptive reasoning iteration, enhancing reasoning capabilities while reducing computational overhead. The evaluation criteria employed in this paper align with those of traditional CoT methods, wherein key segments of the generated output are extracted and compared against the ground truth.", "theoretical_claims": "The theoretical contribution of the paper primarily focuses on the analysis of entropy variation in self-training. The claims are intuitively reasonable, and the proofs provided in the Appendix should be correct.", "experimental_designs_or_analyses": "The paper validates the effectiveness of the proposed method through extensive experiments, focusing primarily on the improvements in reasoning performance and computational efficiency.", "relation_to_broader_scientific_literature": "This study builds upon CoT reasoning and self-training methods, proposing a new framework that combines the advantages of both. Compared to existing CoT reasoning methods, the proposed approach introduces task-specific prompt optimization and adaptive reasoning iteration, thereby reducing issues of over-reasoning and high similarity.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "Strengths：\n1) This paper is grounded in solid theoretical foundations, providing an in-depth theoretical analysis of entropy variation. Additionally, the experimental results are significant, demonstrating that the proposed framework outperforms baseline methods in both performance and computational efficiency, particularly excelling in arithmetic reasoning tasks.\n\n2) In Section 2, this paper provides an in-depth theoretical discussion on the uncertainty measure (information entropy) in self-training, analyzing the mechanism by which the iterative generation of pseudo-labels in semi-supervised self-training leads to progressively improved prediction accuracy. Subsequently, the authors apply this \"entropy reduction\" perspective to CoT reasoning, and in the following experiments, they conduct a quantitative comparison using semantic entropy, supported by both theoretical analysis and extensive experimental validation.\n\n3) The Task-Specific Prompt module proposed by the authors automatically searches for or constructs prompt statements from task samples, rather than using the generic \"Let's think step by step.\" In scenarios that require more precise guidance, this approach may yield better results. The authors' experiments also confirm that targeted prompts can significantly improve the results of the first-round reasoning.\n\n4) The Adaptive Reasoning Iteration module proposed by the authors incorporates a semantic entropy-based exit mechanism, preventing further reasoning when the model is already \"highly confident,\" thus avoiding the introduction of additional errors. Additionally, to reduce excessive repetition of reasoning across different iterations and control reasoning similarity, the module introduces new prompts. This encourages LLMs to explore more diverse reasoning paths.\n\n5) The overall logic is clear, and the writing is smooth.\n\nWeaknesses:\n1) There is a lack of discussion on negative results. While the paper primarily demonstrates the advantages of the method, it does not provide an explanation for the model's poor performance on certain datasets, such as why the improvement in the \"commonsense\" task is limited.\n\n2) The main text lacks a Related Work section, and the Task-Specific Prompt part of Framework Figure 4 does not clearly describe the specific details, making it difficult for readers to understand how to obtain the candidate prompts and select the optimal one.", "other_comments_or_suggestions": "1)Is it better to have a larger maximum number of iterations in the proposed ARI module?\n2)This paper should include an intuitive analysis of the prompts selected by the TSP, such as whether they provide stronger guidance or how they structurally differ from manually crafted prompts.", "questions_for_authors": "Does the number of self-consistency samples have any particular impact on the proposed method?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper connects CoT reasoning with self-training, showing both aim to reduce predictive uncertainty using model-generated outputs. Based on this, the authors propose a new CoT framework with a task-specific prompt and adaptive reasoning iteration module. Experiments on several reasoning benchmarks show notable gains in accuracy and efficiency, especially on arithmetic tasks. All reviewers unanimously recommended for acceptance.
Decision: Accept (poster)