Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning
Authors: Xiangning Yu
Zhuohan Wang
Linyi Yang
Haoxuan Li
Anjie Liu
Xiao Xue
Jun Wang
Mengyue Yang
Venue: NeurIPS 2025 poster ()
Keywords: PNS
Chain-of-Thought(CoT)
Causal Suffciency Necessity
Primary area: deep_learning

Abstract: Chain-of-Thought (CoT) prompting plays an indispensable role in endowing large language models (LLMs) with complex reasoning capabilities. However, CoT currently faces two fundamental challenges: (1) Sufficiency, which ensures that the generated intermediate inference steps comprehensively cover and substantiate the final conclusion; and (2) Necessity, which identifies the inference steps that are truly indispensable for the soundness of the resulting answer. We propose a causal framework that characterizes CoT reasoning through the dual lenses of sufficiency and necessity. Incorporating causal Probability of Sufficiency and Necessity allows us not only to determine which steps are logically sufficient or necessary to the prediction outcome, but also to quantify their actual influence on the final reasoning outcome under different intervention scenarios, thereby enabling the automated addition of missing steps and the pruning of redundant ones. Extensive experimental results on various mathematical and commonsense reasoning benchmarks confirm substantial improvements in reasoning efficiency and reduced token usage without sacrificing accuracy. Our work provides a promising direction for improving LLM reasoning performance and cost-effectiveness. The code will be publicly available upon acceptance at: https://anonymous.4open.science/r/causalmath-1CEF.

Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning
Xiangning Yu 1, Zhuohan : Yang", Haoxuan Li*, Anjie Liu", Xiao X Wang", Mengyue Yang Tianjin University City University of Hong Kong
University College London *Peking University
The Hong Kong University of Science and Technology (Guangzhou) UUniversity of Bristol
Abstract
Chain-of-Thought (CoT) prompting plays an indispensable role in endowing large language models (LLMs) with complex reasoning capabilities. However, CoT currently faces two fundamental challenges: (1) Sufficiency, which ensures that the generated intermediate inference steps comprchensively cover and substantiate the final conclusion; and (2) Necessity, which identifies the inference steps that are truly indispensable for the soundness of the resulting answer. We propose causal framework that characterizes CoT reasoning through the dual lenses of suffi- ciency and necessity. Incorporating causal Probability of Sufficiency and Necessity allows us not only to determine which steps are logically sufficient or necessary the prediction outcome, but also quantify their actual influence the final reasoning outcome under different intervention scenarios, thereby enabling the automated addition of missing steps and the pruning of redundant ones. Exten- sive experimental results on various mathematical and commonsense reasoning benchmarks confirm substantial improvements in reasoning efficiency and reduced token usage without sacrificing accuracy. Our work provides promising direction for improving LLM reasoning performance and ofstteffectiveness. The code is available at: https:///ithu:t
Introduction
Large Language Models (LLMs) have demonstrated impressive advancements in complex reasoning tasks, significantly attributed to the adoption of Chain-of-Though (CoT), CoT prompting guides models to generate intermediate reasoning steps, thereby enhancing performance in areas such as arithmetic problem-solving and commonsense reasoning [3 17
g. Despite these improvements,
CoT reasoning faces two fundamental challenges: (i) Sufficiency: ensuring that the generated intermediate steps comprehensively support the conclusion [50 33] and (ii) Necessity: identifying which steps are indispensable for the soundness of the final answer 54 Figure [Ta illlustrassss three common reasoning patterns frequently observed in LLMs, exemplified here using GSM-8k [10)
question: (1) Sufficient but Unnecessary, where redundant steps reduce reasoning efficiency; (2)
Necessary but Insufficient, in which incomplete reasoning fails to reach the correct answer; and (3)
Sufficient and Necessary, the ideal case that balances correctness and conciseness. These examples highlight the impact of reasoning inefficiencies-esst especially "overthinking", where unnecessary steps may hinder rather than help model performance.
"qqual contribution.
i "Corespondiig authors. I Project leader
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Sufficinnt
Untecessory
X
Sufficienc and Necessory Duaster:
Recensory bot Prugfficnnt
Intervention penersted nodes
Gusaaon hoad
F1
PMS-sellccedd
Skpp Zifosiasient 0nS 2- ANert
Concinlio
Slop zong 1200 2 2 lakel 220
F1..Fk denote the sufficient necessary formulas required onswer the question, with textuual explanation.
222 Jrrslars 3ije
1100 203 12/00-1
oddeed the rest pet
Flo11
Stop
(100) 21206 212-29.1
(6000
dut sho diat help the corrons mastele? Maybe dhoutly 5612 andtiwaass
Question
Stop 1000 2//2/0912
Conciation: 9001
Final path formed by PNS-selected nodes.
(a)
(h)
Figure 1: (a) Illustration of three reasoning Sipes-Suficientt but Unnecessary, Necessary but Insufficient, and Sufficient and Necessary- based on actual model-generated responses to a GSM-8k question: "Compute 9g2 + 99 1 in your head." " (b) Path selection process using our method. Purple nodes denote CoT steps obtained through causal intervention (rollout), while green nodes indicate the minimal steps satisfying both sufficiency and necessity.
Recent research on Mfhiin-ofTThougee (CoT) reasoning has addressed Sufficiency by introducing strategies such as self-consistency decoding (59] and iterative refinement methods like Self- Belf-Refine
[40], aiming to ensure intermediate steps comprehensively support final answers [27 48 17.. Con- currently, efforts targeting Necessity have developed pruning techniques, such as addressing the "overthinking" by reducing the token length 7 36). Chain-of-Draft prompting [67] and identify However, none have utilized rigorous mathematical analyses based on sufficient and necessary conditions [46] to evaluate and prune reasoning paths. These methods predominantly rely correlation based metrics (e.g., attention weights, likelihood scores, or ablation accuracy), which may misleadingly associate frequent or prominent steps with correctness without verifying true causal impact (4). Consequently, correlation alone is insufficient for reliably distinguishing genuinely necessary or sufficient reasoning steps, highlighting the need for causal frameworks rigorously assess their logical contributions.
critical reasoning steps [3), to reduce redundancy in reasoning paths 442
62 52
45
To jointly address the sufficiency and necessity of reasoning steps while ensuring logical and causal soundness, we introduce the concept of causal Probability of Necessity and Sufficiency (PNS) and redefine for CoT reasoning framework. We theoretically analyse the identifiability of PNS in CoT. Based on the identifiability results, we develop reconstruct reasoning sequences by causal intervention (rollout) (shown in Figure lb). Using this algorithm, we effectively reconstruct CoT responses from training data that explicitly meet causal sufficiency and necessity criteria, thus eliminating redundant steps without compromising and potentially enhancing answerr answer accuracy. The reconstructed reasoning CoTT then serve as causally- informed demonstrations, enabling LLMs to acquire causal reasoning capabilities via in-context learning and fine-tuning evaluations on mathematical reasoning benchmarks-including GSM-8k (10), MATH-500 [25]. and AIME 44. as well as the CommonsenseQA [53] cotaset-confirm that our approach significantly reduces reasoning redundancy while maintaining or improving prediction accuracy.
RNS-based evaluation algorithm to systematically
improve the efficiency without sacrificing the accuracy. Empirical
Our main contributions are follows:
1. We propose a conceptual integration of Probability of Necessary and Sufficient causation (PNS) into CoT reasoning.
2. We introduce novel bi-level optimization framework based on PNS for systematically constructing efficient and accurate CoT reasoning sequences.
3. We empirically validate our approach across diverse reasoning tasks, demonstrating substan- tial improvements in both efficiency and accuracy through optimized CoT traces used for in-context learning and supervised fine-tuning.

---

2 Related Work
Reasoning Sufficiency Enhancement via CoT Optimization. CoT reasoning [6] has significantly improved LLM performance on complex tasks, inspiring variants such as Tree-of-Thought [76].
Sraph-of-Thoure [6) and DOTS 78 Further developments include multimodal extensions [65]
latent variable formulations [64], and dynamic memory usage [73]. Others enhance reasoning via self-
correction 47, counterfactual fine-tuning [26]. or prompt design [88. Despite strong performance,
many methods suffer from unnecessary verbosity, inefficient computation, or overthinking [62 7
Beyond textual reasoning, task-grounded reasoning frameworks such as ChessGPT [20 bridge policy learning and language modeling, revealing that reasoning sufficiency can also be optimized in structured decision domains.
Reasoning CoT Redundancy. Recent work targets CoT redundancy by compressing reasoning traces (e.g., C3oT 32], CaT-Valve (39), CCoT (9), pruning superfluous steps [36], using token- budget-aware reasoning [22]. Training-fie frree approaches such as Kimi [54] and external thought injection further optimize reasoning cost. SPIRIT
leverages perplexity to identify key reasoning steps, balancing accuracy and efficiency in both few-shot and fine-tuned CoT settings, while also generalizing well across models. Ton et al. [57] use conditional mutual information to quantify each step's contribution to the final answer, revealing failure patterns without requiring intermediate supervision. However, existing methods often prioritize brevity or representational efficiency, without explicitly enforcing causal sufficiency or necessity.
35
PNS Theory in CoT Reasoning. Extending Pearl [46], introduce the Probability of Necessary and Sufficient causes (PNS) framework to CoT reasoning, applying PN and PS at the step level rather than the model level as Hüyük et [26]. This enables causal pruning of redundant steps, yielding minimal yet faithful CoTs. Relatedly, Yang et al. (72) formalize invariant learning through PNS estimation, offering principled view of causal sufficiency and necessity in representation learning. Building on this foundation, recent work on LLM causal reasoning (29 extends similar principles to in-context learning. Our approach generalizes these insights to step-level reasoning dynamies, providing model-agnostic and theoretically grounded alternative to heuristic compression methods.
Causal Necessity and Sufficiency in XAL Prior studies have leveraged causal necessity and suffi- ciency to explain model behavior. LENS [60] identifies necessary and sufficient output conditions; Darwiche and Hirth 14 compute sufficient reasons via Decision-DNNFF circuits; Mothilal al. 4I] generate diverse counterfactuals; Beckers SS] formalize sufficiency- bbsed explanations for fairness; and Galhotra al. 21 propose LEWIS, a probabilistic counterfactual method. Recent progress in causal representation learning [71, 70) further explores disentanglement and invariance through struc- tural causal models and counterfactual reasoning, demonstrating how causal principles enable robust and interpretable representations. Building on these ideas, our PNS evaluation extends necessity and sufficiency analysis to LLM reasoning chains, using counterfactual rollouts to assess the causal faithfulness of CoT traces and mitigate overthinking.
3 Defining Causal Necessary and Sufficiency in CoT
3.1 Chain-of-Thought (CoT) Reasoning
Definition of-ain-of-Thought (CoT) Reasoning 61
Given input Q = the Chain-of-
Thought (CoT) reasoning process generates the final answer A = a by sequentially deriving inter- mediate reasoning steps S {si, (2,,,2,.... ......, S..). The probability of generating the answer given the
question is defined as:
/ P(a S1, Si...., 9) x II P(s: Sei q) dS.
P(A =
Q -Q=9) |
(1)
Generation
CoT Generation
Explanation: P(A = a | == is the final answer probability. P(a St,..., Sn, q) corresponds to the conditional probability of generating the final answer from the full reasoning trace. I P(s:: 8cí, q) models the sequential reasoning process. The integral marginalizes over all possible reasoning traces S.
Sn

---

BASE MODEL Evalustor
PS ESTIMATION
do(S)
CoT Generator Answer Evaluator
correctness of CoT onswers.
BASE
ROLLOUT MODEL B, Intervention model
A
generating alternative reasoning choins.
Final prediction correctness outcome.
Y
YR
INTERVENTION
replace
PNS ESTIMATION
Perform counterfactual
&
PNS(S, 9)
Evaluator
intervention for
MODELL
PVS estimation.
Fine-tune & In-context Learning
INITIAL CoT Generated base model: full reasoning trace with possible redundoncy.
$1
$3
S3
COMPACT CoT
Remove non-essential steps to obtain minimal sufficient and necessory reasoning.
Figure Causal Optimization Framework CoT Reasoning. Our method identifies and retains only causally essential reasoning steps to form compact COT. (1) A base model generates the initial CoT trace, possibly containing redundant steps. (2) Sufficiency is estimated by checking if the full CoT leads to a correct answer. (3) For each step st, necessity is evaluated via counterfactual substitution & using a rollout model, followed by answer scoring from the base model. (4) The Probability of
Necessity and Sufficiency (PNS) is computed to measure causal contribution. (5) Non-essential steps are pruned to obtain a compact CoT, which is then used for fine-tuning or in-context learning.
3.2 Causal Necessary and Sufficiency in CoT
To rigorously characterize the causal significance of individual reasoning steps in Chain-of of Thought (CoT) reasoning, we propose formal definitions of causal sufficiency and causal necessity tailored to structure and properties of COT.
Definition 2 (Sufficiency, PS) Sufficiency measures whether the reasoning chain S = ($1..... sufficient to produce the correct answer y. Following the counterfactual definition Pearl (46]. the probability sufficiency is defined as:
PS(S. P (A = y A S,
where do(S) means the intervention which set the value of chain variable as S, Ado(S) denotes the counterfactual answer had the reasoning chain S been used, and do(S) denotes the actual answer under the original reasoning Š (which could be null or incorrect). This captures the likelihood that inserting S would have changed an incorrect answer correct
Necessity quantifies whether specific reasoning step St is required for producing the correct answer a y. Inspired by the counterfactual definition of necessity [46], we define the probability of necessity as:
Definition 3 (Necessity, PN)
PN(S, 9) P
PP0:::PAAAPPaaaaaaaa (A A
A
where sæt denotes the set of all correct reasoning steps before position t, represents incorrect or corrupted variant of the original step s,. The counterfactual outcome Ado(s.) is defined as the model's predicted answer when s; is replaced by s,, and the subsequent steps s' >t are generated conditioned on this modified reasoning trajectory.
Definition (Probability of Necessary and Sufficient Cause (PNS) in CoT) Given a reasoning chain S (s1....,S,) that produces the correct answer A y, and alternative reasoning step 8, at position f, let the counterfactual chain be defined as:
S' = (s St, >t),
where Sett denotes the preceding steps, and s' >t are subsequent steps possibly adapted St. Inspired by the counterfactual definition 46), we define the Probability of Necessary and Sufficient
Cause (PNS) for the step

---

PNS(S, St ②) = (As - y, x))
(4)
This quantifies the probability that step S, is sufficient and necessary for the correct answer under counterfactual. Identifiability results are in Appendix
Methodology PNS Estimation for Improving Chain-of-Thought Reasoning
In practice, directly maximizing the full-chain Probability of Necessity and Sufficiency (PNS) for reconstructed CoTs is computationally intractable. To address this. we adopt two-stage causal optimization strategy: we first enhance the chain-level Probability of Sufficiency (PS), followed by step-wise refinement via the node-level Probability of Necessity (PN). Guided by these criteria, propose an iterative pruning only those that are both causally sufficient and necessary for producing the correct answer. The overall process is illustrated in Figure which shows how initial CoT's are evaluated, intervened upon, and refined to produce minimal causal traces. These optimized CoTs are then used as high-quality exemplars for subsequent in-context learning and fine-tuning, enabling the base model to internalize which reasoning paths are truly essential.
that removes and reorders steps preserve
4.1 PNS Estimation and Algorithm for Reconstructing CoT
Algorithm 1: Sufficient and Necessary Optimization of CoT
Input: Initial CoT chain Simt. ground truth answer y, query q. threshold Output: Optimized CoT chain iiml
iin Rollout [S 9);
PS - [[iin y]]:
if PS then
Let Saument f- Siniti
foreach step S; Scurrent (processed in order) do
5t - Genneattelleernaiivee"""" current ss);;
V Vscores t empty list;
for j + 1 1... do
RollontContimnation (2t Caurtn St, B1,B); // is the rollout model; Forms (s (scurrent 3,, 5 (
Ensure semantic disjointness of (5,, 2((i from original (s,, s Symen
Add ((5())
Vicones
// V is the validation model
PNS (s²)
E EvEVisme
PNS vul (s:) > a then
Append to S
11 If St is deemed necessary, keep it
else
Skip s
// Drop unnecessary step
else
L Sinal
Smit
// Original chain not sufficient
return Sinali
Estimating PS (Chain-Level), We approximate CoT trace sufficiency binary: PS 1 if chain yields the correct answer (equivalent to P(A y do(S), = 1; Appendix A). else PS 0. To improve PS, we repeatedly execute Algorithml under the same question context. each execution. the model samples an alternative CoT, and its PS is re-evaluated. This repeated sampling increases likelihood of obtaining a CoT with higher PS. Lemma (proof in Appendix A establishes PNS identifiability (Definition 3.2 when PS 1:
Lemma (Identifiability of PNS under P(A y /o(s)) q) 1) Assume:
5

---

1. Perfect intervention with correct CoT S = St. Syt) yields P(A = do(S), q) -
2. Replacing step with incorrect S, and performing rollout 8'yt (from 5x) yields intervened chain S = (s<t St, s
Then, PNS/S, s. q) = P(Ado(s) assuming perfect intervention and the nature of S from Definition 3.2
A do(S) * y q) simplifies to 1 - P(A = y do(S). q),
When PS 1, PNS validity depends P(A y do(S), 9), reflecting PN's magnitude.
Estimating PN (Node-Level). S sufficient (PS(S, q) 1), Lemma guides PN estimation for each node evaluate PNS(SS, Se, q). We construct 5, by removing s; content and descendants. rollout model B generates revised downstream segment (5,, forming intervened chain
g( (s<t, S2 ) for the i-th rollout, ensuring 5, 53,538 are semantically disjoint from original components rooted at 5,. Each 5 assessed by validation model for coherence and logical integrity (beyond just final answer correctness). PNS is then computed via Monte-Care estimation over l rollouts:
k i=1 PNS(S, St 9) ~
- v(s
1
(5)
Nodes with PNS score below threshold a (and their downstream nodes) are pruned iteratively until all retained nodes satisfy necessity.
Iterative Optimization. Algorithm details the iterative optimization procedure. Starting from the initial CoT trace, we extract its chain and compute PNS. If the chain is sufficient, we perform necessity estimation for each node and prune the chain accordingly. The final optimized trace Sinal consists only of reasoning steps that are both sufficient to produce the correct answer and necessary to preserve logical coherence.
Rollout Strategies for Intervention Chain S (1) We use three strategies (details/prompts in Ap-
pendix B for generating semantically modified steps for S²); (1) Direct Rollout: base model generates replacement from preceding context. (2) Prompt-Based Rollout: structured prompts guide base model substitutions. (3) External Rollout: separate, stronger model generates replacements.
Base/rollout models are consistent for (1)-(2;; external rollouts use a more capable auxiliary.
4.2 PNS-Guided Reasoning Enhancement for In-Context Learning and Fine-Tuning
We use causally filtered CoT traces to improve LLMs under two paradigms: In-Context Learning (ICL) and Supervised Fine- TuninTTnng (SFT). In ICL, optimized CoT's are directly inserted into prompts to guide non-reasoning models. In SFT, we fine-tune ceasoning-capable models using 1,229 high- quality CoTs. Results for both settings are reported in $5.2
5
Experiments
Our experiments are structured around two core questions:
RQ1:
Can our method construct CoT datasets that enhance causal sufficiency and necessity? 5.2.1
RQ2: Can the causally optimized CoT data improve the performance of non-reasoning models via ICL, and further enhance caasolin--capable models through SFT? ($ 5.2.2
5.1
Experimental Setup
Datasets. We evaluate on diverse reasoning benchmarks to ensure robustness across domains and difficulty levels. For mathematical reasoning, we use: (1) GSM-8k [00, with grade-school problems; (2) MATH-500 255,, covering intermediate-level topics; and (3) AIME, with advanced competition problems up 2025 44 T1]. For commonsense reasoning, we use CommonsenseeA multiple-choice dataset requiring everyday inference.
53

---

Table Experimental results for RQI. Comparison of CoT reasoning performance before and after ENS-basee optimization across QWEN and DEEPSEEK variants.
GSM-SK
Commonsense@A
MATH-500
AIME
Method
Tokens
Steps
Acc.
Tokens
Steps
Acc.
Tokens
Steps
Avc.
Tokens
Steps
Acc.
Owen
Variant HTwQ-32BB-rrviiie
Qwea-2.5 2.5-228--11111
(728-instacc)
Prompt: Based
-133.9
113.8
-22.3
8.1
-95.8%
90.0%
109.2
-90.4
-33.0
3.7
-575.6%
69.7%
-177888
281.8
-15.5
9.2
82.6%
-86.5%
-511.9
531.3
-22.3
12.5
->26.7%
16.7%
Direct
113.8
8.1
90.0%
109.2
3.7
69.7%
281.8
9.2
82.6%
531.3
12.5
16.7%
-26.6
+200
-97.0%
-90.4
-1300
+74.5%
-169.44
-+5.1
+87.4%
-522.7
-+2.3
-22333
Extemal
356.4
158.9
23.9
93.3%
197.9%
474.2
1215.9
17.8
:7.4
83.2%
+88.0%
743.2
2200.7
50.3
+11.0
87.6%
194.3%
1719.4
11479.4
108.7
176.7
43.3%
356.7%
DoxpSock Variani (DocpSeck-RI & DespScck-V33
Prompt-Based
-29.6
137.3
->14
5.4
-997.3%
95.0%
191.0
-88..3%
83.0%
-163.4
387.6
16.2
-992.0%
85.915
-2195.44
2438.8
-+100.9
120.3
-+25.0%
25.0%
137.3
3.4
05.0%
191.0
6.2
83.01
387.6
16.2
85.9%
2438.8
120.3
25.0%
Direct
+29.2
-+13
-97.0%
-69..8
+2..5
+85.3%
-11112
+5..
+91.6%
-22662.6
-9969
+22.00
Exterall
1135.1
x3.2
99.0%%
-99.9%
368.2
1667.9
>3.1
87.1%
194.9%
828.8
22145
29.7
36.1
93.2%
>97.6%
3052.5
16339.3
157.8
>68.3
X92.6%
79.29
Evaluation Metrics. We assess (i) reasoning efficiency-measureed by token length and step length and (ii) accuracy, comprising PS(chain) and final-answer accuracy. Token length is com- puted via space-delimited tokenization; step length counts steps separated by double newline for consistency. Accuracy is the average fraction of correctly answered instances on the test set, i.c., Accuracy # correctly answered instances # instances in the
Baselines. For RQI, we compare unoptimized CoT traces from the base model with PNS- PNS-optimized versions (via $ 4..1 in terms of reasoning efficiency, accuracy, and average PNS. A representative
example is shown in Figure 3 with full results in Appendix G
For RQ2, ICL baselines include:(1) Standard, few-shot with original (often redundant) CoT's; (2) Fast-Solve, concise yet complete reasoning; (3) Reduction [19, shortcut conclusions; (4) CoD 667, minimal key phrases; and (5) Ours-ICL, few-shot with PNS-optimized, causally essential steps. Details and prompts are in Appendix C For SFT, we compare: Original, the base model; Noncausal, fine-tuned on raw CoTs; and Causal, fine-tuned the same CoTs after PNS-based pruning.
Models. For RQI, we use Qwen aeven-2.555 7..5572B-Instrus [55] as both the base and rollout model. In the External variant, QwQ-32B- 1-2B-revier ew [56] is used as base, with rollout unchanged. For DeepSeek, DeepSeek- V3 [16] serves as both base and rollout in standard settings, while the external variant as base and V3 as rollout. PNS evaluations share the same configuration.
uses DeepSeek-Ri
RQ2 ICL, experiments primarily use Qwen-2. 1ww-22-Inssruee with additional results from Qwen-2.5 S--B-Innsruct [55] Llama-3, 1-8B-Instruce [1) and DeepSeek-V3. SFT exper- iments fine-tune DeepSeek-R11 -ieek--RR--ssilll -eell--wen-1111 1.B DeepScaleR-1. -B. -B-rrvie [37] and
Phi-4-mini-reasee (66). Training details are in Appendix
5.2
Main Results
5.2.1
PNS Optimization on CoT Trajectories
We apply our method to CoT traces from Qwen and DeepSeek variants, As shown in Table I our PNS-based algorithm reduces both token and step lengths while improving accuracy f indicating effective removal of redundant reasoning.
We compare average PNS values before and after optimization across tasks (AIME and Common- senseQA) and models (Qwen-2 -..en-2..-7BB.. - DeepSeek-RR As shown in Figure 3 PNS values consistently increase after optimization, confirming that the retained steps are more causally sufficient and necessary. The figure illustrates results on 15 sampled questions per setting; more comprehensive results across larger test sets are provided in Appendix G
IIferrenee for reasoning models was performed using VLLM. The max-tokens is 16,384.
7

---

PNS Comparison AIME 2025 (Qwen-2.5- 728- Inntructttt
PNS Comparison
15 CommonsenseeAA Qs (Qwen-2.5 w.w-nn2.-72B.... 12B-nstt
Before PNS Optimication
After PNS Optimization
Before Optimizstion
Optimization
1.0
Averagee PNS (Steps)
Average PWS (Supp)
0.8
0.6
0.4
AUME 1
0.
AMME 1.3
AIME .
AIME, AiME 1.7
AIME, B. 13
II
AIME 0.3
AINE 1.1
AIME 2.32
com 139
0.0
CoM 65
coM 281
CoM 252
CON 23p
COM 111
COM. 366
COM 3:6
53
455
CoM 78
COM 288
cole 238
AIME
AIME,
AlNE
AiME
AINE
CoM
coM
cOM
CDM
(b)
PNS Comparison 15 AIME 2025 (DeepSeek-)) R1)
Comparison Commonsense@A Qs (DeepSeek-R1)
Before PNS Optimization
After PNS Optimization
Before PNS Optimization
After PNS Optimiza tion
Average PNS (Stops)
Average PNS (Socps)
1.0
0.8
0.8
0.6
0.4
0.0
AIME 0
AIME, 1.3
AIME
AIME
AIME 1.8
AIME 1.7
AIME H
AMEE
AIME it 13
ATME 112
AIME 1.2
AIME U.3
AIME 1.3
AIME 1.1
AIME 11.12
CoM 176
COM 65
com
CoM. 212
CON 111
COM. 109
cold .20
com 74
COM 239
CoM. 171
COM,
coM 172
COM. 126
COM 22
cold. .355
(c)
Figure 3: Average PNS values before and after optimization across different models and datasets. Each subfigure displays PNS improvements across 15 sampled problems: (a) Qwen-2 2. Diven22.5-72B-neee I---2---nst evaluated on AIME, (b) Qwen-2. (wxn-22.572B-Im 5-78---ns-stsc on CommonsenseQA, (c) DeepSeek-R1 evalu- ated on AIME, and (d) DeepSeek- R1 Commonsense PNS-optimization CoTs exhibit consis- tently higher PNS values, indicating an increased necessity for retained steps.
We conducted human quality evaluation of 50 CoTs: (S&N), and only 6% insufficient (NbI); see Appendix [oor details.
84% were judged both sufficient and necessary
These findings indicate that our optimized CoTs are not only more concise and accurate, but also demonstrate enhanced causal sufficiency and necessity. Notably, the average PNS per step increases after optimization, suggesting that the retained reasoning steps are more integral to producing correct answers- each step contributes more critically the final outcome than before.
5.2.2 Enhancing LLMs via ICL and SFT with Optimized CoT
We investigate enhancing LLM performance with optimized CoT data via in-context learning (ICL) and supervised fine-tuning (SFT).
In-Context Learning with Optimized CoT. Using optimized CoT traces for ICI. with non-fine- tuned L.Mss (Table Ours-ICI. balances reasoning efficiency and accuracy. Compared to Standard CoT, it consistently reduces token/step usage (often >50%) with minimal/no accuracy loss. For instance, on GSM-8k with DeepSeek-VV V3, Ours-ICL improves accuracy (97.65 to 99.9%) while cutting tokens by 67%; with Llama-3 --8--nnstruct on MATH-500, accuracy increases by 7.1 points (to 54.8%) with more concise reasoning. Ours-ICL also surpasses baselines like Fast- Solve and Reduction [19] accuracy with comparable or better efficiency. On GSM-8k with Quen-2.5 5--5-------- -Is-tust our method (99.5%, 65.3 tokens) outperforms Fast-Solve (91.7%, 72.8 tokens) and Reduction (84.7%, 114.1 tokens). Unlike aggressive pruning (c.g., CoD (61). Ours-ICL maintains substantially higher accuracy, especially complex tasks (e.g., M.TH.MTH-OO 96.2% vs. 55.6% with DeepSeek-V3).
Limitations for ICL. Despite its effectiveness with non-reasoning models, ICL using optimized CoT is sensitive to prompt/example selection. Its benefits diminish complex tasks (e.g., MATH- 500), and performance is constrained by the fixed parameters such models. contrast, SFT more impactful for ceasoning-capable models, allowing deeper integration of reasoning patterns.

---

Table 2: Experimental results for RQ2 (ICL on Reasonngn Models). Lower is better (4) for Tokens/Steps, higher is better (1) for Acc. Change rates (%) relative to the average of methods for that metrre/dattsstt/modeel are in parentheses. Cells significantly better than average are colored blue (deeper for greater improvement).
Method
OAmmoneense
GSM-8k
MATH-500
Tokens
Stepsl
Acc-T
Tokens]
Steps]
Acc-7
Tokens!
Steps]
Acc-1
DeepSeck-V2 V3
Stundard
177.5 (vauss))
5.2 (-511))
83.8%
157.3
7.4 cetuma
97.6%
598.6
26,7
93.2%
Fast- Solve
120.1 (azun))
(-278%)
82.0% (0.7%)
(4U0))
4.9 (oun))
(41291
2 (+-28)
(4220)
87.2% 1-239)
Reduction
19
10344 (111.29
3.6 52.75
83.1% (iles)
104, (9)
97.3%
(-289))
176.3 140
22.0 0o4079
91.6% (-830e)
OoD (67
19.3 299299
2.0 .45.5444
80. 7% 123991
28.7 046-991
1.M 548.999
84.0%
31.2 (M
2.1 185,290
55.6% 634.99
Ours-ICI
44.7 (5) 494
2.7 (33.00
83.6%
52.2
4.3 (.009
99.9%
(-5
136.7 ( 16.9%
6.4 45g
96.2%
(613.899)
Qwen-
Standard
(e43059)
(-2787)
(4.35)
(-478%)
(4400))
281.8 (451.99)
9.2 (ettamm
84.0%
(416,99)
Fust Solve
59.7 121.19
67.0%% 645.990
72.8 63.3)
2.7 :0005))
91.7% (-2290
192.4 c5875)
7,3 (4144)
69.8%
(-25%)
Reducsion
1+
116.6 cocumm
3.5 602779
84.9% (o:2
114.1 (94929)
4.8 69.19
84.7% (04490
233.3 1423
9.4 (edtunee
72.4% (40,/00
OtD 667
14.4
2.1
82.3%
18.8 035
1.1 200990
78.
23.0
1,2 arlo
52.05%
Ours-ICI.
81.6
5.3
99.5%
196-9
81.2%
Standard CoT
209.8 (wtsu0)))
7.2 (-40.1))
70.4% (4.5%)
149.7 (-4817)
7.2 60322903
85.1% catoos:
263.5 (443,79)
9.7 cestans
71.0% (+12399)
SulteSolvve
120. (6399))
5.0 (:22m))
76.5% (41,00
108.4 (7799)
5.5 (4199))
83.6% 611220
200 6 (HlR8))
8.4 (4379)
64.6% (-2.19)
Redlusion
19
178.5
6.8
74.44
131.1
6.4 11122t
84.3%
232.2 142h
9.1
71.8%
16.8
Ours-ICI.
99.1 733
77.6% (43.19)
83 11:19
(411
174.7 (578)
c499
72.6%
(415/99)
Llama-3. 1-88- 1-88--nnstruct
Standard CoT
169.5 (55,35)
7.3 (113.9)
72.2% (:229)
182.8 (-2229)
7.9 61290
79.2% (4.30
741.3 (43922%
46.0 c22aar
47.6% (-6.79)
Solve
140.7
7,9
69.0%
170.6
9.5
72.0%
153.1 6333)
30.9 Lau
46.4 5%
Reduction
19
129.1
46.2%
CoD
60.6 (33
(178)
790
69.25 (397
312.5
29.2; (11.8))
28.05% (i)
Ours-ICI.
120.3 (S15;
7.1 (41278)
72.1% (63.7))
128.3 (976))
8.1 (41.35)
93.1% (417/48)
365.9 :23.45
24.2 (36/55)
54,8% (4229955
Supervised TuninTuning with Optimized COT. We fine-tune reasoning models on 1,229 PNS-
selected CoT traces from MATH (25), MMLU (24, ZebraLoggicBench (34. CommonsenseeQA (33
and AIME (pre-2024) 44). All traces were manually checked for causal sufficiency, necessity, concision, and quality; CoTs selected by our method were not edited-only verified. Table demonstrates Causal-CoT's consistent outperformance over baselines. On CommonsenseQAA, it improves accuracy to 47.2% (from 37.6% on on DeepScaleR-1. (et-ecaaee--1..--Peeeeee- (B-ereeeee)) while halving reasoning steps. On MATH-500, it achieves 78.2% accuracy (33.1 steps) versus the Original model's 76.4% (77.2 steps). Even on difficult tasks like AIME25 (low absolute performance Causal-CoT significantly cuts reasoning length (c.g.. DeepSeek-R1- Ri-Distiillll-- -iee-----we--n-- 5B: 212.4 to 95.4 tokens). The Causal variant also matches/ex- ceeds Noncausal fine-tuning with substantially fewer steps/tokens. For GSM-8k, it reaches 86.2% accuracy (11.6 steps), while Noncausal needs steps for 86. 1% (DeepScaleR-1 1errscallR--1.-PPeeei..... PB-Priieew).
and from 41.3%
Implications of SFT Results. SFT on PNS-selected traces yields consistent gains in accuracy and reasoning efficiency, even with small Causal-CoTT datasets, confirming the high supervision value of enforcing causal sufficiency and necessity. The token overhead from PNS filtering is a one-time curation cost; after fine-tuning, inference becomes cheaper because the model generates concise, accurate CoTs without stepwise rollouts or post-hoc pruning. The strength of our approach lies in reshaping the training distribution toward high-PNS evidence, enabling the model to internalize causally meaningful, non-redundant reasoning patterns. At test time, this manifests as streamlined, interpretable CoTs that improve both efficiency and reliability.
6 Conclusion
This work successfully incorporates PNS into CoT reasoning. The developed method systematically prunes unnecessary reasoning steps, leading to significant improvements in reasoning efficiency, while maintaining or even enhancing the accuracy of the outcomes. The effectiveness of the method has been verified across both in-context learning and supervised fine-tuing scenarios.
Limitation and Future Work. Limitations include potential performance decreases on highly complex tasks. Key challenges involve selecting optimal pruning thresholds, managing PNS esti-
Inference performed via VLLM; The max-tokens is 16.384.

---

Table 3: Experimental results for RQ2 (SFT on Reasoning Models). For Noncausal and Causal methods, change rates (%) are reported relative to the "Original" for each model/dataset. Cells with notable improvements over the original are highlighted.
Commonsense@A
GSM-8K
MATH-500
AIME25
Method
Tokens|
Steps,
Acc.f
Tokens
Steps]
Acc.t
Tokens]
Steps]
Acc-t
Tokens
Steps]
Acc-I
DeepSeek-R RI- Seeuill-Qwen--ae
Original
751.3
21.4
37.6%
332.1
14.4
77.9%
[441.8
77.2
76.4%
4002.3
212.4
23.3%
(huselin))
(buseline)
(huseline)
(baretine)
(Boselino)
(buecline)
(buselin))
(bssclin))
(baseline)
(Baseline)
(buseline)
(busctive)
1271.3
27.9
43.2%
621.4
16.5
83.4%
1456.6
54.7
81.6%
3796.3
116.4
20.0%
Noncausal
(+69.2%)
(-30,4%)
(+14.9%
(+87.195)
(+14.6%)
(+7.195)
(+1,99]
(29.1%)
(+6.8%)
(5.1%)
[-55 15.2%)
(14.2%)
Causal (Ours)
740.0
10.3
47.2%
12.8
84.2%
33.1
78.2%
2948.3
95.4
23.3%
(-1.%%
(-1.99))
(+25.9%)
(-1.9%)
(.1.15)
(+8119)
(-3.77%)
(-57.19)
(+2.4%
(-26.99)
(-55 19)
(0.8)
DeepScaleR -.Se-er
646.7
22.1
41.3%
716.3
27.5
87.5%
1325,0
77.4
84.6%
4897.2
356.0
20.0%
Original
(bostinn)
(besriins)
(hoaline)
(harline)
(hovlinr)
(hancliar)
(beseii
(bvarlin))
(bovrline)
(huselirr)
(huoeliee)
611.5
21.7
40.8%
604.88
15.0
86.1%
1445.5
$2.4
82.8%
5709.8
243.2
16.7%
Noncausal
(3.4%)
(1,88)
(1.2%)
15.6%)
(5.55%)
(165)
(+9,198)
(3.39%
(2.1%%
(+16.615)
(31.7%
(16,5%)
601.2
11.2
47.2%
86.2%
10414
36.3
91.7%
2015.4
53.6
20.0%
Causal (Ours)
(7.0%)
(49.2%)
(+14,3%)
(44.9%)
(5785%)
(155%
(214%)
(3.11%)
(+3,6%)
(58.8%)
(85.0%)
(0.0%)
Phi-4-mini-rcasoning
Original
935.7
18.6
72.4%
783.3
18.6
92.6%
1743,5
62.1
85.8%
6544.0
274.3
30.0%
(busriin)
(besriins)
(boalinns
(huneine)
(hurline)
(basviin))
(basline)
(boselin))
(hunelirr
(huseline)
949.6
14.9
66.4%
566.9
14.3
89.8%
2042.8
65.8
59.8%
8297.8
359.3
23.3%
Noncausal
(+1.5%)
(-19.9%)
(-..5%)
(-27.6%)
(-23.1%)
(.3.0%)
(-17.28)
(-5.9%)
(-30,5%)
(426.89)
(+31.09)
(-22.39%
Causal (Ours)
920.3
14.1
92.4%
1031.
28.7
4140.00
(-18%)
(-24.2%)
(+0.7%
(33.0%)
(-2475%)
(-0.2%
(-40.9%)
(-3.8%)
(+1.0%)
(56)
(-99211)
(0.0%)
mation costs, and ensuring counterfactual generation quality. Future work will aim address these limitations, focusing on improving the causal fidelity and overall performance of LLM reasoning.
Acknowledgments
X. Yu was supported in part by the National Natural Science Foundation of China (Nos. 62472306. 62441221, and 62206116), Tianjin University's s 2024 Special Project on Disciplinary Development (No. XKIS-2024-5-9), the Tianjin University Talent Innovation Reward Program for Literature & Science Graduate Students (No. C1-2022-010), and the Henan Province Key Research and Development Program (No. 251111210500). H. Li was supported in part by National Natural Science Foundation of China (No. 623B2002).
Author Contributions
This work completed through the joint efforts of all authors. X.Y and Z.W were responsible for the main experiment design, model implementation, and mamuscript writing. M.Y contribute to idea formulate, experiment design, theoretical analysis and manuscript writing. L.Y contribute to experimental design. H.L contribute to idea formulate and manuscript writing. The other authors contributed to regular discussion.
References
[1] AI@ Meta. Llama model card, 2024. URL httppsss//gtthbb.com/aaaaaaaaaaa blob/main/MODEL. CARD .ddl
[2] Ameen Ali, Lior Wolf, and Ivan Titov. Mitigating copy bias in in-context learning through neuron pruning. arXiv preprint arXiv:2410.01288, 2024.
[3] Dean Allemang and Juan Sequeda. Increasing the Ilm accuracy for question answering: Ontolo- gies to the rescue! arXiv preprint arXiv:2405.11700 2024.
10

---

[4] Swagata Ashwani, Kshiteesh Hegde, Nishith Reddy Mannuru, Dushyant Singh Sengar, Mayank Jindal. Krishna Chaitanya Rao Kathala, Dishant Banga, Vinija Jain, and Aman Chadha. Cause and effect: can large language models truly understand causality? In Proceedings of the AAAI Symposium Series, volume 4, pages 2-9, 2024.
[5] Sander Beckers. Causal explanations and xai. In Conference on causal learning and reasoning, pages 90-109. PMLR, 2022.
[6] Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17682- 11882-16690, 2024.
[7] Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu. Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of ol-like Ilms. arXiv preprint arXiv:2412.21 187, 2024.
[8] Fengxiang Cheng, Haoxuan Li, Fenrong Liu, Robert Van Rooij, Kun Zhang, and Zhouchen Lin. Empowering Ilms with logical reasoning: A comprehensive survey. In Proceedings the FourthyFourth International Joint Conference on Artificial Intelligence, 2025.
[9] Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations. arXiv preprint arXiv:2412.13171 2024.
[10] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint 0rXiv22110.14100 2021.
[11] OpenCompass Contributors. Aime 2025 dataset. https://hughe haap:::://uggingaaa.ea. co/datasets/ opencooppass/TTN22055 2025. Accessed: 2025-05-14.
[12] Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang, Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, al. The danger of overthinking: Examining the reasoning-action dilemma in agentic tasks. arXiv preprint arXiv:2502.08233 2025.
[13] Yingqian Cui, Pengfei He, Jingying Zeng, Hui Liu, Xianfeng Tang, Zhenwei Dai, Yan Han, Chen Luo, Jing Huang, Zhen Li, al. Stepwise perplexity-guice refinement for efficient chain-of-thhuggee reasoning in large language models. arXiv preprint arXiv:2502.13266 2025.
[14] Adnan Darwiche and Auguste Hirth. On the (complete) reasons behind decisions. Journal of Logic, Language and Information, 32(1):63-88) 88, 2023.
[15] C Nicolò De Sabbata, Theodore R Sumers, and Thomas L Griffiths. Rational metareasoning for large language models. arXiv preprint arXiv:2410.05563, 2024.
[16] DeepSeek-AL. Deepseek-v3 technical report, 2024. URL fttps::/aaxiv.orgge. at////b//22
19437
[17] DeepSeck-AL. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025. Utthttts://aaxi org/ wwtg/b////////// 211.215111110
[18] Shizhe Diao, Pengcheng Wang, Yong Lin, Rui Pan, Xiang Liu, and Tong Zhang. Active prompting with chain-of-thought for large language models. arXiv preprint arXiv:2302.12246. 2023.
[19] Mengru Ding, Hanmeng Liu, Zhizhang Fu, Jian Song, Wenbo Xie, and Yue Zhang. Break the chain: Large language models can be shorteut reasoners. arXiv preprint 24X.v:2406.06580. 2024.
[20] Xidong Feng Yicheng Luo, Ziyan Wang, Hongrui Tang, Mengyue Yang, Kun Shao, David Mguni, Yali Du, and Wang. Chessgpt: Bridging policy learning and language modeling. Advances in Neural Information Processing Systems, 36:7216-7262. 2023.

---

[21] Sainyam Galhotra, Romila Pradhan, and Babak Salimi. Explaining black- box algorithms using probabilistic contrastive counterfactuals. In Proceedings of the 2021 International Conference on Management of Data, pages 577-590, 2021.
[22] Tingxu Han, Chunrong Fang, Shiyu Zhao, Shiqing Ma, Zhenyu Chen, and Zhenting Wang. Token-budget-awarre Ilm reasoning. arXiv preprint arXiv:2412. 18547, 2024.
[23] Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769, 2024.
[24] Dan Hendrycks. Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021.
[25] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprini arXiv:2203.03880 2021.
[26] Alihan Hüyük, Xinnuo Xu, Jacqueline Maasch, Aditya V Nori, and Javier González. Reasoning elicitation language models via counterfactual feedback. arXiv preprint arXiv:2410.03767, 2024.
[27] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card, arXiv preprint arXiv:2412.16726, 2024.
[28] Bowen Jin, Chulin Xie, Jiawei Zhang, Kashob Kumar Roy, Yu Zhang, Zheng Li, Ruirui Li, Xianfeng Tang, Suhang Wang, Yu Meng, et al. Graph chain-of-thoughh Augmenting large language models by reasoning on graphs. arXiv preprint arXiv:2404.07103, 2024.
[29] Jiarui Jin, Yuwei Wu, Haoxuan Li, Xiaoting He, Weinan Zhang, Yiming Yang, Yong Yu, Jun Wang, and Mengyue Yang. Large language models are demonstration pre-selectors for themselves. In Forty-second International Conference on Machine Learning, 2025.
[30] Mingyu Jin, Qinkai Yu, Dong Shu, Haiyan Zhao, Wenyue Hua, Yanda Meng, Yongfeng Zhang, and Mengnan Du. The impact of reasoning step length on large language models. arXiv preprint arXiv:2401.0m 049925, 2024.
[31] Ziqi Jin and Wei Lu. helf-harmonized chain of thought. arXiv preprint arXiv:2409.04057 2024.
[32] Yu Kang, Xianghui Sun, Liangyu Chen, and Wei Zou. C3o::
Generating shorter chain-of- thought without compromising effectiveness. arXiv preprint arXiv: 1664, 2024.
[33] Leo Li, Ye Luo, and Tingyou Pan. Openai-ol ab testing: Does the ol model really do good reasoning in math problem solving? arXiv preprint arXiv:241 1.0-29411.0699, 2024.
[34] Bill Yuchen Lin, Ronan Le Bras, Kyle Richardson, Ashish Sabharwal, Radha Poovendran, Peter Clark, and Yejin Choi. Zebralogic: On the scaling limits of 1lms for logical reasoning. arXiv preprint 200.0:2500200000 2025.
[35] Yule Liu, Jingyi Zheng, Zhen Sun. Zifan Peng, Wenhan Dong, Zeyang Sha, Shiwen Cui, Weiqiang Wang, and Xinlei He. Thought manipulation: External thought can be efficient for large reasoning models. arXiv preprint arXiv:2504.13626 2025.
[36] Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length-harmoniiing fine-tuning for ol-like reasoning pruning. arXiv preprint arXiv:2501. /2570, 2025.
[37] Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing ol ol-preview with a 1.5b model by scaling rl, 2025. Notion Blog.
12

---

[38] Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking. arXiv preprint FrXiv2504.098555 2025.
[39] Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. Cot-valve: Length-compressible chain-of-thought tuning. arXiv preprint (rXiv:2502.09601 2025.
[40] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye. Yiming Yang, al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534-46594, 2023.
[41] Ramaravind K Mothilal, Amit Sharma, and Chenhao Tan. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pages 607-617. 2020.
[42] Tergel Munkhbat, Namgyu Ho, Seohyun Kim, Yongjin Yang, Yujin Kim, and Se- Young Yun. Self-training elicits concise reasoning in large language models. arXiv preprint urXiv:2502.20122, 2025.
[43] OpenAL. Learning to reason with llms, 2025. URL http::////peaaa.ccommaoooaaee/ learning-ton weason-withh-lmm.
[44] OpenAI. :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camey, Alex Iftimic, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, et al. Openai ol system card, 2024. URL /ttpppps//aaxx.ooorr///////22 16720
[45] Soumyasundar Pal, Didier Chételat, Yingxue Zhang, and Mark Coates. Hint marginalization for improved reasoning in large language models. arXiv preprint arXiv:2412.13292, 2024.
[46] Judea Pearl. Causality. Cambridge university press, 2009.
[47] Haritz Puerto, Tilek Chubakov, Xiaodan Zhu, Harish Tayyar Madabushi, and Iryna Gurevych. Fine-tuning with divergent chains of thought boosts reasoning through self-correction in lan- guage models. OpenReview, 2024.
[48] Team Qwen. Qwq: Reflect deeply on the boundaries of the unknown. htttps://qwenlm github. ht/bli////////o qwq-qwq--o previewwwww 2024. Accessed: March 2025.
[49] Adi Simhi, Itay Itzhak, Fazl Barez, Gabriel Stanovsky, and Yonatan Belinkov. Trust me, i'm wrong: High-certainty hallucinations in Ilms, arXiv preprint arXiv:2502.12664 2025.
[50] Zayne Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett. To cot or not to cot? chain- of-thought helps mainly on math and symbolic reasoning. arXiv preprint 17X.1.2409.12183, 2024.
[51] Alessandro Stolfo, Ben WU, Wes Gurnee, Yonatan Belinkov, Xingyi Song, Mrinmaya Sachan, and Neel Nanda. Confidence regulation neurons in language models. Advances in Neural Information Processing Systems, 37: 15019999441, 2024.
[52] Hanshi Sun, Momin Haider, Ruiqi Zhang, Huitao Yang, Jiahao Qiu, Ming Yin, Mengdi Wang, Peter Bartlett, and Andrea Zanette. Fast best-of-n decoding via speculative rejection. arXiv preprint arXiv:2410.20299, 2024.
[53] Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsenseqa:
A question answering challenge targeting commonsense knowledge, 2019. URL https
// /aaxxiv.org aa////ass//1./// 00937
[54] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with llms. arXiv preprint UrXiv:2501.12599 2025.
[55] Qwen Team. Qwen2.5: A party of foundation models, September 2024. URL https:// qwenlm. github. i htt/b/llgggg////eee 5/
13

---

[56] Qwen Team. Qwq: Reflect deeply on the boundaries of the unknown, November 2024. URL https://qwennn: www.nggiih....... 1.t..ioo/lll. httploog/www/g/////o//oooogoggge 32b -
[57] Jean-Francois Ton, Muhammad Faaiz Taufiq, and Yang Liu. Understanding chain-of-thought in Ilms through information theory. arXiv preprint 1.XXx:24111.19888 2024.
[58] Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language models don't always say what they think: Unfaithful explanations in thain-of-thoughh prompting. Advances in Neural Information Processing Systems, 36:74952--4990 2023.
[59] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.1117/, 2022.
[60] David S Watson, Limor Gultchin, Ankur Taly, and Luciano Floridi. Local explanations via necessity and sufficiency: Unifying theory and practice. In Uncertainty in Artificial Intelligence, pages 1382-1392. PMLR, 2021.
[61] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of of-thought prompting elicits reasoning in large language models. Advances neural information processing systems, 11:24824-24833 2022.
[62] Siwei Wu, Zhongyuan Peng, Xinrun Du, Tuney Zheng, Minghao Liu, Jialong Wu, Jiachen Ma, Yizhi Li, Jian Yang, Wangchunshu Zhou, et al. A comparative study reasoning patterns of openai's ol model. arXiv preprint arXiv:2410. 13639, 2024.
[63] Tong Wu, Chong Xiang, Jiachen Wang. and Prateek Mittal. Effectively controlling reasoning models through thinking intervention. arXiv preprint 2:XIv:250..24370, 2025.
[64] Violet Xiang, Charlie Snell, Kanishk Gandhi, Alon Albalak, Anikait Singh, Chase Blagden, Duy Phung, Rafael Rafailov, Nathan Lile, Dakota Mahan, et al. Towards system 2 reasoning in lims: Learning how think with meta chain-of ohain-of-though. arXiv preprint TrXiv:2501.04683 2025.
[65] Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-ol: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10440 2024.
[66] Haoran Xu, Baolin Peng, Hany Awadalla, Dongdong Chen, Yen-Chun Chen, Mei Gao, Young Jin Kim, Yunsheng Li, Liliang Ren, Yelong Shen, Shuohang Wang, Weijian Xu, Jianfeng Gao, and Weizhu Chen. reiisoonnn-- Exploring the limits of small reasoning language models in math, 2025. bttpsss/s//ariiiit orgp///vvor///oooooo abs//bbs 2504. 21233
[67] Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less. arXiv preprint 2502 18600, 2025.
[68] Xiao Xue, Xiangning Yu, Deyu Zhou, Xiao Wang, Chongke Bi, Shufang Wang, and Fei- Yue Wang. Computational experiments for complex social systems: Integrated design of experiment system. IEÉE/CAA Journal of Automatica Sinica, 11(5):1155-1188) 2024.
[69] Hao Yang, Qianghua Zhao, and Lei Li. Chain-of-thought in large language models: Decoding, projection, and activation. arXiv preprint arXiv:2412.03944, 2024.
[70] Mengyue Yang, Quanyu Dai, Zhenhua Dong, Xu Chen, Xiuqiang He, and Jun Wang. Top-n recommendation with counterfactual user preference simulation. CIKM '21, page 2342-2351, 2021. doi: 10.1 10.114/5/459667.4482225
[71] Mengyue Yang, Furui Liu, Zhitang Chen, Xinwei Shen, Jianye Hao, and Jun Wang. Causalvae: Disentangled representation learning via neural structural causal models. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, June 2021.
[72] Mengyue Yang, Zhen Fang, Yonggang Zhang, Yali Du, Furui Liu, Jean-Francois Ton, Jianhong Wang, and Jun Wang. Invariant learning via probability of sufficient and necessary causes. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, pages 79832-79857, 2023.
14

---

[73] Wen Yang, Kai Fan, and Minpeng Liao. Markov chain of thought for efficient mathematical reasoning. arXiv preprint arXiv:2410.176:3 2024.
[74] Wenkai Yang, Shuming Ma, Yankai Lin, and Furu Wei. Towards thinking-optimal scaling of test-time compute for IIm reasoning. arXiv preprint arXiv:2502. 18080, 2025.
[75] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations, 2022.
[76] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in Neural Information Processing Systems, 36, 2024.
[77] Xiangning Yu, Xiao Xue, Deyu Zhou, Li Fang, and Zhiyong Feng. Beyond traditional metries: The power of value entropy in multidimensional evaluation of the service ecosystem. In 2024 IEEE International Conference on Web Services (ICWS), pages 61 1-11.2.. IEEE, 2024.
[78] Murong Yue, Wenlin Yao, Haitao Mi, Dian Yu, Ziyu Yao, and Dong Yu. Dots: Learning to reason dynamically in Ilms via optimal reasoning trajectories search. arXiv preprint arXiv:24/0.0388 2024.
NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide a short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked While "[Yes]" is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] prrovided proper justification is given (c.g., "error bars are not reported because it would be too computationally expensive" " or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
the checklist one of the factors in their evaluation.
"[No] " or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, so please just use your best judgment and write justification elaborate, All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Check- list".
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
15

---

1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims presented in the abstract and introduction accurately reflect the scope and contributions of our work. Specifically, claim that our proposed PNS-based CoT optimization improves reasoning efficiency and accuracy, which is supported by both theoretical formulation and empirical results discussed in and
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We explicitly discuss limitations in $ 6 noting reduced effectiveness high- complexity tasks like AIME. This is attributed to the use of fixed pruning threshold o, which may be suboptimal across diverse reasoning domains.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
16

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We provide a formal definition the Probability Necessity and Sufficiency (PNS) and clearly state the assumptions required for its estimation within the CoT framework. Key theoretical formulations and pruning criteria are detailed in $ with all assumptions explicitly stated. Formal justifications and derivations are included in the appendix to support our main results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
All experimental settings-includding datasets, model variants, training config- urations, evaluation metrics, and optimization parameters- are described in detail in $ $ and Appendix D A public GitHub repository including code and processed data will be released upon acceptance to further support reproducibility. Guidelines:
Justification:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
17

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We use publicly available datasets and will release code and processed data upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cct
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips /pipss.ccppbllc//uiidee wwtcc//gwies/eeeeeeeeeee www.oooaiiiiiiiinnnooooooooeoeee
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Ye]]
Justification: We provide full experimental details in $ and the appendix, including dataset splits, prompt formatting, model variants, inference settings (e.g., max tokens, temperature).
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Ys]]
Justification: We clearly specify our datasets and rollout methodology; all parameters are publicly disclosed, and results are obtained through repeated testing across multiple datasets.
Guidelines:
18

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide detailed descriptions of the compute resources in Appendix ??.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
Our research adheres fully to the NeurlPS Code of Ethics. All datasets used social impacts in our discussion.
publicly available, no private or sensitive data is involved, and we have considered potential
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer:[NA]
19

---

This work is fundamental research on reasoning optimization and does not directly involve applications with immediate societal impact.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (c.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not involve the release of new pretrained models scraped datasets that pose a high risk of misuse. We evaluate existing public models and operate on curated, publicly available datasets with no sensitive content.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All assets used in this work, including datasets, code, and models, publicly available. We have properly credited the original creators and ensured that all licenses and terms of use respected.
Guidelines:
20

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pannessippsssii thcode. can/datasess has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
We release accompanying code. All assets are documented with details on format, usage, and limitations, and are provided via anonymized GitHub repository during review, with public release planned upon publication.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
21

---

Answer:
[NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: LLMs are used in this paper solely for text polishing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy http::/://aaarriieoec/o/o/f/eeee for what should or should not be described.
22

---

Appendix
Appendix Table of Contents
A.
Theoretical Analysis
B.
Intervention Prompts
C.
In-Context Learning Prompts
D.
Supervised Fine- Fine-Tuning (SFT) Hyperparameter Settings
E.
In-Context Learning: Case study
F.
Full Version of Related Work
G.
Additional PNS Comparison Results
H.
Computational Complexity Analysis
I.
Qualitative Analysis of Whether the Final Reasoning Is Sufficient and Necessary
Validator Accuracy and Robustness
K.
Supplementary Experimental Results
A Theoretical Analysis
A.1 Exogeneity and Monotonicity Conditions
For the calculation of the Probability of Necessary and Sufficient (PNS) conditions, is essential that the exogeneity and monotonicity conditions satisfied.
Definition (CoT Exogeneity) When generating each reasoning step St. the generation of the current step depends only the previous reasoning steps Set and the question q, and not any external variables. Formally:
P(st) do(set), q) = P(se|s q)
(A.1)
Where St is the current reasoning step, Set denotes all previous reasoning steps, and q is the input question. According to the definition of exogeneity, the intervention probability can be evaluated by the conditional probability.
Assumption (Monotonicity for CoT) Inspired by the monotonicity assumption causal in- ference [46], define monotonicity for Chain-of-Thought (CoT) reasoning as follows. Let = .S,)...,su) be a reasoning chain leading to the correct answer A = and let S = denote modified reasoning chain where only step t is altered
($1 ,, to an incorrect step Be and subsequent steps s' are rolled out. CoT reasoning satisfies monotonicity and only if. for every reasoning step I. and for every possible alteration the following holds:
P (AA y, Ag = =y = 0.
This condition states that altering a reasoning step to an incorrect version cannot result in correcting a previously incorrect final answer. Equivalently, whenever the modified chain S yields correct answer, the original unaltered chain S must also yield a correct answer, ensuring monotonic progression toward correctness.
A.2 Identifiability of PNS in CoT Under Monotonicity Assumption
Lemma (Identifiability of PNS under downstream-adaptinn reasoning) Assume that the (CoT) reasoning process satisfies both the Exogeneity (Definition A.7 and Monotonicity (Assumption 7 assumptions. Let the correct reasoning chain be denoted by St, (s,,,,t, Sx), and let an alternative corrupted step 5, induce a modified future reasoning sequence st> resulting in the altered chain S' = (S21,3,, s,,s's)). Then the Probability of Necessary and Sufficient Cause (PNS) for the reasoning chain is identifiable and satisfies:
PNS(S, st) P(As y) P(As: = P(A = do(S)) P(A y do(S"))
Note: do(S') is equivalent do(s< St. st).
23

---

We begin with the definition of PNS as a counterfactual joint:
PNS(S, 8,) P (As y, As y)
Using the fact that As y implies either As' y or As y, rewrite:
P(As y) = P(As y, As y) + P(As y, As y))
Thus:
PNS(S, = P(As y) - P(As As y))
Under the Monotonicity assumption (Assumption D] As = As y. This implies that the event (As = y and As = y) is equivalent to the event (As' = y). Therefore:
P(As y, As =) = P(As' y))
Hence:
PNS(S, 5.) P(As = y) P(As y).
Finally, under the Exogeneity assumption (Definition A. these counterfactual probabilities can be identified with interventional probabilities:
PNS(S, 5,) P(A y do(S)) P(A do(S))).
A.3
Identifiability of PNS without Monotonicity Assumption
Lemma (Identifiability of PNS under P(A = y do(()) = without Monotonicity) Assume:
Under the perfect intervention of the correct CoT chain S (sects St, 8yt), the model always produces the correct answer:
P(A do(())
1.
2. Replacing step with an incorrect step S, and allowing arbitrary rollout continuation s'y yields the intervened chain S' ($21, 5,, s'x).
3. We do not assume Monotonicity (Assumption [ i.e., we make no assumption that P P(Aao(S) + y, A y) O. (Note: original text had Adols)) which seems less precise here than Ado(S') for the full chain).
Then the counterfactual joint defining PNS,
PNS(S, 5.) P P(Ado(S) y, A Fv),
is identifiable and simplifies to:
PNS(S, = P(A = y
Start from the definition:
PNS(S, 5.) = P(Ado(S) = y, Adoisy + x))
Apply the law of total probability the event A y:
P(Ato(s) y) P(Ado(s) Ado(s)) Fy) P (Ado(S)) =, Ado(s) = =y). Since P(A y do(S)) by assumption 1, the left side equals 1, so
= PNS(S s.) P(Ado(s) y. Ado(s) =))
Under Exogeneity (Definition A.1 implying hidden confounding between the choice of interven- tion and its outcome), observing the original chain does not influence the outcome of intervening with S'. Thus:
((A)) y, Ado(s) = y) ((A y) P(A = y do(()))
Substitute back obtain
PNS(S, 5,) P(A y do(S')).
and therefore
PNS(S, = P(A = d
24

---

A.4 Equivalence of Perfect Intervention and Full Sufficiency
Theorem (Equivalence of Perfect Intervention and Full Sufficiency) Under the standard Exo- geneity assumption (Definition and assuming there exists at least one alternative chain S (which could be the original chain before applying the correct steps S, or any other relevant baseline) such that it has a nonzero probability of leading to failure, Le., P(A A y: S. q) > 0, the following two statements are equivalent:
1. P(A y do(S), q) = (Perfect Intervention with S guarantees y).
2. PS(S, ②) where PS(S, P(Ado(s) A y, S, q) (Full Sufficiency).
(+) Assume P do(S). q) This means the intervention do(S) guarantees A in all worlds compatible with q. Consider the definition of PS(S, q):
PS(S, q) P(Aao(s) = y A yy q).
Given the condition A * S, we evaluate the probability of Ado(s) = Since P(A y q) including in those specific cireumstances where + y would have occurred with S. Therefore, (Aac(S)) y A + S, q) 1, so PS(S, q) =.
it follows that Ad(Ss) y holds universally under do(S),
(+) Assume PS(S. q) By definition, this means:
P(Ado(S) y A S, q) =
This implies that for any world compatible with q where A + would occur with S, applying (o(S) results in A = y. We want to show P(A = y do(S). = 1, which is P(Aao(S) y q) 1. Consider the outcome Ado(s) given q. The intervention do(S) sets the chain of thought to S and determines the outcome A. This outcome (s) is determined solely by and q (due to exogeneity of with respect other factors once do(S) is applied). If PS(S, q) 1, means do(S) corrects all instances where 5 would lead failure. What about instances where S might lead success (A y)? Since S is the "correct" chain of thought designed to produce y, the intervention do(S) is assumed to robustly produce If it produces when would have failed, and it (by its nature as correct CoT) produces y when S might have succeeded, then Adoss) holds across all situations defined by and any alternative S. Therefore, P P(Ado(S) y q) = 1.
B Intervention Prompts
System Message for LLM Intervention
You are helpful assistant. Continue solving the problem using mathematical expressions only, without repeating previous steps. Provide final answer once, directly linked to the preceding reasoning, without additional summaries explanations. Avoid using summarizing words such as 'so' or "thus,' and refrain from repeating the final result when the calculation already clear. Don't It say something like "Let's continue with the previous reasoning" or other nonsense, just output the following reasoning directly.
Direct and External Intervention Prompt
Question:
[query)
Current reasoning steps:
(context_steps)
Prompt-Based Intervention Prompt
25

---

Ensure the next output node does not match the meaning of: (current_str step)
Avoid repeating the final result directly when the calculation already clear.
Question:
(query)
Current reasoning steps:
[context_stepp
C In-Context Learning Prompts
ICL Baselines.
The ICL baselines consist of the following variants:
Standard:
This baseline uses verbose Chain-of- Shaiinof-Thhuugh (CoT) exemplars that include all intermediate reasoning steps, regardless of redundancy. It reflects the default strategy often employed in prompting LLMs for step-by-step reasoning.
2. Fast-Solve: This bascline encourages the model to produce concise reasoning chains that contain only the minimal steps necessary to reach the correct answer, avoiding verbose or redundant elaboration.
3. Reduction 19:
This method emphasizes rapid completion by prompting models to directly output shortcut solutions, often skipping step-by-step logical progression. It reflects minimalistic strategy that favors brevity over transparency.
4. Chain-of-Draft (CoD) 67 This variant uses prompts composed of minimally informa- tive intermediate phrases-enowe enough scaffold the reasoning process but without detailed Eibboration-simlaatin a rough- nough-draft-style reasoning chain.
5. Ours-ICL: Our method, which leverages traces optimized using the Probability of Necessity and Sufficiency (PNS), presents only causally essential reasoning steps. These exemplars pruned to retain only those steps that significantly contribute to correct outcomes, ensuring both efficiency and fidelity in reasoning.
System Message (Common for All Prompts)
You are helpful assistant who is good at reasoning. Whenever doing multistep reasoning. please use two newline characters to split multiple steps (Anin).
Ours-ICL Prompt (Sufficient and Necessary Reasoning)
User Message:
Instructions
When solving the following questions, your reasoning should: Be Accurate: Ensure your chain of thought leads to the correct answer without skipping any necessary logical steps. Be Efficient: Avoid unnecessary or redundant steps. Each step should be necessary progress toward the solution.
Only include steps that are both sufficient to reach the correct answer and necessary to avoid gaps or confusion. If step can be removed without affecting correctness, remove
Aim for Sufficient and Necessary Reasoning:
Notice the Pattern: In the following examples, compare the original, verbose solution with the optimized solution. Learn to identify and eliminate redundant reasoning steps while preserving logical soundness.
Example 1: (example1)
Example 2: [example2)
26

---

Example 3: (example3)
Now Solve This:
Question: [question]
Your Simplified and Optimized Answer:
Fast-Solve Prompt
User Message:
You are a math assistant that solves problems step by step. Please reason in clear and structured manner, but keep your explanation as concise as possible. Avoid unnecessary repetition or redundant steps. The goal is to solve the problem accurately with the fewest necessary steps.
Now Solve This:
Question: [question]
Your Simplified and Optimized Answer:
CoD Prompt
User Message:
Think step by step, but only keep minimum draft for each thinking step, with 5 words at most. Return the answer the end of the response after separator Now Solve This:
Question: (question)
Your Simplified and Optimized Answer:
Reduction Prompt
User Message:
Let's quickly conclude the answer with shortcut reasoning.
Now Solve This:
Question: (question)
Your Simplified and Optimized Answer:
D Supervised Fine-Tuning (SFT) Hyperparameter Settings
Table 4 details the general hyperparameter configuration used for supervised fine-tuning (SFT) in our experiments. All SFT training was conducted on 8 NVIDIA RTX 3090 GPUs using the ZeRO-3 optimizer for efficient memory distribution. To accelerate training and reduce memory usage, employed bf 16 mixed-precision computation.
The training used the "lash_attentin combined with a cosine learning rate scheduler that decays to a minimum learning rate. Each GPU assigned batch size of due to the large context length of 16,384 tokens. The model was trained for 3 epochs, and max_steps was left as -1 to allow epoch-based termination. These settings balance computational feasibility and performance under long-context, reasoning-intensin tasks.
implementation for efficient attention computation,
The same configuration was applied across all target models, including DeepSeek- R1-Qwen-11.eee DeepScaleR-1 1. SB-Preview, and Phi-4-mint rrasoniing unless otherwise specified.
27

---

Table 4: General SFT Hyperparameters. Hardware: 8 x NVIDIA RTX 3090 GPUs, ZeRO-3
optimizer, b mixed precision.
Parameter
Value
attn_imp implementation
flash attention
bf16
true
learning_rate rate
5.0c-05
Ir schechllerrrrr.e _ype
cosine_ min
per device train batch size
1
max steps
max_length
16384
num num_train_qpochh
3
E In-Context Learning: Case study
To further illustrate the effectiveness of our optimized CoT examples in in-context learning (ICL), we provide case study using the Qwen-2.3 5-2B-.2.---2- 52--72B-Instruc model on a representative problem from the MATH-500 dataset.
Figure compares two responses: directly generated by the 1-2..--72B-Ins model without any in-context examples (blue background), and the other generated under ICL using our optimized CoT example (pink background).
Question:
Ground truth:
Let i+iandw
2012 Find [w].
Direct response
To find [w] where 3011 and 1 i will follow steps /n/n1. Substitute z=1+i
into the expression for w. /n/e2. Simplify the expression for w. /n/n3. Find the magnitude (w)
/n/uStep 1: Substitute ==1+ in/nu/ists substitute z=1+1 into the expression for
12 /n/nStep 2: Simplify the expression
/n/nSimplify the numerator
denominator soparately: 3(1 + +3 3-+++1+111-0 - 3//////n........ (Skip 12 steps) ... Answer:
Response after in contextlearnigg
2-1 1 i/n/n 32 1- 3(1 1 1- 3i 1- 3i/n/n 52 - 5i7
si/n/nw
12 /n/nTo find [w],, the property H Br /n/n//4 3i]
4+33/
3
-22 5/b/n/1z -1 -v169= 13/n/n/w]
/n/n Final Answer:
Figure 4: Case Study: Comparison of direct response from Qwen-2. 22.5-72BB Instruct (blue back- ground) and response under ICL with optimized CoT examples (pink background) on a MATH-500 problem. The optimized CoT enables more sufficient and necessary reasoning.
The direct response exhibits a lengthy reasoning process with several redundant or unnecessary steps and expressions. In contrast, the ICL-guided response is more concise and logically structured, reflecting a clearer and more efficient problem-solving strategy.
This comparison demonstrates how optimized CoT exemplars help guide the model toward more focused and causally sufficient and necessary reasoning.
Full Version of Related Work
Reasoning Sufficiency Enhancement CoT Optimization.
Recent efforts have focused on improving the reasoning capabilities of large language models (LLMs) through the development of Chain-of- Chain-of-Thought (CoT) reasoning and its variants. CoT (61 introduced intermediate reasoning steps to enable L.MSs to perform structured, multi-step reasoning. This foundational idea has since evolved into more sophisticated frameworks such as Tree-of-Though (ToT)[76) and Graph-of-TT oraphhof-Thhouhh (G, which organize reasoning structures into tree and graph
28

---

forms, respectively. Yue et al. [78 introduce DOTS, a method enabling dynamic reasoning trajectory planning via optimal reasoning strategy search, resulting in more adaptive and efficient problem- solving. Jin et al. (28] introduce CoT framework using graph structures for iterative reasoning in LLMS and builds the GRBench dataset for graph-based reasoning evaluation. Yang et al. [69] explores how Co'I prompts affect LLM mechanisms, enhancing knowledge retrieval by activating more neurons.
Several recent innovations further expand the reasoning capacity of LLMs. LLaVA-COT [65] combines CoT with a multimodal visual-language model to enhance reasoning in vision-language tasks. Meta-CoT [64] formulates reasoning as a latent variable process, improving flexibility and generalization. Markov Chain of Thought [73] introduces Markovian transitions across reasoning steps by clearing the context KV cache to extend reasoning depth. Additionally, Puerto et al. 47 propose self-correction mechanism using multiple reasoning paths, significantly improving performance on wwwwledee-nntesive benchmarks. Hüyük et al. [26] propose fine-tuning strategies using counterfactual feedback to enhance LLMss' causal reasoning capabilities. Ma et al. [8 demonstrate that explicit thinking processes are not always necessary and propose simplified "NoThinking" method achieving competitive reasoning performance with reduced computational costs. Diao et al. [8 present Active-Prompt, a method that enhances LLM reasoning by selectively annotating spsc-fpe prompts for automatic adaptation. ECHO S1 unifies diverse reasoning paths to improve the consistency and accuracy of LLM reasoning.
Additional recent studies have explored novel dimensions CoT reasoning. Ding et al. 19 propose "Break the Chain" strategies, integrating heuristic shortcuts to streamline CoT reasoning, significantly enhancing efficiency. Stolfo al. [51 identify entropy and token frequency neurons, elucidating internal mechanisms by which LLMs manage uncertainty and confidence. Ali et al. [ mitigate copy biases in in-context learning through targeted neuron pruning, improving generalization. De Sabbata al. 15 use rational metareasoning to selectively invoke intermediate steps, reducing inference cost while preserving accuracy. Turpin et al. (58) question CoT's faithfulness by systematically evaluating model-generated rationales. Jin et al. [30] demonstrate that artificially lengthening reasoning can superficially boost performance, highlighting potential pitfalls in CoT evaluation. Simhi et al. [49] reveal that LLMs can exhibit high-certainty hallucinations, producing incorrect answers with strong confidence, thereby challenging the reliability of CoT-based outputs.
Despite their benefits. these structured reasoning methods often introduce excessive token length, which becomes problematic in cost-sensitive or latency-constrainet scenarios (62. Moreover, models frequently fail to assess task complexity, leading to over-reasoning simple problems-an issue known as overthinking [12 36). Our proposed approach addresses these limitations by simultaneously ensuring reasoning efficiency and maintaining high accuracy.
F.2 Reasoning CoT Redundancy.
Recent studies on CoT redundancy have sought to mitigate redundancy in CoT reasoning. Token- budget-aware methods, such as Han et al. [22), dynamically allocate reasoning budgets based on task complexity. C3oT (32) leverages GPT-4 as compressor retain only essential reasoning content. CCoT 9 and COCONUT 23 adopt continuous representations to encode reasoning traces
more compactly, while CoT- Valve 39 introduces variable- length CoT', Training-fre free approaches, including Kimi K1.5
(54) and O1-Pruner
36
use prompt ensembles or reinforcement learning to
discard unnecessary reasoning steps.
Recent efforts also investigate finer-grained control over CoT reasoning. Wu ct al. [63] introduces the Thinking Intervention paradigm, allowing targeted interventions reasoning tokens. Yang et al. [74] highlight the impact of excessive CoT length on model performance, proposing adaptive token scaling strategies. Liu et al. (35] demonstrate external thoughts from smaller models can effectively streamline reasoning in larger models, reducing redundancy.
However, most of these methods focus primarily on sequence length reduction or representation compression, without explicitly addressing causal logical redundancy. In contrast, our approach identifies and preserves only those reasoning steps that are both causally sufficient and necessary, grounded in a formal intervention- framework. This results in CoT traces that are not only more compact but also more causally meaningful.
29

---

F33 PNS Theory in CoT Reasoning.
Our work is grounded in the causal inference framework proposed by Pearl A6]. which defines the Probability of Sufficiency (PS) and the Probability of Necessity (PN) to quantify whether a cause is sufficient necessary for given effect. For instance, in image classification, PS measures how likely adding a feature (e.g., "pointy ears") leads to a positive label (e.g., "cat"), while PN measures how likely removing that feature would change the outcome.
While Hüyük et al. (26 introduced PN and PS as evaluation metrics to enhance model-Ievel causal reasoning via counterfactual fine-tuning, our approach advances this direction by applying these metrics to the internal reasoning traces of Chain-of-Thought (CoT) prompting. Specifically, we formalize and operationalize the Probability of Necessary and Sufficient Causes (PNS) for individual reasoning steps, enabling a structured intervention-bant -aaed analysis that identifies steps that are both causally essential and logically minimal.
Unlike prior work that emphasizes model-wide causal consistency, our framework targets step-level causal minimality within COT, yielding concise, interpretable, and causally grounded reasoning sequences. This design allows us to optimize CoTss not merely for brevity or accuracy, but for causal soundness. Crucially, our method is model-agnostee and applies to any LLM capable of generating CoT outputs, representing a significant advancement over heuristic or model-specific compression strategies.
Our approach is essentially causal analysis method for multi-step systems. In fact, it is not only applicable to Chain-of- Chain-of-Thought (CoT)) reasoning, but can also inspire causal thinking in computational experiments [68 77 and other complex systems.
G Additional PNS Comparison Results
This appendix provides comprehensive visualizations average PNS values before and after applying our optimization algorithm across multiple models and datasets. Each figure presents results over 30 sampled problems, highlighting the increased causal necessity of the retained reasoning steps after optimization. See Figures t 8 for detailed comparisons.
PNS Comparison 30 AIME (Owen-2.557) 2aaemn22.5722--tttteee
Before PAS Optimization
After PNS Optimization
1.
Average PaS (Supp)
3.8
0.6
AME U.5
Re
AIME AME
AIME u.
AMEI) .
AMME 1.7 AME
AME
AIME 11
AMEE 2.2
AMME) 12
AMME 1.3
AIME AMMe 1.1
AIE 1112
II
AME II
AIME 2.6
AINE AIME It
1d
AME
AIME 8
AIME LI
AIME 1.12
AIME 1.5
AIME
AME
AMME tIME .10
7
11
NIKE 14
AIME 4
Figure 5: Average PNS comparison for Qwen 2...-2.55772 52- Instruct on the AIME dataset.
PNS Comparison Commonsense@A Os (Qwen 2.5- 1----nsttcc
Before PNS Optimization
After PMS Optimization
3.8
PWS
3.6
2.4
02
coM.13 133
CoM. e
COM 281
205.252
252
2099
COM 111
COM 366
cod COM A35
cod.346
coM 136
con 269
78
200
COM.235 2335
cod da
cold 22
126
coM_262
2922
379
473
cou 222
COM 220
CoM 203
235
col.30 30s
cod 482
COM
com.
cod.
col
cod
COM
coM COM
COM
Figure 6: Average PNS comparison for Qwen-2 2. 5 52B Instruct on the CommonsenseQA dataset.
30

---

PNS Comparison 30 AIME 2023 Qs (DeepSeek-R1)
Before Optimization
Aversge
2.4
02
AIME 1.3 ARME, 1o AIME ft. AINE Le
AIME 1.7
AIME ARME, 1.
AHME 8,13 AIKE 11.2 AIME
AIME 13 AIME, 1.1 AIME I.I2
AIME f.1
AIME it AIME v
AWME 9.0
AHE I1 AINE, 12 AIME L.
AIME La
AIME I.7
AIME, 3.11 AIME 1.0 AIME 14
AIME
AIME
AMME U.
AME
AIME
Figure 7: Average PNS comparison for DeepSeek R1 on the AIME dataset.
PNS Comparison 30 Commonsense@A Qs (DeepSeek-R) Optimization
Before Optimization
8.8
Nvaroge PNS
CoN la
com 272
com con top
com 220
Cow 74
conl z'
col 171
COM ad
com 172
cod 128
CoM 22
com 235
com 147
com 178
com 245
221
117
col id4
com 121
com 122
CoN 76
COM sJ
com Is
cott 223
com Itd
col 12
coll 173
com
CoM
Figure 8: Average PNS comparison for DeepSeek-Ri -1 on the CommonsenseQA dataset.
H
Computational Complexity Analysis
Let us denote the following quantities:
n: the number of reasoning steps in the chain-of offthhugttthhht (CoT);
the number rollouts per step;
stop: the average number of tokens generated per step;
lomi the number of tokens in each verification output (assumed constant);
loken: the time required to generate a single token (assumed constant).
We analyze the computational cost of our PNS-based intervention method by separately evaluating the rollout and evaluation stages.
H.1 Rollout Time Complexity
At each reasoning step i E [l,...,,, the model performs k rollouts, where each rollout generates the remaining (n - f) steps. Each step contains on average lstp tokens. Thus, the total rollout time is given by:
T Trollout
Σ
k- (n-(n- (n- i) -eepp Loken
O(k: kaep broenn n²²)
H.2 Evaluation Time Complexity
Each rollout must also be evaluated to determine its validity. This involves generating both the full continuation of the chain and a final answer output of length l The total evaluation time is therefore:
Teval Σ k [(1e -i) 1step + loat] booken O(k booken (lsep n2 + Lout n))
Since typically lam & lstep n, the second term is asymptotically negligible, yielding:
Toxal Okk lseep troknn n2
(8)
31

---

H.3
Total Complexity
Combining the rollout and evaluation phases, the total computational cost becomes:
T Fisial
T Tollout
+ Teval O Isee booenn n
(9)
Assuming that both step and Loken constants (or near-constants) in practice, the overall time complexity simplifies to:
Tens O(k n²))
(10)
This quadratic dependence on n highlights the computational cost of deeper reasoning chains, while the linear dependence k reflects the tradeoff between rollout breadth and computation time.
I Qualitative Analysis of Whether the Final Reasoning Is Sufficient and Necessary
We conducted a human evaluation of 50 chain-of-thought (CoT) samples generated by model fine-tuned on data curated with our PNS-based algorithm. Five mathematics experts independently judged each sample under three labels:
S&N: reasoning is both sufficient and necessary to support the final answer;
SbU: reasoning is sufficient but contains unnecessary (redundant) steps;
Nbl: reasoning insufficient (missing critical steps).
Table 5: Human evaluation of reasoning quality. "Fully Sufficient" S&N + SbU. "Redundant" flags any chain containing redundant steps and may co-occur with Nbl.
Dataset
Samples
Fully Sufficient
Redundant
S&N
SbU
Nbl
GSM8K
20
19
3
17
2
1
Commonsense QA
15
15
1
14
0
MATH500
15
13
4
11
2
2
Total
50 47 (94.0%) (16.0%)
42 (84.0%)
(10.0%)
(6.0%)
Most outputs are logically sound under the sufficiency/necessitty criterion: 84% of CoTs are both sufficient and necessary, and only 6% (3/50) are insufficient. MATH500 exhibits higher incidence of redundancy and incompleteness, consistent with the greater difficulty of mathematical reasoning in this benchmark.
Validator Accuracy and Robustness
To assess the reliability of the PNS validator V, we evaluate its performance across different LLMs and rollout sizes k. Specifically, we report the mean abxolute error (MAE) between the estimated PNS values and ground-truth labels, where ground-truth PNS is defined as 1 for reasoning chains judged by experts be both necessary and sufficient.
Metric. The evaluation computes the mean absolute difference [PNS PNS| across annotated samples. Lower values indicate better alignment with ground truth.
Table 6: Validator MAE Across LLMs and Rollout Sizes. Lower is better.
We compare validator accuracy rollout sizes using stronger
model strengths. Increasing LLMs both reduce MAE, yielding more stable and ac- curate PNS estimates. GPT-4o achieves the lowest error across all settings, underscoring that validator quality and model strength key reliable counterfactual analysis.
Validator
k=1
k=3
k=5
k=
Qwen-72B
0.315
0.187
0.142
0.116
Qwen-7B GPT-4o
0.411
0.395
0.315
0.293
0.137
0.114
0.090
0.050
32

---

K
Supplementary Experimental Results
To further assess the robustness and generality of our proposed method, compare with three representative reasoning baselines: SPIRIT, ReAct, and Tree-of-Thoughts (ToT). These methods represent distinct paradigms for enhancing large language model reasoning:
SPIRIT
33
uses perplexity identify key reasoning steps and prune redundant tokens.
ReAct 75 integrates reasoning and acting by alternating between thought and action steps, improving interpretability and interactive decision
decision-making. making.
Tree-of-Thought (To'T) (76 introduces a tree-structured reasoning process, maintaining
multiple reasoning trajectories to perform deliberate exploration and self-evaluation. We implement this process using a simple prompt-based approach.
Table / summarizes the results across three benchmarks CommonsenseQAA GSM-8K, and MATH- 500. Each metric is reported Tokens\/ Steps.// Acc.t, where fewer tokens and steps indicate greater reasoning efficiency.
Table Comparison of SPIRIT, ReAct, ToT, and Ours-ICI, across models and datasets. Each metric is reported as Tokens|/ Steps Ace.t.
CommonsenseQA
GSM-8k
MATH-500
Model
Method
Tokens.
Steps!,
Acc.t
Tokens,
Steps.!
Acc.
Tokens.
Steps].
Acc.
SPIRIT
142.3
4.4
83,9
73.1
2.0
95.5
250.5
7.7
89.6
D3
ReAct
181.2
5.4
83.8
179.5
6.9
92.3
412.3
19.0
91.2
ToT
271.8
8.2
80.1
198.9
7.7
91.7
349.1
13.9
72.8
Ours-ICL
44.7
52.2
6.4
96.2
SPIRIT
214.2
7.6
76.3
98.3
4.0
94.2
239.3
10.3
81.4
ReAct
175.0
4.1
83.7
171.7
77.6
Qwen-2.5.721 -nsstrcct
ToT
269.2
7.9
80.0
273.5
10.4
75.1
345.1
14,1
70.6
Ours-ICL
81.6
3.4
83.0
65.3
5.3
99.5
196,9
8.9
81.2
SPIRIT
188.6
6.5
59.3
113.7
5.1
86.7
238.8
9.4
72.9
ReAct
192.9
5.2
79.4
181.6
7.2
84.5
263.0
10.5
70.8
Qwen 2,5-7B Instruct
ToT
335.4
78.0
Ours-ICI.
99.1
3.8
77.6
83.4
4.8
94.1
174.7
7.7
72.6
170.1
7.4
63.2
179.7
7.8
80.9
1218.1
67.3
1.MM-3311
ReAct
382.1
17.9
79.0
521.5
20.6
82.5
1176.9
$3.6
46.0
ToT
$59.9
19.4
75.3
711.7
27.1
69.0
996.7
45.8
30.2
Ours-ICL
120.3
7.1
72.1
128.3
8.1
93.1
365,9
24.2
54.8
Across all models and datasets, our proposed Ours-ICL consistently achieves a superior balance between reasoning accuracy and efficiency. While ReAct and ToT obtain competitive accuracy on commonsense reasoning tasks, they dramatically increase reasoning cost in both tokens and steps. SPIRIT improves efficiency via perplexity-based step selection but underperform complex mathematical reasoning. In contrast, Ours-ICL yields the best overall trade-off, reducing reasoning by up 60-80% while maintaining improving accuracy, especially GSM-8K and MATH-500.
33


Reviews:
[{"review_id": "eYX6IKNddm", "reviewer": "Reviewer_Uvzq", "summary": "This paper introduces a causal framework for optimizing Chain-of-Thought (CoT) reasoning by identifying and preserving only causally sufficient and necessary steps. Using counterfactual metrics (PS and PN), the method prunes redundant reasoning from correct CoTs and applies the optimized traces for in-context learning and fine-tuning. Experiments show improved reasoning efficiency and accuracy across multiple benchmarks.", "questions": "Please refer to the “weakness”.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\nThe paper presents a clear problem setup and task formulation; the proposed method is straightforward and interpretable; the experimental results appear promising.\n\n\nWeaknesses:\n\n1.\tLimited Novelty.\nUsing counterfactual interventions like PS and PN in CoT is conceptually interesting, but causal counterfactual as a core mechanism has been adopted in many existing studies across causal inference and CoT interpretability.\n\n2. Algorithm Soundness.\na. Section 4 describes the proposed method as a two-stage optimization process: first enhancing PS, then improving PN. However, the paper does not clearly explain how PS is actually enhanced. Algorithm 1 does not seem to contain any explicit procedure for stage 1’s improving PS, which is confusing.\nb. It is also unclear how the method handles initially incorrect CoTs. According to Algorithm 1, if the original CoT is not sufficient (i.e., leads to an incorrect answer), the algorithm simply returns the unmodified chain. This seems to limit the practical applicability of the method. If the method is indeed capable of improving incorrect CoTs in practice, the authors should provide a clear explanation or clarification in the main text.\n\n3. On experimental setup and conclusions.\na. A weakness of the experimental part may be the lack of detail regarding metric definitions and inference setup, e.g., it is unclear how accuracy is calculated—was it based on a single rollout, or averaged over multiple rollouts? Additionally, the decoding setup is a bit confusing: the appendix states that inference was performed with temperature=0.6, top_p=0.95, and greedy decoding, which appear to be contradictory. \nb. Regarding RQ1, I am confused by the conclusion that “the PNS-based algorithm can improve accuracy.” According to Algorithm 1, the proposed method appears to apply only to CoTs that already lead to correct answers (i.e., PS = 1). This raises a concern: if only correct CoTs are subject to optimized, then the upper bound should be the original accuracy. I suggest that the authors further clarify the metric calculation.\nc. Regarding RQ2, the authors mention that all SFT examples were manually verified. What is the reasoning behind this, and could it potentially impact the comparison fairness? In other words, if human filtering is still required, it undermines the method’s automation and scalability. Additionally, the paper does not clarify whether the Noncausal SFT baseline was trained only on correct CoTs (or with wrong CoTs included), which significantly affects the performance of SFT and the fairness of the comparison.", "was_revised": true, "final_justification": "The rebuttals have solved most of my concerns, I decide to keep my positive score.", "rebuttal": "**1. [W1] Related Work Discussion about Using PS and PN in CoT.**\n\nThank you for your insightful comment. To the best of our knowledge, this work is the first to formally introduce the causal concept of Probability of Necessity and Sufficiency (PNS) into large language models, specifically to address the issue of “overthinking” in Chain-of-Thought (CoT) reasoning. While prior studies have explored counterfactual metrics such as PS and PN, PNS has not been applied in this context, nor has it been employed as a causal criterion to identify and prune unnecessary reasoning steps. Our approach offers a novel and actionable causal perspective, leveraging effective intervention strategies to improve both the performance and interpretability of the model.\n\n**2.[W2] In Algorithm 1, how is PS improved if the original CoT is incorrect?**\n\nThank you for the question. As PS is defined over the entire reasoning chain (CoT), while PN is defined at the step level, our approach **aims to improve PS by repeatedly executing Algorithm 1 under the same question context**, especially when the original CoT is insufficient (i.e., fails to yield the correct answer).\n\nIn each execution, the model samples an alternative CoT, and its PS is re-evaluated. This repeated sampling increases the likelihood of obtaining a CoT with higher PS, thereby enhancing the reliability of the subsequent PN-based intervention.\n\nAlthough Algorithm 1 does not explicitly separate this process as a standalone stage, it plays a key role in improving PS before any pruning is applied. We will clarify this iterative refinement mechanism in the revised manuscript to better reflect the actual implementation and practical utility of our method.\n\n**3.[W3] Details about metric definition and inference setup (e.g., accuracy calculation and decoding setup)**\n\nThank you for your question. We address each point as follows:\n\n**a. On metric definition and decoding setup:**\n\nThe reported **accuracy** refers to the average proportion of correctly answered instances in the test set. It is computed as: $\\text{Accuracy} = \\frac{\\text{Number of correct CoTs}}{\\text{Total number of CoTs in the dataset}}$\n\nThis is a standard evaluation setup in LLM reasoning benchmarks and is consistent with prior CoT literature.\n\nRegarding the decoding configuration, the mention of greedy decoding in the appendix was a typo. We apologize for the confusion. In practice, we consistently use temperature = 0.6 and top_p = 0.95 across all experiments.\n\n**b. On the meaning of “accuracy improvement” in RQ1:**\n\nThe accuracy gains reported in RQ1 result from the fact that, in practice, we continuously apply Algorithm 1 across the dataset. Through repeated rollouts of each CoT step, the probability of generating a correct answer increases significantly. A CoT with PS < 1 might be overlooked in a single run of Algorithm 1, but is revisited and re-evaluated in subsequent rollouts until the entire dataset undergoes PNS-based optimization.\n\n**c. On fairness concerns regarding manual inspection in RQ2:**\n\nWe confirm that **no manual editing was performed on the CoTs selected by our method**—only human verification of quality was conducted. As shown in RQ1, the CoTs filtered via the PNS method are already causally sound and can be directly used for SFT.\n\nThe **Non-Causal SFT baseline** uses raw, unfiltered CoTs (including both correct and incorrect samples). In contrast, the **Causal SFT baseline** uses CoTs that were optimized via the PNS-based filtering procedure. This distinction is **intentional and fair**, as our goal is to demonstrate how causal filtering improves data quality and, ultimately, downstream performance.\n\nWe will clarify this design choice and its rationale in greater detail in the revised manuscript."}, {"review_id": "OLj4bpHQaw", "reviewer": "Reviewer_rD2Z", "summary": "This paper proposes a causal framework to enhance CoT reasoning from the perspective of sufficiency and necessity.\n\nExisting COT reasoning cannot ensure that all intermediate steps are useful for the final reasoning, nor can it ensure that the intermediate steps are sufficient to make the final reasoning.\n\nThis paper proposes the concept of PNS, which aims to remove redundant steps on the one hand and complete the missing steps on the other hand, so as to improve the performance of reasoning and reduce the cost.", "questions": "1. Does the complete PNS consume more tokens than COT? If so, is there any way to directly generate shorter and better reasoning?\n2. Analysis of the number and complexity of tokens for complete PNS reasoning\n3. Qualitative analysis of whether the final reasoning is sufficient and necessary", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\nThis paper proposes to optimize the reasoning process from the perspective of sufficiency and necessity.\nDealing with the redundancy problem of COT reasoning is indeed a huge challenge, and it is intuitively reasonable to generate sufficient and necessary reasoning processes. Experiments have shown that the PNS method proposed in this paper does improve the reasoning accuracy in the end. The resulting final reasoning length is also shorter than COT reasoning.\n\n**Weaknesses**\n1. This paper claims that it significantly reduces reasoning redundancy, but PNS does not seem to be able to directly generate short reasoning results. It checks the entire chain and each step after COT reasoning, rewrites some steps, and finally prune them. Doesn't this process consume more tokens?\n2. The framework of this article seems to be prompt engineering. Is there a way to make the model eventually learn to generate short and effective reasoning in one generation?\n3. If the PNS reasoning process described in 1 is correct, you may need to add more complexity analysis and cost experiments on the number of all tokens generated during the entire reasoning process.\n4. You may need human evaluation or qualitative experiments to analyze whether the final reasoning generated by PNS is sufficient and necessary.", "was_revised": true, "final_justification": "Thank you for your additional analysis, we would like to raise our rating to 4", "rebuttal": "**1.[Q1 & W1, W2] Does the complete PNS procedure consume more tokens than CoT? If so, can we reduce the token usage or train the model to generate shorter and more effective reasoning directly? Is this approach just prompt engineering?**\n\nThank you for the thoughtful questions. We respond as follows:\n\nOur method adopts a **two-stage framework** (see Lines 120–128):\n\n1. **Data preparation stage**: We apply the PNS-based filtering method to existing CoT data to optimize and clean reasoning traces. This ensures that the selected CoTs satisfy both **necessity and sufficiency**.\n2. **Inference stage**: We use the optimized dataset to perform **in-context learning** or **fine-tuning**, enabling the model to **learn to generate short and effective reasoning in a single forward pass**.\n\nWhile it is true that the PNS filtering process incurs additional token usage during the data preparation phase, this is a **one-time cost**. Crucially, **the actual inference process after fine-tuning consumes fewer tokens**, as the model learns to generate concise and accurate reasoning directly—without requiring step-by-step rollout or post-hoc pruning.\n\nIn this sense, the strength of our method does **not** lie in reducing token usage during the filtering phase, but rather in reshaping the training distribution. By training on high-PNS data, the model internalizes causally meaningful and non-redundant reasoning patterns. The benefit manifests at inference time, where the model naturally produces streamlined CoTs, improving both efficiency and interpretability.\n\nFinally, we would like to clarify that our method is not merely prompt engineering. Although the filtering procedure is prompt-based, the core of our approach lies in learning from causally sound CoTs via supervised fine-tuning (SFT). After this training, the model is able to generalize the PNS principle, generating short and effective reasoning from scratch—without requiring further intervention.\n\nWe will revise the manuscript to clarify this two-stage design and the distinction between one-time filtering overhead and long-term inference efficiency.\n\n**2.[Q2 & W3] Analysis of the Number and Complexity of Tokens for Complete PNS Reasoning.**\n\n(1) We **added a new experiment** (see **Table R1**) showing token cost and rollout complexity across LLMs (e.g., Qwen2.5) and LRMs (e.g., DeepSeek R1) during PNS filtering.\n\n(2) We also compare average tokens per CoT before and after PNS-based fine-tuning (see **Table 3 in the main paper)**, demonstrating reduced inference cost and improved conciseness.\n\n- Across three different LLM families, we observe a consistent trend: while PNS filtering introduces a moderate one-time token cost during data preparation, it yields over 20% token savings during inference.\n- The accuracy of all models is either maintained or improved, confirming that the CoTs retained after causal filtering are both concise and effective.\n- This supports our central claim: PNS filtering reshapes the training distribution, enabling the model to generate more efficient and logically minimal reasoning paths—without requiring inference-time pruning.\n\n**Table R1**. Token usage and complexity during PNS filtering\n| **Model (Base)**            | **Dataset**       | **Rollouts** | **Avg. Tokens** | **Time Complexity** |\n|----------------------------|-------------------|--------------|------------------|---------------------|\n| Qwen-2.5-72B-Instruct | GSM8k             | 9.77         | 1,104.42         | $O(k · n²)$          |\n|                            | CommonsenseQA     | 24.84        | 2,945.61         | $O(k · n²)$           |\n|                            | MATH-500          | 21.66        | 2,315.26         | $O(k · n²)$          |\n|                            | AIME              | 61.81        | 12,145.87        | $O(k · n²)$         |\n| DeepSeek R1            | GSM8k             | 22.62        | 2,861.12         | $O(k · n²)$         |\n|                            | CommonsenseQA     | 30.81        | 4,501.99         | $O(k · n²)$         |\n|                            | MATH-500          | 46.16        | 9,855.70         | $O(k · n²)$         |\n|                            | AIME              | 660.90       | 371,430.57       | $O(k · n²)$          |\n> *Rollouts*: Average number of step-level interventions (regenerations) per CoT.  \n*Avg. Tokens*: Average total number of tokens generated per example during PNS filtering.\n\n**Time Complexity Derivation:**\n\nLet $n$ be the number of reasoning steps in a CoT;\n\n$k$ the number of rollouts per step;\n\n$l_{\\text{step}}$ the average number of tokens generated per step;\n\n$l_{\\text{out}}$ the number of tokens required to generate an answer during verification (typically constant);\n\nand $t_{\\text{token}}$ the time required to generate one token (a constant).\n\nAssuming each rollout at step $i$ generates approximately $(n - i)$ subsequent steps, each with $l_{\\text{step}}$ tokens, the total rollout cost is:\n\n$$\nT_{\\text{rollout}} = \\sum_{i=1}^{n} k \\cdot (n - i) \\cdot l_{\\text{step}} \\cdot t_{\\text{token}} = O(k \\cdot l_{\\text{step}} \\cdot t_{\\text{token}} \\cdot n^2)\n$$\n\nEach rollout also requires evaluating whether the generated result is valid, including generating the answer ($l_{\\text{out}}$ tokens) and the full chain:\n\n$$\nT_{\\text{eval}} = \\sum_{i=1}^{n} k \\cdot \\left[ (n - i) \\cdot l_{\\text{step}} + l_{\\text{out}} \\right] \\cdot t_{\\text{token}} = O(k \\cdot t_{\\text{token}} \\cdot (l_{\\text{step}} \\cdot n^2 + l_{\\text{out}} \\cdot n))\n$$\n\nSince $l_{\\text{out}} \\ll l_{\\text{step}} \\cdot n$, we simplify:\n\n$$\nT_{\\text{eval}} = O(k \\cdot l_{\\text{step}} \\cdot t_{\\text{token}} \\cdot n^2)\n$$\n\nCombining both rollout and evaluation:\n\n$$\nT_{\\text{total}} = T_{\\text{rollout}} + T_{\\text{eval}} = O(k \\cdot l_{\\text{step}} \\cdot t_{\\text{token}} \\cdot n^2)\n$$\n\nSince $l_{\\text{step}}$ and $t_{\\text{token}}$ are typically constant or nearly so, the final complexity simplifies to:\n\n$$\nT_{\\text{PNS}} = O(k \\cdot n^2)\n$$\n\nOverall, our results show that the proposed method **reduces reasoning redundancy**, trading a one-time overhead for sustained improvements in inference efficiency.\n\n**3.[Q3 & W4] Qualitative Analysis of Whether the Final Reasoning Is Sufficient and Necessary.**\n\nThank you for the insightful suggestion. We conducted **an additional human evaluation** on 50 CoT samples, all generated by a model fine-tuned on data optimized using our PNS-based algorithm. Five mathematics experts were involved in validating the 50 samples, assessing whether each reasoning chain was sufficient and necessary to support the final answer.\n\n- **S&N**: The reasoning is both sufficient and necessary to support the final answer;\n- **SbU**: The reasoning is sufficient but includes unnecessary (redundant) steps;\n- **NbI**: The reasoning is insufficient, i.e., missing critical steps;\n\nThe results are summarized as follows:\n\n**Table R2.** Human Evaluation of Reasoning Quality\n\n| **Dataset**      | **# Samples** | **Fully Sufficient** | **Redundant** | **S&N**        | **SbU**       | **Nbl**       |\n|------------------|---------------|-----------------------|----------------|----------------|----------------|----------------|\n| GSM–8k           | 20            | 19                    | 3              | 17             | 2              | 1              |\n| Commonsense QA   | 15            | 15                    | 1              | 14             | 1              | 0              |\n| MATH–500         | 15            | 13                    | 4              | 11             | 2              | 2              |\n| **Total**        | **50**        | **47 (94.0%)**        | **8 (16.0%)**  | **42 (84.0%)** | **5 (10.0%)**  | **3 (6.0%)**   |\n\nThese results confirm that most model outputs are not only accurate but also logically sound under the criteria of sufficiency and necessity.\n\nNotably:\n\n- 84% of CoTs are classified as both sufficient and necessary (S&N),\n- Only 3 out of 50 CoTs (6.0%) were deemed insufficient (Nbl),\n- MATH-500 exhibits a slightly higher rate of redundancy and incompleteness, which is expected due to the complexity of mathematical reasoning.\n\nWe will include this evaluation and Table R2 in the revised manuscript."}, {"review_id": "2j9AL7hqfZ", "reviewer": "Reviewer_iGwQ", "summary": "This paper proposes to characterize CoT with causal probability of sufficiency and necessity. In a CoT, the intermediate steps are sufficient if they can support the conclusion, while necessary if they are indispensable for the soundness of the final answer. By optimizing the sufficiency and necessity of CoT (i.e., PNS optimization) with causal intervention on each intermediate step (actually, the authors only optimized the necessity), the authors reconstruct more concise CoT responses from training data, eliminating redundant steps without affecting the accuracy. Empirically, the PNS optimization method enhances causal sufficiency and necessity of CoTs. While the optimized CoTs contain fewer tokens and steps, they don’t compromise the performance and can sometimes even improve the performance on reasoning benchmarks in both in-context learning and finetuning scenarios.", "questions": "1. Can you discuss how PNS optimization is different from SPIRIT[1]? They look similar as both involve counterfactual intervention and scoring with an LLM.\n2. In Equation 2, what is $\\bar{S}$ in the RHS? It’s not defined in the LHS. Can I understand RHS as the average for any $\\bar{S}$?\n3. Is Equation 4 factorizable as $P(A_S = y) P(A_{S'} \\neq y)$? That would be more intuitive for your Lemma 1.\n4. What is semantic disjointness in Algorithm 1?\n5. Where do the unoptimized CoT traces come from? Are they human annotated, or elicited by zero-shot CoT prompting? If there is no comparison against other CoT optimization methods, how can I trust the initial unoptimized CoT traces are not deliberately made overthinking?\n6. What is the setting for measuring accuracy in Table 1? Is it like in-context learning in Table 2 or finetuning in Table 3?\n7. Table 3: Does original refer to the model before SFT? Does noncausal refer to unoptimized SFT data?\n8. Line 97: in Pearl → by Pearl", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. As CoTs grow longer and longer in recent LLMs, it is very important to optimize the length of CoTs to reduce finetuning and inference workloads. This paper provides an automatic solution for optimizing CoTs without human annotation.\n2. Unlike previous methods that mostly rely on correlation-based metrics, this paper uses causal metrics to guide the optimization of CoT, which is intuitive and may inspire new methods in this field.\n3. Experimental results show that the proposed PNS optimization can reduce the length of CoTs by 2-6x, without compromising performance in both in-context learning and finetuning scenarios. However, it’s unclear whether the proposed method is effective or CoT optimization is easy (see weakness 1).\n\nWeaknesses:\n\n1. The authors only compared PNS optimization against efficient CoT prompting methods. I feel efficient CoT prompting methods might be weak baselines here, since they are just slightly optimized by human. It’s not clear whether the proposed method is effective, or standard prompts are so redundant that any CoT optimization method can achieve good results. The authors need to consider other CoT optimization methods, such as SPIRIT[1], or a CoT optimization method derived from information gain[2]. [1, 2] should also be discussed in related works.\n2. While the authors claim to optimize sufficiency and necessity, in reality they assumed sufficiency is fulfilled and only optimized necessity, as shown in Algorithm 1 and Lemma 1. In that sense, the “addition of missing steps” in the abstract, as well as the “bi-level optimization framework” in the contribution part, are overclaim.\n3. The writing of this paper has much room to improve. The title may be more specific about what causal sufficiency and necessity improves. The authors may discuss the impact of optimizing CoTs and then transit to the challenges of sufficiency and necessity.\n\n[1] Cui et al. Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models. arXiv 2025.\n\n[2] Ton and Taufiq et al. Understanding Chain-of-Thought in LLMs through Information Theory. arXiv 2024.", "was_revised": true, "final_justification": "The authors have successfully addressed my concern on lack of baselines in their response. Regarding the claim of optimizing sufficiency, I assume it's overclaim given that the authors didn't directly respond to my concern in their response. I am generally comfortable with my rating and don't oppose the acceptance of this paper.", "rebuttal": "We sincerely thank you for your constructive and insightful feedback. Our responses are as follows:\n\n**1. [Q1] Can you discuss how PNS optimization is different from SPIRIT [1]? They look similar as both involve counterfactual intervention and scoring with an LLM.**\n\nThank you for raising this important point. We have conducted an additional experiment to directly compare our method with SPIRIT, focusing on its prompt-based ICL version. The results are shown in Table R1 below, and more baseline reproductions will be added in future work.\n\nAlthough both our method and SPIRIT [1] involve selecting reasoning steps, there are **fundamental differences** in their principles and goals:\n\n1. **Causal vs. Heuristic**: Our method is grounded in **causal theory**, using Probability of Necessity and Sufficiency (PNS) to guide selection. In contrast, SPIRIT relies on **perplexity**, a fluency-based heuristic that reflects the LLM’s confidence but not its causal contribution.\n2. **Counterfactual Intervention**: We perform actual interventions (i.e., step modification and rollout) to estimate how altering each step impacts the final outcome. SPIRIT does not perform such counterfactual evaluation.\n3. **Optimization Objective**: Our goal is to **retain only those steps that are both necessary and sufficient**, ensuring causal minimality. SPIRIT, on the other hand, aims to **reduce token count while maintaining fluency**, which may inadvertently discard causally important steps.\n\nWe have added both SPIRIT [1] and Ton et al. [2] to the Related Work section of the revised manuscript:\n> SPIRIT [1] uses perplexity to effectively identify key reasoning steps, achieving a balance between accuracy and efficiency in both few-shot CoT and fine-tuning settings, and also exhibits strong cross-model transferability. Ton et al. [2], based on information-theoretic principles, quantify the contribution of each step in the reasoning chain to the final correct answer using conditional mutual information, enabling the identification of reasoning failure patterns in large language models without requiring manual annotation of intermediate steps.\n\n**Table R1.** Comparison with SPIRIT.\n| **Model**                | **Method** | **CommonsenseQA (Tokens / Steps / Accuracy)** | **GSM-8k (Tokens / Steps / Accuracy)** | **MATH-500 (Tokens / Steps / Accuracy)** |\n| ------------------------- | -------- | ------------------------------------------- | ------------------------------------ | -------------------------------------- |\n| **DeepSeek-V3**           | SPIRIT   | 142.3/4.4/**83.9%**                         | 73.1/**2.0**/95.5%                   | 250.5/7.7/89.6%                        |\n|                           | Standard | 177.5/5.7/83.8%                             | 157.3/7.4/97.6%                      | 598.6 /26.7 /93.2%                     |\n|                           | Ours-ICL | **44.7**/ **2.7**/83.6%                     | **52.2** /4.3 /**99.9%**             | **136.7**/**6.4**/**96.2%**            |\n| **Qwen-2.5-72B-Instruct** | SPIRIT   | 214.2/7.6/76.3%                             | 98.3/**4.0**/94.2%                   | 239.3/10.3/81.4%                       |\n|                           | Standard | 109.1/3.7 /78.2%                            | 113.8/8.0 /93.6%                     | 281.8/9.2/**84.0%**                    |\n|                           | Ours-ICL | **81.6** /**3.4** /**83.0%**                | **65.3**/5.3 /**99.5%**              | **196.9** /**8.9**/81.2%               |\n| **Qwen-2.5-7B-Instruct**  | SPIRIT   | 188.6/6.5/59.3%                             | 113.7/5.1/86.7%                      | 238.8/9.4/**72.9%**                    |\n|                           | Standard | 209.8/7.2 /70.4%                            | 149.7/7.2/85.1%                      | 263.5 /9.7/71.0%                       |\n|                           | Ours-ICL | **99.1** /**3.8**/**77.6%**                 | **83.4**/**4.8**/**94.1%**        | **174.7**/**7.7**/72.6%                |\n| **LLaMA-3.1-8B-Instruct** | SPIRIT   |        170.1/7.4/63.2%     |      179.7/**7.8**/80.9%      |      1218.1/67.3/42.8%             | \n|                           | Standard | 169.3 /7.3 /**72.2%**                         | 182.8/7.9/79.2%                      | 741.3 /46.0 /47.6%                     |\n|                           | Ours-ICL | **120.3**/**7.1**/72.1%                           | **128.3**/8.1/**93.1%**                      | **365.9** /**24.2** /**54.8%**                     |\n\nOur additional experiments corroborate the main findings reported in the paper (see lines 212–222): **our method consistently achieves better accuracy–efficiency trade-offs across diverse models and datasets**. Compared to several baselines, it significantly reduces the number of steps and tokens without compromising accuracy—and often improves it.\n\n**2.[Q2]  In Equation 2, what is $\\bar{S}$ in the RHS? It’s not defined in the LHS. Can I understand RHS as the average for any $\\bar{S}$?**\n\nThank you for pointing this out. In Equation 2,  $\\bar{S}$ denotes a **reasoning chain that fails to produce the correct answer**. The expression on the right-hand side should not be interpreted as an average over all possible , but rather as being restricted to the context where $A \\ne y$.\n\n**3. [Q3] Is Equation 4 factorizable as $P(A_S = y) \\cdot P(A_{S'} \\ne y)$? That would be more intuitive for your Lemma 1.**\n\nThank you for the question. This factorization is mathematically invalid due to a causal dependency between the events $A_S = y$ and $A_{S'} \\ne y$.\n\nEquation (4) defines a joint probability:\n\n$$\nPNS(S, s_t, q) = P(A_S = y, A_{S'} \\ne y)\n$$\n\nBy the chain rule of probability:\n\n$$\nP(A_S = y, A_{S'} \\ne y) = P(A_S = y) \\cdot P(A_{S'} \\ne y \\mid A_S = y)\n$$\n\nSince $S$ and $S'$ differ by only one step, their outputs are highly correlated. Therefore, the conditional probability **cannot be simplified** to a marginal:\n\n$$\nP(A_{S'} \\ne y \\mid A_S = y) \\ne P(A_{S'} \\ne y)\n$$\n\nWe will include this derivation and explanation in the final version of the paper for clarity.\n\n**4. [Q4] What is semantic disjointness in Algorithm 1?**\n\nIn Algorithm 1, semantic disjointness refers to the requirement that the perturbed step and its subsequent steps are substantially different in meaning and reasoning logic from the original step. This constraint avoids trivial rewordings and ensures that the counterfactual chain represents a truly distinct reasoning path. We will clarify this in the algorithm description.\n\n**5. [Q5] Where do the unoptimized CoT traces come from? Are they human annotated, or elicited by zero-shot CoT prompting? If there is no comparison against other CoT optimization methods, how can I trust the initial unoptimized CoT traces are not deliberately made overthinking?**\n\nThank you for the question. The unoptimized CoTs used in our experiments are generated via zero-shot or few-shot prompting from strong language models (e.g., Qwen-2.5-72B-Instruct). These are produced using the same system prompt as in our Ours-ICL setting (see Appendix C), and **no human editing or annotation** is applied. To avoid bias, we use standard open-ended prompts and do not manually craft overthinking chains.\n\nWe have already included **multiple prompt-based baselines** for comparison (see **Table 2** in the paper main text and prompt templates in **Appendix C**). In this revision, we also added SPIRIT as a new baseline in the expanded **Table R1** (which extends Table 2 of the main paper).\n\n**6. [Q6] What is the setting for measuring accuracy in Table 1? Is it like in-context learning in Table 2 or fine-tuning in Table 3?**\n\nAccuracy is consistently computed as follows, and the setting corresponds to in-context learning as in Table 2 or fine-tuning as in Table 3, depending on the experiment.\n$\n\\text{Acc} = \\frac{\\text{Number of correct CoTs}}{\\text{Total number of CoTs in the dataset}}.\n$\n\nThe accuracy improvements in Table 1 arise from repeated applications of Algorithm 1 across the dataset, where each step in the CoT undergoes multiple rollouts. This significantly increases the likelihood of generating a correct answer. CoTs with PS < 1 may be ignored in a single run, but will be revisited in subsequent rollouts. Thus, PS enhancement is implicitly achieved via repeated generation and validation within each CoT execution unit.\n\n**7. [Q7] Table 3: Does “Original” refer to the model before SFT? Does “Noncausal” refer to unoptimized SFT data?**\n\nYes, exactly:\n\n- “**Original**” refers to the base model before any supervised fine-tuning (SFT).\n- “**Noncausal**” refers to the model fine-tuned on unfiltered, unoptimized CoTs.\n- “**Causal**” refers to the model fine-tuned on CoTs filtered and optimized via our PNS method.\n\n**8.[Q8] Line 97: “in Pearl” → “by Pearl”**\n\nThank you for catching this! We will correct “in Pearl” to “by Pearl” in the revised version."}, {"review_id": "w2vqIvAZx3", "reviewer": "Reviewer_Br1J", "summary": "The manuscript proposes to refine Chain of Thought (CoT) reasoning by scoring explanations by causality-inspired metrics designed to track the probabilities of necessity and sufficiency, as originally described by Pearl. This should encourage CoTs that are as short and informative as possible, ideally providing all and only the steps required to answer the relevant query. Experiments demonstrate the effectiveness of the method in improving reasoning in mathematical and common sense tasks using a range of LLMs.", "questions": "$\\textbf{Major comments}$\n\n-The causal allusion is suggestive, but it's not clear to me how it really applies if there is no counterfactual reasoning or structural dependence between variables (unless this is secretly tucked into the mysterious validator $V$ somehow?) Is there an abduction step somewhere that I'm missing? Put another way: would anything about this method be different if we replaced Pearl's probabilities of causation with standard conditional probabilities such as $p(a \\mid s, q)$ and $p(a' \\mid s', q)$? Perhaps this is what the causal vs. noncausal experiment in Table 3 is meant to summarize, but I was not clear on what exactly differentiates these two methods. It's very possible I've missed something here, and I would gladly revise my score upward if the authors could clarify the uniquely causal component of their procedure and/or explain how $V$ operationalizes PNS.\n\n-I expected to see some discussion of the complexity overheads associated with this approach (if not a theoretical analysis, at least a run time comparison). I assume that optimizing for PNS incurs some extra compute? This is potentially important given that CoT is already a very computationally intensive procedure.\n\n$\\textbf{Minor comments}$\n\n-How many rollouts per intervention are typically needed to get reliable PNS estimates in practice? \n\n-In Eq. 4, I assume that $q$ should be a conditioning event on the RHS? \n\n-There is a whole literature on necessity and sufficiency for XAI that may be relevant here (see links below).\n\n-https://proceedings.mlr.press/v177/beckers22a.html\n\n-https://link.springer.com/article/10.1007/s11023-022-09598-7\n\n-https://link.springer.com/article/10.1007/s10849-022-09377-8\n\n-https://dl.acm.org/doi/10.1145/3448016.3458455\n\n-https://dl.acm.org/doi/10.1145/3351095.3372850", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "$\\textbf{Strengths}$: the paper is clear and easy to follow. The central idea is simple and compelling. It stands to reason that users would generally prefer CoTs that maximize the probability of necessity and sufficiency (PNS) over those that do not, all else being equal. The empirical results are strong.\n\n$\\textbf{Weaknesses}$: while the inspiration is clear enough, the execution appears rather heuristic, as far as I can tell. The \"validation model\" $V$ is doing all the heavy lifting here – we literally just equate PNS with the complement of its average – but I see no reason to believe that $V$ is actually measuring PNS, even indirectly (see comments below).", "was_revised": true, "final_justification": "The authors did an excellent job responding to my comments, as well as those of the other reviewers. I had missed some important aspects of the contribution in my original review that were fully clarified during the rebuttal stage. The extra experiments were also quite helpful. In light of these updates, I have decided to revise my score upward.", "rebuttal": "**1.[Q1] How does your method implement causal reasoning—if there is no explicit structural dependence or counterfactual modeling? Would the results differ if standard conditional probabilities were used instead?**\n\n**(1) Why Our Method Is Counterfactual**\n\nCounterfactual reasoning follows the “**abduction–intervention–prediction**” paradigm. In our case:\n\n- **Abduction** is implicitly simulated by fixing the upstream context $(s_{<t}, q)$ for all rollouts.\n- **Intervention** is performed by replacing a single reasoning step $s_t$ with a semantically disjoint alternative $\\bar{s}_t$.\n- **Prediction** is executed by generating a full downstream reasoning chain from the modified input and evaluating the outcome.\n\nWhile we do not build a causal graph explicitly, this process **mimics counterfactual inference** by isolating a cause ($s_t$) and observing the impact of its intervention ($\\bar{s}_t$) on the final answer. This is distinct from simply computing conditional probabilities like $p(a \\mid s, q)$ or $p(a’ \\mid \\bar{s}, q)$, which lack an explicit notion of intervention or structural manipulation. In our setting, we **actively intervene** on the step and generate downstream consequences.\n\n**(2) How PNS Is Computed Operationally**\n\nWhile PNS is theoretically defined in terms of counterfactual probabilities, it is **not directly computable**. In our paper, we derive an empirical approximation (see paper Equation 10):\n\n$$\n\\mathrm{PNS}(\\mathbf{S}, \\overline{\\mathbf{s}}\\_t, \\mathbf{q}) \\approx 1 - \\frac{1}{k} \\sum_{i=1}^{k} V(\\overline{\\mathbf{S}}^{(i)})\n$$\n\nHere’s how we compute it in practice:\n\n1. We perform a **counterfactual intervention** on a single step $s_t$ by replacing it with a semantically disjoint alternative $\\bar{s}_t$ (i.e., not a trivial paraphrase).\n2. From the alternative step, we use a large language model to generate a full reasoning chain $\\bar{S}_{>t}$.\n3. We apply a validator $V$ to check whether the counterfactual reasoning still leads to the correct final answer.\n\nThe validator $V$ is **not a static function, but rather an operation** that executes the new reasoning path and checks its correctness.\n\nWe repeat this process for $k$ different $\\bar{s}_t^{(i)}$, and estimate the **counterfactual success rate**:\n\n$$\n\\frac{1}{k} \\sum_{i=1}^{k} V(\\overline{\\mathbf{S}}^{(i)})\n$$\n\nThis success rate quantifies **how easily the original step $s_t$ can be replaced without affecting the final outcome**:\n\n- A high success rate implies that $s_t$ is not necessary—it can be replaced with alternatives that still lead to the correct result.\n- A low success rate suggests that $s_t$ is causally necessary—its absence results in incorrect answers.\n\nThus, the complement term 1 - $\\text{success rate}$ serves as an empirical estimate of the PNS value for step $s_t$.\n\n**(3) Validator Accuracy and Robustness**\n\nTo assess the reliability of the validator $V$, we conduct experiments varying both the underlying LLM and the number of rollouts $k$. The table below shows the **mean absolute error** between estimated PNS values and ground-truth, where ground-truth PNS is set to 1 based on a reasoning chain judged by experts to be both necessary and sufficient.\n\n**Table R1.** Validator Accuracy Across LLMs and Rollout Sizes\n>Values indicate the mean absolute error $| \\widehat{\\mathrm{PNS}} - \\mathrm{PNS} |$ across evaluation samples. Lower is better.\n\n| **Validator** | **k = 1** | **k = 3** | **k = 5** | **k = 10** |\n| ------------- | --------- | --------- | --------- | ---------- |\n| Qwen-2.5-72B-Instruct  | 0.315     | 0.187     | 0.142     | 0.116      |\n| Qwen-2.5-7B-Instruct   | 0.411     |  0.395   |  0.315   |  0.293    |\n| GPT-4o        | 0.137     |   0.114  | 0.090     | 0.050      |\n\n- Increasing $k$ improves the stability and reliability of the PNS estimate.\n- Stronger LLMs used as validators yield more accurate evaluations.\n- The validator $V$ is crucial for enabling **counterfactual analysis** in practice.\n\nWe will include this explanation in the revised manuscript.\n\n**2.[Q2] Analysis of the Number and Complexity of Tokens for Complete PNS Reasoning.**\n\n(1) We **added a new experiment** (see **Table R2**) showing token cost and rollout complexity across LLMs (e.g., Qwen2.5) and LRMs (e.g., DeepSeek R1) during PNS filtering.\n\n(2) We also compare average tokens per CoT before and after PNS-based fine-tuning (see **Table 3 in the main paper)**, demonstrating reduced inference cost and improved conciseness.\n\n- Across three different LLM families, we observe a consistent trend: while PNS filtering introduces a moderate one-time token cost during data preparation, it yields over 20% token savings during inference.\n- The accuracy of all models is either maintained or improved, confirming that the CoTs retained after causal filtering are both concise and effective.\n- This supports our central claim: PNS filtering reshapes the training distribution, enabling the model to generate more efficient and logically minimal reasoning paths—without requiring inference-time pruning.\n\n**Table R2**. Token usage and complexity during PNS filtering\n| **Model (Base)**            | **Dataset**       | **Rollouts** | **Avg. Tokens** | **Time Complexity** |\n|----------------------------|-------------------|--------------|------------------|---------------------|\n| Qwen-2.5-72B-Instruct | GSM8k             | 9.77         | 1,104.42         | $O(k · n²)$          |\n|                            | CommonsenseQA     | 24.84        | 2,945.61         | $O(k · n²)$           |\n|                            | MATH-500          | 21.66        | 2,315.26         | $O(k · n²)$          |\n|                            | AIME              | 61.81        | 12,145.87        | $O(k · n²)$         |\n| DeepSeek R1            | GSM8k             | 22.62        | 2,861.12         | $O(k · n²)$         |\n|                            | CommonsenseQA     | 30.81        | 4,501.99         | $O(k · n²)$         |\n|                            | MATH-500          | 46.16        | 9,855.70         | $O(k · n²)$         |\n|                            | AIME              | 660.90       | 371,430.57       | $O(k · n²)$          |\n> *Rollouts*: Average number of step-level interventions (regenerations) per CoT.  \n*Avg. Tokens*: Average total number of tokens generated per example during PNS filtering.\n\nOverall, our results show that the proposed method **reduces reasoning redundancy**, trading a one-time overhead for sustained improvements in inference efficiency.\n\n**3.[Q3] How many rollouts per intervention are typically needed to get reliable PNS estimates in practice?**\n\nAs shown in **Table R1(above)**, we conducted additional evaluations to assess how the number of rollouts  $k$  affects the stability of PNS estimates. Both theoretical considerations and empirical results indicate that increasing $k$ improves estimation reliability. In practice, we found that using $k = 3 \\sim 5$ strikes a good balance between computational cost and accuracy. We will clarify this design choice in the final version.\n\n**4.[Q4] In Equation 4, should q be included as a conditioning variable on the right-hand side?**\n\nThank you for pointing this out. Yes, all probability expressions in our method are implicitly conditioned on the question $q$ (e.g., Equation 4). We followed the standard convention of omitting global conditions like the input prompt for notational simplicity. In the revised manuscript, we will explicitly clarify that all probability terms are conditioned on $q$, unless otherwise specified.\n\n**5.[Q5] There is a whole literature on necessity and sufficiency for XAI that may be relevant here (see links below).**\n\nThank you very much for the valuable references — these are all excellent and inspiring works. We will incorporate them into the **Related Work** section of the revised version.\n>  Causal Necessity and Sufficiency in XAI. Our work relates closely to prior studies that explain model behavior via necessity and sufficiency [1–5]. For instance, LENS [1] identifies necessary and sufficient conditions for outputs better than mainstream XAI methods; Darwiche et al. [2] use Decision-DNNF circuits to compute all sufficient reasons efficiently; Mothilal et al. [3] generate diverse counterfactuals using determinant point processes; Beckers [4] formalizes sufficiency-based explanations for action guidance and fairness; Galhotra et al. [5] introduce LEWIS, a probabilistic counterfactual explanation method.\nIn contrast, our PNS evaluation targets LLM reasoning chains, using counterfactual rollouts to assess the causal necessity and sufficiency of CoT traces—offering a causal solution to the overthinking problem in LLMs."}]

Author rebuttal:

Meta-review: This paper presents a causal framework for optimizing CoT reasoning by evaluating step-wise sufficiency and necessity using PNS. Through a two-stage process—resampling to improve sufficiency and pruning via necessity—the method improves both reasoning accuracy and efficiency. Reviewers raised concerns about novelty, clarity, and evaluation, but the authors responded with detailed explanations, new experiments, and additional baselines. Reviewers acknowledged the thorough rebuttal and some of them raised their scores. Therefore I recommend acceptance.
Decision: Accept (poster)