Self-Harmonized Chain of Thought (2025)

Abstract: Chain-of-thought (CoT) prompting has demonstrated the capacity of large language models to perform complex reasoning through intermediate steps. While effective, current CoT methods face challenges: Zero-shot-CoT can lead to reasoning errors, and Few-shot-CoT requires labor-intensive manual demonstrations. Auto-CoT attempts to address these issues by automatically generating diverse demonstrations, but this diversity can lead to inconsistent reasoning patterns. We propose ECHO (Self-Harmonized Chain of Thought), a novel method that unifies diverse solution paths into a consistent and effective reasoning pattern. ECHO employs an iterative process to refine and harmonize automatically generated demonstrations, mitigating the limitations of existing approaches. Our comprehensive experiments across arithmetic, commonsense, and symbolic reasoning tasks demonstrate that ECHO outperforms Auto-CoT by an average of 2.8%. These findings suggest that ECHO represents a significant step towards more robust and generalizable automated reasoning in large language models.

## introduction large language models (llms) have demonstrated remarkable capabilities in various natural language processing tasks. however, their performance on complex reasoning tasks has been a persistent challenge. a recent technique known as chain-ofthought (cot) prompting @xcite has significantly enhanced llms' ability to tackle such tasks by decomposing complex problems into a series of intermediate steps. cot methods typically employ two prompting paradigms: zero-shot-cot @xcite and @xcite . zero-shot-cot uses a universal prompt like "let's think step 1 our code is available: https://github.com/xalp/ echo . ## why does it work? in this section, we offer some insight into why the proposed approach, echo, can be effective by presenting a mathematical framework for understanding its iterative refinement process. research question 1: why will regenerate rationale using others as demonstrations result in convergence? few-shot-cot demonstrates that models naturally follow the patterns shown in few-shot demonstrations, providing reasoning steps before producing the final answer. subsequent research, such as pal @xcite , extends this concept by using code rationales to generate code for solving math problems. these studies collectively show that models not only follow the general questionanswering structure but also inherit the patterns from these demonstrations. research question 2: why will regenerate rationale using others as demonstrations yield better rationale? consider a set of questions @xmath0, q (2) , . . . , q (m) }. assume that we can use zero-shot-cot to arrive at the rationales (with answers): @xmath1, r (2) , . . . , r (m) }. auto-cot performs the following: first, it constructs demonstrations @xmath2, d (2) , . . . , d (m) }, where d (k) = q (k) • r (k) . during the inference stage, such demonstrations are then used as fewshot examples for generating "refined" rationales for the i-th instance, based on d\d (i) , which we denote as r empirically, auto-cot was shown to yield better results than zero-shot-cot. mathematically, this leads to the following hypothesis: in practice, we can keep refining the set of rationales by reconstructing them. specifically, we can first construct repeating the above process, we arrive at r 1 = {r if the above hypothesis for auto-cot is true, it is not unreasonable to state the following hypothesis as its extension: we can repeat the above process t times to arrive at a chain of inequalities: this chain of inequalities argues why our proposed approach works empirically. each iteration of refinement is likely to produce a set of rationales that are at least as good as, if not better than, the previous set. it's important to note that while the above process updates the rationales in a batch mode, in our echo method, we adopted an online approach. in this approach, we utilize the recently updated rationales for the next instance within the same iteration. this online approach allows for more immediate incorporation of improvements, potentially leading to faster convergence to high-quality rationales. however, this behaviour may also negatively impact model performance, as some patterns are specific to particular questions and cannot be generalized effectively to others (we demonstrate performance on combined dataset in section 6.7). ## conclusion in this work, we propose a novel method called echo for improving chain-of-thought prompting in large language models. we have shown that our method results in consistentimprovements in three domains, confirming the feasibility and significance of adopting the self-harmonization mechanism in the cot prompting process. we also carried out extensive experiments and case studies to understand the behavior of the proposed approach. we hope that these insights can help to improve future automatic reasoning frameworks.