Mitigating Demonstration Bias through Global Coevolutionary Reasoning (2025)

Abstract: Recent advances in large language models (LLMs) have demonstrated the effectiveness of chain-of-thought (CoT) prompting. Few-Shot-CoT relies on task-specific, manually labeled demonstrations, limiting its generalization to unseen tasks. While Zero-Shot-CoT eliminates this reliance, it often underperforms. To address this, existing methods aim to automatically generate demonstrations in zero-shot settings. However, these generated demonstrations face challenges due to demonstration bias: 1) selected demonstrations may contain errors, and 2) they may not be suitable or representative enough for all questions. To mitigate these biases, we propose Global Coevolutionary Reasoning (GCR). The method first applies Zero-Shot-CoT to answer all questions, then clusters the results. For each cluster, a random sample is selected, and these selected samples serve as demonstrations for each other. The model then iteratively re-answers the questions and updates their rationales based on these demonstrations, enabling coevolutionary reasoning to progressively improve the quality of the answers. This process of random sampling and coevolutionary reasoning is repeated until all questions have been re-answered. Experimental results on ten datasets using GPT-3.5-turbo and GPT-4o-mini show that GCR outperforms baseline methods without any performance degradation caused by demonstration bias. Additionally, GCR is orthogonal to existing methods and can be seamlessly integrated with them. The code is available at: https://github.com/GouChuan/GCR.

## introduction the paradigm of in-context learning (icl) @xcite has proven effective in large language models (llms), enabling them to perform reasoning tasks based on a few examples. among the strategies within icl, chain-of-thought (cot) (wei et al., 2022) reasoning, including few-shot and zeroshot variants, is a cornerstone for tackling complex problems. few-shot-cot relies on manually labeled demonstrations, limiting generalization to unseen tasks, while zero-shot-cot @xcite removes this reliance, offering broader applicability but often resulting in suboptimal reasoning. the auto-cot @xcite ) framework addresses these challenges by dynamically transitioning from zero-shot to few-shot-cot. it clusters questions based on semantic similarity and selects representative examples from the cluster centroids to construct a few-shot prompt, combin-q: a car finishes a journey in 20 hours at the speed of 60 km/hr. if the same distance is to be covered in 10 hours, how much speed does the car gain? ... q: at what price should the karan mark a sewing machine that costs him rs. 1200/-so that even after offering a 20% discount, he makes 20% profit? ... q: the capacity of a tank of dimensions (8 m × 6 m × 2.5 m) is? ... q: 900 + 5 × 12 = ? … q: joe's age, joe's sister's age and joe's father's age sums up to 100. when joe is as old as his father, joe's sister will be twice as old as now. when joe is as old as his father then his father is twice as old as when his sister was as old as her father. find the age of joe's father? a: sorry, i am unable to solve this type of word problem. ## experimental results the experimental results are displayed in arithmetic reasoning: gcr achieves the best average performance compared with all baseline models, indicating the superiority of our method. compared with the competitive baseline echo, gcr outperforms it by an average of 4.9% with gpt-3.5-turbo and 0.6% with gpt-4o-mini. the largest improvement is observed in gsm8k (+10.8%) and aqua (+10.6%) under gpt-3.5-turbo, and in aqua (+3.2%) under gpt-4o-mini. one possible reason is that these datasets suffer from two types of demonstration bias in the demonstrations automatically selected by auto-cot, namely, the selected demonstrations are not sufficiently representative or contain errors. commonsense reasoning: consistent improvement is observed in commonsense reasoning tasks. gcr outperforms all baseline models across both csqa and strategyqa. compared with auto-cot, gcr improves the average performance by 7.1% under gpt-3.5-turbo and 0.5% under gpt-4o-mini, demonstrating its effectiveness in leveraging external knowledge and logical inference. these results suggest that gcr enhances the ability to reason about implicit knowledge and abstract relationships, making it more robust for real-world commonsense tasks. symbolic reasoning: gcr also achieves better results than auto-cot in symbolic reasoning tasks. notably, due to the absence of a judge mechanism to ensure the iterative process moves in the correct direction in echo (as shown in figure 3 ), echo experiences a sharp performance drop on the lastletter dataset in gpt-3.5-turbo compared to auto-cot (82.6 % → 66.4 %). however, gcr and echo+judge, which incorporate the judge process during iterations, do not exhibit such a performance decline. these results indicate that gcr addresses the demonstration bias present in auto-cot and exhibits strong robustness, avoiding sharp performance declines. ## analysis another way to judge to further validate the generalizability of gcr, we replaced the closedsource model's direct judgment-based method with the judge method designed for open-source models. we adopt the probability disparity proposed in @xcite to assess the quality of a rationale. the open-source model used is mistral_7b_instruct_v3. the probability disparity of a rationale is defined as: here, r represents a rationale, and x 1 t and x 2 t are the top two tokens at the t-th decoding step, chosen based on their maximum post-softmax probabilities from the vocabulary, with x t being part of the answer tokens. to avoid blind confidence, if the probability disparity for a token exceeds a threshold (in this section, 0.75), it is disregarded in the calculation. ## conclusion this paper presents a method for global coevolutionary reasoning, referred to as gcr, where samples are clustered based on rationales obtained through zero-shot-cot. within each cluster, samples are randomly selected using p-sampling, and coevolutionary iterations are performed where samples act as demonstrations for one another. this approach addresses the demonstration bias inherent in auto-cot, enhancing reasoning performance in the absence of manually designed demonstrations. furthermore, it can be integrated with existing methods for reasoning tasks.