Self-Para-Consistency: Improving Reasoning Tasks at Low Cost for Large Language Models (2024)

Abstract: Recently, the self-consistency decoding strategy has shown the ability to improve performance for complex reasoning tasks with large language models (LLMs). However, the costs may be high because the sampling process of the strategy generates some low-probability text, resulting in low-quality reasoning paths. As a consequence, it requires a relatively large sampling number to obtain good aggregation performance. In this paper, we propose an alternative strategy, self-para-consistency. It first generates multiple paraphrases for each test question, then generates reasoning paths for the original and all the paraphrased questions based on greedy decoding, and finally selects the most consistent answer. Since all the candidate paths have relatively high probabilities, the sampling number could be much smaller than the self-consistency strategy. Extensive experiments on complex reasoning datasets demonstrate the effectiveness of our method in reducing the sampling number.

heading: 1 Introduction
text: Recently, large language models (LLMs) like GPT3 were considered to have an emergent ability of Chain of Thought (CoT) prompting (Wei et al., 2022) to perform multi-step reasoning (Wei et al., 2023), though the term "emergent" is still debated (Schaeffer et al., 2023). While promising, the CoTs generated through greedy decoding may fall into local optima. To alleviate this problem, Wei et al. (2022) proposed to sample a diverse set of CoTs and then aggregate them through majority voting, as shown in Figure 1(a).
However, the self-consistency strategy may encounter a key challenge in reality, which is the
∗ Corresponding Author
high cost due to the sampling process producing low-probability reasoning paths and therefore needing a relatively large number to achieve considerable performance. For example, Wei et al. (2022) and Chen et al. (2022) set the sampling number to 40, making the sampling process expensive, especially for complex reasoning tasks with long reasoning paths.
To alleviate this problem, we need to achieve a better trade-off between the quality and the diversity of generated reasoning paths. In this paper, we propose an alternative strategy, referred to as self-para-consistency. Its key idea is to retain the advantage of greedy decoding, which is likely to have a higher average probability than the sampling process. Then the diversity comes from generating different paraphrases of the given question. The intuition of paraphrasing is that LLMs are shown to be sensitive to spurious features in the prompts (Sclar et al., 2023), and therefore may
14162
generate different text in response to paraphrases in different expressions.
As shown in Figure 1(b), the proposed self-paraconsistency strategy consists of three steps. We first prompt an LLM to generate paraphrases and then generate the corresponding reasoning path for each paraphrase and the original question through greedy decoding. The last step is the same as the self-consistency strategy, which is to find the most consistent answer via majority voting. We conduct extensive experiments on 6 reasoning datasets, of which the results demonstrate effectiveness in reducing the sampling number.
The contributions of this work are as follows:
• We propose the self-para-consistency strategy to improve the reasoning performance of LLMs, which is a lower-cost alternative to self-consistency.
• The self-para-consistency strategy can serve as a kind of robustness or uncertainty measurement method.
• The extensive experiments demonstrate the effectiveness of reducing the generation costs by reducing the sampling number.

heading: 2 Related Work
text: Recently, CoT Prompting has demonstrated impressive performance (Wei et al., 2022) but has two challenges. The first is the inconsistency between the reasoning path and the result. The second is the local optimality of the generated reasoning path through greedy decoding (Wang et al., 2022). To address the inconsistency problem, a line of work has employed program languages instead of natural language to depict the reasoning path, referred as program-of-thought (PoT) prompting (Chen et al., 2022; Gao et al., 2023; Lyu et al., 2023). To address the problem of local optimality, Wang et al. proposed the self-consistency strategy based on CoT prompting (Wang et al., 2022).
The Quality-Diversity Trade-Off. Due to the diversity of natural language, there is a quality-diversity trade-off in text generation systems (Montahaei et al., 2019; Zhang et al., 2021). Although tuning the temperature or other parameters of LLMs can switch on the quality-diversity trade-off curve, previous studies suggest another type of diversity improvement (Hu et al., 2017; Wang et al., 2017; Ruan et al., 2020; Shao et al.,
2021). By sampling a latent variable z, text generation models can further improve diversity while still using greedy decoding or beam search. Our method is inspired by this work, as we suppose that different prompts can involve extra diversity while still keeping considerable quality through greedy decoding.

heading: 3 Methodology

heading: 3.1 The Framework and Notation
text: As shown in Figure 1, the proposed self-paraconsistency strategy consists of three steps: 1) paraphrasing the given question into multiple paraphrases, 2) generating corresponding reasoning paths via greedy decoding, and 3) aggregating the answers based on majority voting. We formalize the three steps in the following paragraphs.
Given a testing question x, we first prompt an LLM (parameterized by θ) to generate k − 1 paraphrased questions Gpara = x′ii = 1k−1 where k > 1. The prompt for paraphrasing is denoted by Ipara, then the paraphrasing process can be formalized as:
Pθ(Gpara | x, Ipara) = k−1∏
i=1
Pθ(x′i | x, Ipara, G<ipara)
(1) where G<ipara denotes the subgroup containing the already generated i− 1 paraphrases. It means that all the k−1 paraphrases are generated sequentially. In this way, the LLM tends to generate different paraphrases.
In the second step, we collect the original question and the k− 1 generated paraphrases, and then prompt the LLM to generate k reasoning paths Rpath = rii = 1
k in parallel. This step can be formalized as:
Pθ(Rpath | x,Gpara, Iinst) = Pθ(r1 | x, Iinst) · k−1∏
i=1
Pθ(ri+1 | x′i, Iinst) (2)
where Iinst denotes the instruction prompt to generate reasoning paths. We denote r1 as the reasoning path for the original question x. Equation 2 means that the generation of reasoning paths can be parallelized. Greedy decoding is then performed following Equation 1 and 2. The diversity is no longer from the sampling process of the LLM’s decoder but comes from the paraphrasing process instead.
In the final step, the process is the same as with self-consistency (Wang et al., 2022). We also assume each reasoning path ri is coupled with the answer ai where ri → ai. This can be easily achieved because CoT prompting can generate the answer in the final tokens of ri, and PoT prompting can get the answer by executing the codes, which takes a majority vote over the answers aiki=1 by:argmaxa ∑k i=1 1 (ai = a).

heading: 3.2 Prompting Details
text: We then introduce the two prompts Ipara and Iinst in detail. We show a case in numerical reasoning in Figure 2 where k = 3 and the LLM needs to generate 2 paraphrases following the instruction Ipara in the first step. {question} in Ipara denotes the original question. {{text1}} and {{text2}} denote the placeholders for the output paraphrases.
In the second step, when prompting the LLM with Iinst, there is a slight difference between the paraphrased question x′i and the original question
x. As shown in Figure 2, for generating reasoning paths for paraphrased questions, we put both x′i and x into Iinst because only including x′i will lose some key information in practice (with GPT-3.5) due to the imperfect paraphrasing process. For the original question, we simply remove the line starting with "Rephrased Question: ", as shown in the left gray box of Figure 2.
It worth noting that Iinst can also be combined with in-context learning, where Iinst will consist of T few-shot demonstrations denoted by D′demo = {xi, x′i, ri}Ti=1 without the instruction text for the paraphrased question. For the original question, Iinst will consist of Ddemo = {xi, ri}Ti=1.

heading: 4 Experiments

heading: 4.1 Datasets
text: We conducted experiments on 5 reasoning datasets, which we categorized into three main classes: (1) The in-distribution numerical reason-
ing datasets, comprising GSM8K (Cobbe et al., 2021), SVAMP (Patel et al., 2021), and ASDIV (Miao et al., 2020). (2) An out-of-distribution numerical dataset, GSM8K-hard, modified from GSM8K where the numbers in the questions were replaced with significantly larger values (Gao et al., 2023). (3) A symbolic reasoning dataset, date understanding sourced from BIG-bench (Suzgun et al., 2022), requiring inference of dates based on contextual information.
We realize that there are more reasoning datasets in previous work (Wang et al., 2022; Chen et al., 2022; Gao et al., 2023) like MAWPS (Koncel-Kedziorski et al., 2016), but those datasets are relatively easy for GPT-3.5 with PoT prompting, making them less distinguishable for different methods.

heading: 4.2 Baselines
text: Since PoT prompting has outperformed CoT prompting in complex reasoning tasks and PoT can obtain more consistent output formats than CoTs, the basic baseline in this paper is PoT prompting, also known as PAL (Gao et al., 2023). Based on PoT, the self-consistency strategy is performed with different temperatures T and sampling number k. The sampling number of our method, self-para-consistency, is set to k = 3 as we do not want to increase the costs and k = 3 is the smallest number to perform majority voting.
The GPT-3.5-turbo-0613 version is used for both baselines and our method. The temperature of our method is set to 0.0 in both the paraphrasing and reasoning stages. For date understanding, we use the same few-shot PoT examples as (Gao et al., 2023).

heading: 4.3 Main Results
text: The results of our method and baseline methods are shown in Table 1 and 2. Overall, our method, self-para-consistency, achieves the best performance on 4 out of 5 datasets.
For numerical reasoning, Self-ParaConsistencyk=3 improves the accuracy of the baseline Zero-Shot-PAL by 7.0, 5.0, and 9.5 points on GSM8K, SVAMP and GSMHARD, respectively. For date understanding, Self-ParaConsistencyk=3 improves the accuracy of the baseline Zero-Shot-PAL by 1.8 points.
Compared with Self-ConsistencyT=0.7,k=5, Self-Para-Consistencyk=3 outperforms it on all 5 datasets. Compared with SelfConsistencyT=0.7,k=10, Self-Para-Consistencyk=3 outperforms it on 4 out of 5 datasets with 7 less reasoning paths required.
Compared with different datasets, our method improves most on GSM-HARD, which is typically
an OOD dataset where the training data of LLMs do not contain such large numbers. It shows that paraphrasing may introduce extra improvements for OOD data.

heading: 4.4 Analysis of the Paraphrasing Process
text: A concern of this work is that the paraphrased question may not always be faithful to the original question, which may cause error accumulation for downstream tasks. To alleviate the problem, we concatenate the original and the paraphrased questions in the prompt as shown in Figure 2, which gives the LLM an opportunity to answer correctly even if the paraphrased question is wrong or incomplete.
We show a case in Figure 3 where the paraphrased questions miss some necessary information. The baseline Self-ConsistencyT=1.0,k=5 yields low-quality reasoning paths and then gets a wrong answer. Our method Self-ParaConsistencyk=3 can tolerate imperfect paraphrasing and finally answer correctly.

heading: 4.5 Limitations and Future Work
text: The paraphrasing process may have more applications that need to be explored. In the future, we would like to explore the following scenarios: 1) Integrating our method with self-consistency. For instance, generating k1 paraphrases where each re-
sults in k2 sampled decoding paths, resulting in a total of k1 × k2 paths. This approach could potentially achieve a more favorable balance between quality and diversity in the decoding paths. 2) Combining our method with a paraphrase verifier to eliminate low-quality paraphrases, ensuring only the most accurate versions are used. 3) Employing self-para-consistency as an indicator of uncertainty, which could serve as a measure of the robustness of LLMs in reasoning tasks.

heading: 5 Conclusion
text: In this paper, we propose the self-para-consistency strategy, which serves as an alternative to the selfconsistency strategy with lower costs. The proposed method first prompts the LLM to generate multiple paraphrases sequentially, and then generate reasoning paths in parallel. The diversity comes from paraphrasing instead of nondeterministic decoding strategies. Extensive experiments show the effectiveness of the proposed method.

heading: Acknowledgments
text: This work was supported by the National Natural Science Foundation of China under Grant 62306344, and Natural Science Foundation of Guangdong Province of China under Grant 2024A1515010253.

heading: Future Work
text: In the future, we would like to explore the following scenarios: 1) Integrating our method with self-consistency. For instance, generating k1 paraphrases where each re-
sults in k2 sampled decoding paths, resulting in a total of k1 × k2 paths. This approach could potentially achieve a more favorable balance between quality and diversity in the decoding paths. 2) Combining our method with a paraphrase verifier to eliminate low-quality paraphrases, ensuring only the most accurate versions are used. 3) Employing self-para-consistency as an indicator of uncertainty, which could serve as a measure of the robustness of LLMs in reasoning tasks