Towards A Unified View of Answer Calibration for Multi-Step Reasoning (2024)

Abstract: Large Language Models (LLMs) employing Chain-of-Thought (CoT) prompting have broadened the scope for improving multi-step reasoning capabilities. We generally divide multi-step reasoning into two phases: path generation to generate the reasoning path(s); and answer calibration post-processing the reasoning path(s) to obtain a final answer. However, the existing literature lacks systematic analysis on different answer calibration approaches. In this paper, we summarize the taxonomy of recent answer calibration techniques and break them down into step-level and path-level strategies. We then conduct a thorough evaluation on these strategies from a unified view, systematically scrutinizing steplevel and path-level answer calibration across multiple paths. Experimental results reveal that integrating the dominance of both strategies tends to derive optimal outcomes. Our study holds the potential to illuminate key insights for optimizing multi-step reasoning with answer calibration.

heading: 1 Introduction
text: Chain-of-Thought (CoT) prompting (Wei et al., 2022) has significantly improved multi-step reasoning capabilities of Large Language Models (LLMs) (Zhao et al., 2023b; Qiao et al., 2023). As seen from Figure 1, the process of multi-step reasoning generally contains two primary modules: reasoning path generation which generates one or multiple reasoning paths (Fu et al., 2023; Yao et al., 2023b); and answer calibration which post-processes the reasoning path(s) to calibrate the initial output (Wang et al., 2023f; Zhao et al., 2023a).
In practice, answer calibration is pluggable and can be integrated into path generation models. The answer calibration framework can be divided into step and path levels, applicable to single or multiple paths, as illustrated in Figure 1. For step-
∗ Corresponding Author.
level answer calibration on a single path, the model rectifies errors in intermediate-step answers of a generated path (Zhao et al., 2023a). For step-level answer calibration on multiple paths, the model verifies each intermediate-step answer (Weng et al., 2023) or aggregates the correct step answers (Cao, 2023) from multiple paths. For path-level answer calibration on a single path, the model revises the entire rationale to obtain the correct answer (Baek et al., 2023). For path-level answer calibration on multiple paths, the model produces a result indicating the consensus of all candidate paths (Wang et al., 2023f; Yoran et al., 2023). As answer calibration can identify and rectify errors in the reasoning path, or even holistically utilize multiple candidate paths, it plays a vital role in multi-step reasoning to ensure a precise, consistent and reliable reasoning process (Pan et al., 2024).
However, we argue that the crucial factors driving the success of answer calibration strategies remain obscure, with a comprehensive systematic
25
analysis still underexplored. To bridge the gap, our study investigates: (1) The specific conditions where answer calibration notably boosts multistep reasoning performance; (2) The strengths and weaknesses of step-level versus path-level answer calibration, and the pathway to attaining optimal performance; (3) The robustness and generalizability of answer calibration strategies.
To address these questions, we dissect cuttingedge answer calibration techniques for multi-step reasoning with LLMs, and introduce a unified framework that elucidates step-level and path-level strategies. We define two thresholds to respectively signify the step-level and path-level dominance in the unified framework. We then undertake a comprehensive evaluation of answer calibration strategies, w.r.t. accuracy, faithfulness, informativeness, consistency, and perplexity over steps or paths. Through rigorous experiments on five representative multi-step reasoning tasks involving arithmetic (Ahn et al., 2024) and commonsense, we find that: (1) employing answer calibration can enhance accuracy, with the improvement being more noticeable in zero-shot scenarios (§4.2) and less significant on stronger backbone models (§4.4); (2) The optimal performance of the unified answer calibration strategy typically achieved by synthesizing step-level and path level dominance (§4.3); (3) path-level answer calibration is more beneficial in improving accuracy, and step-level answer calibration is more effective for mitigating low-quality prompting (§4.5); (4) answer calibration can improve consistency on arithmetic tasks but weakens faithfulness, informativeness and perplexity on both arithmetic and commonsense tasks (§4.6).

heading: 2 Related Work
text: Reasoning Path Generation. Previous methods for reasoning path generation mostly focus on two aspects to improve reasoning process, including refining input query or prompts (input refinement) and polishing the reasoning path (rationale polish).
As for input refinement, Zero-shot CoT (Kojima et al., 2022) and Few-shot CoT (Wei et al., 2022) are classic methods to elicit multi-step reasoning ability of LLMs, with “Let’s think step by step” prompts. To decouple planning and execution, Wang et al. (2023d); Sun et al. (2024) devise a plan by prompting and divide and conquer multistep tasks. To enrich prompts, Wang et al. (2024a) leverage structure triples as evidence, Kong et al.
(2024) design role-play prompting, and Xu et al. (2023) employ re-reading instructions. Besides, LLM performance can also be affected by prompt complexity (Fu et al., 2023) and formats, such as program (Gao et al., 2023; Chen et al., 2023b; Sel et al., 2024; Jie et al., 2023; Lei and Deng, 2023; Bi et al., 2024) and table (Jin and Lu, 2023). Further, Wang et al. (2023b); Shi et al. (2023); Liang et al. (2024) propose to adaptively utilize prompts. Apart from refining prompts, Xi et al. (2023b) progressively refine the given questions, Wang et al. (2024d) convert semantically-wrapped questions to meta-questions, and Jie and Lu (2023) augment training data with program annotations.
In terms of rationale polish, recent work mainly focus on step-aware training (Wang et al., 2023g) and path-level optimization. For step-aware training, Zhang et al. (2023) introduce step-by-step planning and Lee and Kim (2023) recursively tackle intermediate steps; Jiang et al. (2023) reconstruct the reasoning rationale within prompts by residual connections; Paul et al. (2024) iteratively provide feedback on step answers; Lanchantin et al. (2023) leverage self-notes as intermediate steps and working memory; Li et al. (2023); Ling et al. (2023); Lightman et al. (2024) propose to verify on intermediate step answers; Li et al. (2024); Wang et al. (2023c) process step-aware verification by knowledge base retrieval. For path-level optimization, Li and Qiu (2023) enable LLMs to self-improve via pre-thinking and recalling relevant reasoning paths as memory; Wang et al. (2024b); Yue et al. (2024) leverage hybrid rationales in formats of natural language and program. Some work also generate deliberate rationales beyond CoT, such as Tree-ofThought (Yao et al., 2023b; Long, 2023), Graph-ofThought (Yao et al., 2023d; Besta et al., 2023) and Hypergraph-of-Thought (HoT) (Yao et al., 2023a).
Answer Calibration. Given generated reasoning path(s), answer calibration methods post-process the path(s) to calibrate the answer, involving stepor path-level calibration on one or multiple path(s).
Step-level answer calibration. Xue et al. (2023); Cao (2023) propose to rectify factual inconsistency and reasoning logic between intermediate steps. Miao et al. (2024); Wu et al. (2024) check the correctness of each intermediate step. Zhao et al. (2023a) post-edit multi-step reasoning paths with external knowledge. Yao et al. (2023c); Hao et al. (2023); Shinn et al. (2023); Yao et al. (2024); Chen et al. (2023a); Aksitov et al. (2023) draw up a plan
and act step by step with LLMs as agents (Wang et al., 2024c; Xi et al., 2023a), encouraging interaction with the environment to provide feedback. Weng et al. (2023); Jiang et al. (2024) unleash the self-verification ability of LLMs, by forward reasoning and backward verification on intermediate step answers. Zhou et al. (2024) propose codebased self-verification on reasoning steps.
Path-level answer calibration. Zelikman et al. (2022) present a self-taught reasoner to iteratively generate rationales. Zheng et al. (2023) progressively use the generated answers as hints to make double-check. Mountantonakis and Tzitzikas (2023) enrich generated reasoning paths with hundreds of RDF KGs for fact checking. Baek et al. (2023) iteratively rectify errors in knowledge retrieval and answer generation for knowledgeaugmented LMs. To cultivate the reasoning ability of smaller LMs, Ho et al. (2023); Wang et al. (2023e,h) propose to fine-tune CoT for knowledge distillation. Huang et al. (2023) demonstrate that LLMs can self-improve with high-confidence rationale-augmented answers. Yoran et al. (2023) prompt LLMs to meta-reason over multiple paths. Liu et al. (2023); Madaan et al. (2024) leverage feedback to improve model initial outputs. Wan et al. (2023) adaptively select in-context demonstrations from previous outputs to re-generate answers. Wang et al. (2023f) leverage self-consistency decoding strategy to majority vote on multiple path answers. Aggarwal and Yang (2023) propose adaptive-consistency to reduce sample budget.

heading: 3 Comprehensive Analysis of Answer Calibration

heading: 3.1 Formulation of Answer Calibration
text: Given a question denoted as Q and its associated prompt P , we leverage the LLM to generate the resultR. R can either encompass a single reasoning path P with an initial answer A or multiple reasoning paths P = {Pi}i∈[1,N ] with a corresponding answer set A = {Ai}i∈[1,N ]. The total number of paths in P is N . In this paper, we analyze under the assumption that each reasoning path comprises a maximum ofM steps. Paths exceedingM steps are truncated, and those with fewer steps are padded. The intermediate step answers for each reasoning path P(i) are represented as {aj}(i)j∈[1,M ].
Step-Level Answer Calibration. Given a single reasoning path P with an initial final path answer
A and intermediate step answers {aj}j∈[1,M ], the objective of step-level answer calibration is to rectify any erroneous aj , so that deriving the correct A. For multiple reasoning paths P, step-level answer calibration seeks to either select the reasoning path with the maximum correct intermediate step answers or aggregate the verified correct steps to form the most accurate reasoning path, leading to a correct final path answer. Self-verification (Weng et al., 2023) is an effective approach for step-level answer calibration on multiple reasoning paths.
Path-Level Answer Calibration. Given a single reasoning path P with an initial final path answer A, the goal of path-level answer calibration is to revise the wrong A. For multiple reasoning paths P = {Pi}i∈[1,N ] with corresponding answers A = {Ai}i∈[1,N ], path-level answer calibration is designed to select the reasoning path from P with the most consistent answer in A. Self-consistency (Wang et al., 2023f) is a widely-used efficacious technique for path-level answer calibration on multiple reasoning paths.

heading: 3.2 Unified View of Answer Calibration
text: Considering the advantages of both step-level and path-level answer calibration, we propose to integrate the two strategies on multiple paths. Given the multiple generated reasoning paths P = {Pi}i∈[1,N ], we define a unified score Di for each Pi (with the final path answer: Ai and intermediate step answers: {aj}(i)j∈[1,M ]):
Di = α ni N︸︷︷︸
path−level
+ (1− α)mi M︸ ︷︷ ︸
step−level
(1)
where ni ∈ [1, N ] is the frequency of Ai existing in A, mi ∈ [0,M ] is the number of correct intermediate steps in Pi, and α is a hyper-parameter. The final answer is Ai∗ satisfying i∗ = argmax
i∈[1,N ] (Di).
To better analyze the effects of varying α in the unified framework, we then define particular choices for α which we call step and path level dominant answer calibration.
Definition 1. Step-Level Dominant Answer Calibration: This choice refers to the level of α at which the step-level score is used as the dominant criterion, with the path-level score given much smaller weight and only serving to break ties when necessary. Specifically, larger mi always results
in larger Di, no matter how small ni is. We denote this as: ∀nj , nk ∈ [1, N ] and mj ,mk ∈ [0,M ],where nj < nk and mj > mk, the scores Dj and Dk should satisfy
α nj N + (1− α)mj M > α nk N + (1− α)mk M
Thus we can obtain
α < 1
M(nk−nj) N(mj−mk) + 1
(2)
If Eq (2) is constant, we can infer that
α < min
  1
M(nk−nj) N(mj−mk) + 1
  = 1
M max(nk−nj) N min(mj−mk) + 1
(3) As 1 ≤ nj < nk, nj+nk ≤ N , and 0 ≤ mk < mj , we can deduce that min(mj−mk) = 1, max(nk− nj) = N − 2. From the above, we deduce:
α < 1
M(N−2) N + 1
(4)
Definition 2. Path-Level Dominant Answer Calibration: For this choice, Di gives priority to the path-level score, with the step-level score given much smaller weight and only serving to break ties when necessary. Concretely, larger ni always conduces larger Di, no matter how small mi is. We denote this as: ∀nj , nk ∈ [1, N ] and mj ,mk ∈ [0,M ],where nj > nk and mj < mk, the scores Dj and Dk should satisfy
α nj N + (1− α)mj M > α nk N + (1− α)mk M
Analogously, we can obtain
α > 1
M(nj−nk) N(mk−mj) + 1
(5)
If Eq (5) is constant, we can infer that
α > max
  1
M(nj−nk) N(mk−mj) + 1
  = 1
M min(nj−nk) N max(mk−mj) + 1
(6) As 1 ≤ nk < nj , and 0 ≤ mj < mk ≤ M , we deduce that min(nj −nk) = 1, max(mk−mj) = M − 0 =M . From the above, we deduce:
α > 1
1 N + 1
(7)
In general, considering step-level and path-level answer calibration dominance, we can obtain two thresholds: 1M(N−2)
N +1 and 11 N +1 . Note that α = 0
and α = 1 are respectively equivalent to the self-verification and self-consistency strategies.

heading: 3.3 Evaluation of Answer Calibration
text: Calculation of ROSCOE Scores. In addition to the classical evaluation metric: Accuracy, Golovneva et al. (2023) have proposed ROSCOE, a suite of metrics for multi-step reasoning, under four perspectives: semantic alignment (ROSCOE-SA), semantic similarity (ROSCOE-SS), logical inference, and (ROSCOE-LI) and language coherence (ROSCOE-LC). Due to space limits, we select some representative scores from ROSCOE as evaluation metrics in the experiments.
Given source ground truth rationale (s) and generated rationale (h) with multiple steps (hi), we calculate five scores (All scores satisfy the principle that larger is better):
(1) Faithfulnessstep (h→ s): To assess whether the model misconstrues the problem statement, or if the reasoning path is too nebulous, irrelevant, or improperly employs input information.
∑N i=1 r-align(hi → s)/N (8)
where N is the number of steps and r-align is used to measure how well hi ∈ h can be aligned with any one of the steps in the ground truth path s.
(2) Informativenesspath (h → s): To measure the level of concordance between the generated path and the source, and if the generated reasoning path is well-grounded with respect to the source.
[1 + cos(h, s)]/2 (9)
where cos(·, ·) is a function for cosine similarity. (3) Consistencysteps (hi ↔ hj): To measure logical entailment errors within the reasoning steps.
1−maxi=2..N maxj<i pcontr(hi, hj) (10)
where pcontr is used to assess the likelihood of step pairs contradicting each other. hi ∈ h and hj ∈ h.
(4) Consistencypath (h ↔ s): To evaluate mistakes in logical entailment between the generated reasoning path h and source context s:
1−maxi=1..N maxj=1..T pcontr(hi, sj) (11)
where pcontr is the likelihood of source and generated steps contradicting each other. sj ∈ s; hi ∈ h.
(5) Perplexitypath (h): As an indicator of language coherence, it calculates average perplexity of all tokens in the generated reasoning path steps.
1/PPL(h) (12)
where PPL denotes the perplexity.

heading: 4 Experiments

heading: 4.1 Setup
text: Evaluation Metrics. In this paper, we aim to conduct comprehensive evaluation on multi-step reasoning, thus we select some scores from ROSCOE (Golovneva et al., 2023) as introduced in §3.3, which contains a suite of metrics allowing us to evaluate the quality of reasoning rationales, not limited to the correctness of final answers.
Datasets. We evaluate on five benchmark datasets involving arithmetic and commonsense multi-step reasoning: GSM8K (Cobbe et al., 2021), SVAMP (Patel et al., 2021), MultiArith (Roy and Roth, 2015), MathQA (Amini et al., 2019) and CSQA (Talmor et al., 2019).
Models. For reasoning path generation, we leverage Zero-shot CoT (ZS CoT) (Kojima et al., 2022) and Few-shot CoT (CoT) (Wei et al., 2022) with complexity-based prompting (Fu et al., 2023). For answer calibration, we employ SelfVerification (SV) (Weng et al., 2023) and SelfConsistency (SC) (Wang et al., 2023f) on multiple
paths. SV is a step-level strategy, which verifies intermediate-step answers and returns the path containing the maximum number of correct step answers. SC is a path-level strategy, which conducts majority voting on final answers of all generated paths and selects the most consistent result.
Implementation. We release the codes and generated results anonymously1. In this paper, the number of reasoning paths N defined in Eq (1) is 10, and number of intermediate steps M is 3 on all datasets except for CSQA where M is 10. We utilize GPT-3.5 with gpt-3.5-turbo engine as the backbone LLM to generate reasoning paths (the model choice justification is elaborated in Appendix B), and the temperature is set to 0.7. We also leverage GPT-4 (OpenAI, 2023) with gpt-4 engine to generate ground-truth rationales given the ground-truth answers for all datasets excluding GSM8K (which already contains them). For evaluation referring to ROSCOE (Golovneva et al., 2023), we respectively lever-
1https://github.com/231sm/Eval_Multi-Step_Reasoning.
N +1
and 11
N +1
are marked as F.
age all-MiniLM-L6-v2/SentenceTransformer, and pretrained gpt2-large (Radford et al., 2019) to obtain token/sentence embedding and calculate perplexity defined in Eq (12). All the reasoning paths for CoT and ZS CoT were generated during 8th to 23rd June 2023, and answer calibration on the generated reasoning paths was conducted during 12th October to 8th November 2023.

heading: 4.2 Analysis on Step-Level and Path-Level Answer Calibration Strategies
text: We respectively incorporate the effective step-level and path-level answer calibration strategies, SelfVerification (SV) and Self-Consistency (SC), into CoT-based models operating on multiple paths. We evaluate their performance using six evaluation metrics, with the results presented in Table 1.
Generally, in terms of accuracy, employing answer calibration is effective. Seen from Table 1, we find that models equipped with SV and SC obviously outperform vanilla methods, as both fewshot and zero-shot CoT employing SV/SC achieve significant accuracy improvements on almost all tasks. Notably, zero-shot CoT with SV and SC achieves much more significant outperformance of accuracy than few-shot settings on almost all tasks, demonstrating that answer calibration is more effective in zero-shot settings. As zero-shot CoT is relatively challenging due to the absence of task-specific in-context learning, answer calibration strategies essentially creating a feedback loop where the model assesses its own performance and adjusts accordingly, could help to mitigate biases and overfitting to specific patterns during inference, allowing the model to better generalize to
new types of problems and datasets. Furthermore, in terms of other metrics, answer calibration can improve consistency on arithmetic tasks but weakens faithfulness, informativeness and perplexity on both arithmetic and commonsense tasks. Observed from Table 1, we find that SV and SC weaken the perplexity score (16 out of 20 cases), suggesting that the rationale generated from multiple paths is more complex than that from a single path with CoT models. However, these two strategies improve consistency scores on arithmetic tasks (10 out of 16 cases; 14 out of 16 cases), intuitively benefiting from multiple paths. As SV verifies answers for intermediate steps and SC considers answers for all paths, they naturally enhance consistency within steps and between input/output (I/O). Additionally, SV and SC worsen faithfulness and informativeness on almost all tasks (15 out of 20 cases for both). The possible reason is that answer calibration on multiple paths focuses more on answer accuracy, while its increased complexity of its rationales tends to result in lower alignment and concordance between the source content and the output path. Generally, despite the benefits of employing SV and SC to CoT-based methods, the improvements are taskdependent and vary across different metrics.

heading: 4.3 Analysis on Unified Answer Calibration Strategies
text: We then integrate step-level and path-level answer calibration strategies, varying α as defined in Eq (1). We present the accuracy of the unified strategies in Figure 2. As observed, accuracy peaks at a specific value of α between the two thresholds
defined in Eq (4) and (7) in almost all scenarios across all tasks (i.e., 8 out of 10 cases), demonstrating that optimal model performance should balance both step-level and path-level answer calibration dominance. Besides, we notice that for “CoT on SVAMP task” in Figure 2(b) and “zeroshot CoT on MathQA task” Figure 2(d), employing integrated answer calibration strategies reaches a peak with α not between the two thresholds, and the overall performance remains stably lower than the initial best accuracy with α = 0 (i.e., SV). The possible reason may related to employing SV (i.e., α = 0) presenting more significant advantages than SC (i.e., α = 1) in the two scenarios. Specifically, CoT on SVAMP respectively achieves accuracy of 85.80% and 84.40% when α values 0 (SV) and 1 (SC), with the difference larger than 1%; Zero-shot CoT on MathQA employing SV and SC achieves accuracy of 52.86% v.s. 49.51%, where the difference is larger than 3%. Except for these two distinctive scenarios, others in Figure 2 obtain the optimal results by synthesizing step-level and path level answer calibration dominance.
In conclusion, the value of α plays a significant role in the performance of both few-shot and zeroshot CoT. Optimal ranges of α for each task are mostly between the two thresholds of step-level and path-level answer calibration dominance. The marked two thresholds represent boundaries for optimizing performance, which could guide further fine-tuning. Besides, the performance variance across datasets implies that the characteristics of each task, such as complexity, size, or the nature of the tasks. Models equipped with answer calibration strategies may require task-specific tuning to achieve the best performance.

heading: 4.4 Effects of Backbone Models
text: We compare accuracy on CoT-based answer calibration strategies with different LLM backbone engines, and present results in Table 2.
As observed from the results, for GPT-3 and Instruct-GPT, both self-verification (SV) and selfconsistency (SC) provide consistent improvements; while on the larger GPT-3.5 model, their improvements are observably weaker, particularly for SV, with which accuracy even slightly drops on the CSQA task. The possible reason is that GPT-3.5 is more prone to making mistakes when verifying on intermediate-step answers for multiple paths. Further, for integrated answer calibration strategies (SV+SC), the model’s performance is close to the better one between SV and SC. Generally, path-level answer calibration is more advantageous than step-level one, with relatively higher accuracy and lower computation cost. Based on these observations, we can infer that answer calibration strategies, especially path-level self-consistency, provide benefits in many cases, particularly on less powerful LLMs.
We further speculate, if the path generation for CoT with strong backbone LLM is sophisticated enough, the answer calibration may be simplified. We can directly conduct path-level answer calibration for multiple paths. But these findings cannot indicate that step-level answer calibration is meaningless for stronger backbone LLMs. As seen from Table 1, LLM equipped with step-level answer calibration is relatively beneficial to improve consistency scores. Besides, as mentioned in Weng et al. (2023), step-level answer calibration can provide explainable answers by verifying on intermediatestep answers, making results more reliable.

heading: 4.5 Effects of Prompting
text: We further demonstrate the effects of prompting with few-shot demonstrations on answer calibration, evaluated on CoT models.
We respectively input prompts of no coherence and no relevance for few-shot CoT referring to Wang et al. (2023a) (examples are listed in Appendix C), and present performance on SVAMP and MultiArith in Figure 3. As seen, the deficiency of coherence and relevance in the prompting observably weaken the performance of all models, with no relevance having a more profound impact than no coherence. In addition, CoT+SV achieves comparable performance with CoT+SC when prompting is standard or not coherent. Further, CoT+SV tends to perform observably better than CoT+SC, when prompting with no relevance, indicating that step-level answer calibration strategy SV, is beneficial to maintain performance under adverse conditions. This observation suggests the robustness of step-level answer calibration. It also highlights the potential benefits of step-level answer calibration strategies to mitigate performance degeneration caused by poor prompting. The possible reason is that step-level answer calibration strategies break down the task into subtasks, and these subtasks are simple enough so that less likely to be influenced by the low-quality prompts.

heading: 4.6 Analysis on Tasks
text: As seen from Table 1,2, and Figure 2, generally, SV and SC present more significant outperformance on arithmetic tasks than on the commonsense task (CSQA). Further, for CSQA, employing answer calibration tends to worsen the consistency scores, which is contrary to the trend ob-
served in arithmetic tasks. The possible explanation lies in the characteristics of each task, such as complexity, size, or the nature of the tasks. In the CSQA task, correct intermediate steps may not always contribute to a coherent reasoning path due to potential irrelevance and redundancy. Specifically, even if we calibrate both intermediate step and path answers, there can be some correct commonsense statements while irrelevant to the question, resulting in worse consistency and perplexity. Conversely, in arithmetic tasks, correct intermediate answers almost guarantee a consistent reasoning path, as all intermediate answers are necessary and will contribute to a correct final answer.

heading: 5 Conclusion and Future Work
text: In this paper, we dissect multi-step reasoning into path generation and answer calibration, and provide a unified view of answer calibration strategies through a comprehensive evaluation. We find that path-level answer calibration is particularly potent in improving accuracy, while step-level answer calibration is more suitable for addressing issues related to low-quality prompting. The improvement is more pronounced in zero-shot scenarios and less significant on stronger backbone models. We also define step-level and path-level answer calibration dominance with two thresholds, and propose to integrate of the two types of strategies, which is promising to achieve optimal performance. Our findings suggest that answer calibration is a versatile strategy that can be integrated into various models to bolster multi-step reasoning capabilities of LLMs. In the future, we aim to develop more sophisticated multi-step reasoning models, drawing on the insights and conclusions from this study.
Limitations
The main limitation for this paper is that we didn’t analyze more answer calibration strategies, such as step-/path-level methods on the single path, and varying the numbers of steps and paths in the unified answer calibration strategies. Besides, we can also employ answer calibration strategies to other path generation models, not limited to CoT-based methods. Further, we should also evaluate answer calibration strategies on more tasks to make the results more sufficient.
Broader Impact
Technical Novelty Emphasis. We have conducted an empirical study of answer calibration and proposed a unified method to address that Step-Level / Path-Level Answer Calibration for a Single or Multiple Paths can be integrated together, with the two thresholds of Step-/Path-Level Dominant Answer Calibration and a hyper-parameter α. Our analysis has the potential to inspire further research and practical implications on unified answer calibration, such as “how the hyper-parameter α can be optimally chosen across different tasks, like iterative tuning”. Our paper is based on an empirical study, and its main contributions are to unify multiple seemingly disparate types of approaches into a common framework, allowing us to investigate empirical questions to obtain more insights, e.g.,
(1) Employing answer calibration can enhance accuracy, with the improvement being more noticeable in zero-shot scenarios and less significant on stronger backbone models;
(2) The optimal performance of the unified answer calibration strategy typically achieved by synthesizing step-level and path level dominance;
(3) Path-level answer calibration is more beneficial in improving accuracy, and step-level answer calibration is more effective for mitigating lowquality prompting;
(4) Answer calibration can improve consistency on arithmetic tasks but weakens faithfulness, informativeness and perplexity on both arithmetic and commonsense tasks.

heading: Acknowledgment
text: We would like to express gratitude to the anonymous reviewers for their kind and helpful com-
ments. This work was supported by the National Natural Science Foundation of China (No. 62206246), the Fundamental Research Funds for the Central Universities (226-2023-00138), Zhejiang Provincial Natural Science Foundation of China (No. LGG22F030011), Yongjiang Talent Introduction Programme (2021A-156-G), Tencent AI Lab Rhino-Bird Focused Research Program (RBFR2024003), Information Technology Center and State Key Lab of CAD&CG, Zhejiang University, and NUS-NCS Joint Laboratory (A-000854200-00).

heading: A Terminology Clarification of Answer Calibration and Model Calibration
text: To avoid the confusion caused by the usage of the already-existing concept “calibration”, we provide a terminology clarification. We emphasize that “answer calibration” defined in our paper differs from “model calibration” (Niculescu-Mizil and Caruana, 2005; Guo et al., 2017; Tian et al., 2023; Xiong et al., 2024). “Answer Calibration” refers to the post-processing methods applied to one or more reasoning path(s), to obtain a final answer. We categorize answer calibration methods as ‘steplevel’ if they break down the reasoning path(s) into their individual steps, and ‘path-level’ otherwise. In most cases, “Answer Calibration” is more akin to “Answer Correction” (Pan et al., 2024), involves correcting mistakes in the initial output. We did give a definition like this in the Abstract, Introduction, and we have already provided clear definitions of “Answer Calibration” in §3.

heading: B Model Choice Justification
text: The choice of GPT-3.5 was driven by its relevance and accessibility for our research objectives. Our research includes an empirical study of answer calibration and a proposal of a unified method, where the backbone LLM is pluggable. To facilitate reproducibility, we have already released the code and LLM-generated data anonymously1 (provided at the bottom of Page 5 in §4), aiming to enhance transparency to some extent and facilitate further research in this area. We remain committed to exploring more transparent models in future work.

heading: C Cases of Low-Quality Prompts
text: We list some examples of prompts in Table 3.

heading: Future Work
text: We remain committed to exploring more transparent models in future work