Towards A Unified View of Answer Calibration for Multi-Step Reasoning (2024)

Abstract: Large Language Models (LLMs) employing Chain-of-Thought (CoT) prompting have broadened the scope for improving multi-step reasoning capabilities. We generally divide multi-step reasoning into two phases: *path generation* to generate the reasoning path(s); and *answer calibration* post-processing the reasoning path(s) to obtain a final answer. However, the existing literature lacks systematic analysis on different answer calibration approaches. In this paper, we summarize the taxonomy of recent answer calibration techniques and break them down into step-level and path-level strategies. We then conduct a thorough evaluation on these strategies from a unified view, systematically scrutinizing step-level and path-level answer calibration across multiple paths. Experimental results reveal that integrating the dominance of both strategies tends to derive optimal outcomes. Our study holds the potential to illuminate key insights for optimizing multi-step reasoning with answer calibration.

## introduction chain-of-thought (cot) prompting @xcite has significantly improved multi-step reasoning capabilities of large language models (llms) @xcite . as seen from figure 1 , the process of multi-step reasoning generally contains two primary modules: reasoning path generation which generates one or multiple reasoning paths @xcite ); and answer calibration which post-processes the reasoning path(s) to calibrate the initial output @xcite . in practice, answer calibration is pluggable and can be integrated into path generation models. the answer calibration framework can be divided into step and path levels, applicable to single or multiple paths, as illustrated in figure 1 . for step- path generation answer calibration answer answer step1 step2 step3 step4 answer b a a a path 3 path 1 path 2 answer answer step1 step2 step3 step4 step1 step2 step3 step4 ## related work reasoning path generation. previous methods for reasoning path generation mostly focus on two aspects to improve reasoning process, including refining input query or prompts (input refinement) and polishing the reasoning path (rationale polish). as for input refinement, zero-shot cot @xcite and few-shot cot @xcite and formats, such as program @xcite @xcite @xcite and table @xcite . further, @xcite al. (2023b @xcite , graph-of-thought @xcite and hypergraph-of-thought (hot) @xcite . answer calibration. given generated reasoning path(s), answer calibration methods post-process the path(s) to calibrate the answer, involving stepor path-level calibration on one or multiple path(s). xue et al. (2023); cao (2023 @xcite , encouraging interaction with the environment to provide feedback. weng et al. (2023); jiang et al. (2024) unleash the self-verification ability of llms, by forward reasoning and backward verification on intermediate step answers. zhou et al. (2024) propose codebased self-verification on reasoning steps. path-level answer calibration. zelikman et al. (2022) present a self-taught reasoner to iteratively generate rationales. zheng et al. (2023) progressively use the generated answers as hints to make double-check. mountantonakis and tzitzikas (2023) enrich generated reasoning paths with hundreds of rdf kgs for fact checking. baek et al. (2023) iteratively rectify errors in knowledge retrieval and answer generation for knowledgeaugmented lms. to cultivate the reasoning ability of smaller lms, ho et al. (2023); wang et al. (2023e,h) propose to fine-tune cot for knowledge distillation. huang et al. (2023) demonstrate that llms can self-improve with high-confidence rationale-augmented answers. yoran et al. (2023) prompt llms to meta-reason over multiple paths. liu et al. (2023); madaan et al. (2024) leverage feedback to improve model initial outputs. wan et al. (2023) adaptively select in-context demonstrations from previous outputs to re-generate answers. wang et al. (2023f) leverage self-consistency decoding strategy to majority vote on multiple path answers. aggarwal and yang (2023) propose adaptive-consistency to reduce sample budget. ## conclusion and future work in this paper, we dissect multi-step reasoning into path generation and answer calibration, and provide a unified view of answer calibration strategies through a comprehensive evaluation. we find that path-level answer calibration is particularly potent in improving accuracy, while step-level answer calibration is more suitable for addressing issues related to low-quality prompting. the improvement is more pronounced in zero-shot scenarios and less significant on stronger backbone models. we also define step-level and path-level answer calibration dominance with two thresholds, and propose to integrate of the two types of strategies, which is promising to achieve optimal performance. our findings suggest that answer calibration is a versatile strategy that can be integrated into various models to bolster multi-step reasoning capabilities of llms. in the future, we aim to develop more sophisticated multi-step reasoning models, drawing on the insights and conclusions from this study.