Reasoning in Flux: Enhancing Large Language Models Reasoning through Uncertainty-aware Adaptive Guidance (2024)

Abstract: Machine reasoning, which involves solving complex problems through step-by-step deduction and analysis, is a crucial indicator of the capabilities of Large Language Models (LLMs). However, as the complexity of tasks escalates, LLMs often encounter increasing errors in their multi-step reasoning process. This study delves into the underlying factors contributing to these reasoning errors and seeks to leverage uncertainty to refine them. Specifically, we introduce Uncertainty-aware Adaptive Guidance (UAG), a novel approach for guiding LLM reasoning onto an accurate and reliable trajectory. UAG first identifies and evaluates uncertainty signals within each step of the reasoning chain. Upon detecting a significant increase in uncertainty, UAG intervenes by retracting to a previously reliable state and then introduces certified reasoning clues for refinement. By dynamically adjusting the reasoning process, UAG offers a plug-and-play solution for improving LLMs’ performance in complex reasoning. Extensive experiments across various reasoning tasks demonstrate that UAG not only enhances the reasoning abilities of LLMs but also consistently outperforms several strong baselines with minimal computational overhead. Further analysis reveals that UAG is notably effective in identifying and diminishing reasoning errors.

## introduction the impressive advancements of large language models (llms) have recently brought about a new era in machine reasoning @xcite @xcite @xcite . for challenging scenarios, decomposing a problem into a series of intermediate steps has been shown to significantly improve the performance of llms @xcite @xcite ). ## preliminary this section outlines the foundational concepts that underpin our uag method. given a problem q, a llm generates an answer a, constructing it to-ken by token through a probabilistic approach as described below: cot @xcite involves supplementing the problem q with several demonstrations d that include detailed reasoning processes. this methodology guides the model to first generate the rationale r, followed by the answer a. applying bayesian theorem (bayes and price, 1763) allows us to further refine our understanding: given that the answer a typically comprises fewer tokens than the rationale r, our primary focus is on optimizing the impact of the reasoning process r. to this end, we define the following criteria: ## uncertainty-aware adaptive guidance uncertainty-aware adaptive guidance (uag) aims to enhance the reasoning capabilities of llms by incorporating uncertainty awareness and adaptability. as depicted in figure 2 , our method progresses through three interconnected phases: uncertainty identification, adaptive reasoning adjustment, and demonstration clustering. each phase plays a crucial role in refining the llm's reasoning process, which we detail in the subsequent sections. ## conclusion in this paper, we start by observing the causes behind llm reasoning errors from the perspective of uncertainty, paving the way for nuanced and adaptive modifications in the reasoning process. driven by this insight, we introduce the uncertainty-aware adaptive guidance (uag). uag strategically mitigates reasoning errors by identifying and eliminating uncertainties with reasoning steps, concurrently leveraging exemplars chosen for their relevance and originality to dispel uncertainty and steer the subsequent reasoning trajectory. comprehensive experimental evaluations across a spectrum of reasoning tasks demonstrate that uag not only surpasses several strong baselines but also exhibits adaptability to a diverse array of models, underscoring its efficacy and wide applicability in enhancing llm reasoning capabilities. further analysis showcases the remarkable versatility of uag, which can function as a plug-and-play module and efficiently identify and eliminate reasoning errors.