Self- P olish: Enhance Reasoning in Large Language Models via Problem Refinement (2023)

Abstract: To enhance the multi-step reasoning capabilities of large language models, researchers have extensively explored prompting methods, notably the Chain-of-Thought (CoT) method which explicitly elicits human-like rationales. However, they have inadvertently overlooked the potential of enhancing model reasoning performance by formulating higher-quality problems. In this work, we start from the problem side and propose Self-Polish (SP), a novel method that facilitates the model’s reasoning by guiding it to progressively refine the given problems to be more comprehensible and solvable. We also explore several automatic prompting varients and propose the Self-Polish prompt bank for the community. SP is orthogonal to all other prompting methods of answer/reasoning side like CoT, allowing for seamless integration with state-of-the-art techniques for further improvement. Thorough experiments show that the proposed method attains notable and consistent effectiveness on five reasoning benchmarks across different models. Furthermore, our method also showcases impressive performance on robustness evaluation. Codes and prompts are available at https://github.com/WooooDyy/Self-Polish.

## introduction large language models (llms) have achieved impressive performance on a variety of nlp tasks @xcite @xcite , but their capability to perform multistep reasoning is considered a limitation, which can not be tackled solely by scaling up the model size @xcite . to address this challenge, many prompting methods have been proposed to elicit reasoning in llms, and * equal contribution. † corresponding author. 1 a reasoning problem often consists of two parts: the context and the final question @xcite . have demonstrated significant effectiveness @xcite @xcite . chain-of-thought (cot) is a breakthrough method that teaches a language model to imitate the step-by-step reasoning process of humans to solve a reasoning task @xcite . many following work has explored variants of cot to improve the quality of rationales of llms @xcite @xcite . there is also a line of work that optimizes the rationales for better consistency and continuity @xcite @xcite , and a representative one is self-consistency (sc). sc generates diverse reasoning paths and answers, and then leverages the majority vote strategy to get the most consistent answer @xcite . despite the boosted reasoning performance of the aforementioned methods, they focus on the answer/reasoning side, and little emphasis has been placed on the problems side. actually, the clarity and logical structure of the problem description are crucial factors for human understanding and model comprehension @xcite @xcite . llms often exhibit poor reasoning performance when confronted with low-quality real-world reasoning problems, which may be excessively long, ambiguous, unclear in focus, or contain irrelevant information @xcite @xcite . to tackle this challenge, we consider refining problems into a better formulation. in this work, we propose self-polish (figure 1 right) that leverages llms themselves to refine reasoning problems without training for better reasoning performance. we first present several principles for refined problems: concise, clear, well-focused, and absent of irrelevant information. to achieve our goal, we propose the self-polish prompt bank which includes several feasible solutions as outlined in the following text. an intuitive strategy is to reformulate problems via instruction-following @xcite , and we call it zero-shot problem refining. next, we include demonstrations in the prompts @xcite to enable models to better internalize and apply the principles, which is defined as in-context problem refining. during the construction of the demonstrations, we incorporated a curated collection of problem-refining patterns, e.g., eliminating irrelevant information, rearranging the logic structure, and organizing local conditions into new ones in parallel. moreover, we explore automatic prompting methods to construct enhanced prompts and mitigate manual efforts, based on the criteria of complexity (complexitybased self-polish) or diversity (automatic self-polish). to further enhance the reliability and consistency of the generated problems, we propose to progressively refine problems until obtaining a convergent answer. experiments show that our method consistently improves the reasoning performance of various models (i.e., text-davinci-002, text-davinci-003, gpt-3.5-turbo) on five benchmarks ( in summary, we make the following contributions: ## related work multi-step reasoning. multi-step reasoning tasks have posed significant challenges for language models @xcite @xcite , and it is considered as an emergent ability of llms @xcite . it is in these tasks that the effectiveness of few-shot prompting begins to surpass that of full training set fine-tuning @xcite . moreover, such capability is considered important in building more complex artificial intelligence such as large language model-based agents (llm-based agents) @xcite . our work represents a significant stride in enhancing the ability of language models to perform multi-step reasoning tasks, through the facilitation of models' comprehension and processing of given reasoning problems. reasoning with prompting. prompting strategies have substantially improved the reasoning ability of llms by a large margin @xcite ). an important line of work in this area is chain-of-thought (cot) prompting which elicits the reasoning ability of models by prompting them to imitate the stepby-step reasoning process of humans @xcite @xcite . another line of work focuses on optimizing the rationales for better consistency and continuity @xcite @xcite . a representative one is self-consistency (sc), which samples ## self-polish prompting in this section, we first revisit previous prompting paradigms aiming at solving reasoning problems. next, we describe the proposed self-polish method detailedly. ## experiments in this section, we conduct experiments to demonstrate the effectiveness and robustness of sp.