R ^3 Prompting: Review, Rephrase and Resolve for Chain-of-Thought Reasoning in Large Language Models under Noisy Context (2023)

Abstract: With the help of Chain-of-Thought (CoT) prompting, Large Language Models (LLMs) have achieved remarkable performance on various reasoning tasks. However, most of them have been evaluated under noise-free context and the dilemma for LLMs to produce inaccurate results under the noisy context has not been fully investigated. Existing studies utilize trigger sentences to encourage LLMs to concentrate on the relevant information but the trigger has limited effect on final answer prediction. Inspired by interactive CoT method, where intermediate reasoning steps are promoted by multiple rounds of interaction between users and LLMs, we propose a novel prompting method, namely R ^3 prompting, for CoT reasoning under noisy context. Specifically, R ^3 prompting interacts with LLMs to perform key sentence extraction, variable declaration and answer prediction, which corresponds to a thought process of reviewing, rephrasing and resolving. The responses generated at the last interaction will perform as hints to guide toward the responses of the next interaction. Our experiments show that R ^3 prompting significantly outperforms existing CoT prompting methods on five reasoning tasks under noisy context. With GPT-3.5-turbo, we observe 3.7% accuracy improvement on average on the reasoning tasks under noisy context compared to the most competitive prompting baseline. More analyses and ablation studies show the robustness and generalization of R ^3 prompting method in solving reasoning tasks in LLMs under noisy context.

## introduction recent advances in large language models (llms) like gpt-3 @xcite , palm @xcite , opt @xcite , and llama @xcite have revolutionized the landscape of natural language * corresponding author processing. a series of studies have proved that multi-step reasoning tasks can be easily solved by llms via different prompting methods @xcite @xcite @xcite @xcite . chain-of-thought (cot), as the representative prompting approach, narrows the gap between human intelligence and machine intelligence by applying the rationales to few-shot prompting @xcite . it has achieved impressive results on arithmetical reasoning tasks @xcite @xcite @xcite @xcite . however, most cot approaches are investigated under a noisy-free context, where all the information provided in the problem of interest is relevant to the final answer. while in real-world scenarios, it is frequent to encounter problems with irrelevant information. @xcite first defined the reasoning tasks in llms under noisy context. as shown in figure 1 , the noisy context in the problem of interest includes "helen baked 12 berry cookies yesterday", which can easily distract llms. they further proposed instructed-cot prompt, which includes a trigger sentence instructing llms to ignore the irrelevant information given in a question ahead of one-turn response exemplar. instructed-cot prompt is intuitive to denoise context but it has limited effect on the final answer prediction. inspired by prior studies @xcite , interactive cot prompting approaches show that conducting multi-round of interaction between users and llms as hints is efficient in progressively guiding toward the correct answers for complex reasoning tasks @xcite . least-to-most @xcite is a cot prompting method that guides llms to decompose a complex problem into simple sub-questions. the prompt first decomposes the problem into subproblems, then the generated intermediate answer is appended to the next subproblem until the final answer is returned. php @xcite is another prompting method with interactive responses that combine the candidate answers and problems for re-evaluation purposes, which allows double-check to the answer prediction. nevertheless, these interactive cot prompting methods are still vulnerable to noisy context. in this paper, we propose a new method named r 3 prompting, that includes sequentially interaction with llms to gradually approach final answers via a thought process of reviewing, rephrasing and resolving. in the review stage, a review prompt is designed to extract key sentences that are essential conditions required for the final answer prediction. in the rephrase stage, we guide llms to reformulate the problem narratives to variables with the hint of extracted key sentences. in the resolve stage, llms predict the final answers taking account of the generated variables. all stages are implemented under few-shot setting with showing the responses of several exemplar problems. a comparison between the existing prompting methods is displayed in figure 1 . we evaluate our prompting method on five different benchmarks containing misleading and irrelevant context, that are addsub @xcite , svamp @xcite , multiarith-ic, singleeq-ic, and gsm-ic @xcite . the last three datasets are manually created by inserting noise to multiarith (roy and roth, 2016), singleeq @xcite and gsm8k @xcite . we follow @xcite to create multiarith-ic and singleeq-ic by randomly inserting irrelevant sentences into problem descriptions from multiarith and singleeq, respectively. specifically, we follow their templatebased method to create templates for noisy context and instantiate with role names in the problems. from the experimental results, we observe that interactive cot has some advantages in solving noisy problems and our r 3 prompting method could outperform a series of cot prompting methods for reasoning tasks in llms under noisy context, which leads to an average improvement of 3.7 absolute points on accuracy. more analyses shed lights on the robustness and generalization of r 3 prompting. prompting with noisy ground truth. a few studies have investigated the effect of prompts with noisy ground truth, including pairing exemplar problems with incorrect answers @xcite and adding irrelevant or misleading context into exemplars @xcite . in particular, @xcite proved that the correctness of math equations in exemplars does not notably influence model's performance, whereas the noise included in the text could hamper the performance dramatically. similar observation has been discussed in another work @xcite . to solve the reasoning task under noisy context, @xcite introduced an intuitive trigger sen-tence in prompts to decrease the influence from the noisy context to llms. different from the simple instruction for denoising in their prompts, our work focuses on designing a interactive cot prompt that can steadily solve problems without being misguided by noisy context. ## r prompting researches on human problem-solving indicate that when solving a reasoning task under noisy contexts, the instinct of humanity is to conduct multiple rounds of thinking @xcite . in this paper, we propose that this process can be simulated in llms by reviewing, rephrasing, and resolving successively. specifically, we design a prompting method with three-stage prompts: (1) review prompt: instruct llms to read the question and extract the key sentences from the noisy contexts. (2) rephrase prompt: with the hint of extracted key sentences, reformulate the narratives into abstractive representations with explicit variables. (3) resolve prompt: with the hint of explicit variables, solve the question and predict the final answers. figure 2 illustrates the proposed interaction and prompt design for each stage. next, we will introduce the detailed design of prompts in each stage. ## conclusion in this paper, we proposed r 3 prompting to enable llms to solve problems under noisy context. it involves prompts for three different stages: reviewing, rephrasing, and resolving. evaluation on eight datasets across two types of noisy context suggests r 3 prompting significantly outperforms the previous baselines. more analyses and ablation studies show that the performance of r 3 prompting is stable and powerful. overall, the advantages of r 3 prompting are presented in the following aspects: 1) achieving substantial performance improvement on arithmetic reasoning tasks under the interference of noisy context; 2) exhibiting strong robustness, maintaining stable performance even when facing progressively increasing amounts of noisy context.