Mirror: Multiple-perspective Self-Reflection Method for Knowledge-rich Reasoning (2024)

Abstract: While Large language models (LLMs) have the capability to iteratively reflect on their own outputs, recent studies have observed their struggles with knowledge-rich problems without access to external resources. In addition to the inefficiency of LLMs in self-assessment, we also observe that LLMs struggle to revisit their predictions despite receiving explicit negative feedback. Therefore, We propose Mirror, a Multiple-perspective self-reflection method for knowledge-rich reasoning, to avoid getting stuck at a particular reflection iteration. Mirror enables LLMs to reflect from multiple-perspective clues, achieved through a heuristic interaction between a Navigator and a Reasoner. It guides agents toward diverse yet plausibly reliable reasoning trajectory without access to ground truth by encouraging (1) diversity of directions generated by Navigator and (2) agreement among strategically induced perturbations in responses generated by the Reasoner. The experiments on five reasoning datasets demonstrate that Mirror’s superiority over several contemporary self-reflection approaches. Additionally, the ablation study studies clearly indicate that our strategies alleviate the aforementioned challenges.

## introduction large language models (llms) have become an important and flexible building block in a variety of tasks. they can be further improved by iterative correction in many tasks @xcite @xcite , such as code generation, arithmetic problem solving and reasoning. during iterative refinement, the critic module, which assesses the current response and generates valuable feedback, is crucial to drive performance improvement. ## lost in the reasoning loop given the observed challenges in enhancing llms' self-improvement without ground truth labels, particularly in knowledge-rich reasoning tasks, our initial experiment aims to address these challenges by breaking them down into two sub-questions. ## the framework of mirror in this section, we introduce our unsupervised selfreflection framework, mirror, depicted in figure 3 . the reward r consists of diversity and consistency terms. diversity is applied to prevent reflection from becoming stuck and to facilitate intraconsistency involved in the stop criteria for selfassessment. the consistency reward also influences direction generation. ## can mirror steer llms in iterative improvements? we evaluate our proposed mirror on mmlu and fever @xcite . fever is a fact-checking dataset featuring three labels for knowledge-rich statements, i.e., supports, refutes and not enough info. foot_2