Auto-Evolve: Enhancing Large Language Model’s Performance via Self-Reasoning Framework (2024)

Abstract: Recent advancements in prompt engineering strategies, such as Chain-of-Thought (CoT) and Self-Discover, have demonstrated significant potential in improving the reasoning abilities of Large Language Models (LLMs). However, these state-of-the-art (SOTA) prompting strategies rely on a fixed set of static seed reasoning modules like “think step by step” or “break down this problem” intended to simulate human approach to problem-solving. This constraint limits the flexibility of models in tackling diverse problems effectively. In this paper, we introduce Auto-Evolve, a novel framework that enables LLMs to self-create dynamic reasoning modules and downstream action plan, resulting in significant improvements over current SOTA methods. We evaluate Auto-Evolve on the challenging BigBench-Hard (BBH) dataset with Claude 2.0, Claude 3 Sonnet, Mistral Large, and GPT-4, where it consistently outperforms the SOTA prompt strategies. Auto-Evolve outperforms CoT by up to 10.4% and on an average by 7% across these four models. Our framework introduces two innovations: a) Auto-Evolve dynamically generates reasoning modules for each task while aligning with human reasoning paradigm, thus eliminating the need for predefined templates. b) An iterative refinement component, that incrementally refines instruction guidance for LLMs and helps boost performance by average 2.8% compared to doing it in a single step.

## introduction llms have demonstrated significant potential in various natural language processing (nlp) capabilities such as understanding, generating, and reasoning @xcite @xcite . despite the impressive progress, llms continue to have challenges in solving multi-step reasoning tasks that require systematic thinking and planning. increasing model size alone is not enough to solve these issues, emphasizing the necessity for developing novel techniques to improve llms' reasoning capabilities @xcite . various prompting strategies have been developed to guide and facilitate the reasoning capabilities of llms. cot @xcite has emerged as a prominent approach, encouraging llms to generate step-by-step explanations mimicking human reasoning. subsequent research efforts have focused on refining the generation process thereby enhancing the quality and consistency of the rationales @xcite @xcite . self-discover @xcite improves models' reasoning capabilities over cot by allowing models to select the most appropriate reasoning path from a fixed set of reasoning modules. however, our analysis of the seed modules in self-discover revealed that a subset of fixed seed modules dominated the usage, limiting the framework's reasoning coverage and performance on diverse tasks (appendix: fig. 9 ). cot's and self-discover's reliance on a limited set of reasoning seed modules such as "think step by step" or "break down this problem" constrains the approaches to tackling a problem, negatively affecting their ability to generalize over diverse tasks. our framework, auto-evolve, builds upon the strengths of previous prompting approaches while addressing their limitations. rather than relying on a fixed set of seed modules, auto-evolve creates custom reasoning modules on-the-fly for each task, allowing llms to come up with a wider range of reasoning structures (instruction guidance in json format that llms can follow to solve a task step by step) that are better suited to handle the specific needs of each task. auto-evolve further incorporates an iterative refinement process allowing models to refine their reasoning structures based on the specific requirements of each task, significantly boosting performance. auto-evolve consists of three core components: we evaluate auto-evolve's performance on big bench hard @xcite , a widely used benchmark with a subset of 23 hard tasks from the big-bench suite @xcite . on average, across four models, auto-evolve demonstrates a 12.8% performance improvement over direct prompt (claude 2.0: 11.7%, claude 3 sonnet: 3.0%, mistral large: 13.5%, gpt-4: 22.9%), a 7% performance improvement over cot (claude 2.0: 10.4%, claude 3 sonnet: 3.3%, mistral large: 8.1%, gpt-4: 6.3%) and a 4% improvement over self-discover 4% (claude 2.0: 6.7%, claude 3 sonnet: 3.9%, mistral large: 2.7%, gpt-4: 2.6%). by combining dynamic prompt generation and iterative refinement, auto-evolve offers a more flexible and adaptive approach to reasoning, pushing the boundaries of llms performance on complex tasks. ## related work there are two key model optimization techniques, model prompting and model fine-tuning. model prompting methods enhance the reasoning capabilities of llms by providing carefully designed prompts that guide the model towards generating the desired output, without modifying the underlying model parameters. on the other hand, model fine-tuning methods involve updating the model's parameters by training on a relevant dataset to specialize the model for a particular task or domain, which can be computationally expensive. while both methods have their advantages and disadvantages, our framework auto-evolve, is closely related to model prompting methods. model prompting methods such as cot prompting @xcite encourages models to generate intermediate reasoning steps that lead to the final desired answer. cot has been shown to boost performance on arithmetic, commonsense, and symbolic reasoning tasks. subsequent work has extended cot by selectively sampling rationales @xcite , improving rationale consistency @xcite ; self-consistency), generating more structured reasoning paths @xcite , and having models first plan the reasoning before solving the problem @xcite ; plan-and-solve). self-discover @xcite , introduces a three-stage process where llms select relevant reasoning modules, adapt them to the specific task, and implement them into a coherent reasoning structure. self-discover outperforms cot @xcite , self-consistency @xcite and plan-and-solve @xcite prompting on various benchmarks. ## auto-evolve framework auto-evolve framework is inspired by two fundamental principles. (1) higher interpretability associated with json structure: llm's reasoning capabilities and performance are enhanced by json structure's higher interpretability @xcite . (2) llms have inbuilt diverse reasoning abilities: llms possess an inherent grasp of diverse thinking styles and essential reasoning modules crucial for tackling variety of tasks since they were trained on enormous data, typically measured in petabytes. sota self-discover adheres to the first principle, but it overlooks the key aspect of the second principle. instead of leverage knowledge hidden within llms, self-discover supplies llms with a fixed set of initial human-designed reasoning modules such as "use critical thinking" and "let's think step by step". on the flip side, auto-evolve advocates for llms' intrinsic ability to independently discern and utilize relevant reasoning strategies for different tasks. auto-evolve comprises of two stages as illustrated in fig. 1 . stage 1 dynamically generates intrinsic task-related reasoning modules and structure (json instructions) by leveraging task examples and three meta-prompts, thereby guiding llms to solve tasks without needing static human-designed seed modules and further training. stage 1 operates at task-level, i.e., one run for each task category. stage 2 uses the finalized reasoning structure produced as an output of stage 1 to solve individual task instances by asking the model to follow the instruction step by step. given the straightforward and uncomplicated nature of stage 2, we focus rest of this section on further elaborating the three components of stage 1 that are illustrated in fig. 2 . we also present a graphical representation in fig. 3 that's accompanied by mathematical notations to elucidate the procedure of stage 1. left half of this figure showcases the generate and imple-ment components, while right half showcases the refine components. the mathematical notations are explained in the following subsections. prompts details are included in appendix fig. 7 . ## conclusion and future work auto-evolve introduces a novel framework that dynamically generates task-specific reasoning structures, eliminating the need for static seed modules and enabling more effective reasoning across diverse problem domains. by seamlessly integrating dynamic prompt generation and iterative refinement, auto-evolve surpasses state-of-the-art methods like cot prompting, achieving performance improvements up to 10.4% and an average gain of 6.8% when evaluated with gpt-4, claude 2.0, claude 3 sonnet and mistral large models. the framework's ability to transfer reasoning structures from larger models to smaller ones, as demonstrated with models like llama 3.1 8b, highlights its broader utility and adaptability across architectures. the broader implications of auto-evolve extend beyond the performance enhancement, as the framework has the potential to advance the development of more interpretable and transparent ai systems by generating dynamic problem specific reasoning modules and explicit reasoning structures. our experimentation has highlighted the pivotal role played by json reasoning structures in solving tasks effectively. in future iterations, we aim to explore the potential of incorporating feedback mechanisms to iteratively improve these reasoning structures, further refining and enhancing the framework's capabilities.