Auto-Evolve: Enhancing Large Language Model's Performance via Self-Reasoning Framework
Authors: Huilin Lu, Krishna Aswani, Pranav Patankar, Priya Dhalwani, Xue Tan, Jayant Ganeshmohan, Simon Lacasse


Reviewer paper summaries: The author proposes a novel framework to generate and refine reasoning modules and reasoning structure on the fly by large language models themselves and conducted a systematic analysis on them to show their effectiveness. The reasoning modules resembles slot filling tasks to encourage language models to complete tasks with explicit goals before giving the final answer. It not only improves the performance across several complex reasoning tasks but also reduced inference calls needed compared to CoT.
Summary of strengths: 1. It discussed the related work very comprehensively and accurately. The author especially pinpoints the difference between Auto-Evolve, Self-Discover and Plan-and-Solve prompting, and provides an illustration to explain the atomic reasoning module.
2. The proposed method not only generate but also refine reasoning structure to break down the problem into subtasks. Finally, it operationalizes the reasoning modules into a reasoning structure.
3. It's a very promising framework to adapt language models to various types of human-AI collaborations.
Summary of weaknesses: 1. The author only elicit reasoning modules from Claude, Mistral-Large and GPT-4. It would be more interesting to see if the reasoning structure generated and refined by closed-source models can be transferred for open-source models' usage to improve the performance.
Comments, suggestions and typos: The figures provided don't have dynamic resolution, and it's better to include figures in pdf format.
Overall assessment scores: 2