Revisiting OPRO : The Limitations of Small-Scale LLM s as Optimizers (2024)

Abstract: Numerous recent works aim to enhance the efficacy of Large Language Models (LLMs) through strategic prompting. In particular, the Optimization by PROmpting (OPRO) approach provides state-of-the-art performance by leveraging LLMs as optimizers where the optimization task is to find instructions that maximize the task accuracy. In this paper, we revisit OPRO for automated prompting with relatively small-scale LLMs, such as LLaMa-2 family and Mistral 7B. Our investigation reveals that OPRO shows limited effectiveness in small-scale LLMs, with limited inference capabilities constraining optimization ability. We suggest future automatic prompting engineering to consider both model capabilities and computational costs. Additionally, for small-scale LLMs, we recommend direct instructions that clearly outline objectives and methodologies as robust prompt baselines, ensuring efficient and effective prompt engineering in ongoing research.

## introduction advancements in large language models (llms) have catalyzed a shift towards prompting-based learning, distinguishing models with capacities exceeding 100 billion parameters for their few-shot learning abilities without extensive retraining (ope-nai, 2020). in-context learning, facilitated through the strategic use of prompts, enables these models to generate task-specific responses, marking a departure from traditional pre-train and fine-tune approaches @xcite . the chain of thought (cot) technique significantly advances llms' problem-solving capabilities by incorporating intermediate reasoning steps, facilitating effective zero-shot reasoning and performance enhancements with prompts like "let's think step by step" @xcite @xcite . while initially dependent on manual prompt creation, recent developments in automated prompt engineering, such as ape @xcite and apo @xcite , leverage llms for dynamic prompt generation and refinement. this iterative process enhances nlp task accuracy through feedback and selection. building on this, the proposition of llms as optimizers @xcite presents the current state-of-theart in automated prompt design, framing prompt refinement as an optimization challenge. this approach iteratively refines prompts to maximize task accuracy, ceasing when performance plateaus or iteration limits are met. the motivation for opro is based on the llms' self-optimization ability. however, our empirical results reveal that smaller-scale llms like llama-2 @xcite do not have sufficient ability to support the self-optimization. we demonstrate that such optimization strategies offer marginal benefits for smaller-scale llms, demanding considerable computational resources for slight performance gains, particularly when contrasted with zero-shot cot prompts. we summarize our contributions as follows: opro @xcite and evoprompt @xcite framework have demonstrated the significant potential of llms in automating prompt design. however, the effectiveness appears to be contingent upon the inherent optimization capabilities of the llms themselves. notably, evaluations within the opro framework have predominantly focused on large-scale models, such as gpt-4 and text-bison, leaving the performance of smallerscale llms unexplored. this observation prompts a critical inquiry: can small-scale llms also serve as optimizers? to delve into this question, we attempt to reproduce the linear regression optimization experiment with llama-2-13b, the motivating example shown in opro @xcite . we adopt the same experiment setting as in opro. specifically, our experiment aims to optimize two parameters, w and b, in a one-dimensional linear regression model with an intercept b, using 50 data points generated from predefined value of w true and b true with standard gaussian noise ϵ. starting from five initial random pairs of (w, b), we engaged llama-2-13b through a meta-prompt strategy similar to opro, directing the model to propose pairs that minimize the objective function, based on historical data of the top 20 performing pairs. a representative meta-prompt and its output is shown in figure 1 . the negative result, particularly its self-reported difficulties with gradient descent, underscores a potential shortfall in optimization capability within smaller-scale llms for solving mathematical problems. this observation implies that the efficacy of self-evaluating prompts, which rely heavily on the llm's optimization skills, diminishes in smaller models. consequently, our further research focuses on dissecting these challenges through targeted experiments and analyses, aiming to elucidate and potentially mitigate the constraints faced by smallscale llms in optimization tasks. ## evaluation in this section, we aim to replicate the opro framework with small-scale llms to assess its efficacy in identifying optimal instruction words. the instruction position is added to the beginning of the llm output. ## limitations of self-optimization prompting in small-scale llms small-scale llms could not support selfoptimization. our analysis, presented in this finding is in line with recommendations from existing literature @xcite , where leveraging outputs from larger llms to enhance smaller models reflects our experimental observations. furthermore, recent literature indicates that without additional inputs, llms struggle to selfimprove @xcite . interestingly, upgrading the scorer model only minimally affects performance, implying the optimizer may not fully leverage the advanced capabilities of a superior scorer in opro's context, leading to suboptimal prompt generation. as a result, due to the limited inference ability, small-scale llms could not support self-optimization for prompting paradigms. ## conclusion with empirical results, we demonstrate that smallscale llms are limited in self-optimization capacity, which causes opro is not effective for smallscale llms. in addition, our findings underscore opro's dependency on scorer performance and manually designed prompts, despite the effort to automate the process. we suggest the future automatic prompting engineering consider both model capabilities and system efficiencies. limitation and future study. our study's scope was limited by computational resources, excluding other self-optimization strategies like evo-prompt and apo due to their extensive prompt generation time. our future research will extend to enhancing the interpretability and depth of error analysis, alternative optimization metrics, bias considerations, or hyperparameter tuning impacts based on our current findings.