S tra G o: Harnessing Strategic Guidance for Prompt Optimization (2024)

Abstract: Prompt engineering is pivotal for harnessing the capabilities of large language models (LLMs) across diverse applications. While existing prompt optimization methods improve prompt effectiveness, they often lead to prompt drifting, wherein newly generated prompts canadversely impact previously successful cases while addressing failures. Furthermore, these methods tend to rely heavily on LLMs’ intrinsic capabilities for prompt optimization tasks. In this paper, we introduce STRAGO (StrategicGuided Optimization), a novel approach designed to mitigate prompt drifting by leveraging insights from both successful and failed cases to identify critical factors for achieving optimization objectives. STRAGO employs a how-to-do methodology, integrating in-context learning to formulate specific, actionable strategies that provide detailed, step-by-step guidance for prompt optimization. Extensive experiments conducted across a range of tasks, including reasoning, natural language understanding, domain-specific knowledge, and industrial applications, demonstrate STRAGO’s superior performance. It establishes a new stateof-the-art in prompt optimization, showcasing its ability to deliver stable and effective prompt improvements.

## introduction recent advancements in large language models (llms), such as chatgpt and gpt-4, have significantly enhanced their analytical, reasoning, and contextual understanding capabilities @xcite @xcite . llms are employed in various applications, such as microsoft copilot and new bing, where users interact with the models through prompts. these prompts play a crucial role in guiding the llms' responses, ensuring outputs are accurate, relevant, and useful. however, the performance of llms heavily depends on prompt quality, and crafting effective prompts remains a complex, labor-intensive task that requires considerable expertise. to overcome the challenge of crafting effective prompts, recent research has focused on creating and optimizing prompts automatically. early approaches utilized reinforcement learning @xcite or gradient-based methods @xcite , though these techniques often require additional training or depend on the model's internal state, limiting their applicability for api-based llms like chatgpt and gpt-4. recent studies have leveraged llms themselves as prompt generators @xcite or optimizers @xcite . advanced search algorithms, such as monte carlo tree search (mcts) @xcite and evolutionary algorithms @xcite , have also been applied to discover effective prompts. additionally, some research has exploited the reflective capabilities of llms @xcite , optimizing prompts by using erroneous examples to guide refinement, either explicitly or implicitly @xcite @xcite . these llm-based optimization methods have demonstrated effectiveness across various tasks and hold promise for improving prompt quality. however, search-based algorithms often suffer from inefficiency in prompt optimization due to the absence of a clear optimization direction at each step. reflection-oriented methods aim to accelerate convergence by focusing on iteratively analyzing and correcting erroneous cases. however, concentrating on failure cases can sometimes negatively affect correct ones, especially when the errors exhibit outlier characteristics. both search-based and reflection-oriented approaches can result in prompt drift, where a newly generated prompt resolves certain failures but inadvertently disrupts previously successful cases. additionally, these methods typically provide the llm with a task description and context without offering specific guidance on how to achieve the desired outcomes, relying solely on the llm's inherent capabilities. for example, opro @xcite supplies historical prompts with corresponding scores and task-specific data, expecting the llm to generate more effective prompts. evoprompt @xcite asks the llm to merge two prompts into a new one without any instructions or strategy for doing so. similarly, apo @xcite presents erroneous cases and asks the llm to correct them with new prompts, but without providing actionable guidance. this heavy reliance on the llm's intrinsic abilities can be problematic for complex tasks, as the model may lack the necessary skills, leading to suboptimal prompt generation. in this paper, we introduce strago (strategic-guided optimization), a novel reflection-based prompt optimization method designed to overcome the limitations of existing approaches. unlike prior methods, strago avoids bias towards failure cases by analyzing both successful and failed outcomes in each iteration, identifying key factors necessary for task success and understanding the causes of failures. using this analysis, strago employs in-context learning to develop specific, actionable strategies that offer detailed, step-by-step guidance for prompt refinement. these strategies, combined with the analysis results, are used to optimize the prompt. our extensive experiments across reasoning, natural language understanding, domain knowledge, and industrial applications demonstrate that this approach effectively corrects failures while minimizing adverse effects on successful cases. this unbiased iterative process, coupled with detailed guidance, achieves the best overall accuracy improvements post-optimization, setting a new state-of-the-art in prompt optimization. our major contributions are as follows: ## case study this section provides an in-depth examination of the strategies developed by the refiner and the optimization processes undertaken by the optimizer, as illustrated through tow cases detailed in the first case pertains to a movie recommendation task. in this scenario, the analyzer identifies the prompt's failure, attributing it to the absence of a clear similarity criterion. to rectify this, the refiner develops a strategy focusing on identifying such criteria, particularly by scrutinizing outlier data. subsequently, the optimizer refines the prompt by addressing this diagnosed error and integrating the strategic insights. the second case involves the snarks task, where the analyzer underscores the importance of focusing on contextual clues, such as specific words or phrases. the refiner then crafts a strategy that not only incorporates these basic experiential insights but also emphasizes the analysis of sentence tone, specifically to discern exaggeration or overstatement. these additional insights are pivotal in determining the ironic intent of a sentence. ## conclusion in this paper, we introduce strago, a strategyguided, reflective-based optimization method that utilizes balanced iterations to analyze both suc-cessful and failed cases. this innovative approach identifies critical factors for achieving objectives while providing insights into the reasons for failures. by leveraging in-context learning, strago delivers detailed, step-by-step guidance for prompt optimization. experiments conducted across diverse tasks-ranging from simple scenarios to domain-specific and complex industrial contexts-demonstrate that strago significantly outperforms existing prompt optimization methods, establishing a new state-of-the-art in the field.