Dual-Phase Accelerated Prompt Optimization (2024)

Abstract: Gradient-free prompt optimization methods have made significant strides in enhancing the performance of closed-source Large Language Model (LLMs) across a wide range of tasks. However, existing approaches make light of the importance of high-quality prompt initialization and the identification of effective optimization directions, thus resulting in substantial optimization steps to obtain satisfactory performance. In this light, we aim to accelerate prompt optimization process to tackle the challenge of low convergence rate. We propose a dual-phase approach which starts with generating high-quality initial prompts by adopting a well-designed meta-instruction to delve into task-specific information, and iteratively optimize the prompts at the sentence level, leveraging previous tuning experience to expand prompt candidates and accept effective ones. Extensive experiments on eight datasets demonstrate the effectiveness of our proposed method, achieving a consistent accuracy gain over baselines with less than five optimization steps.

## introduction llms have demonstrated remarkable capabilities across a wide range of natural language processing (nlp) tasks, including machine translation @xcite , summarization @xcite , and question answering @xcite . the dependency on prompt quality has led to the emergence of prompt engineering @xcite , aiming at crafting effective prompts to elicit the desired responses from llms. as the need for efficient prompt design becomes increasingly evident @xcite , automatic prompt optimization has been introduced to streamline the prompt design process, ensuring that llms are utilized to their full potential @xcite @xcite . automatic prompt optimization can be broadly categorized into gradient-based and gradient-free methods. gradient-based methods @xcite @xcite are devised for open-source llms to enable the optimization of prompts through adjustments based on model gradient. gradient-free methods have emerged as the predominant approach for closedsource llms, which focuses on refining prompts without access to the model gradient @xcite @xcite . starting from initial prompts, these methods usually expand candidate prompts using searching methods @xcite and then accepting the more prominent ones in an iterative manner. this paper focuses on gradient-free methods due to the distinguished abilities of closedsource llms and the challenge of optimizing their prompts with limited model information. we argue that current gradient-free prompt optimization methods have not adequately considered the rate of convergence. typically, these methods demand an excessive number of optimization steps to obtain satisfactory prompts due to the limited access to model details, the vast discrete search space, and the uncertain optimization directions @xcite @xcite . representative work such as opro @xcite even necessitates nearly 200 optimization steps for some nlp tasks. this requirement for excessive optimization steps makes existing methods impractical for real-world applications since users are understandably reluctant to tolerate extensive optimization steps to achieve satisfactory performance levels. therefore, we aim to achieve accelerated prompt optimization, obtaining satisfactory performance via few optimization steps (e.g., < 5). to achieve accelerated prompt optimization, two crucial factors need to be considered: high-quality initial prompts and effective optimization directions. firstly, the initialization of the prompt plays a crucial role in determining the efficiency of the optimization process @xcite , whereas existing approaches pay insufficient attention to the impact of initialization on subsequent optimization. therefore, we aim to obtain initial prompts of high quality, laying a solid foundation to accelerate optimization process. secondly, the accelerated prompt optimization needs to identify the most effective optimization directions in each step, streamlining efficient optimization from the initial prompts. thus, we aim to design a more refined expansion tuned by experience and acceptance of candidate prompts enhanced by examination of failure cases. to this end, we propose a dual-phase approach to achieve the accelerated gradient-free prompt optimization. our approach consists of two phases: high-quality initial prompt generation, and experience-tuned optimization. firstly, we utilize a well-designed meta-instruction to guide the llm in generating high-quality and structured initial prompts that contain task-specific information, including task type and description, output format and constraints, suggested reasoning process, and professional tips. after that, we devise a sentencelevel prompt optimization strategy for efficiently optimization on the long initial prompt, leveraging previous direction tuning experience, together with failure cases, to select sentences in the initial prompt to be expanded and accept effective prompt candidates. extensive experiments (cf. figure 1 ) on three llms across several datasets confirm the effectiveness and superiority of our method. our contributions are threefold: ## related work the gradient-free prompt optimization for closedsource llms typically contains two phases: initialization and iterative optimization steps, where the optimization step consists of expansion and selection stages. initialization. the prompt initialization for optimization can be achieved manually or autonomously. manual initialization often entails professional machine learning engineers formulating prompts, as delineated in @xcite . concurrently, works such as @xcite , @xcite , and @xcite utilize existing manual prompts as the foundational set to harness human creativity. in contrast, automated initialization leverages the power of llm generation, which is exemplified by @xcite , generating prompts from few-shot exemplars and a rudimentary description, and @xcite , fabricating prompts based on meta-prompts and illustrative input-output examples. our method belongs to the automated initialization, improving the initial prompt generation for acceleration. optimization. the optimization step is achieved by expanding prompt candidates by modifying from the initial prompt and selecting the better candidates for the next iteration. the expansion stage can be executed through rephrasing, as in @xcite , where high-scoring prompts undergo evolution akin to a monte carlo search methodology, or through heuristic algorithms that automatically revise prompts, as in @xcite and @xcite . more complex regeneration strategies are employed by works like @xcite , where the optimizer llm progressively expands prompts based on task delineations and historical iterations. the expansion can also be implemented leveraging an open-source llm @xcite . reinforcement learning-based methods have also been adopted for prompt modification @xcite . moreover, the granularity of prompt modification exhibits variation across studies. heuristic-based methods and @xcite ) work operate at the word/token granularity, while classical optimization algorithms like @xcite consider the entire prompt. the se-lection stage generally utilized the performance of the prompt on a held-out validation set @xcite @xcite , while recent work also explores human preference feedback @xcite or score feedback from other llms @xcite . ## experiments in this section, we begin by detailing datasets, baselines, and the implementation of the experiments. following this, we conduct comprehensive and controlled experiments on our method. ## conclusion in this paper, we addressed the issue of low convergence rates in gradient-free prompt optimization methods for llms. our proposed dual-phase approach effectively accelerates prompt optimization by generating high-quality initial prompts and leveraging tuning experience to navigate the optimization process. extensive experiments on several llms across diverse datasets demonstrated the superiority of our method in achieving satisfactory performance within few optimization steps. our approach not only enhances the efficiency of prompt optimization but also improves the overall performance of llms in various nlp tasks. future work will focus on further refining the optimization strategies and exploring their applications in more diverse and complex scenarios.