P3: Prompts Promote Prompting (2025)

Abstract: Current large language model (LLM) applications often employ multi-component prompts, comprising both system and user prompts, to guide model behaviors. While recent advancements have demonstrated the efficacy of automatically optimizing either the system or user prompt to boost performance, such unilateral approaches often yield suboptimal outcomes due to the interdependent nature of these components. In this work, we introduce P3, a novel self-improvement framework that concurrently optimizes both system and user prompts through an iterative process. The offline optimized prompts are further leveraged to promote online prompting by performing query-dependent prompt optimization. Extensive experiments on general tasks (e.g., Arena-hard and Alpaca-eval) and reasoning tasks (e.g., GSM8K and GPQA) demonstrate that P3 achieves superior performance in the realm of automatic prompt optimization. Our results highlight the effectiveness of a holistic optimization strategy in enhancing LLM performance across diverse domains.

## introduction large language models (llms) have achieved remarkable success across a wide range of tasks, becoming a cornerstone of modern ai applications. however, finetuning these models is often costly and data-dependent, driving interest in 'external' tuning methods, e.g., prompt engineering and retrieval-augmented generation, which optimize outputs without retraining. among these methods, prompt engineering stands out as a cost-effective and versatile approach, enabling significant performance gains with minimal adjustments. yet, crafting effective prompts remains challenging, often requiring expertise and iterative manual tuning. recent advances in automated prompt optimization (apo) @xcite @xcite @xcite address this by leveraging llms to refine prompts based on objective metrics or subjective feedbacks (llm-as-judge). while these methods show solid results on several benchmarks, they lack real-time applicability, because their reliance on multiple model calls for each prompt refinement can introduce huge computational overhead. to enable real-time prompt optimization, a new class of techniques for online prompt optimization has emerged, such as bpo @xcite and pas @xcite . these methods can be divided into two stages. in the first stage, they optimize each user prompt and collect (prompt, optimized prompt) pairs as datasets. specifically, bpo optimizes user prompts by rephrasing, and pas adds a complementary instruction as an expansion to the original prompt. in the second stage, the collected dataset is used to finetune a smaller language model, which is capable of dynamically adjusting or extending the original prompt at test time. while promising, these approaches encounter three key limitations: affinity issue: system prompts significantly influence the style and quality of llm outputs. existing methods primarily focus on optimizing user prompts, overlooking the potential of system prompts. to fully unlock the capabilities of prompt optimization, system prompts must adapt to the structural changes introduced by optimized user prompts, ensuring alignment and synergy between the two components. ## methodology in this section, we introduce the background and overall framework of p3. we first give the task formulation for automatic prompt optimization in section 2.1. then we describe the algorithm details of p3 in section 2.2. ## experiments in this section, we conduct thorough experiments to analyze the performance of our proposed methods. we report results with the user prompt optimization iteration number set to 1. additional experiments demonstrating performance improvements with increased iteration numbers are provided in the appendix 7.1. other hyperparemeter settings are listed in appendix 7.6. details of online model training are shown in appendix 7.8. ## conclusion in this work, we introduce prompts promote prompting (p3), a novel framwork addressing three fundamental challenges in online prompt optimization: affinity alignment, response diversity, and inference efficiency. our solution establishes a dual-phase optimization paradigm that combines offline prompt optimization with online adaptation of user prompts.