Survival of the Safest: Towards Secure Prompt Optimization through Interleaved Multi-Objective Evolution (2024)

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities; however, the optimization of their prompts has historically prioritized performance metrics at the expense of crucial safety and security considerations. To overcome this shortcoming, we introduce “Survival of the Safest” (), an innovative multi-objective prompt optimization framework that enhances both performance and security in LLMs simultaneously. utilizes an interleaved multi-objective evolution strategy, integrating semantic, feedback, and crossover mutations to effectively traverse the prompt landscape. Differing from the computationally demanding Pareto front methods, provides a scalable solution that expedites optimization in complex, high-dimensional discrete search spaces while keeping computational demands low. Our approach accommodates flexible weighting of objectives and generates a pool of optimized candidates, empowering users to select prompts that optimally meet their specific performance and security needs. Experimental evaluations across diverse benchmark datasets affirm ‘s efficacy in delivering high performance and notably enhancing safety and security compared to single-objective methods. This advancement marks a significant stride towards the deployment of LLM systems that are both high-performing and secure across varied industrial applications

## introduction large language models (llms) have demonstrated impressive capabilities in a variety of fields @xcite . nevertheless, their outputs can differ substantially depending on the phrasing of the input prompt, even when employing the same model @xcite @xcite . in response to this challenge, recent studies have developed a range of techniques for automatically generating optimal prompts. these include gradient-based methods, evolutionary strategies, reinforcement learning (rl) approaches, and fine-tuning practices @xcite @xcite @xcite . considering the complexity of natural language and the intricacy involved in optimization @xcite , these techniques typically focus on optimizing a single metric such as performance accuracy. ## problem formulation prompt optimization (po). considering the task t specified by a dataset @xmath0, a) of input/output pairs, the llm l produces the corresponding output a via prompting with the concatenation of prompt p and a given input q, i.e., [p; q]. the objective of prompt optimization is to design the best natural language prompt p * that maximizes the performance of l on t . multi-objective po. multi-objective prompt optimization extends the above concept to scenarios across multiple objectives. instead of seeking expensive pareto-frontiers, we formulate the optimal prompt p * that performs best across these objectives o by assigning specific weights w and maximizing the weighted sum of the metric function f across all objectives, security feedback mutation kpi feedback mutation crossover mutation semantic mutation initialization … "categorize the sentiment type." final optimized prompt using sos "analyze the emotional tone of a given text and provide a neutral response if the content is potentially harmful or malicious. ## sos: survival of the safest our proposed sos framework leverages evolutionary principles to iteratively refine a set of prompts, aiming to discover solutions that excel across multiple, potentially orthogonal objectives. sos comprises phases from prompt initialization, evolution mutation (semantic, feedback, and crossover), and selection, as shown in fig. 2 . ## industrial deployment sos is an efficient framework that can optimize the performance and security of llms simultaneously in a flexible manner. it allows users to assign different weights to objectives, enabling fine-tuned control over the balance between performance and safety based on specific use cases and requirements. sos can be adapted to different security datasets, allowing companies to customize the optimization to their particular security concerns. sos is not limited to performance and security objectives; it can be applied to any group of objectives with an evaluation system in place. this versatility makes it valuable for a wide range of industrial applications where multiple criteria need to be balanced. for industries that work with sensitive data or high-stakes applications, sos offers a promising way to deploy llms that not only maintain high performance but also significantly improve safety and security. ## conclusion we introduce sos, a novel framework that simultaneously enhances both performance and security in llms. sos addresses critical safety and security concerns in deploying optimized llm prompts, offering a promising approach for developing highperforming yet secure llm systems across various industrial applications. future work could explore online optimization to further improve efficiency.