Prompt Optimization via Adversarial In-Context Learning (2024)

Abstract: We propose a new method, Adversarial In-Context Learning (adv-ICL), to optimize prompts for in-context learning (ICL). Inspired by adversarial learning, adv-ICL is implemented as a two-player game between a generator and discriminator, with LLMs acting as both. In each round, given an input prefixed by task instructions and several exemplars, the generator produces an output. The discriminator then classifies the generator’s input-output pair as model-generated or real data. Based on the discriminator’s loss, a prompt modifier LLM proposes possible edits to the generator and discriminator prompts, and the edits that most improve the adversarial loss are selected. We show that applying adv-ICL results in significant improvements over state-of-the-art prompt optimization techniques for both open and closed-source models on 13 generation and classification tasks including summarization, arithmetic reasoning, machine translation, data-to-text generation, and the MMLU and big-bench hard benchmarks. In addition, our method is computationally efficient, easily extensible to other LLMs and tasks, and effective in low-resource settings.

## introduction generative adversarial networks (gans) and adversarial learning @xcite have driven significant progress across a range of domains, including image generation @xcite @xcite , domain adaptation @xcite @xcite , and enhancing model robustness @xcite @xcite . at its core, adversarial learning frames training as a minimax game between a generator and a discriminator. the generator aims to generate output realistic enough that the discriminator classifies it as real (i.e., not generated), while the discriminator aims to accurately differentiate between generator output and real training samples. after each round, the parameters of both models are updated based on an adversarial loss, and the process repeats. as the generator improves, the discriminator improves alongside it, finding "weak spots" in generator output that may go undiscovered in non-adversarial training, ultimately resulting in better generator outputs. despite success in other domains, applying adversarial learning to pre-training llms is impractical due to the data and computational overheads associated with training two models. particularly for novel tasks where data is often scarce, it is desirable to have methods that can improve model performance using limited data. in this work, we solve this problem by applying adversarial learning to in-context learning (icl) @xcite @xcite @xcite , which has shown to be an effective method to improve model performance with few training samples. though, effective, icl has shown to be sensitive to changes in prompts @xcite . we introduce adversarial in-context learning (adv-icl), which applies insights from adversarial learning to prompt optimization for icl. adv-icl keeps model parameters fixed and instead updates model prompts in an adversarial manner. this alleviates compute and data requirements, while still allowing improvements in model performance. the generator generates an output given an input and task-specific prompt. ## analysis in this section, we examine several design choices of adv-icl. we further discuss the necessity of the discriminator in appendix a.4, as well as an extended set of analyses in appendix a.5. ## conclusion in this work, we introduce adv-icl, an adversarial training framework for in-context learning using large language models. our method has demonstrated empirical success across a diverse range of tasks and outperforms previous sota prompt optimization methods significantly. effective with limited data samples and a very small number of training iterations, adv-icl holds promise for a wide array of real-world applications.