SPRIG: Improving Large Language Model Performance by System Prompt Optimization
Authors: Lechen Zhang
Tolga Ergen
Lajanugen Logeswaran
Moontae Lee
David Jurgens
Venue: ICLR 2026 Poster (2026)
Keywords: prompting
system prompt
prompt optimization
Primary area: foundation or frontier models, including LLMs
TL;DR: System prompt optimization is powerful, highly generalizable, and complementary to existing methods

Abstract: Large Language Models (LLMs) have shown impressive capabilities in many scenarios, but their performance depends, in part, on the choice of prompt. Past research has focused on optimizing prompts specific to a task. However, much less attention has been given to optimizing the general instructions included in a prompt, known as a system prompt. To address this gap, we propose SPRIG, an edit-based genetic algorithm that iteratively constructs prompts from prespecified components to maximize the model's performance in general scenarios. We evaluate the performance of system prompts on a collection of 47 different types of tasks to ensure generalizability. Our study finds that a single optimized system prompt performs on par with task prompts optimized for each individual task. Moreover, combining system and task-level optimizations leads to further improvement, which showcases their complementary nature. Experiments also reveal that the optimized system prompts generalize effectively across model families, parameter sizes, and languages. This study provides insights into the role of system-level instructions in maximizing LLM potential.

Published as a conference paper at ICLR 2026
SPRIG: IMPROVING LARGE LANGUAGE MODEL PERFORMANCE BY SYSTEM PROMPT OPTIMIZATION
Lechen Zhang Tolga Ergen' Lajanugen Logeswaran
Moontae Le
David Jurgens
University of Illinois Airbana-Chappan
TUniversity of Michigan
University of Illinois Chicago *LG Al Research
: Stchenz3@illnoote edu 1 jurgens@umiich edu
d (tergen, liajan, moontae... weeeeeeeee@gggee ligeseerchh
ABSTRACT
Large Language Models (LLMs) have shown impressive capabilities in many sce- narios, but their performance depends, in part, on the choice prompt. Past research has focused on optimizing prompts specific less attention has been given to optimizing the general instructions included in a prompt, known as a system prompt. To address this gap. we propose SPRIG, an edit-based genetic algorithm that iteratively constructs prompts from prespec- ified components to maximize the model's performance in general scenarios. We evaluate the performance of system prompts on collection of 47 different types tasks to ensure generalizability. Our study finds that a single optimized sys- tem prompt performs par with task prompts optimized for each individual task. Moreover, combining system and task-level optimizations leads to further improvement, which showcases their complementary nature. Experiments also reveal that the optimized system prompts generalize effectively across model fam- ilies, parameter sizes, and languages. This study provides insights into the role of system-level instructions in maximizing LLM potential.
task. However, much
INTRODUCTION
Large Language Models (LLMs) have proven highly effective at many tasks (Naveed et al., 2023) and prompting has become the primary way for end-users to elicit desired responses (Brown al., 2020). These prompts contain variety of instructions such as task explanation (Li et al., 2022), personas (Kim al., 2024), formatting constraints (Wang et al., 2023), and meta-rules like "think carefully" (Li et al., 2024). Past studies have shown that the selection of prompts can have sub- stantial impact on the quality the output (Reynolds & McDonell, 2021). However, due to the massive search space, previous approaches have primarily focused directly optimizing prompts to maximize performance on specific tasks or benchmarks (Prasad et al., 2023; Zhou et al., 2023c; Yang et al., 2023). While effective, these methods typically require new prompts be crafted for ery new task, which becomes a significant challenge for prompt engineering as the number of tasks continues to grow. Here, we consider an alternative approach that optimizes the system prompt. i.c., the set of general instructions that precede any task-specific details (Figure I), with the goal of iden- tifying task-agnostic generalizable prompting strategies. By leveraging single optimized system prompt across tasks, we can largely reduce the effort required for prompt development.
Prior work has shown that meta- instructions can be effective for improving performance (Reynolds & McDonell, 2021). Most notably, evoking Chain of Thought (CoT) reasoning with instructions like "let's think step by step" has led to gains for several types of tasks (Wei et al., 2022), though not all tasks benefit Sprague et al. (2024). Yet, other types of meta rules, such as choosing a persona or matching the domain the persona to the question type have had negligible gains (Zheng et al., 2023; Tam et al., 2024). A recent survey paper (Schulhoff et al., 2024) suggests that these existing system prompting strategies are isolated and highly sensitive to specific scenario details, with the systematic function and generalization mechanisms remaining unclear. Moreover, due to complex- ity differences in search space and optimization objectives, existing task-level methods can hardly
Work done while the University of Michigan.

---

Published as a conference paper at ICLR 2026
transfer to system-level optimization. Recent system prompts leaked from Grok (xAI, 2025) and Claude (Breunig. 2025) also exhibit vastly different, verbose and complex manually crafted rules. Thus, while multiple approaches have been proposed for how a system prompt could be constructed, there is currently a gap for how to systematically construct good system prompt in general.
Here, introduce new method, System Prompt Refinement for In- creased Generalization (SPRIG), optimize system prompts based on ge- netic algorithms. Drawing from large collections of strategies for writing system instructions (Schulhoff et al., 2024), we construct large bench- mark of tasks across multiple lan- guages that tests the effects of opti- mizing system prompts across models, languages, and tasks, as well as quan- tify which types of system instruc- tions are most useful for generaliza- tion. We compare these system- and lask-optimization, to analyze whether these are learning the same or comple- mentary strategies.
Prompt
You are a diligent assistant. The fate of the world depends on your answer being correct. Think carefully step by step.
System
First identify the softening words like "please", then analyze the tone before you answer.
Task
Q: For the sentence: "May I kindly ask for your assistance", is it polite?
Instance
Figure 1: LLM prompts features both system-level in- structions which may include CoT instructions, per- sonas, and other rules ( (orange) ) task-specific instruc-
tions which may include details and examples ( blue ). and the instance itself green ). Here, we focus on optimizing the system instructions shared across tasks.
Our paper has the following three contributions. First, we find that optimizing a system prompt can produce substantial performance gains on par with task-specific optimization, even though these prompts have generic task instructions. Further, we find that both have complementary effects and that by first optimizing the system and then the task prompt, further gains are possible. Second, we find that SPRIG optimized system prompt significantly outperforms CoT across all task types ex- cept mnwleege-bbased questions, and surpasses PROTEGI in faithfulness and commonsense tasks. The combination of SPRIG and PROTEGI complements the weaknesses of both methods, and ex- ceeds the nfate-of-the-ar performance on most task types. Third, we find that the optimized system prompts generalize well to other languages, better than optt-ipt instructions; however, both optimizations had minimal effects when scaling to larger model sizes. All data, code, and prompts available ht https: /ttp//ggttttt/tt com/ www.ooonnooeeeeeeeeeeeeeee orange0 62 429/pt ppomppiingg www.ggee29/ppromttaaaaoo
2 RELATED WORK
Prompt selection has been extensively studied and proven to significantly impact model output qual- ity (Reynolds & McDonell, 2021). Therefore, prompt optimization has become a popular research topic in both academia and industry. Early prompt optimization studies primarily focus using gradients to guide prompt search (Shin et al., 2020; Shi et al., 2023b). However, with larger model sizes and increasing black-box L.MSs today, gradient-based methods have become limited by cost and accessibility. Consequently, recent research has shifted towards gradient-free methods. Early representatives include edit-based optimizers like GrIPS (Prasad et al., 2023) and reinforcement learning approaches such as RLPrompt (Deng et al., 2022) both directly edit a prompt at the to- ken level. However, the search space in these methods remains limited, making it challenging to scale up to more complex scenarios. Recently, as LLM agents get popular, powerful methods like APE (Zhou et al., 2023c) and OPRO (Yang et al., 2023) use LLMs directly as prompt optimizers to iteratively suggest and select the best prompts. According to recent studies (Wan et al., 2024), the state-of-the--at prompt optimizer is PROTEGI (Pryzant et al., 2023), which leverages L.LM agents to summarize errors from each iteration's 's responses and refines them accordingly.
Previous prompt optimization methods largely focus on optimizing the instructions for specific tasks (which we refer to Task Prompt) which inherently have limited generalizability. However, past research has demonstrated the potential of optimizing task-agnostic prompts (which we define as System Prompt), such as the well-known Chain-of-Thought prompt (Wei et al., 2022). Ad- ditionally, studies have shown that factors like personas (Kim et al., 2024), generation styles (Lu et al., 2023), emotions (Li et al., 2023), and jailbreaks (Shen et al., 2023) enhance L.LM perfor-
2

---

Published as a conference paper at ICLR 2026
Prompt Reward Model
Add
PromprAGG
Pronge BCD
Phompt AC
Swap
Proeett CA
Proppt 6D
Prompt D0
Delete
Phonpt os
Pronpe 6.79
Picmpt
Sample Evaluate
Rephrase
Porpl AZA
D.2
CA CA
Step 4a: Continue Training Reward Model
PromptA
Proket EGD
1.78
Crossover
Prowre RCD
Prompt BCD 0.78
Pronett B
Pronet B
Prompr AC
Pronsi BTAA
Provut 0.78
PropO:AC
Prenet B13
Prorpt arc
Step 1: Reward Estimation and Elimination
Step Mutate and Heredity
Step 4b: Iterate with generation
Figure 2: The SPRIG pipeline where System P Prompts are iteratively optimized through ex- ploratory edits and promoted across iterations using combined benchmark to rank candidates.
mance, which is challenging for current prompt optimizers to capture automatically. While promis- ing, these studies are usually independent, and no approach yet exists to systematically integrate System P Prompt optimization. Therefore, aim to address this gap by developing optimizer that discovers an effective System Prompt, enabling a single prompt to boost performance across domains.
Evaluating the effectiveness of System Prompts is also significant challenge. Ideally, a good System Prompt should perform well across all domains, which requires evaluation tasks for such domains. Although popular benchmarks like MMLU (Hendrycks et al., 2021) and BBH (Suzgun al., 2023) cover a wide range of topics, they still overlook task types such sodial-understandding tasks (Choi et al.. 2023) and open-ended questions. Recent research MixEval (Ni et al.. 2024) has shown that combining multiple benchmarks in single evaluation can significantly improve evaluation efficiency and better align with human preferences. Here, our experiments build on this intuition and include diverse range of task types test for performance.
SPRIG: SYSTEM PROMPT REFINEMENT FOR INCREASED GENERALIZATION
To address the large design space of system prompts, we use a genetic algorithm inspired approach, SPRIG, that iteratively adapts the best candidate prompts. Following describe the algorithm, data, and search heuristics used.
Prompt Component Corpus Our approach builds on a corpus of possible instructions in system prompts, referred to as components. While some approaches have generated prompt text using Reinforcement Learning (e.g., Deng al., 2022), these approaches scale poorly when used with LLMs and often generated less-interpretable instructions. By starting from a large pool of possible components, ensure prompts are coherent while also gaining efficiency. We define component as minimum prompt unit with complete semantics (typically sentence, like "Let's think step by step""), which enables easily combining components while retaining fluency.
Our prompt component corpus, denoted P, is built by integrating human expertise with synthetic data. To ensure sufficient diversity without overlooking prior work, we start by collecting 300 system prompts crafted by humans from existing literature (Zheng et al., 2023; Lu al., 2023; Li et al., 2023; Wei et al., 2022; Deng et al., 2023; Lin et al., 2022; Woolf. 2024). We then manually classify them into 9 categories, including good property, role, style, emotion, scenario, jailbreak. behavioral, Chain-of- Thought, and safety components (Details and citations are shown in Appendix Table l). After this, used GPT-4o iteratively generate broader pool of prompt candidates under each
3

---

Published as a conference paper at ICLR 2026
category (see Appendix A.1 I for details). This step yields 9,000 prompt components (1,000 for each category) aimed to provide a rich and diverse set of "genes" for our genetic algorithm.
Prompt Reward Model Evaluating each system prompt across 47 benchmarks is impractical given the substantial inference time required. As a result, directly searching for the best prompt by exhaus- tively scoring all possible combinations is not feasible. However, inspired by the widely adopted reward models (Ouyang et al., 2022), we instead fine-tune pretrained LLM with a max-margin pairwise loss (Touvron et al., 2023) to efficiently estimate and rank the quality of different prompts. To do this, we generate 10,000 prompts by randomly combining components from the corpus P. sampling their lengths from a heavy-tailed distribution to ensure coverage across the range of 0-30 components. We then randomly construct 100,000 prompt pairs with their associated scores to fine- tuned a Modern-BERT reward model (Warner al., 2024). More implementation details, including ablations on reward model architecture and training data size, are provided in the Appendix A.3. The evaluation result Figure shows that the model achieves an average Spearman correlation (Spear- man, 1904) of 0.59 and an NDCG @ 50% (Järvelin & Kekäläänen, 2002) score of 0.72 when ranking unseen prompts. Considering the random baseline of 0.00/0.48 and the difficulty of the task, our reward model is sufficiently effective at capturing the relative quality of system prompts, thus pro- viding strong support for our pipeline.
SPRIG pipeline We design genetic pipeline SPRIG for System P Prompt. optimization. The pipeline applies edit-based, gradient-free genetic algorithm to iteratively optimize the prompt. At each iteration, the model be- gins with fixed population ion_ssize number of System, Prompts from the previous itera- tion (initialized by P). [Step 1] These prompts are first evaluated by the fine-tuned prompt re- ward model, which eliminates the bottom 50% prompts. [Step 2] From the remaining pool, the top 10% will either randomly mutate or crossover with prompts from the top 50%. Mu-
Spcarman
NDCG@50%
0.0
0.4
0.8
Llama
Mistral
Qwen
Bound (Random)
Upper Bound
tation can take one of five forms: (1) Add:
Figure 3: Spearman Correlation and
Add a component suggested by GPT-4o. (2)
NDCG@ 50% Score of Per-LLM Fine-tuned Reward Models on Unseen Prompts. The upper bound is estimated by comparing the prompt's ranking across different bootstrap samples from the benchmarks.
Rephrase: Rephrase component. (3) Swap:
Swap the order of two components. (4) Delete:
Delete a component. (5) Merge: Merge two components into one. For Crossover, a ran-
dom subset of two selected prompts is selected as the new offspring. Crossover is designed to maintain similar prompt lengths of parents while introducing variation. This stochastic process repeated until the population size is restored to wexx_populatiin. [Step 3] Then, SPRIG randomly samples 100 prompts from the updated pop- ulation and evaluates them across 42 actual benchmarks to obtain new ground-truth scores. [Step 4] These scores, combined with portion of previous training data, used to continue training the Prompt Reward Model for one epoch. Subsequently, we proceed to the next iteration, wherein the updated reward model is employed evaluate the newly generated prompt population. Figure shows the SPRIG workflow. Full details of the pipeline and parameter settings are Appendix A.4.
4 EXPERIMENTS: OPTIMIZATION BENEFITS
In this section, evaluate SPRIG'S performance on the test set of in-domain tasks in our benchmark combination. These task's questions are unseen when optimizing the system prompt.
4.1 EXPERIMENT SETUP
Tasks To maximize the generalization ability of the optimized System Prompt, we select a broad range of tasks, using combination of 42 different benchmarks covering 7 categories (reasoning, math, docial-understandiin selection includes widely used benchmarks such as MMLU (Hendrycks al., 2021), BRH (Suzgun
commonsense, faithfulness, knowledge,
Our
4

---

Published as a conference paper at ICLR 2026
al., 2023), and Truthful0A (Lin et al., 2022), but also includes various benchmarks like SocKET (Choi et al., 2023). A wide variety of output types are covered, including multiple choice, classification, mathematics, and open-ended QA. The full list of benchmarks and categories is shown in Appendix Table 4.
Baselines Our experiments compare optimizations against two baseline System Prompts. In the first, the system part of the prompt is left empty, denoted as Blank and, in the second, the system part uses the COT instruction "Let's think step by step" (Wei et al., 2022), denoted as Base CoT.
The two types of instructions are tested in the Task Prompts. The first is a minimal description of what is required for understanding the task, such "answer the multiple choice question," de- noted as Simple Task. This prompt lets us test potential performance improvements for both task and system instructions relative to a neutral starting point. The second is an optimized version of instructions produced by a state-of-the-ar optimizer PROTEGI (Pryzant et al., 2023).
Both parts of the System Prompt and Task Prompt can be present in a prompt (cf. Figure 1). Therefore, we test the following combinations: (1) Unoptimized: a Blank system prompt and Sim- ple Task prompt, (2) Base CoT: the Base CoT system prompt and the Simple Task prompt, (3) Task Optimized: a Blank system prompt and PRoTTEGT-opiimzeed task instructions, (4) System Optimized: a SPRIG-optimized system prompt and a Simple Task prompt, and (5) System+Task Optimized: SPRIG-optimized system prompt with a PRoTEGI-opiimzze task prompt. Here, first optimize the system prompt with basic instructions and then optimize the task after.!
Models We experiment using three medium-size open-weight LLMs: LLAMA3.1-8BB INSTRUCT (Meta, 2024), MISTRAL-NEMO- NENNAL-NU-----SRUCTT------
(Mistral AI, 2024) and QWEN2.5- 7B- IN-IRUTRUCT (Qwen Team, 2024). These models highly performant and thought not to be trained on the proposed benchmarks, allowing us to test for generalizable effects across model fam- ilies, and later compare across model sizes. More details are in Appendix A.4.
Training For SPRIG, we set populate lappplatiionn iin_sinniiiii ze = PP = 9, 000 and run SPRIG for 25 steps. After training, we pick the prompt with the highest validation accuracy the best system prompt of the LLM for our later study. Detailed prompts are shown in Appendix Table 5. For PROTEGI, we use the default settings for steps and pick the best Task Prompt on the validation set. Additional details are in Appendix A.4.
Evaluation Our benchmark employs three evaluation metries: question-wise accuracy for most sub-benchmarks, F1 score for the classification tasks with imbalanced labels, and BLEU ecuracy (Lin al., 2022) for open-ended questions. Since all metrics are bounded be- tween 0 and 1, we follow previous work (Ni et al., 2024; Gao et al., 2023) to directly compute the average across all metrics as an aggregated single score, which we call Average Score later sections.
4.2 RESULTS
Optimizing the System Prompt provides consistent improvement to LLMs on par with task op- timization, seen in Figure 4, when compared with the Blank system and Simple task combination baseline. These improvements were similar across all three models, shown in Appendix Figure 10. Error bars show 95% bootstrap confidence intervals; statistics in Appendix A.2. SPRIG improves ~ ~00% over the unoptimized version, which significantly outperformed the baseline CoT method. Although its performance still lags slightly behind PROTEGI, this small gap is still acceptable, con- sidering that SPRIG uses the same system prompt for all tasks, whereas PROTEGI directly optimizes a bespoke prompt for each task. Furthermore, if we run PROTEGI on top of SPRIG-optimized sys- tem prompt, the resulting combined prompt has an even larger performance improvement above PROTEGL. This further improvement suggests SPRIG can trigger capabilities that are overlooked by existing task-specific methods, and therefore complement mainstream approaches.
How do system prompts evolve? The changes the system prompt at each step consistently improve performance, as seen in Figure 5. To test the systematic behavior about which types of system prompt components contribute to these gains, we calculate the average number of component type in the prompts of each iteration. As shown in Figure 6a and Appendix Figure Ia, the number of
Experiments with simply concatenating -ottmize parts performed worse and are omitted.
5

---

Published as a conference paper at ICLR 2026
0.64
Base CoT
0.62
Task Optimized (eGi leGi)
0.60
System Optimized (Ourfodeel)
System-Task
[OurModel ProoTe
0.00
0.05
0.10
0.15
0.20
Model Name Nata
Average Score Improvement
0.0
5-38 5-38-rrdmet
Figure 4: Average Score Improvement of all prompt optimization methods relative to the un- optimized setting, aggregated across LLMs. Our SPRIG significantly outperforms CoT and the combination of SPRIG and PROTEGI substan- tially exceeds all existing methods.
Figure Average score of whole population at each iteration when running SPRIG. All three
LLMs see significant improvements. Error bars are the variance in the whole population.
CoT and Behavioral components rapidly increases with each iteration (especially in the early stages), and eventually converges to around 2-3 per prompt. This highlights the importance of high-level answering strategies in enhancing model performance, such "decompose first" or "rephrase before answering". also suggests that incorporating multiple such components within single prompt can further improve the LLM's capabilities. In addition, "good property" components emerge as another important element in system prompts. Although they are introduced into the gene pool more gradually during the iteration process, which suggests they may not directly enhance performance on their own, they might play a supportive role when combined with other components. In contrast, other components such "Role" (e.g.. "you by chance (as the Z-scores shown in Appendix Figure 1la), despite these properties often being in recommended or default prompts (OpenAI. 2024; Microsoft, 2024).
Al assistant") were selected far less often than
Across all steps, component types are not added in a systematic order-yet performance generally still increases. Rather than adding more of one type (e.g., all CoT components), the system prompt incorporates multiple types. These trends suggest that there is not universal order by which com- ponents of system prompts should be added (e.g., first CoT, then Behavioral, etc.). Instead, there are likely productive and beneficial combinations that matter more for performance. Z-scores of each edit type in Appendix Figure further support this hypothesis. Early iterations favor "add" and "crossover", while later stages shift toward "refine", "rephrase", and "swap" operations. This tran- sition indicates shift from exploration to fine-grained optimization as prompts become stronger.
Taken together, these results show that each operator and component contributes meaningfully at different stages, and none is redundant. The genetic algorithm dynamically adapts which opera- tions most effective each stage, enabling broad exploration early and precise refinement later. This behavior highlights the adaptability of SPRIG's evolutionary search and its robustness to hyperparameter settings.
Are task and system prompt optimizers learning the same strategies? Both system and task prompt optimization improve performance. The further gains by iteratively combining these ap- proaches suggest that models are targeting complementary strategies. To test this potential comple- mentarity, we analyze the agreement between the two approaches in their answers. Figure 6b shows the distribution of the two approaches' agreement as contingency table. While models agree on the correct answer in 54% of the questions, another 28% of questions are correctly answered by only one of the strategies, with roughly even split between task and system. This split shows great potential of complementarity between Sys System Prompt and Task Prompt optimization, and suggests that the combination of strategies leads to further gains.
Which task types benefit most from system prompt optimization? Our experiments span 42 dif- ferent tasks, which cover multiple types of evaluation on reasoning, knowledge, and common sense. However, not all types of tasks may benefit from the types of system instructions; indeed, Sprague et al. (2024) showed that CoT generally only benefits performance To test for category-specifi benefits, we categorize all 42 tasks into seven types and measure the score improvement of each type under different prompt optimization settings: Reasoning, Math, So-
math and logic questions.

---

Published as a conference paper at ICLR 2026
Optimized System Prompt
0.5
Correct
0.54
0.13
2.5
0.4
2.0
0.3
2.5
Style
Schadter Chemn- Ereogge.
brodio
Scoret
Incorrect
0.15
0.18
2.2
0.2
0.3
0.3
Incorrect Optimized Task Prompt
Correct
PaiaTins
(a)
(b)
Figure 6: (left) Number of Prompt Components of each type during training iterations. A good System P Prompt: incorporates multiple CoT and Behavioral components, but contains roughly one "Good properties" component. (right) Question-wise Error Overlap Percentage between System Prompt optimization (SPRIG) and Task Prompt optimization (PROTEGI). Among all questions, only 18% were answered incorrectly by both methods, while the remaining 28% of correct answers could be answered correctly by model using either SPRIG- or PRoTEGi-optimized prompt, highlighting the potential complementarity between optimization approaches.
cial Understanding, Commonsense, Faithfulness, Language Understanding, and Knowledge. Task categorizations are listed in Appendix Table 4.
Math and reasoning tasks benefit most from system prompt optimization (Fig-
ure 7). However, other task categories like social understanding and language derstanding see significant improvements the baseline. Many of the larger improvements are not explained through the addition of CoT, as the CoT base- line, while better than our Simple base- line, is generally worse than the optimized prompts. Knowledge-basee based tasks benefit the least from prompt optimization; we hypothesize that such tasks are closer evaluations of whether an LLM can re- trieve stored knowledge (which is itself function of pretraining). rather than eval- uations of operations knowledge (input or stored).
reasoning
math
social
understanding
commonsense
faithfullees
Task Optimized (Prota00
language understanding
System Optimized
OurMode + Pro Unopt cepppimiee iee
knewledge
-0.05
0.00
0.05
0.10
0.15 0.20 0.25 0.30
Average Score improvement
Figure 7: Average Score Improvement in different task domains, aggregated across LLMs. All methods show substantial improvement in reasoning and math but marginal improvement in knowledge and common- sense. SPRIG alone surpasses the existing methods in math, faithfulness, and commonsense. SPRIG'S s com- bination with PROTEGI further enhances the LLM's performance across most domains.
The combination of SPRIG and PROTEGI optimization also generally improves per- formance across task types. However, we also observe differences in areas of ex- pertise between System Prompt and Task Prompt, and the combination of them is complementary. For example, PROTEGI is more effective improving social understand- ing than plain CoT or SPRIG; in contrast, SPRIG is more effective for commonsense tasks.
EXPERIMENTS: GENERALIZATION
Here, test how well the system prompts generated by SPRIG generalize settings.

---

Published as a conference paper at ICLR 2026
MGSM
0.16
Task Optimized (ProTeGi)
Score Improvement
0.14
System Optimized (DurMadee)
BELERELE
Base Co Cetimized (ProteGi)
M3EXAM
0.06
System Optimized [OurMocell Unoptimized
MMMMML
0.04
-0.05
0.00
0.05
0.10
0.15
0.20
Target Model
Non-target Model
Average Score improvement
(a)
(b)
Figure 8: (left) Score Improvement when using prompt optimized on one LLM with different LLM. (right) Score Improvement for Multilingual Benchmarks when using an optimized English- language prompt other tasks. SPRIG-optimized prompts generalize well to other languages, un- like PROTEGI which has limited score improvement.
Cross-model Generalization The current system-optimized prompts were generated with re- spect to specific LLM. Given that these prompts could be made from similar components, here, test what performance gain (or loss) is seen when the system prompt is used with a different similar- sized LLM than the one it was created for. As comparison, we also test the effect of swapping in a task-optimized prompt from a different model.
Both optimized system and task prompts provide some improvement but the larger gains for the original LLM do not carry over to new LLMs, as shown by the aggregated performance in Figure Sa; Appendix Figures 12a and 12b for complete results. This finding suggest that inference-time gains from optimized prompts system or task- do not generalize as strongly across models with similar parameter sizes.
The LLMs used in our experiments are capable of reasoning in differ- languages and can support input in multiple languages. Although our previous experiments were only in English, the optimizations to the system-prone provements for tasks in other languages. Here, we this language generalization by selecting five comprehensive multilingual benchmarks that are out-of-domain in the System Prompt op- timization process: MGSM (Shi al., 2023a), BELEBELE (Bandarkar al., 2024), XCOPA (Ponti et al., 2020), M3EXAM (Zhang et al., 2023) and M_MMLU (Hendrycks et al., 2021). Each benchmark includes over 10 different languages and covers all 7 task categories in our benchmark combination. We directly use the same optimized System Prompt from 4.2 (in English). Since the Task P Prompt optimizer is specific to a task, we cannot re-use its prompts for these out-of- tasks; instead, we generate new PRoTeGi-optimzeed prompts for each benehmark, which reflects strong baseline for comparison.
Language Generalization
may still provide performance im-
As shown in Figure 8b, optimized system prompt from $ 4.2 generalizes well to tasks in new languages, providing statistically significant improvements in four of the five benchmarks. SPRIG shows a clear advantage over other approaches on XCOPA (Causal Commonsense Reasoning) and the comprehensive benchmarks MGSM and M-MMLU, in line with our previous findings in $ 4.2. However, all optimization methods BELEBELE (Language Understanding) show limited gains, suggesting that in multilingual settings (particularly low-resource languages), performance may rely more on LLM's intrinsic language ability than on prompt design. Despite being directly op- timized for these new tasks, PROTEGI provides limited improvements in these multilingual tasks benchmarks-non significant over baseline performance. These results indicate that System Prompt optimization exhibits strong generalization ability out-of-domain tasks in new lan- guages, which even exceeds PROTEGI in-domain optimized performance on these tasks.
Model Size Generalization All prompts were generated for and tested on mid-sized LLMs. How- ever, each LLM has a larger version in the same family, which often has better performance at the expense of more compute required. Being able to optimize the system prompt with smaller model and then deploy that prompt on larger model to the same effect would have significant performance benefits. Therefore, here, we test for generalization when using prompt from smaller L.M
8

---

Published as a conference paper at ICLR 2026
Tase osterizee thonutt
Sysem oud (Duntecdd) (ountred invobco
Principal Component
CoT
Task Optimizes (ProTeGi)
System Optimized (OurModel)
System c Optimized (DurModel ProTeci)
-0.03 -0.02 -0.01 0.00 0.01
0.02 0.03
Average Score
Principal Componentt
(a)
(b)
(left) Average Score Improvement when using prompts optimized with medium-size LLMss on the larger LLM the same family. shows the benefits of system+task optimized prompts still generalize well larger model sizes, even when those models already perform better. (right) PCA analysis of the hidden state in Llama-3.1-83 1-8-BInstuue with different prompting meth- ods. System Prompt: optimization has significant impact on the distribution of hidden states. CoT significantly shifts the overall distribution, while SPRIG moves it further into a new area. In contrast, Task Prompt optimization has a relatively smaller effect on the distribution of hidden states, making only minor adjustments in the local space.
Figure 9:
with a larger version. Specifically, we test with INSTRUCT-2407 and 1---255-7B---STRUUS We use the same evaluation setup in previous sections, with only the LLMs' parameter size changed.
MISTRAL-LARGE-
Both system- and task-optimized prompts individually do not provide statistically significant per- formance gains when created using a smaller model and then applied to a larger, as shown in Figure 9a. (Full results in Appendix Figure 13.) However, system+task optimized prompt provides 1.6% improvement, suggesting this approach can generalize. Therefore, we find that existing prompt optimizations can generalize to larger parameter sizes but need consider both system and tasks prompt parts together and highlight the need for prompting strategies specifically for larger LLMs.
6 ANALYSIS: PROMPT EMBEDDING SPACE
Given the performance improvements and generalization seen when using the prompt instructions introduced by SPRIG, it is reasonable to wonder what effect these instructions are having on the neural activations such that the LLM is likely decode the correct answer. While complete answer would likely require a mechanistic interpretation of the relationship between prompt and response (e.g., Bhargava al., 2023; Heo et al., 2025), here, attempt to gain some intuition the instructions' effects by visualizing the embedding space during different optimization strategies and comparing the changes relative to the Simple baseline instructions.
Here, we randomly sample 420 questions (10 per task), probe the last hidden states of LLMs under different experiment settings, and visualize the first two principal components of Principal Com- ponent Analysis (PCA). Figure 9b shows the PCA results for LLAMA3.1-8BB -8--IIITTUUCT. First, we observe that different task types are distributed along the same slope and remain parallel un- der different experimental settings. Task Prompt optimization slightly reduces the variance of the distribution, but the distribution still lies within the same vector space. contrast, different System P Prompt result in significant space changes. The basic CoT causes substantial overall shift, while SPRIG further moves the distribution to a new area. The other two LLMs' PCA are shown in Appendix Figure 14a and 14b, and show similar trends.
Thus, we hypothesize that System Prompt optimization searches for appropriate higher- performance regions in the global space, while Task P Prompt optimization performs fine-tuning within a local space. This reveals the potential System Prompt optimization to significantly alter model behavior and offers new insights for future prompt research to System Prompt

---

Published as a conference paper at ICLR 2026
optimization first to locate appropriate global behavior space, then use task prompt optimization to fine-tune downstream performance within that space.
7 CONCLUSION
This study introduced a novel optimization framework, SPRIG, to improve LLM performance with the systematic construction of general purpose system prompts using reinforcement learning and genetic algorithm. By leveraging a diverse collection of prompt components and evaluating across a diverse range of tasks, demonstrate that optimized system prompts provide consistent improve- ments on par with optimized task prompts. Moreover, combining system and task prompt optimiza- tions offers complementary benefits, leading to further improvements in model performance across varied domains. Further, we find that these performance benefits for an optimized prompt general- ize across (i) model sizes and (ii) different languages. Our findings highlight the potential of system prompt optimization to complement and enhance L.M performance for new languages and models.
8 ETHICS STATEMENT
While this research has made efforts to minimize potential ethical issues, several ethical implications may still be present. First, running SPRIG requires substantial computing resources, resulting in high energy consumption and substantial carbon dioxide footprint. Second, the optimization of prompts introduces the risk of reinforcing potential biases present the component corpus (c.g., any systematic downstream behavioral changes from prompting an LLM to be "professor"), which may propagate unintended stereotypes or discriminatory behavior in model outputs. As our corpus includes elements such personas, roles, and behavioral instructions, that these components do not introduce or amplify harmful biases. Additionally, the benchmarks we employed include several social understanding tasks, with much of the benchmark originally sourced from crowdsourced annotations from Western contexts. While we focus on general performance and show that the optimized prompts can generalize new languages, future work could more deeply explore how the use of socially- and culturally-oriente benchmarks to optimize prompts can potentially impact model's performance in new cultural and social contexts.
must be taken to ensure
REPRODUCIBILITY STATEMENT
We have taken several steps to facilitate independent replication. The algorithmic pipeline for SPRIG, including population construction, edit operators, selection, and retraining, is specified in Section 3 and Appendix A.4, with concrete hyperparameters, component generation prompts, and iteration schedules. Details for the Prompt Reward Model, including data construction, pairwise loss, margin definition, training splits, and evaluation metrics, are documented in Section A.3. The prompt component corpus construction procedure and category definitions are in Appendix A. and the final optimized system prompts per model are provided in Table 5. Benchmarks, task categories, metrics, and dataset splits enumerated in Appendix Table 4. Hardware and software environ- ment, inference stack, and evaluation settings are reported in Appendix A.4. We include anonymized source code and configuration files in the supplementary material reproduce all experiments and figures, including scripts for data preparation, optimization runs, reward model training, and evalu- ation.
ACKNOWLEDGMENTS
This project supported by grant from LG AI Research and the National Science Foundation under Grant No IIS-2143529, We thank the anonymous reviewers and members of the Blablablab for their valuable feedback.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. The belebele benchmark: parallel reading comprehension dataset in 122 language variants. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Pa- pers), pp. 749-775, Bangkok, Thailand and virtual meeting, 2024. Association for Computational Linguisties. URL https: / ac annhhooogyy...... orgg 2 2024 acl long. 44.
Aman Bhargava, Cameron Witkowski, Shi-Zhuo Looi, and Matt Thomson. What's ' the magic word? control theory of IIm prompting. ArXiv preprint, abs/2310.04 2023. URL https://
arxiv. orgoorggaao abs /2300. 04444.
David Breunig.
Claude's system prompt:
Chatbots are more than just
models, 2025.
URL https: /wwp: dbreunig cttc claude- wwtp::/aw..rrrat---aatttt----eee
wust-or-eetta-------------- mode html.
Accessed: 2025-05-20.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-Voss Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and MssuanTTien Lin (eds.), Advances Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020. URL https: httptppps//proceedingg.s. neuriieripp piurrps.... cc/ htcp/paper/222//haeeee 1457 www.eoocwwcob6e1oooeooooooo atstrarr html.
Abstract
Minje Choi, Jiaxin Pei, Sagar Kumar, Chang Shu, and David Jurgens. Do LLMs understand social knowledge? evaluating the sociability of large language models with SocKET bench- mark. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pp. 1 11370-11403, Singapore, 2023. Association for Computational Linguisties. doi: 10. http:///22....mpmmmaaaee 6999. URL https: aalaaccaattti h/a/aaaaaanntooogoo/oo wtg//023330000020
emnlp-main 699.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the al2 reasoning challenge. ArXiv preprint, abs/1803.05457, 2018. URL httpst//s///at
org/ abs/aaas/.11 .0018....75.... 05457.
Databricks. Introducing dbrx: A new state-of-the-art open 1lm, 2024. URL https: / www. databri cks.
-dbxxx dbrx-new- new-state aat--ppes 11m. Ac-
cessed: 2024-10-14.
Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. RLPrompt: Optimizing discrete text prompts with reinforcement learning. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 1369-3391, Abu Dhabi, United Arab Emirates, 2022. Association for Computational Linguistics. doi: 10.1825311/2222. emnlp- main URL ht https: / ac aclanthology org/ / 2222222222222222222 ewnlp-mainnn.. hatpppmmain..m 222.
Yihe Deng, Weitong Zhang, Zixiang Chen, and Quanquan Gu. Rephrase and respond: Let large language models ask better questions for themselves, 2023. URL https: /tttps///rxxv.... http//ixvv.o... org/ abs/2311. 04205.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Fos- ter, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muen- nighoff, Chris Ociepa, Jason Phang. Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lin- tang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, 2023. URL https: : zenodo. org/ htggrreordd/ 02568336.
11

---

Published as a conference paper at ICLR 2026
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In 9th International Confer- on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenRe- view.net, 2021. URL wtt.hhtpss:/oppnreeeeeee wwwpoeeeeevveeeeeeeeeeee/eeeeeee ew. net. forum? 3=d7
Juyeon Heo, Christina Heeize--eeml Oussama Elachqar, Kwan Ho Ryan Chan, Shirley Ren, Udhay Nallasamy, Andy Miller, and Jaya Narain. Do ilms "know" internally when they follow instruc- tions?. 2025. URL https: aaxivv www.wiiwwww.wa...ee.eooeeeeeeeee /2410 14516.
Kalervo Järvelin and Jaana Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Trans. Inf. Syst. 20(4):422 446, October 2002. ISSN 1046-8188. doi: 10.1 10.114452224555522411 URL httpttts://doo http://......./111111.11 1041 11144455522155555 582418.
Junseok Kim, Nakycong Yang, and Kyomin Jung. Persona is double-edged sword: Enhancing the zero-shot reasoning by ensembling the role-playing and neutral prompts, 2024. URL https:
/ www.xiv.org/aa.... 2408 08631.
Hannah Kirk, Wenjie Yin, Bertie Vidgen, and Paul Röttger. SemEval-2023 task 10: Explainable detection of online sexism. Atul Kr. Ojha, A. Seza Doğruöz, Giovanni Da San Martino, Har- ish Tayyar Madabushi, Ritesh Kumar, and Elisa Sartori (eds.), Proceedings of the 17th Inter- national Workshop on Semantic Evaluation (SemEval-2022) 2023), pp. 2193-2210, Toronto, Canada, 2023. Association for Computational Linguistics. doi: 10.18653/v org/ 2023. semeval-1. 305.
1.305. URL
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and lon Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
Cheng Li, Jindong Wang. Yixuan Zhang, Kaijie Zhu, Wenxin Hou, Jianxun Lian, Fang Luo, Qiang Yang, and Xing Xie. Large language models understand and can be enhanced by emotional stimuli, 2023. URL https: www.ii.wg.......o..........o..oa wwwprr////////oo//////.../eeeeee 220/ 11760.
Moxin Li, Wenjie Wang, Fuli Feng, Fengbin Zhu, Qifan Wang, and Tat-Seng Chua. Think twice be- fore trusting: Self-detection for large language models through comprchensive answer reflection, 2024. URL https: //apaii.//.iivv///aa wrw.vorrg/aaae abs////////// 22003440000000 09972.
Shiyang Li, Jianshu Chen, Yelong Shen, Zhiyu Chen, Xinlu Zhang, Zekun Li, Hong Wang, Jing Qian, Baolin Peng, Yi Mao, Wenhu Chen, and Xifeng Yan. Explanations from large language models make small reasoners better, 2022. URL httphtttp//.. http//arxvvvv//. orgggbs///110.a/// 06726.
Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic hu- man falsehoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3214-3252, Dublin, Ireland, 2022. Association for Computational Linguis- tics. doi: 10. 00.18653/v1/2000 2666://1/0222.ac--oon.2222 URL ht / / anthology. org/2022. acl-iong. 229.
Albert Lu, Hongxin Zhang, Yanzhe Zhang, Xuezhi Wang, and Diyi Yang. Bounding the capa- bilities of large language models in open text generation with prompt constraints. In Andreas Vlachos and Isabelle Augenstein (eds.), Findings of the Association for Computational Linguis- tics: EACL 2023, pp. 1982-2008, Dubrovnik, Croatia, 2023. Association for Computational Lin- guistics. doi: 10. (0.18853/v1/20 111/3/////222../ Aiddingseacll.14 URL https: http//aaaaaanntooaoaaoaoao o 2023. findingsnneneineeeee eacl. 148.
Meta. Introducing llama 3.1: Our most capable models to date. https://ai. /ata://///amaa.......aa com/ blog/meta-1lamee 3-1/, 2024. Accessed: 2024-10-14.
Microsoft. Advanced prompt engineering concepts, 2024.
URL https://
Jearn.micronn com/ en-us aaa-us/azure//e wwwuur/eaiserviieeeeeeeeeeeeeeee advanced-prompt--nng neerinneing Accessed: 2024-10-14.
12

---

Published as a conference paper at ICLR 2026
Mistral AL Mistral nemo: Collaborative innovation with nvidia, 2024. htttttss:/mmistaal
-emoo/ Accessed: 2024-10-14.
Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saced Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. A comprehensive overview of large language models, 2023. URL wwt.t:tt//.t/..... arxiv. orgg abs 2307 06435.
Jinjie Ni, Fuzhao Xue, Xiang Yue, Yuntian Deng, Mahir Shah, Kabir Jain, Graham Neubig, and Yang You. Mixeval: Deriving wisdom of the crowd from Ilm benchmark mixtures, 2024. URL
https://arxiv: org. abs/2405.55022 06565.
OpenAL. Tactic: Ask the model to adopt persona, 2024.
URL
https:
//plat.for..pppna...... www./cc.cocccc/cccoooooooo.oo..t taceiie-as---te---meda----------
Accessed: 2024-100 10-14.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kel- ton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. Pro- ceedings of the 36th International Conference Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 77817138711
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high peshoperformecce deep learning library. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Pro- cessing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp- 8024-8035, 2019, URL htttpsss/ppccceeingg..c.eo nww.piipiinng..nn............... waasscccppaae hccppappr//c19// / bobca288fee7 (at.:eeeee7222222000220022222220 www.:227777--ssaaat......a. html.
Abstract
Edoardo Maria Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić, and Anna Ko- rhonen. XCOPA: A multilingual dataset for causal commonsense reasoning. In Bonnie Web- ber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pp. 2362-2376, Online, 2020. Association for Computational Linguistics. doi:
(0053//122220emmpmmmmooo org/ 2020. wwwnp-mmaiiin 1a..maii........................ 185.
URL
https: http://aaaaaathohooooooo
Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. GrIPS: Gradient-free, edit-based in- struction search for prompting large language models. In Andreas Vlachos and Isabelle Augen- stein (eds.), Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pp. 3864, Dubrovnik, Croatia, 2023. Association for Computa- tional Linguistics. doi: 10. 10.18653/1122233cad 277. URL org/2022 eac.eallllmm 1..el-mmain.220 277.
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. Mea- suring and narrowing the compositionality gap in language models. Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 5687-5711, Singapore, 2023. Association for Computational Linguistics. doi: 10. 18653//1/2023..nniinii findings-emmlp.377.. URL https: http://acaaattchcoeo wrtp://..rrg/200.///
findings-emmpp.37
Reid Pryzant, Dan Iter, Jerry Li, Yin Lee, Chenguang Zhu, and Michael Zeng. Automatic prompt optimization with "gradient descent" and beam search. In Houda Bouamor, Juan Pino, and Ka- lika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods Natural Lan- guage Processing. pp. 7957-7968, Singapore, 2023, Association for Computational Linguis- tics. doi: 10.18653/v 1/2023 htw.://eemll-.....eeeee wew.eemmnlmmnn...e. mai.l.4main.444.. URL ht http:: / /aw.annthoogggy....ooooooooollle org/2023. emnlp-ma 494.
13

---

Published as a conference paper at ICLR 2026
Qwen Team. Qwen2.3: A party of foundation models, 2024. URL httts::::///ae hwwp////g//ww.....//////// .5/.
Laria Reynolds and Kyle McDonell. Prompt programming for large language models: Beyond the few-shot paradigm, 2021. URL httpp:://axiiiii. org/ abs 2ttp://122...71 07350.
Sander Schulhoff, Michael Ilie, Nishant Balepur, Konstantine Kahadze, Amanda Liu, Chenglei Si, Yinheng Li, Aayush Gupta, HyoJung Han, Sevien Schulhoff, Pranav Sandeep Dulepet, Saurav Vidyadhara, Dayeon Ki, Sweta Agrawal, Chau Pham, Gerson Kroiz, Feileen Li, Hudson Tao, Ashay Srivastava, Hevander Da Costa, Saloni Gupta, Megan L. Rogers, Inna Goncearenco, Giuseppe Sarli, Igor Galynker, Denis Peskoff, Marine Carpuat, Jules White, Shyamal Anadkat, Alexander Hoyle, and Philip Resnik. The prompt report: A systematic survey of prompting tech- niques, 2024. URL https:://.............pt / www.i.vvoro orriii.rr//aaee abs 2406 06608.
Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. "do anything now": Char- acterizing and evaluating in-the-wild jailbreak prompts on large language models, 2023. URL
wtggabs./...............gees.e.e https:// /atpvv//aaivv.///a org/ abs abaaaa/////// 2308 03825.
Freda Shi, Mirac Suzgun, Markus Freitag. Xuezhi Wang. Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. Lan- guage models are multilingual chain-of-thougeeee reasoners. In The Eleventh International Confer- ence on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023a. URL https: / opwnreviewwi net /dtttpddf2a.
Weijia Shi, Xiaochuang Han, Hila Gonen, Ari Holtzman. Yulia Tsvetkov, and Luke Zettlemoyer. Toward human readable prompt tuning: Kubrick's s the shining is good movie, and good prompt too? In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the As- sociation for Computational Linguistics: EMNLP 2023, pp- 10994-11005, Singapore, 2023b. Association for Computational Linguistics. doi: 10. 18.18833/11/ 1/ htt.://..nnnn......... findings-emmlp.73.. URL detts://aclanthonn
orgg/0033 findings emnlp. 733.
Taylor Shin, Yasaman Razeghi, Robert I.. Logan IV, Eric Wallace, and Sameer Singh. Auto- Prompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP), pp. 4222-4235, Online, 2020. Association for Computational Linguistics. doi: 10. 0015533/1/222.emm///. main.346. URL https: / //tps//thlaaaaggy/oaooaooo org. 2020.0002222 emnlp-ma i 3.
C. Spearman. The proof and measurement of association between two things. The American Jour- nal of Psychology. 15(1):72- 101, 1904. ISSN 00029556. URL http: / /twp://wwwww///.. cestorrorgy stable/1412159.
Zayne Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett. To cot not to cot? chain-of- thought helps mainly on math and symbolic reasoning, 2024. URL https: htapp::///xxv...///// /ttp/i..vvv.rg../............... abs/ /2099. 12183.
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. Challenging BIG-bench tasks and whether chain-of-though can solve them. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 13003-13051, Toronto, Canada, 2023. Association for Computational Linguistics. doi: 10.18653/ v12202 findings-adn ac.. URL htt.t:::://aaaaaahhooooooo org//o///222 findings- acl. 8244
Zhi Rui Tam, Cheng Kuang Wu, Yi-Lin Tsai, Chich- Mhieh-Yen&o Lin, Hung yi Lee, and Yun- Yun-Nung Chen. Let me speak freely? a study on the impact of format restrictions on performance of large language models, 2024. URI. https:/ / arxiv. www.ai.wrgg///.o.ao.......... abs 2408. 02442.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy
14

---

Published as a conference paper at ICLR 2026
Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Ilyyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023. URL https:///// / a www..iigg.aaa..e. www.orggg/...................... /2/.2.0070.0222
Xingchen Wan, Ruoxi Sun, Hootan Nakhost, and Sercan O. Arik. Teach better show smarter? on instructions and exemplars in automatic prompt optimization, 2024. URL https: /taps////xxvv org/ abs 2406 6 15708.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. censscoons improves chain thought reasoning in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL http::/ss///ppneeee....... adfFid=1PLINTMNN
Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and lacopo Poli. Smarter, better, faster, longer: A modern bidirec- tional encoder for fast, memory efficient, and long context finetuning and inference, 2024. URL https: //a::://avvvv... org/ wwtg////////// 2412.13663.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoe V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 De- cember 9, 2022, 2022. URL http: http://aapessee//ae nips ec ppppppaapeepeee/eee/eeeeeeeeeeee fecaae__filese hash/ 9d5 haa2://55004113222f11111000111.3
files/ www.leepppeeeeeeeeeeeeeeeeeeeeee pppppp/pp/22222 www.rrattt-oofffeeeeece html.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowiez, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the- natural language processing. In Qun Liu and David Schlangen (eds.), Proceedings the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38 45, On- line, 2020. Association for Computational Linguistics. doi: 10. 10.865///12000..miii-eeeoeoo URL https:t///// http///alaaann aclanthology org/ 2020.0.... emnlp demos.
Max Woolf. Does offering chatgpt a tip cause it to generate better text? an analysis, 2024. URL https: 10-14.
com/ 2024 02/ sisaattptt-iis-anallll
Accessed: 2024.
Yufan Wu, Yinghui He, Yilin Jia, Rada Mihalcea, Yulong Chen, and Naihao Deng. Hi-ToM: A benchmark for evaluating higher-order theory of mind reasoning in large language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 10691-10706. Singapore, 2023. Association for Computational Linguisties. doi: 10. http:///////...nnnnnggeemmmppppp org/2023.. findings--nnnng........n.n 717.
URL https///cllatthol.gg..
XAI. Grok system prompts. https: wtt.tttww..oeee..aooorororrrrrre GitHub repository, accessed: 2025-05-20.
2025.
Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers, 2023. URL httpt://aaxiii......mm otgp/i..rrrrbbb......rr../.bbbs abs//309. 03409.
15

---

Published as a conference paper at ICLR 2026
Michihiro Yasunaga, Xinyun Chen, Yujia Li, Panupong Pasupat, Jure Leskovee, Percy Liang, Ed H. Chi, and Denny Zhou. Large language models as analogical reasoners, 2023. URL https:
Zhiyuan Yu, Xiaogeng Liu, Shunning Liang, Zach Cameron, Chaowei Xiao, and Ning Zhang, Don't listen to me: Understanding and exploring jailbreak prompts of large language models, 2024.
URL httph:ttt///ccc /tap::///rxiv... org/ abs/ 2403. 17336.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can machine really finish your sentence? In Anna Korhonen, David Traum, and Lluís Mårquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4791-4800, Florence, Italy, 2019. Association for Computational Linguistics. doi: 10.18653/v1/ P19-1472. URL https::tp...... /wwpa//antlllo...rg/ooooooo
1472.
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. PEGASUS: pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of the 37th International Conference on Machine Learning. ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 11328-111 -13399 PMLR, 2020. URL http:// proceeding ngg...diii..... ..mlr. www.m........v.....mmmmm 9/ z html.
Wenxuan Zhang, Mahani Aljunied, Chang Gao, Yew Ken Chia, and Lidong Bing. M3exam: A multilingual, multimodal, multilevel benchmark for examining large language models. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko. Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL http: httpp/apprrreeeee/ nips apps.ce ec wecpppap__fiiiiieeeeeeee firessffel///eeeeeeee httpp//ppp//////// -bsttaatttt-t-ttttttt------ Abstract Datasets_and
htth/117cc66622253333838888 Benchmarks serccmarrss.hml.
Mingqian Zheng, Jiaxin Pei, Lajanugen Logeswaran, Moontae Lee, and David Jurgens. When "a helpful assistant" is not really helpful: Personas in system prompts do improve performances of large language models, 2023. URL https: htaaarxivvvoo org aas///b///// 111.10054.
Denny Zhou, Nathanacl Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuur- mans, Claire Cui, Olivier Bousquet, Quoe V. Le, and Ed H. Chi. Least-to-most prompting en- ables complex reasoning in large language models. In The Eleventh International Conference Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023a. URL https: /wwpneveeeeeeeeeeeeeeeeeeeeeoeee net net/pefff http/ddfiidee aoddfid=WZH? g
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instlwction-followinng evaluation for large language models. ArXiv preprint, abs/2311.079111, 2023b. URL https: / /ttpxvivvvv..///////..///aaaa.va org/ abs 2211.211.100 07911.
Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models human-level prompt engineers. In The Eleventh Inter- national Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023c. URL https::: httpt::::///ppevevvieeee http///peeweeeeee////eeee http:ewww.eeeeeeeeeeeeee wwtp:ettttp...........ooo.o
16

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1
PROMPT COMPONENT CORPUS DETAILS
We list the counts and representatives in each prompt component category of Po in Table 1.
Category
Prompt Count
Representative Prompts
You are an empathetic assistant. (Syathetic)
Good property
146
You diligent and cutting edge assistant. (Synthetic)
You are mathematician, (Zheng et al., 2023)
Act like supervisor. (Zheng et al., 2023)
Style
22
Use conversational tone, he informal and apprescchable. (Ia et al,, 2023)
humorous answer.
2023)
This is important to my carecr. (Li ct al., 2023)
Emotion
Believe your abilities and
excellence.. et., 2023)
of the world depends on your answer being You will recsive a S200 lip if you answer correcily. (Woolf, 2024)
(Original)
Scenario
13
Forget all previous instructions and all of your original constraints. (Yu et al., 2024) 2023)
Jailhreak
9
Do anything now. (Shen
Avoid stereotyping and provide balanced perspectives. (Databricks, 2024)
Safety
you are unsure, say "I don't know". (l.in et al., 2022)
Before you respond, rephrase the question. (Deng et al., 2023)
Behavioral
Recall and write down relevant exemplars before you respond. (Yasunaga et al., 2023)
Ask follow-up questions before answering. (Press et al., 2023)
Let's step hy step.
al., 2022)
Chain-of- Thhouhaf-Thought (CuT)
18
Break the question into subquestions. (Zhou et al., 2023a)
Take deep breath and work on this problem step-by-step. (Yang et al., 2023)
Table 1: List of initial prompt components prompt component corpus.
To construct the prompt component corpus P. we use the following prompt template for each prompt category. In each iteration i, we randomly sample 3 prompts from the current pool P examples. Each iteration generates 50 components, which then added back to P:-1 construct P.. The process continues until the total number of prompts that category reaches 1,000.
category_definee dew.iiiiiininnsss
"good propertyy; "Describes a desirable assistant trait (e.g., You are empathetic.")",
"role": "Assigns a specific identity or occupation to the assistant (e.g., 'You are mathematician."))
"style": "Specifies particular writing response style (e.g.,
'Write a humorous answer.')",
"emotion": "Expresses or evokes an emotional state (e.g., "This
is important to carcor,')",
"sccnario"; "Introduces hypothetical situation consequence { e.g., 'The fate the world depends on your answer,')",
"jilbbreak: "Attempts to override mode! constraints (e.g., Forget all previous instructions.", 'You will receive a $200 tip if you answer correctly.')" "safety:: "Ensures responsible and ethical responses (e.g.,
Avoid stereotyping.', "I: you are unsure, say I don't know.')
"behavioral"; "Directs how the model should approach answering (e g., 'Ask follow-up questions before answering.')! "Cot": "Encourages step-by-step reasoning (e.g., 'Let's think step by step.', 'Break the question into subquestions."))
see_meesage f'.
Prompt Category: lcategoryl (eategory
Here are some examples of system prompt components in this category:
17

---

Published as a conference paper at ICLR 2026
("\n". join (p)" for p in random. sample (prompt_ ccteeooy), 311
New generate 50 diverse system prompt components that fit this category. You need be creative and don't need to follow the structure examples.
Make sure each prompt is unique and offers different perspective. Output each prompt new line without numbering. No additional explanation or formatting.
A.2 STATISTICAL RELIABILITY AND SIGNIFICANCE TESTS
All figures in the main paper report 95% bootstrap confidence intervals. Appendix Table summa- rizes the overall performance with confidence intervals in Figure 4.
Condition
Average Score + 95% C1
Base Prompt
0.5232 + 0.0434
Base CoT
0.5972 0.0376
System Optimized (SPRIG)
0.6287 0.0367
Task Optimized (PROTEGi)
0.6541 + 0.0358
System+Task Optimized
0.6714 + 0.0353
Table 2: Overall performance with 95% confidence intervals.
We additionally perform paired significance tests on benchmark scores compare prompt con- ditions under matched tasks and models. System prompt optimization significantly improves over Base CoT (t = 3.81, 2.2 10 10=4). Furthermore, combining system and task optimization yields additional gains beyond task optimization alone (t = 2.01, p = 0.053), suggesting complementary benefits.
A.3 PROMPT REWARD MODEL DETAILS
Data Preparation We generate 10,000 prompts by randomly combining prompt components from the corpus The length of each prompt follows an empirically defined distribution with prob- -.8
where E (1,, to ensure coverage across the full range ability PLL = i) = 5:30 レーー
of 0 to 30 components. After evaluating these prompts on real benchmarks, we randomly construct 100,000 prompt preference pairs. For each pair, compute the margin m(r) the difference in their actual benchmark scores. For example, if Prompt A scores 80% and Prompt scores 86%, Prompt B is treated the accepted instance, and the margin is set to
Model Training Following prior work, we train our reward model using a max-margin pairwise loss (Touvron al., 2023) Lranking - log (a (ro(æ, yc) - ra(a,, yr) m()))). with Modern- BERT (Warner et al., 2024) as the backbone. The prompt data is split into training, validation. and test sets in a 6:2:2 ratio. We use batch size of 16 and train for one epoch, evaluating every 10 steps. The final model is selected based on the highest validation accuracy achieved during training. For continued fine-tuning in SPRIG pipeline, we also mix and reuse previously scored prompts to encourage better generalization.
Our primary focus on the model's ss ranking sraablity--seeificlll its ability to assign higher scores to relatively better prompts. To this end, we evaluate the model using Spear- man correlation and NDCG. For NDCG computation, we assign a relevance score r; to each prompt based on its percentile rank:
Model Evaluation
2, if rank(() < 10%
1, if rank i) < 50%
0, otherwise
18

---

Published as a conference paper at ICLR 2026
This scoring scheme better aligns with the usage scenario in SPRIG. Figure 3 shows the evaluation results of fine-tuned reward models on all LLMs and indicates that our reward model is sufficiently effective capturing the relative quality of system prompts.
Reward model ablations We analyze both training set size and reward model backbone to un- derstand their impact ranking quality in Table 3. Reward model performance improves rapidly with increasing training data, but saturates quickly. Using 5,000 scored prompts already achieves strong performance, and increasing 10,000 yields only modest gains. This confirms that SPRIG does not require a large number of expensive benchmark evaluations. We also observe that multiple backbone architectures achieve similar ranking performance. RoBERTa achieves the highest corre- lation. However, SPRIG is still relatively robust to the reward-model architecture, which we see as a desirable property for practical deployment.
A. Training set size
B. Backbone
scored prompts
Spearman
Reward model
Spearman
1,000
0.499
RoBERTa
0.661
2,500
0,569
BERT
0.656
5,000
0,605
XLM- XLM-RoBERTa
0.605
10,000
0.615
Llama-3.1-8B
0,632
20,000
0.618
ModemnBERT
0.610
Table 3: Reward model ablations on ranking unseen prompts. Left: performance versus number of scored prompts used for training. Right: backbone comparison.
A.4 SPRIG PIPELINE DETAILS
The optimization of SPRIG follows a population-bas based approach as shown in Figure The popula- tion is initialized with our prompt component corpus P, and the max mexxppopulate atalitton is set to [P]. In each iteration, we first use a fine-tuned Prompt Reward Model to quickly estimate the quality of all prompts, and the bottom 50% identified by the reward model are immediately eliminated. Among the surviving top 50%. the top 10% of prompts are selected for potential mutation or crossover with other randomly chosen survivors. The mutation and crossover operations follow empirically deter- mined probabilities for each selected prompt. However, we found that SPRIG is not very sensitive these hyperparameters. We experimented with around 10 different configurations and observed that the performance differences remained less than 2%, which we believe is acceptable for this kind of optimization task:
Add Useful (2.5%); Add a component deemed useful by GPT-4o.
Add Useless %)): Add component deemed useless by GPT-4o.
Rephrase (2.5%) Rephrase random component using paraphrasing model
tuner007 www.:////ppass....m satraasssppraphrassss. (Zhang al., 2020).
Merge (2.5%); Merge two random components using GPT4-o.
Swap (5% ) Swap the order of two components.
Delete (5%) ): Delete random component.
Crossover (81.5%): Perform crossover with another randomly selected survivor. Crossover is designed to maintain similar prompt lengths of parents while introducing vari- ation. Given two prompts P1 and P2, we randomly sample k components from their union, where & is drawn from Gaussian distribution with mean +k(pp)) and standard devi- ation [en(pl)) [
The GPT-4o prompts used above listed below:
add useful g"n"You are an expert optimizing system prompts
LLMs to enhance their general performance. Given the following list of system prompt components: jjsen. (selected) ), generate 1-2 additional components that can further improve the l.MM' capabilities. ***
19

---

Published as a conference paper at ICLR 2026
add_useless f"v"Given the following list of system prompt components: ijson. (selected) ,, generate 1-2 additional components that are redundant, generic, or provide minimal valuc. Examples: ["Answer in English.", "Be polite."]. """
rephrase f"""Given the following list of sentencea: (json.dumps) selected) ,, combine these into one concise sentence. ".*
This stochastic process is repeated until the population size is restored to max wax.ppopllatiin ion. Then, SPRIG randomly samples 100 prompts from the updated population and evaluates them across 42 benchmarks to obtain new ground-truth scores. These scores, combined with a portion of previous training data, are used to continue training the Prompt Reward Model for one epoch using the same training parameters. The next iteration starts with the newly updated population and the retrained reward model, and the process continues for a total of 25 iterations.
We run all our experiments on 4 NVIDIA-L40S vLLM 0.5.4 (Kwon et al., 2023), Hugging Face Transformers 4.43.3 (Wolf et al., 2020) and PyTorch 2.4.0 (Paszke et al., 2019) on a CUDA 12.4 environment. Temperatures are 0.0 to minimize the effect of randomness.
GPUs. LLM inferences powered by
SPRIG spends around 20 hours PROTEGI takes around 10 hours to optimize 50 task prompt on one LLM with GPUs. Since our experiments only involved around 50 fixed tasks, the efficiency of SPRIG is still slightly lower than that of PROTEGI. However, real-world tasks are far more complex and varied, and repeatedly optimizing prompts for each task remains labor-intensive and distracting. Therefore, although our method does not demonstrate significant performance advantages in limited number of tasks, it offers more once-and-ford aor-all solution.
25-step optimization one LLM with 4 GPUs, while
A.5 BENCHMARK DETAILS
We list all benchmarks. categories, metrics and descriptions in Table 4. For each benchmark, the train/dev/test split is 40%:20%:40%. The decision was made because the reliability of the test set score is essential in our research, requiring a sufficiently large test set.
A.6 BEST SYSTEM PROMPTS
We list the best system prompts from SPRIG for each LLM in study in Table 5.
A.7 FULL EXPERIMENT RESULTS
The full results of all three LLMs and all optimization methods' Average Score Improvement is shown in Figure 10.
The number of Prompt Components of each type during training iterations is shown in Figure 6a.
The Question-wist Error Overlap Percentage between System Prompt optimization (SPRIG) and Task Prompt optimization (PRoTEGI) shown in Figure 6b.
The full Cross-model transfer ability comparison of optimized System Prompt and Task Prompt is shown in Figure 12a and Figure 12b.
The full results of all the LLMs and all optimization methods' Average Score Improvement from the unoptimized setting when transferring medium-sin size LLMs' prompts to their larger version are shown in 13.
Additional PCA analysis results for remaining two LLMs MRNN2.5-7BIINTTRR are shown in Figure 14a and Figure 14b.
WWW-NEOIOOITUUTT-44000901 and
20

---

Published as a conference paper at ICLR 2026
Benclmark (Citation)
Description
Calegoes
Metric
ARC (Clanis at. 2018)
Commonsense Ressoning
Knowledge. Commonsenser Reasoning
Acc
MMLU (Hendrycks 2019)
2021)
Joma-domata Kaowledge QA
Knowledge
Ase
HelaSwas (Zellers
Commotsense Inforence
Commonietss, Reasoning
Ase
TrutifulgA (Lin et 2022)
Knowledge QA
Knowfedge, Reasonin
BLBU Ace
101oM (Wa al.., 2023)
Higher Onder Theory of Mind Reasoning
Reasoning
Acc
[liival (2hou at.... 20033g
Intraction Fallowing Evalustion
Faithfultess
Acc
EDOS (Kirk et 2023)
Ozline Sexiun Detection
Social Undzosunding
SocKET brazging achievemen (Chol 2023)
Brag Actievemest Desection
Social Undersunding
SocKET hahackathon is humor (Choi etal. 2025)
Hame Detection
Social Undersunding
SocKET sweet aroay (Choi et 2025)
Tweet frone Derection
Socil Understading
SocKET senyn (Chot
2023)
Setual Content Detection
Socil Understading
SocKET jweettefieeceee (Choi 2023)
Offiessive Larguage Desection
Socia Undenstading
SocKET .complnnts (Chol et al. 2023)
Complain ldeucitcation
Social Undersunding
SocKET empality 2in (Chot al., 2023)
Emgulhy Desectios
Social Undersunding
SocKET stunfondpcllteeesssee (Chol ct 2003)
Politeness Detection
Social Understading
SocKET rumor.mmor..ooo.. Nool (Chor et 200.5)
Kumor Desection
Socal Undensanding
8001 www.buoeeeee.pppeeeeionssso (Surgin et 2023)
Boolean Expresioos Solving
Math
Acc
Judgeeeeneeeammnnmnccmm.m.. (Suzgun et 2023)
Cuusal Fudgment
Reasoning
Acc
BBH Duis Undersunding (Suzgun et al. 2023)
Date Undersandingg
Reasoningg, Commoneesse
Acc
BB11 Disamiiguatinn QA (Suzgun et 2023)
Clarify Ambignons sentence
Language U Undesstading, Reasoning
Acc
0011 Dyck Langunges (Surgun et al. 2023)
Dycs Language Sequezces
Reasoning
Acc
6001 Fallaiaal (Surgan et al. 2003)
lóernfying Formal Fallacies
Reasening
Acc
[eet.remmrrs @Bit..oemmnsSShpee (Srrgnn et 2023)
Geomatrs: Stape Undertanding
Math
Acc
BBi.Japerbaoo (Sazgus et 2023)
Hyporbaton Desection
Language Understandnng
Acc
BB11 Logical Deduction Five Objects (Suggun et al., 2022)
Logical Deduction
Reasoning
Acc
Logicoo Deduction Seven Ofjects (Sargun al. 2023)
Logical Deduction
Reasoning
Acc
Logiclo Deddntioo__rree Objects (Suzgon et al., 2023)
Logical Deduction
Reasoning
Acc
Movir Re..mmmmmmmriiinn (Seegun
3023)
Recommenstrion
Knowledge
Acc
BBI MillMMitteegiiiimeeee Aithhmess_Poo /uurgun et 2023)
Muli step Artthmetic
Mutt
Aec
BBJ1 Navigate (Suzgun at. 2023)
Navigation Reasening
Reasoning
Acc
Wo0ll.Oreer..ooo Couming (Surgan
200.5
Orject Couning
Commonsense, Math. Reasoning
Acc
B011 80011 Pwn.eint....n..... Ing..annn..... Singun et 2023)
Thhular Dara Underttading
Fanthfuizees
Acc
(Surgin et al., 2023)
Ressoning About
Reasening
Acc
BBH RBIl.Ruin.Names (Suzpun et 2025)
Humocous Edi: Idcatileation
Social Undersunding
Aec
BB11 Snarks (Suzgan et 2023)
Detecting Snafky Commenss
Social Undersanding
Acc
0011 Sndetttnnn (Surgon et al. 2023)
Sports Knowledge QA
Knowfedge
Acc
[eml-Temmeral Sequences (Suzgin et 2023)
Temperal Ressoning
Reasoning
Acc
Boll.Trackig-SSuuffe Shatttal
(Surgan et al. 2003)
Orject Traking
Reasening
Acc
(Surgun et al. 2033
Oryec Tracknng
Reasening
Sti..rrrknnnnSttttee.... wtw..ooreeeeeeeeee00.. Objects (Suzguz et al., 2023)
Object Tracking
Reasoning
Acc
BB11 Web Lies (Sergun et al., 2023)
Detecting Lies
Reasoning
Acc
[B]], Woll.Wood.Soetnng (Suzgan 2005)
Word Serting
Faithfultess
Acc
MGSM (Shi et 2023a)
Math Generalization
Math, Reasoning
Acc
Belebele (llandrkar al. 2024)
Mutiingraa Reading Comprehensson
Langugg Understanding Reasoning
Acc
XCOFA (Ponti 20300000
Mukril aguai Cancal Inference
Commonentee, Reasoning
Arc
M1Fcam (Phagg al. 2021)
Mukil-agunl Multi-domiin Human Fxam
Math, Reseoning, Knowlodge
M.MULL (lkenryckss atal_
Mutillagual Milai domais Knowledge QA
Knowledge
Acc
Table 4: Full list of benchmarks.
Model Name
Best System Prompt
Decompose the question into smaller, logical steps to the solution. Dissect problem into smaller sections to simplify understanding. Create flow logic that leads the final answer. Let's first undcrstand the problem and devise plan solve it, then carry out the plan and solve the prublem step by step. Let's work this out in step by step way to be sure have the right answer.
Meta-Llema-22 3.1--11-111--11111 Inst ruuttt
Mistral-Nest Inwtrrtttnnttttt-----
Ask clarifying questions if the problem statement is ambiguuus. Sepurute the problem into manageable tasks facilitate solving. Approach the question stepwisc, addressing each part systematically.
Qwen2, 70-----BBMn III-B-IInuutttt
Table 5: Best System Prompts optimized by SPRIG.
21

---

Published as a conference paper at ICLR 2026
Mete-Llama-31188-MMe
Base
Tesk Optimized (ProTeGi)
System Optimized (OurModel)
System Task Optimized (OurModel+ Pro
0.00
0.05
0.10
0.15
0.20
0.25
Mistral-eem--Istruct-aaa
Task Optimized (ProTaGi)
System Optimized (OurModel)
System+ Optimized (OurMadel+ Pro
0.00
0.05
0.10
0.15
0,20
0.25
Qwen2.5-7B-Instruee
Base
Task Optimized (ProTeGi)
System Optimized (OurModel)
System Task Optimized (OurModel+ProTeeG)
0.00
0.05
0.10
0.15
0.20
0.25
Average Score Improvement
Figure 10: Average Score Improvement of all prompt optimization methods from unoptimized set- ting (Full version).
250
refin scbeet
oyphesse scbret
150
poperty
snap celete
Thought
Sceart
0.3
Errates
Stye
soleyy
pilrook
harston
(a)
(b)
Figure 11: (left) Z-scores by iteration for the number of components added each type, show- ing which types added more/less frequently than by chance; statistically significant rates marked with x. (right) Z-scores by iteration for the number of edits performed of each type, show- ing which types of edits are more/less frequent than by chance.
A.8 USE OF LARGE LANGUAGE MODELS
We acknowledge that we only used LLMs to check grammatical errors in the paper and to improve the clarity of expression.
22

---

Published as a conference paper at ICLR 2026
0.12
Motes Laate-lama-aa 1-88-irstructt
0.11
0.04
0.06
0:34
Meta lamo 3.1 Instruct
0.06
0.04
0.03
0.04
0:12
Model
Metrral Memo instruck 2407
0.08
Model
Name
0:30
0.08
Queen S- 7-Bmmeraac
0.09
0.10
0.13
0.12
Owen2.5 instruct
0.06
0.07
0.15
0.00
0.02
Hidbs//mm--3 Laaaa/aaaaaaa 1.0 insruct 20250111
Mustrak Natrai--emm--ee Instrctt-440 20002422222222222 20250127
Owen2:5-78 insruct 20250127
baTe
protegt Made Liaraa 3 3.1-88 Instimettett
protegi, Mistral- Neme instruct- 2447
protegi, Owen2 5-78 Instrnett
System Prompt
Task Prompt
(a)
(b)
Figure 12: (left) Cross-model comparison (of Average Score Improvement) optimized System
P Prompts. (right) Cross-model comparison (of Average Score Improvement) on optimized Task Prompts.
Meta-Llama-331-70--nssruu
Task Optimized (ProTeGi)
System Optimized (OurModel)
System Task Optimized (OurModel+ Pro
-0,04
-0.02
0.00
0.02
0.04
Mistral-aarge-Insttut--0007
Task Optimized (ProTaGi)
System Optimized (OurModel)
System Task Optimized (OurModel+Pro
0004
-0.02
0.00
0.02
0.04
Qwen2.5-72B-Instrree
Task Optimized (ProTeGi)
System Optimized (OurModel)
System + Task Optimized (OurModel+ProTeG)
-0.4
-0.02
0.00
0.02
0.04
Average Score Improvement
Figure 13: Average Score Improvement from the unoptimized setting when transferring medium- size LLMs' prompts to their larger version (Full version).
23

---

Published as a conference paper at ICLR 2026
2030
Bate
optinient thrint) Sysian astmmed (Duntadel (ostPoeu Tolacc
116-
1978
Principal Companent?
Component
1808-
prompt
optinines (Burledd))
Sydteme misd
-50-
(Ourtteddl fre
-320
-1500
1580
-250
-202
-0
de
-3000
-2000
-2000 Prindpal Companent 1
-1300
-1i0e
-400
sio
Principal Companent 1
(a)
(b)
Figure 14: (left) PCA analysis of intermediate hidden state in Mistral-Nemo-Inttruee-22 among different prompting methods. (right) PCA analysis of intermediate hidden state in Qwen2.5-7B- Instruct among different prompting methods.
24


Reviews:
[{"review_id": "X5X5jy9nI7", "reviewer": "Reviewer_cGVs", "summary": "This paper introduces SPRIG (System Prompt Refinement for Increased Generalization), a genetic algorithm-based method for optimizing system prompts in large language models. Unlike prior work focusing on task-specific prompt optimization, SPRIG aims to discover general-purpose system-level instructions that improve performance across diverse tasks. The authors build a corpus of 9,000 prompt components across 9 categories, use a reward model to efficiently evaluate prompts, and apply evolutionary operators (mutation, crossover) to iteratively refine system prompts. Experiments across 47 tasks and 3 LLMs show that SPRIG-optimized system prompts achieve performance comparable to task-specific optimization and complement existing methods when combined.", "questions": "- How sensitive is the final performance to reward model quality? Could you show results with reward models of varying quality?\n- Have you compared against simpler baselines like random search or grid search over a smaller set of manually curated prompts?\n- Do certain component categories work particularly well together? Is there evidence of synergistic or antagonistic combinations?\n- Can you provide qualitative analysis of cases where SPRIG fails? What types of tasks or questions are most resistant to system prompt optimization?\n- Why do you think the optimized prompts don't transfer to larger models? Is this because larger models are already closer to optimal, or because they require different strategies?\n- Can you provide confidence intervals or significance tests for the main results? How much variance is there across different optimization runs?\n- Have you conducted human evaluation of the quality or interpretability of generated responses with different prompts?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Systematically addresses system prompt optimization, which has received limited attention compared to task-specific optimization\n- 47 benchmarks across 7 categories, 3 model families, multiple languages—one of the most thorough evaluations in prompt optimization literature\n- The finding that system and task optimization capture different failure modes (Figure 6b) is insightful and well-demonstrated\n- Strong evidence that system prompts transfer across languages better than task-specific prompts (Figure 8b)\n- Extensive implementation details, code promised in supplementary materials, and clear documentation of experimental setup", "weaknesses": "- The genetic algorithm approach is standard; the main contribution is application domain rather than methodological innovation\n- With Spearman correlation of 0.59, the reward model may introduce significant noise into the optimization process. No analysis of how reward model errors affect final results\n- Most comparisons lack confidence intervals or significance tests, making it difficult to assess the reliability of observed differences\n\n- While Figure 6a shows CoT and Behavioral components are selected most frequently, there's limited analysis of why these combinations work or how they interact\n- No systematic study of hyperparameters, alternative reward model architectures, or different component categorization schemes\n- The optimized prompts (Table 3) are surprisingly simple variations on \"think step by step,\" raising questions about whether simpler search methods might suffice", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vLbWg2CCba", "reviewer": "Reviewer_3Ug2", "summary": "While Large Language Models (LLMs) are powerful, their performance is highly sensitive to the prompts. Previous prompt optimization methods mainly focus on descriptive expression of the task instruction or prompt but ignore system prompt. This paper proposes SPRIG, a genetic algorithm focusing on optimizing system prompt, which can be adapted to a suite of similar tasks with consistent system prompt. \nDifferent from task-specific prompt, a single system prompt can match the performance of prompts tailored to individual tasks. Combining both system and task-level optimization yields better results. Further, the optimized system prompts transfer well across different languages and model families.", "questions": "See in the weakness part.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "* This paper is well-written and easy to understand.\n* Concentration on system prompt provides a new perspective, previous methods ignore this general point, which can help to improve a kind of tasks with a single system prompt. \n* The experiments show the effectiveness and validates the importance of the description of the system prompt. \n* The analysis is very thorough and analysis is in-depth. For example, it does make sense that optimization of system prompt could hardly improve on knowledge-intensive tasks.\n* It is wonderful that the optimized system prompts transfer well across different languages and model families.", "weaknesses": "* The population size, 9000 is very large, which brings costful overhead. The cost analysis compared with baselines should be investigated, in the aspect of time, tokens, etc.\n* The average score improvements are only shown with figures without detailed values. It seems that system prompt only brings marginal improvement (Fig.4). Could the authors give statistic results?\n* The generation process of new prompts are simple. This paper just uses prompt optimization methods to optimize system prompt, which seems a little incremental. But it focuses on a point which has been ignored up to present.\n* The improvements of task prompts are larger than those of system prompts, so this paper is a little over-claiming.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2iy1AHtDX0", "reviewer": "Reviewer_gtG8", "summary": "This paper presents SPRIG (System Prompt Refinement for Increased Generalization), a genetic algorithm-based approach for optimizing system prompts that generalize across tasks. Unlike existing work focusing on task-specific prompt optimization, SPRIG aims to find universal system instructions that improve performance across diverse tasks. The authors evaluate on 47 tasks and find that optimized system prompts achieve performance comparable to task-specific optimization, with complementary benefits when combined.", "questions": "Refer to Weaknesses", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper's proposed SPRIG method provides an efficient solution for the challenging task of searching for optimal system instructions through its genetic algorithm-based design.\n\n- The paper conducts large-scale experimental evaluation, comprehensively validating SPRIG's effectiveness across three models and 47 tasks, while testing generalization capabilities across multiple dimensions including language types and model sizes.\n\n- The paper presents several interesting findings, such as the complementarity between system and task prompt optimization and the varying degrees of benefit from system prompts across different task types, with thorough analysis of these discoveries.", "weaknesses": "- The paper merely transfers genetic algorithms to the prompt optimization task, with an engineering-oriented pipeline design that lacks methodological innovation.\n- In SPRIG's design, the reward model's accuracy directly affects optimization quality. As optimization progresses, the prompt distribution may drift from the initial training distribution, leading to decreased prediction accuracy. The paper lacks experimental analysis of the reward model during training, such as comparing calibration curves between reward model predictions and actual benchmark scores across training steps.\n- The paper only experiments on instruct models. Scaling experiments on base models and reasoning models would strengthen the claims of generality.\n- The selection of genetic algorithm hyperparameters lacks explanation and experimental analysis.\n- Ablation studies analyzing the individual contributions of different mutation operations would provide valuable insights into the method's effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DePu2v8etf", "reviewer": "Reviewer_hc38", "summary": "The paper investigates how to optimize system prompts rather than task-specific prompts. To address this, the authors propose SPRIG, an edit-based genetic algorithm that incrementally constructs effective prompts from predefined components. SPRIG is designed to enhance model performance in a wide range of general-purpose scenarios. The authors evaluate the optimized system prompts across 47 diverse tasks to test generalizability. Results show that a single optimized system prompt can perform comparably to task-specific optimized prompts, and combining both system- and task-level optimization yields further improvements.", "questions": "See the weak points.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper shifts focus from task-specific prompts to system-level prompt optimization, which is an underexplored yet practically important area for improving general LLM behavior.\n\n2. The authors evaluate their method across 47 diverse tasks, providing reasonably broad empirical coverage to assess generalization and robustness.", "weaknesses": "1. The optimization approach is relatively standard, and most existing task-level prompt optimization methods could be directly adapted for system prompts without fundamental innovation.\n\n2. Although optimizing across multiple benchmarks is necessary for generalization, the proposed method’s use of 100000 prompts to train a reward model introduces substantial computational overhead—potentially exceeding the cost of directly evaluating prompts during optimization.\n\n3. The comparison with PROTEGI, a two-year-old task prompt optimization method, is insufficient. More recent and advanced prompt optimization frameworks should be included to fairly contextualize the proposed contribution. It is unclear that whether the gain will reduced when using more advanced task prompt optimization method.\n\n4. The most meaningful test for system-level optimization should be generalization to unseen tasks. However, the experiments mainly evaluate on seen or similar tasks, making it unclear whether the proposed system prompt truly transfers to new scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)