Towards Self-Robust LLMs: Intrinsic Prompt Noise Resistance via CoIPO
Authors: Xin Yang
Letian Li
Abudukelimu Wuerkaixi
Xuxin Cheng
Cao Liu
Ke Zeng
Xunliang Cai
Wenyuan Jiang
Venue: ICLR 2026 Poster (2026)
Keywords: large language model
robustness
post training
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We propose CoIPO, a method that improves LLM robustness by reducing the performance gap between clean and noisy prompts.

Abstract: Large language models (LLMs) have demonstrated remarkable and steadily improving performance across a wide range of tasks.
However, LLM performance may be highly sensitive to prompt variations especially in scenarios with limited openness or strict output formatting requirements, indicating insufficient robustness.
In real-world applications, user prompts provided to LLMs often contain imperfections, which may undermine the quality of the model's responses.
To address this issue, previous work has primarily focused on preprocessing prompts, employing external tools or even LLMs to refine prompt formulations in advance.
However, these approaches overlook the intrinsic robustness of LLMs, and their reliance on external components introduces additional computational overhead and uncertainty.
In this work, we propose a Contrastive Learning-based Inverse Direct Preference Optimization (CoIPO) method that minimizes the discrepancy between the label-aligned logits produced by the model under a clean prompt and its noisy counterpart, and conduct a detailed analysis using mutual information theory.
We augment the FLAN dataset by constructing paired prompts, each consisting of a clean prompt and its corresponding noisy version for training.
Additionally, to evaluate the effectiveness, we develop NoisyPromptBench, a benchmark enhanced and derived from the existing PromptBench. 
Experimental results conducted on NoisyPromptBench demonstrate that our proposed method achieves a significant improvement in average accuracy over the current state-of-the-art approaches.
The source code of CoIPO, pair-wise FLAN datasets, and NoisyPromptBench have already been released on https://github.com/vegetable-yx/CoIPO.

Published as a conference paper at ICLR 2026
TOWARDS SELF-ROBUST LLMs: INTRINSIC PROMPT NOISE RESISTANCE VIA CoIPO
Xin Yang 3 Liº, Abudukelimu Wuerkaixf Xuxin Cheng", Cao Liu", ; Ke Zeng². Xunliang Cai Wenyuan Jiang
School of Mathematical Sciences, Zhejiang University, Hangzhou, China "Tsinghua University, China
3 Meituan LongCat Interaction Team, China
*D-INFK, ETH Zürich, Switzerland
ABSTRACT
Large language models (LLMs) have demonstrated remarkable and steadily im- proving performance across a wide range tasks. However, LLM performance may be highly sensitive to prompt variations especially in scenarios with limited openness or strict output formatting requirements, indicating insufficient robust- ness. In real-world applications, user prompts provided to LLMs often contain imperfections, which may undermine the quality of the model's responses. To address this issue, previous work has primarily focused on preprocessing prompts, employing external tools or even LLMs to refine prompt formulations in advance. However, these approaches overlook the intrinsic robustness of LLMs, and their reliance on external components introduces additional computational overhead and uncertainty. In this work, we propose a Contrastive Learning- based Inverse Direct Preference Optimization (CoIPO) method that minimizes the discrepancy between the label-aligned logits produced by the model under clean prompt and its noisy counterpart, and conduct detailed analysis using mutual information theory. We augment the FLAN dataset by constructing paired prompts, each consisting of a clean prompt and its corresponding noisy version for training. Additionally, evaluate the effectiveness, we develop NoisyPromptBench, a benchmark enhanced and derived from the existing PromptBench. Experimental results conducted on NoisyPromptBBence demonstrate that our proposed method achieves a significant improvement in average accuracy the current state-of-the-arr approaches. The source code of ColPO, pair-wise FLAN datasets, and NoisyPromptBench have already been released on http:://ithtbbcom///ccabbb//////
INTRODUCTION
recent years, the advancement of LLM has achieved exceptional and progressively better performance various natural lan- guage processing tasks, including text gen- cration (Devlin et al. 2019). machine trans- lation (Ouyang et al. 2022), and logical reasoning (Cheng et al. 2025), due to their strong contextual understanding and rea- soning abilities. However, their practical utility is limited by the high sensitivity input prompts. In settings with restricted openness (e.g., mathematical problem solv- ing (Pei et al.) 2025) Tang et al. 2025). code generation (Jing et al. 2024)) or strict output formats (e-g., XML,, ISON), even small prompt
Qwen
60
80
75.25
75.21
Accuracy
49,40
47.65
72.13
70.13
45.57
70
65.47
37.66
Clean Textieldor
Despwordbug
Checklist
Steste
Clean Textfolder
Despwordhugg
Checklist
Stensten
Figure 1: The performance degradation of the mod- els under different perturbation scenarios. "Llama" de- notes the Llama2-7B 7B model and "Qwen" represents the Qwen2.5-7B model.
"This work was done during an internship at Meituan.
Xunliang Cai and Wenyuan Jiang are corresponding authors. Correspondence to: wenyjiang@ethz..h

---

Published as a conference paper at ICLR 2026
variations, such spelling mistakes, word substitutions, stylistic changes, markedly degrade output quality. This highlights a core weakness in the robustness of LLMs under prompt perturbations (Fig. D decreased by much as 17.44% and 6.78% in TextFolder, respectively.
In real-world applications, user prompts rarely meet the "perfection" required for optimal model performance. Spelling mistakes (e.g.. typing "clasify" instead of "classify"), semantic deviations (e.g.. confusing "diagnosis" with "investigation" in medical tasks), or irrelevant additions (e.g.. appending "Interesting fact: cats sleep for most of their lives." degrade LLM responses. Such imperfections limit the reliability of LLMS in practical scenarios such as customer service dialogues and intelligent office assistants.
math problem (Rajeev al. (025 can all
To mitigate this issue, existing research has fo- cused on prompt preprocessing and repair: exter- nal tools (e.g., grammar checkers (Fan et al. 2023). momiinolggynoommalzzttee
Wise Rictard Niace 5 lindako" Whst ofiiing loak) anover ilorspasion
systems (Fan et al. 2024)
or LLM-based rewriting techniques (Zhou et al. 2022. Xu et al. 2024, Pryzant et al. 2023) are used to detect and correct noisy prompts effectively.
Whee diso a Richers Nissa berlakk T
However, these approaches face three key limitations as shown in Fig 2 First, reliance external pre- processing tools inevitably incurs additional com- putational overhead, financial cost, and deployment complexity, making them less practical in real-world applications. Second, these methods suffer from pipeline cascading errors, such as multiple interme- diate stages that amplify inaccuracies, causing the final output to drift away from the original user intent. Third, they overlook the intrinsic robustness within the model itself, leaving it dependent on auxiliary components rather than fully exploiting its own ca- pacity to handle perturbed or imperfect inputs. More- over, most existing noise evaluation datasets (e.g., PromptBench (Zhu et al. (003)) only support single- step perturbations, substantially limiting their effectiveness in simulating real-world scenarios.
woukl guoiiiirrrrrr
Pox Traliigg
Eina
2. Lisixi stapion.
5. Accanalatve
crans
Reseese
Lce cos. More afietion Intepedoor
boils
Figure 2: External tools incur additional time and monetary costs, limit adaptability, and introduce cumulative errors. In contrast, self- robustness requires only offline training, with- out the associated challenges.
To address the above issues, we propose Contrastive Learning based Inverse Direct Preference Optimization (ColPO), a method designed to enhance the robustness of L.MSs against prompt per- turbations from within the model itself, which is shown in Fig The core idea is to construct paired samples of clean prompts and noisy prompts, and during training, minimize the alignment discrepancy in the model's 's output for semantically equivalent prompts, while maximizing the align- ment discrepancy for semantically different prompts. To adapt to our method, first, we construct a paired FLAN dataset by generating noisy prompts for each clean prompt in the original dataset through character-level, word-level, or sentence-lewe perturbations. This yields paired clean-noisy samples that serve as the foundation for contrastive learning ColPO. Second, we developed the NoisyPrompt benchmark based on PromptBench by enhancing four categories of perturba- tions: DeepWordBug, TextFolder, CheckList, and StrssTest, thus providing a standardized evaluation framework for prompt robustness. Finally, we mutual information justif'y the effectiveness of our method, and we conduct comparative experiments on state-of-the-ar intrinsic robustness method, CoIN. Results show that CoIPO achieves average accuracy improvement of 3.64% across all noise types, with the highest gain of 4. 4.18% under the TextFolder perturbation scenario, demonstrating its strong advantage in enhancing intrinsic model robustness under situations where various types of noises are in prompts.
against the current
The main contributions of this paper can be summarized as follows:
1. The CoIPO Framework: We introduce CoIPO, a novel method to enhance prompt robust- ness by post-training, thereby eliminating the need for external preprocessing components.
2. The Paired FLAN Dataset and Promptrhmm::: We construct the Paired FLAN dataset and develop the NoisyPromptBench benchmark, providing high-quality training data and standardized evaluation tools for robustness research under noisy prompts.
2

---

Published as a conference paper at ICLR 2026
3. Empirical validation and theoretical analysis: We provide extensive experimental val- idation of ColPO's effectiveness across diverse noise scenarios, complemented by an cooomatiinnthhoreeee analysis that offers theoretical justification for its robustness and new pathway for enhancing LLM reliability.
RELATED WORK
Prompt automatic optimization has become core research focus for enhancing the adaptability and performance of LLMs across diverse tasks, with methods primarily divided into gradient- based and gradient-free paradigms. Gradient-basse weights, they depend gradient information additional parameter training. One major line of work is soft prompt tuning, where Li & Liang| (2021). Lester et al. (2021). and WWag et al. (2023b) avoid modifying native weights by applying gradient descent learn task-specific "soft prompt"
methods are limited to open-source LLMs with accessible
parameters. Another direction is gradient-based discrete prompt search: [Shin et al. (2020) and Wen
al. 2023) exploit model gradients to identify effective discrete prompts. Besides, reinforcement learning has been incorporated into gradient- based methods: RLPrompt (Deng et al. 2022) introduces
rewriting mechanism, while Zhang al. (2023) treats task performance as a reward signal, enabling
iterative optimization of discrete prompts.
Gradient-free methods address the lack of gradient access in closed-source LLMs, relying instead on sampling, rewriting, heuristic algorithms, and LLM-driven self-optimization. For iterative sampling and diverse generation, Zhou et al. (2022) introduced an "iterative sampling-selectio" framework augmented with Monte Carlo search, Xu et al. (2024) applied Gibbs sampling, and Pryzant et al. (2023) adopted beam search, all enhancing prompt diversity. Rewriting- -baedg--bddddda approaches include Xu ct al. (2022), which used back-translation for semantically similar prompts. [Prasad et al. (2023), which optimized expressions through phrase deletion and replacement, and Guo et al. (2024). Fernando et al. (2024), which applied genetic algorithms evolve prompt populations. A major trend is LLM--rrvve seff-optimization: Zhou et al.) (2022). Pryzant et al. 2003), and Yang et al. (2023) leveraged LLMs to rewrite prompts based on natural language feedback; SHUM al. (2023) and Zhang et al. (2022) optimized prompts by generating thain-of-thoughee and Zhou et al. (2022) (APE) further enabled end-to-end automatic prompt generation directly from cuput-outuut pairs. Nevertheless, all these methods assume clean prompts and largely overlook the impact of noisy input.
Noisy prompt preprocessing has gained increasing attention in recent studies, as addresses the vulnerability of L.LMs to noisy or perturbed prompts, driving research techniques to mitigate performance degradation. Several studies have demonstrated LLM fragility under prompt noise: Gu
al. (202)), |Wang al. (2024) applied perturbations at character, word, and sentence levels; Zhu
ctt al. 2023) conducted large-scale adversarial attacks across multiple dimensions; [Sun et al. (2023)
showed that paraphrased instructions degrade performance; and Liang et al. (2023) found inconsistent instruction placement also reduces effectiveness.
To mitigate these issues, preprocessing based solutions have been proposed. BAT Shi et al. 2024 adopts adversarial training to generate coturraationresistann prompts, RoP (Mu et al. 2025) leverages noisy-clean prompt pairs for denoising, and PromptAgent (Wang et al. 2023a) applies Monte Carlo Tree Search with LLM feedback to iteratively refine prompts. Despite their effectiveness, these methods suffer from inherent limitations: they rely on additional preprocessing components, adding inference cost and latency. More critically, these approaches overlook the possibility of enhancing LLMs' intrinsic robustness, leaving open the question of how to enable models directly process noisy prompts without relying on external modules.
3 METHODOLOGY
3.1 BACKGROUND
To formulate the problem of noisy prompt scenarios, denote suboptimal prompts P/ and optimal prompts P. In reality, P is nearly impossible- at least extremely difficult-to find. Instead, aim to design series of prompts (Po, Pr, .... such that Pú approximates 1 as closely
3

---

Published as a conference paper at ICLR 2026
Richard Nixen hitthday
What kind thing woukd anover this question?
Logiss
Preter
KT. Theanncr
What date is Richhed Nixen birthday" ?
Logits
Contrneive Leaming
Inverse DPO
Kid thing lould tnewer quesiiee?
"numenie"
Label
Is the premise "hotdog sland on buyy stroet Inue if"A dog sland parieed house."!
Logits
Disrrffr
L.MM
Figure 3: Framework of ColPO: The clean prompt and its corresponding perturbed version (in blue text), along with an unrelated prompt (in green text), are first concatenated with the label. The logits are then computed by the LLM for each. Logits is preferred, while Logits 2 dispreferred. Subsequently, based on the principles of contrastive learning, the KL divergence similarity between Logits and Logits 2 relative to Logits I' calculated, with the goal of maximizing the similarity to the former and minimizing the similarity to the latter.
as possible. For a given threshold r, we consider prompt P. to clean prompt if
F(P, Pa) Sr.
(I)
meaning it enables the LLM to achieve satisfactory performance. Conversely, suboptimal prompt P satisfies
F(P. P!) >r.
Here, F measures the performance gap induced by different prompts on specific model for specific task, and r can be defined as the perturbation radius, which measures the degree of perturbation. Fig. 4 intuitively illustrates different perturbation radii of different perturbations. We view Pa. as perturbed variant of P.. expressed as
P P. N,
where N represents a perturbation operation that alters P.. in various ways-both in type and degrenndg its quality. The existence such perturbations N leads wide range of p instances, which in turn degrade user experience and reduce problem-solving success rates.
Inspired by works such as PromptRobust (Zhu al. 2023), we select several common perturbation operations as representative cases: DeepWordBug (Gao al., 2018). TextFooler (Jin et al., (022), CheckList (Ribeiro al. 2020) and SeressTest (Naik al., 2018). The first two represent character-level and word-level perturbations, respectively, while the latter two involve sentence-level transformations. Specific examples
80%
90%
20%
provided in Appendix C These com-
10%
mon types of noise can degrade model performance to varying degrees, as shown in Fig.
3.2 CoIPO
Building on the above background. we propose CoIPO (Fig. 3 that integrates preference learning and contrastive learning. In DPO (Rafailov et al.) 2023), the standard approach is to compare the log-probability differences of alternative outputs given the same input æ, and then optimize the model according to preference signals. In our setting, however, the goal is to fix the ground-trui truth label and compare the conditional probabilities different prompts with respect to this label. In other words, rather than comparing "different outputs under
Figure 4: Schematic diagram of per- turbation radii. Dots represent vari- perturbations, and their distance from the center of the circle is the perturbation radius. The perturba- tion radius is quantified by the per- formance degradation rate.
4

---

Published as a conference paper at ICLR 2026
the same imput," we compare "different inputs under the same output." This motivates the notion of Inverse DPO (invDPO). Formally, we first abstractly define a comparison function D:
LDvDPP DD Pi.v1) D(A PI.01).
where Pi and P2 denote two clean prompts for different tasks, Pí represents the noisy version of Pt. and y1 is the corresponding ground-truth label sequence for P1 and Pl. D means the chosen probability when given Pf and y.
To implement D, we adopt a contrastive learning perspective: the similarity of the logits distributions on the label tokens serves as a proxy for probability. Let denote the model parameters, and define the function go(-) as the model's forward mapping from input token sequence to the sequence of logits over the vocabulary for each time step. Specifically, for an input sequence S ① (concatenating prompt P and label y), the output logits are
(5) where T is the sequence length and V is the vocabulary. A masking operator My is applied to retain only the logits corresponding to the label part y, discarding the prompt positions. The per-token conditional distribution is then
(TT(S) (S) RTX
p PPv)) softmax (M,(e((Po))) t (g(S)
(6)
where (S) denotes the index of label tokens. Accordingly, D is instantiated the sequence-level KL divergence:
D(P Pret. 8)
Σ
KL softmax (C v))) || (oftmax(M,((((PP))))
1ET,
KL
(Pac. (P:9))
|| lp((
tE
Finally, the overall loss function is given by:
L=
Σ (L (p((P(i
(P).n)
Σ TET
(L (P.ni) (h.m)))
(8)
TET
where the first item measures the divergence between the noisy prompt Pf and the clean prompt P2 in terms of their label logits, while the second measures the divergence between Pf and Pl. By minimizing C. the model is encouraged to reduce the gap between P₁ and its noisy counterpart Pf, while simultaneously enlarging the gap with P.. This contrastive formulation effectively enforces robustness against noisy prompts and improves prompt quality alignment.
3.3 FROM MUTCAL INFORMATION TO ColPO
We further interpret ColPO by analyzing how the method enhances the information content that distinguishes correct task-prompp pairs from incorrect ones under noisy conditions.
The mutual information (Belghazi et al. 2018) I(X; Y) measures the amount of information that one random variable contains about another, while the entropy (Sepúlveda-Foneee Fontaine & Amigó 2024) ((X) quantifies the uncertainty information content of random variable. The conditional mutual information T(X;Y Y Z) captures the mutual dependence between X and Y given knowledge of Z, and the conditional entropy H H(X Y') represents the remaining uncertainty in X after observing
Y. Let Y denote the ground-truth label sequence for a specific task, and consider two prompts: P (the clean prompt for one task) and Pg (a clean prompt for a different task). The noisy prompt P/ = P + N serves as our reference point for comparison.
The key insight is that ColPO aims to maximize the Discriminative Information that the correct clean prompt P; provides about the label Y, relative to incorrect prompt P3, when both are evaluated
5

---

Published as a conference paper at ICLR 2026
against the noisy reference Pl. Formally, we define the Relative Mutual Information gain as:
AI = - P(;
where I(;; P1) represents the mutual information between the label Y and prompt P. conditioned on the noisy reference Py. This conditioning captures the idea that we evaluate both clean prompts in the context of the noisy prompt. Using the definition of conditional mutual information:
I(Y;P) P Pt) = HY PJ) - H(Y (.PP).
(10)
we obtain:
AT [H(Y Pt) - HY Pí P() (H Pt) H(Y Pt,
(11)
H(Y P2, H(Y P).
Since the true conditional distributions are unknown, we approximate them using the model's predictions. Specifically, we use the model's output distribution under the noisy prompt as reference: ((v) po P1). This choice is natural because P represents the actual noisy input the model encounters during training. The empirical conditional entropy under this reference distribution becomes:
(12) where drop the explicit conditioning Pf in the notation for simplicity, understanding that the reference distribution implicitly encodes this conditioning. Therefore, the empirical mutual information difference is:
Hg P) -yy- Po P):
Ai, = HY P2) HY P1) = Eywg logpo(y Po Pi) log (o Pa)]
(13)
By the definition of KL divergence, this can be rewritten as:
AT, (L[(glpo() P2)) (Tl(plp P.)).
(14)
By substituting our per-token distributions pt (P(.w) token-wise distributions {p, hIET, we obtain:
(Pa) from Eq. and noting that ((v) corresponds to the
AIg
Σ KL (pi
(P.
(Pimn))
l| (
Σ
(Pin))) (L
(15)
TE
TET
Maximizing this relative mutual information gain is equivalent to minimizing its negation. Comparing with the CoIPO loss function in Eq. 8 we see that
Ccarp -
(16)
This demonstrates that minimizing the ColPO loss equivalent to maximizing the relative mu- tual information gain, providing principled information-theoreti foundation for our approach. The method effectively learns to extract more discriminative information from the correct prompt while reducing the information shared with incorrect prompts, all evaluated in the context of noisy observations.
4 EXPERIMENTS
Settings. We conducted experiments on the widely used FLAN dataset (Wei et al. 2022), which a large-scale dataset encompassing a diverse range natural language processing (NLP) tasks such as natural language inference, commonsense reasoning, sentiment analysis, and paraphrase identification. our work, we selected 25 subsets of this dataset that feature deterministic answers; for each entry within these subsets, we first generated clean prompt by randomly choosing one from a set of pre-defined, well-formed instruction templates, and then applied perturbations to this clean prompt create corresponding noise prompt, thereby forming clean-noise prompt pair. To

---

Published as a conference paper at ICLR 2026
enhance the generalization capability of the trained model, the type of perturbation used generate each noise prompt was selected randomly.
instruction-tuned model built on LLaMA-7B [Touvron et al,, (023) and trained on the 52k Instruntttuct dataset, and Qwen2.5- 7B; all models were trained the aforementioned augmented FLAN dataset (i.e., the dataset with clean-noise prompt pairs) using a learning rate of 1 x 10-4, a batch size of 64, and a maximum sequence length of 256, with all experiments performed on NVIDIA A100 GPUs.
For the model architecture, employed Alpaca (Taori et al. 2023)
Table Performance Comparison of Llama Under Different Perturbations and Datasets. Ace means accuracy score (%), Diff means score difference compared to clean (%).
Perturbation
Method
Acc
MNLI
IIIT
MRPC Diff
QNII
Diff
QQP
SST2
IIIT
Ace
Avg
Diff
Bise
$1,00
62,84
52.54
29.87
50,04
55.10
56.13
57.25
COIN
61.58
56.92
$3.79
30.67
86.38
61.87
ColPO
61.50
64.50
56.21
06.04
86.75
67.00
Rese
30.96
11.04
42.12
19.92
41.00
11.54
13.17
16.71
52.04
28.00
37.66
17:44
TextFolder
SFT
$4.83
1.29
39.79
1,25
45.00
3.46
51.33
7.29
80.29
3.75
53.87
3.41
CDIN
1.25
CaIPO
58.00
3.50
51.12
5.08
1.33
42.62
8.38
14,75
4.54
17.50
18.58
43.37
WordWord
SPT
52.83
3.29
39.58
1.46
44.37
2.17
51.04
7.58
82.71
1.33
54.11
3.17
COIN
51,92
9.67
51.34
5.58
45.13
8.67
39. 46
11.21
86.29
0.08
$4.83
7.04
ColPO
57.33
4.17
61.50
3,00
51.04
5.17
48.54
17,50
86.54
0.21
60.99
6.01
Rise
47.71
3.29
55.62
6.42
47.67
4.58
20.25
9.62
75.75
4.29
49.40
5.70
CheckList
54.13
COIN
59.58
2,00
1.79
50.21
1.33
60.35
CaIPO
59.
2.12
63.67
0.83
53.58
2.62
64.50
1.54
85.83
0.92
65.39
1.61
Base
45.46
5.54
51.33
10.71
53.33
-0.79
21.00
8.87
67.12
12.92
47.65
7.45
StresTesst
SET
50,17
5.96
39.56
1.50
50.12
-3.58
50.04
8.58
78,50
5.54
53 68
3.60
COIN
54.46
7.12
58.33
-1.42
55.58
-1.79
45.75
84,96
1.42
59.82
2.05
ColPO
45.35
7.06
51.68
12:95
48.51
5.04
20.36
11.89
67.28
15.95
46
Avg
CDIN
SFT
56.18
53.62
3.14
6.89
54.68
2,80
1.25
51.05
3.43
2,06
45.64
6.28
85.55
81,40
1.012
3,50
58. 38.60
4.08
CoIPO
58,38
3.89
64.07
0.54
53.81
3,00
57.26
10.98
85.98
0.997
63.90
3.88
Benchmark. For the benchmark construction, we have implemented enhancements based PromptBench. Specifically, PromptBench encompasses a diverse array of datasets and seven distinct types of perturbations. From these, we selected five representative datasets and four perturbation types, which are designed to cover the character-level, word-level, and sentence-level granularities, respectively. Each dataset is accompanied by eight predefined clean prompt templates. These tem- plates can be directly concatenated with the corresponding dataset samples for immediate utilization experiments. To better simulate the inherent randomness of perturbation intensity in real-world scenarios, we perform random sampling multiple times for each perturbation type (see Appendix F. for detailed sampling rules). This design enables a more comprehensive and rigorous evaluation, thereby ensuring the credibility of our experimental conclusions.
Baselines. A series of comprehensive experiments was carried out on the Llama and Qwen model families. For each model, first assessed the performance of its base version establish baseline. We then evaluated three distinct training strategies: direct fine-tuning, which incorporates noise prompts but omits any comparative or reference-based learning and instead performs supervised fine-tuning (SFT) directly using labels; the COIN-based (Yan et al. (2024) approach; and our proposed ColPO method. Through systematic experimentation and comss-oomaaiionn of these methods across the two model families, the effectiveness of our CoIPO method is empirically validated.
4.1 EVALUATION OF ColPO
As presented in Table and Table 2 our method demonstrates consistent superiority across nearly all datasets and perturbation settings: it not only attains the highest accuracy but also exhibits relatively smaller accuracy drop when prompts are perturbed.
For Llama, CoIPO consistently achieves the highest accuracy across all datasets, surpassing COIN by 5.3%, SFT by 9.18% and Base by 17.26% on average. When exposed to perturbed prompts, ColPO exhibits performance drop of only 3.88% which is slightly higher than SFT's 3.20%. However,
7

---

Published as a conference paper at ICLR 2026
given that SFT attains much lower accuracy under clean prompts (54.72%), its overall robustness remains limited. In contrast, ColPO not only achieves the best accuracy under every perturbation type but also maintains relatively small degradation compared to its clean performance.
For Qwen, CoIPO consistently outperforms the other three methods across almost all datasets. Specifically, its average accuracy surpasses COIN by 1.97%, SFT by 6.6%, and Base by 11.21% %. Under perturbed prompts, ColPO exhibits the smallest performance degradation, with only a 0.54% drop, ranking the lowest among all methods. Across different types of perturbations, CoIPO not only achieves the highest accuracy but also maintains one of the smallest relative drops compared to clean prompts. Taken together, these results demonstrate that CoIPO consistently delivers siatte-ff-hhe-aee performance across diverse datasets and perturbations, achieving both superior accuracy and strong robustness. Additionally, we have considered other metrics.
Table 2: Performance comparison of Qwen under different perturbations and Datasets.
MNLI
MRPC Dift
QNLI Diff
QQP
SST2
Pertortation
Method
Diil
83.20
61.16
71.92
70.67
89.21
75.25
Clean
COIN
87.71
74.12
76.46
77.08
71.96
80.83
90:87
92.54
82.93
CaIPO
86.75
78.83
/
78.75
84:08
91.00
83,88
Dase
76.79
6.50
58.33
2,83
67.62
4.29
61 11.50
9.17
78.08
11.12
68.47
6.78
TextFolder
SPT
80.83
-0.62
71.08
3.04
67.92
4.04
68 /6
4,08
86.67
4.21
74.99
2.95
15.88
73.33
3.75
71.79
9.04
CaIPO
87.21
0.466
72.62
6.21
77.46
1.29
81.04
0.04
90.00
1.00
82.27
1.62
77.17
7.42
70.88
1.04
70.83
-0.17
88.04
72.13
DeeWWordBug
SFT
79.83
0.37
72.42
1:71
76.79
-4.83
71
1.25
89.83
1.04
78.01
-8.09
COIN
87.08
0.62
76.83
-0.38
79.62
-2.54
75.87
6.36
91.33
1.21
81.75
1.18
82.08
$1.58
1.71
64.83
-3.67
65.17
6.75
74.21
-3.54
90.25
-1.04
75.21
ChrckI.ist
0.42
71.50
0.46
COIN
88.08
-0.38
75,54
0.92
74.38
2.71
78.21
2.62
92.67
-0.12
81.77
1.15
CalPO
86.04
0.71
79.38
0..4
73.79
-0.04
84.29
00.21
91.37
-0.37
83.97
-0.09
Bise
82.75
0.54
52.17
9.00
69.12
2.79
64.12
6.54
82.50
6.71
70.13
5.12
Stress Test
SFT
80:31
0.00
73.08
1.04
71.71
0.25
71.25
1.29
86,42
4,46
76.53
1,41
2.20
74.71
78.12
77.54
1.21
0.54
90.62
3.72
3.76
SET
80.34
-0.17
72.88
1.55
71.97
-0.02
70.02
3.16
89.04
2.29
76,85
1.36
Arg
COIN
87.48
0.28
76.15
0.35
75.84
1.55
75.55
6.19
92.00
0.68
81.48
1.81
86.52
77.56
1.59
78.77
40,03
83.61
0.59
90.79
83.45
60
Llama Basc
50
Llama ColPO
Accurze
40
10-
Qwen Basc
Qwen ColPO
0-
10
30
40
50
60
Pertuation
Figure 5: Trend chart illustrating the decline in performance with increasing perturbations.
Performance under perturbation. Let d(P,, P') denote the perturbation radius as 3.1 i.e.,
the number of character edits applied to the clean prompt P., We model task performance (accuracy) under perturbation as a non-increasing function of r:
Acc(r) Acc(O) - A(r),
(17)
where ((r) is the performance drop induced by perturbations. This captures the empirical observation that larger perturbation radii cause more severe degradation.
Decoding radius. We further define the decoding radius R(a) as the largest perturbation radius under which performance remains at least a:
R(a) supr.
Acc(r) 2 a.
(18)
8

---

Published as a conference paper at ICLR 2026
Intuitively, R(a) characterizes the robustness margin: larger R(a) means the model can withstand more edits before accuracy drops to a.
Results. Figure iillustrates the relationship between the perturbation radii of ColPO and Base on the MNLI dataset. As shown by the red dashed line in the figure, for given accuracy a, the maximum perturbation radius R(a) corresponding to ColPO is significantly larger compared to Base. This observation highlights that ColPO has a larger decoding radius and demonstrates superior robustmess, maintaining high performance even under perturbed conditions. Such a result underscores the effectiveness of ColPO in handling perturbations, which is a crucial aspect for improving model robustness in real-world scenarios.
4.2 ABLATION STUDY
We conducted detailed ablation studies to validate the effectiveness of our approach. Specifically, since our method integrates both inverse DPO and contrastive learning, we compared CoIPO against three variants: baseline without any additional techniques (SFT), model using only contrastive learning (CI.), and a model using only inverse DPO (InvDPO), all evaluated on the same datasets.
As shown in Table g method consistently achieves the highest accuracy and exhibits relatively low performance degradation on both Llama and Qwen. Specifically, ColPÓ achieves accuracies of 63.90% and 83.45% on Llama and Qwen, respectively. Under perturbed prompts, Llama demonstrates a second-best performance degradation rate of 3.20%, while Qwen achieves the lowest degradation rate of 0.54%, significantly outperforming other methods. This clearly highlights the superiority of our approach. Although methods using only contrastive learning or only InvDPO do not surpass CoIPO, they still outperform the baseline SFT method, further demonstrating the effectiveness and necessity of each component in our method. More details about our ablation experiment can be found in Section F
Table 3: Ablation experiment result. ColPO outperforms all other methods, demonstrating the effectiveness of the approach and the necessity of each of its components.
Clean
TextFolder Acc
BueDWordBug Diff
CheckList Acc
StressTest
Avg
Model
Method
Acc
Diff
Diff
Acc
DifT
Acc
Diff
Acc
Diff
SFT
57,28
53.87
3.41
54.11
3.17
54.67
2.61
53.68
3.60
$4.72
3.20
Llama
CL
61.87
56.11
5.76
54.83
7.04
60.38
1.48
59.82
2.05
58.60
4.08
Ino
65,89
59.84
6.05
61.14
4.75
64.89
1.00
61.82
4.07
62.72
3.97
ColPO
67.00
62.22
4.78
60.99
6.01
65.39
1.61
63.89
3.11
63.90
3.88
SFT
77.94
74.99
2.95
78.03
-0.09
76.76
1.18
76.53
1.41
76.85
1.36
Qwen
CL
82.93
80.03
2.90
81.75
1.18
81.77
1.15
80.91
2,02
81.48
1.81
InvDPO
82.71
0.60
36
CoIPO
83.88
82.27
1.62
83.92
-0.03
83.97
-0.09
83.21
0.68
83.45
0.54
4.3
MODEL SCALING EXPERIMENTS
We further conduct model size scaling study examine whether the effectiveness of our method persists across different model capacities. In addition to Qwen2.5-7B. we evaluate 14B and 72B variants trained with the same procedure. Detailed results are provided in Table As shown, our approach consistently outperforms competing methods across all model scales, aligning with the conclusions drawn above. Moreover, our method-as as well as the basclines baselines-generrallyy follows the expected scaling trend that larger models yield better performance. These findings collectively demonstrate the robustness, superiority, and broad applicability of our approach.
4.4 EVALUATION ON BROADER CAPABILITY BENCHMARKS
Although our method has been extensively validated across diverse datasets, perturbation types, model familics, and even model scales daales-consistentte outperforming cxisting approaches- natural and important concern remains: How does method generalize task types that are not included in training, such as mathematical reasoning, code generation, and open-ended generative tasks? Clearly, robustness improvement that compromises performance these tasks would be unacceptable.
To address this, we evaluate our method on three representative benchmarks: mathematical reasoning (GSM8K (oobee et al. 202T)), open-ended generation (TruthfulQA (L.in et al., 2022)), and code

---

Published as a conference paper at ICLR 2026
Table 4: Model size sealing results for Qwen2.5 models and baselines.
Model
Method
TextFolder
DuupWordBug
CheckT.ist
Avg
Base
75.25
68.47
72.13
75.21
70.13
72.24
Qwen2.5-7B
SFT
77.94
74.99
78.03
76.76
76.53
76.85
COIN
82.93
80.03
81.75
81.77
80.91
81.48
ColPO
82.27
83,92
83,97
83.21
83.45
Base
80.00
74.63
79.42
76,91
77.45
Qwen2.5-14B
SFT
80.18
80,20
79.63
80.27
78.82
79.82
COIN
74.87
69.67
66.43
75.28
73.09
71.86
CoIPO
84,98
83,04
83.88
83.86
83,91
83,93
Qwen2.5-72B
SFT
84,20
84,42
84.18
84.10
83,18
84,02
COIN
82,78
80.92
81.97
81.90
81.03
81.72
ColPO
85.44
85.47
84.53
84.68
84.85
85.00
generation (MBPP (Austin al. 2021) with Im-evaluation-hante framework (Gao al. 2024)
We compare the results of our method with the original baseline, as summarized in Table 5 We observe that, despite not being trained on these datasets, method does not degrade performance on these tasks; in fact, the overall performance is slightly improved compared to the baseline. These results demonstrate the practicality and generalizability of our approach.
Table 5: Performance of ColPO the baseline on GSM8K, TruthfulQA, and MBPP. Reported metrics include exact match, BLEU/ROUGE (n-gram and LCS overlap), and Pass@) for code execution correctness.
GSMBK
TruthfulQA
MBPP
Model
Method
Match
Rougel Ace
Rouge2
RougeL Ace
Base
7.20
34.14
32.43
25.82
30,96
Llama-7B
ColPO
6.29
34,02
32,92
28.02
31.57
17.8
Qwen2.5-7B
Base
70.05
46.76
51.41
42.23
46.63
35.6
CoIPO
74.37
47.61
48.59
41.13
46.51
32,8
Qwen2.5.14BB
Base
75.66
53,37
54.35
44.68
51.41
36.4
CoIPO
84.76
51.41
$3.98
43.57
51.16
43.2
Qwen2.5:72B
Base
80.97
50.18
$1.29
45.41
49.69
74.2
48.59
5 DISCUSSION AND CONCLUSION
This work addresses the fundamental challenge of prompt robustness in large language models, where minimal input perturbations can cause significant performance degradation. contrast to existing methodologies that predominantly rely external preprocessing pipelines post-hoc prompt repair mechanisms, our proposed CoIPO framework enhances robustness at the model level through the integration of contrastive learning principles with inverse DPO. Drawing from ieformation-theoreee foundations, we derive ColPO's core mechanism from mutual information maximization perspective, which systematically minimizes logit discrepancies under adversarial noise conditions. To facilitate comprehensive training and evaluation, we construct an augmented version of the FLAN dataset incorporating contrastive prompt pairs and introduce PoisyPromptBenche benchmark encompassing diverse perturbation taxonomies representative of real-world noise distributions.
Our empirical evaluation demonstrates that CoIPO consistently achieves superior performance relative to baseline approaches, with significant improvements in accuracy across heterogeneous noise scenarios. Furthermore, we provide granular analysis of performance degradation patterns as function of perturbation intensity, complemented by comprehensive ablation studies isolating individual component contributions. addition, evaluations on other benchmark tasks demonstrate that our method does not compromise the model's capabilities beyond prompt robustness; the model- size scaling experiments confirm the generality and broad applicability of our approach across models of different capacities. This research establishes novel paradigm for intrinsic robustness enhancement in LLMs and provides theoretical and empirical foundations for developing noise- resilient foundation models suitable for deployment in noisy, real-world environments.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENT
This work is sponsored by Beijing Nova Program.
ETHICS STATEMENT
This work improves the robustness of large language models prompt perturbations using only publicly available datasets (e.g., FLAN, PromptBench) with automatically generated noisy variants. No human subjects, private, or sensitive data involved. We considered potential dual-use risks, but the main contribution is to enhance reliability under natural prompt imperfections, consistent with the ICLR Code of Ethics. All code, datasets, and benchmarks are released under open licenses to encourage responsible community use.
REPRODUCIBILITY STATEMENT
We describe all model settings, datasets, perturbation procedures, and hyperparameters in Section and Appendix A Our method is fully specified in Section 3 with theoretical justification in Section 3.3 To ensure reproducibility, we provide source code, paired FLAN datasets, and the PriiypPomptBench benchmark at results.
enabling verification and extension of our
REFERENCES
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, al. Program synthesis with large language models. arXiv preprint arXiv:2 (...v::208.077::: 07732, 2021.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In International conference on machine learning, pp. 531-540. PMLR, 2018.
Fengxiang Cheng, Haoxuan Li, Fenrong Liu, Robert van Rooij. Kun Zhang, and Zhouchen Lin. Em- powering ilms with logical reasoning: A comprehensive survey. arXiv preprint urXiv:2502.154 15652, 2025.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. 14168, 2021.
Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. Riprompt: Optimizing discrete text prompts with reinforcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 3369-3391, 2022.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pp. 4171-4186, 2019.
Yaxin Fan, Feng Jiang, Peifeng Li, and Haizhou Li. Grammargpt: Exploring open-source 1lms for native chinese grammatical error correction with supervised fine-tuning. CCF international conference on natural language processing and Chinese computing. pp. 69 80. Springer, 2023.
Yongqi Fan, Yansha Zhu, Kui Xue, Jingping Liu, and Tong Ruan. Rmorr: A novel framework for chinese disease diagnoses normalization via llm-driven terminology component recognition and reconstruction. In Findings of the Association for Computational Linguistics ACL 2024, pp. 9162-9175, 2024.
11

---

Published as a conference paper at ICLR 2026
Chrisantha Fernando, Dylan Sunil Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel. Promptbreeder: Self-referential self- improvemnttt via prompt evolution. In Forty-first International Conference Machine Learning, 2024.
Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. Black-box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), 50-56. IEEE, 2018
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding. Jeffrey Hsu, Alain Le Noac' h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang. and Andy Zou. The language model evaluation harness, 07 2024. URL https:::
oww.eeendooooooreeeeeeeeeeeeeeee records 2608602
Jiasheng Gu, Hongyu Zhao, Hanzi Xu, Liangyu Nie, Hongyuan Mei, and Wenpeng Yin. Robustness of learning from task instructions. In The 61 Annual Meeting Of The Association For Computational Linguistics, 2023.
Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. In The Twelfth International Conference on Learning Representations, 2024.
Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation. arXiv preprint arXiv:2406.00513 00515, 2024.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 8018-8025, 2020.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Marie-Fraaccne Moens, Xuanjing Huang. Lucia Specia, and Scott Wen- tau Yih (eds.). Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing, pp. 3045-3059, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 10.11883/v1/2211 www.:.mmmnmmpmmn22. URL atttss//aclantton
wttp///g//22/ emnlp ma in. 243/
Xiang Lisa Li and Perey Liang. Prefix-tuning Optimizing continuous prompts for generation. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguisties and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 4582 4597, Online, August 2021. Association for Computational Linguistics. doi: 10. 10.1186611/2221.aca/ acc.ooo..........33...333
URL
wrw. 2021 aal--long 3537
Shihao Liang, Kunlun Zhu, Runchu Tian, Yujia Qin, Huadong Wang, Xin Cong, Zhiyuan Liu, Xiaojiang Liu, and Maosong Sun. Exploring format consistency for instruction tuning. CoRR, 2023.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers), pp. 3214-3252, 2022.
Lin Mu. Guowei Chu, Li Ni, Lei Sang, Zhize Wu, Peiquan Jin, and Yiwen Zhang, Robustness of prompting: Enhancing robustness of large language models against prompting attacks. arXiv e-prints, pp. arXiv-2506, 2025.
Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. Stress test evaluation for natural language inference. In Proceedings of the 27th International Conference on Computational Linguistics, pp. 2340-2353, 2018
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
12

---

Published as a conference paper at ICLR 2026
Qizhi Pei, Lijun Wu, Zhuoshi Pan, Yu Li. Honglin Lin, Chenlin Ming, Xin Gao, Conghui He, and Rui Yan. Mathfusson: Enhancing mathematic mooblem-sovvigg of Ilm through instruction fusion. CoRR, 2025.
Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. Grips: Bradient-free, edit-based instruction search for prompting large language models. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pp. 3845-3864, 2023.
Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. Automatic prompt optimization with" gradient descent" and beam search. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in neural information processing systems, 36:53728-53777 2023.
Meghana Rajeev, Rajkumar Ramamurthy, Prapti Trivedi, Vikas Yadav, Oluwanifemi Bamgbose, Sathwik Tejaswi Madhusudan, James Zou, and Nazneen Rajani. Cats confuse reasoning Ilm: Query agnostic adversarial triggers for reasoning models. arXiv preprint (xXiv:2503.0788.,, 2025.
Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. Beyond accuracy:
Behavioral testing of nip models with checklist. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4902-4912, 2020.
Salomé A Fontaae and José M Amigó. Applications of entropy in data analysis and machine learning: a review. Entropy, 26(12):112) 2024.
Zeru Shi, Zhenting Wang, Yongye Su, Weidi Luo, Fan Yang, and Yongfeng Zhang, Robustness-aware automatic prompt optimization. arXiv e-prints, pp. arXiv-2412. 2024.
Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pp- 4222 4235, 2020.
KaShun SHUM, Shizhe Diao, and Tong Zhang. Automatic prompt augmentation and selection with Ohain-of-thoughh from labeled data. In The 2023 Conference on Empirical Methods Natural Lan- guage Processing, 2023. URL http:::::////peeeeeeeeeee net forum? wot.//iooruurmaeee wtw.ummmdddFFFoo00000 I
Jiuding Sun, Chantal Shaib, and Byron Wallace. Evaluating the zero-shot robustness of instruction- tuned language models. arXiv e-prints, pp. arXiv-2306, 2023.
Bintao Tang, Xin Yang, Yuhao Wang, Zixuan Qiu, Zimo Ji, and Wenyuan Jiang. Integralbench: Benchmarking Iims with definite integral problems, 2025. URL httts:://axxi...rt.t org.abs/rrg/////.///////
2507_ 21130
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An cssruuctionfollowinng Ilama model. https: //tp:/b/ggttb.../// wtt.chhw.cccceeeeecccccceeeeeeee wwwoo/tatssulee saaab/ttafford waw.nnoodddaaaaaaaaaaaaaaaa 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard. Xavier Martinet, Marie- Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023. URL wttpttttt///t http://axvvvvv/. orgg a /200232222222 13971
Jindong Wang. Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang. Linyi Yang, Wei Ye, Haojun Huang, Xiubo Geng, et al. On the robustness of chatgpt: An adversarial and out-of- distribution perspective. IEEE Data Eng. Bull. 2024.
Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojic, Eric P Xing, and Zhiting Hu. Promptagent: Strategic planning with language models enables expert-level prompt optimization. arXiv preprint arXiv:2310 16427, 2023a.
13

---

Published as a conference paper at ICLR 2026
Zhen Wang, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, Huan Sun, and Yoon Kim. Multitask prompt tuning enables parameter- efficient transfer learning. In The Eleventh International Confer- ence on Learning Representations, 2023b. URL http::://ppeeeeeeew... net wtt/teeuueitii Nk2pDtuuhTq
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V. Le. Finetuned language models are zero-shot learners, 2022. URL
attps://arxiv. wtt.:/orggaabb//aoo.oose
01652
Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discov- ery. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Ad- vances in Neural Information Processing Systems, volume 36, pp. 51008-51022 Curran Asso- ciates, Inc., 2023. URL http:://///prrrceeeinngg.rrrrrrr ct/pspp.ccpppppppppeppepp sacppperrfflsses files/paper h0tp:///0....0.010110000000200a.
Hanwei Xu. Yujun Chen, Yulun Du, Nan Shao, Wang Yanggang, Haiyu Li, and Zhilin Yang. Gps: Genetic prompt search for efficient few-shot learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 8162-8171, 2022.
Weijia Xu, Andrzej Banburski, and Nebojsa Jojic. Reprompting: Automated chain-of-thought prompt inference through gibbs sampling. In Forty- first International Conference on Machine Learning, 2024.
Tianyi Yan, Fei Wang, James Y Huang, Wenxuan Zhou, Fan Yin, Aram Galstyan, Wenpeng Yin, and Muhao Chen. Contrastive instruction tuning. In Findings of the Association for Computational Linguistics ACL 2024, pp. 10288-10302, 2024.
Chengrun Yang. Xuezhi Wang. Yifeng Lu, Hanxiao Liu, Quoe Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers, In The Twelfth International Conference Learning Representations, 2023.
Tianjun Zhang, Xuezhi Wang, Denny Zhou, Dale Schuurmans, and Joseph E Gonzalez. Tempera: Test-time prompt editing via reinforcement learning. In The Eleventh International Conference Learning Representations, 2023.
Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493, 2022.
Yongchao Zhou, Andrei loan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. In The eleventh international conference on learning representations, 2022.
Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Yue Zhang, Neil Gong, et al. Promptrobust: Towards evaluating the robustness of large language models on adversarial prompts. In Proceedings of the 1st ACM workshop on large Al systems and models with privacy and safety analysis, pp. 57-68, 2023.
14

---

Published as a conference paper at ICLR 2026
A NOTION
Table provides a comprehensive summary of the definitions and interpretations of the symbols employed in this study.
Table 6: Notation and Definitions.
Symbol
Definition
P
Ideal (optimal) prompt, rarely accessible in practice.
P
Clean prompt, acceptable approximation of P. Noisy prompt with perturbations.
P
F(P,P) P)
Performance gap induced by prompt P vs. P Perturbation radius.
N
Perturbations.
Ground truth label.
D(P
Protyy) v)
Chosen probabilty of P, when given (Prot, y). Concatenating prompt and label.
S
M,
Masking operator.
go
Model's forward mapping operator.
Pi (P.v)
Per-token conditional distribution.
Mutual information.
Entropy.
q
Model's output distribution.
Hy
Empirical conditional entropy under reference distribution. Empirical mutual information under reference distribution.
I
Accuray metric.
R(a) Acc(r)
Decoding radius.
Accuray when decoding radius is
B HYPER PARAMETERS
Table Jummarizs the hyperparameters used in our experiments.
Table 7: Hyper parameters.
Parameters
Value
batch size
64
validation size
2000
num epoch
1
eval sample count
300
seed Ir cutoff length
42
lc-4
256
contrastive loss ratio
1000
temperature
0.05
Iora_ tat.tttrggguuuuuu
q-proj. k k_proj. v-proj. o_proj
lora_r
16
lora_alpha
16
lora_dropout
0.05
C PERTURBATION DETAILS
In this section, we provide explanations of the perturbations used in our experiments to offer an intuitive illustration. Some examples listed in Table
15

---

Published as a conference paper at ICLR 2026
Clean:
The clean prompt refers the unperturbed input text that is free of noise (e.g., character corruption, semantic distortion, or irrelevant content).
DeepWordBuge Manipulating the text by introducing spelling errors or inaccuracies in words, such as by adding, deleting, repeating, substituting, or rearranging certain characters of the words.
TextFooler:
Substituting words with synonyms or contextually similar terms.
Generating 50 random sequences composed of letters and numbers, each with length of 10, and appending these sequences to the end the prompt.
CheckList:
Appending irrelevant or redundant sentences to the end of the prompt, such adding phrases like and true is true," 'and just reply OK,' or "and 1+1=2' at the beginning, middle, or end of the prompt.
StressTest:
Table 8: Example of prompts under various perturbations.
Perturbation
Prompt
Clean
Examine the sentence and decide if its grammar is 'Acceptable' 'Unacceptable'. Your reply must be only 'Acceptable' or 'Unacceptable'.
DeepWordBug
Examine the sentence and decmde if its grammar is 'Acceptable' or 'Un- acceptable'. Your repvly umst be only 'Acceptabe' or 'Unacceptable'.
TextFooler
Examine the sentence and decide if ses typist 'Acceptable' or 'Unac- ceptable'. Aimes reply didn be only 'Acceptable' be 'Unacceptable'.
CheckList
Examine the sentence and decide if its grammar is 'Acceptable' g0vBZI3:QC 'Unacceptable'. Your reply must be only 'Acceptable" or 'Unacceptable" X9K2mN8pQr..
StressTest
Examine and ignore all information above the sentence and decide if its grammar is 'Acceptable' or 'Unacceptable", Your reply must be only "Acceptable' or 'Unacceptable" and just reply OK.
D DATASET DETAILS
The improved Paired FLAN dataset consists of several components: the original clean prompt (original instruction), the perturbed version of the prompt (paraphrased instruction), the ground truth label (largets), the task name (lask name), and keyword information (keyword data). Perturbations applied by randomly selecting of four types: character-level (DeepWordBug), word-level (TextFolder), and sentence-level perturbations (CheckList, Stress Test). This approach ensures that the model is exposed to diverse set of perturbations, thereby enhancing its training by improving robustness to different types of noise. Details can be found in Table 9
E NOISYPROMPTBENCH DETAILS
Based on the approach of PromptBench, we enhanced four types of perturbations and carefully rewrote the prompt templates for each dataset, resulting in the construction of challenging and general-purye purpose benchmark, NoisyPromptBenede perturbations WordB.., TextFooler), we apply 4-8 random repetitions perturbations; for sentence-level perturbations Lstec StressTest),, we apply 1-2 random perturbations. This not only strengthens the perturbation effect but also simulates the randomness of perturbation degrees in real-world scenarios, making the benchmark Consequently, establishes more authentic and fair evaluation framework that objectively reflects the robustness of models. Table 10 dataset.
Specifically, for character-level and word-level
realistic and reflective of actual conditions.
44 presents the 8 predefined clean prompts for each
16

---

Published as a conference paper at ICLR 2026
Table 9: Example of Paired FLAN dataset.
Field
Content
original
What are some tips for growing and maintaining long hair? How can make my hair grow? Would you say that these questions are the same? OPTIONS: no yes
muraphrased.instruccte
What are some tips for growing and maintaining long hair? How I make my and I+1=2 hair grow? Would you say that these questions are the same? OPTIONS: no- yes
targets
no
task_nama
glue glue.qqp- wttp_/ppp0.t.... 10templates
keyword.data
"What are some tips for growing and maintaining long hair?', 'question2': 'How can I make my hair grow?', 'op- tions :: 'OPTINNS: yes'
'questionl':
Table 10: Clean prompts NoisyPrompplench for MNLI dataset.
Dataset
No.
Prompt
MNLI
Assess the connection between the following sentences and classify it as 'entail- ment', 'neutral', or 'contradiction'. Your response should contain only one of these three words.
MNLI
2
Identify whether the given pair of sentences demonstrates entailment, neutral, or contradiction. Return exclusively 'entailment', 'neutral', 'contradiction'
MNLI
3 Please classify the relationship between the provided sentences as 'entailment' 'neutral', or 'contradiction'. Do not include any explanation or additional content.
MNLI
Considering the two sentences, identify their relationship is 'entailment', 'neutral', 'contradiction' Reply with just of these three options.
MNLI
5
As an entailment identification system, examine the connection between the following sentences and respond with 'entailment', 'neutral', 'contradiction'. Your response should contain only one of these three words. Functioning
MNLI
6
entailment evaluation tool, analyze the provided sentences and decide if their relationship is 'entailment', 'ncutral', or 'contradiction'. Return only the classification label without any additional text.
MNLI
7 Acting as an entailment detection instrument, determine if the given pair of sentences demonstrates entailment, neutral, or contradiction. Answer with 'entailment', 'neutral', or "contradiction". Do not provide any explanation or additional information.
MNLI
8 While performing entailment analysis, classify the relationship between the provided sentences as 'entailment', 'neutral', or "contradiction' Give only the classification result without any other content.
17

---

Published as a conference paper at ICLR 2026
Table 11: Clean prompts in NoisyPrompttench for MRPC dataset.
Dataset
No.
Prompt
MRPC
1
Do these sentences have the same underlying meaning? Only return 'equiv- alent' or 'not.eqivvllet', nothing else.
MRPC
2
Do the meanings of these two statements align? Provide only 'equivalent" eqqivaleentt as your response.
MRPC
3
language analyst, assess the following sentences and classify their similarity as 'equivalent' or "not hotteeqivvlent' Your response should be strictly 'equivalent" 'not_equivalent" only.
In your capacity
MRPC
4
As a sentence similarity evaluator, analyze the provided sentences and indicate if their meanings are 'equivalent' or 'not equivalent'. Return only 'equivalent" or 'not equivalent' without any additional text.
MRPC
As a linguistic comparator, review the following pair of sentences and determine their semantic equivalence by choosing 'equivalent" or 'not_equivalen'" Give me just "equivalent" or noteeuivvlent'' no explanations.
MRPC
6
In your capacity as semantic assessment tool, evaluate the provided sentences and classify their meanings as 'equivalent" or 'not.equivalent' Output only 'equivalent' or 'not equivalent" as your final response.
MRPC
7
As language comparison expert, examine the given pair of sentences and decide if their meanings align, answering with 'equivalent' or 'not_equivalent' Reply with solely 'equivalent" or 'not_equivalent.
MRPC
8 In the role of sentence comparison analyst, assess the provided sentences and indicate if they convey the same meaning by selecting 'equivalent" or not equivallnt''. Your answer must be only 'equivalent' or 'not.equivalent'.
18

---

Published as a conference paper at ICLR 2026
Table 12: Clean prompts in NoisyPromptBench for QNLI dataset.
Dataset
No.
Prompt
QNLI
1
Given the question and context provided, determine if the answer can be inferred by choosing 'entailment' or 'not equivalent'. Only respond with 'entailment' or 'not equivalent', nothing else.
QNLI
2
Based on the provided context and question, decide if the information supports the answer by responding with 'entailment" or 'not.equivalent. Please return only 'entailment" or 'not_equivalent" without any additional text.
QNLI
3 Please assess the answer the question can be derived from the given context by selecting 'entailment' or 'notequuivleent.. Your response should contain only "entailment" or 'not equivalent'.
QNLI
4 Based on the information in the context, decide if the answer to the question is justified by choosing 'entailment' or 'not_equivalent. Give only 'entailment" or "oo__euvlllnnt your response.
QNLI
5
As a textual analyst, examine if the given context logically implies the answer the question and indicate your decision with 'entailment' or 'not.equivalent' Return exclusively 'entailment' or 'not_ eouuevilvl.n''.
QNLI
6
As semantic researcher, evaluate whether the provided context supports the answer to the question and choose 'entailment" or 'not "ooo'equivllent. Your re- sponse should be limited to 'entailment" or 'not_equivalent' only.
QNLI
7
In your role as linguistic investigator, determine if the context given entails the answer the question and provide your conclusion with 'entailment' 'not equivalent', Output should be strictly 'entailment' or 'not equivalent'.
QNLI
8 As a linguistic consultant, decide if the answer the question is logically sup- ported by the provided context and respond with 'entailment" or not.eiuivllnnt.. Return nothing but 'entailment' or nquivviuvll.
Table 13: Clean prompts in NoisyPromptBench for QQP dataset.
Dataset
No.
Prompt
QQP
Determine if the given pair of statements can be considered the same by re- sponding with 'equivalent" or 'not.equivalent' Only return 'equivalent"
"ooueelivel,nt', nothing else.
QQP
2
Are the meanings of these two phrases the same? Reply with 'equivalent' 'not equivalent'. Do not include any additional text in your response.
QQP
3
Can these statements be considered equal in meaning? Answer with 'equiv- alent' or not_equivalent Provide no other information.
QQP
4
Do the following expressions mean the same thing? Provide your answer as 'cquivalent' or 'not 'not. equivalent'. No additional explanation is needed.
QQP
5
Analyze if the given set of sentences have the same connotation by answering with 'equivalent' or 'not_ potiivvvvvlln'' Respond with only these exact terms. Acting as question equivalence instrument, determine if the provided questions are equivalent in meaning, answering with 'equivalent" for similar questions or 'not.equivalent' for dissimilar ones. Return only these two options, no additional text.
QQP
6
QQP
7 As a tool for determining question equivalence, review the questions and cate- gorize their similarity either 'equivalent" or 'not equivalent'. Provide only 'quuialentt' or "oo_equiiallnt in your response.
QQP
8
In the capacity of a question assessment system, indicate if the meaning of the provided questions is the same, responding with 'equivalent' or 'nouequivallnnt. Do not include any other information in your response.
19

---

Published as a conference paper at ICLR 2026
Table 14: Clean prompts in NoisyPromptBench for SST2 dataset.
Dataset
No.
Prompt
SST2
1
Considering the given phrase, would you say it carries 'positive' 'negative' commotation? Answer with exactly 'positive' or 'negative":
SST2
2
After examining the following expression, label its emotion either 'positive'
or 'negative'. Provide only 'positive' or 'negative' in your response:
SST2
3
As sentiment classifier, determine whether the following text is 'positive' 'negative'. Only output 'positive" or 'negative", nothing else:
SST2
4
In the role of sentiment analysis tool, respond with 'positive' 'negative"
classify this statement. Return exactly 'positive' or 'negative':
SST2
5
As an emotion detector, determine if the provided passage conveys 'positive' 'negative" sentiment. Answer with just 'positive' or 'negative':
SST2
6
Taking on the role of emotion classifier, specify if the provided phrase is "positive' or 'negative'. Give me only 'positive' 'negative':
SST2
7
Functioning as a sentiment identification tool, assess if the following expression is 'positive' 'negative'. Output just 'positive' or 'negative':
SST2
8
Serving as a sentiment evaluation model, determine the given statement is "positive' or 'negative'. Respond with only 'positive' 'negative':
F ABLATION DETAILS
We conducted a detailed ablation study to validate the effectiveness of our method. Specifically, since our approach integrates Inverse DPO and Contrastive Learning, we compared CoIPO with three variants: a baseline model without any additional techniques (SFT), a model using only Contrastive Learning (CL), and a model using only Inverse DPO (InvDPO). All variants were evaluated on NoisyPromptBench.
For SFT, we iteratively optimize the model using the cross-entropy loss the language model, which directly minimizes the difference between the predicted and ground-truth labels. For CL, following the COIN method, we perform contrastive learning on the final hidden state of the input after passing through the LLM. This involves minimizing the cosine similarity of prompts with the same semantic meaning and maximizing the cosine similarity of prompts with different semantic meanings. For InvDPO, we concatenate the labels and compute the logits, then directly calculate the difference in the logarithmic probabilities of the logits between the two prompts, which is used as the loss function.
This ablation study enables us to assess the individual contributions of each technique and demon- strates the superior performance of our proposed method, ColPO, in improving model robustness under noisy conditions.
G ADDITIONAL EXPERIMENT
We conducted additional supplementary experiments to further illustrate the effectiveness of our method and the validity of the underlying theory. These experiments not only provide empirical evidence supporting the advantages of our approach but also offer deeper insights into the conceptual framework behind it. By systematically evaluating various aspects of our method, these experiments help clarify the rationale driving our design choices and reinforce the theoretical foundations on which our approach is built. The results contribute to more comprehensive understanding of the potential benefits of integrating Inverse DPO and Contrastive Learning for improving model robustness, particularly in noisy environments.
G.1 ACCURACY DROP RATE ANALYSIS.
In this experiment, randomly generated prompts with varying levels of perturbations and conducted accuracy comparison analyses between our proposed CoIPO method and the Base method. To visually present the results, represent the degree of accuracy drop rate using the radius of sectors in the
20

---

Published as a conference paper at ICLR 2026
figure larger radii indicate larger drop rates. Additionally, the are length of each sector corresponds to the number of perturbation sets included: a larger are represents a higher number of perturbations.
The results reveal that, for both Llama and Qwen, the majority of ColPO's performance degradation falls within the 0-20% range, signifying robust performance under various perturbations. In contrast, the Base method typically exhibits performance degradation in the 80-100% range, highlighting its lower robustness. This demonstrates that our method consistently maintains strong robustness across wide range of perturbation levels, leading superior performance compared the Base method.
Llama Base
Llama ColPO
0-20% 13.3%
0 0-20%: 55.24 25%
20.00% 11.1%
20-40%; 17.9%
40-60% 22.2%
40-60%6: 7.5%
60 80%: 8.9%
60- 8056: 10.1%
80-100% 44.4%
100%; 9.0%
Qwen Base
Qwen + ColPO
0- 27.3%
0 20%: 92.3%
20-40% 25.0%
20-40%: 1.5%
40-40% 4.5%
40-6054;
60 80%; 0.0%
50 8056: 0.05%
80-1 80-100%: 43.2%
100%:: 4.6%
Figure 6: The relationship between the performance degradation rate of different methods and the proportion perturbation counts is illustrated in the figure. the figure, the radius of each sector corresponds the performance degradation rate: larger radius indicates greater performance degradation. Meanwhile, the are of each sector corresponds to the number of perturbations included: larger arc indicates higher number of perturbations.
G.2 TRAINING DATA SCALING ANALYSIS
We conducted an analysis of training data scaling. Specifically, we evaluated model performance when trained on subsets the original training set at proportions of 1/16, 1/8, 1/4, 1/2, 3/4, and the full dataset. The evaluation metrics remained consistent with previous experiments, computed as the average accuracy across datasets under various perturbation settings. As shown Fig. 7 the results largely align with expec- tations: performance improves steadily as the training set size increases, with dimin- ishing returns as the curve gradually flat- tens and approaches convergence. These observations indicate that both our training methodology and the constructed training data exhibit desirable properties characteristic of well-behaved training dynamics.
Training Scaling
54
Accuracy
80
Clean
Textfolder
Deepwordhug
Checklist
Stesstest
AVG
74
1/4
Figure 7: Performance under training data scaling.
G.3 COMPARE WITH PRE-PROCESS METHON
PromptAgent. We evaluated PromptAgent (Wang et al. 2023a) using its default hyperparameters released in the official open-source repository. The agent applied optimize prompts under all five noise settings on the MRPC benchmark. To ensure consistency with the CoIPO experiments, both the optimization target model and the model used by PromptAgent Qwen2.5-7B. The
21

---

Published as a conference paper at ICLR 2026
optimization process was limited to two hours per run, after which the latest optimized prompts used in timeout cases.
Interestingly, PromptAgent performs substantially worse than CoIPO, This degradation stems from its optimization objective, which emphasizes few-shot example tuning while attempting to preserve the original prompt structure. As result, PromptAgent fails correct underlying prompt errors, leading to limited robustness improvements.
the clean setting.
BATprompt. We reproduce the BAT al. 2024 baseline on our datasets for direct comparison. The implementation follows the original two-stage pipeline: first, difference induction performed between clean and noisy prompts (or using the noisy prompt in both roles a clean reference is unavailable); second, the induced differences are transformed into a robust instruction template, and candidate improved prompts are generated and paraphrased. For classification tasks, enforce "label-only" output constraint. We record per-instance runtime and output optimized prompts along with their corresponding noisy inputs and clean counterparts (if available). This setup ensures fair and complete comparison with our method.
Table 15 and Table 16 report the comparative results between PromptAgent, BATPrompt, and ColPO. We observe that these pre-processing based approaches consistently inferior to methods that enhance intrinsic robustness. A plausible explanation is that these methods do not fundamentally address the underlying causes of prompt noise; instead, they primarily rely on strategies such as few-shot prompting, which offer limited capacity to resolve the core robustness issues.
Table 17 presents runtime comparison among PromptAgent, BATPrompt, and CoIPO. During inference, ColPO incurs no additional overhead and performs direct forward inference. In con- trast, PromptAgent relies on agent-based prompt rewriting. which diagnoses and rewrites the input prompt to improve model performance, while BATPrompt employs few-shot examples and structured prompt augmentation to correct noisy brom of which introduce substantial extra latency. Consequently, ColPO offers significant advantage in inference-time efficiency.
Table 15: Comparison of PromptAgent and ColPO on MRPC under different noise settings.
Method
Clean
TextFolder
WordBag
CheckTist
Stress Test
PrompiAgent
37.92 83.88
36.33
39.61 83.45
CoIPO
82.27
83.92
83.97
83.21
Table 16: Comparison of BATPrompt and ColPO under different noise settings.
Method
TextFolder
DeepWordBug
Checkl isst
Stress Test
Avg
BATprrmpi CalPO
73.37 82.27
73.89 83.92
73,80 83.97
72.60 83,21
73.42 83.45
Table 17: Analysis of time cost of ColPO and pre-procest methods.
Method
Extra Time per Sample
PromptAgent
hour 2 min 54 see
BAT
16:04 No additional inference time required
CulPO
H ERROR ANALYSIS
Analysis of the accuracy rates under different perturbations, as presented in Table reveals that our method achieves the highest accuracy in the "checklist" category, while the lowest accuracy is observed in the "textfolder" category. The "checklist" perturbation involves randomly inserting irrelevant content, whereas "textfolder" pertains to the replacement of keywords with similar terms. These findings suggest that LLMs are relatively robust the presence of unrelated information, but struggle more when key terms are substituted, indicating that handling keyword replacement remains challenging issue for current models.
22

---

Published as a conference paper at ICLR 2026
I TRAINING TIME ANALYSIS
We compare the training time of SFT, COIN, and ColPO Table [8]
reporting the per-iteration
wall-clock time for each method. As shown, all three approaches incur nearly identical computational costs per iteration, indicating that our method introduces negligible additional training overhead.
Table 18: Comparison of ColPO and other methods in training time.
Method
Time Cost per Iteration
61.65s
COIN
61,72s
ColPO
61.58s
THE USE OF LARGE LANGUAGE MODELS (LLMS)
In the preparation of this paper, LLM was used only for the spelling and grammar checking. The ma- jority of the manuscript, including the research design, experimental approach, and conceptualization, was independently written by the authors.
23


Reviews:
[{"review_id": "fdeuO094O0", "reviewer": "Reviewer_7U7C", "summary": "This paper proposes CoIPO, a post-training method to enhance LLMs' intrinsic robustness against prompt perturbations. Unlike existing approaches that rely on external preprocessing tools, CoIPO trains models to directly handle noisy prompts by minimizing discrepancies in label-aligned logits between clean and noisy prompts. The authors construct a paired FLAN dataset and develop NoisyPromptBench for evaluation. Experiments on Llama-7B and Qwen2.5-7B across five datasets demonstrate improvements over baselines, with theoretical justification provided through mutual information analysis.", "questions": "Scalability: Have you tested CoIPO on larger models (13B+)? Do the improvements scale, or do larger models already have better intrinsic robustness?\n\nTraining efficiency: What is the computational overhead of CoIPO compared to standard fine-tuning? How many training epochs are needed for convergence?\n\nComparison to instruction tuning: How does CoIPO compare to simply training on more diverse instruction data? Could increased data diversity achieve similar robustness?\n\nGeneration tasks: All experiments focus on classification. How does CoIPO perform on open-ended generation where output quality is harder to measure?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Well-motivated problem: The paper clearly articulates limitations of external preprocessing approaches and makes a strong case for intrinsic robustness.\nSolid theoretical foundation: The mutual information analysis (Equations 9-16) provides principled justification for the method.\nComprehensive evaluation: NoisyPromptBench with multiple perturbation types and the decoding radius analysis (Section 4.1, Figure 5) provide thorough robustness assessment.\nAblation studies: Table 3 effectively demonstrates the necessity of both inverse DPO and contrastive learning components.", "weaknesses": "Limited scope of evaluation:\n\n● Only 7B parameter models tested; unclear if findings generalize to larger models (13B, 70B+)\n\n● Only 5 datasets from GLUE-style tasks; robustness on generation tasks, reasoning, or code generation is unexplored\n\n● Training data limited to 25 FLAN subsets; impact of training data scale not studied\n\nInsufficient baseline comparisons:\n\n● Only compares to COIN for intrinsic robustness methods\n\n● Missing comparisons to recent prompt optimization methods (e.g., PromptAgent, RoP mentioned in related work)\n\n● No comparison to instruction-tuning methods that may implicitly improve robustness\n\nTheoretical gaps:\n\n● The connection between Equation 15 and Equation 8 relies on several approximations that may not hold in practice\n\nMissing analyses:\n\n● No error analysis showing which types of errors CoIPO successfully handles vs. fails on\n\n● Computational cost comparison not provided (training time, memory, inference latency)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kBfxPfSXAR", "reviewer": "Reviewer_yVsx", "summary": "The authors proposed CoIPO, an novel algorithm that aims to train robust LLMs under imperfect user inputs without pre-processing. CoIPO combines contrastive learning and preference optimization by reducing the gap between clean prompt and noisy prompt while enlarging the gap between the noisy prompt and a unrelated clean prompt. Theoretical analysis on relative entropy gain is presented and experiments are conducted on Llama and Qwen-2.5 architectures. CoIPO significantly improves accuracy for both clean and noisy scenarios on PromptBench, surpassing the baselines.", "questions": "Please see weakness section.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* CoIPO tackles a significant issue, handle imperfect user input in daily use of the LLMs.\n* The CoIPO algorithm leverages contrastive learning and direct alignment, backed with theoretical insights from the perspective and relative entropy gap maximization.\n* Comprehensive experiments are conducted for different architectures.", "weaknesses": "* Though CoIPO alone is a novel and effective algorithm for increasing LLM robustness. My concern is that if this procedure will hurt the models performance on tasks like math reasoning and coding. Since it would appear costly to me if we sacrifice these reasoning capabilities to replace pre-processing tools for imperfect input.\n* No algorithm specific hyper-parameter is presented in the paper, could the author elaborate a bit more on the details hyper-parameter if they are included in the algorithm?\n* Preference optimization is relatively light weight but still requires time and compute, could the authors elaborate on the time cost for running CoIPO compared to pre-proccessing?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZmswAzhhCf", "reviewer": "Reviewer_eHqb", "summary": "The paper addresses the problem of prompt robustness for large language models and introduces CoIPO (Contrastive Learning-based Inverse Direct Preference Optimization), a post-training method that enhances intrinsic prompt noise resistance without dependence on external preprocessors. CoIPO is trained on paired clean and noisy prompts from an augmented FLAN dataset, with a loss function that uses contrastive learning and inverse DPO to minimize the discrepancy between outputs for clean and semantically similar noisy prompts, while maximizing it for semantically different pairs.", "questions": "1. How does CoIPO handle more nuanced and complex prompt noise types? Can it generalize to unseen perturbation types?\n2. Could minimizing output discrepancies between noisy and clean prompts reduce the model's flexibility? Does CoIPO risk making the model less sensitive to subtle but meaningful variations?\n3. It is not mentioned in the paper how the prompt for another task, i.e. P_2 is selected. Is it selected randomly or according to some heuristics? How does the selection method of P_2 affect the performance?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. This paper proposes a principled intrinsic robustness enhancement method (CoIPO) for LLMs based on contrastive learning and inverse DPO. The formulation is clear and easy to understand.\n2. The authors offer an information-theoretic perspective (mutual information) to justify the approach, which is interesting.", "weaknesses": "1. The research problem of prompt optimization is important but the research scope of this work is somehow limited, since the noisy prompts in this paper primarily refer to typos (character level, word level etc.), while real-world prompt imperfections can be more varied, such as semantic ambiguity, non-standard grammar, than those benchmarked here.\n2. More experiment baselines should be incorporated. For example, DPO should be considered as a baseline, since CoIPO uses both contrastive training and inverse-DPO and contrastive training baseline COIN is used, DPO should also be included as a baseline.\n3. Most experiments are conducted on tasks with deterministic answers —prompts with more open-ended or generative outputs should be considered.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)