Adversarially Pretrained Transformers May Be Universally Robust In-Context Learners
Authors: Soichiro Kumano
Hiroshi Kera
Toshihiko Yamasaki
Venue: ICLR 2026 Poster (2026)
Keywords: Adversarial Robustness
Transformer
In-Context Learning
Primary area: learning theory
TL;DR: Adversarially pretrained transformers are robust across tasks.

Abstract: Adversarial training is one of the most effective defenses against adversarial attacks, but it incurs a high computational cost. In this study, we present the first theoretical analysis suggesting that adversarially pretrained transformers can serve as universally robust foundation models—models that can adapt robustly to diverse downstream tasks with only lightweight tuning. Specifically, we demonstrate that single-layer linear transformers, after adversarial pretraining across a variety of classification tasks, can generalize robustly to unseen classification tasks through in-context learning from clean demonstrations (i.e., without requiring additional adversarial training or examples). This universal robustness stems from the model's ability to adaptively focus on robust features within given tasks. We also identify two open challenges for attaining robustness: the accuracy–robustness trade-off and sample-hungry training. This study initiates the discussion on the utility of universally robust foundation models. While their training is expensive, the investment would prove worthwhile as downstream tasks can obtain adversarial robustness for free. The code is available at https://github.com/s-kumano/universally-robust-in-context-learner.

Published as a conference paper at ICLR 2026
ADVERSARIALLY PRETRAINED TRANSFORMERS MAY BE UNIVERSALLY ROBUST IN-CONTEXT LEARNERS
Soichiro Kumano
Hiroshi Kera
The University of Tokyo
Chiba University, National Institute of Informatics Cera@chiba-te jp
ac. jp
Toshihiko Yamasaki
The University of Tokyo
yamasaki&cvm. t. u-tokyo.ac.
ABSTRACT
Adversarial training is one of the most effective defenses against adversarial at- tacks, but incurs a high computational cost. In this study, we present the first theoretical analysis suggesting that adversarially pretrained transformers can serve as universally robust foundation models-mmdehh that can adapt robustly to diverse downstream tasks with only lightweight tuning. Specifically, we demonstrate that single-layer linear transformers, after adversarial pretraining across a variety of classification tasks, can generalize robustly to unseen classification tasks through in-context learning from clean demonstrations (i.e., without requiring additional adversarial training examples). This universal robustness stems from the model's ability to adaptively focus on robust features within given tasks. We also identify two open challenges for attaining robustness: the accuracy-robustnes trade-off and sample-hungry training. This study initiates the discussion on the utility of uni- versally robust foundation models. While their training is expensive, the investment would prove worthwhile as downstream tasks can obtain adversarial robustness for free. The code is available at https:/ / /githb........ con/s s kumano/ / uuniiiiis sall lally-an haty-robuuttte -nnousttii--ie innooustti-oooneetttt-oo-oo ceentext-learnee learner.
INTRODUCTION
Adversarial suamples-subllc and often imperceptible perturbations to inputs that lead machine learning models to make incorrect predictions-tevell fundamental vulnerability in modern deep learning systems (Szegedy et al., 2014). Adversarial training is one of the most effective defenses against such attacks (Goodfellow al., 2015; Madry et al., 2018), where classification loss is minimized under worst-case (i.e., adversarial) perturbations. This min-max optimization significantly increases the computational cost compared standard training. Despite extensive efforts develop alternative defenses, most such approaches have subsequently been shown to offer only spurious robustness (Athalye al., 2018; Croce& Hein, 2020; Tramer et al., 2020). Consequently, adversarial training remains the de facto standard, and practitioners must incur this cost to obtain ad adversarially robust models.
Recently, it has become standard practice to leverage foundation models for target tasks. Thanks to large-scale pretraining, these models can be adapted to diverse downstream tasks through lightweight tuning. This raises a natural question: Can udversarially trained foundation models enable efficient and robust adaptation to a wide range of downstream tasks? Although training such models is expensive, the investment would be justified if numerous downstream tasks could inherit adversarial robustness for free, without requiring costly adversarial training themselves. While this a promising research direction, the utility of such universally robust foundation models remains largely unexplored, as the computational and financial costs make empirical evaluation across multiple runs impractical.
In this study, we present the first theoretical analysis suggesting that adversarially pretrained trans- formers can serve as universally robust foundation models. Specifically, we show that single-layer linear transformers, after adversarial pretraining across variety of classification tasks, generalize

---

Published as a conference paper at ICLR 2026
robustly to previously unseen classification tasks through in-context learning (Brown et al., 2020) from clean demonstrations. Namely, these transformers achieve robust adaptation without requiring additional adversarial examples or training. In-context learning is transformer capability that enables efficient adaptation to new tasks from a few input-output demonstrations in the prompt, without any parameter updates.
Our analysis builds upon the conceptual framework of robust features (class-discrimingg (emman-iteerpetabble) and non-robust features ihuman-imperceptible yet predictive) (Ilyas et al., 2019; Tsipras et al., 2019). Based on this framework, we show that adversarially pretrained single-layer linear transformers can adaptively focus on robust features within each downstream task, rather than non-robust or non con--ddiitiive features, thereby achieving universal robustness. This framework also reveals that universal robustness holds under mild conditions, except an unrealistic scenario where non-robust features overwhelmingly outnumber robust ones.
and
We also show that two open challenges in robust machine learning (Schmidt et al., 2018; Tsipras et al., 2019) persist in our setting. First, adversarially pretrained single-layer linear transformers exhibit lower clean accuracy than their standardly pretrained counterparts. Second, to achieve clean accuracy comparable to standard models, these transformers require more in-context demonstrations.
Our contributions are summarized as follows:
We provide the first theoretical evidence for universally robust foundation models: under mild conditions, adversarially pretrained transformers with a single linear self-attention layer can adapt robustly to unseen classification tasks through in-context learning.
Based on the framework of robust and non-robust features, we characterize the condition for successful robust adaptation. Moreover, we show that universal robustness arises from the models' adaptive focus on robust features within each downstream task.
We identify two open problems for these transformers: the eecuracy-robustnee trade-off and cample-hungry in-context learning.
This study explores the potential of universally robust foundation models, which can provide diverse downstream tasks with adversarial robustness without additional adversarial training. A key challenge is the cost of adversarial pretraining. We assume that, as with standard foundation models, such efforts would be undertaken by large organizations, which could offset development costs through licensing or API fees. The growing demand for safe and reliable AI strengthens this incentive. Encouragingly, advances in acceleration techniques for adversarial training, such as fast adversarial training (Wong al., 2020) and adversarial finetuning (Jeddi et al., 2020), suggest that the cost of adversarial training could approach the cost of standard training. We view our theoretical analysis as an important first step toward fostering the practical development of universally robust foundation models.
RELATED WORK
Additional related work can be found in Appendix A.
Adversarial Training. Adversarial training (Goodfellow et al., 2015; Madry et al., 2018), which augments training data with adversarial examples (Szegedy et al., 2014), is one of the most effective adversarial defenses. Its major limitation is the high computational cost. To address this, several methods have focused on efficient generation of adversarial examples (Andriushchenko & Flammar- ion, 2020; Kim et al., 2021; Park & Lee, 2021; Shafahi et al., 2019; Wong et al., 2020; Zhang et al., 2019a) and adversarial finetuning (Jeddi al., 2020; Mao et al., 2023; Suzuki al., 2023; Wang et al., 2024a). However, these methods require task-specific adversarial training. In this study, we introduce the concept of universally robust foundation models, which can adapt robustly to wide range of downstream tasks without requiring any adversarial training examples.
Robust and Non-Robust Features. It is widely hypothesized that adversarial vulnerability arises from models' reliance on non-robust features (Ilyas et al., 2019; Tsipras et al., 2019). While robust features are class-discimminative. -umaaninterprettalle and semantically meaningful, non- robust features are subtle, often imperceptible to humans, yet statistically correlated with labels and therefore predictive. Humans can rely only on robust features, whereas models can leverage both types of features to maximize accuracy. Tsipras al. (2019) showed that standard classifiers
2

---

Published as a conference paper at ICLR 2026
depend heavily non-robust features, making them vulnerable to adversarial perturbations that can manipulate these subtle features. They also showed that adversarial training encourages models to rely primarily on robust features, which enhances robustness but often reduces clean accuracy- phenomenon known as the securacy-robustnes trade-off (Dobriban et al., 2023; Mehrabi et al., 2021; Raghunathan al., 2019; 2020; Su et al., 2018; Tsipras et al., 2019; Yang al., 2020; Zhang et al., 2019b). Subsequent studies have confirmed that adversarially trained neural networks exhibit greater reliance on robust features (Augustin et al., 2020; Chalasani et al., 2020; Engstrom et al., 2019; Etmann et al., 2019; Kaur et al., 2019; Santurkar et al., 2019; Srinivas et al., 2023; Tsipras al., 2019; Zhang Zhu, 2019). In this study, we incorporate the concept of robust and non-robust features into the data assumptions for our theoretical analysis. Based on this framework, we find that adversarially pretrained single-layer linear transformers prioritize robust features over non-robust features, and exhibit the recuracy-robustnee trade-off.
THEORETICAL RESULTS
Notation. For n E N, let [a] == {1,.... n). Denote the i-th element a vector a by a, and the element in the i-th row and j-th column of a matrix A by Ai.- Let U(S) be the uniform distribution over a set S. The sign fuunction is denoted as sgn(.) .). For d1, d2 E N, let 1d, and 1d be the di-dimensional all-ones vector and di d2 all-ones matrix, respectively. The di x dj identity matrix is denoted as Id Similarly, we write the all-zeros vector and matrix as Od, and Odi,d2, respectively. We use Z, S. and only to hide constant factors in informal statements.
3.1 PROBLEM SETUP
Overview. We adversarially train a single-layer linear transformer on d E N distinct datasets. The e-th training data distribution is denoted by D for E [d]. The e-th dataset consists of N +1 samples, {(x:)) (c) (c))
) - L.d. ~ Drr The transformer is encouraged to adaptively learn data structures from N clean in-context demonstrations { yn) ) and to generalize to the (N + 1)-th perturbed sample N + where represents an adversarial perturbation. We then evaluate the adversarial
robustness of the trained transformer on test dataset { v/t)) N+ iidd Dic which may exhibit different structures from any training distributions.
Transformer. We first define the input sequence for a transformer
ZA
D2
EN
A UN 0
EN+1
IR (d++)))
V2
where 01...., ZN E IRd are training data, yi..... UN E {+1} are their binary labels, ØN+1 Rd is a test sample (query), and A E IRd is an adversarial perturbation (see below). The transformer is expected to learn data structures adaptively from N demonstrations {( yn > and to predict the label of 41. The (d + 1, N 1)-th element of Za serves as placeholder for the prediction of NN+1 + A. We define a single-layer linear transformer f (R+ ()) (N+++) + (X++)) which commonly employed in theoretical studies of in-context learning (Ahn al., 2023; Cheng et al., 2024; Gatmiry et al., 2024; Mahankali et al., 2024; Zhang et al., 2024b), as follows:
1 f(Za: = N PZAMZIQZA
M =
I, 0
0
E (R(++)X(N+D))
(2)
0
where P ((d+1)x(d+1) serves the value weight matrix and Q E R(d+1)x(d+1) serves as the product of the key and query weight matrices. Following prior work on in-context learning (Ahn al., 2023; Cheng et al., 2024; Gatmiry et al., 2024; Li et al., 2025), we adopt a mask matrix M to prevent the tokens from attending to the query token.
Training Data Distribution. The transformer pretrained on d distinct datasets. Inspired by Tsipras et al. (2019), we consider the following data structure that explicilly separates robust and non-robust features (cf. Section 2) according to their dimensional indices:
3

---

Published as a conference paper at ICLR 2026
Assumption 3.1 (Individual training data distribution). Let [d] be the index of a training data distribution and D be the c-th distribution. sample (æ, y) ~ Der satisfies the following:
((O. (U( 0))
(v ===1)
Te = v, y U({+1))
vi [d], c: 2i
(3)
(y -1)
where 0 1. For any + j. 24 and x, are independent, given y.
In this distribution, each sample has feature that is strongly correlated with its label (i.e., robust feature) in the e-th dimension and weakly correlated features (i.e., non-robust features) in the remaining dimensions. The correlation between the non-robust features and the label is bounded by A. The robust features mimic human- intmannnterpretablle semantically meaningful attributes natural objects (c.g., shape). The non-robust features mimic human-immereeee iiperceiimpeee yet predictive attributes (c.g., texture).
Test Data Distribution. The test data distribution may exhibit more diverse structures than the training data distributions, and may include non- non-predictive features in addition to robust and non- robust features.
Assumption 3.2 (Test data distribution). Let the index sets of robust, non-robust, and irrelevant features be Syab, Svul, Sir C [d], respectively. Suppose that these sets are disjoint, i.e., S Seul Sire Srob and that Srob Uvul U [d]. Let the number of robust, non-robust, and irrelevant features be drob := |Srobl dvul == |/ and dirr == S respectively. Let the scales of the robust, non-robust, and irrelevant features be > 0, 0, and y 2 0, respectively, Let Die be test data distribution. A sample (a. v) ~ D satisfies the following:
(1. Label) The label y follows the uniform distribution U{
(2. Expectation and Moments) For every i E Sin [[[x]] = O. For every i E [d] and n E {2, 3, 4}. there exist constants C 0 and Ci,, 2 0 such that
C,a
(i S,ob)
Cana"
(i E Srob)
Blya::
C
(i Svul).
[B[(;; E[yx:])"] S
Cin
(i E Sonl)
(4)
Sin)
Or,
(i E Stre)
(3. Covariance) There exist constants 0 S grob: Ivul < 1 such that
K i Slob Ejesmnus El(yx; E Elym;))(ur; B[yæ,)) } hobbroob (5)
{ E Ejesmuss.. B[(yx; - Elyx;))(uz; [[y=,1)] < H S Qvuidvul- (6)
(4. Independence) For every E Sm is independent of and all æÿ for j +
In contrast to the training distributions, the test distribution may contain d,ob robust features and due irrelevant features. These irrelevant features simulate natural noise or redundant dimensions commonly found in real-world data. For example, in MNIST (Deng, 2012), the top-left pixel is always zero and thus not predictive. Assumption 4 requires each irrelevant feature be independent of both the label and all other features. The robust and non-robust features are not necessarily mutually independent.
Assumption 2 (Expectation) ensures that the robust and non-robust features are positively correlated with the label. Given sufficient data, is always possible to preprocess features to positively align with their binary labels. For example, with large N, this can be achieved by multiplying each feature x; by sgn [E[yz,]) x sgn (E Vn)),,), ensuring Ely(ssn(E[(yz (E[yx:])=;) = [[[yz,]] 0.
Assumption (Moments) bounds the n-th central moment of each feature by a constant multiple of the 1-th power of its scale for n E (2, 4). This condition ensures that the feature distribution does not exhibit excessively large fluctuations relative to its scale (n = 2), extreme asymmetry (n = 3), or heavy tails (n 4). For example, with appropriate constant factors, exponential distributions satisfy this condition. Morcover, empirical studies suggest that pixel values (or contrasts) after typical filtering (e.g., Gabor filtering) approximately follow exponential distributions (Ruderman, 1994;
4

---

Published as a conference paper at ICLR 2026
Srivastava et al., 2003). This observation suggests that filtered pixel values (and contrasts) are broadly consistent with this assumption.
Assumption 3 bounds the number of features whose total covariance with other informative fea- tures (i.e., robust and non-robust features) is negative. As stated in Theorem 3.6, we typically assume that Grob and qvul small (but not necessarily infinitesimal). This assumption prevents unrealistic cases where useful features are overly anti-correlated with others, which could hinder learning. When all the predictive features are conditionally independent given the label, grob and Qvul = satisfy this assumption. Empirically, Goob and gvul appear to be small in real-world datasets (cf. Fig. A2).
These conditions encompass wide class of realistic data distributions.
Example 1: Training data distribution. Each training distribution D is a special case of the test distribution D Specifically, it contains drob 1 robust feature with scale a 1 and dvul = d 1 non-robust features with scale 3 = X. There are no irrelevant features, i.e., dirr = 0. By construction and due to the properties of uniform distribution, this distribution satisfies all the conditions in Assumption 3.2.
Example 2: Standard distributions. With appropriate constant factors, the test distribution class includes standard distributions, such as uniform, normal, exponential, beta, gamma, Bernoulli, binomial, etc. For example, consider the normal distribution. For Sroh, Assumption 2 is satisfied by setting ye; ~ (aa,, a²) with C; = 1, Ct.2 = 1, G.,3 O, and Ci = 3. Assumptions 3 and 4 are satisfied when all the features are mutually independent.
Empirical evidence suggests that preprocessed MNIST (Deng, 2012). Fashion-MNISS (Xiao et al., 2017), and CIFAR-10 (Krizhevsky, 2009)
Example 3: WNIST/Fshhion-MNST/CCEAAA
y approximately satisfy our assumptions. Consider MNIST. Let {an N J==1: {aa Jx=1 E [0. 1]784 denote the samples of digits zero and one, respectively. We assign
1 to digit zero and with Ø (1/2N) Σ> (w..) + ) and align (1))
-1
digit one. We center the data via x' f-
features with the label using x" +- sgn E (xx (0) on )) @ x'. In this representation, common background pixels have near-zero expectations (i.e., y 0), while discriminative pixels such as the left and right ares of zero or the vertical stroke of cne-correlate strongly with the label (i.e., a R 0.2) (cf. Fig, A2). Additionally, some pixels that are occasionally activated by atypical samples (e.g., corners activated by slanted digits) exhibit weak correlation with the label (i.e., 3 0.01), reflecting non-robust but predictive attributes. Empirical analysis reveals that most pixels exhibit positive total covariance with others, consistent with Assumption 3 (cf. Fig. A2). The main departure from Assumption 3.2 is that real-world datasets exhibit a gradual transition in feature robustness rather than an explicit binary separation between robust and non-robust features.
Example 4: Linear combination of orthonormal bases. Under mild conditions, any distribution in which robust and non-robust directions form an orthonormal basis can be transformed into our setting via principal component analysis (cf. Appendix C).
Adversarial Attack. We assume that the query (test sample) # is subject to the adversarial perturbation A constrained by the fox norm, i.c. | |0 S c, where e 2 denotes the perturbation budget. In practice, is chosen to be consistent with the scale of non-robust features (c.g., x À for the training distributions and ss 3 for the test distribution). This ensures that perturbations can manipulate non-robust features while leaving robust features intact and remaining imperceptible to humans.
Pretraining With In-Context Loss. For pretraining, consider the following problem based on the in-context loss (Ahn et al., 2023; Bai et al., 2023; Mahankali et al., 2024; Zhang et al., 2024b):
min I. (xie) o-U((())(((+.....)))) Nart
Ne max ||
(7)
This formulation encourages the transformer to extract robust, generalizable representations from N clean in-context demonstrations and to accurately classify the adversarially perturbed query. We impose the constraint on the transformer parameters to prevent the problem from becoming ill pooeed.ed. We choose (0, 1]d instead of [-1, to simplify the theoretical derivation.
5

---

Published as a conference paper at ICLR 2026
1.0
0.5
0.0
10
20 0
10 20
0
10
20 0
10
20
Std. P
Std.
Q
Adv. P
Adv. Q
Str.
adv. P
Str.
adv. Q
Figure I: Parameter heatmaps learned via adversarial training (7) with = 20 and À = 0.1. For the standard, adversarial, and strong adversarial regimes, we used = 0, -+(d11)(A/3) = 0.098, and 2-A
(3-1))2+3 0.95, respectively. We optimized (7) by stochastic gradient descent. Detailed experimental settings can be found in Appendix D.
3.2 LINEAR CLASSIFIERS AND ORACLE
Standard Linear Classifiers Extract All Features and Are Therefore Vulnerable. As a warm- up, consider standard training of a linear classifier parameterized by Rd on the e-th training distribution D Standard training yields w := arg [00 ((.. [-yw x] la. This classifier utilizes all the features, i.e., the robust feature at the e-th dimension and the other non-robust features. Although wttd achieves correct predictions on clean samples (Elywwstd æc] 0), it is L+(-)1A/22) d This implies that when d is small, the perturbation must satisfy Z 1 to flip the prediction. In this regime, the perturbation alters the robust feature and is no longer human-ime mamaan--pperepptblee However, dl increases, the threshold decreases Z A, which matches the scale of the non-robust features. Such perturbations are human-imperceepibbl yet sufficient to cause misclassification.
vulnerable to adversarial perturbations: E|min| >o yuutd (æ + A) S 0 for 2
Linear Classifiers Can Be Specifically Robust but Not Universally Robust. Consider adversarial St -yw (x + A) For < E 1, the optimal solution wdv equals one at the e-th dimension and zero otherwise. The classifier relies solely on the robust feature at the c-th dimension and ignores the other non-robust features. Unlike the standard model, this model can correctly classify both clean and adversarial samples for 0 S c < 1: thus, linear classifiers can be robust to specific training distribution. However, "dv tailored to D is vulnerable on the other distributions Dy: indexed by c' + c; thus, linear classifiers cannot universally robust.
training: [o.1] E6 [mmax
Universally Robust Classifiers Exist. Although linear classifiers cannot exhibit universal robustness across all c, universally robust classifiers do exist. For example, the classifier h((z)) == sgn (I;) with i arg [a [zy] always produces correct predictions for clean data Dtr for any and perturbed data æ + with || no S
3.3 ADVERSARIAL PRETRAINING
In this section, we analyze a global minimizer of the minimization problem (7).
Optimization Challenges. Although the training distributions simple, the minimization problem (7) remains nontrivial due to the non-linearity and non-convexity of the model with respect to the trainable parameters P and Q. The non-linearity of the self-attentio and the mnner-maximizaion also obstacles. Indeed, the minimization problem (7) can be reformulated as the following non-linear maximization problem:
Lemma 3.3 (Transformation of original optimization problem). The minimization problem (7) can transformed into the maximization problem (0. Σ max(0, > by b,h,,)), where ha.j E TR. is constant depending on (i.j), and there exists mapping from b to P and Q.
The proof can be found in Appendix E. This lemma highlights the inherent difficulty of optimizing (7), as it requires selecting a binary vector b that balances d(d + interdependent non-lineat terms.
Emmmnn Allos yud (x
A)] u
(T[(ar)
ela) {1 + (d 1)(22)) de

---

Published as a conference paper at ICLR 2026
Global Solution. By exploiting the symmetric property of b and further transformation of the problem in Lemma 3.3, we identify the global solutions to (7) for certain perturbation regimes.
Theorem 3.4 (Parameters learned via adversarial pretraining). The global minimizers of (7) are
11. Standard; 6 =0) P = petd =
0 Od.d+1 1
and Q Qstd := [1a+1d 0s+1]
2. Adversarial; I+(d (ild-DDX//2)
p padv
0ddd+1 and Q Qadv 1d+1
La
0.
0
0
Strongly adversarial; 6 2
2- +3
P= 0=41,d4 and Q = 0d+1,d+1
The proof and optimal parameters for different can be found in Appendix E. Importantly, the optimal P and Q are independent of any specific training distribution (i.e., index c), reflecting that the transformer acquires learning capability from demonstrations rather than memorizing individual tasks. Experimental results obtained using gradient descent align with our theoretical predictions (Fig. 1).
Failure Case. the strongly adversarial regime, the global optimum becomes = Q causing the transformer to always output zero regardless of the input. Namely, no universally robust single- layer linear transformers exist, despite the existence of universally robust classifiers (cf. Section 3.2). The perturbation scale 2 + 3 2-A
13322)2+3 decreases in d: it transitions from c = 1 when d =
to c + as > oc. moderate dimensions (d ~ }t) adversarial perturbations must satisfy Z to break the robustness. They are comparable to the scale of the robust features and thus perceptible to humans, contradicting the concept of adversarial perturbations. However, in extremely high dimensions (d 2 )a), it suffices perturb by only Σ X, which on the same scale as the non-robust features and is typically imperceptible, thus preserving the concept of adversarial perturbations. This can rephrased follows: under our training distributions, single-layer linear transformers cannot achieve universal robustness when the non-robust dimensions (i.e., d 1) substantially outnumber the single robust dimension.
3.4 UNIVERSAL ROBUSTNESS
In this section, we show that adversarial pretraining, combined with in-context learning from clean demonstrations, can yield universal robustness both seen and unseen distributions.
Standard Pretraining Leads to Vulnerability, We begin by showing that standard model fails to classify adversarially perturbed inputs.
Theorem 3.5 (Standard pretraining case). There exist constant 0 and strictly positive
function (ddrb, dyuut dirr; 3, 7) such that
E (1. ((e. N.. 11ii
UN+il/(Za: pstd Q Jati, [e
min
s g(drob, dyul. din ar B,7)
{
Cldroba + d
(drob + dyyul + dirre
}.
(8)
Prediction original data
Adversarial effect
The proof can be found in Appendix This result analyzes the expectation of the product of the true label and the model prediction for the query. A positive value indicates correct classification, whereas a nonpositive value indicates misclassification. Since g(drob; dvul, dirr: Ot, 3, )) is always positive, when C(drosaa + d - (dras + deut + dirr)e ) is nonpositive, this implies incorrect classification.
Standard models extract both robust and non-robust features and thus are vulnerable. Assume diee = 0. Like standard linear classifiers, the standard model leverages both robust features dyoba and non-robust features 3. This also leads to vulnerability to adversarial perturbations, -(drob + d. The prediction becomes incorrect when C(droba + d - (drob dyui)e < 0, i.e. when droba-deu18 When the perturbation size is the same scale the non-robust features S (), doob-dvul

---

Published as a conference paper at ICLR 2026
the inequality can be rearranged as d 2 - In typical cases where the scale of the robust features is much larger than that of the non-robust features (a: >> B3), we can informally conclude:
(Informal restatement of Theorem 3.5) Assume that the scale of the robust features is much larger than that of the non-robust features (ax > 3), the perturbation size is on the same scale the non-robust features (e Σ 3), and there are no non-predictive features (dirr = 0). If d 2 2 drab. 3
then the standardly pretrained single-layer linear transformer is vulnerable to adversarial attacks.
Non-predictive features accelerate vulnerability. Redundant dimensions dire do not contribute to the first term (i.e., accuracy) but increase the second term (i.e., vulnerability). Therefore, they degrade robustness without improving predictive performance. In addition, dirr amplifies the adversarial effect rate of d which is comparable to the effect of the useful dimensions, drobe and dyule.
Adversarial Pretraining Leads to Universal Robustness. We now establish universal robustness of the adversarially pretrained model.
Theorem 3.6 (Adversarial pretraining case). Suppose that grob and vvul defined Assumption 3.2 are sufficiently small. There exist constants C1, C2 0 such that
min UN+I]f(ZA: Padv 3a E Ma) Die Sf
C (droba dvmn + (ddroaa + d2
Prediction original data
- Cs (dhusaa + deul.3 1) (drabar + d + dirr?
Viitr N + +1 72 }
+ dier
(9)
VN
Adversarial effect
The proof and a more general statement can be found in Appendix F and Theorem F.I, respectively, For notational simplicity, we assume small Grob and fvul- However, we do not require infinitesimal 9rob and (vul- See Theorem F.1 and Appendix C. In contrast to Theorem 3.5, this theorem provides lower bound. A positive right-hame hand side implies correct classification under adversarial perturbations.
Adversarially trained models prioritize robust features. Assume di = O. Up to constant factors, the lower bound reduces to (droba + dvv113 + () + dvui,32 (deat, + dluut3)e), The important factor is drob + d - (Crob + dv,, which determines the sign. As shown in Theorem 3.5, the standard model extracts the robust and non-robust features at scales droba and dvut respectively, In contrast, the adversarially trained model extracts them at quadratic scales dioba² and 8 Since the robust features typically have larger magnitude (a2 >> B²), the adversarially trained model places greater emphasis on the robust features and mitigates the influence of the non-robust features, compared to its standard counterpart.
Adversurially trained models are universally robust. As shown above, up constant factors, the prediction remains correct as long as draba + d2 - (drob + dvui13) 2 0. This condition fails when 2 dooa' +d When the perturbation size is the same scale as the non-robust features (e S [), the inequality can be rearranged as dyul draaala- B) In typical cases where a > B, we can informally conclude:
82
(Informal restatement of Theorem 3.6) Assume that the scale of the robust features is much larger than that of the non-robust features (a >> 9), the perturbation size is on the same scale the non-robust features (c S 8), and there are no non- non- -on-predie features (drr 0). If d 2 (g ? then the adversarially pretrained single-layer linear transformer is robust to adversarial attacks.
This threshold substantially improves on the standard model's robustness condition. For example, when a = 160/255 and / = 8 8/255, the standard model is potentially robust up to dyul S. 20d whereas the adversarially pretrained model remains robust up to d 400d,ob- This result also suggests that the adversarially pretrained model is potentially vulnerable when the non-robust dimensions significantly outnumber the robust ones, consistent with the failure case in Section 3.3.
8

---

Published as a conference paper at ICLR 2026
Adversarially trained models are more robust attacks that exploit non-predictive f Theo- rem 3.6 shows that even when the adversary exploits redundant dimensions, their effect is significantly attenuated. For simplicity, assume N + The adversarial effect from the irrelevant features scales as d which is linear in dir- In contrast, the clean term scales as d a and de d i.e., quadratically in the number of the informative features. Thus, long the informative features dominate in magnitude and number, the influence of the non-predictive features on the model's robustness remains limited.
3.5 OPEN CHALLENGES
In this section, we show that two open challenges in robust classification (Schmidt et al., 2018; Tsipras et al., 2019) persist in our setting.
CecuracyRRobustess Trade-Off. Inspired by Tsipras et al. (2019), we consider a situation where robust features correlate with their labels with some probability, whereas non-robust features always correlate.
Theorem 3.7 (Accuracy-robustnes. trade-off). Assume drob 1, dval = d - 1, and dir 0. In addition to Assumption 3.2, for (ar, y) ~ Dee, suppose that ye, takes a with probability p > 0.5 and with probability 1 - p for Stob. Moreover, ye, takes 3 with probability one for i E Syul Define f(P,Q) = E ((...,,)) itd 'ie [uN+ (f++If(Zo (Zo; P, 1). Then, there exist strictly
positive functions 91 (d, a, 3) and 92 a. B) such that
f(petd, Qud) = [i((, a, () + - 1)8) (w.p. p)
(10)
(d, a, () (d 1)33 (w.p. 1 1-p)
f(pady Qudv) S g2(d, a, (-p 1)a² + (d - 1) (w.p. 1-p).
(11)
The proof can be found Appendix G. Unlike Theorems 3.5 and 3.6, this theorem considers the expectation over {<r, yn...,)))))) ) instead of {<I,,, 3x)) The query (æN+1. -V+11 stochastic. If 2 3. the standard model consistently produces correct predictions. However, if S (2p-1)(g) the adversarially trained model produces incorrect predictions with probability 1 p. This trade-off arises because the adversarially trained model discards the non-robust but predictive features.
Need for Larger In-Context Sample Sizes. We informally summarize Theorem H. as follows (omit- ting constant factors for clarity):
(Informal summary of Theorem H.1) Assume the same conditions as in Theorem 3.7. Consider Ba [f P, Q)4++1N4+1) Assume d S P - 0.5, and small N regime. With probability least 1 exp(-N), the standard model makes correct predictions. With probability the adversarially trained model makes correct predictions. most VN.
This result indicates that in law-sample regimes, the adversarially pretrained model requires substan- tially more in-context demonstrations achieve clean accuracy comparable that of the standard model. This stems from the model's 's reliance on the robust features, which are statistically underrep- resented in small-sample regimes.
EXPERIMENTAL RESULTS
Additional results and detailed experimental settings provided in Appendix D.
Verification of Theorem 3.4. We trained single-layer linear transformers (2) using stochastic gradient configured with d = 20 and À = 0.1. We used = 0, (d-2)) 0.098, and 5/2 + 2 (3-1)x²43 = 0.95 for the standard, adversarial, and strongly adversarial regimes, respectively. The heatmaps of the learned parameters shown in Fig. 1. These results align with the theoretical predictions of Theorem 3.4.
descent over [0, 1]d with the in-context loss (7). The training distribution
3
Verification of Theorems 3.5 to 3.7. We evaluated the standardly and adversarially pretrained single-layer linear transformers with the theoretically predicted parameters (i.e., the parameters in

---

Published as a conference paper at ICLR 2026
Table I: Accuracy (%) of standardly and adversarially pretrained single-layer linear transformers. Left values represent clean accuracy. Right values represent robust accuracy. For D (cf. Assumption 3.1), we used d = 100 and 0.1. For D (cf. Assumption 3.2), we constructed the test distribution from multivariate normal distributions with dzob = 10, dvul 90, dir 0, a = 1.0, and / = 0.1. For the real-world datasets, the values were averaged across all 45 binary classification pairs from the 10 classes. The perturbation budgets were set as follows: = 0.15 for D 0.2 for Die, 0.1 for MNIST and CIFAR-10, and 0.15 for Mashion-MNIST See Appendix D for details.
D
Die
MNIST
FMNIST
CIFARIO
Standardly pretrained model Adversarially pretrained model
100 /
100 /
94
91 / 20
68
21
100 / 100 99 / 95
93 /
72
89 / 62
64 34
the standard regime and adversarial regime in Theorem 3.4) on Dtr Die, MNIST (Deng, 2012), Fashion-MNIST (Xiao et al., 2017), and CIFAR- (Krizhevsky, 2009). These results are provided in Tab. 1. The results suggest that the standard models achieve high clean accuracy but suffer severe degradation under adversarial attacks, consistent with Theorem 3.5. In contrast, the adversarially pretrained models maintain high robustness, supporting Theorem 3.6, while their clean accuracy is lower, aligning with the accuracy-robustness trade-off described in Theorem 3.7.
5 CONCLUSION AND LIMITATIONS
We theoretically demonstrated that single-layer linear transformers, after adversarial pretraining across classification tasks, can robustly adapt to previously unseen classification tasks through in- context learning, without any additional training. These results pave the way for universally robust foundation models. We also showed that these transformers can adaptively focus robust features, exhibit the accuracy-robustees trade-off, and require a larger number of in-context demonstrations.
Our limitations include assumptions about data distributions and architectures. While we assume that the data distributions consist of clearly separated robust and non-robust features, real-world datasets typically exhibit more gradual transition (cf. Section 3.1, especially Example 3). Single-layer linear transformers lack the practical characteristics of multi-layer models and softmax attention. Although these theoretical assumptions are standard and comparable in strength to those in prior work (cf. studies on in-context learning in Appendix A), they limit the applicability of our results.
Universally robust foundation models conceptually expected adapt any task and any form of perturbations. However, our theoretical results assume classification tasks and foo perturbations. Extending these results other tasks and perturbation models is left for future work.
The cost of adversarial pretraining is also limitation of universally robust foundation models. We expect that such efforts will be undertaken by large organizations, which could offset development costs through API fees. In addition, acceleration techniques for adversarial training, which have been extensively studied in the literature, can reduce this cost to level comparable to standard training. Our theoretical analysis is an important first step toward fostering the practical development of universally robust foundation models. See also the last paragraph in Section
REPRODUCIBILITY STATEMENT
All experimental procedures described in Section 4 and Appendix D. The source code to reproduce our experimental results can be found in https: / //githb.. com/ s- kumano/ / uuninvvi vivveeaalll y- robust-ttt-t- an-context--- learner. Proofs the theorems are provided in Appendices E to H.
THE USE OF LARGE LANGUAGE MODELS (LLMs)
We used LLMs improve our writing. No essential contributions made by the LLMss.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTE AND DISCLOSURE OF FUNDING
S. Kumano was supported by JSPS KAKENHI Grant Number JP23KJ0789 and by JST, ACT-X Grant Number JPMIAX23C7, JAPAN. H. Kera was supported by JST PRESTO Grant Number JPMJPR24K, JST BOOST Program Grant Number JPM/BY24C6, and JSPS Program for Forming Japan's ss Peak Research Universities (J-PEAKS) Grant Number JPJS00420230002.
REFERENCES
Kwangjun Ahn, Xiang Cheng, Hadi Daneshmand, and Suvrit Sra. Transformers learn implement preconditioned gradient descent for in-context learning. In NeurlPS, volume 36, pp. 45614-45650, 2023.
Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algorithm is in-context learning? investigations with linear models. In ICLR, 2023.
Ahmed Aldahdooh, Wassim Hamidouche, and Olivier Deforges. Reveal of vision transformers robustness against adversarial attacks. arXiv:2/06.03755 2021.
Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast adversarial training. In NeurlPS, volume 33, pp. 16048-16059, 2020.
Usman Anwar, Johannes Von Oswald, Louis Kirsch, David Krueger, and Spencer Frei. Adversarial robustness of in-context learning in transformers for linear regression. arX1v:2411.05/89, 2024.
Robert B Ash. Information Theory. Courier Corporation, 1990.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In ICML, pp. 274-283, 2018.
Maximilian Augustin, Alexander Meinke, and Matthias Hein. Adversarial robustness on in-and out-distribution improves explainability. In ECCV, pp. 228-222822 228-245,20 2020.
Yu Bai, Fan Chen, Huan Wang, Caiming Xiong, and Song Mei. Transformers as statisticians: Provable in-context learning with in-context algorithm selection. In NeurIPS, volume 36, pp. 77125-572111 2023.
Yutong Bai, Jieru Mei, Alan Yuille, and Cihang Xie. Are transformers more robust than cnns? In NeurlPS, volume 34, pp. 26831-26844 2021.
Philipp Benz, Soomin Ham, Chaoning Zhang, Adil Karjauv, and In So Kweon. Adversarial robustness comparison of vision transformer and MLP- -ixer to CNNs. In BMVC, 2021.
Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, and Andreas Veit. Understanding robustness of transformers for image classification. In ICCV, pp.
10231-10241, 2021.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In NeurIPS, volume 33, pp. 1877-1901, 2020.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, al. Extracting training data from large language models. In USENIX Security, pp. 2633-2651 2021.
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee. Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. In ICLR, 2022.
Prasad Chalasani, Jiefeng Chen, Amrita Roy Chowdhury, Xi Wu, and Somesh Jha. Concise explana- tions of neural networks using adversarial training. ICMI, pp. 1383-1391, 2020.
Xiang Cheng, Yuxin Chen, and Suvrit Sra. Transformers implement functional gradient descent to learn non-linear functions in context. In ICMI,, 2024.

---

Published as a conference paper at ICLR 2026
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter- free attacks. In ICML, pp. 1206-2216, 2020.
Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, and Furu Wei. Why can gpt learn in-context? language models implicitly perform gradient descent meta-optimizers. In ACL, 2023.
Edoardo Debenedetti, Vikash Sehwag, and Prateek Mittal. A light recipe to train robust vision transformers. In SaTML, pp. 225-253, 2023.
Li Deng. The MNIST database of handwritten digit images for machine learning research. Signal Processing Magazine, 29(6):141-13 2012.
Edgar Dobriban, Hamed Hassani, David Hong, and Alexander Robey. Provable tradeoff's in adversar- ially robust classification. IEEE Transactions on Information Theory, 2023.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image worth 16x16 words: Transformers for image recognition scale. In ICLR, 2021.
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Aleksander Madry. Adversarial robustness prior for learned representations. arXivv1996.00945. 2019.
Christian Etmann, Sebastian Lunz, Peter Maass, and Carola-Bibiane Schönlieb. On the connection between adversarial robustness and saliency map interpretability. ICMI., 2019.
Hao Fan, Zhaoyang Ma, Yong Li, Rui Tian, Yunli Chen, and Chenlong Gao. Mixprompt: Enhancing generalizability and adversarial robustness for vision-language models via prompt fusion. In ICIC, pp. 30--328-999 2024.
Spencer Frei and Gal Vardi. Trained transformer classifiers generalize and exhibit benign overfitting in-context. In ICLR, 2025.
Deqing Fu, Tian-qi Chen, Robin Jía, and Vatsal Sharan. Transformers learn achieve second-order convergence rates for in-context linear regression. In NeurlPS, volume 37, pp. 98675-987 98716, 2024.
Shaopeng Fu, Liang Ding, and Di Wang. "short-length" adversarial training helps llms defend "long-length" jailbreak attacks: Theoretical and empirical evidence. arXiv:2502.04204, 2025.
Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant. What can transformers learn in-context? a case study of simple function classes. In NeurlPS, volume 35, pp. 30583-30598, 2022.
Siddhant Garg and Goutham Ramakrishnan. BAE: BERT-based adversarial examples for text
classification. In EMNLP, pp. 6171- 2020.
Khashayar Gatmiry, Nikunj Saunshi, Sashank Reddi. Stefanie Jegelka, and Sanjiv Kumar. Can looped transformers learn implement multi-step gradient descent for in-context learning? In ICML, 2024.
Angeliki Giannou, Liu Yang, Tianhao Wang, Dimitris Papailiopoulos, and Jason D Lee. How well can transformers emulate in-context newton's 's method? arXiv:2403.03/88, 2024.
Micah Goldblum, Liam Fowl, and Tom Goldstein. Adversarially robust few-shot learning: A meta-learning approach. In NewrIPS, volume 33, pp. 17886-17895 2020.
Ian J Goodfellow, Jonathon Shiens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Yufan Hou, Lixin Zou, and Weidong Liu. Task-based focal loss for adversarially robust meta-lcarning. In ICPR, pp. 2824-2829, 2021.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In NeurIPS, volume 32, pp. 125-136, 2019.
12

---

Published as a conference paper at ICLR 2026
Ahmadreza Jeddi, Mohammad Javad Shafiee, and Alexander Wong. A simple fine-tuning is all you need: Towards robust deep learning via adversarial fine-tuning. arXiv:2012.13628, 2020.
Xiaojun Jia, Sensen Gao, Simeng Qin, Ke Ma, Xinfeng Li, Yihao Huang, Wei Dong, Yang Liu, and Xiaochun Cao. Evolution-based region adversarial prompt learning for robustness enhancement in vision-language models. arXiv:2503. 12874, 2025.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits, Is BERT really robust? a strong baseline for natural language attack on text classification and entailment. In AAAI, volume 34, pp. 8018-8025, 2020.
Simran Kaur, Jeremy Cohen, and Zachary C Lipton. Are perceptually-aligne gradients a general property of robust classifiers? NeurlPS WS, 2019.
Hoki Kim, Woojin Lee, and Jaewook Lee. Understanding catastrophic overfitting in single-step adversarial training. In AAAI, volume 35, pp. 8119-8127, 2021.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Jonathan Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, and Emma Brunskill. Supervised pretraining can learn in-context reinforcement learning. In NeurIPS, volume 36, pp. 43057-43083, 2023.
Lin Li, Haoyan Guan, Jianing Qiu, and Michael Spratling. One prompt word is enough to boost adversarial robustness for pre-trained laiionla models. In CVPR, pp. 24408-24419, 2024.
Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. BERT-ATTACK: Adversarial
attack against BERT using BERT. In EMNLP, pp. 6193-6202, 2020.
Tianle Li, Chenyang Zhang, Xingwu Chen, Yuan Cao, and Difan Zou, On the robustness of transformers against context hijacking for linear classification. arXiv:2502.15609 2025.
Licong Lin, Yu Bai, and Song Mei. Transformers as decision makers: Provable in-context reinforce-
ment learning via supervised pretraining. In ICLR, 2024.
Chang Liu, Yinpeng Dong, Wenzhao Xiang, Xiao Yang, Hang Su, Jun Zhu, Yuefeng Chen, Yuan He, Hui Xue, and Shibao Zheng. A comprehensive study on robustness of image classification models: Benchmarking and rethinking. IJCV, 133(2):567-589, 2025.
Fan Liu, Shuyu Zhao, Xuelong Dai, and Bin Xiao. Long-term cross adversarial training: A robust meta-learning method for few-shot classification tasks. In ICML WS, 2021.
Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. AutoDAN: Generating stealthy jailbreak prompts on aligned large language models. ICLR, 2024.
Lin Luo, Xin Wang. Bojia Zi Shihao Zhao, and Xingjun Ma. Adversarial prompt distillation for vision-language models. arXiv:2411.15244 2024.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. ICLR, 2018.
Arvind Mahankali, Tatsunori B Hashimoto, and Tengyu Ma. One step of gradient descent is provably the optimal in-context learner with one layer of linear self-attention. In ICLR, 2024.
Kaleel Mahmood, Rigel Mahmood, and Marten Van Dijk. On the robustness of vision transformers to adversarial examples. In ICCV, pp. 7838-7847, 2021.
Chengzhi Mao, Scott Geng, Junfeng Yang, Xin Wang, and Carl Vondrick. Understanding zero-shot adversarial robustness for large-scale models. In ICLR, 2023.
Mohammad Mehrabi, Adel Javanmard, Ryan A Rossi, Anup Rao, and Tung Mai. Fundamental tradeoffs in distributionally adversarial training. In ICMI., pp. 7544-7554, 2021.
13

---

Published as a conference paper at ICLR 2026
Muhammad Muzammal Naseer, Kanchana Ranasinghe, Salman H Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming- Sing--suann Yang. Intriguing properties of vision transformers. In NeurIPS, volume 34, pp.
2021.
Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv:2311. 17035, 2023.
Geon Yeong Park and Sang Wan Lee. Reliably fast adversarial training via latent adversarial perturbation. In ICCV, pp. 7758-7767, 2021.
Sayak Paul and Pin- Yu Chen. Vision transformers are robust learners. In AAAI, volume 36, pp. 2071-2081, 2022.
Fábio Perez and Ian Ribeiro. Ignore previous prompt: Attack techniques for language models. In NeurlPS WS, 2022.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John C Duchi, and Percy Liang. Adversarial training can hurt generalization. In ICML WS, 2019.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John Duchi, and Percy Liang. Understanding and mitigating the tradeoff between robustness and accuracy. In ICML, 2020.
Daniel L Ruderman. The statistics of natural images. Network: computation in neural systems, 5(4):
517, 1994.
Shibani Santurkar, Andrew Ilyas, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Image synthesis with single (robust) classifier. In NeurlPS, volume 32, 2019.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adver- sarially robust generalization requires more data. In NeurIPS, volume 31, 2018.
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In NeurlPS, 2019.
Rulin Shao, Zhouxing Shi, Jinfeng Yi, Pin- Yu Chen, and Cho-Jui Hsieh. On the adversarial robustness of vision transformers. TMLR, 2022.
Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. "Do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. In ACM CCS, pp. 167 1-655, 2024.
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. Large language models can be easily distracted by irrelevant context. In ICML, pp. 31210-311 31227, 2023.
Zhenmei Shi, Junyi Wei, Zhuoyan Xu, and Yingyu Liang. Why larger language models do in-context learning differently? ICML, 2024.
Suraj Srinivas, Sebastian Bordt, and Himabindu Lakkaraju. Which models have prrceptuulll--aignne gradients? an explanation via off- robustness. In NeurIPS, volume 36, 2023.
Anuj Srivastava, Ann B Lee, Eero P Simoncelli, and S-C Zhu. On advances in statistical modeling of natural images. Journal of mathematical imaging and vision, 18(1):17-33, 2003.
Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin- Yu Chen, and Yupeng Gao. Is robustness the cost of accuracy?-a comprehensive study on the robustness of 18 deep image classification models. In ECCV, pp. 631 2018
Satoshi Suzuki, Shin'y ya Yamaguchi, Shoichiro Takeda, Sekitoshi Kanai, Naoki Makishima, Atsushi Ando, and Ryo Masumura. Adversarial finetuning with latent representation constraint to mitigate accuracy-robustness tradeoff. In ICCV, pp. 4367-4378, 2023.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICIR, 2014.
14

---

Published as a conference paper at ICLR 2026
Shiyu Tang, Ruihao Gong, Yan Wang, Aishan Liu, Jiakai Wang, Xinyun Chen, Fengwei Yu, Xiang- long Liu, Dawn Song, Alan Yuille, et al. RobustART; Benchmarking robustness on architecture design and training techniques. WrXh::2109002211 2021.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. In NewrlPS, volume 33, pp. 1633-1645, 2020.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be odds with accuracy. In ICLR, 2019.
Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent. In ICML. pp. 35151-35174, 2023.
Johannes Von Oswald, Maximilian Schlegel, Alexander Meulemans, Seijin Kobayashi, Eyvind Niklasson, Nicolas Zucchet, Nino Scherrer, Nolan Miller, Mark Sandler, Max Vladymyrov, et al. Uncovering mesa-optimization algorithms in transformers. In ICLR WS, 2024.
Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial triggers for attacking and analyzing NLP. In EMNLP-IJCNLP, 2019.
Ren Wang, Kaidi Xu, Sijia Liu, Pin- Yu Chen, Tsui- Wei Weng, Chuang Gan, and Meng Wang. On fast adversarial robustness adaptation in model-agnostic meta-learning. ICLR, 2021.
Sibo Wang, Jie Zhang, Zheng Yuan, and Shiguang Shan. Pre-trained model guided fine-tuning for zero-shot adversarial robustness. In CVPR, pp. 24502-24511, ,,,,,,411,,2,,,,,, 2024a.
Xin Wang, Kai Chen, Xingjun Ma, Zhineng Chen, Jingjing Chen, and Yu-Gang Jiang. Advqdet: Detecting query-based adversarial attacks with adversarial contrastive prompt tuning. In ACM MM, pp. 6212 2221,, 2024b.
Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does LLM safety training fail? In NewrlPS, volume 36, pp. 80079-8011 2023a.
Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, et al. Larger language models do in-context learning differently. wrXiv:2303.03866, 2023b.
Noam Wies, Yoav Levine, and Amnon Shashua. The learnability in-context learning. In NeurIPS, volume 36, pp. 36637-36651, 2023.
Eric Wong, Leslie Rice, and Zico Kolter. Fast better than free: Revisiting adversarial training. In ICLR, 2020.
Boxi Wu, Jindong Gu, Zhifeng Li, Deng Cai, Xiaofei He, and Wei Liu. Towards efficient adversarial training on vision transformers. In ECCV, pp. 307-325, 2022.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: novel image dataset for bench- marking machine learning algorithms. /rXiv 077447, 2017.
Fan Yang, Mingxuan Xia, Sangzhou Xia, Chicheng Ma, and Hui Hui. Revisiting the robust general- ization of adversarial prompt tuning. arXiv:2405.Il 1144 2024.
Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Russ R Salakhutdinov, and Kamalika Chaud- huri. A closer look at accuracy vs. robustness. In NeurIPS, volume 33, pp. 8588-8601, 2020.
Chengxiang Yin, Jian Tang, Zhiyuan Xu, and Yanzhi Wang. Adversarial meta-learning. arXiv:1806.03316, 2018
Yuan Zang, Fanchao Qi, Chenghao Yang. Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. Word-level textual adversarial attacking combinatorial optimization. In ACL, pp. 6066-6080. 2020.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. In NewrIPS, 2019a.
15

---

Published as a conference paper at ICLR 2026
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, pp. 7472-7482, 2019b.
Jiaming Zhang, Xingjun Ma, Xin Wang, Lingyu Qiu, Jiaqi Wang, Yu-Gang Jiang, and Jitao Sang. Adversarial prompt tuning for vision-language models. In ECCV, pp. 56-72, 2024a.
Ruiqi Zhang. Spencer Frei, and Peter I. Bartlett. Trained transformers learn linear models in-context. JMLR, 25(49): 2024b.
Tianyuan Zhang and Zhanxing Zhu. Interpreting adversarially trained convolutional neural networks. In ICML, pp. 7502-7511, 2019.
Yufeng Zhang, Fengzhuo Zhang, Zhuoran Yang, and Zhaoran Wang. What and how does in-context learning learn? bayesian model averaging, parameterization, and generalization. arXiv:2305.19422 2023.
Yiwei Zhou, Xiaobo Xia, Zhiwei Lin, Bo Han, and Tongliang Liu. Few-shot adversarial prompt learning vision-language models. In NeurIPS, volume 37, pp. 3122-3156, 2024.
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv:2307.15043 2023.
16

---

Published as a conference paper at ICLR 2026
A Additional Related Work
17
B Clarification on Single-Task Pretraining and Task-Specific Adversarial Training
19
Additional Theoretical Support and Insights
19
C.I Linear Combination of Orthonormal Bases can be Transformed into Our Test Distri- bution.
19
C.2 Sufficient Number of Datasets Provide Universal Robustness
20
C.3 Effects of Grob and Gvul
20
C.4 Disadvantage of Standard Finetuning: Parameter Selection Perspective
20
C.5 Naive Adversarial Context may not Improve Robustness
21
Additional Experimental Results
D.I Support for Assumption 3.2.
D.2 Verification of Theorem 3.4.
21
D.3 Verification of Theorems 3.5 3.7 and H.I
22
E Proof of Lemma 3.3 and Theorem 3.4 (Pretraining)
22
F Proof of Theorems 3.5 and 3.6 (Robustness)
35
G Proof of Theorem 3.7 (Trade-Off)
42
Proof of Theorem H.1 (Need for Larger Sample Size)
43
A ADDITIONAL RELATED WORK
In-Context Learning. In-context learning has emerged remarkable property of large language models, enabling them to adapt to new task from few input-output demonstrations without any parameter updates (Brown et al., 2020). Recent work has shown that in-context learning can implement various algorithms (Bai et al., 2023; Garg et al., 2022). One research direction has linked in-context lcarning with preconditioned gradient descent through empirical (Akyürek et al., 2023; Dai et al., 2023; Garg al., 2022; Von Oswald et al., 2023; 2024) and theoretical analyses (Ahn et al., 2023; Bai et al., 2023; Cheng et al., 2024; Gatmiry et al., 2024; Mahankali et al., 2024; Zhang et al., 2024b). Additional results have indicated that in-context learning can implement ridge regression (Akyürek et al., 2023; Bai et al., 2023), second-order optimization (Fu et al., 2024; Giannou al., 2024), reinforcement learning (Lee al., 2023; Lin et al., 2024), and Bayesian model averaging (Zhang et al., 2023). terms of robustness, some studies have shown that in-context learning can act nearly optimal predictor under noisy linear data (Bai et al., 2023) and noisy labels (Frei & Vardi, 2025). Moreover, it has been demonstrated that in-context learning is robust shifts in the query distribution (Wies et al., 2023; Zhang al., 2024b), but not necessarily to shifts in the context (Shi et al., 2023; 2024; Wei et al., 2023b; Zhang et al., 2024b). In this study, we focus on the adversarial robustness of in-context learning, rather than the underlying algorithms or its robustness random noise and distribution shifts. Specifically, we examine whether a single adversarially pretrained transformer can robustly adapt to a broad range of tasks through in-context learning.
Norm- and Token-Bounde Adversarial Examples. Adversarial examples originally intro- duced as subtle perturbations to natural data, designed to induce misclassifications models (Croce & Hein, 2020: Goodfellow al., 2015; Madry al., 2018; Szegedy et al., 2014). These perturba-
17

---

Published as a conference paper at ICLR 2026
tions are typically constrained by norm-based distance from the original inputs. The robustness of transformers to such norm-bounded adversarial examples has been studied primarily in vision transformers (Dosovitskiy al., 2021). Several studies have shown that standard vision transformers are as vulnerable to these attacks as conventional vision models (Bai et al., 2021; Mahmood et al., 2021), though some have reported marginal differences (Aldahdooh et al., 2021; Benz al., 2021; Bhojanapalli et al., 2021; Naseer et al., 2021; Paul & Chen, 2022; Shao et al., 2022; Tang et al., 2021). In contrast, adversarial attacks on language models are often neither norm-constrained nor imperceptible to humans. They involve substantial token modifications (Garg & Ramakrishnan, 2020;
Jin et al., 2020; Li al., 2020; Zang et al., 2020), the insertion of adversarial tokens (Liu et al., 2024; Shen et al., 2024; Wallace et al., 2019; Wei et al., 2023a; Zou et al., 2023), and the construction of entirely new adversarial prompts (Carlini et al., 2021; 2022; Nasr et al., 2023; Perez Ribeiro, 2022;
Wei et al., 2023a). These attacks aim not only to induce misclassification (Garg & Ramakrishnan, 2020; Jin et al., 2020; Li et al., 2020; Wallace et al., 2019; Zang et al., 2020), but also to provoke objectionable outputs (Liu et al., 2024; Perez& & Ribeiro, 2022: Shen et al., 2024; Wei et al., 2023a; Zou et al., 2023) or to extract private information from training data (Carlini et al., 2021: 2022; Nasr al., 2023). They are generally bounded by token-level metrics (e.g., y., the number of modified tokens). In this study, we focus exclusively on norm-bounded adversarial examples. Token-bounded ones are out of scope.
Adversarial Training. Adversarial training, which augments training data with adversarial exam- ples, is one of the most effective adversarial defenses (Goodfellow al., 2015; Madry et al., 2018). Although originally developed for conventional neural architectures, adversarial training has also proven effective for transformers (Debenedetti al., 2023; Liu et al., 2025; Shao et al., 2022: Tang et al., 2021: Wu et al., 2022). A major limitation of adversarial training is its high computational cost. To address this, several methods have focused on more efficient generation of adversarial examples (Andriushchenko & Flammarion, 2020; Kim et al., 2021; Park & Lee, 2021; Shafahi et al., 2019; Wong et al., 2020; Zhang et al., 2019a) and adversarial finetuning of standard pretrained models (Jeddi et al., 2020; Mao et al., 2023; Suzuki al., 2023; Wang et al., 2024a). More recently, researchers have introduced adversarial prompt tuning, which trains visual (Mao et al., 2023; Wang et al., 2024b), textual (Fan et al., 2024; Li et al., 2024; Zhang et al., 2024a), or bimodal prompts (Jia et al., 2025; Luo et al., 2024; Yang et al., 2024; Zhou et al., 2024) in an adversarial manner. However, these methods require retraining for each task. In this study, we explore the potential of adversarially pretrained transformers for robust task adaptation via in-context learning, thereby eliminating the task-specific retraining and associated computational overhead.
Adversarial Meta-Learning. Adversarial meta-learning seeks to develop a universally robust meta-learner that can swiftly and reliably adapt new tasks under adversarial conditions. Existing approaches adversarially train a neural network on multiple tasks, and then finetune it on a target task using clean (Goldblum al., 2020; Hou et al., 2021; Liu al., 2021; Wang et al., 2021: Yin et al., 2018) or adversarial samples (Yin et al., 2018). In this study, we similarly aim to train such a meta- learner. However, rather than relying on neural networks and finetuning, we employ a transformer as the meta-learner and leverage its in-context learning ability for task adaptation.
Related but Distinct Work. We here review theoretical work on the adversarial robustness of in-context learning. Assuming token-bounded adversarial examples. prior studies have shown that even a single token modification in the context can significantly alter the output of a standardly trained model on clean query (Anwar et al., 2024), and deeper layers can mitigate this (Li et al., 2025). Assuming norm- and token-bounded examples, Fu et al. have shown that adversarial training with short adversarial contexts can provide robustness against longer ones (Fu et al., 2025). They considered a clean query and adversarial tokens appended to the original context. In this study, we explore how adversarially trained models handle norm-bounded perturbations to query in a clean context. As a result, we reveal their universal robustness that can be generalized to a new task from a few demonstrations.
18

---

Published as a conference paper at ICLR 2026
B CLARIFICATION ON SINGLE-TASK PRETRAINING AND TASK-SPECIFIC ADVERSARIAL TRAINING
Single-Task Task Adversarial Training and In-Context Learning. We should clarify that model trained on a single task, unlike one pretrained on multiple tasks (i.e., our main setting), lacks in- context learning capability. Namely, approach that combines adversarial training (not adversarial pretraining) and in-context learning is not feasible. Specifically, the parameters of model trained on single task differ significantly from those on multiple tasks (shown in Theorem 3.4). Such models cannot provide correct answers for new tasks via in in-context learning even in standard settings (without adversaries). The interesting property of transformers is that when trained on multiple tasks rather than single one-as with large language models they develop distinctly different parameters that enable in-context learning capability.
Performance Comparison with Task-Specific Adversarial Training. As the no-free-lunch theo- rem indicates, task-specific approaches achieve higher performance than the approach that combines adversarial pretraining and in- learning. Specifically, the following holds in terms of robust accuracy: (1) task-specific adversarially trained models 2 (2) adversarially pretrained models with in-context learning > (3) task-specific standardly trained models x (4) standardly pretrained models with in-context learning z 0%. More precisely, Models (2) have the limitation robustness as predicted in Theorems 3.4 and 3.6. However, if training and test distributions match, Models (1) do not. However, we emphasize that Models (1) require users to perform adversarial training for each individual task and cannot generalize to test distributions other than the one was trained on. This makes it unsuitable for our research focus on universally robust foundation models that can generalize across a wide range of tasks without task-specific adversarial training.
C ADDITIONAL THEORETICAL SUPPORT AND INSIGHTS
C.1 LINEAR COMBINATION OF ORTHONORMAL BASES CAN BE TRANSFORMED INTO OUR TEST DISTRIBUTION.
Our test data distribution, Assumption 3.2, can implicitly represent data distributions comprising robust and non-robust directions forming an orthonormal basis. Consider d orthonormal bases, {e. We set dur = O, namely = drob + dyul- Each data point is represented as æ = c1e1 02e2 + ... + caed. where coefficients
sampled probabilistically. These coefficients satisfy Elyc]] = Cj for E Stob and ,3 for E Syul- In addition, [E[[cc; E[yc)))"] S a for i E Slob and Sr for i E Svul- Given a dataset of N i.i.d. samples {(z,.. yn) yn)) 1 if cn, is independent of Cn,j for + j conditional on y, and N is sufficiently large, then the covariance of yæ can be approximated as:
N
Σ 2
Σ
Un
Un
E (yx
El ((yx)) (yxc Elya)))
(A12)
E
(ya) - es
Σ (yics - EElycs)es
(A13)
E((yc: - E[vc:l) -Llcylyee
(A14)
d
C,ea + ieS,ob
Σ Cr.Bee,e iES
(A15)
This implies that through principal component analysis for E, we can obtain d orthonormal bases, (e,). By projecting sample In onto these bases, we obtain transformed sample x', := (c,,1, .., Cn,a). This demonstrates that when data is sampled from a distribution comprising robust and non-robust directions forming orthonormal basis, if the coefficients
19

---

Published as a conference paper at ICLR 2026
mutually independent and the sample size is sufficiently large, can preprocess the data to satisfy Assumption 3.2. Importantly, this preprocessing relies solely on statistics derivable from training samples.
C.2 SUFFICIENT NUMBER OF DATASETS TO PROVIDE UNIVERSAL ROBUSTNESS
What determines the sufficient number of datasets needed to provide universal robustness to trans- formers? We conjecture that this may be determined by the number of robust bases. In this paper, we trained transformers using d datasets. This stems from training with datasets where only one dimension is robust (in other words, datasets with single robust basis), the number of dimensions d, and the assumption that all dimensions might contain robust features. If we assume that robust features never appear in the latter d' dimensions, following the procedure in Appendix E, we can train robust transformers using only d d' datasets that describe the first d - d' robust features. From this observation, we conjecture that the sufficient number datasets required to provide universal robustness to transformers depends on the number of robust bases in the assumed data structure.
C.3 EFFECTS OF Grob AND Qvul
We here analyze how grob and Qvul affect the robustness of adversarially trained transformer. As defined in Assumption 3.2, these parameters control the proportion of features whose total covariance with other features is negative. Theorem F.l suggests that the transformer prediction for unperturbed data can be expressed
Cldroba + dvu { -rrob droba 2 + (1 deu + (drobaaa + drul
(A16)
where
(maxieS C) C2.2)
C
(A17)
min C
Examining the term (I-cgrab)ee (1-Cgrob)drabaa (1 cq-cc 3 we observe that larger values of Groh and Qeul generally diminish the magnitude of transformer predictions. This indicates that negative correla- tions between features degrade the robustness of adversarially trained transformers. Additionally, the coefficient e is characterized by Sa. Ci,2, which represents a variance coefficient. This suggests that smaller feature variances enhance the robustness of adversarially trained transformers. For example, if each feature variance C2 is sufficiently small, even Grob 1 and Jvul 1 may be tolerated without significantly compromising robustness.
C.4 DISADVANTAGR OF STANDARD FINETUNING: PARAMETER SELECTION PERSPECTIVE
In this study, investigate task adaptation through in-context learning. As an alternative lightweight approach, standard finetuning- whhre all or part of the model parameters are updated- can also be employed. However, key drawback of standard finetuning is that it requires parameter updates, whereas in-context learning does not. Moreover, finetuning necessitates careful selection of which parameters to update. Our analysis shows that improper parameter selection during finetuning can compromise the robustness initially established by adversarial pretraining. Consider adversarially pretrained parameters, padv and Qadv, and D.r as downstream data distribution.
First, we examine the scenario where only P is updated while keeping Qudv fixed, formulated as:
min (e(x.1(++)xx(+)) ((=...w)))) tol [- UNN P,
P.Qadv) 1e+1.Naa.
(A18)
In this case, as shown in the proof in Appendix E, P patd pady) is the global solution. Consequently, as demonstrated in Theorem 3.6, the model's robustness is preserved.
Conversely, consider training Q while keeping padv fixed, formulated as:
min Qe ((s....))) N1 an Ad De [-yN+ (Zoo padv Q)44+1.441.
E
(A19)
In this scenario, Q- autd is the global solution. As established in Theorems 3.7 and H.1, while this configuration enables the transformer to perform well on unperturbed queries, it fails to maintain robustness against perturbed inputs.
20

---

Published as a conference paper at ICLR 2026
These findings highlight critical insight: achieving robust task adaptation through standard finetuning requires careful parameter selection; otherwise, the pretrained model's ' adversarial robustness may be compromised. This parameter sensitivity represents a disadvantage compared to in-context learning, which preserves robustness without requiring parameter updates.
C.5
NAIVE ADVERSARIAL CONTEXT MAY NOT IMPROVE ROBUSTNESS
One approach to enhancing the robustness of a standardly trained transformer to incorporate adversarial examples into the context. In this section, show that this is not the case in setting. Consider the following transformer input:
1
22
EN + AN ON+1 + AN+1
(A20)
y1
y2
UN
The adversarial perturbations for the context, A 1..... A N, are defined as A. == cyn 1d. In this setting, for e (+(=)1(A/2)) the standard transformer prediction is given by:
E
min A St
yN+1lf(Z': petd ( S..
(A21)
(a)) M DD
This result suggests that, in our setting, naive adversarial demonstrations do not improve the perfor- mance of the standard transformer. Intuitively, because adversarial training generates new adversarial examples at each step of gradient descent, fixed adversarial demonstrations may fail to counter newly generated adversarial perturbations to the query.
D ADDITIONAL EXPERIMENTAL RESULTS
All experiments were conducted Ubuntu 20.04.6 LTS, Intel Xeon Gold 6226R CPUs, and NVIDIA RTX 6000 Ada GPUs
D.1 SUPPORT FOR ASSUMPTION 3.2.
The statistics of preprocessed MNIST, Fashion-MNIST and CIFAR-10 are provided Fig. A2. Preprocessing conducted follows: (i) selection of two different classes from the ten avail- able classes and assignment of binary labels to every sample from the training dataset, creating {( ) (ii) centering the data via x' t x ã with ã == (1/N) Σ In, and (iii) aligning features with the label using x" + sgn V,, @ x'. These preprocessed datasets exhibit that each dimension has a positive correlation with the label and that few dimensions have negative total covariance. The main distinction from Assumption 3.2 is that their features are not clearly separated as robust or non-robust. Instead, they gradually transition from robust to non-robust characteristics.
D.2 VERIFICATION THEOREM 3.4.
We trained a single-layer transformer (2) with the in-context loss (7). The training distribution was configured with 20 and 0.1 in Fig. and with d 100 and À 0.1 in Fig. A3. For standard, adversarial, and strong adversarial regimes, we used e = 0. 1+(d11)(/2) = 0.098, and
++ (a-1)A 0.95 in Fig. and e 0, [+(d-1)(A2) 0.06, and 3/2 ( 0.77 in Fig. A3. Optimization was conducted using stochastic gradient descent with momentum 0.9. Learning rates were set to 0.1 for all regimes in Fig. 1, and to 1.0 for standard and strong adversarial regimes and 0.2 for the adversarial regime in Fig. A3. Training ran for 100 epochs with a learning rate scheduler that multiplied the rate by 0.1 when the loss did not improve within 10 epochs. In each iteration of stochastic gradient descent, we sampled 1,000 datasets {(x! yn ) with N 1, 000. The distribution index was randomly sampled from U[d)). meaning that in each iteration, each of the 1,000 datasets may have different e values. After each parameter update, we projected the parameters to [0, 1ld. Adversarial perturbation which represents the optimal attack. The heatmaps of the learned parameters in Figs. and A3 completely align with the theoretical predictions of Theorem 3.4.
calculated as A = -cyn sgn( egn(Ptt1. Z,M Z Q :.).
21

---

Published as a conference paper at ICLR 2026
D.3 VERIFICATION OF THEOREMS 3.5 TO 3.7 AND H.1
We evaluated standardly and adversarially pretrained single-layer transformers on Dur Die, the preprocessed MNIST, Fashion-MNIST, and CIFAR-I0 datasets. For network parameters, we used the theoretically predicted pstd and Qsta as standard model parameters and padv and Qadv as udversarially trained model parameters. This approach allowed to circumvent the computationally expensive adversarial pretraining for every distinct d setting. As described previously, our empirical results completely align with the theoretically predicted parameter configurations.
Configuration in Figs. A4 and A5. In Fig. A4, the basic settings were d = 100, A = 0.1,
N I, 000, and = 0.13. In Fig. A5, they were drob = 10, d = 90, dur = 0, a = 1.0. = 0.1. Y 0.1, and 0.2. The basic perturbation budget was set to 0.1. We considered 1,000 batches where each batch contained 1,000 in-context demonstrations (i.e., = 1000), and 1,000 queries. The test distribution Die was constructed based normal distribution. During sampling, ye, was sampled from ((a, ) for i E Syob. (/3, (²) for E Svul. and (N 72) for i E Sim. Each dimension is independent, given y.
Configuration in Fig. A6. The preprocessing procedure is described in Appendix D.1. As batches, we considered 45 binary class pairs from ten classes. The basic perturbation budget was set to 0.1. In the first row of Fig. A6, used all training samples in the training dataset. As queries, we used all test samples in the test dataset.
Analysis. In Figs. A4 A6, standard transformers consistently demonstrate vulnerability to adversarial attacks, whereas adversarially trained transformers maintain a certain level robustness, validating Theorems 3.5 and 3.6. However, adversarially pretrained transformers exhibit lower clean accuracy, supporting Theorem 3.7.
In Figs. A4 and A5, observe that a larger number of vulnerable dimensions increases model vulnerability. Conversely, Fig. A5 shows that a larger number of robust dimensions enhances model robustness. Robust models are less susceptible increasing vulnerable dimensions and benefit more from increasing robust dimensions.
Additionally, as predicted in Theorems 3.5 and 3.6, standard training exhibits vulnerability to increasing redundant dimensions, which is more detrimental than the harmful effect from increasing vulnerable dimensions, since redundant dimensions do benefit predictions and are only harmful for robustness. In contrast, adversarially trained transformers exhibit significant resistance increases in these dimensions.
The second row of Fig. A6 indicates that standard transformers still achieve high classification accuracy in small demonstration regimes, whereas adversarially trained transformers show degraded performance. These results align with our theoretical predictions, Theorem H.I.
E PROOF OF LEMMA 3.3 AND THEOREM 3.4 (PRETRAINING)
Lemma 3.3 (Transformation of original optimization problem). The minimization problem (7) can be d+1
transformed into the maximization problem max Σ I(d++) max(0. -5=1 byha.)), where hi, E IR is constant depending on (i. and there exists mapping from b to and Q.
Proof. See "Overview" below detail. The proof sketch is as follows:
Let simplify the transformer definition (2) as fax; 01, 02) = 0, where 01, O2 (0, 1}. Then the optimization problem (7) becomes
max yf + A;01,02) e (0.1)
min
min
max vnax-y(02200((r) +1).
(A22)
A
6 02 A
This can be transformed to:
min max max-y(01x20, +A)) 0,E00())11 A
min 0 (1x A)).
(A23)
6 0.E0(1011
22

---

Published as a conference paper at ICLR 2026
MNIST
FMNIST
CIFAR10
0.2
0.2
0.1
ym
0.0
0.0
0.0
151
784
0
326
784
24
3072
0
0
ΣS Cor(
576
784
626
2667
Sorted dimension
Sorted dimension i
Sorted dimension
Figure A2: Statistical properties of preprocessed MNIST, Fashion-MNIST and CIFAR-10 datasets. First row: Blue lines represent the mean of (1 (1/N) En= 3n across 45 binary class pairs and shaded regions represent the sample standard deviation. Orange lines represent typical perturbation magnitude. Green dashed lines represent the (pseudo) threshold between robust and non-robust dimensions. Second row: Blue lines represent the total covariance of each dimension with other dimensions and shaded regions represent sample standard deviation across the 45 binary class pairs. Green dashed lines represent the boundary between positive and negative total covariance.
1.0
0.5
100
0.0
0
50
100
50
100 0
50
100
50
100
50
100 0
50
100
Std. P
Std. Q
Adv.
Adv. Q
Str. adv. P
Str. adv. Q
Figure A3: Parameter heatmaps induced by adversarial training (7) with d 100 and X 0.1. For the standard, adversarial, and strong adversarial regimes, we used e = 0, [+ 1+dd-1)(A/2) 0.06, and
+
1-1 0.77, respectively. We optimized (7) by stochastic gradient descent.
2-
1.0
1.0
0.8
0.8
Accuracy
0.6
0.6
Adv
0.4
0.4
0.2
0.2
0.0
0.0
0.00
0.05
0.10
0.15
0.20
0
50
100
150
Perturbation size
Input dimension d
Figure A4: Accuracy (%) of standardly and adversarially pretrained single-layer transformers. Lines represent mean accuracy across batches and shaded regions represent unbiased standard deviation (notably small in magnitude). We used 1,000 batches, each containing 1,000 in-context demonstrations (N 1000) and 1,000 query examples. Base configuration parameters were d 100. 0.1, and = 0.15.
23

---

Published as a conference paper at ICLR 2026
1.0
1.0
1.0
0.8
0.8
Stdd (dval)
Accuracy
0.6
0.6
0.6
Adv (deal)
0.4
0.4
0.4
(dire)
Std
--- Adv (dur)
0.2
0.2
0.2
Adv
0.5
10
10a
10a
Perturbation size
Robust dimension dzob
Vul. dim. deal or Irr. dim. dux
Figure A5: Accuracy (%) of standardly and adversarially pretrained single-layer transformers. Lines represent mean accuracy across batches and shaded regions represent unbiased standard deviation. We used 1,000 batches, each containing 1,000 in-context demonstrations (N 1000) and 1,000 query examples. Base configuration parameters were dzob 10, dvul = 90, dir 0, a = 1.0, 3= 0.1, 7 0.1, and 0.2.
1.0
1.0
1.0
- Std
0.8
0.8
0.8
Adv
0.4
0.4
0.2
0.2
0.2
0.0
0.0
0.0
Accuracy
0.0
0.2
0.0
0.2
0.0
0.2
Perturbation size
Perturbation size
Perturbation size
1.0
1.0
1.0
0.8
Std (e 0)
0.6
Adv 0)
0.4
0.4
0.4
Std (e 0.1)
Adv (e 0.1)
0.2
0.2
0.2
0.0
10
10a
10P
T0
102
109
10
102
Demonstration size
Demonstration size N
Demonstration
Figure A6: Accuracy (%) of standardly and adversarially pretrained single-layer transformers. Lines represent mean accuracy across binary classification tasks (derived from all possible pairs ten classes) and shaded regions represent the unbiased standard deviation. The perturbation size was basically = 0.1.
24

---

Published as a conference paper at ICLR 2026
Since 01 takes only 0 1, the optimal strategy is always 01 1 when y2²00(2 A) is positive and 81 = when negative. This transforms the problem to:
max min 01 (ya² 02 0y + A)) 6 0,,0110011 à
max max min yr 2x20022000 02(x A)).
(A24)
0e f
&
Denoting all terms except 0g h:
max max(0, min 02 (x +)) 026(0,1)
max max(0, B2ht). (2e
(A25)
This provides intuition for the problem (0.11 E Zi=1 max(0, Dj=1 byheo) in Lemma 3.3.
I(d+1)
d+1
Theorem 3.4 (Parameters learned via adversarial pretraining). The global minimizers of (7) are
1I. Standard; E e= =0)
patd
0d.d+1
and Q =Qutd [Ld+1,d Oa+1].
141
Adversarial; e 1+(d- (XA/2)
P padv
0add+ 1 1
and Q Qudv :
La
0g
02
3. Strongly adversarial; 2 2 +
2-d
P = 0s41,d+1
and Q 0d+1,d411
(d-1)A243
Proof. This special case of the following theorem.
Theorem E.1 (General case of Theorem 3.4). The global minimizer of (7) is as follows:
If
A(A(d 2) 4)
0sE5
(A26)
2(\(d - 1) + 2)
0a.a+1 1 1
then
and Q = [1a+1,d 0s+1].
If
1+ (d 1)(/22)
(A27)
04.041
Ia 0,
then
and Q
1
0g
0
If
3 2 - A
2 2
(A28)
2
(d
1) 3
then -04+1,++1 and Q 0.+
Proof.
Overview. The loss function L(P,Q) is determined only by the last of P and the first d columns of Q. Let
Ou T
P =
Q Q: [A 02t1l.
(A29)
where b gd+1 and A = (a aul E [(d+l)xd With 6, A, and G ZA ZAMZ /N, the loss function L(P, Q) can be represented as:
C(P, Q) == E c. ((o Ae
max -IN+1 P,QU:,LNHH
(A30)
25

---

Published as a conference paper at ICLR 2026
1 PZAMZZQZA Za + N
B ((((nn)))))) N+1
max -UN-1
(A31)
| Sc
441,N41
UN++I GATTGAATNNTT + A) >1
= (((((a)))))) N+1
max
(A32)
Using b and A, we redefine the loss function as [(b, A) == C(P, Q). Since G does not include A and m 1l.o Sc
A e||w/|| for w R. the inner maximization can be solved as:
Cb,, A) e ((....)))) N1: [-yN+16 6 GAæN+1 +e||b GAIli].
(A33)
When 0 < < 1 and S A v 1, then GA = GTA since all the elements of G are nonnegative. Thus,
min A)
0EbE1,05A5E A<i
min 055E1,05AS1
(.(( Nit V- GAEN+i cb cb GA1].
E
(A34)
Let the i-th row of be g. Rearranging the argument of the expectation as:
-UN+1b
GAEN+1 " GA
Aj.k
Σ bog.,(UN++RN-1AA-)
(A35)
Thus, the objective function can be represented as:
d++1
ΣΣ
A.M.
Σ
b. e. c ((a www.:rooooooNNNN...... - -e)]
(A36)
OSbELUSAS
j=1k=1
Since the objective function is linear with respect to and A, respectively, the optimal solution exists on the boundary:
d+1 d
d+1
max be (0.1))d (0. 11xx
Aj.
Σ
b,E (.
-e]] -
- (A37)
b=1 Am1
This is maximized by AjAa- otherwise. Now,
1 Sddt b,E (M.. (
o)) 2 0 and
d+1 d
d+1
max
ΣΣ
Σ
b,E (. ((...)))) (1,0)///11.,00NNNNHLLLL -e]]
(A38)
(0.01)) 4+1
j=l km
where ((x) == max(0, x)). Calculating the expectation and optimizing b, we obtain the solution.
Calculation of the expectation. First, we consider the expectation given c. Since Yn = i c and Tn U(0, A) otherwise, the expectation of yn be calculated as:
[1
(i ===) c)
Elynnn |c]
Eluma T
cJ =
[
(A39)
(i +
The expectation of G can calculated as:
(( y |G
(( Z MA
(A40)
Σ == Eu, x cl
ΣN
Ei Nn En 1
(A41)
Σm= Ex Exnii 20 [c]
N
Ex [In 20 c)
Ex un Ie c/]
(A42)
Ex. Mhn 1B/r xn
26

---

Published as a conference paper at ICLR 2026
For yn 1 and i,J + c, B[a |c] So x² /
X2/3 and Elun,stnn | Elxn, |c
= X2/4. /4. Thus,
(i = c) A(j=ji,
A
(i (i=c)\(+(id+1)
1²
(EE|d,i++)A((=))
E (( Jiis lc]
(vE(0jif(A(=cd-])
(A43)
X2
1
(i (==++)A(==cdt1)
(i (=d+1)A(ffddt1)
Note that
E ((m...
G c]
oth
A²/3
A2/4
22/4
22/4
X/2
22/4
22/4
X/2
X2/4
X2/3
X2/4
X2/4
X/2
X2/4
X2/4
X/2
X²/4
A2/4
X2/4
A2/3
X/2
X2/4
A2/4
1/2
2/2
X/2
A/2
A/2
1
A/2
A/2
bo-th.
(A44)
A2/4
X2/4
X2/4
A/2
X2/3
X2/4
A/2
A2/4
A2/4
X²/4
A2/4
X/2
X2/4
X2/3
X/2
X/2
X/2
X/2
X/2
X/2
A/2
Let
1s(j; k; = E [( (1 1.8 (9i (IN -
(A45)
Let c 1 and c_ X/2
c. By Eqs. (A39) and (A43),
C
(i [d] A (j -.j=i,+11p A (I: = (c (c= 1)
6_
(i A =j (c= i)
C
(i [d]) [0++
36
(i ([((())))(((((1))(44))))
12
J
6
3 ({E()){j==xx===<(4{]
3
24 ((=()))(=c+)))((=))((++
(A46)
14 ((=())x(fiicd))))(==\o++
1 ($=())(ii))(+::c)c(+}}
3 (1=())(=cc4+)))4+44k44
2
46 6 (((1))((Hic444)))444444444
64
(1 (=d+1)Aj=cd+()x((=)
6
(i (=d41))6==c(+1)x(44
$4 (=d=1))(fcd+)))AA=
(i ==
Then, we compute the expectation along c. Note that
E ((m( (AA
M(::;)))
(A47)
Let Hig.k Ed hs k; c). The summation of h; along be calculated as:
27

---

Published as a conference paper at ICLR 2026
For (i [d]) A (j- = 1) A (k= 1),
(hi = + X2
d
High [1,k= i i; i)
= 1)e_ (A48)
=1
(A49)
For (i E [d] (j = i) ^ (k +1),
HasA= = h4]x= -
Σ #b (A50)
c
X2
3 + X2
+
2)E_
(A51)
3
3
=:
(A52)
For (i (t 1 (k- 1), d+
d
Hig. = h₂6j = + 1; = 1;
i) + Σ -h==d+LA=tc++ i)
(A53)
oF
A ld - 1)c-
=4+ +
(A54)
2
(A55)
For (1 [d] = d+11 A (1: +1),
Hig, = = d d Maaa:-------kiiiio--------------
d
+
Σ
1
(A56)
X
=
(d
(A57)
2
(A58)
For (i [d) # i, + 1) A (k- = i),
Hes = hu6j i, ., - i)
+
Σ
(A59)
A
X2
+
--
(d - 2)e_
(A60)
2
2
4
=: T3
(A61)
For (i (de(d) [d) + +i 1) A #1) Aj A0j=j k).
Haj.a hi # -h)
Σ
hb+ 1: 1,kcct+
(A62)
A c_ + 1--2244- 2 1 2d-
A X2
(A63)
2
=: T'-
(A64)
For +
28

---

Published as a conference paper at ICLR 2026
+
Σ
h42
(A65)
Gi.ik
x²
X²
Je f-
4 c4
(d 3)
(A66)
4
=: l..
(A67)
For = d 1),
HiJA h46 + - + Σ
hil - k)
(A68)
c/k
=4+ (d- 1)6- 11)--
(A69)
=18.
(A70)
For (i= ((==d+))((++++ 1 k),
H1 h4 c; k kc=k) k)
E
#c k)
(A71)
of
=4+ = 2 1)e
(A72)
(A73)
For (i = =d+1+((f++0 # (d+))
Hij. = =
d
+
Σ
(A74)
ejjk
X
=
E+
2dd
2)E-
(A75)
T4.
(A76)
Optimization of A and b. From Eq. (A38), we redefine the objective function as:
d+1 d
d+1
d max bE(0.1)
be
e.
(UN +)]
11
d+1
d+1
max
Σ by Hi
(A77)
be(0.1)441
j=1 k=1 k=1
Recall that we set AjA = 1 if EdH by Hijjk 2 0 and O otherwise. Let [d]' = (i E [d] by = 1}
and d' = [[]]]. Now,
d+1
beHlii
Σ Φ
b4r1Ha+1,0414
1[k [d])
Σ
Hid+1,
Veldy' degt
Σ
bat1Hatiss I E [d)) +
Σ Hass
seld
d
be41 + 1lj e H
j=1 Risk
+ 1 Hess +
High
(A78)
kE(d),iz
29

---

Published as a conference paper at ICLR 2026
By Eqs. (A55), (A58) and (A70),
HettHat1d + 1[k [d]] +
E
Hidd11k
k=1
d
b84177+ + E [d])rs +
Σ r4
(A79)
k=1
iE|d' .itR
d' (0bb-(117771 + 13 + (d' 1)ra) + (d (b)obb4+17 + d'ra).
(A80)
=:s (d' bo41)
=:22 Id+t)
By Eqs. (A49), (A64) and (A73),
Σ
ba+1Ha--d5 + 1bj [d] + E Hisd
j=1
(disss .ifs
bat1r3 + 1 [d][r
Σ Y5
(A81)
iE(d"" .iンj
d' o(ba+1T3s + + (d' 1)ro)
(d d' d') (bbb--1rs + drrt).
(A82)
=1eg(df.b--))
=ioald bet1)
By Eqs. (A52), (A61), (A67) and (A76),
byt1 H 11j E [d] + 1 H +
Σ
Hish
j=1 kデj
veldy
d d
ΣΣ
ba+1r4 + 1[j E [d] + [[ E [d] +
T6
(A83)
j=1 kgj
VE(d) A
dd' (0bbbtrr's + + + is + (d' 2)ro) + (d - d' 00l battr' + 12 + (d' 1)ma)
Ede ba+1)
=saald' (aaadd
(d - d' d')olbu417a + To + (d' 1)r6) + (d d') - d' 300bbtirs + ddrr). (A84)
=iogld' bats)
Now,
d+1
d+1
Σ belliggi
= d''(ssldd' + (d- (d dd)d(s
+ d' Moss(d bat1))
j=1 Am
+ (d d')o(sa(d' ba+1)) + d' (d' 1)d(ss(dd (d',ba+1)) bat1))
+ d' d' d')o(sa(d' ba+1)) + d' d' ((s bat1))
+ (d - ('d d' ())))(s((dd,bbt)))
(A85)
=: score(d', ba+1).
(A86)
We shall now summarize the discussion Lemma E.2. The rest of the proof left to Lemma E.3.
Optimization of transformed problem.
Lemma E.2. Let (xx) max(0, x), d E x =1-c, and c. := A/2 In addition, for d' E {O..... d} and ba-i E (0.1),
X²
a1 = c +
3 (d l)c_
(A87)
30

---

Published as a conference paper at ICLR 2026
1
X²
22 = 6-
E +
(d 2)e-,
(A88)
3
X
33 = 4 +
(d l)c_,
(A89)
2
X
14 = c_
4
(d 2)e_1
(A90)
A
1²
15 = E + -6_ +
(d 2)
(A91)
2
4
1 1 c +
X2
X²
Te = 2 2
+
c++
(d (d-33)m 3)
(A92)
4
4
= 4 + (d - l)c_
(A93)
s1 (641) bd+177 13 + (d' 1)r4:
(A94)
22 (6+1) Dat1T7 d'rs,
(A95)
so(d', (541)
DatIT3 + (d' - 1)rs:
(A96)
sa(d', bot1)
bat1l's + d'rs,
(A97)
s5(d', b.41)
ba+IT4 + + + (d' 2)re:
(A98)
so(d' bo+1) batIT4 + + (d' - 1)r6:
(A99)
s7(d", bb41) = bat174 + + (d' - 1)r6.
(A100)
ss(d' bbt1) = -ba4114 + d'r6.
(A101)
score(d', b(41) = d'ó(s) ba+1)) (d-dddd d') ba+ + d'olsald' ba+1))
+ (d - d')o(sa(d ba+1)) + (d' 1)o(ss(dd bat1))
+ d' - d' (oso" (',, + d' d' ((
+ (d - d'd d' (ds bat1)).
(A102)
Considering the following optimization problem:
d'E(0....d..bbb score(d', ba+1).
max
(A103)
Then, setting P.QE g(d+1)((++1) to
0a b
P
Q [AA 0st1), b 1
0
0
ba+1), (A104)
Ask
1[ba+177 + 1|k < d' (d' 1Kk < d'Jrr 2 0]
(j d==++1)
1[ba+1r3 + 1 S d']r (d' - 11 < d')rrs 2 0]
()
1[ba + 1Ij 1 +d]2+ s d'Jrs (d' 1|j < d') 1 S d'Irre 2 oj
Aj
(A105)
the global maximizer of (A103) is the global minimizer of (7).
Proof. See the above discussion.
Lemma E.3. The global maximizer of (A103) is as follows:
31

---

Published as a conference paper at ICLR 2026
(a) If
M(s( 2) + 4)
0 0
(A106)
2(\(d (ild-11-- +
then d' and bu41 = 1. This corresponds to la+1 and A ld+1,d-
(b) If
X(d 1) +
(A107)
2d
then d' and bu+1 = 1. This corresponds to b la+1 and A [L 0.]".
(c) If
3 2 2 A
(A108)
2
2
22 1) 3
then d' = 0 and ba-1 = 0. This corresponds to = 1d+1 and A = 0s+1,d.
Proof. For notational simplicity, we abbreviate terms including variables such æı, #2y... (e.g.. of + 32z +...) using the notation @ (2, In particular, when the expression is strictly O (21,
nonnegative (e.g., 2 = or nonpositive, These terms are not essential to the analysis and too long. They can be derived by simple basic arithmetic operations. These concrete values can be showed by our python codes.
O_ __ 22,-.. .... respectively.
We define E,,..., as
X 3
2 - X
n
22 1) +3 E1
(A109)
2
12 - X(A2(d ( 2) +)
62
(A110)
1)) +3) 63
13 c= -2 1) +4
(A111)
2(A(d - 1) +2)
X(A(d - 2) + +
11 6
61,
(A112)
2(A(d - 1) +2)
X2 - 2) + +4
13s = 0 0 2(A(d- - 2) +4)
6:
(A113)
TB = - 6- X(A(d -) 3) + 6)
2(A(d - 2) + +4)
66:
(A114)
17 = ( A(d - 1) +2
6.
(A115)
2d
X 3dd= SdX² 24d + 4X2
34A + 48
(d( -
18A + 6
(A116)
2 2 5d + 18d1 + 21
Since
X(d 1) A)(3 22)
9 C 2(A(d - 1) + () - 1)) + 3) 20,
(A117)
(2 A)2
63 65
(A(d - 2) + ()(d
1) + 2) 20,
(A118)
(d (2 2)2
65 2d(A(d - 2) 44) 2 0,
(A119)
(2- (-3x + 6dd + 2X² 18A + 24)
67 6.6 2d(3 5d + 18d1 + 222 18A + 24)
20,
(A120)
X2(2-X)
E3
2 0,
(A121)
(A(d - 1) 2) - -5x= + 18d1 + 2X 18A 18A+ 24)
32

---

Published as a conference paper at ICLR 2026
X(2 2)2
c4 66 = 2(A(d - 2) + 4)(A(d - 1) + 2 0,
(A122)
X(3 2) - 1)(1-
66 62 =
20,
(A123)
2(A(d 2) + 4) - + 3)
for d 2, they are ordered as
62 2 co E S cs S E3 c1.
(A124)
In score, b8+1 appears as 1T4,73, 144174, 14. or 174177, each with a positive coefficient in d and d'. Thus, if ra, T4, r7 S O, then ba+1 should be zero. If ra, 14, 0, then ba-1 should be one. Considering Ineq. (A124), for d 2 2, the optimal bo+1 is one if S c and zero c cs.
Din-Dimessional Case. dd=
score(d', bo+1)
1d' ()(((ba+177) + ((bs+1ra)) + 1[d = 1|(0(ba+177 r3) + @(ba+1r3 +ri)) (A125)
11 (((bbu+)) + ((ba-1c+)
+ L[d' = 1](0(be+14+ + ++) + o (bbd-14+ +6+)).
(A126)
As e., is always positive for s E < 1, = d = 1 and ba+1 = are the optimal. This aligns with the following case analysis.
Weak Adversarial (Case 1). Assume d 2 2 and 0 S E S Eg. As S cg S Eg. bo+1 = 1
is the optimal. By Ineq. (A124), T1, T3, T3, T6. 2 0. The sign of 12 depends on Thus,
81 (d', 1), 82 (dd, sa(",, 1), sa('', 1), 87(d', 1), ss 1) 2 0 for s d' s d. In addition, for
d' 2 2,
as 1) 2 +22
(A127)
13
- (d- ( 2) + (3d 2) +
X²
31
- 1) 1)+ 12)
(A128)
6
12
2
X² - (7(2-))(5-3)) 21)
(::e5 E6)
(A129)
12(A(d- 22) +
20.
(A130)
Thus, d'(d' - s)ss 1) is nonnegative for 0 < d' < d. Similarly, by sa(d', 1) 2 r4 t 2 0 for
d' 2 d'(d' - 1) (6(('' 1) is nonnegative for S d' 2 d. Thus,
score(d', 1) d'si (d- d')2
d' s')sa(( (d' 1) + d''d'' - ((d (d' 1) + d' (d d')s7 (d', 1)
d'sz
+ -dd=
(A131)
= dr + d'r3 + d' 1)r4 dr3 + d'r; + d'(d= - 1)rs
+ dr, + + + d'rs + d'(d - () 2)rG-
(A132)
This monotonically increases in d'. Therefore, d' = d is the optimal. By Lemma E.2, b = 18+1- In addition, from $1 (d, 1), ss(d, 1), ss (d. 1) 2 A 14+1,d-
Weak Adversarial (Case 2). Assume d 2 and 6 < c cs. Ase S c,, but1 = 1 is the optimal. By Ineq. (A124), T3, PA, Ts. F7 2 0 and r2, ro < 0. Thus, (dd" 1), s2 sa(d', 1), sald', 1) 2 0. In addition,
S6 (sd',, 1) 2 ss(d,1) 2 X²(2 A)
12(A(d - 1) 2) 20
(::5C))
(A133)
X(2 2)a
97 s7 1) 2 s7(d,1) 2 8(X(d - 1) 22
20
(a)
(A134)
Due to the following inequality, s3 (x (1) is always larger than s6(d', 1):
sy (d 1) - (d( = 24
13 (d 1d+1++
512
X +
c (>² (d (d + 2) + 12(1 - A))
(A135)
12
2
12
33

---

Published as a conference paper at ICLR 2026
1(3 -)) 6(A(d- - +4) +4)
(::2 ct)
(A136)
20.
(A137)
If sa ssdd, 20, 0,
d dscore(d', (d', 1) (2+ X(d - 1) 2dc (A2 - + + 18X(d 1) + 24)
2 0. (A138)
dd
24
We used
(2-
2 A(d 1 2de 2
X(d - +2 20 (5)
(A139)
If so (dd, S 0,800,8..8 1s(d 20.
dsscoredd'1) = O(d, d' A) - E (3dx2((d d')2 2d ) 61(2 - A1
-21 (d (
2
11
7d
+
d²
dd'
12
4
+ xd + d'(4 366 + +88)
(A140)
2(2 A)
2 ((, 24(A(d 1) + 2)
d' d'(9d' X(2 A) +6 41(3d+ + 7) + 24)
(:)
(A141)
2->)(d>> + dd(112 7A) - 11X2 30 +4)
(A142)
12(A(d - 22)
2 0.
(A143)
We used for 0 S d' < d.
d' (9dd' - +) + 1 47)
+7) + 24)
S ddi3dd22 - A) + 28A+200
(A144)
If so 1) S 0, ss(,,, 10 S.
dscore(d', 1) dd'
= =(d, 12 + +3dd(3 8 8dA( -- + 2 + +d 6)1+333
c (3d
((A X(A - 2 +6 12(d- 1)
(A145)
A(2 A) d' d-33xx 6dd 6X2 -201 12)
2 ((d 12(A(d- 1) + 2)
(::ESEe)
(A146)
A IV (2- A) - sdx² 24dA 2\ + 22 600 +M++0 48)
(:'d' S d)
(A147)
24(A(d- - 1) + 2)
0.
(A148)
From the above discussion, for any case, (sc, 88 2 0), (96 < 0 and 2 0), or (sp, $8 < 0). the derivative of score(d', 1) with respect to d' is nonnegative. Thus, d' d is the optimal. By
Lemma E.2, b = 14+1. In addition, from $y (d, 1), ss(d, 1), ss(d 2 0, A = 14+1,d-
Adversarial, Assume d > 2 and e ET. By Ineq. (A124), 11, 13.73 2 0, r7 = 0, and r2, T4. T6 S Thus, sa(d', ba+1), sa(" bat1) 2 0 and s2d',, ba+1), sodd
b))1), ss('', bat < O. Now,
(d - d' d2)(2 A)2
$2 (d', 1) = (d', 0) >
20 0 (:6=G).
(A149)
4d
Thus,
score(d', ba-1) d' (dd) 0) + (dd + (d- (dsdds ba+1)
+ d'd'' ()d(ss(dd bb+1)) (d d')o(sr(d' ba41))
(A150)
= d's 0) + d'r (d 1)d'r rs'rs dbat113
34

---

Published as a conference paper at ICLR 2026
+ d' (d' 1))((b++111))) 72 + (d' 2)ro)
+ d' (d d')bba++ira + + (d' 1)ro).
(A151)
Since r4 is nonpositive, this indicates that score changes by dr3 + d' (d 1)rs at least by switching bd+1 to one from zero. Moreover,
dr3 + d' ())4-1)r+* 2 (d (d-1)(d-d)( 1)²
20 (:'e= = 67).
(A152)
4d
Therefore, ba+1 = the optimal. From Ineq. (A124) and E = E7, 07(d', ba+1) s5(d'' bat1) 2 If so (d', s7((" 2 0,
d score 1) = O(d, d', A) - (+dd,
(A153)
dd'
= @(d, A) - (d,
(:'6=G)
(A 154)
20 (:'d' d' S ds.);
(A (515)
where
ss(d', 1) 2 0 d' 3d1² - 6d1 + 21² - 181 + 24 dos.
(A156)
6A(A 2)
When s5 1) < 0, s7 2 O, then dasooredd .1) 2 0 similarly holds. If 95 (d', 1), (d' S 0, decore(d' .1]
dd'
del 2 for d' _ 1. Comparing score(d', 1) with d' = 1 and d, we obtain score(d, 2 score(d 1, 1). In summary, d' d is the optimal. By Lemma E.2, 1441- In addition, from sa 1) 2 0, 81 (d, = 0, and (d, 1) < 0, A = [Ta 0.]
Strong Adversarial. Assume 81 (d'' ba41)..... (d', ba+1) are nonpositive. Therefore, d' and ba+1 = 0 are the optimal. By Lemma E.2, b 0 and 0s+1,d
and 2 61. By Ineq. (A124), rıy..., 17 nonpositive. Thus,
L
F PROOF OF THEOREMS 3.5 AND 3.6 (ROBUSTNESS)
For notational convenience, occasionally describe representations and equations under the assumption that Srub := (1...., dzob), Svul == {dzob + n.....droh. + dyul), and Siee = {drob dyul 1,.... drob + dyul d. This assumption is made without loss of generality.
We use uniform big-O and -Theta notation. Denote f(x) = (g((z)) there exists a positive constant C> O such that [x)] <
Clg(2)) for every in the domain. Denote f(x) (gg(=)) if there exist C1, C2 > such that C (1)g()) & [f(z) & C21g(z)) for every x in the domain.
For notational simplicity, abbreviate the following matrix:
Cia Cza
Caros a C1
C, G Ciyy
(A157)
Cdiortiraa
Cd
Theorem 3.5 (Standard pretraining case). There exist constant 0 and strictly positive function g( (drob. doul, dir: a, 3,,) such that
E.
min
yN+ils(ZA: pstd 1
1841,N+1
((me. Natii aa Die Se
35

---

Published as a conference paper at ICLR 2026
g(drobi drul, dir, a, 7,
{
Cldrab + )run
(drbb + dyul + dirrle }. (8)
Prodiction original data
Adversarial effect
Proof. Since 1. A Ia+i,d, and ZAMZA is positive semidefinite, every entry in b M A is nonnegative. Thus, solve the inner minimization
min Se
min >
t ZaMZ AWN++ (#N+I A)
(A158)
b ZA MAM AANNAA(NNINNNN -cl.).
(A159)
Using (ar, v) ~ Dic
1
E
Za A
N
IC Jear:
TE| [[(yae]
(A160)
18 [yv
]
E [yy]
[[ Elye))
[l(w=)) [
0a
+
(A161)
Blya
0,
0
Since the second term is positive semidefinite,
E
1
T
Z
1a+1
N
d+
E Elyxx [yyx |1
E[yar]]
El(yx E E[yæ)
(
Elyx))
= -1
+
1441
(A162)
B Blyv
0
E[yyx
|E [ya]]
EE [yyæ]]
2
1/41
14+1-
(A163)
TE Elyxx
Since every entry of Elyan
TE [ya]] and IE [[ya is nonnegative,
Elyæe [[[ye]]
E[yæ
E Representing Eib ZA JAMZAA/N [o(drob. drul, dier, a, B,7) g(dro drul, dier a, B, 7)
d+ Z ZaMZ]1sH1
2 I
1a+1 2 1.
(A164)
Elyx
using some positive function g(drob; dvvut, dirr: a, 3,7) > O, there exists positive constant C such that
T ZAMZ ALyNHIaNT ela)
N
g[drobi dyul: dirr a, B, v,
(Elyn+1*N+11 cld)
(A165)
[g(drobi dvul. din, a, B, r)
g(drob druts dirr a, B, () + dyul +
de)
(A166)
S g( (drob, drui, dues a, B, (C((ooob + drul (drob + drul + dir)e).
(A167)
Theorem 3.6 (Adversarial pretraining case). Suppose that Grob and Zvul defined in Assumption 3.2 are sufficiently small. There exist constants C1, C2 such that
min E ((. (oe.. Dr ||
UN+ UN+Iff(2, padv Jadi))++1N++1
2 Cidrobaa + 3 + (d 2 + d
Prediction for original data
36

---

Published as a conference paper at ICLR 2026
Co (chobaa + dru13 10( (dhoba + drul
dirr? VN
+dyr dhrr
N
1 72
se.
(9)
Adversurial effect
Proof.
This is the special case of the following theorem.
Theorem F.1 (General case of Theorem 3.6). There exist constants C, C, C"
such that
E (( Dm
min |e
UN UN+ of+ilf(Za: pady Met
2 C(droba dvul Carob (qrob) d 2 + (1 (qvul)dvul + ('rrobaa² dduu
droba dral + + dim V Vdrr + 72 }1) (A168)
dirr?
-C" (dropa drui 1)
VN.
where
(maxies,.uus.... (i C2,2)
(A169)
min C
In particular, if there exists a constant C" > 0 such that - crrob 2 C and eqvul 2 C then there exist constants C1, C2 >> 0 such that Ineq. (9) holds.
Proof. Similarly to Eq. (A33), can solve the minimization as min A So
yN
min bl ZAMZ AYN+1 AUN A)
(A170)
| Sc
N
N ZAMZI AANZAAYNIONNHI
1 ZAMZAA
(A171)
By Eq. (A161), we can rearrange the first term as
E
b
ZUMZZ
N
B Blya LE lyæ Elyæ ']
1
E Blgh
1 E Blyx))(va Blyæ:)
(A172)
The first term of Eq. (A172) can be rearranged as
E yaa IE lyze
1
B[yN+1œN41]
Elya:
C.C,a² C.C,aBB
C,C,a8
0
Cla
C,C,
1T
0
C222
21
C
(A173)
0
Cja
C.8
Σ Cia+ +
ECB+11 +1
CaB+
Σ cia²
Σ
C2
(A174)
HiEStbh
iESval
VEESobb
iESual
min iES
)
(droba + drul. (d + d
c,a²+ Σ
c33².
(A175)
SES
1E5...
Consider the second term of Eq. (A172). Now,
[B((y Elyar:))(rr, Blya,)))]
37

---

Published as a conference paper at ICLR 2026
VGav O220² Von v G22 C1av C3,208
(i,j E Stob)
[i,j Svul)
(A176)
Stob j Svul) V (i E Syul A E Stob)
Let
8 E Stob U
Bi Elyr, - E[yx,11] < Elyx; {))
(A177)
JES,oLUS.al
The second term of Eq. (A172) can be computed
1 [B[(v - B[yar) Elya)) JE
v C2,2a Ejeshoh V222a Ejesual V C3,2
Trobdrab
Viiaa Ejesroh (Oipa(EiESnn Vaaa + Ejesuul v C228)
Ca C
0
(A178)
VCi2ßl (EjEsEnesion VC2200 + Ejesmn v VC,23
gvuldvul
( Ejesoo VC + Ejesmn VC228)
Voig
Σ Va E VC228
LiES
iESvul
Σ
CipCipa
Σ
CivC1
(A179)
iESTann
iESwinS
max (E
VC222
max iE(SrobUSS,.)
C, Vi.2
x (droh + ( (vuldvut
(A180)
max KESyabUSSnl
C,
max (ESrob USyul
C2,2 (droboa + (ral)) + (
(A181)
By Lemma F.2, we can compute the second term as
DUZZAZZA ]]
1
E
= =0 (droba + drui8 1) draba drmn + dirr
dirr
+ 11)))) N
+ dir
(A182)
Finally,
ZAMZIA A |]]
E
T ZAMZ AyN+1 - E.F
bT
C₂ (ddroba + d 3) + d
Σ cta²+ Σ CP
min USv
iES
iES
c:)(
(
C2.2 (droba + dval))(roobhobaa + qvuldyyul
max (ESrahh
max iEStah
+ O (drober + drul3 + 1) (hoobrr + drul. + dirry
(V / +1 -1)p2)
+ due
(A183)
VN
38

---

Published as a conference paper at ICLR 2026
Lemma F.2.1 If (1).,.)......
(ZN, uN) are ii.d. and follow Die, then
1 b ZAM
I8
(droba dru1B + 1) droba druiß + dirr
VN N +1 )3²)
(verr
dirr
+ dir
(A184)
where = 14+1 and A = L Og].
Proof. We can rearrange the given expectation
1 N
b ZaMZA4
A
1.]
E
E m
ΣN
Bn
La
(A185)
In T
0,
n=
N
In x n
(A186)
d+1
m=1 Un x
N
E
Yn
+
In.
Fn,
(A187)
N
i=1
By the Lyapunov inequality, for N + i...d. random variables X, 51,11.... XN,
N
N
27
N X
Xn
+
E[X|
(A188)
N
N
N
n==
Thus, using (xt, y) ~ Die
d
Un
Ind
In,i
N
j=1
N
E
y
f
E
y
33
02
(A (A189)
N
1=1
1=1
From Lemma F.3, we can compute the second term of using
E
y
+
Σ
Iz
E[yz.] +
E [zy,,
(A190)
==1
1=1
O(a(droba + d 1))
(i E Stob)
((3(rroba + drui + 1))
(i E Scut)
(A191)
0(}))
(i E Sim)
From Lemma F.3, compute the first term of using
2
E
y
2
E[x2 +
El Elyyyaj
Σ Blaynan7] J,K=1
(A192)
1=1
O022(( ((drable d + 1)2 + d
(i E Srob)
( + d 3 + 1)2 + 2 21)
(i E Smn)
(A193)
(drabb) + 3 1)2 + d 21)
(i E Sm))
39

---

Published as a conference paper at ICLR 2026
Thus,
2
N
E
2/
+
3ij
+
E
y +
32
32
N
j=1
j=1
=0 drob
a(drobba + drul3 +1)+ V
dier
N am
dur
+ dvul Ildroba dvu13 + 1) + V N
B7
+ dirr (+++)))
VN 3/ (choona + d + 1) v /dnr)))
(A194)
= =0 (droba + dyulß + 1) droba + d + dirry
(v V +1
dier
+ dire
(A195)
VN
Lemma F33. If (x, v) Dio, then
(a)
O
(i,j E Stor)
O(32)
(i,j Svul)
O
(i=j) =]) A Sir))
(A196)
0(a8)
( Slob AjE e Svul) V (iE Sval Aj E Srob)
(i+j) (i Sir V Sim)
(b)
(aa))
(i, Stab)
(
(i,j Svul)
Oaa²))
(EE Sroh A E Svull
E Myzz=au
OaaB²))
(EE Syul Aj E Scub)
(A197)
S Aj Srob)
(BB2)
(1 ( S AJ Syul)
0
(j E Sire)
(c)
B E[zzzaa11
(aat)
ki.j,k E S
(
(i,j E Svul)
O
(j (j=k)
O(a²8)
(i StOb) A {(j E Slob Ak E Svul) V (j Syul AR Syob)}
0(a:5%)
Srul) A {(j E Srob A E Svul) V (j E Srul A E Stob)] (A198)
O(a289)
(i SlOb A j,K SSul) V (i Smul ^j Srob)
O(a2??)
(1 (1 Aj,k Srob) V =j= dir A Stob)
0(327))
(i Smn Aj,KE Svul) v(j
= =k din Ai E Svul) (Em) Mil E E Svu) V (j Syal A & Stob))
0aaB73)
0
(i Sin V Sm))
40

---

Published as a conference paper at ICLR 2026
Proof. We first note that
O((a)
(i Stob)
[|a [[[(xx)))]
Bl(vx; Blyx,,)) Elyr:]
O(33)
E Sval)
(A199)
(i E Sur)
B[ya?]]
[(((r);)) )²]
(A200)
El(x2;
Blyx,11)1 (1V() ( - 2E yyze)
(A201)
0(a9)
E Srub)
O(3%)
(i E Svul)
(A202)
E Sirr)
B|a]]
B[
B1 41[yya]]Elyrr 61E [1yrr:1] 3Eyyr
(A203)
O(aa))
(i E Stob)
O(8))
(i Svul)
(A204)
((7))
E (er))
(a) For (i # 5) A (i Sir j E Sin). B[x,,,1 = [[,,][[r;] 0. Using the Cauthy- Schwarz inequality,
B[ryate < V B|/3]| Biaa]]
(A205)
O((2)
(i,j E Stoh)
0(32)
(1,j E Svul)
(A206)
0(7²)
(h,j E Sim) A (i=
((a))
(1 Srob Aj E Soal v (i E Sout Aj E Srob)
(b) For Sirr, Elyzy=ā] E[y]E[+]] = 0. 0. For S + Elyxzaf] E[r, [[yx)]
0. Using the Cauthy-Schwarz inequality.
O(a²)
(i,j E Spob)
O(89)
(i,j E Svul)
0(a?8)
(l Slob Aj E Svul)
E[ya,a$] S /Brrg] B[a:]]
(A207)
O
(1E Aj E Scob)
O
(iE SHr Aj E Srob)
O
(i SHZE AJ E Sval)
(c) For (j **)) k)) A (j e Smn k E Sun). E|
0. For j k. using the Cauthy-Schward
inequality,
0679)
(j = k) A (i.j,k Eim)
E E[
E[ayfl E[2:]|
O
(j k) (j, k dirr A E Stob)
(A208)
O(B8)2)
(j A E dirr i E Svul)
Using the Cauthy-Schwaad inequality,
E[a3]]
Elr2ly [[a:]]
(A209)
O(a4)
(i, k E Stob)
O(84)
(i,j E Syul)
O(a)3)
(E Slob) A {(j E Slob A A Svul) V (j AkI Srob)}
O(a,83)
(iE Svul) A ((j E Stob A KE Svul) V (j E Svul A k E Stob)}
(A210)
O(a23²)
(EE A.jk E Svul) V Vii Svul A j,k E Syob)
O(a2??)
(i SH ^j,k E Stob)
0(822))
S Aj,k E Svul) (EE)) A {(j E Slob A Svu) V (j Soul A Stob)]
O(a87?)
41

---

Published as a conference paper at ICLR 2026
G PROOF OF THEOREM 3.7 (TRADE-OFF)
Theorem 3.7 (Accuracy-robuutness trade-off). Assume drob 1, dvul = 1, and d 0. addition to Assumption 3.2, for (æx Dee, suppose that yz; takes with probability > 0.5 and -a with probability 1 - p for Stob. Moreover, yæ, takes 3 with probability one for i E Svul Define (().Q) = E ((...,))) "i INN+ P, Q)a++1441l- Then, there exist strictly
positive functions gn (d, a 3) and (2(d, ot, (3 such that
f(padd Qaid) (g, a, () + (d o 1))) (w.p- p)
(10)
Lgi a, () (d ( 1)3) (w.p. 1 1-p) p)
1)a² + (d - 1)0²} (w.p. 1 p). f(pady Qudy s a, 8)
(11)
Proof. Using and A defined in Appendix can rearrange f(P,Q) as
f(P.Q Q) = (( INN+I P. P.)))1-1,N+1
(A211)
N b E ((C ZoMZo AUN+1
(A212)
Standard Transformer. Similarly to the proof of Theorem 3.5, using some positive function g(d, a, 3) > 0, we can represent E[b" Zo Zo A/N| lø(d, a, g(d, a, 8)}, Thus,
g(d, a, 3)
1
bE N [[mn.8n)) ZoM AyN+1
|
UN+1
(A213)
ggld 3))
g(d, 33yN+1
IN+L5
(A214)
a (d - 1)3
(w.p. p)
(A215)
a + (d - 1)8
(w.p- 1 - p)
Adversarially Trained Transformer. Now,
E
(( ZoM Zo!
N
(B(x) }]
[[yæ]
(A216)
Elyæ 1
1
a
(2p
1)a8
(2p
1)a3
(2p
1)a
(2p 1)a3
82
82
(2p 1)38
82
82
(A217)
(2p 1)a8
82
82
(2p 1)a
B
Thus,
Tafa + (d 1 1)(2p 1)88 (2p 1)} T
31(2p - 1)a (d-1)8++1)
b T
(1om)))))8_ Lo.M. A
(A218)
N
B1(2p - 1)a + (d 1) +1)
42

---

Published as a conference paper at ICLR 2026
Therefore,
N E ((or Zo. AyN+IENH1
Tafa + (d - 1) 1)3 + (2p- 1)}
UN+12N+1,1
8(12p 1)a + (d 1)3++
B
(A219)
B((2p- (d-1))++
B
(2p
((d-1) - - (d-1)(
(w.p. p)
(A220)
((2= + (d- (w.p. - p)
In particular,
aa + (d 1)(2p 1)3 1)) (d- 1)8 ((2p 1)a + (d- 1)3 + +1}
(d
= a)a (d- (d-1)88)
(A221)
where
C a 1122
(2p - 1) a + (d - 1) 1)3 (2p 1)
(A222)
(2p- 1)a (d-
(2p 1)a - (d- 1)3
2p
(A223)
H PROOF OF THEOREM H.1 (NEED FOR LARGER SAMPLE SIZB)
Theorem H.1 (Need for Larger Sample Size). Assume the same assumptions in Theorem 3.7. Then,
ERs+i.e4+ [f pstd Qatd) J4+1,N+1] > 0 (w.p. at least 1 - PPNN
(A224)
In addition, suppose that there exists a constant 0 < C < such that (d - 1)3 +1 < Co. Moreover, assume that N is an number. Then, as p 2 with p > 3. for 4 S N 2.
E ERg |UN+ [f (Zo: padv Qudddyy Jo+1,N1++++1] >
0.483
w.p. at most
< - -NNN
(A225)
VN
Proof. Using and A defined in Appendix calculate
E Bww P, ()4)))+L++1]== N
T ZoM AE
(A226)
Now,
Zo Mo
N
a²
N
8 N
N
$
N
Em= 2n
LEN
En=
In
1n
82
B
1=
2n,1
g2
82
8
(A227)
1= In
12
g²
/ 1
Em Dn.1
8
N
Standard Transformer. From the configuration of and A, all the entries of bT Zo Z are same. Since all the entries of E[yN+1æN+1) positive, with some positive function g(d, a, 3) >
N T ZoMZn AE AEiyn-1 = g(d, a, a) N TT ZoM 2 1a-1.
(A228)
43

---

Published as a conference paper at ICLR 2026
Now,
1
N d+1 Zo 1++1
2 N
Un + 2(d 1) 3 N
= -d 30 + 2( 1)3 + 1+ 1 +
Σ
Σ
Yn
(A229)
N
N
n=1
n=1
((d- - 1)8 172 + 2((d 1),3 + }}
nn
(A230)
N
[[(dd 1)8 11 aa²² + 2((d 1)3 1}
N
Σ
(a fn
(A231)
>0 (w.p. at least 1 - (1- > PN.).
(A232)
Adversarially Trained Transformer. Note that ElyN+1 [(2p - 1)ex 3 ... 3). Thus,
1 - N d+1 ZoM I:B(yn+1eN+11
3 = (2p 1)ar a² + (d - 1)
N
N
Σ
ynInn, +
Σ
FnI1
N
N
n=1
n=1
B N
+ (d- - 1)8
Σ
nn, + (d - 10)52 + B
(A233)
N n=1
= [(2 1)a + (d 1).²² {(d -1)3 + 1H1
+ [(pp 1)a((d - 1)3 13 + (d- 1
(A234)
This indicates ExsTi 1 (zo padv Qudvv ]J411N441] > 0 only if
(2p 1)as (d - 32 (d 1d +1
Σ yn.
(A235)
(2p 1) (d 1) + 1 + (d 1
Representing bn = a(2X, - 1) with Xn taking with probability p and with probability
1 p.
(2X,, -1) > - 1)a 8 (d-1)) (d--)
(2p -
(d- 1),32
N
N
(2p - (dd (d
X,
1 a (2p
(A236)
2
n=1
Let ~ B(N,P) p), where B(N, p) is the Binomial distribution. Consider the following probability:
N
] a (2p 1o
1 (2p 1)a² (d- (dd=
PY~B(N.))
Y
1
(A237)
2
When p + 1/2.
1 ((pp -
+ (d ((d
Py~B(ND)
Y>
2
(2p - 12p-1)a( (d - + +
N
(d - 1)3 +
+ PY~BB(N1/2)
Y
1
(A238)
2
N
- (1 - c))
< <PY~B(N112)
Y> 2
(A239)
44

---

Published as a conference paper at ICLR 2026
N - 1 Y
S BYYB(111
(A240)
2
From Ash (1990), for an integer < k < 2/2,
PY~B(N1/2) 1
exp ND
2
(A241)
8N
(1-
where is the Kullback-Leibler divergence. Substituting k
2/2 -1
N
PY~B(N112))
Y
2
1 1
N x122))
exp
-ND
(A242)
1 8N( } ( }}
2
exp -ND
1/2
N/
11 /))
(A243)
/2(1 N²0 ) N
2 N
Note that
D N 1 2 (
1
2
2
[1
[ + 2 N
2
In
1- N
+
1m
In
N
(A244)
For N 2 4,
1
exp -ND (
2 N 11/2) 0.483.
(A245)
22 N72)
In summary,
Y N
0.483
PY~B(N.1/2)
Y
2 -1 1
=
Py
PYN,(1112))
2 -1
< 1
(A246)
N
45


Reviews:
[{"review_id": "MeIZw87QgH", "reviewer": "Reviewer_SXX6", "summary": "The paper provides a theoretical investigation of the robustness of single-layer linear transformers to adversarial perturbations to query tokens when performing in-context classification.", "questions": "- Do authors have any comments or thoughts about whether universally robust multi-layer linear transformers, or softmax transformers, are realizable or not? I would in particular appreciate any insights that authors may have on the interplay between robustness and depth of the transformer. \n-", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Generally well written paper.\n- The core insight of the paper (theorem 3.6) -- that universally robust linear transformer (for very particular class of classification tasks) are realizable is moderately surprising and interesting imo.\n- Other main results of the paper (accuracy-robustness tradeoff, need for larger in-context datasets) are well presented though not that surprising.\n- The theoretical results are interpretable and provide some intuition about accuracy-robustness trade off.\n- The paper empirically validates its theoretical results.\n- Limitations of the paper are well-acknowledged.", "weaknesses": "Most of the weaknesses of the paper relate to the (narrow) assumptions authors make to make the theoretical results tractable. While typical of theory papers, these are nevertheless weaknesses as they limit the relevance of these results to practical contexts.\n- The paper only studies single-layer linear transformers. This is a major weakness of the paper; as all the results pertain to this narrow class of models, I am not certain whether there is something special about this class of models, or can we expect some variation of these results to hold for other types of transformers as well. Given the relative difficulty of theoretical investigation for other types of tansformers, including some sort of empirical investigatons of other transformer classes would be helpful.\n- The paper only considers a single type of perturbation (L-infty norm).\n- The setup used in the paper is somewhat artificial.\n- Theorem G.1 is not empirically validated (I guess this is because these transformers don't length generalize?)\n\nMinor:\n- In lines 393-394, I was initially confused by the current phrasing and would suggest rephrasing to \"For example, when α = 160/255 and β = 8/255, the standard model becomes vulnerable at dvul ≳ 20drob, whereas the adversarially pretrained model **becomes vulnerable at dvul ≳ 400drob**\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OKrLLvNyWk", "reviewer": "Reviewer_2dSm", "summary": "The authors demonstrate that adversarially pretraining on a mix of classification tasks can make a (single-layer, linear) transformer act as a “universally robust” in-context learner: once pretrained, it can adapt to new classification tasks from clean demos and stay robust to attacks on the query, without any downstream adversarial training. The theory says robustness comes from the model learning to attend to task-shared robust features.", "questions": "1. The author assumes clean demonstrations for the new task. How sensitive is the in-context robustness to label noise in the demonstrations? \n2. How much of the benefit comes from task diversity in adversarial pretraining vs. adversarial pretraining itself? In other words, if I adversarially pretrain on fewer tasks, does the cross-task robust ICL degrade smoothly or abruptly?\n3. It is known that non-robust feature can also transfer. Can the author provide an in-depth analysis of the learned transformer weight to clarify which component are robustly learned parameters and which is not.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. the paper is in general well written with clear notation and explanations \n2. The author provides a formal analysis that proves task-transferable robustness under a matched threat model. \n3. The author provides a good experiental setup with clear sample creation scheme.", "weaknesses": "1. The proofs are for single-layer linear transformers with matched adversaries; the paper text, as described, markets this as universally robust foundation models. For it to be an universally robust model, I would expect an transferability for different perturbation class and distribution shift setup. The whole story depends on the downstream adversary matching (or being close to) the one used in pretraining.\n\n2. Another key theoretical baseline missing is that the author did not compare the no-pretraining case. Can the author demonstrate an separation or difference in sample complexity between adversarial training + ICL over direct linear regression on the test distribution. And is there an way to provide both an upper and lower bound on the error. E.g. LR > Pretrain + ICL> Adv + ICL > Robust LR ? \n\n3. The explanation for the connection between theorem 3.5 and 3.6 is confusing. I do not observe an strong separation and see why adversarial pretraining leads to an strictly smaller robust error. \n\n4. The design of the training dataset is quite arbitrary and to me it directly encodes the robust feature and thus it is able provide generalize to the test dataset which contains more weaker signals. To demonstrate the universality, I think the author should compare the training and test dataset with general distributions.\n\nIn addition, the theory already says transfer happens when the new task uses the same robust features as the pretraining mix. That’s a strong precondition. The author should discuss the scenario where the training data covers an diverse set of robustness features. Otherwise, this distribution coverage assumption is too strong. Similar thing should be analyzed in the experient as well.\n\n5. As the author mentions, theorem 3.7 requires small N regime, the author should consider include an sample complexity bound that can be directly compared against standard ICL case as in [1].\n\n\n[1] Ruiqi Zhang, Spencer Frei, and Peter L. Bartlett. Trained transformers learn linear models in-context. JMLR, 25(49):1–55, 2024b.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P2A6N1058e", "reviewer": "Reviewer_rGBV", "summary": "In this paper, the authors theoretically demonstrate that single-layer linear transformers, after adversarial pre-training across multiple classification tasks, can robustly adapt to previously unseen classification tasks via in-context learning without requiring any additional training. The authors also conduct small-scale empirical experiments to provide preliminary evidence supporting their theoretical claims.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper is clearly written and easy to follow.\n- The topic is promising. If robustness can indeed be efficiently achieved during pre-training and transferred to downstream tasks, it would be highly meaningful.\n- The paper seems to provide solid and convincing theoretical analysis.\n- The authors also offer empirical results that provide a certain level of support for the proposed approach.", "weaknesses": "- The paper is limited to single-layer linear transformers, and it remains unclear whether the theoretical results can generalize to more realistic multi-layer non-linear transformers, which form the basis of modern foundation models.\n- The empirical evaluation is also restricted to single-layer linear transformers. I suggest conducting experiments on multi-layer non-linear transformers and on more complex datasets to better demonstrate the generalizability of the conclusions (not necessarily large-scale, but at least small-scale experiments on nonlinear multi-layer transformers would strengthen the claim.)\n- It would be helpful if the authors could discuss the practicality of achieving robustness during pre-training and transferring it to downstream tasks, compared to performing robust fine-tuning at the adaptation stage (for example [1]). \n\n[1] AutoLoRa: An Automated Robust Fine-Tuning Framework. ICLR 24", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "diyjFcoi7C", "reviewer": "Reviewer_oCaH", "summary": "This paper considers the adversarial robustness of pre-trained transformers using in-context learning. They derive a set of theoretical results for a single-layer \"linear\" transformer and show that adversarially trained transformers can be universally robust in-context learners. Their results improve the mathematical validity of previously empirically and intuitively understood concepts in adversarial robustness, such as the number of redundant, robust, and non-robust features, the effect of the strength of the adversarial perturbation, and the robustness-accuracy tradeoff. Finally, they briefly validate some of the theorems with simple experiments.", "questions": "- Lemma 3.3: This lemma seems integral to the rest of the derivations. However, the intuition behind the lemma and its connection to the following theorems are slightly difficult to grasp. Could the authors elaborate on this further and add the additional elaborations to the text?\n\n- Line 199: “Taylor’s law (Taylor, 1961), is observed in a wide range of natural datasets and distributions.” To improve the flow of reading, it would be good to mention that examples of this are provided further below in the text.\n\n- Section 3.2: “Warm-up” may be a misleading term, as it usually pertains to warm-up during training of a neural network. Perhaps it’s better to use another term or simply call the section “linear classifiers and oracle.”\n\n- Section 3.2: Although a reasonable assumption, the authors should mention that $w \\in [0,1]^d$ and provide a brief justification.\n\n- Line 288: “non-linearity and non-convexity in the trainable parameters P and Q”: parameters cannot be linear or non-linear, convex or non-convex; it is a property of functions or sets. Please elaborate or correct this notation.\n\n- Line 289: “High non-linearity” is a vague term; perhaps it is better to bundle all mentioned non-linearities as simply “non-linearity” as a boolean attribute.\n\n- Thrm 3.4: Shouldn’t the second case be “$0<\\epsilon \\leq$” instead of “$\\epsilon =$”? Additionally, there is a discontinuity between the thresholds in cases 2 & 3. Could the authors elaborate on why this is and what happens in the left-out values of $\\epsilon$?\n\nWhile I currently suggested \"Accept\", should these minor considerations be addressed, I believe the paper is worthy of a \"Strong accept\" score.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The theoretical results are significant in improving the mathematical understanding of adversarial robustness under norm-bounded perturbations.\n- The paper is organized and written well, and while the theory is dense, most of the main text is presented and discussed with clarity and intuition.", "weaknesses": "- The experiments are brief, and Table 2 is a somewhat expected result, regardless of the contributions of the paper. However, the brevity of the experiments is made up for by the theoretical results.\n- The proofs are far too long and tedious. I understand that proving the theorems in a more succinct manner is non-trivial, but this still makes verifying the proofs rather difficult. Regardless, at a cursory glance, the proofs seem correct and seem to follow sound logic.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)