Feature compression is the root cause of adversarial fragility in neural networks
Authors: Jingchao Gao
Ziqing Lu
Raghu Mudumbai
Xiaodong Wu
Jirong Yi
Myung Cho
Catherine Xu
Hui Xie
Weiyu Xu
Venue: ICLR 2026 Poster (2026)
Keywords: Adversarial learning
Deep neural network
Robust learning
Primary area: learning theory
TL;DR: Adversarial fragility of deep neural networks is due to feature compression.

Abstract: In this paper, we uniquely study the adversarial robustness of deep neural networks (NN) for classification tasks against that of optimal classifiers.  We look at the smallest magnitude of possible additive perturbations that can change a classifier's output.  We provide a matrix-theoretic explanation of the adversarial fragility of deep neural networks for classification. In particular, our theoretical results show that a neural network's adversarial robustness can degrade as the input dimension $d$ increases.  Analytically, we show that neural networks' adversarial robustness can be only $1/\sqrt{d}$ of the best possible adversarial robustness of optimal classifiers. Our theories match remarkably well with numerical experiments of practically trained NN, including NN for ImageNet images. The matrix-theoretic explanation is consistent with an earlier information-theoretic feature-compression-based explanation for the adversarial fragility of neural networks.

Published as a conference paper at ICLR 2026
FEATURE COMPRESSION IS THE ROOT CAUSE OF AD- VERSARIAL FRAGILITY IN NEURAL NETWORKS
Jingchao Gao, Ziqing Lu, Raghu Mudumbai, Xiaodong Wu, Jirong Yi, Myung Cho, Catherine Xu, Hui Xie, Weiyu Xu" Department of Electrical and Computer Engineering University of lowa
ABSTRACT
In this paper, we uniquely study the adversarial robustness of deep neural networks (NN) for classification tasks against that of optimal classifiers. We look the smallest magnitude of possible additive perturbations that can change a classifier's ' output. We provide maarix-thhereete explanation of the adversarial fragility of deep neural networks for classification. In particular, our theoretical results show that neural network's adversarial robustness can degrade as the input dimension d increases. Analytically, we show that neural networks' adversarial robustness can be only 1/vd of the best possible adversarial robustness of optimal classifiers. Our theories match remarkably well with numerical experiments of practically explana- trained NN, including NN for ImageNet images. The compure-oommeesinn--oo bbsed tion is consistent with an earlier information-theoreti explanation for the adversarial fragility of neural networks.
INTRODUCTION
Deep learning or NN based classifiers are known to offer high classification accuracy in many clas- sification tasks. Neural networks are known to have great power in fitting even unstructured data Zhang et al. (0117). However, it is also observed that deep learning based classifiers almost uni- versally suffer from adversarial frraiiiity and show poor robustness under adversarial perturbations Szegedy et al. (2014); Goodfellow et al. (2014). Specifically, when small amount of adversarial noise is added the signal input of deep learning classifier, its output can dramatically change from an accurate label to an inaccurate label, even though the input signal is barely changed accord- ing to human perceptions.
The reason for this fragility has remained mystery, though this question has been extensively re- searched see e.g. Akkharr Mian (2018); Yuan et al. (2017): HHuag et al. (2018): Wu et al. (2024); Wang et al. (2023) for surveys. This previous work has not yet resulted in a consensus on a theoreti- cal explanation for adversarial fragility; instead, we currently have multiple competing explanations such as (a) smoothness, quasi- lincariti of the decision boundary or size of gradients Goodfellow
ct al. 2014): Li & Spratling (203); Kanai et al. (2023); Eustratiadis et al. (2022) SSimon Gabriel
al. 2019 (b) lack of smoothness of the decision boundary in the form of high curvature Fawzi
et al. (016); Reza et al. (2023); (Singla et al. (2021), (c) closeness of the classification boundary to
the data manifold Tanay & Griffin (2016); Zeng et al. (2023): Xu et al. (2022), and (d) the existence of highly predictive, "non-robust" features llyas et al. (2019). However, there are recent works, for example, et al. (2023), which show that non-robust features from data are not enough to fully explain the adversarial fragility of NN based classifiers. While many defenses have been proposed based on these explanations Allen-Zhu & Li (2022), they "all end up broken without exception", e.g. see Zhhan et al. (2024); Bryniarski et al. (2022); Li et al. (2023). Closest to this work is an earlier Cooornation-theorett feature-compression hypothesis Xie et al. (2199).
So despite these efforts, there is still no clear consensus on theoretical understanding of the fun- damental reason for the adversarial fragility of neural network based classifiers [i et al. (2023). It might be tempting to explain the adversarial fragility of neural network based classifiers purely as the gap between the average-case performance (the performance of the classifier under random
"Jingchao and Ziqing are the co-first authors.

---

Published as a conference paper at ICLR 2026
average-case noise) and the worst-case performance (the performance of the classifier under well- crafted corst-case perturbation), for example through the linearity of the model Goodfellow et al. (014).. However, argue that this average-case case-versuse wersus-worst-coee worst-case gap cannot explain the dramatic fragility of NN based classifiers. Firstly, it is common that there is a gap between average-case and worst-case performance: it exists for almost every classifier (even including theoretically optimal classifiers), and is not particularly tied to NN based classifiers. Secondly, we can show that there ex- ist classifiers which have very good average-case performances, and their worst-case performances are provably orders of dimension better than the worst-case performances of NN based classifiers. So there are deeper reasons for the NN adversarial fragility than just attributing it to the worst-case- versus-average-con degradation.
In this paper, we study the adversarial robustness of NN classifiers from a different perspective than the current literature. We focus on comparing the worst-case performances of NN based classi- fiers and the worst-case performances of optimal classifiers. We look at the smallest magnitude of possible additive perturbations that change the output of the classification algorithm. We provide a matrix-theoretic explanation of the adversarial fragility of deep neural networks. In particular, theoretical results show that neural network's 's adversarial robustness can degrade the input dimension increases, compared to the worst-case performance of optimal classifiers. Analytically, of the best possible adversarial robustness.
show that NN' adversarial robustness can be only Va
To the best of our knowledge, this is the first theoretical result comparing the worst-case per- formance of NN based classifiers against the worst-case performance of optimal classifiers, and showing the ((vd) gap between them. In particular, through concrete classification examples and matrix-theoreti derivations, we show that the adversarial fragility of NN based classifiers comes from the fact that very often neural network only uses a subset (or compressed features math- ematically defined in Section (5) of all the features to perform the classification tasks. Thus in adversarial attacks, one just needs to add perturbations to change the small subsets of features used by the neural networks. This conclusion from matrix-theae theoretic analysis is consistent with the earlier information--hherrttic
cotiure-ccompession--aaaee hypothesis that neural network based classifier's fragility comes from its utilizing compressed features for final classification decisions [ et al.) (011). Different from Xie et al. (2019) which gave higher-level explanation based on the fea- ture compression hypothesis and high-dimensional geometric analysis, this paper gives the analysis of adversarial fragility building on concrete NN architectures and classification examples. Our re- sults are derived for linear networks (Section 2 and non-linear networks (Section 4. for two-layer and general multiple-layer neural networks with different assumptions on network weights, and for different classification tasks involving exponential numbers (in d) of data points (Section 33) As a byproduct, we developed a characterization of the distribution of the QR decomposition of the products of random Gaussian matrices in Lemma]
Due to unstructured data in datasets such as MNIST and ImageNet, this paper is to start with more structured data and classifier models which enable concrete theoretical analysis before extending results to general models and data (Section 5).
Related works In Vardi al. (2022), the authors showed that trained two-layer ReLU neural net- work can be fragile under adversarial attacks, compared to another robust NN classifier. Compared with Vardi et al. (2022), make new technical contributions in that: 1) conceptually,
propose
the "feature compression" explanation for the adversarial fragility; 2) our results work for a much larger number of datapoints (can grow exponentially in input dimension d), while Vardi et al. (2022) relies on the inner products between data points X, and x, being small and thus can only handle a small number of data points; 3) our paper compares NN classifiers against the optimal classifier in terms of tolerable perturbation size and give the O / /d) perturbation gap against the optimal classi- fier; while Vardi et al. (2022) compares between NN classifiers; 4) In fact, our paper (Theorem 5) handles much more challenging datapoints than Vardi et al. (2022), because we allow different dat- apoints to have inner products of magnitude d (or angle 180 degrees), while Vardi et al. (2022) does not apply to this. 5) Our results apply beyond binary classification and to a large number of labels (Theorem 1, Theorem 6 and discussion in Section 5 6) Our technical derivatioos//echaniit are different from Vardi et al. (2022): we rely on random matrix theory, while Vardi et al. (2022) builds on KKT conditions for stationary point of gradient flow. Compared with Daniely & Shacham (2020) and Bartlett et al. (202T). besides all the differences mentioned above, our predicted NN's tolerable perturbation is of scale Ö(1) = ((()x|/d)) where x E IRd is a vector, for our example in
2

---

Published as a conference paper at ICLR 2026
Theorem 6, and this bound is actually tighter (better) than the
O(vd) predicted by these two references. The experiments match well with our theory. More recently, Frei et al. (2023)
extended the adversarial fragility results of Vardi et al. (2022) to input data modeled by Gaussian mixture models: Melamed et al. (2023) proved the adversarial fragility of two-layer NN trained us- ing data from now-dimensional manifold. However, the results from |Frei et al. (2023) essentially still require the centers of the Gaussian mixture models to be nearly orthogonal, and Melamed et al. (2023) requires data be from low-dimensional manifold. By comparison, data points used in our theorems do not fit these conditions. Compared with [Iaya et al. (2019), our results show that it is not the "non-robust features" or "robust features" in training data that lead to the fragility or robust- ness, but instead it is the "feature compression" property of the NN classifier itself. For example, Li et al. (2023) showed that even using "robust features"' (sometimes they are even hard to extract), the trained neural networks are still highly fragile under stronger adversarial attacks, implying that adversarial fragility does not purely come from data. Moreover, in contrast to the belief in Myas) argue that those "non-robust features" are not necessarily non-robust: they may just be a compressed part of robust features. Thus the "non-robust features" may also be needed for building an adversarially robust classifiers. For example, this paper's Theorem [ that the utilized "non-robust" compressed feature (the orthogonal residue of a vector after projecting it onto a subspace) is just part of the robust feature: this compressed feature should not be cleaned out from data in building robust models. Compared with Simon-Gaat Gabriel et al. (2019), we attribute fragility more to the angle of gradient rather than its size.
et al. (2019).
Notations We denote the E norm of an vector x R" by ||x|| or |x||2 V x Let a NN based classifier G(-) : Id - I* be implemented through a l-layer NN which has -- hidden layers and has ( + 1 columns of neurons (including the neurons at the input layer and output layer). We denote the number of neurons at the inputs of layers 1, 2, ..., and l as n1, f2. and ni respectively. At the output of the output layer, the number of neurons is R1+1 = k, where & is the number of classes. We define the bias terms in each layer as 82 E 1R"2, 82 E RT',.... 81_1 E R" 8 E R= and the weight matrices Hf, for the i-th layer are of dimension R* xn. The element- wise activation functions in each layer are denoted by (-), and some commonly used activation functions include ReLU and leaky ReLU. So, the output y when the input is x is given by y = =((x)= = oH (Ht1 ... o (H + 81) ... + 81-1) + +,))
2 FEATURE COMPRESSION CAUSES SIGNIFICANT DEGRADATION IN ADVERSARIAL ROBUSTNESS
We first give a theoretical analysis of linear NN based classifiers' adversarial robustness and show that its worst-case performance can be worse than the worst-case performance of optimal classifiers in the order of input dimension d, in Theorems [ and 4 We then generalize our results analyze the worst-case performance of non-linear NN based classifiers for classification tasks with more complicatedly-disrricn disttribtdd data in Theorem and Section 5
For now, to demonstrate the concept of feature compression, we consider d training data points (x,; i), where = 1,2, ... d, each x; is a d-dimensional vector with each of its elements following the standard Gaussian distribution A (0, 1), and each i is a distinct label (namely d classes in total). We will later extend the number of training data points from d to be exponential in input dimension d.
Consider a two-layer (to be extended to multiple layers in later theorems) neural network whose hidden layer's output is = a O(Hix + 51), where H₁ E Rmmxd z E Rm and m is the number of hidden layer neurons. For each class i, suppose that the output in the output layer neuron of the neural network is given by fx) w. o H + ō1), where w, m By the softmax function, the probability for class i is given by 0; = Σ efi where & is the number of classes. In our results, "with high probability" means that the probability increases to '1' as the dimension increases.
efi
Theorem For each class i, suppose that the neural network satisfies the following, then
15(x,) = 1, i. j= =
(1)
0f i
3

---

Published as a conference paper at ICLR 2026
with high probability, for every >> 0, the smallest distance between any two data points is
min j=1.....d
|x4 x,ز2 2 (1- (1-c)/dd.
1=1.2...d
For each class i, one would need to add a perturbation e of size ||c||2 2 (1-c)v2d the classification decision if the mimimum-distanne classifier used.
to change
For each i, with high probability, one can add perturbation of size ||el|2 C such that the classification result of the neural network is changed, namely fz( + e) > fi(: + for certain + where C a constant independent of d and m.
We consider condition 1 because accurate classifier requires output fx(), achieve it maximum at the i-th output neuron, namely when j = i. Moreover, this assumption facilitates analysis and the corresponding assumptions and predictions match the numerical results of practi- cally trained NN as in the numerical result section. The *I' can be changed to any positive number.
Remarks 1:
Remarks 2: From this theorem and later theorems, we can see the optimal classifier can tolerate average-case perturbations of magnitude (d), and worst-case perturbations magnitude O(vd); while the NN classifier can tolerate average-case perturbations of magnitude O( ((v),, but can only tolerate worst-case perturbations of magnitude O(1). Both have average-case versus worst-case degradations, so the adversarial fragility does not come from those.
Proof. To prove the first claim, need the following lemma (proof provided in the appendix).
Lemma 2 Suppose that Zv, Zz and Za are iid. random variables following the standard Gaussian distribution (0,,),, Let a be a constant smaller than 1. Then the probability that EL_ 23 S ad is at most (a(e²=a)) Moreover, as a + 0, the natural logarithm of this proba- bility divided by d goes negative infinity.
For each pair of and Xjxx,-X will be d-dimensional vector with elements independent zero- mean Gaussian random variables with variance So by Lemma 2 we know with high probability that the distance between x, and x, will be at least (1 - e) By taking the union bound over ( pairs of vectors, we have proved the first claim.
We let X = [11, ..., Xa] be a Rdd matrix with its columns as x;'s. Without loss of generality, assume that the ground-truth signal is xa corresponding to label d.
We consider the QR decomposition of X as X = Q2R, where Q2 E Rdxd satisfies Q²Q²= = Iaxd and R Rdxd is upper-triangular matrix. We further consider the QR decomposi- tion of Hı RH here Q1 E R.xd satisfies QTQi Q1 Taxd, and Rn dxd dnd is an upper- triangular matrix. Because of condition T the weight matrix H2 between the hidden layer and the output layer is H2 Rt R-1Qu RH R Red = [Ri,, R2,d,... Ra-i,ds Ra,a)* When the NN input is X Q2li. the output
R-1 Take the last column of Rt namely
y H2H1xx R-1 R.d [0,0, ... 00
We let the adversarial perturbation be Ra-1.d, Ra Then the NN input is xa + and we will have a successful target attack such that fa(xa + = 0 and Ja (xa + e)
Q2b, where
(0, O, 1
In fact, when the input is X + c, the output at the d output neurons ý R-1 (R.d + b). We then notice that the inverse of R is an upper triangular matrix given by
0 Rs Rd-1.d-1
Rg
R
Ra.d
0
0
Ra.d
where we only explicitly express the last two rows.
So (Xa + e), fa(xx + e)) is equal to
4

---

Published as a conference paper at ICLR 2026
Ra- 1,4
( Ra-1a) + Ra- 1, (- (Raad) + Rad
Rd +
1.
Ru
of Badd
1g-1d-11
T1 1-
0
0
Rayd
The magnitude of this perturbation is
||||2 = IQ2bll2 = (Raa-18-1 Ra)1,a)2 Ra,a)² < |Ra-1a [Ra-ial r,a + (Raa).
(3)
By random matrix theory Hassibi & Vikalo 20055; Xu et al. (2004) Radd is the absolute value of random variable following the standard Gaussian distribution N [10, 1). Moreover, Ra-1.d-1 the square root of a random variable following the chi-squared distribution of degree 2; and Ra-1,a is standard normal random variable. Thus, there exists a constant C' such that, with high probability,
under an emror with |ell2 I C, the predicted label will be changed.
Note that X I (Q2).. where (Q2).. is the i-th column of Q2. Namely is composed of d features corresponding to the d columns of the Q2 matrix. An optimal classifier will use all these columns for decisions and is able tolerate adversarial perturbation of magnitude O(vd). However, to attack the NN classifier, we only need perturb the compressed feature direction (Q2).dd used in making decisions. This is the fundamental reason why this NN classifier shows O(vdl degradation in adversarial robustness compared with optimal classifiers. The proof also shows that adversarial fragility is there even when x is not Gaussian distributed, as long as the ratio between Ra.d and the f2 norm of Red is small.
Remarks:
Now we go beyond 2-layer neural networks, and consider x being generated from linear generative Gaussian models. For these Gaussian matrices, we obtain novel characterization of the distribution of the QR decomposition of their products (see Lemma 7 and its proof in the appendix).
Theorem Consider the setup in Theorem 7 but a multiple-layer linear neural network whose hid- den layers' output is z Hı. where H, Rm Xns, and ny d. For each class i, sup- pose that the output at the output layer neuron is given by f((x)= = w" z, where W; E m x1, For each class i, suppose that the neural network satisfies 47p. We assume that x; = G Givi, where G₁, G2- G2 are independent x normalized generator matrices with each element of each generator matrix being an independent zero-mean Gaussian variable with variance 3, and v; 's are independent vectors with the elements being independent standard unit-variance Gaussian random variables. Then
with high probability, for every >> 0, the smallest distance between any two data points is
min i$j. i=1,2....d j=1.2....
X x, (L-e)v2d.
For each class i, one would need to add perturbation e of size [|a||2 2 (1-c)v2d to change the classification decision if the minimun-distanee classification rule is used.
For each class i, with high probability, one add perturbation of size ||e||2 S C such that the classification result of the neural network is changed, namely f, (x; +e) > fx( + e) for a certain j f i, where C is constant independent of d.
So far we have assumed that for each class i, condition (1 holds. This condition facilitates char- acterizing the adversarial robustness of neural networks via undom-matrin maatixx-theoreee theoretic analysis. We now extend results to general NN weights which do not necessarily satisfy CI.. Moreover, the number of classes is not restricted to
Theorem Consider a multi-layer linear neural network for the classification problem in Theorem ! Suppose that the input signal x corresponds to ground-truth class i. Let us consider an attack target class j + Let the last layer's weight vectors for class and j be w; and w; respectively. Namely the output layer's outputs for class i and j are respectively:
f(()) ww H1- and fy(x) w/ H1-1..,1xx,
where H; E RTi/ and n, = We define two probing vectors (each of dimension d x 1) for class and class j as
probe; (wF Hi-1...Hi)? and probe; (w3 H H,)""
5

---

Published as a conference paper at ICLR 2026
Suppose have the following QR decomposition:
probee
i11
r12
where Q Rdx2, We the projections of x; and x5 onto the subspace spanned by the two columns of Q be xG and x, respectively. We assume that
041 011
[5, x,,}
02:2
0,2
If for some input x A. -)> 1i(x + 3), then we say that the perturbation A changes the label from class to class j. To change the predicted label from class to class j. we only need to add perturbation x on the subspace spanned by the two columns of Q, and the magnitude of A satisfies
Inia - (r120, + F220x2)) pprobe; probe,) ||
S Va 0.2
| S
From Theorem one just needs to change the components of in the subspace spanned by the two probing vectors. This explains the adversarial fragility from the feature compression perspective: one only needs to attack the compressed features used for classification to fool the classifiers.
FEATURE COMPRESSION WITH EXPONENTIALLY MANY DATA POINTS
In the following, we consider case (proof provided in the appendix) where the number of data points (2d-1) within a class is much larger than the dimension of the input data vector, and the data points of different classes are more complicatedly distributed than considered in previous theorems.
Theorem Consider 2d data points (X,; where 2,,,.... 2d X; E d is the input data, and is the label. For each i, we have x, Azs, where 2; is d x vector with each of its elements being +1 or and A is d d random matrix with each element following the standard Gaussian distribution (0, 1). The ground-truth label is +1 if z, (d) if (2(d) -. We C+1 denote the set of x; such that the corresponding 2, (or label) is +1, and let C denote the set of x; such that the corresponding 2, (d) (or label) is -1.
+1 (namely z, 's last element is
+ +1), and
Consider a multiple-layer linear neural network whose hidden layers' output is o H..I...Hx where H; E R 1xx; 1gn;-xxm,, n1 d and is the input. For each class C41 C- suppose that the two output neurons are
(x) wI10 and f-1(x) =
For input x,, suppose that the neural network satisfies
z. = +1, f=1( (.) and f-1(x;)
+1 = -1,
(4)
1, (i = +1.
Denote the last element of corresponding to the ground-truth input x, by 'bit'. Then
with high probability, there exists constant > O such that the smallest distance between any two data points in the two different classes is at least avd, namely minx,eC+1, x,EC-1 ||x; - xil12 2 avd.
Given a data x X., with high probability, one can add a perturbation e of size ||e/|2 D such that f-bee(x
> (x, + +), where D) is a constant independent of d.
As we can see in the proof, because the NN makes classification decisions based on the compressed features the direction of the vector Q.., namely the last column of Q (Q is from QR decomposi- tion of A), one successfully attack along the direction Q. using much smaller magnitude of perturbation. Using the results of QR decomposition for products of Gaussian matrices in Lemma 7 the proofs of Theorem and Theorem
extend Theorem 5o multiple-layer NN models.

---

Published as a conference paper at ICLR 2026
4 MULTIPLE-LAYER NON-LINEAR NEURAL NETWORKS
We extend results to general non-linear multiple-layer NN based classifiers, showing that one just needs to attack the input along the direction of "compression".
Consider multi-layer neural network for classification and an arbitrary point x E RRd From each class i, let the closest point in that class to x be denoted by x xxx We take >> as a small positive number. For each class i, We let the the NN based classifier's output at its output layer be fi(x), and we denote the gradient (w.rt. input x) of fx(x) by Vx((). We assume the first-order approximation error of f: i's in the neighborhood of is small relative to c, of order o(c).
Theorem 6
We consider the points x + CX1 and x + cX2. Suppose that the input to the classifier is x + cX1 Then we can add perturbation to eX1 such that (dot equality in the sense of furst-oddr approximation of e, namely o(c))
fi + EX1 + ee = fi(x + Ex2) and f2(x + EX1 + e) = f2(x EX2).
Moreover, the magnitude of satisfies ||e||2 S (1) (x1 x2)1l2, where (1110)) is the projection onto the subspace spanned by (x) and f2(x).
Suppose that the input to the NN classifier is + ex1. If the elements of (xx),, f2(x), and X2 - are independent standard Gaussian random variables, then with high probability we can change the elassifier's 's neurons' output values to those values produced by an NN input x + cX2, using an adversarial perturbation whose magnitude is only s(Vd) of e/ell22 (X|x2-Xx1] xi
In Theorem 6 only, notationwise, we use X1 and X2 as increments rather than data points from two classes. In order to make the classifier wrongly think the input is x + CX2 instead of the true signal x + cx, at the corresponding two output neurons, we only need to add a small perturbation instead of adding a full perturbation exx² - x ). due to compression of X2 Xx along the directions of gradients V (x) and /x()x). Similary, we can use a small perturbation to + ex1 such that f2 + ex1 + e) - fi(x + ex1 + e) = f2(x + ex2) f1 + ex2), with small magnitude x2)l12, where (V()(x) (x) fa(x)) the projection onto the subspace spanned by fxi (x). 2(-) denotes the asymptotic lower bound.
Remarks:
||e|22 S [+
From Theorem 6|s proof in the appendix, in order for the NN to have good adversarial robustness locally around x, the direction of X2 x1 should be in the span of the gradients (x) and V (xx However, the subspace spanned by V fi (x) and V (x() may only contain "compressed parts of
E(x² x1). leading to adversarial fragility.
5 COMPRESSION RATIO LEADS TO THE ADVERSARIAL FRAGILITY: A SIMPLE
ALGEBRAIC EXPLANATION
In this section, we consider general non-linear NN based classifiers for general classification tasks, and extend results from the local analysis around input x in last section to "global" analysis.
Let us consider classifier with multiple classes. Let us focus on a data point x2 belonging to Class 2, and assume that the closest data point (in l norm) belonging to a different class X1. Assume x1 belongs to Class 1. Let us consider the direct path from the input x2 to X1, and the function g(x) = f2(x) - (x). where fz()) and f.)) are, respectively, the corresponding neuron outputs for Class 2 and Class 1. By following this path, g(x) goes from g(x2) to g(x1), and the change experienced by g(x) is thus D = g(xi) - (x2)). The length of this path is || X x2ll and we parameterize this path by the length parameter 0 S 7 S ||x] x2ll. going from X2 to X1.
Following this path, we can write D in another way:
|xx xx
D
xx))) cos( d7,
(5)
xx X2 is a point along the path, V means the gradient of the function, and 8, is the angle between the gradient of g(x) (at the point X,,x,,22 ) and the direction X1 X2.
where Xy,11x22 -x,-xa1 X1 +
Suppose that the adversarial attack uses an infinitely small step size. The adversarial attack starts with the point x = X2 and in each step (iteration). goes in the negative direction of the gradient of gx)). We assume that at the end of the attack, the change in g(x) is also D. Let the length of
7

---

Published as a conference paper at ICLR 2026
the path the adversarial attack follows be 2 and parameterize the path by the length parameter 0 I Y S 2, where y means the length of the path followed by the adversarial attack so far. Then from the perspective of the adversarial attackers, D can also be written in another way:
D = % - -IVo(xx))l
(6)
where x, is the point at which the path length the attacker has traveled so far is ry.
Notice that both (5) and (6) lead to change D in g(x). However, because (5) has this compression term (0,) (often small in absolute value, sometimes even negative), the direct path's length ||x] X₂ || needs to be much bigger than the path length followed by the adversarial attack (as- suming that the magnitudes of the gradients of (xx) are roughly the same at locations of interest).
We further notice that ||x- x1 is the minimum perturbation needed for changing the optimal classifier's 's result from Class to Class 1. But as explained, ||x: - X1 |l needs to be much bigger than the adversarial attacker's perturbation magnitude 2. This feature compression factor (f,)) explains the experimentally observed phenomenon that the adversarial attack on the NN classifier can have an attack of a much smaller magnitude than that required by the optimal classifier. We can also see that this adversarial fragility comes from the NN classifier's Vg(x)* compression (namely cos(0,)) (0))) )) of the "good" feature X1 X2. The key in this analysis is that we analyze the adversar- ial attack performance with respect to the NN classifier's s feature compression along the direction looked at by the optimal classifier, not comparing the worst-case adversarial perturbation against the average-case random perturbation. From this analysis, we can see feature compression is also a necessary condition for adversarial fragility to happen, if the magnitudes of the NN classifier's s gradients do not experience dramatic changes within regions of interest. Please see the appendix for an illustration of this feature compression explanation.
NUMERICAL. RESULTS
We present numerical results verifying theoretical predictions on advensarial fragility. In particular, we focus on the setting described in Theorem (linear networks) and general non-linear networks.
Linear networks: Consider the model of data in Theorem 5 Let X be d x 2d matrix where each column of X represents an input data vector of dimension d. Next, we train linear neural network with one hidden layer for classification. The input layer of the neural network has dimension d, the hidden layer has 3000 neurons, and the output layer of dimension 2. We denote the 3000 d weight matrix between the input layer and the hidden layer as H1, and the weight matrix between the hidden layer and the output layer as 2 x 3000 matrix H2. We use the identity activation function and we use the Adam package in PyTorch for training. The loss function we use in the training process is the Cross-Entropy loss function. The number of epochs is 20.
For d = 12, each "run" starts by generating a random matrix A, and the data matrix X accordingly. In generating the data matrix X, we multiply each of A's columns by 5 except for the last column (Note that this modification will not change the theoretical predictions in Theorem 5 This is be- cause the modification will not change the last column of matrix in the QR decomposition of A). Then we train a neural network as described above. We repeat training until 10 valid runs (training accuracy is 1) are collected. Then, in Table 1 we report the results of the 10 valid "experiments".
Let W1 and W2 be the first and second row of W = H H1, respectively. Note that W1 and W₂ are the two probing vectors mentioned in Theorem 4 For each valid "experiment", we consider two different angles, 0, and 0z. 0, is the angle between Wi - W2 and the last column of A. The physical meaning of cos(0,) is how much of the feature (the last column of A) is projected (or compressed) onto W, W2 in the neural network make classification decisions. By similar derivation as in Theorem 4 cos(6,)) quantifies how much perturbation we can add to the input signal such that the output of the classifier is changed to the opposite label. For example, when | cos( O. 1910 in Experiment 1 of Table ! we only need perturbation 0.1970 of the & magnitude of the last column of A (perturbation is added to the input of the neural network) to change the output of this neural network to the opposite label. On the other hand, the optimal decoder (the minimum distance decoder or classifier) would need the input to be changed by at least the & magnitude of the last column of A so that the output of the optimal decoder is changed to the opposite label. The second angle fg is the angle between the first row (namely W, ) of W = H2H, and the last row of the inverse of A. As modeled in Theorem 5 W, should be aligned oppositely aligned with the last row of
8

---

Published as a conference paper at ICLR 2026
Table 1: Cosine of angles of trained models with training accuracy equal to 1, d 12.
Experiment No.
3
7
8
10
cos(0)
0.1970
0.1907
-0.0017
-0.2119
-22409
-0.5054
-0.7796
-0.9865
-0.065
-0 -0.4739
0.3992
03902
0.988
03988
0.390
03088
00079
0.9972
035000
01902
61870
0.5888
02008
0.085
000788
Table 2: Averages of cosines of angles, for cos((22) > 0.9, d 12
Avg. of cos((0))
Avg. of [o]
Avg. of [ooa(0))) - 1oi
0.36-15
0,3280
0.0367
the inverse of A, and thus [os()22] should be close to We also consider the quantity "fraction" 6, which is the ratio of the absolute value of R,,dd over the lg magnitude of the last column of A. Theorem 5 theoretically predicts that cos(O)) | (or the actual feature compression ratio) should be close to "fraction" (the theoretical feature compression ratio).
From Table for Experiment 7), one can see that using Theorem the actual compression of the feature vector (A's last column) onto the probing vectors (W1 W2) and "fraction" ø (the theoretical compression ratio) accurately predict the adversarial fragility. For example, let us look at Experiment 9. The quantity of ø is 0. 1480, and thus Theorem [5] predicts the adversarial robust- ness (namely smallest magnitude of perturbation to change classification result) of the theoretically- assumed NN model is only 048480 of the best possible adversarial robustness offered by the optimal classifier. In fact, by the actual computationally trained NN experiment, 0.148 4880 is indeed very close to coss(8 (=0)
which is the size of actual perturbation (relative to the & magnitude of A's last classifier's decision to the opposite label. We see
column) needed to change the practically-traing that when the theoretically predicted compression ratio is small, the actual adversarial robustness quantified by cos(01) )l is also very small, experimentally validating Theorem 5 purely theoretical predictions. We also notice cos(f2) is very close matching the prediction of Theorem
We further conduct 50 experiments and see that there are 20 experiments with training accuracy 1. Among all these 20 experiments with training accuracy 1. we noticed that there are 18 cases with the absolute value of cos(02) (2) over 0.9. Furthermore, for these 18 experiments, we report 3 statistical values Table From Table 2 the average value of cos(01) [)] 0.3645. means that on average, need 0.3645 of the l2 magnitude of the last column of A to be added to the input signal such that the output of the classifier changed to the opposite label. Moreover, we can conclude from Table 2 that on average, |o| is 0.3280. It represents that the theoretically-predieee
compression ratio needed to change the classifier output is on average 0.3280. We also observe that the average value of (oss())) ) - | is 0.0367, meaning the actual result is close
theoretical analysis. For higher dimension inputs up to d = 7, 8, 9, 10, 12, 14, 15, 16, 17, the averaged measurements of the "fraction" e (the theoretical predicted robustness) and the compressed feature cos(O) ) (the empirical robustness of the trained neural network) are displayed in Figure [
They match really well.
Adversarial attack analysis non-linear networks trained on ImageNet dataset: We train multiple ImageNet classifiers and compute the feature compressions of these models. Details are in Appendix B.4 (Similar results are conducted on the MNIST classification, too. Please see B.2) We take the experiment on model Inception- Resioon--esee-222 Resnet--v2/Zzgedd et al.] (2017) as an example. To describe that feature compression leads to adversarial fragility, we focus on two classes of inputs chosen from
perturbations
0.6
0.5
Magnitude of cos
0.3
0.2
0.1
0.002
Thearetiratly predicted rubsuteens decorp..
0.0
20
1.
0.000
0.0
0.2
0.4
0.6
alpha
Figure 1: Theoretical predictions o (from QR decomposition) match empirical NN's cos((f))
Figure 2: Magnitudes of cos(O.), for [0.1]

---

Published as a conference paper at ICLR 2026
ImageNet: English springer and Afghan hounds. We name the English springer picture as Xspr and the Afghan hound picture as X/nd-
We examine the compression rate cos(Oa), the difference magnitude L ||xind Xspr 2l2, the least numerical perturbation magnitude Q, and the theoretical expected signed "length" M traveled by the adversarial attacker. These key values are computed in the following ways: (1) cos((..) represents the feature compression rate. We let x., = 0x spr + (1 - a: Xhndd for a E [o, 1]. We compute the at the consecutive images Xa, and Xhond - Xapr- For each experiment, we compute cos(On, (0..) regarding each a and their averages in Table The last row of Table iss is the average compression rate. These ratios are generally small, representing the adversarial fragility of trained NN classifiers on ImageNet. For each English springer image, we select the corresponding closest Afghan hound image that has the smallest 12-norm difference from it. To describe the change in magnitudes of cos) we take the 100 different scalars o, a E [0, 1}, and plot cos((f.)) in terms of a, as shown in Figure 2 (2) Q represents the minimum magnitude of the perturbation (numerically found by the least-likely class attack Kurakin et al. (0117) on x*pr required to cause the trained NN to make the classifier misclassify the perturbed image as Afghan hound. Let the perturbed image be x
angle O,, between the gradient V V(Sspr (x.)
Xspr + Eq where ||Eg]| = Q. (3) M represents the expected signed "length" traveled by the adversarial attacker performing the gradient descent [(ane(x)))) attack. M is defined below according to (5) M Σ., (V(eer((Xa) - ))1l cos(O.) -1)) IVLJspr(X.ppr )-1inelXxpr)))
f(nn(Xxpr)) represents the gradient at the benign English springer image, and the numerator is the value of the change D in (5). Since the adversarial perturbation is small in magnitude, the adversarial attacker will travel a short distance from the benign image, there is big change for the gradient of the g(-) function in Section and we can approximate the gradient on the path traveled by the attacker as [ (Xspr) Shndd (Xppe)) )ll- Thus by 5 and (6). the absolute value of M above represents the theoretical prediction for the needed magnitude of the adversarial perturbation.
where aa is the total number of scalars. Note that the denominator V
Table 4 displays the values of L. Q and M. We compute these key values with 4 different inputs Xapr. The numerical result in Table 4 confirms our prediction that [M LMM/(0.3) (0.5L) = (0.5L., which means that the theoretical predicted Tength traveled by the adversarial attacker is close to the mag- nitude of the actual practically found adversarial perturbation (0(..)) found by the adversarial attack algorithm such that the output of the classifier is changed to the Afghan hound. In these ex- periments, the difference between [ and Q/(0.5L) might be due to that the least-likely class attack does not find the sinimum-mmagniuue perturbation and approximations in our analysis (for example, assuming that the gradient around the benign image remains the same). This exper- iment on the ImageNet also confirms that the feature compression leads to adversarial fragility.
Exp. No.
0.0
0.0011
0.0022
-0,0022
0.000S
0.1
0.0037
-0.0021
0.0015
0.000S
0.2
0.0033
0.0076
0.0028
-0,0156
0.3
0.0020
0.0011
0.0005
0.0059
Exp. No.
0.4
0.0081
0.0035
0.0008
0.0020
3/ 1//10.L)
-0.0765
-0083186
-0.03863
-0.08U6
0.5
-0.0057
-0.0060
00.0199
-0,0098
5.8144
5.8278
5,8893
5,8207
0.0091
0.0042
0.00151
7.
198.2214
100.801
193.3122
171.4597
0.7
0.0008
-0.0055
0.0033
0.00135
Q/(0.5))
0.0722
0.0828
0.0607
0.0679
0.8
0.00144
-000044
0.0018
0.00159
0.3
0.00015
0.0008
0.0020
0.0024
Table 4:
Adversarial attack analysis on Ima-
1.0
0.0012
0.0004
0.0008
0.0011
geNet
Avg.
0.0022
0.0348
0.0024
0.00151
Table 3: ImageNet: cos(Oa) regarding each
consecutive image
7 CONCLUSION
explanation of the NN adversarial fragility by firstly comparing worst- case peformance against optimal classifiers. Analytically we show that NN' adversarial robustness sometimes can be only 1/vd of the best possible adversarial robustness. This perspective may provide new approaches for improving NN adversarial robustness, such as regularization of feature compression during training. Limitations of this paper include the need to extend detailed theoret- ical analysis and numerical experiments to more general data distributions, NN architectures and trainings (such training in
We provide
Frei et al. (0023)
10

---

Published as a conference paper at ICLR 2026
REFERENCES
N. Akhtar and A. Mian. Threat of adversarial attacks on deep learning in computer vision: a survey. arXiv:1801.000 00553, 2018.
Zeyuan Allen-Zhu and Yuanzhi Li. Feature purification: How adversarial training performs robust deep learning. In IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS), pp. 977-988. IEEE, 2022.
Peter L. Bartleut Sébastien Bubeck, and Yeshwanth Cherapanamjeri. Adversarial examples in multi- layer random relu networks. In Advances in Neural Information Processing Systems, volume 34, 2021.
Oliver Bryniarski, Nabeel Hingun, Pedro Pachuca, Vincent Wang, and Nicholas Carlini. Evading adversarial example detection defenses with orthogonal projected gradient descent. In Interna- tional Conference on Learning Representations, 2022. URL httttss//ppperrrieeee net/
Amit Daniely and Haim Shacham. Most relu networks suffer from l² adversarial perturbations. In Advances in Neural Information Processing Systems, volume 33, pp. 6629-6636, 2020.
Eustratiadis, H. Gouk, D. Li, and T. Hospedales. Attacking adversarial defences by smooth- ing the loss landscape, August 2022. URL http://arxiv. wwt..//.ggabbb..ooooo/ http://aarx...rr//b///222...0000 00862 arXiv:2208.0068 00862 [cs].
A. Fawzi, S. Moosavi-Dezfooli, and P. Frossard. Robustness of classifiers: from adversarial to random noise. In Proceedings of the 30th International Conference on Neural Information Pro- cessing Systems, NIPS' 16, pp. 1632-1640, USA, 2016. Curran Associates Inc. ISBN 978-1-5108- 3881-9.
Spencer Frei, Gal Vardi, Peter L. Bartlett, and Nathan Srebro. The double-edged sword of implicit bias: Generalization vs. robustness in relu networks, 2023. URUhttps://arxiv..o 2303. 01 456
I. Goodfellow, J. Shlens, and C. Szegedy. Explaining and hamnessing adversarial examples. 11Xiv:1412.6572 [cs, stat], December 2014.
B. Hassibi and H. Vikalo. On the sphere-decoding algorithm expected complexity. IEEE Trans- actions Signal Processing, 53(8):2806-28) 2818, 2005. doi: 10. (0009//SP..00.8505555..
X. Huang, D. Kroening, M. Kwiatkowska, W. Ruan, Y. Sun, E. Thamo, M. Wu, and X. Yi. Safety and trustworthiness of deep neural networks: a survey. (8/2.08/100834 [es], December 2018. arXiv: 1812.08342.
Andrew Ilas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In H. Wal- lach, H. Larochelle, A. Beygelzimer. F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL.
wwwpiinnggnnnuuiieee www.nneeii.... cc.pssscppppppeeeeeeeeeeee forrrffleee fwwess/papee/e//11/// 2hw.eee00ff22211133355555---0-00 .dff
File/
S. Kanai, S. Yamaguchi, M. Yamada, H. Takahashi, K. Ohno, and Y. Ida. One-vs-the-rest loss to focus on important samples adversarial training. In Proceedings of the 40th Interna- tional Conference Machine Learning, pp. 15669-15695. PMLR, July 2023. URL htRps:::: mlr
2017. / wttpppppwwwww..eaapappe
html ISSN; 2640-3498.
Alexey Kurakin, Ian Goodfellow, and Samy Bengoo Adversarial examples in the physical world, abs wwt.://gg/.......oo////
Ang Li, Yifei Wang, Yiwen Guo, and Yisen Wang. Adversarial examples are not real features. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in
Neural Information Processing Systems, volume 36, pp. 17222 17237. Curran Associates, Inc., 2023. URL https:: /tt.o//eeo////e/ac......e.eeeee.
http:e/perreeeeeeeeeeeeeepp fi ElL29858577777005552222555551550

---

Published as a conference paper at ICLR 2026
L. Li and M. Spratling. Improved adversarial training through adaptive instance wise loss smooth- arXiv:2303.14077 [es].
ing, March 2023. httl:tttpp//xx..oo////o..o
Odelia Melamed, Gilad Yehudai, and Gal Vardi. Adversarial examples exist in two-layer relu net- works for low dimensional linear subspaces, 2023.
M. Reza, A. Rahmati, T. Wu, and H. Dai. CGBA: curvature-aware geometric black-box attack. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 124-133, Paris, France, October 2023. IEEE. ISBN 9798350307184, doi: 10.1000CCCV500702223332 URL htt.tttsss//eeeeelorrtteee ieee http///ommmmmmmm/1//s/3///
Carl-Johann Simon-Gabriel, Yann Ollivier, Leon Bottou, Bernhard Schölkopf, and David Lopez- Paz. First-order adversarial vulnerability of neural networks and input dimension. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 5809-5817. PMLR, 2019.
V. Singla, Singla, Feizi, and Jacobs. Low curvature activations reduce overfitting in ad- versarial training. In 2021 IEEE/CVF International Conference Computer Vision (ICCV), pp. 16403-16413, Montreal, QC, Canada, October 2021. IEEE. ISBN 978- 1-66542-812-5. doi: D0.1099/CCV44992.2221..00 / 97106597
BRLhttts:/// http:/eeeeeeeeeeeeeeeeee
ieee org/ wwtg/oocwgen/////ooooooomm/////e
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014. URL http:ttp://eee http//rxiivio http:::/axxi..or//////// ats///bs/1133 111... 6199
Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander A Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the Thirty- First AAAI Conference Artificial Intelligence (AAAI-17), pp. 4278-4284, 2017.
T. Tanay and L. Griffin. A boundary tilting persepective the phenomenon of adversarial examples. 160.v::00..0700 07690 [es, stat), August 2016. arXiv: 1608.07690.
Gal Vardi, Gilad Ychudai, and Ohad Shamir. Gradient methods provably converge to non- robust networks. In Advances in Neural Information Processing Systems, volume 35, 2022. URL
D. Wang, W. Yao. T. Jiang. G. Tang, and X. Chen. A survey on physical adversarial at- tack in computer vision, September 2023. URL http:://axxv..org///o/oo/ arXiv:2209. 14262 [cs].
B. Wu, Z. Zhu, I. Liu, Q. Liu, Z. He, and S. Lyu. Attacks in adversarial machine Learning: a systematic survey from the life-cyele perspective, January 2024. URL http::///txx...otttt abs/ bbs/2330.co 09457 2Xiv:2302.00457 [cs].
Hui Xie, Jirong Yi, Weiyu Xu, and Raghu Mudumbai. An information-thheorriee explanation for the adversarial fragility of classifiers. In 2019 IEEE International Symposium on Information Theory (ISIT), pp. 1977-1981, 2019, doi: 10. 10.11099/ST..20119.88888 888497777.
Q. Xu, G. Tao, and X. Zhang. Bounded adversarial attack deep content features. pp. 15203 15212, 2022. URL htttps::/oppnaaccesss ttee:s.thhwtve http://mccss..tteeee...oooooo/tn html /Xu, Rounded Ltt_eeedduueeereeeaeeeeeeeeeeeee
CVPR 2022 paper.html
Weiyu Xu, Youzheng Wang, Zucheng Zhou. and Jing Wang. A computationally efficient exact ml sphere decoder. IEEE Global Telecommunication. Conference, 2004. GLOOBECOM '04., volume 4, pp. 2594-2599 Vol.4, 2004. doi: 0001109/....0002000011177777
X. Yuan, P. He. Q. Zhu, and X. Li. Adversarial examples: attacks and defenses for deep leam- ing. 172 [cs, stat], December 2017. URL http:: http/v/xxvv///////////t////.atva wwtp://ww.r/ab/b/////oa/ 07107 arXiv: 1712.07107.
12

---

Published as a conference paper at ICLR 2026
B. Zeng, L. Gao, Q. Zhang, C. Li, Song, and S. Jing. Boostiog adversarial attacks by leveraging de- cision boundary information, March 2023. URL arXiv:2303.05719 [es].
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In Proceedings of the 5th International Confer- ence on Learning Representations (ICLR 2017), 2017. URL /ELLtttpe//eeeeeeviee.eeett Forum?id-Sy8gdBB. 9xx
Jie Zhang, Kristina Nikolić, Nicholas Carlini, and Florian Tramèr. Gradient masking all-at-once: Ensemble everything everywhere not robust, arXiv preprint 4rXiv:2411.1884, 2024.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A. PROOF OF LEMMA 2
Proof. Using the Chernoff Bound, we get that
E (
P z S da)
inf
t<0
edda
However, we know that
E(el2))= L P(z)ete dr
1
c(l- dx.
2t
--0
Evaluating the integral, get
B(e
1
V2
2T
V22
4t
V2 4t
This gives us
d
11 gldo
V2
S(1)
6 /
48
Since d 2 1 and the base is positive, minimizing ((1) equivalent to maximizing ea 22 46. Tak-
ing the derivative of this with respect t, we get e av22 as O, we get t 2-1_ Plugging this back into f(t), we get
- Taking the derivative
P(X < da) -
elaad
We now notice that the exponent g(a) 2 log(ae! aa) goes towards negative infinity - 0, because log(a) goes to negative infinity as a + O.
A.2 LEMMA 7 ITS PROOF
Lemma Let H = HI-1... H2,,, where each H; (1 sisl-1) is an 0i+1 X ny matrix composed of ii.dd standard zero-mean unit-variance Gaussian random variables, and H; 's are jointly independent. Here without loss of generality, we assume that for every i, 14411 2 n.
We let Rú, R2, ..... and R be I independent upper triangular matrices of dimension n X nı with random elements in the upper-triangular sections. In particular, for each R,, 1 < i 1- 1, its off-diagonal elements in the strictly upper triangular section are i...d. standard Gaussian random variables following distribution (0, 1); its diagonal element in the j-th row is the square root of random variable following the chi-squared distribution of degree T4+1 - j + where 1 S S n..
Suppose that we perform QR decomposition on H. namely И = QR. where R is of dimension nı X ny. Then R follows the same probability distribution as RI-1R1-2" R2 R1, namely the product of R1, Ro,, ..., and Ri-1-
Proof. We prove this by induction over the layer index i. When i = 1, we can perform the QR decomposition of H, Q, Rı, where Rú is an upper triangular matrix of dimension mi X nis Q1 is matrix of dimension 7/2 X with orthonormal columns. From random matrix theories Hassibi & Vikalo (2005); Xu et al. (2004), we know that R₁ 's off-diagonal elements in the strictly upper triangular section are i.i.d. standard Gaussian random variables following distribution A/00, 1).; its diagonal element in the j-th row is the square root a random variable following the chi-squared distribution of degree n22 j +1.
Let us now consider Hg dimension ny X n2. Then
H₂ H2Q1R1.
14

---

Published as a conference paper at ICLR 2026
Note that H2Q1 is matrix of dimension ng n1, and the elements of H2Q1 again ii.d. ran- dom variables following the standard Gaussian distribution (0,, 1). To see that, we first notice that because the rows of H2 independent Gaussian random variables, the rows of H2Q1 will be mu- tually independent. Moreover, within each row of H2Q1. the elements are also independent, (0. 1) random variables because the elements are the inner products between a vector of na independent (o 1) elements and the orthonormal columns of Q2. With Q1 having orthogonal columns, these inner products are thus independent because they are jointly Gaussian with correlation.
Then we can replace H2Q1 with matrix H2 of dimension fig x ni, with elements of H22 being i.i.d. (0, 1) random variables. We proceed to perform QR decomposition of H₃ = Q2 where R2 is of dimension 71 X n1. Again, from random matrix theories, we know that R2 s off off-diagonal ele- ments in the strictly upper triangular section are i.i.d. standard Gaussian random variables following distribution (0, 1).; its diagonal element in the j-th row is the square root of random variable following the chi-squared distribution of degree ng - j
Because
H2H = Q2R2R1,
and the products of upper triangular matrices are still upper triangular matrices, Q2(RgRi) is the QR decomposition of H2H1.
We assume that Hi+1 Hi...H, has QR decomposition Qi+ Ri- Ri. Then by the same argu- ment as going from H₃ to H2H1, we have
H442Hl41 Hi... = 2
R1)
working the QR dccomposition of H H H..HH,, Hi, where Qi42 is n443 X n, matrix with orthonormal columns.
By induction over we complete the proof.
A.3 PROOF OF THEOREM 3
Proof. From the proof of the first part of Theorem 1 we know that for any > 0, with high
probability, ||v; - Vyll > (1 - 2d for every pair (i,j) where i + j. Now consider particular pair (i. and fix vi and Vj. We further notice
||x x,| | Gi v2)l--
Because the elements of G1 Gi(v; - v,) are independent Gaussian random variables with variance [[v;-v,)]) 11 due to Chernoff bound similarly appearing the proof of Lemma 2 ||G (v:-v,) > (1-v)l((v; v with probability least - -8d where $ > 0 is constant. By induction over t, we also have ||G,.. ... Gv(v, - vi)ll 2 (1 - c))((v; v with probability at least -Add where 3, > 0 is constant. By the union bound over all pairs of (i.j) where + j. we conclude that with high probability,
min
||x: x,112 2 (1 - e)
HS 1=,,,...d j=12
For proving the second part, we use Lemma Firstly, we take H' = H1 Hi, and let this correspond to Hy similarly in the proof of Theorem [ We also let Hi correspond to Hy in the proof I Then the arguments in the proof of ( apply. We obtain that, in order to change the prediction label, only need perturbation of magnitude at most R + R Ra.al. where R the upper triangular matrix resulting from the QR decomposition of G GI- Moreover, by Lemma 7
Ra- + RRa- + RR.al S |Ra-111B--1111 11
where || ||/]] 1B the sum of the absolute values of elements in the bottom 2 x 2 submatrix of Ray and R i's are the upper triangular matrices coming from the QR decomposition of G,, 1 < S t. |[ will all be bounded by a constant D at the
Because with high probability, || | same time, we can find a perturbation of size bounded by constant D such that it changes the output decision of the neural network classifier.
15

---

Published as a conference paper at ICLR 2026
A.4 PROOF OF THEOREM 4
Proof. Suppose x = Xi is the ground-truth signal. We use p and P, as shorts for probe; and probej. So
(p x1) = 111
(pys x;) = r1244 + 422
We want to add A to x such that (P,. x; + A (p.. + A). Namely, (P. -p, A) < xP),
(p) x²). This is equivalent to
P
P:, A)
(p: x
Pp x;) = 7114,1 (r120:1 + 122012)
We also know that
(pj P; A) (r12 -1 r22
So, by the Cauchy-Schwar inequality, can pick A such that
p Pi A) = || 22 (r12 r11) + 22
So there exists arbitrarily small constant > 0 and perturbation vector A such that
T110 (1224,1 r222;;))
| <
+
(7)
v - ru 2 +
and (p, x; + A) < (pj. X; + A),
(8)
leading to misclassified label because
f((x+A)
A.5 PROOF OF THEOREM 5
Proof. The proof of the first claim follows the same idea as in the proof of the first claim of Theorem
The only major difference is that we have 2d-1 x 2d-1 = 22(d-1) pairs of vectors to consider for the union bound. For each pair of vectors x; and Xy. x; -x; still have i.i.d. Gaussian elements with the variance of each element being at least 4. By Lemma and the union bound, taking constant sufficiently small, the exponential decrease (in d) of the probability that ||x; x,) smaller than a will overwhelm the exponential growth (in d) of 22(d-1), proving the first claim.
Without loss of generality, assume that the ground-truth signal is x; corresponding to label +1. Then we consider the QR decomposition of A,
A=QR,
where E Rdddd satisties Q" Q Laxds and iddd is an upper-triangular matrix. Because the output neural is f+1(x) = +1 whenever the last element of x is +1 and the other elements of x are free take values +1 -1, w H4-1 HH must be equal the last row of A Rt R-1Q"
We notice that the inverse of R is an upper triangular matrix given by
0
0
Ra
Ra 1.d
Ra 1.4
Ra.d
0
Ra.a
where we only explicitly express the last two rows.
Then the weights satisfy
W H ... Hi
Rdddd
and
w H1-1 ... H1
Qr
Rad
16

---

Published as a conference paper at ICLR 2026
where Q.d is last column of matrix Q. We design perturbation
e Q X ebasis
where Cbasis = (0,0, .,.,,,,,00 -2 ). We claim that under such a perturbation c, the input will be X; e and we have
f+1(x +e) =1, and f-i(x; + +e) 1,
thus changing the classification result to the wrong label.
We know that x; Az, QR so x, (zi)a x Ra namely the +1 label multiplied by Ra,d- Then f+1 (xi + e) is equal to
Q(Rz, + @basis). The last element of Rz, is just
w H ... Hi(x; (xxxx + = (Ra,d - 2Rd,d) = -1,
Radd
and f-i(x; + e) is equal to
HI- H1( +e) (Ra.d 2Rd,d) +1,
Ra,d
thus flipping the prediction and achiecing successful adversarial attack.
The magnitude of this perturbation is
||el| = 112 = 2Ra.a-
By random matrix theory Hassibi & Vikalo (2005) Xu al.) (2004)for the QR decomposition of the Gaussian matrix A, we know that Ra.a is the absolute value of random variable following
the standard Gaussian distribution NO,, 1). Thus, there exists constant such that, with high probability, under an етrоr with ||e|| S D, the predicted label of the neural network will be changed.
A.6 PROOF OF THBOREM 6
Proof. Suppose that we add a perturbation q the input x + ex1, namely the input becomes x + CX1 + q. Then
+xx - q) ox)
fx(x txi - q) = fx(x + V
We want to pick a q such that
fix+ cx + q) ~ fi(x ox2) 12(x + ex1 + q) ~ f2(x ex2).
make this happen. However, we claim we can potentially take perturbation of much smaller size achieve this goal. We note that
Apparently, we can take q = c(x; -x, x1)
fi + + + Vf(())
f2xx cx + +)) {
+ (x2(
We want
f1() + (V Vx(() f() (xV
+xx))+V/((x) qx (x()) ++V
Namely, we want
vv)(()) x1 + < (x x1 -=
17

---

Published as a conference paper at ICLR 2026
Direction looked at by the neural network classifier
Function contour
(class 1)
Compressed feature
(class 2)
Feature looked at by the optimal classifier
Figure 3: Illustration plot of the feature compression concept.
So
x1), q= (x2 -x)) X1).
Then we can just let q be the projection of c(x² x1) onto the subspace spanned by ( and fx()..
If (x),, (x)), and X₂ - x₁ all have independent standard Gaussian random variables as their elements, then the square of the magnitude (in & norm of that projection of x2 chi-squared distribution of degree 2. At the same time, the square of the magnitude of X2 X1 will follow the chi-squared distribution with degree d. Moreover, as d + the square of the magnitude of X2 x is ((d) with high probability. Thus changing from x + eX1 to x + eX2 will be ( times more difficult than changing the classifier's 's label using an adversarial attack.
will follow
A.7 ILLUSTRATION OF THE FEATURE COMPRESSION CONCEPT
We illustrate the concept of feature compression in Figure 3 Here the ground-truth signal x² belongs to Class 2. The direction between X2 and its closest point x in Class 1 is the feature the optimal classifier should look at (namely the direction for the "weakest" separation between Class 1 and X2). However, the NN classifier instead looks at the direction (namely compressed feature direction) having angle with the optimal direction X - X2. So in order to change label from Class Class 1, the attacker can just attack along the "compressed feature" direction. For the NN classifier's neuron output function, it will take much smaller distance change the function value to the same value as at X1, enabling successful attack with small magnitude. However, if the direction looked the NN classifier is the same as the direction for the "weakest" separation between Class 1 and X2, we will not have adversarial fragility. Our way of researching this phenomenon is novel because we compare compressed feature direction against the direction looked at by the optimal classifier, sereedd-ffttt-urreeeeeeeee attack against NN's average-
instead of comparing the performance of case performance under random-directtin noises.
B SUPPLEMENTARY NUMERICAL RESULTS ON MULTIPLE NONLINEAR NEURAL NETWORKS, AND ON MULTIPLE DATASETS
B. PERTURBATION ANALYSIS ON NON-LINEAR NETWORKS TRAINED ON GAUSSIAN DATA POINTS
We trained l-hidden-layer (and deeper) non-linear neural networks with ReLU activations to test Theorem 6 To generate vectors x, Xı and X2, define vectors Z41 and z of dimension d. The first elements z+1 are the same as those of z and take random values +1 or -1. The last element of 241 is +1 and the last element of z is -1. Then we define vectors
18

---

Published as a conference paper at ICLR 2026
b1 Az+1, and b2 = Az For 10 values of [0.1], set ab, + (1 a)bb for every scalar a. In Theorem 6 take X1 = b1 - X = (1 - a)b;; - (1 aabb and X₂ = b - x = ab2 ab; for every scalar a. With d = 12, we calculated the projection of x1 X2 onto the subspace spanned by (x) VV/((x) as (
-x2)- We define the following ratio = [ (x] - x - -22l22- By Theorem 6 and the discussions that follow it, we know p is "compression rate" locally: the rate of the compression of the critical feature X2 X1 (the feature the optimal classifier should look at) onto the gradient (the feature actually looked at by the classifier). P also the ratio of tolerable worst-case perturbation of the trained neural network classifier to that of the optimal classifier (locally). The smaller is, the less adversarially robust the trained neural network is, compared with the optimal minimum-ddstanne classifier.
For every a, we calculate the sample mean and medians of p over 50 accurate l-hidden-layer non- linear neuron networks in Table|5) For example, when a = 0.444, p has a mean of 0.3272, meaning the trained classifier is only 0.3272 (0.3272 ~ 0.10 when considering the energy of perturbation) as adversarially robust as the optimal minimum-die distaance classifier. The ratios are similarly small we train neural network classifiers with more layers.
a
0.111
0.222
0.333
0.44
0.556
D.667
0.778
03899
Avg.
0 0.7278
0.3275
0.3273
0.3270
0.3272
0.3275
0.3274
0.3281
0.3280
0.527
Medium
0.3270
0.3261
0.3258
0.3255
0.3303
0,3307
0.3293
0,3322
0,3315
0.3324
Table 5: Averages and mediums of p
In the perturbation analysis, input data b and the non-lineat model architecture follow the settings of Theorem with d = 17. We examine (Os), the magnitude m of the last column of A, and the least perturbation magnitude 6. d represents the minimum magnitude of perturbation (numerically found via gradient- based adversarial attack) on the input b required to cause the trained neural network to flip the classifier's 's output on b. Let bade b + 5 be the perturbed input. 0, is the angle between (fi (badv) - fz (bado)) and the last column of A. Over 10 sampled input b, and 10 sampled input b2, the averaged key values c0.( 5, and magnitude
is displayed in Table 6 The numerical result in ( confirms our prediction that cos().) x 8/m, which means that the compression ratio (0,) the amount of adversarial perturbation (6/m ratio) needed to add the input signal such that the output of the classifier is changed to the opposite label. This confirms that feature compression (i (Cos(0))) leads to adversarial fragility.
B.2 ADVERSARIAL ATTACK ANALYSIS ON NON-LINEAR NETWORKS TRAINED ON MNIST DATASET
We trained a 6-layer convolutional neural network for classification on the MNIST dataset. The convolutional model has convolutional layers, dropout layers, and fully connected layers. The first convolution layer takes an input with 1 channel, has kernel size 3 x 3, and stride of 1, and produces an output with 32 features. The second convolutional layer takes input with 32 features, has kernel size 3 x 3 and stride of 1, and outputs 64 features. The first dropout layer drops 25% of the outputs and the second dropout layer drops 50% of the outputs. The first fully-connected layer flattens output from the convolutional layers with 9216 features and outputs 128 features. The final classifier has 10 output classes. The loss function used in the training is Negative Log Likelihood loss, and Adadelta is the optimizer. The number of epochs is 2.
To describe that feature compression leads to adversarial fragility, we focus on two classes of inputs 11, representing inputs labeled as 1, and 1,. representing inputs labeled as 7. We create artificial inputs A₁ out of T by blocking out enough pixels on the upper left stroke of "7", so that the artificial input A is classified way, we can directly compare the artificial image "1" and the original input image "7", and calculate half of their distance in l norm arguably as the smallest perturbation needed to change the "7"
label 1 by the neural network. By generating an artificial image "1" this
input key values-
cos(e.)
magnitude (m)
6/m
bj
O. 155244514 D. 10802033400094400
5. 19229935050066333 5. 19281350500644355
0.15450701
b2
O. 0.14989976
D. 44703444333445555
0.14630501
Table 6: Adversarial attack analysis for input d = 17
19

---

Published as a conference paper at ICLR 2026
image such that human eyes (as substitute for the optimal classifier) perceive it as "I" image. Otherwise, because of writing styles or different locations where we write "1" and "7" on the images, it may not be reasonable to take the &g norm of their difference metric of how far away these two images are from each other.
Figure is example of the input image I7 (left of 4 and its corresponding artificial image A1 (right of 4 : We examine 8, the magnitude L = ||A, 17ll2, and the least numerical perturbation
Digit
Digit (Artificial)
7
Figure 4: Clean image "7" and artificial image "I"
magnitude Q. L is the magnitude of the difference between the artificial image A₁ and the clean image J7. Q represents the minimum magnitude of perturbation (numerically found by fast gradient sign method) on the clean input 1 required to cause the trained neural network to make the classifier misclassify the perturbed image as "1". Let the perturbed image be 14 17 + Eq. where |
Q. Here 0g is the angle between the image difference A - 17 and the gradient ((f((1)) (17)))
Here f-(-) is the logit of class and (-( is the logit of class 1. We take the gradient of fr([7) (T7) and evaluate the gradient (ff((Tr) - fi(T)) at the clean image I7.
The key values of cos(O.), L, Q, and M are displayed in Table 7 We let M denote the expected signed "length" traveled by the adversarial attacker performing the gradient descent attack. We compute these key values with 5 different clean inputs 17 and their corresponding artificial images prediction that cos(0:) x (0(..L),, which means the
A1. The numerical result in 7 confirms compression ratio cos(0: (0:) is the amount of adversarial perturbation (Q.(L.))) needed to add to the input image such that the output of the classifier is changed to the opposite label. In experiments 3 and 4, the difference between cos(At) and Q/(0.5L) might be due to that the fast gradient method does not find the caniiimummmanittuue perturbation. This experiment the MNIST dataset also confirms that the feature compression () (O,)).) leads to adversarial fragility.
We let X aA1 (1 TT. for [0, 1). M is defined below according 5 and in this experiment, M is:
cas - cos)))) - 1))
M
where is the total number of scalars. We also compute the angle 6 between the gradient (Xf( fi (X)) at the consecutive images X, and A₁ - (the difference between the artificial image and the clean input). For each experiment, we compute cos(f.)) regarding each a and their averages in Table [ The last row of Table the average compression rate. These ratios are generally small, representing the adversarial fragility of trained NN classifiers.
B.3 ADVERSARIAL ATTACK ANALYSIS ON INCEPTION-RESNET-V2 TRAINED ON IMAGENET DATASET
We use the Inception-Resnett Szegedy et al. (017) for classification on the ImageNet dataset. To describe that feature compression leads to adversarial fragility, we focus two classes of input chosen from ImageNet: English springer and Afghan hounds, whose sample pictures are shown in Figures 5 and 6 We name the English springer picture Xepr and the Afghan hound picture |[ Xspr the least numerical perturbation magnitude Q. and the theoretical expected signed"length" M (+ signed according to the angle
as Xindd We examine 6,, the magnitude
20

---

Published as a conference paper at ICLR 2026
Experiment No.
M
0.22555
-0.6393
0.9546
0.4523
0.2034
0.7255
0.9724
1.5866
1.1309
1.5961
2.1024
3.4354
2.5806
2.7998
(/(0.5L)
0.2548
0.6902
0.5661
0.8187
0.8076
M/[0.3L]
0.2826
0.6082
0.5557
0.3499
-0.799
cos(6t)
00.2170
-0.318
-0.2790
-0.2734
-0.3302
Table 7: Adversarial attack analysis on MNIST. Ideally, theoretical [ should be close to (0..L.), which is the case for 1, 2, 3,5, and approximately for 4. The gap for image may be due to approximation errors in the analysis or the suboptimality of the found adversarial perturbation.
Experiment No. a +
2
3
0.0
-0.1537
-0.4023
-0.3749
-02003
0.1
-0.1692
-0,119
00,0075
0,2019
0.2015
0.4240
0.3861
0,2024
0.1574
0.4273
0,4037
0,2374
0.4
-0.1422
-0.9988
-0,3098
0.2325
0.5
0.0882
-0.9988
0..689
0.2408
0.6
0.0231
0.3424
0.3182
0.2459
0.7
0.0794
0.2326
0.1864
0.1630
0.8
-0.0813
-0.0409
-0.0123
0. 1286
0.9
-0.0794
-0.0127
-0.0366
0,0186
1.0
0.1127
0.0153
0.0729
0.0317
Avg.
-0.1136
-0.2831
00.2634
-0.1788
Table 8: MNIST: cos(O:) regarding each consecutive image
[Affgaanhoundd) hound']
0
50
50
100
100
150
150
200
200
250
250
50
100
150
200
250
100
150
200
250
Figure 5: English springer
Figure 6: Afghan hound
21

---

Published as a conference paper at ICLR 2026
0.010
Magnitude of
0.006
0.002
0.0
0.4
0.6
0.8
1.0
alpha
Figure 7: Magnitudes of cos(O,), for a E [0, in experiment
between the path direction of the attacker and the gradient, please see Section 5 (5) and the definition of M below) traveled by the adversarial attacker when performing the gradient descent attack.
L is the magnitude of the difference between Xspr and Xhnd- Q represents the minimum magnitude of the perturbation (numerically found by the least-likely class attack Kurakin et al.) (2017)) on x required to cause the trained neural network to make the classifier misclassify the perturbed image as Afghan hound. Let the perturbed image be xpp Xspr + EQ where ||Eell = Q. Here Oe is the angle between the difference in image Xhnd Xspr and the gradient ((fsp=(Xxpr) - Shaal [>hnd(Xspr) Here fap is the logit of class English springer and find the logit of class Afghan hound. We take the gradient of (xepr) - fhndd (Xppr) and evaluate the gradient V V(fspr (Xape) (xpr(Xapp) at the image xspr.
We let Xa = ax 0X spr + (1 X for a E [0, 1J. We compute the angle 0a between the gradient (ffopr(X.)
fhnnd fhnd(Xaa) (xx)) the consecutive images Xa. and Xhnd Xspr- For each experiment, we compute cos(f regarding each a and their averages in Table 9 The last row of Table 9 is the average compression rate. These ratios are generally small, representing the adversarial fragility of trained NN classifiers on ImageNet. For each English springer image, we select the corresponding closest Afghan hound image that has the smallest 12-norm difference from it. To describe the change in magnitudes of cos(Oa). we take the 100 different scalars E [0, 1], and plot cos(Ox)] in terms of a, as shown in Figure
We let M denote the expected signed "length" traveled by the adversarial attacker performing the gradient descent attack. A is defined below according to (5), and in this experiment, M is:
M E. ( ( (Xx) - fand fand (x,)] cos(O.)) (L/(na - [[V (Xapr)
-1) 1))
where na is the total number of scalars. Note that the denominator / (ffspr(Xep)) - fhnn (Xppr) I represents the gradient at the benign English springer image, and the numerator is the value of the change D in (5). Since the adversarial perturbation is small in magnitude, the adversarial attacker will travel short distance from the benign image, there is no big change for the gradient of the g(-) function in Section and we can approximate the gradient on the path traveled by the attacker as [[
Shmaa (Xprr)) Thus by (5) and (6). the definition of M above represents the theoretical prediction for the needed magnitude of the adversarial perturbation, after we take the absolute value of M.
The key values of L, Q and M displayed in Table 10 We compute these key values with 4 different English springer images as NN inputs.
The numerical result in Table 10 confirms our prediction that |M/(0.L) Q/(0.5L), which means that the theoretical predicted length traveled by the adversarial attacker is close to the mag- nitude of the actual practically found adversarial perturbation (0(.L) found by the adversarial attack algorithm such that the output of the classifier is changed to the Afghan hound. these ex- periments, the difference between |M/(0.5L.) and (/(0..T.) might be due to that the least-likely
22

---

Published as a conference paper at ICLR 2026
Experiment No. a
22002
2
0.0
0.00TI
2.0020
0,0000
0.1
0.0037
0.2
0,0036
0.3
-0.0020
LIO0
0.0003
0.0059
0.4
0.0091
000033
0.0058
0,0026
0.5
0.0057
0.0060
0.0199
0.0098
0.6
-0.0094
-0.0042
0.0011
0.0051
0.7
-0,0009
-0.0055
0.0039
0.0035
-0,034
0.0018
0.00559
0.000S
0.0020
0.0024
0.0012
-0.0004
0.0008
-0.0011
Avg.
0,0022
0.03488
0.0024
0.0034
Table ImageNet: cos( regarding each consecutive image
class attack does not find the minimum- magniitude perturbation and approximations our analysis (for example, assuming that the gradient around the benign image remains the same). This experi- ment on the ImageNet also confirms that the feature compression leads to adversarial fragility.
Experiment No.
M/(0.5L)
-0.0765
-0.03486
0.03865
-0.8006
9
5.8044
5,8278
5 5.8693
5.8207
198.2214
160.8016
193.3129
171.4597
Q/(0.L)
0.0722
0,0828
0.0607
0.0679
Table 10: Adversarial attack analysis on ImageNet
B.4 ADVERSARIAL ATTACK ANALYSIS ON MODERN NEURAL NETWORK MODELS TRAINED ON IMAGENET DATASET
To extend our theory of compression rate to modern architectures, we conducted additional experi- ments on the following modern classifiers: ViT-base- peasse-pttch1--1ee 6224, Inception- Resnet-v2, Resnet50, and VGG-16.
ViT- TTEbase--atchh6-2222 was pretrained on ImageNet-2 dataset consisting of 14 million images and 21k classes while Eccpttio-ResNNe-vv
Resnet50, and VGG-16 were trained on the ILSRRC 2012 ImageNet dataset with Ik classes and ~ 1.2M images. We evaluate the fragility of these models on the ImageNette dataset, which contains approximately 44k validation images.
For each classifier and every validation image, we perform an FGSM attack. If the attack is suc- cessful, we record the original validation image x, and the closest image Xtrg belonging to the adversarially predicted class Yodu- This is the predicted label corresponding to the adversarially per- turbed image Xade The distance L between and Xtrg is measured in (2 norm. Over all such pairs, we examine the average compression rate cos(f) (which taken in the absolute value for every pair of images), the average least numerical perturbation magnitude Q, and the theoretical expected signed "length" M, following the same way as we did in on the images belong to English springer and Afghan hounds classes.
The key values cos(O), M/0.5T,, and Q/0.5L. are summarized Table 11 As predicted by our theory, the theoretical columns of the table. These results show that our feature compression theory can indeed explain the adversarial fragility of NN classifiers, including modern NNs trained on large-scale datasets. L is the magnitude of the difference between x and Xtrg.
are close Q/ (0.5) across all models, shown by the last two
C I. TECHNICAL DETAILS FOR REPRODUCING NUMERICAL RESULTS
Computing infrastructure: For all numerical experiments, we used 32 NVIDIA GeForce RTX 2080 Ti GPUs. The memory limit is 32GB.
Code Implementation: We provide our code implementations of the three models tested for adver-
sarial fragility:
23

---

Published as a conference paper at ICLR 2026
Key Values -> Model +
Avg cos(O)
Avg M/0.57.
Avg Q/0.57
[TT-base-patchl6-.
0.0014
-0.0144
0.0130
Inception-Resnet-v2
0.0027
-0.0495
0.0202
Resnet50)
0.0024
-0.0097
0.0106
VGG-16
0.0034
-0.0144
0.0179
Table 11: Compression rates across multiple modern architectures
Linear-network: https://érive.
http://www.oo/////o/o//////////a https://árive.
Nonlinear neural network trained MNIST:
Nonlinear neural network (Inception-Reesnetx2)) ImageNet:
https://árive ottpgp/cw...oaaoo.o....o.eeeeeea
24


Reviews:
[{"review_id": "9YlMTuzRSw", "reviewer": "Reviewer_J8qY", "summary": "This paper introduces a theoretical framework that attributes the adversarial fragility of neural networks to the phenomenon of feature compression. The authors analyze the effect of dimensionality reduction within network representations and show that such compression significantly reduces adversarial robustness compared to an ideal optimal classifier. The study combines formal derivations with controlled numerical experiments and demonstrates that the degree of compression observed in trained models aligns with theoretical predictions.", "questions": "1. Idealized assumptions. Most theoretical results depend on simplified settings such as Gaussian input data and independent linear layers, which restrict their direct applicability to modern architectures like convolutional networks or transformers.\n\n2. Limited experiments. The experimental validation focuses on synthetic or simplified datasets. The lack of large-scale experiments on realistic benchmarks makes it difficult to assess the practical relevance of the conclusions.\n\n3. Incomplete practical implications. While the theory identifies the cause of fragility, it does not provide clear guidance for designing more robust models or training procedures.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Strong theoretical foundation. The paper presents a clear and rigorous mathematical framework that links the structure of neural networks with their lack of robustness under adversarial perturbations.\n\n2. Novel conceptual insight. The feature compression hypothesis offers a fresh and intuitive way to interpret why modern neural networks are vulnerable, distinguishing itself from gradient-based or geometric explanations.\n\n3. Internal consistency. The theorems and conclusions are internally consistent and appear sound within the given assumptions.\n\n4. Empirical alignment. The observed experimental behavior supports the proposed theory, indicating the analysis captures an essential property of neural networks.", "weaknesses": "1. Idealized assumptions. Most theoretical results depend on simplified settings such as Gaussian input data and independent linear layers, which restrict their direct applicability to modern architectures like convolutional networks or transformers.\n\n2. Limited experiments. The experimental validation focuses on synthetic or simplified datasets. The lack of large-scale experiments on realistic benchmarks makes it difficult to assess the practical relevance of the conclusions.\n\n3. Incomplete practical implications. While the theory identifies the cause of fragility, it does not provide clear guidance for designing more robust models or training procedures.\n\n4. Presentation density. Some proofs and derivations are overly long, which may distract from the main conceptual contributions. Streamlining these sections would improve readability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cCZ71XqqDp", "reviewer": "Reviewer_xkm2", "summary": "This paper proposes a framework to explain the long-standing mystery of adversarial fragility in neural networks.\nThe central claim is that feature compression—where neural networks effectively rely on a small subset or projection of the input feature space for classification—is the fundamental cause of adversarial vulnerability.\n\nMain ideas and results:\n\nFormal comparison with optimal classifiers:\nThe authors analytically show that neural networks’ adversarial robustness can be as small as O (1/sqrt(d)) of the optimal classifier’s robustness, where d is the input dimension.\n\nDerivations:\nUsing QR decompositions of weight matrices, they prove that a successful adversarial perturbation can be achieved by modifying only the compressed subspace of features (Theorem 1–4).\n\nFor both linear and multi-layer networks, they characterize the small perturbation magnitude (constant-scale) that flips classification, while optimal classifiers need perturbations of O(1/sqrt(d)).\n\nGeneralization to nonlinear networks:\nThe paper extends to multi-layer nonlinear networks (Theorem 6), showing that adversarial perturbations are aligned with the projection of the true discriminative direction onto the gradients’ span — i.e., the compressed subspace.", "questions": "1) Can “feature compression” be quantitatively measured for arbitrary networks? e.g., via singular value decay or layerwise Jacobian rank?\n\n2) Does adversarial training reduce compression (and thus improve robustness) in measurable ways?\n\n3) How sensitive are results to activation type (ReLU vs. tanh) or normalization layers?\n\n4) In nonlinear models, how does compression evolve layer-by-layer during training?\n\n5) Can you comment on how this framework could guide architectural design and optimization to avoid compressive fragility?\n\n\nOverall I think the idea of the paper is interesting, but the theoretical statements need clarification.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 3, "strengths": "Conceptual originality: This work claims to move beyond heuristic explanations (e.g., linearity or gradient magnitude) by introducing a precise matrix–QR decomposition view of compression in feature space. This work claims to link geometric and information-theoretic interpretations into a unified mathematical framework.\n\nTheoretical originality: The derivations seem to bridge random matrix theory with robustness analysis — a fresh angle rarely seen in adversarial ML work. The paper claims novel results on the distribution of QR decompositions of products of Gaussian matrices (Lemma 7).\n\nIntuition: Figure 2 (p. 16) provides an intuitive geometric picture—attacks succeed by perturbing along compressed feature directions. The “compression ratio = robustness ratio” insight is crisp and testable. I find the intuition interesting.", "weaknesses": "**Clarity of formal statements:** The theoretical statements are very dense and unclear. I will list below some of the things that I find dubious/unclear. If you are able to clarify, I will be willing to increase my score. \n\nl.143-146: the input dimension is $d$ and the number of training points is also $d$ ? That's an very unsual assumption. And also there are $d$ class labels ? I don't understand. \n\nl.153: does that mean that all the results hold in the limit d to infinity ? so for example in Theorem 1, the perturbation $e$ is a vector whose size goes to infinity? Same question for Theorem 3, the input dimension goes to infinity ? what about the widths of the network ? I'm surprised you can apply random matrix theory without taking all dimensions to infinity. It's not clear what is infinite and what is not, making it difficult to apprehend what is the precise mathematical claim.\n\nTheorem 6:  \"let the closest point in that class to x be denoted by x + x_i\", so you are introducing a new notation \"+\"? In that case, what does \"x + \\epsilon x_1\" mean ? Furthermore, the gradient $\\nabla f_i (x)$, is it a gradient with respect to the parameters, with respect to the input ?\n\nl.336-339, the sentence makes no sense to me.\n\n**Scope of theoretical assumptions:** The main theorems assume Gaussian inputs and idealized linear or piecewise-linear models. It remains unclear how much these results extend to more realistic training dynamics, nonlinearities, and non-Gaussian natural data. For example, the assumption of exponential number of data points (Theorem 5) is not realistic at all, and taking such assumption does not seem to make much sense. Please explain where this is coming from. \n\n**Comparative analysis with existing theories:** The paper differentiates itself from “non-robust feature” and “curvature” explanations, but the relationship with these existing theory is not clear. For example, can compression be quantified independently of gradient alignment? Do non-robust features and compressed features coincide ?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X1oPogxG3D", "reviewer": "Reviewer_w7wf", "summary": "This paper investigates the adversarial fragility of neural networks from a matrix-theoretic perspective. By analyzing the adversarial robustness of neural networks, the authors conclude that their robustness can be only $1/ \\sqrt{d}$ of the best possible robustness achievable by optimal classifiers. The theoretical results are further supported by numerical experiments that empirically validate the proposed analysis.", "questions": "1. In *Theorem 1 (page 3)*, you assume a hard label formulation for the neural network such that $f_j(x_i) = 1$ when $j = i$ and $f_j(x_i) = 0$ otherwise. However, in line 150 on the same page, you define $f_i(x) = w_i^{T}\\sigma(H_1 x + \\delta_1)$. I am concerned about the validity of this assumption, even though *Remark 1* mentions that the constant “1” can be replaced by any positive number. Specifically, what would be the impact if we instead adopt the typical classifier formulation, where $f_j(x_i) = v_j$ and $\\arg\\max_j v_j = i$? Would the main result of Theorem 1 still hold under this more general setting? I think a more detailed discussion of how this relaxation affects the theoretical conclusion would strengthen the paper.\n\n\n2. I am a bit confused about the assumption regarding the training dataset described on \npage 3, lines 143--156 (the second paragraph of Section 2). It seems that your primary theorem is based on training data points $(x_i, i)$, where the input dimension, the number of labels, and the number of data points are all equal ($d$). I think this setting should be clarified further. In particular, why is such an assumption reasonable, or do your theoretical derivations only work on these assumptions?  Although you mention that the number of training data points can be extended to be exponential in the input dimension $d$, it would still strengthen the paper to elaborate on this setting in more detail, for example, by citing previous works that adopt similar assumptions and by discussing why this exponential scaling is theoretically meaningful or realistic in your context.\n\n3. In page 5, line 220, you mention that, by random matrix theory, the eigenvalue distribution follows certain properties, which are then used to derive concentration results. However, as far as I know, such claims from random matrix theory rely critically on the assumption that the data entries are i.i.d. Gaussian random variables. If this assumption changes, for example, if $x$ is not Gaussian or its components are not independent, the results may no longer hold, and, to the best of my knowledge, the corresponding theoretical guarantees remain unknown. If relevant results exist, please provide appropriate citations. Otherwise, this reliance on strong i.i.d. Gaussian assumptions appears to substantially limit the applicability of your theory to real-world data.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The authors provide a thorough and mathematically rigorous theoretical analysis of the adversarial robustness of neural networks. The paper offers clear derivations grounded in matrix theory and connects them to the concept of feature compression, presenting a coherent framework that links theoretical insights with empirical observations.", "weaknesses": "1. You provide a thorough theoretical analysis, but there is a lack of clear discussion and verification of the underlying assumptions. This omission affects the credibility and correctness of your conclusions.\n\n2. Regarding the experiments, on the last page you mention \"Feature compression on deep non-linear networks trained on MNIST and ImageNet in the supplemental materials\". I believe these real-world experiments should be included in the main body of the paper rather than placed in the appendix. They are crucial for demonstrating that your findings are not only theoretically valid under idealized assumptions but also meaningful and applicable in practical settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GQ3NmOmPOE", "reviewer": "Reviewer_hfv7", "summary": "This paper investigates the root cause of adversarial fragility in neural networks and proposes that feature compression, which is the tendency of neural networks to depend on a reduced subset of features, fundamentally limits their robustness. Through a matric theoretic and geometric analysis, the authors show that neural networks are inherently more fragile than optimal classifiers, especially as input dimensions increase. They extend their theory from linear to multilayer nonlinear models and verify the predictions with synthetic and small scale real data experiments. The results align well with the theoretical framework, offering a clear and mathematically grounded explanation for why adversarial perturbations can easily fool deep networks.", "questions": "1. Generality of the Theory: Your analysis is derived primarily under Gaussian and linear assumptions. How sensitive are the theoretical conclusions to deviations from these assumptions — for example, when dealing with real-world, structured, or non-Gaussian data distributions?\n2. Feature Compression Measurement: In practice, how can one quantitatively measure “feature compression” in a trained deep neural network? Are there empirical indicators or diagnostics that could verify the compression–fragility relationship beyond the controlled synthetic setup?\n3. Extension to Modern Architectures: Have you tested whether the same compression–fragility correlation holds for modern architectures such as transformers or convolutional networks trained on large-scale datasets? If not, what challenges do you foresee in extending the theoretical framework to them?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper introduces a matrix-theoretic perspective on adversarial fragility, offering a rigorous mathematical foundation to the widely discussed but poorly formalized “feature compression hypothesis.” The authors connect earlier information theoretic intuition with precise linear algebraic analysis. The proofs are carefully structured, and theoretical predictions are supported by corresponding numerical validations. The idea is well conceived.", "weaknesses": "· Model Selection and Scope Limitations: The theoretical analysis focuses primarily on linear or shallow architectures with Gaussian assumptions. Although the paper claims to extend results to nonlinear deep networks, the derivations rely on strong linearity or independence assumptions that may not hold for modern architectures (e.g., transformers or convolutional models). The leap from linear matrix theory to practical deep models remains under-justified.\n· Limited Robustness Evaluation: The experiments demonstrate numerical consistency but do not include robustness comparisons under standard adversarial benchmarks (e.g., FGSM, PGD on CIFAR-10 or ImageNet). Without such empirical robustness metrics, it is difficult to assess how well the proposed “feature compression factor” explains real-world adversarial behavior beyond toy examples.\n· Dataset Diversity and Representativeness: Most numerical results are based on synthetic Gaussian data or simple setups , with limited exploration of diverse or structured datasets. The extension to MNIST or ImageNet is only briefly mentioned and lacks quantitative reporting. Consequently, it remains unclear whether the compression phenomenon generalizes across modalities and realistic.\n· Insufficient Discussion on Defensive Implications: While the paper identifies feature compression as the root cause of fragility, it does not provide actionable strategies for mitigating it. The lack of defense-oriented discussion (e.g., modifying training objectives, architectural choices, or regularization methods to counter compression) weakens the practical contribution.\n· Assumption Verifiability and Practical Relevance: Many theoretical conclusions depend on strong statistical independence or idealized random matrix properties that are difficult to verify in trained neural networks. This gap between the theoretical setup and actual deep learning practice raises questions about how well the results explain real-world adversarial behavior.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)