Evaluating the Adversarial Robustness of CNNs Layer by Layer
Authors: Yaowen Wang
Daniel Cullina
Venue: Accepted by TMLR ()

Abstract: In order to measure the adversarial robustness of a feature extractor, Bhagoji et al. introduced a distance on example spaces measuring the minimum perturbation of a pair of examples to achieve identical feature extractor outputs. They related these distances to the best possible robust accuracy of any classifier using the feature extractor. By viewing initial layers of a neural network as a feature extractor, this provides a method of attributing adversarial vulnerability of the classifier as a whole to individual layers. However, this framework views any injective feature extractor as perfectly robust: any bad choices of feature representation can be undone by later layers. Thus the framework attributes all adversarial vulnerabilities to the layers that perform dimensionality reduction. Feature spaces at intermediate layers of convolutional neural networks are generally much larger than input spaces, so this methodology provides no information about the contributions of individual layers to the overall robustness of the network. We extend the framework to evaluate feature extractors with high-dimensional output spaces by composing them with a random linear projection to a lower dimensional space. This results in non-trivial information about the quality of the feature space representations for building an adversarial robust classifier.

Published in Transactions on Machine Learning Research (02/2026)
Evaluating the Adversarial Robustness of CNNs Layer by Layer
Yaowen Wang
gmw5684 @ppsu. edu
Deportment of Electrical Engineering The Pennsyleania State University
Daniel Cullina
cullina @psun edu
Department of Electrical Engineering
The Pennsylvania State University
Reviewed on OpenReview: openreveee. net/ net/forum? id= 2Ga9KzsaYB
Abstract
In order to mcasure the adversarial robustness of a feature extractor, Bhagoji et al. introduced a distance on example spaces measuring the minimum perturbation of a pair of examples to achieve identical feature extractor outputs. They related these distances to the best possible robust accuracy of any classifier using the feature extractor. By viewing initial layers of neural network a feature extractor, this provides method of attributing adversarial vulnerability of the classifier as whole to individual layers. However, this framework views any injective feature extractor as perfectly robust: any bad choices of feature representation can be undone by later layers. Thus the framework attributes all adversarial vulnerabilities to the layers that perform dimensionality reduction. Feature spaces at intermediate layers of convolutional neural networks are generally much larger than input spaces, this methodology provides information about the contributions of individual layers to the overall robustness of the network. We extend the framework to evaluate feature extractors with high-dimensional output spaces by composing them with random linear projection to lower dimensional space. This results in non-try non-trivial information about the quality of the feature space representations for building an adversarial robust classifier.
1
Introduction
Over the last several years, a line of work has characterized the optimal adversarial loss achievable for a particular data distribution and adversarial constraint specification (Bhagoji et al.. 2019 2021 Pydi & Jogi
2020
Trillos al. 2023 Dai et al. 2023 These works characterize the problem of minimizing adversarial loss over all possible classifier functions to some kind of optimal transport problem
the class conditional distributions. In order to set up this problem, one needs the distance structure of the data distribution with regard to the distance appearing in the adversarial constraint. These characterizations of optimal loss provide a lower bound for loss achievable using particular architecture and a benchmark performance level that trained classifier can be compared to. At a higher level, they provide an objective method determine whether a given adversarial constraint is reasonable to use with particular dataset.
More recently, this method has been extended to analyze the robustness of a feature extractor, equivalently the initial layers of food-forward network (Bhagoji et al., 2023). Just as the minimum loss on data distribution ion is minimization over all possible classifiers, the robustness of a feature extractor is a minimization over all classifiers using that feature extractor. The robustness of a feature extractor can be computed via a characterization that uses the distance structure of the data distribution using a specially defined distance that takes into account the behavior of the feature extractor. This allows the adversarial loss of the model be decomposed layer-by-layer: including layer of the model in the feature extractor decreases the size of

---

Published in Transactions on Machine Learning Research (02/2026)
the class of models using that feature extractor and thus increases the minimum loss. This increase can be considered the contribution of that layer to the overall adversarial loss of the model.
Unfortumately, for wide models, the decomposition tends to be uninformative. Layers are assigned none of the adversarial loss until dimensionality reduction to below the input space dimension occurs. Since the development of CNNs, state of the art architectures have used huge feature spaces. Dimensionality reduction has to occur eventually, but this methodology often results in the attribution of all the loss to the final layer.
In this paper, we propose variation on the feature extractor distance tailored to wide networks, equivalently feature extractors with high dimension feature spaces. We explore basic properties of this feature extractor distance and propose and implement an algorithı to approximately compute it for ReLU CNNs feature extractors. We use the algorithm to perform some initial exploration into the effects of architectural and training method decisions on layer-by-layer robustness.
As pre-trained models and the transfer learning paradigm are more widely used, analysis of these models as feature extractors rather than fully specified classifiers become more and more relevant. We illustrate the potential feasibility of the layer-by-layer approach to this task.
At this point, adversarial examples have been known to be a serions problem in neural network for over a decade (Szegedy et al. 2013 Goodfellow et al. 2015). Adversarial training has succeeded at mitigating this vulnerability to am extent, but is computationalyy expensive and does not achieve information information-theoretic theoretic limits on adversarial robustness (Bhagoji et al.) 20211. Certified methods give stronger guarantees, but are usually even more computationally expensive and limited in the scale of the adversarial budget.
We would like to provide new analysis tools to enable continued progress training adversarially robust models. With these (ools, we can better identify different behaviors in models as training method is varied. By localizing adversarial vulnerabilities within a model, we allow computational resources to be spent efficiently. Code used in our experiments is available
Contributions. We introduce a novel collision search algorithm that leverages gradient information to evaluate differentiable feature extractors. We provide a theoretical framework for understanding the robustness of feature extractors in high-dimensional feature spaces. We utilize random projections to transform feature extractors with high-dimensional feature spaces to representative low-dimensional versions whose robustness can be efficiently analyzed. From our experiments on convolutional networks, we find that different training methods produce significant variation in layer-by-layer robustness.
2 Related Work
Robustness of Feature Extractors. Our work primarily extends the theoretical framework established by Bhagoji et al. (2023), which links adversarial robustness to optimal transport distances. However, a significant. limitation of prior work is its reliance on combinatorial search methods that scale poorly with depth. Also, applies only to restricted activation types. We advance this framework in two key ways: First, we introduce gradient-based collision search algorithm that evaluates robustness for any diffcrentiable feature extractor, enabling efficient scaling to deep architectures supported by various differentiable aclivation functions, Second, we integrate random projections to handle high-dimensional feature spaces, overcoming the dimensionality constraints that limited previous theoretical evaluations.
Layer-wise Analysis and Model Utilization. Understanding how robustness and information flow evolve through network layers is an active area of research. Song et al. (2021) focused on optimizing layer-wise quantization under Lipschitz bounds, while Craig et al. (2023) derived bounds for information flow through subnetworks. More recently, Gavrikov et al. (2024) investigated how training schemes (including adversarial training) alter layer utilization, identifying which layers become critical for performance.
While Gavrikov et al. (024 focuses on functional criticality, our work provides geometric lower bound untargeted robustness via feature collisions. Unlike prior approaches that bound Lipschitz constants
http:://gtthhb.co////ahot////// rabusttnes
2

---

Published in Transactions on Machine Learning Research (02/2026)
analyze parameter utilization, our method identifies concrete collision points. This dual functionality allows us to not only measure vulnerability but also generate the specific ad adversarial examples that exploit it, offering a direct probe into the geometry of middle-layer representations.
Random Projections and Optimal Transport. Our of Gaussian random projections is philosophi- cally rooted in the Sliced Wasscrstein (SW) distance. Optimal adversarially robust classification is intimately
connected to optimal transport distances between class distributions (Bhagoji et al. 2019 Pydi & Jog 2020:
Trillos et al. 2023). However, estimating these distances suffers from the curse of dimensionality. SW distance
bypasses this by projecting high-dimensional measures onto lower-dimensional subspaces (Kolouri et al. 2018
Deshpande al. 2019 Bonneel al.) 2015 Nadjahi et al., 2021). Whereas SW methods typically average 1-D distances many slices, we employ higher-dimensional (e.g.; 10-D) random projection to approximate the optimal loss, balancing computational efficiency with the preservation of geometric structure.
3 Framework
In this section, we build toward the definition of our robustness measure by discussing key technical aspects of prior work. Section 3.1 characterizes the optimal adversarial cross-entropy loss in terms of a convex optimization problem with constraints described by a bipartite conflitt graph. Section 3.2 explains an earlier definition of robustness measure for feature extractors based on the optimal adversarial loss of classifier built on that feature extractor.
We consider the following adversarial classification scenario. (For comprehensive summary of notalion, please refer to Table in Appendix A.) There is an example space X' = R equipped with a distance metric d and a label space y = {0, 1..., k - 1}. A labeled example (x, y) is sampled from a distribution P over X' x y. An adversary selects a point ă E X such that d(x, S E. The classifier receives I, uses a classifier to produce probability vector h(ā) E P(Y), and incurs loss ((h(T), y)- The expected adversarial ((h(T),)) y)].
loss of h is L(e, P, =
3.1 Characterizing Optimal Adversarial Loss
We brielly summarize the characterization of optimal adversarial cross entropy loss for two-class disccete data distribution Bhagoji et al. (2021).
The distance structure of the data be summarized in a bipartite conflict graph (V,Ee) in which, the vertices V represent labeled examples, and the edges & indicate the presence of intersections under given attack budget e. When such an intersection exists, it signifies a fundamental limitation: no classifier can correctly classify both points in the adversarial setting. In the graph the left vertices are examples from class zero, the right vertices are examples from class one, and an edge (I,x²) present when d(xxx') s 2s. The polytope Pe C RVV defined by the following incqualities:
5 LL qs go (1,(),
For labeled example (ar, v) and soft classifier h X PO)), Mxxy is the probability that h assigns to the correct class. Bhagoji et al. show that there is an h such that zuf h(x)y 2 q(n,)) for each labeled example (x. in the support of the data distribution if and only if Pe. Thus P. is the space of achievable correct-classffication ptw.ssssfiiaatooooooooaillllllli probability vectors.
Using this characterization of the correct classification vectors, we obtain a convex optimization problem for the minimum adversarial loss:
(c, min i(e, P.h) min oEP.
Pv log 1 qv Σ
NEV
The optimal loss represents the lower bound on the loss that any possible classifier can achieve when attempting to correctly classify an ad aeresarii example under the current perturbation.
3

---

Published in Transactions on Machine Learning Research (02/2026)
3.2 A Previous Approach to Feature Extractor Robustness
This analysis has been extended to multi-layer classifiers. Consider a classifier h: R - PP())) that is the composition of & layers: h = softmax hę he-1 o h1, where h4 : IR + R and ny = k = (2). We can consider its initial j layers fj R + R" f = h o... o h1 as a feature extractor. The key idca of Bhagoji et al. is to evaluate any feature extractor f via the set of all classifiers that use f as their initial layers: Hz {g f such that g: R - P())) (Bhagoji et al. 2023). To measure the adversarial robustness of f, they consider the best achievable ad adversarial loss over this family: (s = infREn, L(s, P.).
This definition has some nice properties. As more layers are considered to be part of the feature extractor, the robustness can only decrease because Hgos S Hy. The set of all classifiers is Hia so the identity feature extractor has perfect robustness, i.c. no reduction from imput space loss lower bound.
However, there is key weakness to the definition: any injective feature extractor achieves perfect adversarial robustness. A feature extractor f Rn - R will typically be injective close enough to only introduce a negligible amount of loss. By this definition, only feature extractors that perform significant dimensionality reduction typically cause loss of robustness.
Colliding adversarial examples and feature layer distance Let f be a feature extractor and let (a, v) and (z', v/) be labeled example from different classes (y # y'). A pair of adversarial examples collide when the perturbations cause the examples to be mapped to the same point in the feature space: &. and H' satisfy d(r,
i) S e, d(x', ,,) S and ((ī) = ((").. This guarantees a classification error for at least one example for any backend {(f())) = ((f(")). The minimum adversarial budget required to find inputs that collide in the feature space defines a distance between examples:
df(x,x')=
inf max(d(ar 3), d(x', (,a%)(3)=f(1))
Computing the Robustness Measure The distance dy can be used in place of the input distance d to build conflict graph and characterize optimal adversarial loss over Hy. Define & the edge set of the conflict graph for dy, analogously to Pe: Ef = ((x,x') ) dy (x, x²) s e). Define Pfs C RV is defined analogously to Pe, but with constraints coming from E
Bhagoji et al. show that for discrete distributions P, (E, can be characterized similarly to (E, P):
P.f) = inf P,h) = min gEP,
Σ
Po lo 1 go
NEN,
BEV
If Efe is known, then (r P)f) can be efficiently computed by solving convex optimization problem.
4 Measuring Robustness of High-dimensional Feature Extractor
To evaluate feature extractor f f Re RR; we consider (, P, f), where R - Rm is random linear function: a(x) Ar, A E Rmxn is a matrix with i..d. standard Gaussian entries. Note that because is random function, d (x,,2²) and (e, o f' random variables.
Discussion An injective leature extractor does not commit to any aspect of the behavior of the classifier using it: subsequent layers of the classifier could invert it. However, a feature extractor causes some classifiers extending it to have simpler representations than others. We would like to measure the robustness of a "typical" classifier extending f and we use the composition of a random dimension-reduccnng linear function with f as a proxy for this that is computationally tractable analyze. The random linear function is comparable randomly initialized but untrained classifier network using the features.
4.1 Properties of the Robustness Measure
Now we discuss key properties of the feature extractor distance doof and the robustness measure ī P.o
4

---

Published in Transactions on Machine Learning Research (02/2026)
Invariance of d to scaling of f For any c + (e, cf.) = P.))), collision A for random linear layer A is still collision after scaling by constant. This also means that the scale of the distribution of A does not affect our definitions.
Good Feature Extractor
Bad Feature Extractor
Rotation invariance The distribution of ran- dom linear maps is rotation invariant, so any ro- talion of the feature space leaves all distances un- changed. When m I nj, the random matrix A has rank with probability one. It has singular value decomposition A UEVT, where V R= and U R= are orthogonal and E E Rmxx is diagonal and invertible. Then UEV* f()) JEV") if and only if ((z) = (i²'.
Figure 1: Random non-lineat level sets for composi- tions of a random linear function with good and bad feature extractors. The lines represent the level sets of random linear function composed with the feature extractor. The bad feature extractor contains all of the features in the good extractor plus many more. Most of these are low quality so a random linear combination has low robustness.
Thus using ii.d. Gaussian matrices results in the same distance as a uniform distribution over or- thogonal projections to dimension-m subspace.
Invertible function non-invariance On the other hand, applying generic invertible lincar transformation to the feature space R does affect the distribution of the distance: it is not true that [= bo [e P. f). Applying the linear transformation to fax is equivalent to applying the inverse to the random matrix A, which changes its distribution. All linear maps still appear, but with skewed densities.
Adding, removing, rescaling features While d does not depend on the absolute scaling of features, it does depend the relative scaling, It is easier for the adversary to satisfy AS Af(ā') by modifying the examples in ways that change large magnitude features.
If we modify a feature extractor f to add an additional feature, giving f', this could increase the distance between x and x' or decrease it. Adding constant feature leaves the distance unchanged.
Figure illlustrates how a "good" feature extractor (where level sets separated) can become "bad" one (where level sets are clui cluttered), when enough weak features are added.
Specializing to & In the remainder this sec- tion and in our experiments, we use & distance as our input space distance.
3
4.2 Linear Feature Extractors
Suppose that f is affine linear, so f(x) Hr + k, and that (ank(H) = r, so the singular value decomposition of Rrxr Then Bhagoji al. observe that the optimal adversarial examples f 5 5 and H' = x' - where d /VVT(z (z z'). The perturbation is the projection of half the difference of examples onto the orthogonal complement of the nullspace of H: the adversary should not allocate any of their budget to moving within the level sets of Thus (x 2') [VV (x- 2'Il. The left part of Figure illustrates this.
Figure 2: Visualization of dy (z,,x) for two feature extractors f R2 - Rd, one linear and one piecewise- linear. Blue lines are level sets of f. Each of the balls has radius dz x'). The level set of the colliding adversarial examples is highlighted. These are not classifiers and the highlighted line is not decision boundary. When the feature space is more than one dimension smaller than the input space, level sets do not disconnect the input space.
H = UEVT where Σ
5

---

Published in Transactions on Machine Learning Research (02/2026)
Proposition 1 (Informal version). Let f(x) ALæ, where L E R is fized linear feature eztractor and the entries of A E Rmxx are i.i.d. standard Gaussian. Let v4 TI ||4|| is the lz operator norm. Then T4 a variation of stable rank for L. If < mn < vTA, then dz(2, x') concentrates tightly around y/mli(rs=>\) 2|
LLIl where [|L||a is a Schatten p-norm and
Precise statement and proof are in Appendix D Note that [L - =x'll is just the distance between the feature representations of æ and z'. If we were only interested in linear layers, computing dash would not be justified: [ Il)l and || are much more straightforward to compute, and not random! However, these norms do not obviously generalize to a nonlinear h, which would have different linear approximations at different points in the input space, while dash does still make sense. When evaluating nonlinear h, we would efficiently and accurately estimate it by sampling.
still like to pick that dash concentrates that
Input space calculation The special case of (z( x²) for a random linear function a is more intuitive. This corresponds to L = I E R=oxne and ||]| = no- The orthogonal complement of the millspace of A E Rmxn is a uniformly distributed dimension-m subspace. So da(x, x²)² has mean ing da (x, concentrates around
m || x'13 and
ymm || x x'll2 2,7o
when m is large enough.
4.3 Relationship between distances at different layers
Now we compare daof, (x, x²) and deo x'). As we noted before, d/, (2,x²) 2 dfo+1 (2,, x'): adding an additional layer can only decrease distances by introducing new collisions between inputs. In contrast, daofz- could be either larger or smaller than deef;- Including an additional layer hy+1 to the feature extractor introduces new possibilities for collisions, which reduces robustness, but it also reweights the existing feature information in way that may reduce the effect of weak features. Figure 3 presents paired adversarial examples identified for the 7-layer model. We observe that as the feature extractor becomes deeper, the average adversarial distance daosi (I, x²) increases as well.
Original image
Layer
Layer
Layer 3
Layer
Layer
Layer
Layer 7
3
3
3
3
3
3
3
Distance: 0.000
Distance: 1.357
Distance: 2.008
Distance: 2.065
Distance: 2.651
Distance: 3.055
Distance: 3,231
Distance: 3.623
7
7
7
7
7
7
7
7
Figure 3: Colliding adversarial examples and their & perturbation distance for one pair across different layers.
4.4 Varying the output dimension of the random projection
As we vary the output dimension m of the random lincar function, the distance daof reveal different information about the structure of the feal ure space representation. At one extreme, if m 2 nj. then the linear map a is injective with probability one and (x, x') = daaf (x, If m = 1, we roughly learn how much of the feature representation usable by a lincar classifier. By using a single fixed m to analyze multiple layers with different feature space dimensions n,, we climinate some of the effects of this dimension variation on measured robustness. For most of our experiments, we use 10 to facilitate comparison with the logits.

---

Published in Transactions on Machine Learning Research (02/2026)
5
Algorithm for approximating feature-layer distances
When we use l distance to constrain our adversary, for f R - Rd, dj (x,,²' bocomes
min || - = || 2 subject to f(ī) f( E.
This is slight relaxation: to get a smoother problem we have replaced the original objective max(]āē æ||2, ||ă 22)12) with the sum of squares. Now present the algorithm that we use to approximately solve this optimization problem.
Our algorithm is related to a method used to compute distance from an example to the decision boundary of a classifer. The linear approximated decision hyperplane is used in DeepFool (Mooavii-Dezfooli et al., 2016), and the FAB attack paper (Croce & Hein 2020) employs convex combinations to make the solution closer to the original point.
A Algorithm Collision Search Algorithm
Require: Original points z; x'; initial step a E (0, 1); decay rate y E (0, 1); maximum number of iteration Nmaxi convergence threshold e. Initialize z00) (x x2)/2, g0 (x + x2)/2, for i == O... Niax 1 do
3:
Compute Jacobians, build collision subspace:
H4 E f f(z(1) + (E - (5 0
Convex combination and projection:
(alit)) ( proj ((1 - (0)) ((()) + (x, H.)
Break converged:
- -(((t))) <e and (I|=() x=13 -1101(0) x2113) ((1 x:12 -1
<
6:
((+1)) = (0)) x 7.
7: end for
Return colliding adversarial examples g((+1), ((((+))
The algorithm can be viewed as approximate version of projected gradient descent. Because f is nonconvex piccewise-linet linear function, it is intractable to project onto the true constraint set f(ū) = f(0). Instead, we locally approximate the constraint set as an affine subspace. The derivative of (ž, + f(i) f(i') at (ž, F') is the matrix (Df -Df(3') e mm22no where D represents the Jacobian matrix the gradient. The affine subspace on which the linear approximation zero has dimension at least 2ng - m. contrast. to related algorithms
compute minimum distance to the classification boundary, the subspace usually has codimension greater than one. i.c. is not hyperplane. Thus projection onto the subspace requires to solve linear system and it is preferable to choose m to be not too large. Details are shown in Appendix B.
We know that i = E' implies f(ū) = f(i'), so we initialize è and V' by minimizing the objective subject to this constraint. Thus we set z(0) = (0(0) = (x + 22)/2, the midpoint of the original examples. We found that starting with points satisfying the collision condition worked much better than any algorithm initialized at the original examples (z(0) and (0)) x').
5.1 Convex Combination Projection
We address the NP-hard collision-search problem (Katz et al.) 2017) via a simple iterative method that alternates between convex combination and projection onto locally lincarized subspace. At iteration i, this subspace is Hd. If the subspace were the true collision manifold, then the minimizer would be proj((z, 2²), H.). However, Hi is only an accurate approximation locally, we improve the objective by mixing the projection
7

---

Published in Transactions on Machine Learning Research (02/2026)
(010)
2010
(2,
H1
Ho
(0(00) 2010)
(50)) (o0))
(x(2) 5112)
H*
(a(2), 5'(21)
(+0), pra))
(i x(*)
(ar, x²)
(a) First two steps. Initialize at (Ia, 26) and project orthogonally onto the first linearized collision sub- space Ho to obtain (z(1), _((1), Relincarize to form Hi and project again to get (,(9),(()) (())). Red markers indicate the feet of the projections.
(b) Convergence of the linearize- projectt-reettttto scheme. The true collision set C curved. At iteration i, we lincarize at ( ,0()) to obtain the affine subspace He and convex combination with (x, project to (2(+1) ,,(+1)). The iterates approach the low point (#',"") on C; H denotes the limiting linearization the solution.
Figure 4: Geometry of our linearize project algorithm on the collision manifold: (a) initialization and the first two projections; (b) iterative convergence toward the optimal point on the true collision set.
of the current estimate (,(0,x70) with the projection of original point (x, xx,x"):
(xli+1), (10H),,{((+1)) (1 (0) proj (1T10, ,((5)), Hi) + a () proj ((x, Hi) = =proj ((1 - a (z.10 _((0) + a (I,, Hi)
The second expression for the itcrate holds bocause the projection is affine. Using this saves computation by performing the convex combination first.
Since the midpoint [ + x') is itself perfect collision in function space, initialize search there and compute its Jacobian to define the first affine subspace. the very first projection step, we set a 1, so that (ı(1) (r, Ho), using both original points to project onto Ho- Thereafter, gradually reduce (9 to introduce bias toward the original point This diminishing bias keeps every iterate within a controlled neighborhood of a feasible input, preventing large jumps that might violate constraints or break the validity of the local linear approximation. Figure 4 shows the convex- convex-combination and projection operation in first two step as well as the convergent case in which the initial point x exactly projects to the current iteration point z(()
The algorithm proposed in Bhagoji et al. (2023) is tailored to ReLU networks: it greedily searches ReLU activation patterns to formulate sequence of linear programs. However, network depth increases when non-ReLU activations used, this combinatorial approach becomes intractable. In contrast, our Jacobian- subspace method sidesteps these limitations by relying solely on gradient information. By constructing an affine subspace from the model's local Jacobians, we can handle differentiable activation functions and deep architectures efficiently.
5.2 Computation and Complexity Analysis
Our algorithm alternates between (i) computing local Jacobians to linearize the collision constraint and (ii) projecting onto the resulting affine subspace.
Time Complexity. The computational cost generate a collision pair scales O (N T (m Cop + m²no)). where N is the number of random projections, T is the number of iterations, no is the input dimension, and is the projection dimension (default 10). Here, Cop represents the cost of standard backpropagation
8

---

Published in Transactions on Machine Learning Research (02/2026)
pass through the feature extractor. The term Cup reflects the computation of the Jacobian (via vector-Jacobian products), while n²nno accounts for solving the linear system for the projection. In practice, with small m, the runtime is dominated by the backpropagation steps.
Memory Complexity. Peak memory usage is determined by the storage of intermediate activations required for gradient computation. This scales linearly with O(Batch x Depth x Resolution). For our 7-layer CNN on CIFAR-10, we can process batches of 100 pairs on 40GB GPU; however, for deeper architectures like WideResNet or higher-resolution inputs, the batch size must be reduced (e.g., to 15) to remain within VRAM limits.
6 Experiments
In this section, we test several theories and evaluate the robustness of convolutional models layer by layer the MNIST (LeCun et al., 1998) and CIFAR-10 (Krizhevsky & Hinton 2009) datasets. Since MNIST is a relatively simple dataset, it allows us to easily explore robustness properties. And the MNIST dataset highlights a larger robustness gap between layers and provides a range values for testing. For our baseline, we use an adversarially trained model on MNIST with projected gradient descent (PGD) (Madry et al. 2017 and set perturbation budget =
To design the experiment, we selected 100 examples from two classes in the training dataset for evaluation For the robustness analysis, we focus fixed binary class pairs to keep the setup simple and reproducible. On MNIST and Cifar-10 we use class 3 vs. 7. For each dalasel we randomly sample $ = 100 training examples per class and evaluate our metric on all cross-class pairs, resulting in S = 10000 pairs. Since our collision-search algorithm is run on every pair and for multiple random projections, the overall cost scales quadratically in S, and larger subsets quickly become computationally prohibitive. In Appendix Section C, we show that increasing the subset to 200 and changing the class pair or the sampled subset yields qualitatively similar layer-wise robustness curves, indicating that our conclusions are robust to these choices.
Algorithm Optimal Loss Calculation via Conflict Graph
Choose feature extractor fy ; "o R"
Specify the number of random projections N, and projection dimension
3: for 1 to N do
4: Generate random projection matrix A; E Rmx,,, with elements drawn i.i.d. from W(O0
5: Build new feature extractor (a, o fz)(2) = (z)
6: for example pairs (x,2²) do
7: Approximate da (x, n') using Algorithm
end for
for Emin ... Emax do
10: Build conflict graph (V, E2), where:
11:
V is the set of data, v, x VV
12: Find Lopi(E) using Algorithm from Bhagoji et al. (021
Eg = (xxxx') fa 2x,2²) e
13:
14: end for
15:
end for
16: Return List of averaged optimal losses Lopt
For the sclection of CNN models, we trained three different architectures, following the work in (Gowal et al. 2019), where all layers utilize ReLU activation functions. We employed two robust training methods: PGD adversarial training, and TRADES adversarial training (Zhang et al.) 2019). Each of these methods was applied with robustness constraints based on the 12 norm. For all models, we used a cosine learning rate schodule [Loshchilov Hutter 2016) with an initial learning rate set to 0.01 to ensure consistency across training approaches.
Optimal Loss
We use optimal loss of f) defined Section 3.2 to represent the robustness of the feature extractor. Recall that higher loss represents lower robustness performance. Algorithm 2 computes
9

---

Published in Transactions on Machine Learning Research (02/2026)
the optimal loss using conflict graph-based procedure. The process begins by selecting feature extractor and generating sequence of random matrices with i.i.d. Gaussian entries. Each random matrix A; is combined with the feature extractor fj to create lower-dimensional feature mapping I + Ar), making the representation suitable for robustness analysis. The algorithm then evaluates all cxample pairs to estimate pairwise distances and identifies collisions according to a perturbation--aser threshold e. These collisions define the edges a conflict graph, where nodes represent examples and edges connect pairs within the specified perturbation distance. For each adversarial budget the algorithm invokes Algorithm 1 from [hhaooj] et al. 2021 to compute the optimal loss over the confiict graph. Finally, the results are averaged over multiple random matrices to produce the final optimal loss estimates.
6.1 Validating the robustness metric
To evaluate the true robustness loss on both the training and test datascts, we additionally compare our based robustness metric,
collision-based loss to the FAB attack, which provides a standard rather than relying only on the training-time attack loss. FAB computes, for single input, the minimum perturbation required to change its predicted label, whereas our collision-based procedure at the logit layer jointly moves pair (z, x²) from different classes to point where their logits coincide.
In our setup, the worst-case loss is log(2), indicating that even when collisions occur, we still have a 50% chance of correctly classifying a randomly chosen element of the pair. As shown in the figure 5a when the value of increases, the adversarial loss measured by FAB rises, while our loss saturates its maximum value (2). Thus, the FAB loss remains consistently higher than our dilisioonbaseed loss, which is expected because FAB directly optimizes misclasiffiaation of single input, whercas metric is designed to capture how easily feature representations of different classes can be brought to coincide.
Altack.lcss
Aotack
Optainanal Las
optiaal col Losa
2.5
0.6
Cood 00000 53
2.4
0.4
5.2
0.1
5.0
0.4
0.0
1.2
1.5
2.0
2.4
2.5
3.2
3.5
0.0
0.4
0.8
(a) MNIST Medium CNN (PGD-2).
(b) CIFAR-10 Medium CNN (PGD-0.5).
Figure 5: Comparison of FAB Attack Loss Our Optimal Collision Loss.
6.2
Varying the dimension of the projected features
3.7
3.6
07
Inouc
Riti Layer
Layeri Project on
Layerá Project
Choss Gott00 10
Second Layer
Thind Layer
2.4
Layerd
Layer1
0.4
0.2
0.2
5.0
30400000000
0.
3.2
Figure 6: MNIST Medium CNN (PGD-2): First conv layer with varying random projection dimen- sions (m + 784).
Figure 7: Medium CNN optimal loss using the method in Bhagoji et al. 2023
Since the output dimension of the convolution is higher than the input, when no random projection applied, the only collisions occur when the pairs identical. Therefore, the distance between pairs reflects the exact.
10

---

Published in Transactions on Machine Learning Research (02/2026)
input distance, shown by the brown line in the figure 6 When projecting to higher-dimensiona. space, we introduce less noise while preserving more useful feature information. As result, the dimension increase, and the loss decreases.
The distances scale roughly with ym, but not exactly, which reflects the somewhat different information about feature structure accessed by different choices of m.
6.3 Baseline layer-by-layer robustness results
When applying the method of Bhagoji et al. (003)), the search algorithm cannot find any nontrivial collisions because none of the initial linear layers have an output dimension smaller than the example space dimension. The robustness measure for these initial layers remains essentially the same as at the input level. This is illustrated
Layer dimensions for all models investigate are listed in Table 3 in Appendix [
In Figures x and we have baseline layer-by-layer results for MNIST and CIFARIO respectively. As we move through the layers, robustness increases, suggesting that low-quality features are being pruned. This effect is far outweighing any reductions in robustness due to the introduction of new opportunities for collisions as the feature dimension spaces decrease. Clearly, the dimensions of the feature spaces have significant impact on the robustness measure, but they do not explain the curves by themselves. For both datasets, there is a relatively large improvement from layer one to layer two and from layer four to layer five. These layers perform dimensionality reduction by a factors of four and six, which are larger than reductions adjacent layers. However, from layer six to the logits, we do not observe much robustness improvement despite dimensionality reduction from 512 to 10.
As we move through the layers, we observe that the optimal loss decreases, indicating that the model gradually learns more about the data's robustness, and the features become increasingly separable. Notably, there is a distinct gap between the convolutional layers and the fully connected layers. Additionally, the random projection appears to perform similarly to the learned matrix. The last few linear layers, however, seem to contribute little to the overall robustness.
0.4
0.4
Second
Layer
5.2
Fourth Fith FC Laylr
0.2
Fourh rC Layer
5.0
004000000000000
Logt:
0.0
0.5
2.0
Figure 8: MNIST Medium CNN Model trained
Figure 9: CIFAR-10 Modium CNN Model trainod
with PGD-2: All layers robustness.
with PGD-0.5: All layer robustness.
6.4 Effects of architecture on layer- robustness
When comparing the layer-wise robustness of the small CNN in figure 10 and the Large CNN in figure 11 on MNIST, we observe that robustness does not change uniformly throughout the models. The same observation shows in robustness on CIFAR in figure 12 Although the small CNN exhibits lower overall robustness, its first convolutional layer appears to perform better, and its second layer achieves similar level of robustness to that of the medium CNN.
6.5 Effects of training method on layer-by-layer robustness
In Figures we examine the medium CNN models trained with methods other than plain PGD. These be compared to Figure TRADES have layer-one robustness cliffs at lower adversarial budgets than
11

---

Published in Transactions on Machine Learning Research (02/2026)
Figure 10: MNIST Small CNN Model trained with PGD-2: All layer robustness.
Figure 11: MNIST Large CNN
Figure 12: CIFAR-10 Large CNN Model trained with PGD-0.5: All layer robustness.
Model trained with PGD-2: All layer robustness.
PGD. Besides, TRADES achieves more separation between fully-conneeted layer curves and more gradual growth of those curves.
bey
Dan PC
Figure 15: CIFAR-10 Medium CNN Model trained with TRADES-0.5; All layer robust- ness.
Figure 13: MNIST Medium CNN Model with benign training: All layer robustness.
Figure 14: MNIST Medium CNN Model trained with TRADES-2: All layer robustness.
Relationship between intermediate layer robustness and test robust accuracy In Figures 17a and 17b] we see the Lest. robust accuracy of the models. As expected, benign training achieves very limited robustness, TRADES significantly outperforms plain PGD. It is unclear to us whether the layer-one behavior of TRADES should be viewed as an undesirable side-effect that suggests room for improvement, or an indication that layer-one robustness curves not appropriate objectives.
6.6 Conclusion
An interesting finding is that, for models exhibiting greater robustness under the norm-based attack, the optimal loss the logit layer remains relatively low, which indicates high overall robust yet the first convolutional layer does not consistently follow this trend. fact, this layer often learns less robust features even when the model a whole is quite robust. This suggests that prioritizing the robustness of the first convolutional layer early in the training process is a potential avenue for improving a model's robustness.
At the same time, our experiments show that robustness does not grow uniformly across layers or blocks or depend on feature space dimension a simple manner. Some adjacent layers have similar robustness, and robustness levels often stabilize before the final linear layer. Of course, this behavior varies with different datasets, training methods, and architectures.
6.7 Limitations and Future Work.
Our method requires Jacobian computations for input pairs, imposing a memory overhead that scales lincarly with input resolution and network depth. This tradc-off necessitates smaller batch sizes for very deep or high-resolution models compared to standard training. Additionally, while our collision search is algorithmically compatible with various constraints, our curent theoretical concentration bounds are derived for the rotationally invariant I2 geometry. Developing rigorous thcoretical guarantoes for the 4o norm involves addressing non-trivial challenges regarding the geometry of intersection in high dimensions, which constitutes key direction for future work.
12

---

Published in Transactions on Machine Learning Research (02/2026)
Broader Impact Statement
Analysis of feature extractor robustness can be used to detect weaknesses in classifiers and localize these to particular layers. This is intended to serve the goal building more robust and trustworthy models and is likely to be most useful for these purposes. For example, weak layers can be retrained or rearchitected. However, there is some potential for any method of assessing robustness to also be useful for the design and implementation of attacks: locating weaknesses in classifiers so they can be exploited rather than fixed. The potential for misuse is mitigated by the fact that our collision finding algorithm has been designed for use in the white-box setting.
Acknowledgments
This research used the Delta advanced computing and data resource which is supported by the National Science Foundation (award OAC 2005572) and the State of Illinois. Delta is a joint effort of the University of Illinois Urbana-Champaign and its National Center for Supercomputing Applications.
This work used Delta CPU and Delta GPU at NCSA through allocation CIS240583 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services Support (ACCESS) program, which is supported by U.S. National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296.
Boerner et al. 2023
References
Arjun Nitin Bhagoji, Daniel Cullina, and Prateck Mittal. Lower bounds on adversarial robustness from optimal transport. In Advances in Neural Information Processing Systems, pp. 7496 7508, 2019.
Arjun Nitin Bhagoji, Daniel Cullina, Vikash Sehwag, and Prateek Mittal. Lower bounds cross-entropy loss in the presence of test-time adversaries. Tn International Conference on Machine Learning, pp. 863-873.
PMLR, July 2021.
Arjun Nitin Bhagoji, Daniel Cullina, and Ben Y Zhao. A theoretical perspective on the robustness of feature extractors. In The Second Workshop on New Frontãers in Adversarial Machine Learning, 2023.
Timothy J. Boerner, Stephen Deems, Thomas R. Furlani, Shelley L. Knuth, and John Towns. Access: Advancing innovation: Nsf's advanced cyberinfrastructure coordination ecosystem: Services & support. In Practice and Experience in Advanced Research Computing 2023: Computing for the Common Good, PEARC '23, pp. 173-176, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 450345099. doi: 10.1145, 10.1145/3369951.1 (3559951.33577777 URL httpt::///o:ot http:::///oooor/////// 1144//566/9991.11 3597559
Nicolas Bonneel, Julien Rabin, Gabriel Peyré, and Hanspeter Pfister. Sliced and radon wasserstein barycenters of mcasures. Journal of Mathematical Imaging and Vision, 51 51(1):22-45, 2015.
Jared Craig, Jayson Andle, Thomas S. Nowak, and Sahand Y. Sekeh. A theoretical perspective on subnetwork contributions to adversarial robustness. arXiv preprint arXiv:2307.0380, 2023.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary allack. In International Conference on Machine Learningg pp. 2196 2205. PMLR, November 2020.
Sihui Dai, Wenxin Ding, Arjun Nitin Bhagoji, Daniel Cullina, Heather Zheng, Ben Zhao, and Prateek Mittal. Charactcrizing the optimal (-1 loss for multi-class classification with a test-time attacker. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 49467-49489. Curran Associates, Inc., 2023.
Ishan Deshpande, Ziyu Hu, Liyuan Sun, Nicholas Kolkin, Justin Solomon, Changil Kim, Sanja Fidler, Deva Ramanan, and Krishna Singh. Max-sliced wasserstein distance and its use for gans. In Proceedings of the IRFF/CVF Conference on Computer Vision and Pattern Recogrition,, pp. 10648 10656, 2019.
13

---

Published in Transactions on Machine Learning Research (02/2026)
Paul Gavrikov, Shashank Agnihotri, Margret Keuper, and Janis Keuper. How do training methods influence the utilization of vision models? In NewrIPS 2024 Workshop on Interpretable AI: Past, Present and Future, 2024. URL
Tan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples.
In International Conference on Learning Representations, 2015.
Sven Gowal, Krishnamurthy (Dj) Dvijotham, Robert Stanforth, Jonathan Uesato, Relja Arandjelovic, Brendan O'Donoghue, and Pushmect Kohli. Scalable verified training for provably robust image classification. In Proceedings of the IEEE/CV /CFF International Conference on Computer Vision (ICCV), pp. 4842 4881, 2019.
Guy Katz, Clark Barrelt, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. Reluplex: An efficient smn solver for verifying deep neural networks. In Computer Aided Verification (CAV), volume 10426 of Lecture Notes Computer Science, pp. 97-117. Springer, 2017. doi: 10.10077998-3--19-6833
Soheil Kolouri, Phillip E. Pope, Charles E. Martin, and Gustavo K. Rohde. Sliced wasserstein auto-encoders. In International Conference on Learning Representations, 2018.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Universily of Toronto, 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document. recognition. Proceedings of the IEEE, 86(11):2278 2324, 1998.
Nya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arxiv preprinl DrXiv:1608.03900 2016.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt,, Dimitris Tsipra,, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint or 1rrxix11760000 2017.
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference Computer Vision and Pottern Recognition, pp. 2574-2582, 2016.
Kimia Nadjahi, levgen Redko, Guillaume Bouchard, Mare Sebban, and Nicolas Courty. Fast approximation of wissed-wasserstein distances using concentration of measure. In Advances in Neural Information Processing Systems, volume 34, pp. 22238 22250,, 2021.
Muni Sreenivas Pydi and Varun Jog. Adversarial risk via optimal transport and optimal couplings. Tn Proceedings of the 37th International Conference on Machine Learning, pp. 7814-7823, 2020.
Chulin Song, Rajeev Ranjan, and Haoliang Li. A layer-wise adversarial-awarg quantization optimization for improving robustness. arXiv preprint arXiv:21 10. 12908, 2021.
Christian Szegedy, Wojciech Zaremba, 1lya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint ar arXim:1312.6199, 2013.
Nicolás Garcia Trillos, Matt Jacobs, and Jakwang Kim. The multimarginal optimal transport formulation of adversarial multiclass classification. Journal of Machine Learning Research, 24(45):1 1-6,1 56, 2023.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off wetw robustness and accuracy. In International Conference on Machine Learning, pp. PMLR, 2019.
14

---

Published in Transactions on Machine Learning Research (02/2026)
A Symbol Description
Table 1: Summary of commonly used notation.
Symbol
Description
X C Ro
Input space with dimension Roo-
y = {o..... 1)
Label space for k classes.
(z, y) P
A labeled example drawn from data distribution P.
d(x,x')
Distance metric in the input space (typically 82).
Adversarial perturbation budget.
f R -> m
A feature extractor comprising the first j layers of network.
h: X + A k-1
A full classifier mapping inputs to class probabilities.
Hf
The set of all classifiers that use f as their feature extractor. A random projection matrix with i.idd Gaussian entries.
A E Rmxxy
a: C: + R.
Random linear projection function, defined as a(z) Az. The minimum imput distance required to cause collision in feature space f. The feature-layer distance after random projection.
22())²)
daos (2,2))
P.f)
The optimal adversarial loss (lower bound) achiovable by any classifier using feature extractor f-
B Distance Calculation
B.1 Projection operation
The projection onto a collision affine subspace in the norm can formulated as the following optimization problem:
8
min
8
Subject to:
[Df(x'), -Df(x"))
88 = -f("))-f(r) f(x')
In practice, we solve this optimization problem using Py PoTch'sss least squares solver in code.
For the first linear layer, the problem of finding the distance can be framed as solving the following optimization problem:
8
min
8'
Subject to:
AL(x + 5) AL(w' +8)
where A a random projection matrix, and represents the weights of the first linear layer. The optimal is the projection of (x x')/2 onto the orthogonal complement of the nullspace of AL. Assuming AL has full row rank,
||61|2 4 (x - " AL(x -z))
15

---

Published in Transactions on Machine Learning Research (02/2026)
When A E R (so it is single row drawn from ii.d. Gaussian distribution), This simplifies
|012 |AL(x x')l2
|
-n (b:
4||L1/2 4 B
4 R (b,0,))
where b; are random Gaussians, d, are singular values of L, and wi are the coefficients of (x - x²) is the basis of right singular vectors. Thus w, measures the usefulness particular singular vector for distinguishing a from x'. Weak features contribute proportionally more the denominator. By assigning these smaller singular values (or even zero), ||o|| could be increased.
distribution one pair different layer
layer
12
layer
10
Number
2.0
2.5
Value
Figure 16: One pair distance under different random projection matrix
B.2 Concentration
We measured the distance between one pair in the first and sccond layers with 100 different random gaussian projection matrix, and the distribution of these dist is presented in the histogram. The figure 16 shows that the distances are concentrated, clearly highlighting the differing distributions between the two layers.
C
Experiments
C.1 Robust accuracy of the models
10
Medim om with P
Nodium crn PGD
Medum cm with IMAD5--2
Modium cmn with TRADES
Medium om with Berégn
Lage om with PGD-2
Smai con PGD-2
Medium cmn Beniyn
Conwentiny
Large POD-2
0000000000 400000
0.6
0.4
0.2
12
0.0
0.4
08
(a) MNIST
(b) CIFAR-10.
Figure 17: Comparison of AutoAttack test loss for MNIST and CIFAR-10 models.
C.2 Runtime and Convergence
To demonstrate the computational leasibility our approach, we report the wall-clock runtimes for generating collision pairs in Table [ Measurements were conducted on a single NVIDIA A100 GPU.
As shown in the table, the computational cost increases marginally with network depth, reflecting the accumulation of backpropagation steps required to compute the Jacobian for deeper feature layers. However,
16

---

Published in Transactions on Machine Learning Research (02/2026)
even for the final layers, the runtime remains approximately 1 second per batch of 100 pairs, confirming that the method is efficient enough for extensive layer-wise analysis.
Regarding the stopping criterion for Algorithm 1, we employ a strict convergence threshold to ensure the stability of the found collisions. The search terminates when the update magnitude for the adversarial pair becomes negligible between iterations and +
||(++1) 112 || - 111 < 10
Layer Time (seconds)
Datasel
Model
Res.
Batch
L1
L2
L3
L4
L5
L6
L7
MNIST
Medium- C-NNum--NNN
28
100
0.66
0.74
0.76
0.77
0.76
0.78
0.82
CIFAR-10 Medium-CNN
32
100
0.82
0.90
0.95
1.04
1.02
1.03
1.06
Table 2: Average wall-clock time to generate a batch of collision pairs (100 pairs) across layers L1-L7
C.3
Standard Deviation
For two experiments, we have added error bars representing the standard deviation due to the random projection matrix for 7 layer medium CNN model. This was found by repeating with 15 random projection matrices, Figure 18 shows that the variance is low, confirming the metric's stability.
8.6
0.5
0.5
2.4
0.4
5-2
0.2
Fourth
Fourth
3.0
0.0
Suih
0.0
1.6
2.0
(a) MNIST Medium CNN PGD-2: Standard deviation
(b) CIFAR-10 Medium CNN PGD- 0.5: Standard deviation across 15 projections.
across 15 projections.
The proposed optimal loss curves show very low variance across different random projection initializations, confirming the reliability of the metric.
Figure 18: Stability Analysis.
C.4 Activation function and batchnorm
9.7
2.5
0.5
Seeand Third
3.3
Third Ricua
Second
first
0.3
Fiutt Rala
Secend Detu
The Batchnoem
0.2
0.1
Second Detshnom
Third Rulu
Third Batchnom
0.0
0.5
1.
2.4
2.7
0.2
Figure 19: The ReLU performance
Figure 20: The Batchnorm performance
We evaluated the effect of including the ReLU activation functions as part of the feature extractor. As the ReLU is non-injective, enables new types of input collisions and could potentially reduce robustness because of this. On the other hand, ReLUs can zero weak features and improve robustness. The results in figure 19 indicate that the ReLU activation in the first and second layers effectively reduces the number of collisions,
17

---

Published in Transactions on Machine Learning Research (02/2026)
Model
Layer Description
Out Dim. MNIST
Out Dim. CIFAR
Convl: 32, 3 x 3, stride
25088
32768
Conv2: 32, 4 x 4, stride 2
6272
8192
Conv3: 64, 3 x 3, stride
12544
16384
Med.
CNN
Conv4: 64, x 4, stride
3136
4096
FC: 512
512
512
FC: 512
512
512
FC: 10
10
10
Convl: 16, x 4, stride
3136
4096
Conv2: 32, 4 x 4, stride
5408
7200
Small CNN
FC: 100
100
100
FC: 10
10
10
Convl: 64, 3 x 3, stride
50176
65536
Conv2: 64, 3 x 3, stride
50176
65536
Conv3: 128, 3 3, stride 2
25088
32768
Large CNN
Conv4: 128, 3 3, stride 1
25088
32768
Conv5: 128, 3 x 3, stride 1
25088
32768
FC: 512
512
512
FC: 10
10
10
Convl: 64, 5 x 5, stride 1, MP3
12544
16384
Conv2: 192, 5 x 5, stride 1, MP3
9408
12288
Conv3: 384, 3 3, stride MP3
18816
24576
Conv4: 256, 3 3, stride MP3
12544
16384
AlexNet
Conv5: 256, 3 x 3, stride 1, MP3
2304
4096
FC: 4096
4096
4096
FC: 4096
4096
4096
FC: 10
10
10
Block 1: Ros, 3 x 3, 160 filters
125440
163840
Block 2: Res, 3 3, 160 filters
125440
163840
Block 4: Res, 3 3, 320 filters
125440
163840
Wide ResNet 28-10
Block 6:
Res,
3, 320 filters
62720
81920
Block 8:
Res, 3 3, 640 filters
62720
81920
Block 10: Res, 3 x 3, 640 filters
31360
40960
Block 12: Res, x 3, 640 filters
31360
40960
FC: 10
10
Table 3: Model Architectures and Feature Space Dimensions for MNIST and CIFAR-10.
18

---

Published in Transactions on Machine Learning Research (02/2026)
suggesting lower optimal loss in these layers. However, the situation changes with the third ReLU layer, where the activation function leads to an increase in collisions.
A similar pattern was observed with the batch normalization (BatchNorm) layers in figure 20 Adding BatchNorm after each layer showed that the first and second BatchNorm layers contributed to reduced collisions, whereas the third BatchNorm layer resulted more collisions.
C.5 Different activation function
We experiments using GELU and Tanh activations MNIST. As shown in Figure (included in the revised appendix), these experiments yield qualitatively similar robustness curves and convergence behaviors to the ReLU models. This supports our claim that the collision-search method is not strictly tied activations like ReLU.
to
9.4
Aist
First
Second
Thirt
Thist
8.2
Layer
0.2
Tourth
fith
rits Layar
Sieth
Soch
3.0
0.0
Sewerth
1.5
(a) The optimal loss for the MNIST model with Gelu activation function.
(b) The optimal loss for the MNIST model with Tanh
activation function.
Figure 21: Comparison of optimal loss curves for different activation functions.
C.6 Impact of sample size and class choice.
To assess robustness of the conclusions, we perform a sensitivity analysis the Medium CNN. First, we increase the subset size to S 200 per class (40,000 pairs). Second, repeat the analysis with alternative MNIST class pair (2 5). Third, re-sample different set of S = 100 examples for the original class pair. As shown in Fig. 22 the layer-wise robustness curves and the relative ordering of layers remain qualitatively unchanged across these variations, indicating that our conclusions are stable with respect to sample size and class choice.
App
Lane"
lper
(a) Sample size 200.
(b) Class Pair vs 5.
(c) another sample of 100 pairs.
Figure 22: Sensitivity Analysis. The layer-wise robustness trends remain consistent when (a) increasing sample size to 200, (b) changing the class pair, and (c) another 100 pairs.
C.7 Different learning rate
To investigate how the learning rate affects layer-wise robustness, we retrained the model using a flat learning instcad of a cosine schodule. At similar learning rate levels, the model trained with flat learning rate exhibits no significant difference in layer-wise robustness compared to the one trained with a cosine schedule. However, when the learning rate is reduced to 0.001 in figure 24 the optimal loss curve narrows starting from
19

---

Published in Transactions on Machine Learning Research (02/2020)
2.0
0.5
6.4
0.4
0.2
5.
0.
50
0.4
0.8
12
1.
2.0
2.4
23
32
35
00
0.4
0.5
1.5
2.0
2.4
2.8
5.6
Figure 23: MNIST Medium Can Model trained with PGD: All layer robustness, learning rate is 0.01 throughout training
Figure 24: MNIST Medium Cnn Model trained with PGD: All layer robustness, learning rate is 0.001 throughout training
the second layer. This indicates that the model achieves good robustness with just the first two layers, while the subsequent layers contribule only marginally to robustness.
C.8 Changing the width of single layer
8.5
3
0.5
Inout RPP
hppi RP
Tiret Layer
Fintt Layer
2.3
Second Layer
0.3
Sacond Layar
Tuird Layer
Thisl Layer
Pourth Laysr
Fourhh Layer
Hith Layer
Fith PC Laye
631
saci loyer
0.1
Sleth layer
Loye
Loyk
Figure 25: MNTST Medium Can Model trained with PGD: All layer robustness, and the first convolutional layer has more filters
Figure 26: MNIST Medium Can Model trained with PGD: All layer robustness, and the first convolutional layer has fewer filters
When we decrease the number filters in the third layer, the overall robustness of the model slightly decreases in figure 26 and figure 25 but the robustness the first layer improves, which aligns with our conclusion in the main paper. Conversely, increasing the number of filters in the first layer leads the opposite effect, with slight decrease in first-layer robustness and potential increase in overall robustness.
C.9
Robust training with varying perturbation budgets
3.6
0.5
Cho55 0/983) 002
0.5
0.5
Input R.P
Input RP First Layer
Fist Layer
Swcond
Second
0.2
Layer
0.2
Layer Layer layer Logit Layer
Layer
0.1
Slath leyer
0.1
Layer
0.0
Figure 27: MNIST Medium Cun Model trained with PGD-1: All layer robustness
Figure 28: MNIST Medium Cun Model trained with PGD-3: All layer robustness,
When training with different perturbation budgets, we observed that, compared to PGD-2, PGD-1 exhibits similar robustness in the first. and second layers. However, in the remaining layers, collisions are detected,
20

---

Published in Transactions on Machine Learning Research (02/2026)
indicating lower robustness as reflected by the higher optimal loss. In contrast, PGD-3 demonstrates greater robustness than PGD-2, with the third layer showing improved distance and lower optimal loss compared to the second layer pattern not observed in PGD-2. The robustness distance in the other layers follows the same properties previously identified.
C.10 More common architecture
WideRessNet-28 has three stages, each consisting of four blocks, for total of twelve blocks. AlexNet, on the other hand, includes five convolutional layers with max pooling and three fully connected layers. The feature dimensions are shown in Table g
For MNIST, the performance of AlexNet aligns with the observations reported for other medium-sized CNN models in the paper. However, when evaluating CIFAR, it appears that the max pooling operation concentrates robustness in the layer following the third pool.
In the case of WideResNet-28 each block. However, for CIFAR, the final linear layer retains the most robustness-related information. While the robustness of individual blocks remains similar, the final linear layer enhances overall robustness, bringing it to a level comparable with other modcls.
MNIST, observe that robustness information is gradually learned by
0.7
3.7
3 0.5
0.5
Insu RJP
0.4
Irout &P
Rot Layer
Dlace
Socand Layer
Dlace
2.3
Third Layer
Blacc
fourhh Layer
0.2
Blacc
Rth Layer
13 Bicck
6.3
Severts tayer
Slath byer
0.0
Light
12 Bicck
0.0
Loght
Figure 30: The WideResNet PGD-2 robustness
Figure 29: AlexNet Robustness on MNIST
on Mnist
3.5
0.5
0.5
Irput R.P
5.4
Rist Layer
Blccc
Second Layer
0.3
4B0ccc
2.5
Thim Layer
BloCc
Lager
B
Layer
Block
Steth wyar
0.1
Blcck
53
Logi layer
Laye
0.0
0.4
0.6
Figure 32: The WideResNet PGD-0.5 robustness CIFAR
Figure 31: AlexNet Robustness CIFAR
C.11 PCA
Here, we used PCA instead of training an actual model, and then performed random projection on the PCA down to 10 dimensions for comparison. This illustrates how removing low-quality features improve robustness we measure it. The PCA on CIFAR-10 retains much greater distance compared to real model.
C.12 Using feature extractor for training
To show that feature extractors really matter for robustness, we run a simple freeze-k test: freeze the first k layers (k 0... 7), re-initialize the rest, do epoch of PGD-2 adversarial training the unfrozen part,
21

---

Published in Transactions on Machine Learning Research (02/2026)
2.6
0.
PCA.00
PC410
6.4
PCA300
0.
PCA30
PCA
5.0
50
53
0.6
0.0
12
18
24 3.0
3.5
33
08
03
055
58
1.2
1.3
1.5
2.1
2.7
33
50
39
Figure 33: Mnist PCA, then random projection
Figure 34: Cifar10 PCA, then random projection
to 10
to 10
and record the adversarial training loss for e E [2.0,3.0]. Lower loss means the frozen ext extractor already makes robust learning easier for the new head.
2.5
apozch Miy toweinn
2.0
3
2.0
0.5
0.23
0.00
5.0
2.0
5.6
0.0
24
1.6
2.0
32
3.0
4.8
52
Figure 36: Mnist adv trained on PGD-2 model epoch
Figure 35: Mnist benign model epoch
For the fixing more clean features hurts robust adaptation. For the adversarially pre-trained ("robust") model, the loss decreases with k, indicating that its features provide a robusterobustness--rn test. an external sanity check that different feature extractors change downstream robust training. Our contribution fast, training-free free test that scores extractor robustness. In experiments, our score aligns with the freeze-k behavior, so practitioners can pick good extractors without re-training many models.
("benign") model, the one-epoch adversarial loss increases as k grows, showing that
backbone. This freeze-k
D Analysis of the distance for linear layers
Throughout, let 14 Lil [ where ||]l is a Schatten p-norm and ||]|| is the f operator norm. Then ra is a variation of stable rank for L.
Proposition 2. Let f(x) = ALx, where Xn a fixed linear feature extractor and the entries of A gmxai are i. standard Gaussian. For 2 2 max(e m/4 m²c TT),
1 log(2/e)
d(x,
+ ym
(og(2/e)
4m 2
Vog [og(m²/e)
4m
Tog(m2/0
4|1217
M
with probability 1-6.
This bound implics concentration of the distance when m >> and m <
|
|
|o1?
=
72
Bocause
F4 E. σi
Fh vT4 S
r2,
Ta Vra
and m vra sufficient get concentration.
Proof. For f(x) Hæ, dz( )2 is equal to
min SER"1
|| subject to H + 8) 0
22

---

Published in Transactions on Machine Learning Research (02/2026)
where z'=x", 2. Then the lagrangian is
M(6 = x (xx
and the dual problem is
max J -HHTA + min(o 0" 6 + A
max m H X
where the minimizing 8 is - X. This is unconstrained, so it is easier to work with than the primal problem. u Let À = cp. For fixed j with ||l = 1, the maximizing c is 2x H' so the dual can be reexpressed as
max 1 HT
We have shown the matrix HHT appearing the denominator close || This suggests that the optimizing the ratio is roughly the same as optimizing the numerator.
Pick p² THall (x (T
| 2 = || |
(x HT 10) max p p²T HHTH HHTH p -H|||==
|
(x HT n¹ p: HHH μ
||>|||||<|]
||
I max
S max
#=ll==1 p² H H min 11" H
Thus we need upper and lower bounds on || Then (Alx); x² distribution with m degrees of freedom. By Proposition with || m||La|1? || m m||Lx|| ||a|| - | and r4 m.
NO0, [|x]|??) so ||xx||
2 |x]||² times
(1 ml
Lx
Pr[l| sts
4m
Lo
To get bound of e/2, take 1 ml m Tar|
/ 4m Lox log(2/e).
|
m [ - 2 Lz 1| m log(2/e)
ml| La
V log(2/c)
V
max =1 n HHHH
|1113 4m L 12 log(m=//e)
1+
4m
LI
/lo((22/2
P
For t 2 ||a|12: again using Proposition
(1 (f-m)l Lar 2/²
0 S - Lx| < 2m L
log Pr[||ALLT] 2]
16m [x]l
2ml La
2m|| t ml |2
To gel bound of e/2, we take
m Lx||2 Z|| m log(2/c)
6/2 2 c m/4
t
2m Tar + Lx l2g(2/e)
</2 > e m/44"
At the boundary between the cases, t= 3 which is three times the mean and too big of a threshold to interest us. Using only the first case, we gel
|
m||Lar|2 + Lx| m log(2/c)
m m
1+ mm
v
1og(2/c)
min P|==1 TT HHT
||
4m L (m)))
1 4ml
L
log(m²
23

---

Published in Transactions on Machine Learning Research (02/2026)
Proposition 3. Let H AL, where L E Ruxxm is a fized liner feature extractor and the entries of A E Rmxxa are i..dd slandard Gaussion. Then for 2 mme for all i,j E [m],
(HH - 1111)))))) S 4||1 V 1og(2m2/))
with probability 1 1-6.
Furthermore, for all μ E R. with ||||| = 1, 10 HHHH"' - |11| - 2lggl/202////2222///
Proof- Consider the singular value decomposition of L: I. = UEVT, where U RRIXM L.I." UT Then B = AU E mm also has i.id. standard Gaussian entries. We have H11 = ALL" AUE'UTA" A = BΣ so (HH*); = -E k= B For i + j, (HH"),; EkS & BiB.BBB The first (H ) and Proposition 5 case one (H To achieve error
claim follows by applying Proposition
probabilities of 2m we lake | E((HH));)) );y]] = 4||L (lgg2m/e)
For all μ E 1m with |al| 1, using the triangle inequality we obtain
IpT (HH" 1111121) - Σ
IIallls) - s 1)...)² AlLll3//og(2nm
and (1m1, < 11 m.
Proposition (Chernoff bounds for linear combinations of x² random variables). Let Ei Bi. where (B&) are i.i.d standard Gaussian. Then for S ||
(t |0
log Pr[Z S 54
and for 2 ||012
(1-11011))
0 1- - 1 s 2r
Pr|Z 2
(t d- | |
2r4||a|13 <t ||0
where T4
||]] l13
Proof. The cummlant generating function of Z is
"
log M(A) Σ
log(1 2070).
Observe that
E
0 i.k
2
of E [ = Σ o. = ||| ||
1=1
Var
2 o B2
2
of Var BP
Σ
20 21|0111 2114
1=1
24

---

Published in Transactions on Machine Learning Research (02/2026)
lower tail For S 0, log - S S2+ + = (by comparing derivatives). We take 0- L-llo 2|ll
and for t i ||o1];
1
log Pr((HHH) n 1 t - -0 +
log
2
1 - 20700
k=1
12
S --0% +
(20 0 + 20²
2
k=
= (L ||0 12)0
+ 0 0 |0l1
(t 1
4||0|1
upper tail For < 1, log I-s S
1
and for 2 v 21 I-
S 2 22 For all 2 0, Pr((HHT) ): 2 4 s
#t (0). Let
( o
1 ok 12 k=
11
min (1_110
0 s 1011 2r 11
0- = min
8
4 maxx of
2rallo <t t 1
This choice satisfies 2020- S 1 for all i, so apply our upper bound log - Then for 2 |
log Pr[(HH!)) 2 tJ 0-0+
log
2070"
k=1
220 0r + 80;002) S --0' + Σ
k=1
- (t - ||o 12)00 40" ||o
(t- ||
0 l 1| 2ra|lol8
16
[all
+
2r < t |011
4| &
7
Proposition 5 (Chernoff bounds for linear combinations of x² random variables). Let Z = Eh=1 Br where (Bx) and (B²) are i.i.d. slandard Gaussian. Then
0 0.15 22ralo013
log Priz 2 f
To
V2rallo| <t
V2|
Proof. The cumulant generating function of Z is
n
log (()
log(1 - o202).
The off-diagonal entries have symmetric distribution, so the upper and lower tail bounds are the same. For < 1, log I & -ー and for 2² < 1 1-zz I 2z2. For 2 0, pick
0- = min t
2 V2
and
E
0 BL
2
BE =
i=1
Var
2/ 2.00 BrNBBB.
Σ
Σ
of
|0
|
i=1
25

---

Published in Transactions on Machine Learning Research (02/2026)
Then
R2
log Pr((HHH)) )ig 2 1 S -0" +
log
2
0, 0.2
k=1
1
S 110 +
20
2
k=1
-00 + 0 1
5.55 2r 11ol
+ 2
V2ra|lo113 < 1
V21lo &
26


Reviews:
[{"review_id": "kVQdDXd6uI", "reviewer": "Reviewer_SDCx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies adversarial robustness at the level of individual layers in convolutional neural networks. Building on prior work that measures the robustness of low-dimensional feature extractors. This work extends the framework to high-dimensional CNN feature spaces via random projections and a conflict-graph-based optimal-loss computation. It proposes a practical algorithm that evaluates, for each layer, how robust its feature representation is under an $\\ell_2$-norm bounded adversarial budget. They then conduct an empirical study across several CNN architectures and training schemes (standard, PGD, TRADES) on MNIST and CIFAR-10, relating layer-wise robustness profiles to training hyperparameters and to a freeze-k test that probes the usefulness of learned feature extractors.", "claims_and_evidence_yn": "No", "claims_explanation": "Within the tested setting (CNNs on MNIST and CIFAR-10), the technical development and empirical results are coherent and the main qualitative observations about layer-wise robustness are reasonably supported.\n\nHowever, the manuscript makes broader claims that are not convincingly justified by the current experiments or analysis. For example:\n\n> The contributions section states: “We introduce a novel collision search algorithm applicable to any feature extractor.” \n\n> Later, the related-work discussion adds: “our work introduces a novel collision search algorithm that can evaluate robustness for any feature extractor, regardless of the type of activation function used. This is achieved by leveraging gradient information, enabling the method to be widely applicable.”\n\nHowever, only CNN feature extractors, that too just 2 unique CNN feature extractors have been experimented with, we do not know if the findings hold with deeper architectures, like WideResNet50, ConvNeXt-small/base, ResNet101, or with Vision Transformer-based extractors.\n\n> In the methodology section, the Jacobian-subspace approach is further described as allowing the framework to “handle arbitrary activation functions and deep architectures efficiently and keep the result accuracy the same time.”\n\nHowever, only one Activation function has been explored.\n\nAdditionally, all experiments are performed on convolutional networks of moderate depth, with low-resolution inputs, and there is no complexity analysis or empirical study on larger-scale architectures or datasets. The paper also focuses entirely on $\\ell_2$ robustness, without discussion of $\\ell_{\\infty}$, which is also a standard in much of adversarial training practice. Because of this gap between the scope of claims and the presented evidence, the claims are not fully supported.", "audience_yn": "Yes", "audience_explanation": "- Interesting and timely goal: understanding how robustness is distributed across layers rather than only at input or logits.\n\n- Clear extension of prior theory to practical CNNs, with an implementable algorithm and reasonably well-organized experiments.\n\n- The empirical layer-wise robustness curves and freeze-k experiments are intuitive and support the basic methodological claims.", "requested_changes": "1. **Align generality and efficiency claims with presented evidence (Critical).**  \n   The manuscript should explicitly limit strong claims such as “applicable to any feature extractor”, “evaluate robustness for any feature extractor, regardless of the type of activation function used”, “enabling the method to be widely applicable”, and “handle arbitrary activation functions and deep architectures efficiently” to the regime that is actually studied (CNNs on MNIST and CIFAR-10), or provide additional evidence. Either temper these statements and mark broader applicability as future work, or add at least a few experiments on a more modern or deeper architectures and, ideally, a higher-resolution setting, with various activation functions.\n\n2. **Clarify scalability and computational cost (Critical).**  \n   A more explicit discussion of computational and memory complexity is needed, for example, as a function of the number of layers, feature dimension, number of samples, and input resolution. Reporting representative runtimes or resource requirements for the main experiments would help assess whether the Jacobian-subspace collision search is realistically usable for larger models and datasets, and would substantiate claims of efficiency for deep architectures.\n\n3. **Discuss the choice of $\\ell_2$ norm and relation to $\\ell_\\infty$ (Critical).**  \n   The framework and experiments focus on $\\ell_2$ robustness, while many existing adversarial training works also (or sometimes only) use $\\ell_\\infty$. The paper should justify the decision to work in $\\ell_2$ (for example, mathematical tractability, empirical behaviour) and discuss whether the framework could be adapted to $\\ell_\\infty$. Even a conceptual discussion would help readers interpret the results relative to the also common $\\ell_\\infty$-robust models.\n\n4. **Improve positioning with related work (Non-critical but important).**  \n   The related-work section would benefit from explicitly connecting to recent analyses of layer utilisation and robustness–fairness trade-offs, for example:\n   - Gavrikov et al., “How Do Training Methods Influence the Utilization of Vision Models?” (NeurIPS 2024 Interpretable AI workshop) [1], which investigates how different training schemes, including adversarial training, change which layers are critical to performance.  \n   - Medi et al., “FAIR-TAT: Improving Model Fairness Using Targeted Adversarial Training” (WACV 2025) [2]. The direction of this work is quite different from the submission, since it focuses on targeted adversarial training for fairness, but its discussion of decision boundaries and how they are reshaped by adversarial training could provide useful context. A short comparison would clarify what new insight this collision-based metric adds beyond these perspectives.\n\n5. **Address minor presentation issues (Non-critical).**  \n   Ensure that figures are referenced in order of appearance. At present, “Figure 4” appears in the text before “Figure 3”, and “Figure 2” is never referenced in the main text. Consider adding a brief limitations paragraph that clearly states the main scope restrictions (datasets, architectures, norm choice, and computational considerations).\n\n\n\n\n\nREFERENCES\n\n[1] Gavrikov, Paul, et al. “How Do Training Methods Influence the Utilization of Vision Models?” NeurIPS 2024 Workshop Interpretable AI: Past, Present and Future.\n\n[2] Medi, Tejaswini, Steffen Jung, and Margret Keuper. “FAIR-TAT: Improving Model Fairness Using Targeted Adversarial Training.” 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2025.", "broader_impact_concerns": "**The paper does not include a Broader Impact Statement.** \n\nSince adversarial robustness analysis is a dual-use topic, the work can be used both to strengthen defenses and to design more targeted attacks or model extraction strategies. A short Broader Impact section should be added that reflects on potential beneficial uses (for example, better detection and reinforcement of weak layers, more efficient robust training) as well as possible misuse, particularly in safety- and security-critical applications.", "additional_comments": "The last page is blank. Could we remove that, please?", "extra_scores": {}, "extra_text": {}}, {"review_id": "YmdFbk21bv", "reviewer": "Reviewer_caFn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors extend Bhagoji et al.’s feature-extractor robustness framework to wide CNNs by proposing a variation of the feature extractor distance for high-dimensional layers and implementing an approximate algorithm for ReLU CNNs. They introduce a novel collision-search algorithm and use random linear projections to evaluate each layer’s adversarial robustness contribution. In experiments on MNIST and CIFAR-10, the paper demonstrates that different training methods (e.g. PGD vs TRADES) lead to significant variation in layer-wise robustness, offering insight into how architectures and training decisions affect adversarial vulnerability.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper backs its claims with both theoretical analysis and empirical results. It presents a clear mathematical framework (with propositions and bounds) and provides algorithms for computing the robustness metric. Extensive experiments (Figures 5–15) on standard datasets verify the theoretical findings, showing concrete layer-by-layer robustness curves and validating the proposed distance metric. The evidence (e.g. observed robustness gaps between convolutional and fully-connected layers) is convincing and directly addresses the paper’s claims.", "audience_yn": "Yes", "audience_explanation": "The paper tackles adversarial robustness—an active and important research area—with new analysis tools and theory, which aligns well with TMLR’s focus. By providing a deeper layer-wise understanding of CNN robustness and a generic algorithm for feature-extractor analysis, the work offers broadly relevant insights for ML robustness and interpretability. Its combination of formal analysis and experimental findings is likely to engage readers interested in adversarial learning and model evaluation.", "requested_changes": "Clarify algorithmic details: The description of the collision-search algorithm and its implementation is somewhat terse. Including pseudocode or more detailed discussion (e.g. computational complexity, convergence criteria) would improve reproducibility.\n\nBroaden evaluation: The experiments are limited to relatively small CNNs on MNIST/CIFAR-10. Evaluating on larger models or additional datasets (or providing a runtime/performance analysis) would strengthen the empirical claims.\n\nDiscuss limitations and comparisons: It would be helpful to compare the proposed metric with other robustness measures (or with and without random projection) and to discuss the impact of projection dimension. Also, commenting on the computational cost of the method and its scalability would be useful.", "broader_impact_concerns": "The paper does not discuss broader impact. In terms of ethics, there are no obvious negative implications beyond the general concerns of adversarial ML. It would be appropriate to note that layer-wise robustness analysis could help build safer models (positive impact) but also that understanding vulnerabilities might be misused by adversaries (negative). A brief Broader Impact statement could mention that improved robustness metrics advance security in applications (e.g. safety-critical systems) while cautioning about potential misuse for crafting attacks.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "jYT0jdsKfd", "reviewer": "Reviewer_YcjA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a framework for analyzing the robustness of a feature extractor layer by layer. It proposes a novel, architecture-agnostic method for generating collusion adversarial pairs for a feature extractor. The analysis highlights the influence of various factors--such as training methods, activation functions and architecture--on layer-wise robustness of the feature extractor.", "claims_and_evidence_yn": "No", "claims_explanation": "1. Presentation and organization of the paper: The current version of the manuscript is challenging to follow, particularly in Sections 2 and 3. The overall organization of the paper could be significantly improved to enhance clarity and flow.  \n2. Contributions: The present work builds upon Bhagoji et al. (2021, 2023). The authors  need to clearly highlight the advantages of the proposed method. Currently, the explanation of the proposed approach is entangled with the description of Bhagoji et al. (2021, 2023) in Section 2.\n3. Validation experiment: The comparison between the proposed optimal collusion loss and the FAB loss (Section 5.1, Figures 5 and 6) might lead to incorrect interpretation. This is because the FAB attack generates the minimum perturbation required to change the class label of an input, whereas the proposed collusion algorithm generates adversarial pairs with similar feature representation-- and both the samples may not necessarily be misclassified. \n4. Missing experimental details: The paper considers 100 samples from two classes in the training set of MNIST and CIFAR-10 datasets for evaluation. The details of selected classes and the justification for choosing a subset of the dataset are missing.  Additionally, the paper misses out to explain the impact of sample size on the robustness analysis.", "audience_yn": "Yes", "audience_explanation": "The paper presents a theoretical framework for analyzing the layer-wise robustness of a feature extractor. Researchers in the machine learning community working on adversarial robustness may find the observations in this work valuable.", "requested_changes": "1. [Critical] Please address the points discussed in the claims section, i.e., (a) Presentation and organization of the paper, (b) Contribution, (c) Validation experiment, and (d) Missing experimental details.\n2. It would be beneficial to move the related works section after the introduction and include a section or subsection to describe the commonly used notation. Clearly highlight the advantages of the proposed method over existing methods.\n3. Consider normally trained models for analysis. Also report the standard deviation for the proposed loss curves.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)