Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustness
Authors: Longwei Wang
Ifrat Ikhtear Uddin
KC Santosh
Chaowei Zhang
Xiao Qin
Yang Zhou
Venue: NeurIPS 2025 spotlight ()
Keywords: Adversarial Robustness
Symmetry
Equivariance
Primary area: general_machine_learning

Abstract: Adversarial examples reveal critical vulnerabilities in deep neural networks by exploiting their sensitivity to imperceptible input perturbations. While adversarial training remains the predominant defense strategy, it often incurs significant computational cost and may compromise clean-data accuracy. In this work, we investigate an architectural approach to adversarial robustness by embedding group-equivariant convolutions—specifically, rotation- and scale-equivariant layers—into standard convolutional neural networks (CNNs). These layers encode symmetry priors that align model behavior with structured transformations in the input space, promoting smoother decision boundaries and greater resilience to adversarial attacks. We propose and evaluate two symmetry-aware architectures: a parallel design that processes standard and equivariant features independently before fusion, and a cascaded design that applies equivariant operations sequentially. Theoretically, we demonstrate that such models reduce hypothesis space complexity, regularize gradients, and yield tighter certified robustness bounds under the CLEVER (Cross Lipschitz Extreme Value for nEtwork Robustness) framework. Empirically, our models consistently improve adversarial robustness and generalization across CIFAR-10, CIFAR-100, and CIFAR-10C under both FGSM and PGD attacks, without requiring adversarial training. These findings underscore the potential of symmetry-enforcing architectures as efficient and principled alternatives to data augmentation-based defenses.

Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustness
Longwei Wang', Ifrat Ikhtear Uddin', KC Santosh lf,
Chaowei Zhang Xiao Qin Yang Zhou
longwei. wang@usd edu, ifratikhtear uddingcoyotes. usd. edu, kc. ac.ssatosh@usd edu cuzhang@yzu. edu. en, xqin@auburn. edu, yangzhou@auburn.cm
AT Research Lab, Department of Computer Science, University of South Dakota, USA
2 School of Information and Artificial Intelligence, Yangzhou University, Yangzhou, China à DDepartment of Computer Science and Software Engineering, Auburn University, USA
Abstract
Adversarial examples reveal critical vulnerabilities in deep neural networks by exploiting their sensitivity imperceptible input perturbations. While adversarial training remains the predominant defense strategy, it often incurs significant compu- tational cost and may compromise clean-data accuracy. In this work, investigate an architectural approach to adversarial robustness by embedding group-equivariant convolutions aanvolutions-sppccffclllll rotation- and scale-equivarian layers into standard convolutional neural networks (CNNs). These layers encode symmetry priors that align model behavior with structured transformations in the input space. promoting smoother decision boundaries and greater resilience to adversarial attacks. We propose and evaluate symmetry-aware architectures: parallel design that processes standard and equivariant features independently before fusion, and a cascaded design that applies equivariant operations sequentially. Theoretically, we demonstrate that such models reduce hypothesis space complexity, regular- ize gradients, and yield tighter certified robustness bounds under the CLEVER (Cross Lipschitz Extreme Value for nEtwork Robustness) framework. Empirically, our models consistently improve adversarial robustness and generalization across 100.R-10, CIFAR-I00, and CIFAR-IOC under both FGSM and PGD attacks, without requiring adversarial training. These findings underscore the potential symnetry-enforcing architectures as efficient and principled alternatives to data iigmeentation-basee defenses.
Introduction
Adversarial robustness, defined as the capacity of deep neural networks to produce consistent predic- tions under small and often imperceptible input perturbations, remains fundamental and unresolved challenge in modern machine learning. Adversarial attacks exploit a model's ''s sensitivity to small, norm-bounded input perturbations, leading to incorrect and often high-confiddene predictions [I] These perturbations typically exploit the model's reliance on spurious, non-semantic features that do not align with the true data-generating process [2] A key contributing factor to this vulnerability is insufficient training data: when datasets are limited in size or diversity, models tend to overfit to superficial statistical patterns, such as background textures or local pixel correlations. rather than learning robust and generalizable representations 3 4).
Corresponding Authors.
Code available at: hhttss:// http:///ttthttoooooooootaaaaaaaa
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Adversarial training has become a dominant approach to mitigate this vulnerability. It enhances model resilience by explicitly injecting adversarial examples into the training process. thereby guiding the model to focus on more discriminative, semantically grounded features 15 6 These adversarial examples expand the effective support of the training distribution, allowing models to develop wider decision margins and improved generalization to perturbed inputs [2] 7 Despite its
success, adversarial training is not without limitations: it is computationally expensive, may degrade performance on clean data, and is often specialized to the attack types seen during training. Moreover, it addresses robustness reactively by modifying data rather than proactively by redesigning the model architecture.
This motivates a fundamental question: Can architectural priors alone improve adversarial robustness by encouraging models to align more closely with the geometric structure of data? In this work, we explore this question through the lens of equivariance, the principle that model outputs should transform predictably under known input transformations. In particular, we investigate whether embedding symmetry priors via group-equivare
convolutions can enhance adversarial robustness in convolutional neural networks (CNNs) even in the absence of adversarial training.
Equivariance provides a principled mechanism for enforeing inductive biases that align with underly- ing symmetries in data. While standard CNNs are translation-equivarinn by design, they are not inher- ently equivariant to other common transformations such as rotations and scalings. touuieqquivaiiant convolutions generalize standard convolutions to be equivariant under larger transformation groups, such as the discrete rotation group P4 or scale groups IO. These architectures encode transforma- tion consistency directly into the weight-sharing scheme of the network, allowing the model to process rotated or rescaled inputs without relying on data augmentation. As result, equivariant CNNs have demonstrated improved sample efficiency, stronger generalization, and greater interpretability across domains such as medical imaging, remote sensing, and physics-infomeed learning [2]
Despite their success in structured learning tasks, the relationship between equivariance and adversar- ial robustness remains underexplored. Intuitively, adversarial perturbations often introduce changes that lie off the data manifold or violate known symmetries. By constraining the model to respond con- sistently along group-inducee orbits and suppressing sensitivity to off-orbit perturbations, equivariant architectures may provide a natural defense mechanism. This raises key research question: How does architectural equivariance influence a model's 's resilience to adversarial perturbations, both theoretically and empirically?
In this paper, bridge the gap between symmetry enforcement and adversarial robustness by conducting a systematic study of CNN architectures that integrate standard, rotation-equivariant, and scale-equivariant convolutions. We propose two model designs to incorporate equivariant layers and evaluate their robustness properties across a spectrum of adversarial and natural corruption settings. Our main contributions are summarized as follows:
We present a theoretical analysis demonstrating that equivariant architectures contract the hypothesis space, regularize gradient behavior, and admit tighter certified robustness bounds under the CLEVER (Cross Lipschitz Extreme Value for nÊtwork Robustness) framework.
We propose and compare two symmetry-aware CNN architectures parallel and cascaded that integrate standard, rotation-equive -quivarnnuu and scale-equivariant convolutional layers. We show that the parallel design better preserves complementary feature spaces and achieves superior robustness. We explore two fusion strategies simple concatenation and weighted summattio for combining features from multiple symmetry branches. Our findings indicate that concatenation consistently outperforms weighted fusion in adversarial settings.
We validate our approach through extensive experiments on CIFAR- 10, CIFAR-IOC, and CIFAR-100 datasets, using FGSM and PGD attacks. Our results show that equivariant CNNs, particularly the parallel design with combined rotation and scale branches, significantly outperform standard CNNs in adversarial accuracy without requiring adversarial training.
2 Related Works
Trustworthy machine learning, which focuses on developing and deploying machine learning models that are not only accurate but also robust, private, fair, and explainable, has attracted active research in recent years (3]86). Adversarial robustness, a core pillar of trustworthy MI, addresses the vul-

---

nerability of neural networks to imperceptible perturbations. In this work, we explore the intersection of symmetry-awan architectures and adversarial robustness, drawing from two key research areas.
2.1 Equivariant Neural Networks
Equivariance in neural networks ensures that transformations applied to input data lead to predictable and consistent transformations in the learned representations, aligning models with the inherent symmetries in the data. A landmark advancement in this field is the introduction of Group Equivariant Convolutional Networks (G-CNNs) [8) which extended traditional convolutional operations to group transformations, including rotations [87] [88]. These networks demonstrated significant gains in performance and efficiency on symmetry-ricce datasets such MNIST and CIFAR-10. Subsequent progress led to the development of Harmonic Networks, which employed circular harmonics to achieve continuous rotational equivariance [9] By eliminating the need for discrete approximations, these models improved the flexibility of equivariant frameworks [89]. Further extending these ideas, Scale-Equivariant Steerable Networks were introduced to address scale transformations, enabling the processing of multi-scale inputs without explicit data augmentation [90]. Steerable CNNs provided versatile framework for handling equivariance under range of transformations, paving the way for applications in domains such as medical imaging, astrophysics, and 3D object recognition [91].
More recently, spherical equivariance has garnered attention, with spherical CNNs being developed to handle data defined on spherical domains [12] 992 [93] These models have found use in global-conter
tasks, including climate modeling and astrophysics [1] Additionally, equivariant networks have been applied in molecular biology, utilizing molecular symmetries to predict chemical properties [94). Despite these advancements, the susceptibility of equivariant models adversarial perturbations remains an open area of investigation.
2.2 Adversarial Robustness
The discovery of adversarial examples exposed significant limitation in neural networks, revealing their vulnerability to small, carefully designed perturbations [.] These adversarial inputs exploit weaknesses CNN architectures, leading to incorrect predictions. The Fast Gradient Sign Method (FGSM) formalized this issue as single-step attack based on the direction of the loss gradient [95]. Later, Projected Gradient Descent (PGD) was introduced as stronger iterative attack method, becoming benchmark for adversarial testing [5]. Efforts to defend against such attacks have primarily focused on adversarial training, where models are trained using adversarially perturbed data to improve robustness [5) 96 (97). However, this approach often results in reduced accuracy on clean data (2) To mitigate these trade-offs, researchers have proposed architectural innovations, such feature denoising modules (98) and preprocessing techniques like compression, resizing, and randomization [10] [01), which aim to diminish the effect of adversarial perturbations.
Advanced defenses have also leveraged model uncertainty and interpretability. Randomized smooth- ing has emerged as certified defense strategy [102] [03] [04] while ensemble methods have demonstrated improved robustness by combining multiple decision boundaries [105] [64] Despite
these promising developments, the potential to integrate the preeerrr-preservigg principles of
equivariant networks into adversarial defense strategies remains largely untapped, leaving valuable avenue for future research.
Equivariance and Adversarial Robustness
Neural networks are known to be vulnerable to adversarial perturaatinns: small, human-immerceeeeeeeeeeeeee imperceptible modifications to input data that cause incorrect predictions with high confidence. This phenomenon is often attributed the model's overreliance on non-semantic features and irregular decision boundaries. One principled approach to mitigating this sensitivity is to enforce architectural inductive biases aligned with known symmetries of the data distribution. Among such biases, equivariance has emerged as theoretically grounded and empirically effective mechanism.
3

---

3.1 Equivariance in Neural Networks
Definition 1 (Equivariant Function). Let act on X via Tg and on y S IR via a representation p(g) E Aut()).. A function f x + 3 is said to be G-equivariant if:
f(Tga) Tzf(x), E, D.
Standard CNNs translation-equivaaiiee due to weight sharing across spatial positions. However, they lack equivariance to transformations like rotation and scaling. Group-equivariant CNNs (G- CNNs) generalize convolution to act equivariantly under more general groups G, such as Cn. SO(2). or dilation groups, therehy promoting symmetry-aligned representations.
Formally, let G be group with associated action the input space X' C. R2 and let p: G - Aut (IR*) be linear representation G acting on the feature space. A function f f :x - 1& said to be equivariant with respect to the group G if it satisfies:
f(g x)) = p(g)f(z), E
(1)
where g x denotes the transformed input under the action of g E G. Equivariance ensures that applying transformation the input leads to predictable transformation in the output, thereby promoting stability and consistency in feature representations.
3.2 Adversarial Robustness and Margin Bounds
Definition (Adversarial Robustness). A classifier f :Rd + R is said to be (s.pu-robust at input
E E if:
f(x + 8) [(x), vo E RR, |0lpp S.
Definition (Margin Function). Let fc(x)) denote the logit score of the predicted class c, and fz( the score of class j # The class margin is:
ge.( = fc(x) - fz(x).
Definition (Certified Robustness via Lipschitz Bound). Let ge be locally Lipschitz with constant > O near v. Then for any 5 E R.d such that ||0|| I E, we have:
19c,(x + 9c. S L|(ö|pp
Consequently, robusiness against class j certified if:
(p)
(x.j(x)
Definition 5 (CLEVER Bound (106)). Let geg be the murgin function above. The CLEVER bound estimates a certified perturbation radius as:
(p] min (x) == min 9c.((1) (x)
jtc I
where 1/p + 1/9 = 1, and
L
sup (2,.))
VVge (2')
is data-dependent estimate of the local Lipschitz constant of go.j in the dual norm.
4 Theoretical Analysis of Adversarial Robustness with Equivariant Convolutions
This section presents comprehensive theoretical framework for analyzing the adversarial robustness of troup-equivariant neural networks. We develop the mathematical foundations necessary for understanding the relationship between equivariance and model sensitivity, formalize certified robust- ness bounds under Lipschitz constraints, and show how equivariant architectures induce smoother gradients and larger certified margins.

---

4.1 Mathematical Preliminaries and Equivariant Structures
Definition 6 (Input Space and Model), Let X C Rd be the input space, and let f : IR be neural network mapping inputs to logit vectors. We assume is differentiable almost everywhere. Definition 7 (Orbit and Quotient Space). The orbit of point E X under G is the set [a]e {g- x g E G}. The quotient space //G is the collection of distinct orbits in X. Definition 8 (Jacobian and Lipschitz Constant). If f is differentiable at æ, the Jacobian is Jy(x) f(()) E kxd, and the local Lipschitz constant is L(x) == ||
Definition (Adversarial Perturbation). vector RRd is an adversarial perturbation at if f(x + 8) * f() and ||0|pp I c.
4.2 Jacobian Structure and Lipschitz Regularity
Definition 10 (Global Lipschitz Continuity). A function f is globally Lipschitz if there exists L > 0
such that:
(z] fx2))l & L ||æ xzll: Vz 22 E BR
Definition 11 (Jacobian under Equivariance). If flg = p(g)f(x)) then the Jacobian transforms as:
Jf(g x) = p(g)
where Dg 1 is the Jacobian of the inverse transformation.
Lemma (Jacobian Norm Invariance [107). If p(g) and Dg are orthogonal matrices, then:
4.3 Certified Robustness via CLEVER Bounds
We analyze the certified adversarial robustness of group- eouu--qquivaiiatt networks through the CLEVER framework [I06], which provides lower bound on the minimum input perturbation required to induce misclassification. We show that equivariance yields gradient invariance over group orbits, leading to consistent and stronger robustness certification.
Lemma 2 (Transformation of Margins under Group Equivariance 108 ) Let f be G-equivariant, ie.,
f(g x) = f(g)) where p:: G - GL,(R*) is linear representation. Then for any j # 9o.ig x) (e)(g))((x) Paj P
Lemma (Gradient Transformation of Margin Function 107
Differentiating the margin function
under the group action yields:
Vgealy x) ge. Dg
where Dy-1 E Iaxdd is the Jacobian of the inverse group action. Theorem 1 (Orbit-Invariance of Margin Gradient Norm). If pig) and Dg norm-preserving (e-g,, orthogonal matrices), then for all E G,
|| [Vge,ai x)11, (e)e(())les
As a result, the Lipschitz constant of the margin function is invariant across the group orbit:
L(5)
sup xEBx(xxaa.)
g
where [x]o:= x G).
The urbit-invariance both the classification margin and its gradient norm establishes compelling theoretical foundation for the robustness of group-cquivariant networks. These models are not merely robust at isolated input points but offer uniform guarantees across entire equivalence classes of inputs linked by symmetry transformations. By construction, group-equivarian architectures preserve margin values consistently under group actions, ensuring that the discriminative separation between classes remains stable across symmetrically transformed instances. Furthermore, they inherently suppress gradient sensitivity in directions aligned with the symmetry structure the data, effectively filtering out perturbations that respect these invariances. As result, equivariant networks exhibit tighter and more reliable CLEVER--eerified robustness bounds throughout the input space, Detailed proof is provided in Appendix A.I
5

---

4.4 Squivarriane--Indueed Gradient Smoothing
core source adversarial vulnerability in neural networks the irregularity of their input-output mappings, often reflected in sharp or non-smooth gradients. eoouu--quiaaiiant networks mitigate this by imposing geometric constraints that smooth gradients over symmetrie input transformations. The idea of this section is to analyze how group equivariance suppress adversarial vulnerabilityy by smoothing the gradients of the network with respect its inputs. Specifically, we show that group symmetries induce consistent, low-variance gradient fields along symmetric transformations, and reduce sensitivity to adversarial perturbations that deviate from these structured directions. key source of adversarial fragility in neural networks stems from the irregularity of their input- output mappings, often manifesting as sharp or non-smooth gradients. Group-equivariane models mitigate this issue by embedding geometric constraints that align the model's behavior with inherent symmetries in the data, leading to smoother gradients and more stable decision boundaries.
Definition 12 (Logit Gradient and Jacobian Matrix), Let f Bdd R be differentiable classifier, and let (x) denote the logit for class j. The Jacobian of f is:
(x) T
x(x)==V1(X
E R
[
where (x) Hd denotes the gradient of the j-th logit with respect to the input. Each row of (x) characterizes how sensitive a particular output is to infimitesimal changes in different input directions.
Lemma 4 (Gradient Transformation under Group Equivariance function, i.e.,
Let f be a G-equivariant 109
f( x) = f
with p(g) a representation and D9-1 the Jucobian of the inverse group action. Then:
Vg x) = p(g) [x) Dg
Definition 13 (Orbit-Avcraged Gradient Field). Define the per-logit gradient vector as ox(x) = (x)(). Then, the orbit-averaged gradient is:
(x)x)
Σ
Visl
Σ DEG
(O)
|G|
|G|
Lemma 5 (Smoothing via Orbit Averaging 108). Let ô E IRd be a small perturbation. Then:
(x) - (x)ll < 10, + 8) - -x,(xx) (x))l
especially when orthogonal to the group orbit [x]e. Thus, orbit-averaging suppresses high- frequency variations in the gradient field.
Theorem 2 (Directional Suppression of Off- Offi-trbit Perturbations). Let f Fd + I2t be a differentiable function that is equivariant under the action of compact group G, i.e.,
I[g x) = p(g) (x) for all g E G, where p(g) E GL(R*) is a linear representation and g is the group action the input space. Suppose perturbation vector 5 E RRd can be decomposed as:
5
dG + 51.
where 8G E (x(]) lies in the tangent space of the group orbit at x. and 8 1 (xx]a)). Then:
[V fx(i)] > f 6g) vf(x)
In particular, if f is orbil-averaged over G, then [ + 8c) - V + 0, and off-orbit perturbations dominate gradient variability.
This theorem formalizes the observation that equivariant networks inherently suppress gradient sensi- tivity along commetry-respectnngg directions, while remaining susceptible to orthogonal, adversarial ones. This anisotropy in gradient variability contributes improved adversarial robustness and smoother decision boundaries. Equivariance not only constrains functional outputs under transfor- mations but also regularizes local geometry of the model's decision surface. Gradient smoothing and directional suppression enhance robustness by reducing sensitivity to adversarial perturbations, especially those orthogonal to the data manifold's symmetric structure. Detailed proof is provided in Appendix A.2

---

4.5 Robustness Analysis of Scale Equivariance
Scale-equivariant neural networks do not satisfy the same assumptions required for certified robustness under isometric transformations such as rotations. Specifically, scale transformations alter the norm of the input, violating the orthogonality condition required in Lemma and Theorem Nevertheless, scale-equivariant architectures contribute to adversarial robustness through a different mechanism namely, gradient smoothing via multi-scale orbit averaging.
Let & E IRd an input and G, a finite group of scaling transformations. The orbit of x under G, is
defined
0,(z) (T) s G.h.
where (x(x) = S--.. We define the orbit-averaged gradient of a class logit function Øy as:
1
(o)
(T)(T
G
sEG,
Unlike in the rotation-cquivariant case, the norms VV (Ta3) are not preserved across the orbit, but the averaging process acts form of regularization. reduces the gradient variance across local neighborhoods, thereby smoothing the decision boundary and dampening sensitivity to adversarial perturbations that rely on sharp gradients.
Scale-equivariant convolutional neural networks (CNNs) achieve robustness by explicitly enforcing equivariance across multiple spatial scales. This is typically done using scale-group convolutions, defined as:
+ f+)))
th * f(T sEG,
where th is the filter bank corresponding to scale + denotes convolution, and (x))) = 8 The output scale-indexed feature map that captures the input structure across different resolutions.
This structure induces a smoothing effect both in the feature and gradient spaces. Specifically, consider the aggregated feature response at given layer:
((x) =
Σ w. (x),, with 2x(x) = th xT)
sEG,
and its gradient with respect to the input:
Vh(z)= = Σ w. (z.
sEG,
Although the gradient norms [v)())) scale with their aggregation smooths the overall gradient field by suppressing high-frequency components. This is analogous to low-pass filter in the frequency domain and reduces the model's vulnerability adversarial perturbations that exploit sharp local gradient changes. While scale-equivariant models fall outside the domain of certified robustness guarantees derived under norm-preserving assumptions, they introduce robustness via a complementary mechanism: smoothing the activation and gradient fields through multi-scale aggregation. This mechanism stabilizes the model's output under input perturbations and effectively regularizes its sensitivity, promoting robustness in practice.
5
Equivariance Enhanced Architectural Designs
5.1 Group Equivariant Convolutions
Group Equivariant Convolutional Networks (G-CNNs) generalize standard convolutional architectures by incorporating symmetry priors directly into the model design [8]. These networks are constructed to preserve equivariance under transformations defined by group G, such as translations, rotations, or scalings. To achieve this, the conventional convolution operation is replaced by a group convolution, which aggregates features across transfrradformed versions of both the input and the filters. In this work, we focus on two widely applicable instances: oolation-cquivane convolutions. Detailed formulations of these operations are provided in Appendix C.I
and scale-oquivariant
7

---

5.2
Squivarriane--Efforee Architectural Designs
We investigate two architectural strategies that integrate standard, rotation- -uautiio-equvvriaan,, and scale- equivariant convolutions to enhance robust feature extraction: the parallel design and the cascaded design. Each approach offers a distinct balance between representational diversity and computational efficiency. Comprehensive architectural details are provided in Appendix C.2
6 Experiments and Discussion
This section details the experimental setup used evaluate the impact of adding rotation- and scale-equivariant convolutions on adversarial robustness and generalization. The experiments were conducted using three widely recognized datasets CIFAR-10, CIFAR-100, and CIFAR-10C to ensure a comprehensive evaluation of adversarial robustness, and generalization under natural corruptions. CIFAR-I0C (T10) is variant of CIFAR-10 designed evaluate corruption robustness. It includes 19 types of natural corruptions (c.g., Gaussian noise, motion blur, fog, and pixelation) applied at five levels of severity.
6.1 Models
To investigate the impact of equivariance on adversarial robustness, we designed and evaluated five CNN architectures with varying symmetry-aware modifications.
Baseline Standard CNN serves as the benchmark model, implemented with either 4 or 10 convo- lutional layers. Parallel GCNN replaces the first convolutional layer with two parallel branches: a standard convolution branch and a rotation-equivariann branch based on the discrete group P4. Parallel GCNN with Rotation- and Scale-Equivariant Branches extends the above by introducing third eqle-rqquvarr branch, enabling the model to process inputs across multiple geometric transformations. Cascaded GCNN adopts sequential structure, where the input is first processed by a cration-equivariin layer, followed by standard convolutions. Weighted Parallel GCNN uses the same three-branch structure as the previous parallel design but replaces feature concatenation with learnable fusion weights optimized during training.
6.2 Comparison of Adversarial Robustness under FGSM and PGD Attacks
We evaluated five models Baseline Standard CNN, Parallel GCNN, Parallel GCNN with Rotation- and Ccall--Eqivariant Branch, Cascaded GCNN, and Weighted Parallel GCNN CIFAR-10 and CIFAR- 100 datasets under FGSM and PGD attacks. To ensure a comprehensive understanding of the impact of network depth on robustness, we experimented with both 4-layer and 10-layer CNN architectures for all models.
In Figure I we present the adversarial robustness comparison of five models using 4-layer architec- tures on CIFAR-IO and CIFAR-100. The evaluation considers adversarial accuracies under FGSM and PGD attacks across range of perturbation magnitudes (e). For CIFAR- 10, the Parallel GCNN with Rotation and Scale Branch exhibited the highest robustness. The Parallel GCNN also performed well, but its robustness declined more rapidly compared the combined model. On CIFAR-100. the overall robustness was lower due to increased class diversity and complexity. The Parallel GCNN with Rotation- and Scale-Equivarial Branch remained the most robust particularly at higher perturbations. The Parallel GCNN showed competitive performance at low perturbations but lagged behind the combined model, particularly higher perturbations.
In Figure 2 we consider 10-layer architectures on CIFAR-I0 and CIFAR- 100 datasets. On both CIFAR-10 10 and CIFAR-100, the Parallel GCNN with Rotation and Equivraqu Branch main- tained the highest adversarial robustness across all perturbation levels. The Parallel GCNN with Rotation-Equivarian Branch also demonstrated strong robustness, though it was consistently outper- formed by the combined model. Both the Cascaded GCNN and Weighted Parallel GCNN showed limited robustness, with adversarial accuracies dropping below 15% for FGSM and almost negligible for PGD attacks at higher perturbation levels.
To validate our theoretical framework under strict symmetry constraints, we evaluated fully equivari- ant architectures where all convolutional layers are equivariant, without any standard convolution branches. The 10-layer fully equivariant model achieves 73.01 % FGSM and 64.96% PGD accuracy
8

---

FGSM Edstion ICHFAR-101
Epsi (CIFAR.199
THE
5.3 632 632 C25
645 C.0
6:0
5/0
AI
610 514 110
531 6A2 640 620
011
9.1.
9:20
9:34 0.2
FSSH AccursCy VS Epstion (CIFAR-100)
PGD Accurssy vs Epsiion (C FAR-1000
Restie DCNA
15-
Dilcaind 0004
R0S5
26
Figure I: Adversarial robustness comparison of five models using 4-yayer architectures on CIFAR-10 and CIFAR-100. Shaded regions represent #1 standard deviation over 5 random seeds.
Table 1: Performance Analysis of Different Models under Various Corruption Types and Perturbation Levels CIFARIOC (%)
BASHLINECNN
CASCADED
PARALLEL GENN
PARALEEU
Curmuption Type
Brightnes
14.79
2.9
0.64
16
8:27
0.46
0.02
0.00
18.38
5.86
1.85
0.65
8:38
0.20
0.00
0.00
9.13
0.37
0.01
0.00
Contrast
1.5
0.71
0.50
3355
1.31
0.00
0.00
0.00
7.35
2.52
8.46
0.05
0.69
0.00
1.00
0.00
0.50
0.90
0.00
0.01
Deficme Biur
752
0.32
0.04
0.00
13.15
322
0.39
0.20
2.72
0.05
0.00
0.00
2.80
0.94
0.00
Flasti
0.00
Fog
1/4
0.45
0.12
803
601
0.00
6.97
0.31
0.04
1.50
0.00
0.00
1.85
000
0.00
Foot
9.32
1.45
0.30
0.10
3.55
0.57
00:
0.00
11.90
5322
1.00
0.78
3.04
0.10
1.01
0100
330
0.15
031
0.01
Gameita Blar
7.24
1.12
0.27
0.10
2.54
002
0.00
0.0
11.02
286
0.59
2.02
0.02
0.00
0.00
2.23
0.4
0.00
Cassiia Noise
0.34
000
Glas
5.51
0.33
0.13
001
0.03
0468
010
0.00
0.00
Impuhe New
5.90
100
0.31
0.14
1.57
000
0.00
0.00
12.00
205
0.34
0/06
1.56
0.01
0.00
0.00
1.44
001
0.00
0.00
JPEG Compeesiion
8.68
189
0.45
0.15
1.42
0.11
0.00
0.00
17.44
4.4
1.39
0.31
4.35
0.06
0.00
0.00
3.35
0.05
0.00
0.00
Mation Buur
6.34
0.23
602
0.00
0:59
0.00
0.14
0.00
Saturate
9.24
0.29
0.59
0:44
0.00
1.30
7.90
0.20
8.00
7.81
0.0
Shot Nois
5.62
1,07
0.38
0.17
2.36
0:05
0.00
0.00
1600
3.09
0.51
0.33
2.71
0.01
0.00
0.00
2.33
0.03
0.00
0.00
Snew
9.71
0.53
0.23
0.0
1.23
0.10
0.00
383
0.0
Spatter
Speckle Note
5.25
0.38
0.18
2.48
004
0.03
0.55
0.30
0.01
8.0
2.24
0.00
Zoen Blar
6.39
0.03
0.20
0.05
1.47
0.00
0.00
0.00
10.41
2.34
0.59
0.34
1.07
0.01
0.00
0.00
1.39
0.03
0.00
0.00
Men
7.24
0.38
0.10
0.03
0.51
0.05
0.00
3,01
0.00
Su
22.63
20.55
20.16
=007
+1.64
20.13
20.00
20.00
+3.03
+1.29
20.33
20.16
+2.20
=007
+0:00
#0.00
-2.01
20.10
=0.00
20.00
[i] R & S represents Rotation and Scaling. e represents perturbation threshold where i (1...., 4) with values {0.01, 0.02, 0.03, 0.04) respectively. Bold: Best performance across all models. Results averaged over 5 runs with standard deviation shown.
at = 0.01 on CIFAR-I0, confirming that orbit-invariant gradient regularization compounds benefi- cially when symmetry is enforced end-to-end. Complete results for fully equivariant architectures are provided in Appendix D
Our equivariant models achieve these robustness improvements without adversarial training. For context, we compare our 10-layer equivariant model against a standard CNN trained with PGD adversarial training in Appendix E

---

0.53 632 632 c25
645
6:8
AIl
ALL
610 534
531 632 647 6A0
011
9.20 9:31
FSSH Accurscy Epsiion (CIFAR-100)
PGD Accursoy vs Epsiion FAR-0010
Restie CCNA
0004
Figure 2: Adversarial robustness comparison of five models using 10-layer architectures on CIFAR-10 and CIFAR-100. Shaded regions represent standard deviation 5 random seeds.
Table presents the performance analysis of various models on CIFAR-10C under range of corruption types and perturbation levels. The Parallel GCNN model consistently achieved the best performance across most corruption types, especially for lower perturbation thresholds. Parallel GCNN with Rotation and Scale Equivariance does not perform so well compared with baseline model under the data corruption. We assess the robustness of the proposed Parallel GCNN models using the maximum invariant perturbation metric [111. which quantifies the largest input perturbations a model can tolerate without altering the model's prediction. To further understand the contribution of each equivariant convolutional module, we perform ablation studies under default settings. The experimental details and visualizations of these studies are provided in Appendix F
7 Conclusion
In this work, we conducted a systematic investigation into the role of architectural symmetry en- forcement in improving adversarial robustness. By incorporating rotation- and vcale-equivariant convolutions into standard CNNs, we demonstrated that symmetry-awaan models could achieve improved resilience against adversarial attacks without relying on adversarial training or extensive data augmentation. Our theoretical analysis showed that equivariant architectures reduced hypothesis space complexity, regularized gradient behavior, and yielded tighter CLEVER-certifes robustness bounds. These models consistently preserved classification margins under group transformations and suppressed gradient sensitivity in directions aligned with the data manifold's symmetry structure. Future work could extend these insights to larger-scale datasets, broader threat models, and more expressive network architectures.
10

---

Acknowledgment
This work was supported by the National Science Foundation under Grant No. #2346643 and OAC-2313191, the U.S. Department of Defense under Award No. #FA9550-23- 1-045 the U.S. Department of Education under Grant No. P1 P1162240151 and National Research Platform (NRP) Nautilus HPC cluster [[]2]. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation, the U.S. Department of Defense, or the U.S. Department of Education.
References
[1] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
[2] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry, Robustness may be odds with accuracy. International Conference on Learning Representations, 2019.
[3] Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang. Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving alignment and robustness with circuit breakers. Advances in Neural Information Processing Systems, 37: 83345-83373, 2024.
[4] Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks, In International conference on machine learning, pages 6105-6114. PMLR, 2019.
[5] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018
[6] Mingli Zhu, Shaokui Wei, Hongyuan Zha, and Baoyuan Wu. Neural polarizer: A lightweight and effective backdoor defense via purifying poisoned features. Advances in Neural Informa- tion Processing Systems, 36:1 132-112215 1153, 2023.
[7] Junhao Dong, Piotr Koniusz, Junxi Chen, and Yew-Soon Ong. Adversarially robust distillation by reducing the student-teacher variance gap. In European Conference Computer Vision, pages 92-111. Springer, 2025.
[8] Taco Cohen and Max Welling, Group equivariant convolutional networks. In International conference on machine learning, pages 2990-2999, PMLR, 2016.
[9] Daniel Worrall, Stephan Garbin, Daniyar Turmukhambetov, and Gabriel J Brostow, Har- monic networks: Deep translation and rotation equivariance. In Proceedings of the IEEE conference on computer vision and puttern recognition, pages 5028-5037, 2017.
[10] Jianqi Chen, Hao Chen, Keyan Chen, Yilan Zhang, Zhengxia Zou, and Zhenwei Shi. Diffusion models for imperceptible and transferable adversarial attack. IEEE Transactions Pattern Analysis and Machine Intelligence, 2024.
[1I] Joao Esteves, Ameesh Makadia, Kostas Daniilidis, and Adrien Gaidon. 3d object classification and retrieval with spherical cnns. In Advances in Neural Information Processing Systems, pages 12759-12770, 2018.
[12] Taco S Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical cnns. In International Conference on Learning Representations, 2018.
[13] Z. Zhou, Y. Zhou, Z. Zhang, L. Lyu, D. Yan, R. Jin, and D. Dou. Flexible, efficient, and stable adversarial attacks machine unlearning, In Proceedings of the 42nd International Conference on Machine Learning (ICML 25), Vancouver, Canada, July 13-19 2025.

---

[14] L. Wang, M. Nayyem, A. A. Rakin, K. Santosh, C. Zhang, and Y. Zhou. Explainability guuded defense: Attribution-awes -ware model refinement against adversarial data attacks. In Proceedings of the 25th IEFE International Conference on Data Mining (ICDM'2)) 25), Washington, DC, November 12-15 2025.
[15] J. Jia, J. Liu, C. Huo, Y. Shen, Y. Zhou, H. Dai, and D. Dou. Efficient federated learning with timely update dissemination. Knowledge and Information Systems (KAIS), 2025.
[16] J. Liu, B. Ma, Q. Yu, R. Jin, J. Zhou, Y. Zhou, H. Dai, H. Wang, D. Dou, and P. Valduriez. Efficient federated learning with heterogeneous data and adaptive dropout. ACM Transactions on Knowledge Discovery from Data (TKDD), 19(8):146:1-14633 2025.
[17] B. Palanisamy, L. Liu, Y. Zhou, and Q. Wang. Privacy-preserving publishing of multilevel utility-controlle graph datasets. ACM Transactions on Internet Technology (TOIT), 18(2): 24:1-24:21, 2018.
[18] Y. Zhou, J. Ren, D. Dou, R. Jin, J. Zheng, and K. Lee. Robust meta network embedding against adversarial attacks. In Proceedings of the 20th IEEE International Conference on Data Mining (ICDM'20) 20), pages
Sorrento, Italy, November 17-20 2020.
[19] Z. Zhang, Z. Zhang, Y. Zhou, Y. Shen, R. Jin, and D. Dou. Adversarial attacks on deep graph matching. In Advances in Neural Information P Processing Systems 33 (NeurIPS'20) 20), Virtual, December 6-12 2020.
[20] Y. Zhou, Z. Zhang, S. Wu, V. Sheng, X. Han, Z. Zhang, and R. Jin. Robust network alignment via attack signal scaling and adversarial perturbation elimination. In Proceedings of the 30th Web Conference (WWW'2/) 2/), pages -895, Virtual Event / Ljubljana, Slovenia, April 19-23 2021.
[21] X. Zhao, Z. Zhang, Z. Zhang. L. Wu, J. Jin, Y. Zhou, R. Jin, D. Dou, and D. Yan. Expressive 1-lipschitz neural networks for robust multiple graph learning against adversarial attacks. In Proceedings of the 38th International Conference on Machine Learning (ICML'21), pages 12719- 12735, Virtual Event, July 18-24 2021.
[22] J. Ren, Z. Zhang, J. Jin, X. Zhao, S. Wu, Y. Zhou, Y. Shen, T. Che, R. Jin, and D. Dou. Integrated defense for resilient graph matching. In Proceedings of the 38th International Conference on Machine Learning (ICML'21), pages 3982-8997, Virtual Event, July 18-24 2021.
[23] Z. Zhang, Z. Zhang, Y. Zhou, L. Wu, S. Wu, X. Han, D. Dou, T. Che, and D. Yan. Adversarial attack against lingull knowledge graph alignment. In Proceedings of the 26th Conference on Empirical Methods in Natural Language Processing (EMNIPP' pages 5320-5337, Virtual Event/ / Punta Cana, Dominican Republic, November 7-11 1111222 2021.
[24] Z. Zhang, J. Jin, Z. Zhang, Y. Zhou, X. Zhao, J. Ren, J. Liu, L. Wu, R. Jin, and D. Dou. Validating the lottery ticket hypothesis with inertial manifold theory. In Advances in Neural Information Processing Systems 34 (NeurIPS'2) 21), pages 30196-30210, Virtual, December 6-14 2021.
[25] G. Zhang, Y. Zhou, S. Wu, Z. Zhang, and D. Dou. Cross-lingual entity alignment with adversarial kernel embedding and adversarial knowledge translation. CoRR, abs//2004.7887, 2021.
[26] Y. Zhou, J. Ren, R. Jin, Z. Zhang. J. Zheng, Z. Jiang, D. Yan, and D. Dou. Unsupervised adversarial network alignment with reinforcement learning. ACM Transactions on Knowledge Discovery from Data (TKDD), 16(3):50:1-500 10::22,, 2022.
[27] J. Jin, Z. Zhang, Y. Zhou, and I.. Wu. Input-agnostic certified group fairness via gaussian parameter smoothing. In Proceedings of the 39th International Conference on Machine Learning (ICML'22), pages 10360, Baltimore, MD, July 17-23 2022.
[28] Z. Zhang, Y. Zhou, X. Zhao, T. Che, and L. Lyu. Prompt certified machine unlearning with randomized gradient smoothing and quantization. In Advances in Neural Information Processing Systems 35 (NeurIPS'22 22), New Orleans, LA, November 28-December 9 2022.
12

---

[29] J. Jin, J. Ren, Y. Zhou, L. Lv, J. Liu, and D. Dou. Accelerated federated learning with decoupled adaptive optimization. In Proceedings of the 39th International Conference on Machine Learning (ICMI.'2),, pages 00298-10322 Baltimore, MD, July 17-23 2022.
[30] T. Che, Z. Zhang, Y. Zhou, X. Zhao, J. Liu, Z. Jiang, D. Yan, R. Jin, and D. Dou. Federated fingerprint learning with heterogeneous architectures. In Proceedings of the 22nd IEEE International Conference on Data Mining (ICDM 22), pages 31-40, Orlando, FL, November 28-December 2022.
[31] J. Liu, J. Huang, Y. Zhou, X. Li, S. Ji, H. Xiong, and D. Dou. From distributed machine learning federated learning: A survey. Knowledge and Information Systems (KAIS), 64(4): 885-917, 2022.
[32] T. Che, Y. Zhou, Z. Zhang. L. Lyu, J. Liu, D. Yan, D. Dou, and J. Huan. Fast federated machine unlearning with nonlinear functional theory. In Proceedings of the 40th International Conference on Machine Learning (ICML'23) pages 4241 14164426,, Honolulu, HI. July 23-29 2023.
[33] J. Ren, Y. Zhou, J. Jin, L. Lyu, and D. Yan. Dimension-independear certified neural network watermarks via mollifier smoothing. In Proceedings of the 40th International Conference Machine Learning (ICML'23), pages 28976-29008, Honolulu, HI, July 23-29 2023.
[34] T. Che, J. Liu, Y. Zhou, J. Ren, J. Zhou, V. S. Sheng, H. Dai, and D. Dou. Federated learning of large language models with parameter-efficien prompt tuning and adaptive optimization. In Proceedings of the 28th Conference on Empirical Methods in Natural Language Processing (EMNLP'S 223), Singapore, December 6-10 2023.
[35] J. Liu, J. Jia, B. Ma. C. Zhou, Zhou, Y. Zhou, H. Dai, and D. Dou. Multi-job intelli- gent scheduling with cross-device federated learning. IEEE Transactions on Parallel and Distributed Systems (TPDS), 34(2):535- 551, 2023.
[36] J. Liu, Jia, T. Che, C. Huo, J. Ren, Y. Zhou, Dai, and D. Dou, FedASMU: Efficient asynchronous federated learning with dynamic staleness-aware model update. In Proceed- ings of the 38th AAAI Conference on Artificial Intelligence (AAAI pages 13900-13908, Vancouver, Canada, February 20-27 2024.
[37] J. Liu, T. Che, Y. Zhou, R. Jin, H. Dai, D. Dou, and P. Valduriez. AEDFL: Efficient asyn- chronous decentralized federated learning with heterogeneous devices. In Proceedings of the 24th SIAM International Conference on Data Mining (SDM'24), pages 833-841 Houston, TX, April 18-20 2024.
[38] Y. Zhou, Z. Zhang, Z. Zhang. L. Lyu, and W.-S. Ku. Effective federated graph matching, In Proceedings of the 41st International Conference on Machine Learning (ICML'24) Vienna, Austria, July 21-27 2024.
[39] Y. Xiao, 7. Zhang. Y. Fang. D. Yan, Y. Zhou,, W. Ku, and B. Hui. Advancing certified robustness of explanation via gradient quantization. In Proceedings of the 33rd ACM International Conference Information and Knowledge Management (CIKM' 24), pages 2596-2606, Boise, ID, October 21 21-2522 2024.
[40] J. Liu, J. Ren, R. Jin, 7. Zhang, Y. Zhou, P. Valduriez, and D. Dou. Fisher information-base efficient curriculum federated learning with large language models. In Proceedings of the 29th Conference on Empirical Methods in Natural Language Processing (EMNLP'22 24),, pages 10497-10523, Miami, FL. November 12- 2024.
[41] J. Liu, J. Jia, H. Zhang, Y. Yun, L. Wang. Y. Zhou, H. Dai, and D. Dou. Efficient federated learning using dynamic update and adaptive pruning with momentum on shared server data, ACM Transactions on Intelligent Systems and Technology (TIST), 15(6):
2024.
[42] Zhou, H. Cheng, and J. X. Yu. Clustering large attributed graphs: An efficient incremen- tal approach. In Proceedingy of the 10th IEEE International Conference on Data Mining (ICDM'IO 10), pages 689-698 Sydney, Australia, December 14-17 2010.
13

---

[43] Y. Zhou, H. Cheng, and J. X. Yu. Graph clustering based on structural/attribute similarities. Proceedings of the VLDB Endowment (PVLDB). 2(1):718-722 2009.
[44] H. Cheng, Y. Zhou, and J. X. Yu. Clustering large attributed graphs: A balance between structural and attribute similarities, ACM Transactions on Knowledge Discovery from Data (TKDD), (2)):-33, 2011.
[45] Y. Zhou and L.. Liu. Clustering analysis in large graphs with rich attributes. In D. E. Holmes and L. C. Jain, editors, Data Mining: Foundations and Intelligent Paradigms: Volume 1: Clustering, Association and Classification. Springer, 2012.
[46] H. Cheng, Y. Zhou, X. Huang, and J. X. Yu. Clustering large attributed information net- works: An efficient incremental computing approach. Data Mining and Knowledge Discovery (DMKD), 25(3):450 2012.
[47]
Longwei Wang and Qilian Liang. Representation learning and nature encoded fusion for heterogeneous sensor networks. IEEE Access, 7:39227-39235, 2019.
[48] Longwei Wang, Wen Chen, and Jun Li. Congestion aware dynamic user association in heterogeneous cellular network: A stochastic decision approach. In 2014 IEEE International
Conference on Communications (ICC), pages (636-2640. IEEE, 2014.
[49] Longwei Wang, Aashish Ghimire, K Santosh, Zheng Zhang, and Xueqian Li. Enhanced robustness by symmetry enforcement. IEEE CAI, 2024.
[50] Y. Zhou and L. Liu. Social influence based clustering of heterogeneous information networks. In Proceedings of the 19th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD' 13), pages 338 -366, Chicago, IL, August 11 11-14 2013.
[51] Z. Su, L. Liu, M. Li, X. Fan, and Y. Zhou. ServiceTrust: Trust management in service provision networks. In Proceedings of the 10th IEEE International Conference on Services Computing (SCC'13), pages 272-279, Santa Clara, CA, June 27-July 2 2013.
[52] Q. Zhang, L. Liu, Y. Ren, K. Lee, Y. Tang, X. Zhao, and Y. Zhou. Residency aware inter- VM communication in virtualized cloud: Performance measurement and analysis. In Proceedings of the 2013 IEEE International Conference on Cloud Computing (CLOUD' 13), pages 204-211, Santa Clara, CA, June 27-July 2 2013.
[53] Y. Zhou and L. Liu. Activity centric multi-label classification for mining heterogeneous information networks. In Proceedings of the 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD' 14), pages 1276-1288 New York, NY, August 24 24 2014.
[54] 7. Su, Liu, M. Li, X. Fan, and Zhou. Reliable and resilient trust management in distributed service provision networks. ACM Transactions on the Web (TWEBB, 9(3): 2015.
[55] Y. Zhou and L. Liu. Social influence based clustering and optimization over heterogencous information networks. ACM Transactions on Knowledge Discovery from Data (TKDD), 10(1): 1-53, 2015,
[56] Y. Zhou, L. Liu, and D. Buttler. Integrating Vertex-centric clustering with edge-centric clustering for meta path graph analysis. In Proceedings of the 2 ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD' ())) pages 1563-1572, Sydney, Australia, August 10-13 2015.
[57] X. Bao, L. Liu, N. Xiao, Y. Zhou, and Q. Zhang. Policy-driven autonomic configuration management for nosql. In Proceedings of the 2015 IEEE International Conference on Cloud Computing (CLOUD' 15), pages 245-252 New York, NY, June 27-July 2 2015.
[58] Y. Zhou, L. Liu, K. Lee, C. Pu, and Q Zhang. Fast iterative graph computation with resource aware graph parallel abstractions. In Proceedings of the 24th ACM Symposium on High- Performance Parallel and Distributed Computing (HPDC' 15), pages 179-190, Portland, OR, June 15-19 2015.
14

---

[59] Y. Zhou, L. Liu, S. Seshadri, and L. Chiu. Analyzing enterprise storage workloads with graph modeling and clustering. IEEE Journal on Selected Areas Communications (JSAC), 34(3): 551-574, 2016.
[60] Y. Zhou. Innovative Mining, Processing, and Application of Big Graphs. PhD thesis, Georgia Institute of Technology, Atlanta, GA, USA, 2017.
[61] Y. Zhou, S. Wu, C. Jiang, Z. Zhang, D. Dou, R. Jin, and P. Wang. Density-adaptinn local edge representation learning with generative adversarial network multi-label edge classification. In Proceedings of the 18th IEEE International Conference on Data Mining (ICDM' 18), pages 1464-1469, Singapore, November 17-20 2018.
[62] Y. Zhou, A. Amimeur, C. Jiang, D. Dou, R. Jin, and P. Wang. Density-aware local siamese autoencoder network embedding with autoencoder graph clustering. In Proceedings of the 2018 IEEE International Conference on Big Data (BigData 18), pages 1162 1167, Seattle, WA, December 10-13 2018.
[63] Longwei Wang, Chengfei Wang, Yupeng Li, and Rui Wang. Explaining the behavior of neuron activations in deep neural networks, Ad Hoe Networks, 111:102346, 2021.
[64] Longwei Wang. Chengfei Wang. Yupeng Li, and Rui Wang. Improving robustness of deep neural networks via large-difference transformation. Neurocomputing. 450:411 1-119, 2021.
[65] Kenan Xiao, Longwei Wang, Ashish Gupta, and Xiao Qin. Looking beyond content: Modeling and detection of fake news from a social context perspective. In HICSS, pages 1-10, 2022.
[66] J. Ren, Y. Zhou, R. Jin, Z. Zhang. D. Dou, and P. Wang. Dual adversarial learning based network alignment. In Proceedings of the 19th IEEE International Conference on Data Mining (ICDM' 19), pages 1 1288-1293 Beijing, China, November 8-11 2019.
[67] Y. Zhou, L. Liu, Q Zhang, and B. Palanisamy. Enhancing collaborative filtering with multi- label classification. Proceedings of the 2019 International Conference on Computational Data and Social Networks (CSoNet' 19), pages 323-338, Ho Chi Minh City, Vietnam, Novem- ber 18-20 2019.
[68] Y. Zhou, C. Jiang, Z. Zhang, D. Dou, R. Jin, and P. Wang. Integrating local vertex/edge embedding via deep matrix fusion and siamese multi-label classification. In Proceedings of the 2019 IEEE International Conference on Big Data (BigData' 19), pages 1018-1027, Los Angeles, CA, December 9-12 2019.
[69] Y. Zhou, Ren, S. Wu, D. Dou, Jin, Z. Zhang, and Wang. Semi-super classification- based local vertex ranking via dual generative adversarial nets. In Proceedings of the 2019 IEEE International Conference on Big Data (BigData' 19), pages 1267-1273, Los Angeles, CA, December 9-12 2019.
[70] Y. Zhou and L. Liu. Approximate deep network embedding for mining large-scale graphs. In Proceedings of the 2019 IEEE International Conference on Cognitive Machine Intelligence (CogMI' 19), pages 53 60, Los Angeles, CA, December 12 14 2019.
[71] S. Wu, Y. Li, D. Zhang, Y. Zhou, and Z. Wu. Diverse and informative dialogue generation with sontext-specific commonsense knowledge awareness. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL'20), pages 581 5811-5820, Online, July 5-10 2020.
[72] S. Wu, Y. Li, D. Zhang. Y. Zhou, and Z. Wu. TopicKA: Generating commonsense knowledge- aware dialogue responses towards the recommended topic fact. In Proceedings of the Twenty- Ninth International Joint Conference on Artificial Intelligence (IJCAI'20) 20), pages Online, January 7-15 2021.
[73] Y. Zhou, J. Ren, R. Jin, Z. Zhang, D. Dou, and D. Yan. Unsupervised multiple network alignment with multinominal GAN and variational inference. In Proceedings of the 2020 IEEE International Conference on Big Data (BigData 20), pages 868-877, Atlanta, GA, December 10 13 2020.
15

---

[74] Longwei Wang, Peijie Chen, Chengfei Wang. and Rui Wang. Layer-wise entropy analysis and visualization of neurons activation. In International Conference on Communications and Networking in China, pages 29-36. Springer International Publishing Cham, 2019.
[75] Longwei Wang, Xueqian Li, and Zheng Zhang. Dense cross-connected ensemble convolutional neural networks for enhanced model robustness. arXiv preprint arXiv:2412.07022, 2024.
[76] Navid Nayyem, Abdullah Rakin, and Longwei Wang. Bridging interpretability and robustness using lime-guided model refinement. arXiv preprint arXiv:2412./8952 2024.
[77] Longwei Wang, Navid Nayyem, and Abdullah Rakin. Enhancing adversarial robustness of deep neural networks through supervised contrastive learning, arXiv preprint arXiv:2412.19747, 2024.
[78] Longwei Wang, Ifrat Ikhtear Uddin, Xiao Qin, Yang Zhou, and KC Santosh. Explainability- driven defense: trad-cam--uided model refinement against adversarial threats. In Proceedings of the AAAI Symposium Series, volume 6, pages 49-57, 2025.
[79] Robin Narsingh Ranabhat, Longwei Wang, Xiao Qin, Yang Zhou, and KC Santosh. Multi- scale unrectified push-pull with channel attention for enhanced corruption robustness. In Proceedings of the AAAI Symposium Series, volume 6, pages 34-41, 2025.
[80] Ifrat Ikhtear Uddin, Longwei Wang, and KC Santosh. Expert-guided explainable few-shot learning for medical image diagnosis. arXiv preprini arXiv:2509.080// 080000 2025.
[81] Y. Zhou, L. Liu, K. Lee, B. Palanisamy, and Q. Zhang. Improving collaborative filtering with social influence over heterogeneous information networks. ACM Transactions on Internet Technology (TOIT), 20(4):36:1 36:29, 2020.
[82] R. Jin, D. Li, J. Gao, Z. Liu, L. Chen, and Y. Zhou. Towards better understanding of linear models for recommendation. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery and Data Mining (KDD'21), pages 776-785, Virtual Event, Singapore, August 14 18 2021.
[83] Robin Narsingh Ranabhat, Longwei Wang, Amit Kumar Patel, al. Promoting shape bias in cnns: Frequency- based and contrastive regularization for corruption robustness. arXiv preprint arXiv:2509. 11 1355,, 2025.
[84] S. Wu, M. Wang, D. Zhang, Y. Zhou, Y. Li, and Wu. Knowledge-aware dialogue genera- tion via hierarchical infobox accessing and infobox-dialogue interaction graph network. In Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI'2)). pages 3964-39777 3970, Virtual Event / Montreal, Canada, August 19 19-27 2021.
[85] S. Wu, Y. Li, M. Wang, D. Zhang, Y. Zhou, and Z. Wu. More is better: Enhancing open- domain dialogue generation via multi-source heterogeneous knowledge. In Proceedings of the 26th Conference Empirical Methods in Natural Language Processing (EMNLP'2I) 211), pages 2286-2300, Virtual Event Punta Cana, Dominican Republic, November 7-11 2021.
[86] Guimu Guo, Da Yan, Lyuheng Yuan, Jalal Khalil, Cheng Long, Zhe Jiang, and Yang Zhou. Maximal directed quasi-clique mining. In 2022 IEEE 38th international conference data engineering (ICDE), pages 1900-1913. IEEE, 2022.
[87] Zhiqiang Chen, Yang Chen, Xiaolong Zou, and Shan Yu. Continuous rotation group equivariant network inspired by neural population coding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 1462--1477, 2024.
[88] Hayder Flesedy. Symmetry and generalisation in machine learning. arXiv preprint TrXiv:2501.038.5 2025.
[89] Sharvaree Vadgama, Mohammad Mohaiminul Islam, Domas Buracus, Christian Shewmake, and Erik Bekkers. On the utility of equivariance and symmetry breaking in deep learning architectures on point clouds. arXiv preprint arXiv:2501.01999 2025.
16

---

[90] Daniel Worrall and Max Welling. Deep scale-spaces; Equivariance over scale. Advances in Neural Information Processing Systems, 32, 2019.
[91] Maurice Weiler and Gabriele Cesa. General e (2)-equivariant steerable cnns. Advances in neural information processing systems, 32, 2019.
[92] Ramzan Basheer and Deepak Mishra. Current symmetry group equivariant convolution frameworks for representation learning. arXiv preprint vrXiv:240.003327 2024.
[93] Maksim Zhdanov, Nabil Iqbal, Erik J Bekkers, and Patrick Forré. Ads-gnn-a conformally equivariant graph neural network. In ICLR 2025 Workshop on Machine Leurning Multiscale Processes.
[94] Kristof Schütt, Picter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller. Schnet: A continuous- convolutional neural network for modeling quantum interactions. Advances in neural information processing systems, 30, 2017.
[95] lan Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adver- sarial examples. arXiv preprint vrXiv:1412.6522 2014.
[96] Jon Vadillo, Roberto Santana, and Jose Lozano. Adversarial attacks in explainable machine learning: A survey of threats against models and humans. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 15(1):e1567, 2025.
[97] Changhoon Kim, Kyle Min, and Yezhou Yang. Race: Robust adversarial concept erasure for secure text-to-image diffusion model. In European Conference Computer Vision, pages 461-478. Springer, 2024.
[98] Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pages 501-509, 2019.
[99] Chuan Guo, Mayank Rana, Moustapha Cissé, and Laurens van der Maaten. Countering adversarial images using imput transformations. In International Conference on Learning Representations, 2018
[100] Stephen Casper, Carson Ezell, Charlotte Siegmann, Noam Kolt, Taylor Lynn Curtis, Benjamin Bucknall, Andreas Haupt, Kevin Wei, Jérémy Scheurer, Marius Hobbhahn, et al. Black-box access is insufficient for rigorous ai audits. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 2254-2272, 2024.
[101] Sheng Y Peng, Pin- Yu Chen, Matthew Hull, and Duen Chau. Navigating the safety land- scape: Measuring risks in finetuning large language models. Advances in Neural Information Processing Systems, 37:95692 957115, 2024.
[102] Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning, pages 274-283. PMLR, 2018
[103] Desheng Zheng, Wuping Ke, Xiaoyu Li. Yaoxin Duan. Guangqiang Yin, and Fan Min. Enhanc- ing the transferability of adversarial attacks via multi-fcature attention. IEEE Transactions on Information Forensies and Security, 2025.
[104] Xu Zhang, Bo Peng, Jianjun Lei, Chao Xue, Yuxuan Yao, and Qingming Huang. Adversarially robust object detection via deviation calibration and content preservation. IEEE Transactions on Circuits and Systems for Video Technology, 2025.
[105] Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian J Goodfellow, Dan Bonch, and Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. In International Conference Learning Representations, 2018.
[106] Tsui-Wei Weng, Huan Zhang, Pin- Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. Evaluating the robustness of neural networks: An extreme value theory approach. In International Conference on Learning Representations, 2018.
17

---

[107] Fabio Anselmi, Georgios Evangelopoulos. Lorenzo Rosasco, and Tomaso Poggio. Symmetry- adapted representation learning. Pattern Recognition, 86:201-208, 2019.
[108] Fabio Anselmi, Lorenzo Rosasco, and Tomaso Poggio. On invariance and selectivity in representation learning. Information and Inference: A Journal of the IMA, 5(2):134-158, 2016.
[109] Fabio Anselmi, Joel Z Leibo, Lorenzo Rosasco, Jim Mutch, Andrea Tacchetti, and Tomaso Poggio. Unsupervised learning of invariant representations. Theoretical Computer Science, 633:112-121, 2016.
[110] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness common corruptions and perturbations. In International Conference on Learning Representations, 2019.
[111] Satoshi Hara, Kouichi Ikeno, Tasuku Soma, and Takanori Machara. Maximally invariant data perturbation as explanation. arXiv preprint arXiv: /806.880044 2018.
[112] Derek Weitzel, Ashton Graves, Sam Albin, Huijun Zhu, Frank Wuerthwein, Mahidhar Tatineni, Dmitry Mishin, Elham Khoda, Mohammad Sada, Larry Smarr, et al. The national research platform: Stretched, multi-tenant, scientific kubernetes cluster. In Practice and Experience in Advanced Research Computing 2025: The Power of Collaboration, pages 1-5, 2025.
18

---

A Technical Appendices and Supplementary Material
A.1 Proof of Theorem
Proof. We begin by computing the gradient of the margin function Jc,j at the transformed input g x. Given the equivariance property:
f(g x) = p(g)
we write the individual logits as:
figg x) = [x())((x)),
Σ Pim (g) (x).
Assuming p(g) is diagonal (as in classification tasks where each logit transforms independently), the above reduces to:
fi x) (ix()))(x()
Thus, the margin function transforms
g x) felg x) - fz(g x) = - (is(())
Now compute the gradient using the chain rule. Since x is diffcoomorphism and f is differentiable, we have:
Vo x) = Valeos2)) Vgasla x) - Joc, (g (a- x) = (fela x) - f5l1 x))
Applying the chain rule to each term:
V felg x) v (pcc(s)fe(x)) Poolg) Vc(()) D
ff Pji(g)) Vff(x) D9
Hence:
Voailg = Vielg x) VSslg x)
= fco( PDs (x
(f)) - -P55 D.
We now compute the noom::
Ve x)11 (o P))(
Dg '1l '11.
Assume that Per (g)
{+1,- -1} and that Dg orthogonal:
|Dg vvlg || VVER E.d
Then:
VV x)l1,
(o Dg Dg '11
(g) - Pis V1)
[V
Therefore, the norm of the gradient of the margin function is invariant under the group action:
[Vgc, x)l1
[ Vg E.
Since this holds for every g E G, the Lipschitz constant of ga.j over the group orbit [x]c satisfies:
(1) = sup
V ll. = [[
x x
This completes the proof.
19

---

A.2 Proof of Theorem 2
Proof. The key idea is that group equivariance induces structured regularity over the orbit of the imput space. For any 9 E G, by applying the chain rule to the equivariant condition f(g x) = (()) we obtain the Jacobian transformation law:
Jr(9 x) - p(g)Jr(x)Dg
(2)
where Dg 1 E gdxd denotes the Jacobian the inverse group action g x. Because both p(g)
and Dg orthogonal, the Jacobian norm is preserved:
[ x)l12 = ||J (x)
(3)
Now define the orbit-averaged gradient for each class logit ʃj (x)
as:
xx) = |G|
Σ Vfzlg- (x(gxx). x).
(4)
DEG
Using Eq. (2. we get:
(x)
Σ gEG
P (L)VV\(())
(5)
G
This is linear averaging operator acting on (x)( which has the effect of projecting the gradient onto the invariant subspace under the group. High-frequency components in directions orthogonal to
Tx (xxlc) tend to cancel due to the group symmetries, while components aligned with the orbit are reinforced.
We now analyze the gradient difference along perturbations in each direction.
Case 1: Perturbation dG E Tx(()]):
Let = x xx+ Since 8G lies tangent to the orbit, we have x' [x]a, and hence by the equivariance property and Eq. (3).
vf(x') Vf(xx)
and thus,
VV 86) ff(x) l2 0.
Case 2: Perturbation 8_ T [x([)]))
Let x" x + -1. Because this point lies off' the symmetry manifold, the equivariant structure
offers no constraint on the variation of the gradient. Consequently,
|| 51) V/(x)ll
not bounded by symmetry and may vary substantially especially esppeclll directions where V contains high-frequency components not captured by the group action.
Moreover, the difference between smoothed and unsmoothed gradients satisfies:
(x) (x) < [ 81) (xx)))
(6)
This shows that the orbit-averaged gradient stabilizes variation, while off-orbit perturbations signifi- cantly alter the local gradient field.
Combining Eqs. A.2 and (6. conclude:
V V > ||V 5G) Vf(()
which proves the theorem.
B Limitations
Despite its theoretical and empirical strengths, this work has several limitations. First, the theoretical framework focuses solely on local Lipschitz bounds (e.g., CLEVER), which may not capture broader robustness phenomena such as margin distributions adversarial risk. Second, empirical evaluations are limited small-scale datasets and common f,-norm attacks, leaving open questions about scalability and robustness to more diverse threat models. Finally, the integration of equivariant layers confined to early stages of the network, and the computational trade-offs of group convolutions remain unquantified.
20

---

Equivariance Enhanced Architectural Designs
C.1
Group Equivariant Convolutions
Group Equivariant Convolutional Networks (G-CNNs) extend the conventional convolutional frame- work by embedding symmetry priors directly into the architecture [8] These models are designed to maintain cquivariance under transformations defined by a group G, such as rotations, translations, or scalings.
To implement such equivariance in convolutional neural networks, the standard convolution operation replaced with group convolation that aggregates information over the transformed versions of both input and filters. We now instantiate this framework for two practically important cases: rotation and scale transformations.
C.1.1 Rotation-Equivariant Convolutions
To embed rotational symmetry into the network, we consider a discrete rotation group = P4, which consists of four planar rotations: 0º, 9,90; 180", and 270". In this setting, group convolution is defined as:
[f * gl()) = ((ooo))g(()
NEG
where f is the input feature map, g(v) is the group-transformel filter corresponding to transformation v E G. and u denotes the spatial coordinate. This formulation guarantees that the resulting feature map transforms predictably under group actions:
[f * وماو u) = p(g)(ff * (u), Vg E G,
(8)
thereby satisfying the equivariance condition in Equation (T). By construction, these layers promote feature consistency across rotated versions of the input, enhancing robustness and generalization.
C.1.2 Scale-Equivariant Convolutions
Scale transformations are another common form of variability in visual data, especially when objects appear at different sizes distances. To build scale-equivariant architectures, define discrete scale group G, = {01, 02 O3k C R=,, which acts on the input space via isotropic resizing:
a x Resize((, a), a Gs:
where Resize(æ, a:) uniformly scales the spatial dimensions of the input by factor a, typically using bicubic interpolation. Each scaled input is then processed independently using a shared convolutional function fscates resulting in multiple feature maps:
Facale, [scate(a; x.), i= l. k.
(10)
These outputs are then resized back to common resolution and combined via an aggregation operation, such channel-wist concatenation averaging:
Fscate (x: = Aggregate FFcale. F F)cale,k)-
(11)
The resulting representation is equivariant the scale group G., satisfying:
Fscate (3 x)) = scateFs VB G.,
(12)
where p(3) is the corresponding transformation (e.g., channel permutation spatial rescaling) applied to the feature space. This construction allows the network maintain consistent semantic representations across range of object sizes, improving its ability to generalize under scale variation,
21

---

Parallal Design
Cascaded Design
Equie
Corvcibio
Comokitn
Figure 3: Parallel design of the CNN architecture. Input is processed through standard, rotation- equivariant, and scale-equivariant branches in- dependently. The resulting features are subse- quently fused to form unified representation.
Figure 4: Cascaded design of the CNN architec- ture. Convolutional layers are applied sequen- tially: standard, then rotation-equivariin fol- lowed by scale-equivariant.
Both rotation- and scale-cquivarian convolutions are special cases of the general group-equivariant convolutional framework. They embed structured inductive biases into the network that enable efficient learning and enhanced generalization.
C.2 Equivariance Enforced Architectural Designs
We explore architectural strategies that combine standard, rotation-equivariant, and scale- equivariant convolutions for robust feature extraction: the parallel design and the cascaded design. Each strategy targets different trade-off between representation diversity and processing efficiency.
C.3 Parallel Design
In the parallel design, the input is processed simultancously through three branches: standard convolutional branch for translational equivariant features, rotation-equivarii branch that encodes orientation equivariance via group convolutions, and a scale-equivariant branch that captures multi- scale features.
Let RC*Hx denote the input image, where C, H, and W are the number of channels and the spatial height and width, respectively. The outputs from each branch are given by:
F standard = fstandaard F Foot = fxx((x),
F Fscale = fscale(Xx (x),
(13)
where fstandard frots and fscale are the respective processing functions. The final representation is obtained by fusing the branch outputs:
fused g standard, F rot scaaee;;
(14)
where g()) denotes the fusion strategy. Figure 3 illlssrrate this design, emphasizing the independence of the branches and their contribution to robust feature diversity.
C.4 Cascaded Design
In contrast to the parallel strategy, the cascaded design applies equivariant operations sequentially. The input first passes through standard convolutional layer, followed by rotation aaatioonequivariann processing, and finally ealiveqquvarriant processing:
Foot Fro( standardd (x), seale = fscale (Fra);
(15)
resulting in the final representation:
F Fiused = g (FFsaate)
(16)
This design simplifies the model by reusing intermediate representations. However, may also introduce feature redundancy or suppress early-layer diversity, as subsequent operations act on already transformed representations. Figure depicts the cascaded architecture, highlighting the flow of information through progressively specialized layers.
22

---

Table Adversarial robustness of fully equivariant architectures on CIFAR-10 (%). Deeper models benefit from compounding gradient regularization when equivariance is enforced end-to-end.
Architecture
g
0.01
E
0.03
0.10
FGSM
PGD
FGSM
PGD
FGSM
PGD
FGSM
PGD
4- Layer Equivariant 65.65
52.20
53.78
23.30
47.08
15.85
37.95
7.01
10- 10-Layer Equivariant
73.01
64.96
67.09
52.37
60.23
37.80
44.93
12.46
Table 3: Adversarial robustness of fully equivariant architectures CIFAR-100 100 (%). Consistent improvements across depth validate that orbit-invariant gradient regularization scales effectively.
0.01
0.03
0.05
0.10
Architecture
FGSM
PGD
FGSM
PGD
FGSM
PGD
FGSM
PGD
Layer Equivariant
38.40
21.59
26.63
8.92
22.06
5.66
15.96
2.39
10-Layer Equivariant
50.60
36.29
42.02
21.34
36.09
12.01
24.68
4.08
D Fully Equivariant Architectures
To validate that our theoretical predictions hold for end-to-end equivariant models, we evaluate architectures where all convolutional layers enforce rotation equivariance under the P4 group (4 discrete rotations). These models are constructed by sequentially stacking rotation-equivariant blocks, with each block comprising equivariant convolution, group-aware batch normalization, ReLU activation, and max pooling.
Tables 2 and demonstrate that fully equivariant models achieve substantial adversarial robustness without adversarial training. The 10-layer architecture consistently outperforms the 4-layer variant across all perturbation levels, with improvements of 7-15% in FGSM accuracy and 10-30% in PGD accuracy at moderate perturbations. This depth-dependent improvement validates our theoretical prediction: when symmetry constraints are enforced throughout the network, orbit-invariant gradient regularization compounds beneficially across layers. This contrasts sharply with standard CNNs, where increased depth often amplifies adversarial vulnerability. The results confirm that equivariance must treated as a network-wide architectural principle rather than localized feature extraction, demonstrating that certified robustness guarantees translate into measurable improvements under practical gradicnt-basee attacks.
E
Comparison with Adversarial Training
To contextualize the robustness improvements from architectural symmetry enforcement, we compare our approach against standard adversarial training. Table Table[ppreseee presents results on CIFAR-10, where standard CNN trained with PGD-based adversarial training is compared against 10-layer fully cquivariant G-CNN trained without any adversarial examples.
Our equivariant model achieves superior FGSM robustness and maintains competitive PGD robustness despite not being trained on adversarial examples. This demonstrates that symmetry-aware archi- tectures provide intrinsic robustness through geometric constraints alone, offering complementary and potentially more efficient alternative to data augmentation- based defenses. The slight advantage in FGSM accuracy and competitive PGD performance suggest that equivariant priors effectively regularize the decision boundary without the computational overhead of adversarial training.
Table 4: Comparison with Adversarial Training on CIFAR-
0.01
0.02
0.03
0.04
0.05
Model
FGSM
PGD
FGSM
PGD
FGSM
PGD
FGSM
PGD
FGSM
PGD
CNN + AT
74.5
67.0
70.2
60.4
66.1
54.0
61.7
48.3
57.3
42.1
G-CNN (no AT)
73.01
64.96
70.16
58.87
67.09
52.37
63.77
45.52
60.23
37.80
23

---

Original Image
Standard CNN(0.16)
Group CNN(0..4)
Original Image
Standard CNN(0.05)
Group CNN(0.15)
Figure 5: Visualization of perturbation tolerance for the Bascline CNN and the Parallel GCNN. The top row shows the original image (left), followed by the maximum perturbation (== 0.16) for the Baseline CNN (middle), and the maximum perturbation (c 0.44) for the Parallel GCNN (right).
Similarly, the bottom row shows the original image (left), with the maximum perturbation (c = 0.05) for the Baseline CNN (middle), and the maximum perturbation 0.15) for the Parallel GCNN (right).
Perturbation Tolerance Visualization and Ablation Study
F.1 Visualization
We evaluated the robustness of the Parallel GCNN models using the metric of maximum invariant perturbation [II]] This metric quantifies the highest level of perturbations that model can withstand without significant degradation in performance. The visualization the figure compares the perturbation levels tolerated by the baseline CNN and the Parallel GCNN As shown in the figure 5] the Parallel GCNN model demonstrates a substantially higher tolerance for perturbations compared to the bascline CNN. For example, the baseline CNN can tolerate perturbation rate of e 0.05, whereas the Parallel GCNN maintains stable predictions even at higher perturbation levels such as 0.15.
F.2 Ablation Study
To evaluate the influence of equivariance convolution modules on the proposed model, we con- ducted ablation studies in default settings. The experiments compare the performance of four models-Baseline CNN, Parallel GCNN with Rotation- Eqqivaria Branch (Rot), Parallel GCNN with Scale-Equivariant Branch (Scale), and Parallel GCNN with Rotation- and Scale-Fquivariant Branch (Rot& Scale) on CIFAR-100 under FGSM and PGD attacks across varying perturbation levels. The Parallel GCNN with Rotation- and Scale- Equlv--ariari Branch consistently outperformed other models, achieving the highest robustness across all settings. Rotation-equivarinn convolutions demonstrated better performance than scale-equivariamt convolutions individually, particularly under higher perturbation magnitudes. The Baseline CNN lacked adversarial robustness, showing the superiority of symmetry- aware architectural designs.
Under the more challenging PGD attacks, the Parallel GCNN with Rotation- and Scale-Equivariant Branch continued to outperform other models. At 0.01, it achieved 29.62% accuracy, compared to 24.72% for the Rotation-FEi Equivvaantt GCNN, 21 21.34% for the Baul--aquvaaraatt GCNN, and only 2.49% for the Baseline CNN. Even 0.04, the combined model retained 6.04% accuracy,

---

TGGM Modic CIAR--10
Acaraty
Roulte 12hM (6-1
Firald
COW Mitt Scand
GCM (he Scla
2011
414
101
Figure 6: Ablation study results on CIFAR-100 comparing the adversarial robustness of four models under FGSM (left) and PGD (right) attacks.
while the Rotation-Equivariant GCNN and Scale-Equivariant GCNN dropped to 3.81 % and 1.38%, respectively. The Baseline CNN showed minimal robustness higher perturbations, with accuracy falling below 1 1%.
Our experiments reveal several key insights for the integration of the equivariance layer in the CNN model. The parallel design consistently outperforms the cascaded design in both clean and adversarial settings. By maintaining the independence of standard and equivariant properties, the parallel design achieves diverse and complementary feature representations that enhance adversarial resilience. Within the parallel design, simple concatenation of outputs consistently delivers higher robustness against FGSM and PGD attacks. This approach retains richer and more balanced feature representations, which crucial for adversarial defense. Weighted sum fusion, while effective clean data, is more vulnerable to ad adversarial attacks due to over-reliance on learned weights, which may fail generalize under adversarial perturbations. The cascaded design exhibits diminished performance and robustness due to feature redundancy. The sequential dependence between standard and equivariant layers restricts the network's ' ability to capture diverse patterns, leading to reduced effectiveness in adversarial scenarios.
G Experiments Computing Resources
All experiments were performed on the Lawrence Supercomputer and NRP Nautilus HPC systems. Training and evaluation used single GPU per experiment. Node specifications are provided in T
Table 5: Hardware configuration for experiments
Component
Configuration
CPUs
Dual 12-core SkyLake 5000 series
GPUs
Nvidia Tesla P100 16GB or Nvidia RTXA6000 64GB
RAM
64GB
SSD
240GB
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims made in the abstract and introduction are clearly aligned with the contributions and scope of the paper. The paper emphasizes enhancing adversarial robustness through equivariant architectures and demonstrates improved results on CIFAR- 10, CIFAR-I00, 100, and CIFAR-IOC datasets using group-equivariant convolutions. The claims are supported by both theoretical analysis and experimental results.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We have discussed the limitations of our methodology in Appendix B where reflect on potential areas for improvement and further exploration
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
The theoretical analysis is clearly presented, with detailed assumptions and proofs, particularly regarding the relationship between equivariance and adversarial robust- ness. The paper provides a formal framework showing how equivariant architectures reduce hypothesis space complexity, regularize gradients, and tighten certified robustness bounds using the CLEVER framework.
Justification:
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
The paper fully discloses the experimental setup, including the models, datasets, and evaluation strategies. It details the comparison of various models, the attack types used (FGSM and PGD), and experimental configurations, making it possible for others to replicate the results.
Justification:
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We will provide open access to the code. The datasets used (CIFAR-10. CIFAR- 100, and CIFAR-10C) are publicly available. Once the code is shared, the experiments can be fully reproduced using these datasets. For CIFAR-IOCC users will need to download it from an external source and update the data path accordingly. However, for CIFAR-10 and CIFAR-100, additional setup is -imill run the provided code.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental details are well described, including the split of the data set (CIFAR-10, CIFAR-100, CIFAR1OC), the attack methods (FGSM and PGD), and the architecture details.
26

---

7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Error bars representing =1 standard deviation over random seeds are shown shaded regions in Figures | and Standard deviations are consistently below 1 % across all models and experimental conditions (average std <0.8%).
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In the appendix section of this paper, we have clearly outlined the hardware configuration used to run the experiments for the proposed methods.
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httppp////arrrr....ooo////////aa
Answer: [Yes]
The research adheres to the NeurIPS Code of Ethics, focusing on improving adversarial robustness in machine learning models without any known ethical issues related to data privacy, fairness, or security. There is no mention of potential ethical concerns or violations.
Justification:
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
The paper focuses primarily on improving the adversarial robustness of AI models, with specific emphasis enhancing model resilience in challenging conditions. While the robustness these models may have potential real-world applications, such as in medical science and autonomous vehicles, our work does not directly address the societal impacts of these applications. Therefore, the paper does not explore societal implications, and this question is marked NA.
Justification:
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
The paper focuses solely enhancing the adversarial robustness of Al models to reduce mistakes challenging conditions. As such, does not involve the release of data models that carry a high risk of misuse, and therefore, no specific safeguards are required
Justification:
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
The datasets used in this paper (CIFAR-10, CIFAR-100, and CIFAR-10C) are publicly available. We have ensured that the terms of use for these datasets are respected, and the corresponding citations are included in the paper.
Justification:
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
27

---

Answer: [No]
Justification: Our work does not create new assets such as datasets, however, we have shared code.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve crowdsourcing or research with human subjects. focuses improving adversarial robustness in AI models, and no human participants or crowdsoureing methods were used in the experiments.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Our Work does not involve research with human subjects, and therefore, IRB approval or equivalent was not required. The focus of the paper on improving adversarial robustness in AI models, which does not involve human participants.
Justification:
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The paper does not use large language models (LLMs) as part of the core methodology research. LLMs were used only for editing and rephrasing text, such as improving grammar. This usage did not impact the scientific rigor or originality of the research.
28


Reviews:
[{"review_id": "kcNVOml9tk", "reviewer": "Reviewer_BaeD", "summary": "The authors present three key theoretical findings regarding the correspondence of equivariance and adversarial robustness: group-equivariant convolutions (1) maintain the Lipschitz constant across the group orbit, (2) yield smoother gradients, and (3) suppress gradients in the off-orbit directions. Experiments show that group-equivariant convolutions alone can improve adversarial robustness on CIFAR-10(C) and CIFAR-100.", "questions": "* Inhowfar does the usage of the standard convolution branch in the architectures detailed in Appendix C.3 not undermine most of the theoretical results presented in this paper, as some of the layers used do not fulfill those theoretical guarantees?\n* Would you expect a purely group-equivariant convolutional model to exhibit the same benefits? I am willing to update my score if this is either discussed sufficiently or, ideally, shown experimentally.", "limitations": "Limitations are discussed in Appendix B.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n* Two insightful theorems on invariance of the Lipschitz continuity across the group orbit (Theorem 1) and the suppression of off-orbit perturbations (Theorem 2) in group-equivariant convolutions are proven formally.\n* Experiments on CIFAR-10(C) and CIFAR-100 that incorporate a single group-equivariant layer show that adversarial robustness is significantly improved even without traditional methods such as adversarial training.\n\n**Weaknesses**\n* Experiments are done with a mix of standard and group-equivariant convolutions, and the theoretical findings thus do not directly transfer to the investigated architectures.\n* No results from reference methods, such as standard adversarial robustness, are reported and it is thus difficult to put the reported numbers into context based on this paper alone.\n* The formal proofs assume differentiability at $x$, which is not necessarily a given in neural networks.", "was_revised": true, "final_justification": "The authors present interesting findings on the interplay between equivariance to rotation and scale, and robustness. My initial concern was that their theoretical proofs were not sufficiently supported by experimental results, as these were limited to partially equivariant architectures. However, their rebuttal fully addressed these concerns. Combined with the additional results on larger models (ResNet-50) and datasets (ImageNet-100), I am raising my score to a solid \"Accept\" rating.", "rebuttal": "**We thank this reviewer for the great suggestion!**\n\n---\n\n### **Weakness 1**  \n_Experiments are done with a mix of standard and group-equivariant convolutions, and the theoretical findings thus do not directly transfer to the investigated architectures._\n\n**Answer:**  \nWe appreciate this important observation. Our theoretical analysis is developed under the assumption of purely group-equivariant models, where all layers respect the symmetry constraints imposed by the group action. However, as presented in Appendix C.3, our implemented architectures—particularly the parallel design—intentionally combine standard convolutional layers with group-equivariant branches. \n\nThis hybrid structure was chosen for two key reasons. First, it enables a balance between representational expressiveness and computational efficiency. Second, and more importantly for this work, it provides a controlled framework to isolate and evaluate the impact of symmetry-enhancing components on adversarial robustness. Our central aim is not to enforce full equivariance end-to-end, but **to investigate how the inclusion of symmetry-aware submodules influences a model’s robustness to adversarial perturbations**. While the hybrid architecture does not fully meet the assumptions of our theoretical framework, the equivariant branches themselves do. Their behavior aligns closely with the theoretical predictions derived under group-equivariant assumptions. This is empirically supported by our ablation studies in Appendix D.2, where models containing only rotation-equivariant or scale-equivariant branches consistently outperform standard CNN baselines in adversarial robustness, even in the absence of adversarial training. Thus, the hybrid architecture serves as a practical testbed for validating the theoretical insights and allows us to draw meaningful conclusions about the role of equivariance in improving model robustness.\n\n---\n\n### **Weakness 2**  \n_No results from reference methods, such as standard adversarial robustness, are reported._\n\n**Answer:**  \nThis is a fair and valuable critique. Our goal was to investigate the intrinsic robustness of symmetry-enforced architectures without adversarial training or data augmentation, in order to isolate the architectural contribution. The table below compares the adversarial robustness of a Standard CNN with adversarial training against a Fully Equivariant G-CNN without adversarial training on CIFAR-10. Remarkably, the G-CNN achieves comparable or even superior performance under certain perturbation levels, highlighting the intrinsic robustness conferred by equivariance alone.\n\n### Adversarial Robustness Comparison on CIFAR-10\n\n| Epsilon (\\(\\ell_\\infty\\)) | FGSM Accuracy (%) - Standard CNN | PGD Accuracy (%) - Standard CNN | FGSM Accuracy (%) - G-CNN | PGD Accuracy (%) - G-CNN |\n|---------------------------|----------------------------------|-------------------------------|----------------------------|--------------------------|\n| 0.01                      | 74.5                             | 67.0                          | 73.01                      | 64.96                    |\n| 0.02                      | 70.2                             | 60.4                          | 70.16                      | 58.87                    |\n| 0.03                      | 66.1                             | 54.0                          | 67.09                      | 52.37                    |\n| 0.04                      | 61.7                             | 48.3                          | 63.77                      | 45.52                    |\n| 0.05                      | 57.3                             | 42.1                          | 60.23                      | 37.80                    |\n\n---\n\n### **Weakness 3**  \n_The formal proofs assume differentiability at \\(x\\), which is not necessarily a given in neural networks._\n\n**Answer:**  \nWe fully agree and appreciate the reviewer’s technical rigor. The theoretical results rely on local Lipschitz continuity and Jacobian-based reasoning, which strictly hold almost everywhere for ReLU networks due to their piecewise linearity. While differentiability may not hold at every point, prior works (e.g., [Weng et al., 2018], [Anselmi et al., 2019]) similarly adopt this assumption as a tractable and widely accepted approximation. We have clarified this point in the revised text, explicitly noting that our results apply under the standard assumption of almost-everywhere differentiability.\n\nWeng, Tsui Wei, Cho Jui Hsieh, and Luca Daniel. Evaluating the robustness of neural networks: An extreme value theory approach. In ICLR 2018\n\nFabio Anselmi and Tomaso Poggio. Symmetry-adapted representation learning. Pattern Recognition 2019\n\n---\n\n### **Question 1**  \n_In how far does the usage of the standard convolution branch undermine most of the theoretical results...?_\n\n**Answer:**  \nThank you for raising this important and nuanced concern. In our parallel architecture, the standard convolutional branch operates independently and is fused with the outputs of the equivariant branches only at later stages of the network. As such, it does not directly contribute to the theoretical guarantees derived under the assumption of full group equivariance.\n\nHowever, the primary role of the standard branch is to complement the equivariant representations with additional expressive capacity rather than to interfere with or override the symmetry-induced regularization effects. To better understand the extent to which the inclusion of the standard branch affects the theoretical insights, we have conducted additional experiments (see response to the next question), where we compare models with and without the standard branch. These results help quantify the influence of each branch and assess how the theoretical benefits attributed to equivariance manifest in hybrid settings.\n\n---\n\n### **Question 2**  \n_Would you expect a purely group-equivariant model to exhibit the same benefits? I am willing to update my score if this is either discussed sufficiently or, ideally, shown experimentally._\n\n**Answer:**  \nYes—based on our theoretical framework, we indeed expect a fully group-equivariant model to offer even stronger robustness guarantees compared to hybrid architectures. To validate this expectation empirically, we have conducted additional experiments using both 4-layer and 10-layer models composed entirely of group-equivariant layers. These models were trained on CIFAR-10, CIFAR-100 to assess their performance across datasets of increasing complexity.\n\nThe experimental results (included below) demonstrate that purely group-equivariant models consistently achieve superior robustness under both FGSM and PGD attacks, outperforming the hybrid models presented in the main paper. These findings support the theoretical claim that enforcing full equivariance throughout the architecture enhances the model's ability to resist adversarial perturbations. We have added these results to the revised manuscript to further strengthen the empirical foundation of our analysis.\n\n---\n\n### Adversarial Robustness of **4-layer Fully Equivariant Network** on CIFAR-10\n\n| Epsilon | FGSM Accuracy (%) | PGD Accuracy (%) |\n|---------|-------------------|------------------|\n| 0.01    | 65.65             | 52.20            |\n| 0.02    | 58.54             | 32.04            |\n| 0.03    | 53.78             | 23.30            |\n| 0.04    | 49.92             | 18.63            |\n| 0.05    | 47.08             | 15.85            |\n| 0.06    | 44.72             | 13.03            |\n| 0.07    | 42.89             | 11.05            |\n| 0.08    | 41.09             | 9.50             |\n| 0.09    | 39.51             | 8.17             |\n| 0.10    | 37.95             | 7.01             |\n\n---\n\n### Adversarial Robustness of **4-layer Fully Equivariant Network** on CIFAR-100\n\n| Epsilon | FGSM Accuracy (%) | PGD Accuracy (%) |\n|---------|-------------------|------------------|\n| 0.01    | 38.40             | 21.59            |\n| 0.02    | 30.71             | 12.09            |\n| 0.03    | 26.63             | 8.92             |\n| 0.04    | 24.05             | 6.95             |\n| 0.05    | 22.06             | 5.66             |\n| 0.06    | 20.58             | 4.53             |\n| 0.07    | 19.39             | 3.79             |\n| 0.08    | 18.03             | 3.13             |\n| 0.09    | 16.99             | 2.75             |\n| 0.10    | 15.96             | 2.39             |\n\n---\n\n### Adversarial Robustness of **10-layer Fully Equivariant Network** on CIFAR-10\n\n| Epsilon | FGSM Accuracy (%) | PGD Accuracy (%) |\n|---------|-------------------|------------------|\n| 0.01    | 73.01             | 64.96            |\n| 0.02    | 70.16             | 58.87            |\n| 0.03    | 67.09             | 52.37            |\n| 0.04    | 63.77             | 45.52            |\n| 0.05    | 60.23             | 37.80            |\n| 0.06    | 56.89             | 30.78            |\n| 0.07    | 53.46             | 24.18            |\n| 0.08    | 50.44             | 19.18            |\n| 0.09    | 47.57             | 15.15            |\n| 0.10    | 44.93             | 12.46            |\n\n---\n\n### Adversarial Robustness of **10-layer Fully Equivariant Network** on CIFAR-100\n\n| Epsilon | FGSM Accuracy (%) | PGD Accuracy (%) |\n|---------|-------------------|------------------|\n| 0.01    | 50.60             | 36.29            |\n| 0.02    | 45.42             | 27.98            |\n| 0.03    | 42.02             | 21.34            |\n| 0.04    | 38.83             | 16.02            |\n| 0.05    | 36.09             | 12.01            |\n| 0.06    | 33.33             | 9.29             |\n| 0.07    | 30.94             | 7.31             |\n| 0.08    | 28.52             | 5.87             |\n| 0.09    | 26.41             | 4.73             |\n| 0.10    | 24.68             | 4.08             |"}, {"review_id": "jsi1qLOZDy", "reviewer": "Reviewer_EmhH", "summary": "The paper looks at the role of symmetry aware network architecture for robustness to adversarial attacks. In particular, group-equivariant convolutions (rotation and scale equivariant layers). It proposes two symmetry-aware architectures: a parallel model, processing standard and equivariant features separately before fusion, and a cascaded model, applying equivariant layers sequentially. Theoretically, it provides analyses showing reduced hypothesis space complexity and improved robustness bounds. Empirical evaluations on three datasets demonstrate improvements in robustness under FGSM and PGD attacks without relying on adversarial training.", "questions": "How much computational overhead do equivariant layers add compared to standard CNN layers?\n\n\nHow robustness guarantees scale with network depth or complexity? Deep architectures may have different performance from shallow models, would this be true for robustness as well.", "limitations": "Evaluation is limited to standard adversarial attacks, potentially missing broader adversarial settings.\n\nChosen datasets are relatively simple, unclear how performance would generalize to complex scenarios.\nEquivariant architectures would result in additional computational cost which is not reported.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths\n\nThe architecture used group-equivariant convolutions to incorporate symmetry priors into CNNs. It provides theoretical analysis, demonstrating that equivariant architectures regularize gradient behavior and lead to tighter certified robustness bounds.\n\nEmpirical results show improved robustness and generalization across several standard benchmarks (CIFAR-10, CIFAR-100, CIFAR-10C).\n\nThe parallel architecture outperforms other designs providing useful insights in choosing architecture for symmetry.\n\nOverall well written. No major typos or errors in the key theoretical equations, lemmas, or proofs.\n\n## Weaknesses\n\nThe paper’s entire theoretical robustness framework relies on local Lipschitz bounds. While reasonable this limits theoretical analysis to local neighborhood robustness. It does not guarantee robustness against global adversarial perturbations or more general forms of distribution shifts, limiting the practical scope.\n\nModels are tested only on relatively small datasets. It is unclear whether the approach is scalable to larger, more complex datasets (e.g., ImageNet).\n\nThere is no discussion on computational cost introduced by equivariant transformations.\n\nFigure 1 and 2 do not include error bars. Since datasets are small it would be better to train with multiple seeds. Also, it is important for reproducibility and reliability.\n\nThe motivation behind the chosen architectures in 6.1 is arbitrary. Not clear what are the pros and cons and how one would select the suitable architecture.\n\nTheoretical analysis is for discrete and finite group transformations, would results extend to continuous/Lie group transformations which are common in computer vision tasks.", "was_revised": true, "final_justification": "To my comments authors have included additional results. Also, promised to include error bars in final version. I have therefore increased my score.", "rebuttal": "**We thank this reviewer for the thoughtful and constructive feedback.**\n\n**Weakness 1:** The paper’s entire theoretical robustness framework relies on local Lipschitz.\n\n**Answer:**  \nWe agree with the reviewer that our theoretical analysis is focused on local robustness. While this is a common and well-accepted approach for formal robustness analysis, we acknowledge that it does not extend to global robustness or general distribution shifts.  \nOur primary goal in this work is **to establish a precise and tractable theoretical connection between group equivariance and local certified robustness—specifically, how symmetry-enforcing architectures influence the model's local Lipschitz properties and decision margin sensitivity.**  \nThis framework serves as a mathematically grounded tool that complements empirical evaluation, offering insights into the role of architectural inductive bias in enhancing robustness. Extending our theoretical framework to global robustness, distributional robustness, or manifold-aware perturbations is an important and promising direction for future work.\n\n---\n\n**Weakness 2:** Models are tested only on relatively small datasets.\n\n**Answer:**  \nWe initially selected CIFAR-10, CIFAR-100, and CIFAR-10C as our primary benchmarks to facilitate controlled evaluation across architectural variants, attack settings, and theoretical alignment, **all within a feasible computational budget**.  \nTo that end, **we have conducted additional experiments on the ImageNet-100 subset, a 100-class subset of ImageNet commonly used for mid-scale evaluation**. Our results demonstrate that group-equivariant architectures continue to yield consistent improvements in adversarial robustness on this more challenging dataset, while maintaining manageable computational overhead.\n\n### Adversarial Robustness on **ImageNet-100** – 4-layer Equivariant Model\n\n| Epsilon | FGSM Accuracy (%) | PGD Accuracy (%) |\n|---------|-------------------|------------------|\n| 0.01    | 25.36             | 14.52            |\n| 0.02    | 19.94             | 13.58            |\n| 0.03    | 18.36             | 12.24            |\n| 0.04    | 17.56             | 10.74            |\n| 0.05    | 17.16             | 9.12             |\n| 0.06    | 16.84             | 7.96             |\n| 0.07    | 16.62             | 6.96             |\n| 0.08    | 16.48             | 6.22             |\n| 0.09    | 16.32             | 5.44             |\n| 0.10    | 16.22             | 4.90             |\n| 0.30    | 11.00             | 3.30             |\n\n---\n\n### Adversarial Robustness on **ImageNet-100** – 10-layer Equivariant Model\n\n| Epsilon | FGSM Accuracy (%) | PGD Accuracy (%) |\n|---------|-------------------|------------------|\n| 0.01    | 37.62             | 17.92            |\n| 0.02    | 25.84             | 12.25            |\n| 0.03    | 25.41             | 8.95             |\n| 0.04    | 25.18             | 5.96             |\n| 0.05    | 25.00             | 5.48             |\n| 0.06    | 24.84             | 4.22             |\n| 0.07    | 24.76             | 2.98             |\n| 0.08    | 24.70             | 2.84             |\n| 0.09    | 22.34             | 2.72             |\n| 0.10    | 21.26             | 2.64             |\n\n---\n\n**Weakness 3:** There is no discussion on computational cost.\n\n**Answer:**  \nThank you for pointing this out. Please see our response in question 1.\n\n---\n**Weakness 4:** Figures 1 and 2 do not include error bars.\n\n**Answer:**  \nWe appreciate this valuable suggestion. we have included error bars in the draft.\nDue to limited space, the data is not included here.\n\n---\n\n**Weakness 5:** The motivation behind the chosen architectures.\n\n**Answer:**  \nWe appreciate the reviewer’s concern and agree that a clearer explanation of the architectural design choices is warranted.  \nTo summarize:\n\n- The *Parallel G-CNN* offers modular symmetry enforcement by maintaining independent branches (e.g., standard and rotation-equivariant), which enhances robustness but increases model size and computational cost.  \n- The *Parallel G-CNN with Rotation- and Scale-Equivariant Branches* extends this by incorporating additional symmetry priors and enabling more diverse feature extraction.  \n- The *Cascaded G-CNN* provides a more compact design by stacking symmetry-aware layers sequentially, although this may introduce an information bottleneck if the early equivariant layer restricts feature expressiveness.  \n- The *Weighted Parallel G-CNN* replaces hard feature concatenation with learnable fusion weights, offering a flexible mechanism to dynamically adapt to task-specific signal strength from each branch.  \n- The *Standard CNN* serves as the baseline for comparison.\n\n---\n\n**Weakness 6:** Theoretical analysis is for discrete and finite group transformations, would results extend to continuous/Lie group transformations which are common in computer vision tasks.\n\n**Answer:**  \nThis is an excellent observation. Our current theoretical framework is developed for finite discrete groups (e.g., **P4** for discrete rotations), which simplifies the analysis and aligns with widely adopted G-CNN implementations in practice.  \nThe extension to continuous group requires generalizing the group action. For discrete groups, actions are defined over a finite set of transformations. In contrast, Lie groups possess a smooth manifold structure, and their actions on the input space are described by differentiable maps. To analyze equivariant robustness in this setting, we consider group elements in a neighborhood of the identity, represented as exponentials of Lie algebra elements. This provides a foundation for defining infinitesimal transformations—the continuous analogs of discrete group elements—and enables us to formulate sensitivity metrics along smooth group orbits.\nExtending orbit-averaged Jacobian smoothing techniques to Lie groups requires the use of **Haar measures**—the invariant integration measure on the group. For compact groups such as $\\( \\mathrm{SO}(2) \\)$, this integration is well-defined and finite. For non-compact groups like $\\( \\mathrm{SE}(2) \\) $ or $\\( \\mathbb{R}^2 \\)$, we must either impose bounded supports or consider localized versions of the group action.\n\n\n---\n\n**Question 1:** How much computational overhead?\n\n**Answer:**  \nIt is worth noting that in our parallel architecture, equivariant branches are introduced only at the first layer of the network. This design ensures that the computational overhead introduced by group-equivariant operations does not scale with network depth, allowing us to retain the benefits of symmetry enforcement while maintaining computational efficiency across deeper models.\n\nEmpirically, we find that rotation-equivariant convolutions based on the **P4** group introduce approximately **1.5×** more FLOPs than standard convolutional layers with equivalent dimensions, primarily due to additional orientation channels.  \nScale-equivariant branches, which process multiple rescaled versions of the input, incur modest additional runtime and memory overhead—typically around **1.7×**—owing to interpolation and multi-scale channel stacking.\n\nIn the full parallel G-CNN configuration, which includes standard, rotation-, and scale-equivariant branches, the total computational cost scales roughly linearly with the number of branches. However, because each branch can be made shallow or low-dimensional, our design supports flexible, budget-aware trade-offs between robustness, representation capacity, and efficiency.\n\n---\n\n**Question 2:** How do robustness guarantees scale with network depth or complexity?\n\n**Answer:**  \nThis is an excellent and important question. From a theoretical perspective, the gradient regularization induced by equivariance propagates through the layers of a network, preserving certain structural properties of the Jacobian and contributing to robustness.  \nHowever, in deeper architectures, the influence of symmetry constraints may become diluted, particularly if a large portion of the network consists of standard layers.\n\nAlthough full-scale experiments on very deep models were constrained by available computational resources, **we conducted additional evaluations on ResNet-18 and ResNet-50 architectures to explore scalability**. These experiments indicate that while the absolute robustness tends to decrease with depth—likely due to increased model complexity and overfitting—the equivariant models (G-CNN variants) continue to outperform their standard CNN counterparts.  \nThis confirms that the robustness advantage of equivariant design remains valid even as model depth increases, though the relative gain may be reduced.\n\n### **ResNet50_GCNN** on CIFAR-100\n\n| Epsilon | FGSM Accuracy (%) | PGD Accuracy (%) |\n|---------|-------------------|------------------|\n| 0.01    | 32.09             | 11.58            |\n| 0.02    | 27.18             | 2.04             |\n| 0.03    | 23.21             | 0.47             |\n| 0.04    | 19.14             | 0.14             |\n| 0.05    | 15.15             | 0.03             |\n| 0.06    | 11.75             | 0.01             |\n| 0.07    | 8.81              | 0.01             |\n| 0.08    | 6.31              | 0.00             |\n| 0.09    | 4.58              | 0.00             |\n| 0.10    | 3.54              | 0.00             |\n\n---\n\n### **ResNet18_GCNN** on CIFAR-100\n\n| Epsilon | FGSM Accuracy (%) | PGD Accuracy (%) |\n|---------|-------------------|------------------|\n| 0.01    | 37.34             | 16.85            |\n| 0.02    | 31.19             | 2.73             |\n| 0.03    | 26.55             | 0.34             |\n| 0.04    | 22.72             | 0.05             |\n| 0.05    | 19.70             | 0.00             |\n| 0.06    | 16.98             | 0.00             |\n| 0.07    | 14.94             | 0.00             |\n| 0.08    | 13.43             | 0.00             |\n| 0.09    | 11.89             | 0.00             |\n| 0.10    | 10.33             | 0.00             |"}, {"review_id": "ENBuH6SvFH", "reviewer": "Reviewer_b95s", "summary": "This paper investigates how incorporating group-equivariant  convolutions (rotation and scale) into CNNs can improve adversarial robustness without adversarial training. The authors provide a theoretical analysis showing that equivariant architectures provide stronger robustness guarantees through the CLEVER framework. The evaluation is carried out with diverse CNN architectures integrating equivariant layers. Experiments on CIFAR-10/100/10C demonstrate an increase in adversarial robustness against FGSM and PGD attacks.", "questions": "- Could you clarify the theoretical treatment of scale equivariance?\n- Can you provide insight on how your approach would perform if integrated into equivariant network architectures from the literature?\n- What is the computational overhead of group convolutions compared to standard convolutions?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths\n- The paper is well-structured with clear sections\n- Provides rigorous theoretical analysis with formal proofs linking equivariance to adversarial robustness through CLEVER bounds\n- The work addresses an interesting question from a novel architectural perspective\n- Code provided is clear and complete\n\n\n## Weaknesses\n- The mathematical treatment lacks clarity for scale equivariance. Some formulations are unclear to me and appear to work only for rotations and not scaling (e.g., line 192, I think a scaling term should be present).\n- The evaluation is limited to small datasets (CIFAR variants), which raises questions generalization to larger, more complex datasets.\n- The related work cites equivariant networks (G-CNNs, Harmonic Networks, Steerable CNNs), but the authors develop their own simplified models instead of using these existing architectures.\n- No direct comparison with adversarial training is provided, making it difficult to assess the relative benefits of the architectural approach.\n\ntypo : line 273:  C4 instead of P4", "was_revised": true, "final_justification": "After the authors' rebuttal and discussions, I am updating my recommendation to Accept.\n\nIssues Resolved:\n\n- Mathematical clarity on scale equivariance: One of my primary concerns was the lack of clarity in the mathematical treatment of scale equivariance. The authors provided a comprehensive response addressing this issue and committed to adding clearer explanations and formal definitions in the revised manuscript.\n- Experimental validation: The authors conducted additional experiments with larger datasets as requested. The new comparison with adversarial training methods provides valuable information on the method's effectiveness.\n- Architectural considerations: The productive discussion with Reviewer 1 regarding architectural choices led to new experiments that convincingly increase the overall quality of the paper.\n\nWeight Assignment:\nI assign high weight to the resolution of the mathematical clarity issue (50%) and the strengthened experimental validation (30%). The architectural robustness demonstration carries moderate weight (20%).", "rebuttal": "**We thank this reviewer for the thoughtful and constructive feedback.**\n\n---\n\n### **Weakness 1:**  \n_The mathematical treatment lacks clarity for scale equivariance. Some formulations are unclear to me and appear to work only for rotations and not scaling (e.g., line 192, I think a scaling term should be present)._\n\n**Answer:**  \nWe appreciate this insightful question.\n\nWhile the theoretical analysis in Section 4 effectively establishes robustness guarantees for group-equivariant architectures under the assumption of norm-preserving transformations, such as rotations with orthogonal Jacobians, the extension to scale-equivariant models requires additional consideration.\n\nScale transformations, unlike rotations, do not preserve norms. Specifically, a scaling transformation $x \\mapsto \\alpha x$ has a Jacobian $D_\\alpha = \\alpha I$, meaning that the gradient norm transforms as $\\|D_\\alpha v\\| = \\alpha \\|v\\|$. This directly violates the assumption used in Lemma 1 and Theorem 1, which relies on the orthogonality of both the group representation $\\rho(g)$ and the transformation Jacobian $D_{g^{-1}}$.\n\nAs a result, the CLEVER-certified robustness bounds derived under the assumption of norm invariance do not apply directly to scale-equivariant networks. Instead, these bounds must be rescaled to account for the effect of dilation on the gradient magnitude. In particular, the Lipschitz constant of the margin function becomes scale-dependent, and its growth must be explicitly quantified.\n\nDespite this, scale-equivariant networks still promote adversarial robustness through a different mechanism: **orbit-averaged gradient smoothing**. By aggregating gradients across scaled versions of the input, scale-equivariant architectures reduce high-frequency fluctuations and suppress gradient sensitivity to perturbations that deviate from the scale-induced orbit. This aggregation process, while not norm-preserving, effectively stabilizes model behavior under scale transformations and yields smoother decision boundaries.\n\nTo formalize this, we can define the **orbit-averaged gradient field** over a discrete scale group $G_s = \\{\\alpha_1, \\ldots, \\alpha_k\\} \\subset \\mathbb{R}^+$ as $\\bar{\\phi}_j(x) = \\frac{1}{|G_s|} \\sum_{\\alpha \\in G_s} \\nabla f_j(\\alpha x),$\nwhere each term captures the gradient at a different scale-transformed version of the input. Although these gradients vary in norm due to scaling, the averaging process reduces local gradient variance, contributing to robustness. This is particularly effective when combined with architectural fusion mechanisms, such as channel-wise concatenation or weighted summation of multi-scale features.\n\nWhile scale-equivariant models do not satisfy the same Jacobian norm invariance as rotation-equivariant models, they still enhance robustness by regularizing the gradient landscape across scales. \n\n\n---\n\n### **Weakness 2:**  \n_The evaluation is limited to small datasets (CIFAR variants)_\n\n**Answer:**  \nWe initially selected CIFAR-10, CIFAR-100, and CIFAR-10C as our primary benchmarks to facilitate controlled evaluation across architectural variants, attack settings, and theoretical alignment, **all within a feasible computational budget**.  \nTo that end, **we have conducted additional experiments on the ImageNet-100 subset, a 100-class subset of ImageNet commonly used for mid-scale evaluation**. Our results demonstrate that group-equivariant architectures continue to yield consistent improvements in adversarial robustness on this more challenging dataset, while maintaining manageable computational overhead.\n\n### Adversarial Robustness on **ImageNet-100** – 4-layer Equivariant Model\n\n| Epsilon | FGSM Accuracy (%) | PGD Accuracy (%) |\n|---------|-------------------|------------------|\n| 0.01    | 25.36             | 14.52            |\n| 0.02    | 19.94             | 13.58            |\n| 0.03    | 18.36             | 12.24            |\n| 0.04    | 17.56             | 10.74            |\n| 0.05    | 17.16             | 9.12             |\n| 0.06    | 16.84             | 7.96             |\n| 0.07    | 16.62             | 6.96             |\n| 0.08    | 16.48             | 6.22             |\n| 0.09    | 16.32             | 5.44             |\n| 0.10    | 16.22             | 4.90             |\n| 0.30    | 11.00             | 3.30             |\n\n---\n\n### Adversarial Robustness on **ImageNet-100** – 10-layer Equivariant Model\n\n| Epsilon | FGSM Accuracy (%) | PGD Accuracy (%) |\n|---------|-------------------|------------------|\n| 0.01    | 37.62             | 17.92            |\n| 0.02    | 25.84             | 12.25            |\n| 0.03    | 25.41             | 8.95             |\n| 0.04    | 25.18             | 5.96             |\n| 0.05    | 25.00             | 5.48             |\n| 0.06    | 24.84             | 4.22             |\n| 0.07    | 24.76             | 2.98             |\n| 0.08    | 24.70             | 2.84             |\n| 0.09    | 22.34             | 2.72             |\n| 0.10    | 21.26             | 2.64             |\n\n---\n\n\n---\n\n### **Weakness 3:**  \n_the authors develop their own simplified models instead of using these existing architectures._\n\n**Answer:**  \nThis is a valuable observation. While our architectural design draws inspiration from established equivariant models such as G-CNNs , Harmonic Networks, and Steerable CNNs, we intentionally adopt a modular and interpretable architecture to facilitate controlled experimentation and analysis. Specifically, our design allows for the systematic ablation of individual symmetry-enforcing components—e.g., rotation-only, scale-only, and different fusion strategies (parallel vs. cascaded)—which is more difficult to achieve in tightly integrated models like Steerable or Harmonic CNNs.This level of interpretability and control is a key motivation behind our simplified implementation.\n\n\n---\n\n### **Weakness 4:**  \n_No direct comparison with adversarial training is provided._\n\n**Answer:**  \nThis is a fair and valuable critique. The table below compares the adversarial robustness of a Standard CNN with adversarial training against a Fully Equivariant G-CNN without adversarial training on CIFAR-10. Remarkably, the G-CNN achieves comparable or even superior performance under certain perturbation levels, highlighting the intrinsic robustness conferred by equivariance alone.\n\n### \n\n| Epsilon | FGSM  (%) - Standard CNN | PGD  (%) - Standard CNN | FGSM  (%) - G-CNN | PGD  (%) - G-CNN |\n|---------------------------|----------------------------------|-------------------------------|----------------------------|--------------------------|\n| 0.01                      | 74.5                             | 67.0                          | 73.01                      | 64.96                    |\n| 0.02                      | 70.2                             | 60.4                          | 70.16                      | 58.87                    |\n| 0.03                      | 66.1                             | 54.0                          | 67.09                      | 52.37                    |\n| 0.04                      | 61.7                             | 48.3                          | 63.77                      | 45.52                    |\n\n---\n\n### **Question 1:**  \n_Could you clarify the theoretical treatment of scale equivariance?_\n\n**Answer:**  \nPlease see the response in  **Weakness 1** \n\n\n---\n\n### **Question 2:**  \n_Can you provide insight on how your approach would perform if integrated into equivariant network architectures from the literature?_\n\n**Answer:**  \nOur modular framework—particularly the parallel and cascaded fusion designs—is architecture-agnostic and readily extensible to more expressive equivariant models. In principle, the rotation-equivariant branch used in our current implementation could be replaced with more advanced architectures such as Harmonic Networks, Steerable CNNs, or LieConv, which provide continuous and steerable representations under group transformations.\n\nIntegrating such models may further enhance the robustness and representation capacity of our approach, especially in tasks requiring fine-grained geometric invariance. However, these architectures typically come with increased computational overhead and implementation complexity, which may limit their practicality in some settings.\n\nWe will experiment with steerable convolutional variants within our framework. These extensions, along with scalability and implementation considerations, are discussed in the revised manuscript’s Future Work section.\n\n---\n\n### **Question 3:**  \n_What is the computational overhead of group convolutions compared to standard convolutions?_\n\n**Answer:**  \nIt is worth noting that in our parallel architecture, equivariant branches are introduced only at the first layer of the network. This design ensures that the computational overhead introduced by group-equivariant operations does not scale with network depth, allowing us to retain the benefits of symmetry enforcement while maintaining computational efficiency across deeper models.\n\nEmpirically, we find that rotation-equivariant convolutions based on the $\\mathrm{P4}$ group introduce approximately **1.5×** more FLOPs than standard convolutional layers with equivalent dimensions, primarily due to additional orientation channels. Scale-equivariant branches, which process multiple rescaled versions of the input, incur modest additional runtime and memory overhead—typically around **1.7×**—owing to interpolation and multi-scale channel stacking.\n\nIn the full parallel G-CNN configuration, which includes standard, rotation-, and scale-equivariant branches, the total computational cost scales roughly linearly with the number of branches. However, because each branch can be made shallow or low-dimensional, our design supports flexible, budget-aware trade-offs between robustness, representation capacity, and efficiency.\n\nWe will provide detailed runtime and FLOP analysis in the Appendix."}]

Author rebuttal:

Meta-review: The authors investigate how group-equivariant convolutions (rotation and scale equivariant layers) can enhance adversarial robustness without adversarial training. They present three key theoretical findings: equivariant convolutions (1) preserve the Lipschitz constant across group orbits, (2) produce smoother gradients, and (3) suppress off-orbit gradients. Theoretical analysis with formal proofs is impressive. All reviewers agree to accept this paper.
Decision: Accept (spotlight)