Title: Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustness
Venue: NeurIPS 2025
Accepted: yes
Keywords: Adversarial Robustness

Symmetry

Equivariance
Abstract: Adversarial examples reveal critical vulnerabilities in deep neural networks by exploiting their sensitivity to imperceptible input perturbations. While adversarial training remains the predominant defense strategy, it often incurs significant computational cost and may compromise clean-data accuracy. In this work, we investigate an architectural approach to adversarial robustness by embedding group-equivariant convolutions—specifically, rotation- and scale-equivariant layers—into standard convolutional neural networks (CNNs). These layers encode symmetry priors that align model behavior with structured transformations in the input space, promoting smoother decision boundaries and greater resilience to adversarial attacks. We propose and evaluate two symmetry-aware architectures: a parallel design that processes standard and equivariant features independently before fusion, and a cascaded design that applies equivariant operations sequentially. Theoretically, we demonstrate that such models reduce hypothesis space complexity, regularize gradients, and yield tighter certified robustness bounds under the CLEVER (Cross Lipschitz Extreme Value for nEtwork Robustness) framework. Empirically, our models consistently improve adversarial robustness and generalization across CIFAR-10, CIFAR-100, and CIFAR-10C under both FGSM and PGD attacks, without requiring adversarial training. These findings underscore the potential of symmetry-enforcing architectures as efficient and principled alternatives to data augmentation-based defenses.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates how incorporating group-equivariant  convolutions (rotation and scale) into CNNs can improve adversarial robustness without adversarial training. The authors provide a theoretical analysis showing that equivariant architectures provide stronger robustness guarantees through the CLEVER framework. The evaluation is carried out with diverse CNN architectures integrating equivariant layers. Experiments on CIFAR-10/100/10C demonstrate an increase in adversarial robustness against FGSM and PGD attacks.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- The paper is well-structured with clear sections\n- Provides rigorous theoretical analysis with formal proofs linking equivariance to adversarial robustness through CLEVER bounds\n- The work addresses an interesting question from a novel architectural perspective\n- Code provided is clear and complete\n\n\n## Weaknesses\n- The mathematical treatment lacks clarity for scale equivariance. Some formulations are unclear to me and appear to work only for rotations and not scaling (e.g., line 192, I think a scaling term should be present).\n- The evaluation is limited to small datasets (CIFAR variants), which raises questions generalization to larger, more complex datasets.\n- The related work cites equivariant networks (G-CNNs, Harmonic Networks, Steerable CNNs), but the authors develop their own simplified models instead of using these existing architectures.\n- No direct comparison with adversarial training is provided, making it difficult to assess the relative benefits of the architectural approach.\n\ntypo : line 273:  C4 instead of P4\n\nQUESTIONS:\n- Could you clarify the theoretical treatment of scale equivariance?\n- Can you provide insight on how your approach would perform if integrated into equivariant network architectures from the literature?\n- What is the computational overhead of group convolutions compared to standard convolutions?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper looks at the role of symmetry aware network architecture for robustness to adversarial attacks. In particular, group-equivariant convolutions (rotation and scale equivariant layers). It proposes two symmetry-aware architectures: a parallel model, processing standard and equivariant features separately before fusion, and a cascaded model, applying equivariant layers sequentially. Theoretically, it provides analyses showing reduced hypothesis space complexity and improved robustness bounds. Empirical evaluations on three datasets demonstrate improvements in robustness under FGSM and PGD attacks without relying on adversarial training.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\nThe architecture used group-equivariant convolutions to incorporate symmetry priors into CNNs. It provides theoretical analysis, demonstrating that equivariant architectures regularize gradient behavior and lead to tighter certified robustness bounds.\n\nEmpirical results show improved robustness and generalization across several standard benchmarks (CIFAR-10, CIFAR-100, CIFAR-10C).\n\nThe parallel architecture outperforms other designs providing useful insights in choosing architecture for symmetry.\n\nOverall well written. No major typos or errors in the key theoretical equations, lemmas, or proofs.\n\n## Weaknesses\n\nThe paper’s entire theoretical robustness framework relies on local Lipschitz bounds. While reasonable this limits theoretical analysis to local neighborhood robustness. It does not guarantee robustness against global adversarial perturbations or more general forms of distribution shifts, limiting the practical scope.\n\nModels are tested only on relatively small datasets. It is unclear whether the approach is scalable to larger, more complex datasets (e.g., ImageNet).\n\nThere is no discussion on computational cost introduced by equivariant transformations.\n\nFigure 1 and 2 do not include error bars. Since datasets are small it would be better to train with multiple seeds. Also, it is important for reproducibility and reliability.\n\nThe motivation behind the chosen architectures in 6.1 is arbitrary. Not clear what are the pros and cons and how one would select the suitable architecture.\n\nTheoretical analysis is for discrete and finite group transformations, would results extend to continuous/Lie group transformations which are common in computer vision tasks.\n\nQUESTIONS:\nHow much computational overhead do equivariant layers add compared to standard CNN layers?\n\n\nHow robustness guarantees scale with network depth or complexity? Deep architectures may have different performance from shallow models, would this be true for robustness as well.\n\nLIMITATIONS:\nEvaluation is limited to standard adversarial attacks, potentially missing broader adversarial settings.\n\nChosen datasets are relatively simple, unclear how performance would generalize to complex scenarios.\nEquivariant architectures would result in additional computational cost which is not reported.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present three key theoretical findings regarding the correspondence of equivariance and adversarial robustness: group-equivariant convolutions (1) maintain the Lipschitz constant across the group orbit, (2) yield smoother gradients, and (3) suppress gradients in the off-orbit directions. Experiments show that group-equivariant convolutions alone can improve adversarial robustness on CIFAR-10(C) and CIFAR-100.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n* Two insightful theorems on invariance of the Lipschitz continuity across the group orbit (Theorem 1) and the suppression of off-orbit perturbations (Theorem 2) in group-equivariant convolutions are proven formally.\n* Experiments on CIFAR-10(C) and CIFAR-100 that incorporate a single group-equivariant layer show that adversarial robustness is significantly improved even without traditional methods such as adversarial training.\n\n**Weaknesses**\n* Experiments are done with a mix of standard and group-equivariant convolutions, and the theoretical findings thus do not directly transfer to the investigated architectures.\n* No results from reference methods, such as standard adversarial robustness, are reported and it is thus difficult to put the reported numbers into context based on this paper alone.\n* The formal proofs assume differentiability at $x$, which is not necessarily a given in neural networks.\n\nQUESTIONS:\n* Inhowfar does the usage of the standard convolution branch in the architectures detailed in Appendix C.3 not undermine most of the theoretical results presented in this paper, as some of the layers used do not fulfill those theoretical guarantees?\n* Would you expect a purely group-equivariant convolutional model to exhibit the same benefits? I am willing to update my score if this is either discussed sufficiently or, ideally, shown experimentally.\n\nLIMITATIONS:\nLimitations are discussed in Appendix B.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}