CERTIFIED VS. EMPIRICAL ADVERSARIAL ROBUSTNESS VIA HYBRID CONVOLUTIONS WITH ATTENTION STOCHASTICITY
Authors: Joy Dhar
Song Xia
Manish Kumar Pandey
Maryam Haghighat
Azadeh Alavi
Ferdous Sohel
Wenyu Zhang
Nayyar Zaidi
Venue: ICLR 2026 Poster (2026)
Keywords: Certified Defense
Empirical Defense
Adversarial Robustness
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: We introduce Hybrid Convolutions with Attention Stochasticity (HyCAS), an adversarial defense that narrows the long-standing gap between provable robustness under ℓ2 certificates and empirical robustness against strong ℓ∞ attacks, while preserving strong generalization across diverse imaging benchmarks. HyCAS unifies deterministic and randomized principles by coupling 1-Lipschitz, spectrally normalized convolutions with two stochastic components—spectral normalized random-projection filters and a randomized attention-noise mechanism—to realize a randomized defense. Injecting smoothing randomness inside the architecture yields an overall ≤ 2-Lipschitz network with formal certificates. Extensive experiments on diverse imaging benchmarks—including CIFAR-10/100, ImageNet-1k, NIH Chest X-ray, HAM10000—show that HyCAS surpasses prior leading certified and empirical defenses, boosting certified accuracy by up to ≈ 7.3% (on NIH Chest X-ray) and empirical robustness by up to ≈ 3.1% (on HAM10000), without sacrificing clean accuracy. These results show that a randomized Lipschitz constrained architecture can simultaneously improve both certified ℓ2 and empirical ℓ∞ adversarial robustness, thereby supporting safer deployment of deep models in high-stakes applications.

Published as a conference paper at ICLR 2026
CERTIFIED VS. EMPIRICAL ADVERSARIAL ROBUST- NESS VIA HYBRID CONVOLUTIONS WITH ATTENTION STOCHASTICITY
Joy Dhar', Song X Manish Kumar Pandey"), Maryam Haghighat', Azadeh Alavi', Ferdous Sohel", Wenyu Zhang', Nayyar Zaidi"
Indian Institute of Technology Ropar "Nanyang Technological University 'RoentGen Health
"Queensland University of Technology 'RMIT University "Murdoch University 'Deakin University
ABSTRACT
We introduce Hybrid Convolutions with Attention Stochasticity (HyCAS), an ad- versarial defense that narrows the long-standing gap between provable robustness under &g certificates and empirical robustness against strong &oo attacks, while preserving strong generalization across diverse imaging benchmarks. HyCAS uni- fies deterministic and randomized principles by coupling 1-Lipschitz, spectrally normalized convolutions with two stochastic components-ppecttee normalized candom-projection filters and a randomized attention-noise mechanism to re- alize a randomized defense. Injecting smoothing randomness inside the archi- tecture yields an overall s 2-Lipschiiz network with formal certificates, Exten- sive experiments on diverse imaging benchmarks cechmaarks-including including CIFAR-10/100, ImageNet-1k, NIH Chest X-ray, HAM1 00000000-000 show that HyCAS sur- passes prior leading certified and empirical defenses, boosting certified accu- 7.3% (on NTH Chest x-ray) and empirical robustness by up to z 3.1% (on HAM1 0000), without sacrificing clean accuracy. These re- sults show that randomized Lipschitz constrained architecture can simultane- ously improve both certified & and empirical &oo adversarial robustness, thereby supporting safer deployment of deep models in high- stakes applications. Code: https:
racy by up
com com./mitt11203/BBCA. HYCSA
INTRODUCTION
Despite their impressive accuracy, deep learning architectures in computer vision remain vulnera- ble adversarial attacks. Such vulnerabilities threaten safety-critical deployments in fraud detec-
tion (Pumsirirat & Liu 2018), autonomous driving (Cao et al. 2021), and clinical decision sup-
port (Dhar et al.j (0255) where mistakes carry high costs. In response to these adversarial vulner-
abilitics, carly research focused on empirical defences, most notably adversarial training (Madry
al. 2018 Dhar al. et al.
2025 Ding al. 2019: Shafahi et 2019 Sriramanan al. 2021 Cheng
2023 Dhar et al. 20266_ However, these methods are frequently broken by intricately crafted
adversarial attacks (Carlini & Wagner 2017: Yuan et al. [222 [eeddrykks et al. [021]] Duan al.,
2021
Li al.) (003). This limitation has fuelled interests in certified robustness techniques, which
offer provable guarantees that a classifier's prediction cannot change within a specified perturbation
radius (Raghunathan et al. 2018 Wong & Kolter 2018 Hao et al. 2022).
Randomized Smoothing (RS) Lécuyer et al. 2019 Cohen et al. 2019) certifies robustness by aver- aging model's predictions over noise-perturbed inputs at inference, and therefore scales to modern deep architectures. Yet RS is inherently rigid: large noise budgets erode clean accuracy, whereas small budgets certify only narrow Eg radii. Recent baselines seek to bypass this trade-off with test- time adaptations-botth generic (Croce et al.] 2022) and RS-specific (Alfarra et al.) 2022b Sūkeník 2022 Hong al. 2022). These defences, however, are mostly heuristic- based and they quickly succumb to stronger, tailored attacks (Croce et al. 2022 Alfarra et al. 2002a| Hong et al.
2022
rekindling the familiar "cat-and-mouse cycle of empirical defences (Athalye et 2018
Equual contribution;" Independent researcher.

---

Published as a conference paper at ICLR 2026
Tramer et al.) 2020). Moreover, they are rarely benchmarked against state-of of-te-of-the-arr ahe empirical at- tacks such as APGD (Croce & Hein, 2020) or AutoAttack (Croce & Hein 2 or on domain- specific distributions, such as medical -mmging datasets, thereby leaving their real-world efficacy uncertain. We move beyond pure test-time fixes and inject fresh, independently drawn noise dur- ing both training and inference. This two-phase strategy (i) preserves RS' s formal guarantees, (ii) alleviates the rocuracy-robustness trade-off, and (ii) is validated against both certified and strong empirical attacks across diverse imaging benchmarkss
To bridge the gap between certified and empirical defenses, we introduce Hybrid Convolu- tions with Attention Stochasticity (HyCAS). HyCAS offers provable Eg adversarial robustness, boosts empirical adversarial resilience to strong lo attacks, and generalizes across eight di- verse vision benchmarks. is a randomized defense whose architecture combines a determinis- tic Lipschitz-constraiie stochastic smoothing modules: (i) spectrally normalized cundom-projection filters and (ii) random- ized attention-noise injection. These components inject controlled smoothing noise, thereby incor- porating stochasticity into the architecture and yielding an overall < 2-Lipschitz network that admits formal certification while consistently enhancing empirical robustness to strong for attacks.
beeignn-mmplmentter
via spectrally normalized convolutions-wiify two
The key contributions of this paper can be summarized as follows:
Hybrid defense. We introduce HyCAS, a randomized Lipschirz-conssraiied defense that pro- vides both certified &2 guarantees and strong empirical Eoo robustness across diverse vision benchmarks.
2. Theoretical guarantees. We derive tight l robustness certificate for HyCAS and show that remains competitive in empirical adversarial robustness against state-of-the-art foo attacks.
3. Plug-and-play design. HYCAS integrates a I-Lipschitz deterministic impe-immlementted via spectrally normalized convolutions-wit with two stochastic modules: spectral normalized random- projection filters and randomized attention noise injection. These components inject controlled smoothing noise, thereby incorporating refined stochasticity into the network. Together they form a < 2-Lipschitz network that admits formal certification while boosting empirical robustness.
4. Comprehensive evaluation. Experiments several benchmarks confirm that HyCAS outper- forms prior certified and empirical defenses while allowing controllable trade-offs between cer- tificate and empirical adversarial robustness.
2 RELATED WORK
Deterministic certified defences. Bounding a network's global Lipschitz constant makes its pre- dictions provably stable to small la perturbations. Early studies constrain fully- colneceddd layers via spectral normalisation or orthogonal parameterisations (Miyato et al.) (018).. Layer-wise Or- thogonal Training (LCT) (Xu et al. 2022) and the Spectral- Lattice (SLL) (Araujo et al.) 2023) extend these ideas to CNNs, yet often sacrifice clean accuracy high-resolution data. Our deterministic backbone inherits its 1-Lipschitz guarantee while compensating for the accuracy drop through stochastic branches.
Stochastic certified defences. Randomised smoothing (RS) converts any base classifier into a Gaus- sian ensemble whose majority vote is certifiably robust (Cohen al. 2019). Subsequent work
enlarges certificates via adversarially trained bases (Salman et al.) 2019). consistency regularisation (eoong & Shin 2h2m) and noise-aware denoising (Carlini et al. (023). Mixing multiple noise scales further tightens guarantees, as shown by Dual R$ (DRS) (Xia et al. 2024), Incremental RS (IRS) (Ugare et al.), and Adaptive RS (ARS) (Lyu et al. 20224). Our HyCAS departs from pure input-noise smoothing by injecting internal randomness via dual stochastic noise-yel still preserves global < 2-Liiscciizz certificate.
Empirical defences. Empirical methods drop certificates to maximise robustness against high- budget foo attacks. PNI (He et al., 2019) learns layer-wise Gaussian noise during adversarial train- ing, boosting both clean and robust accuracy. Learn2Perture
(Jeddi et al.) (020) generalises this idea by jointly optimising ceature-perturbattio modules in an EM-like loop. CTRW (Ma et al. 2023) resamples convolution kernels at inference under learned mean-variance constraints, while RPF (Dong & Xul 2023) freezes part of the first convolution layer as Gaussian projections, both outperforming strong PGD-trained baselines. In contrast, CAP (iiang al. 2023) infuses lung-
In our experiments, we use natural vision and medical imaging datasets diverse imaging benchmarks.
2

---

Published as a conference paper at ICLR 2026
edge priors, bolstering adversarial robustness in COVID-19 CT prediction. Despite these gains, these empirical defences provide no certified worst case guarantees (Yang et al. 2022 Liu et al. 2027). and many rely on input independent randomization (He et al. 2019 Jeddi et al. 2020 Dong Xu 2023); these non-certified randomized defences have often been circumvented by adaptive attacks that explicitly average over the internal noise (Athalye al. 2018 Tramer et al. 2020
Most prior defenses optimize for either certified or empirical robustness and are validated on a single regime-usually natural images, with only few addressing specialised medical data. HyCAS bridges this gap by incorporating a deterministic 1-Lipschitz architecture with stochastic smoothing modules (e.g. random- random-projection projection and attention-nois mechanisms), thereby forming randomized defense that robustly generalizes across diverse imaging benchmarks. A modest reduction in clean accuracy yields simultaneous performance gains in certified fz and empirical foo robustness (Fig. Consequently, HyCAS aims to surpass the strongest deterministic certifiers and the leading empirical defences. Further details appear Appendix A.
(related work) and Appendix A.2 (preliminaries).
Ob.c
2-Lipschitz
FDPAN
GEDPAN(E; S.M
Go(:1)
2
2-Lipschitt
I-Lipschitz
SNCAN
RANI
Cancan(Z:
Ob.c
Input.x
2-Lipschitz
2-Lipschitz
Jacohtia
RPFAN
Orthogocal
M..
ab,cc
Freaas--DDPAN, Figure 1: Overview of HyCAS mechanism. It consists of three parallel streams- FDPAN, SNCAN, and RPFAN- each built from i-Lipschitz with Randomized Attention Noise Injection (RANI) residuals. Per-channel convex gating fuses streams to form (= Each stream S 2-Lipschitz; the fused stream and the stacked network remain < 2-Lipschitz, enabling a margin-based &a certificate.
3 HYBRID CONVOLUTIONS WITH ATTENTION STOCHASTICITY
Randomized defenses often incorporate stochasticity into deep network structures by (i) tuning data- dependent spper-parameters (e.g. noise scale, sampling rate) (ii) data- independent architectural modifications. However, deep networks remain highly vulnerable on vision benchmarks, where im- perceptible perturbations can sharply degrade accuracy. Randomization alone is often insufficient; coupling it with Lipxchitz-constrainee deterministic architecture yields stronger certified and em- pirical robustness.
To address these limitations, propose Hybrid Convolutions with Attention Stochasticity (HyCAS), which replaces each convolutional layer in standard CNN backbones with Lipschitz- bounded stochastic streams that inject refined smoothing and controlled randomness into the net- work via two complementary. data- data-indepea independent components (i) a Lipschitz-constraiine constraine determin- istic architecture and (ii) dual stochastic design thereey
thereby improving adversarial robustness.
Let E #C be an input feature map with spatial dimensions (H, W) and C channels with label y E by (h) the (global) fz-Lipschitz constant map ("I--Liscchizz means [[λ(u) h(v)]l2 < Ll(u-ul|2- Our proposed #yCtesintegrateed any base classifier with parameters 0. The smoothed classifier induced by HyCAS is:
(1...., K). We denote by I1-I|2 the Euclidean norm vectorized tensors and
go (x) = arg max Pr.o/fo(a e: 2) = c]. eey
(1)
where Ω2 - (E, tb Ma), ~ (0, r2 denotes Gaussian noise with mean 0 and standard deviation
matching the dimensions of to enable randomized smoothing, È induces deterministic Lipschitz- constrained structure, v integrates implicit structural randomness (first-level stochastic defense), and M., injects the explicit attention noise (second-level stochastic defense). The classifier go(-)
3

---

Published as a conference paper at ICLR 2026
returns whichever class fo is most likely to return, taking expectations over the distributions W(r E; e; 12,, An overview is given in Fig. pseudocode is provided in App. A.7 A.8 Algorithms /3
HyCAS processes every feature map through three parallel streams: (a) Frequency-awant Determin- istic Projection with Attention Noise (FDPAN); (b) Spectrally Normalized Convolution with Atten- tion Noise (SNCAN); and (c) Random Projection Filter with Attention Noise (RPFAN). Their outputs are fused by a data-independent convex channel gate that down-weights high-sensitivity streams, thereby weakening naïve adversarial attacks.
Specifically, let B = {FDPAN, RPFAN, SNCAN] be the set of streams and for each stream b E B, let Gb(:; denote its output feature map and those output feature maps fused by channel- wise convex gate Obbee. For learnable, data-independent logits Ab.c. we define channel-wise convex exp(As,e)
weights ab,c = Even exp(Ay,e) such that Es Abbc = ab,e 2 thereby we obtain the HyCAS block output is
a(T).ane
ab. [Cb(,; + R Σ BEB
ab.e (G)( Ma): c c=1,..,,C.
(2)
BEB
where R denotes RANI module.
Convex fusion and expected-logit map are < 2-Lipschitz. If each stream satisfies Lip(Gr) 2, and the gate is convex (Eq. [2. then the per-channel fusion has Lip(z + x(x)) I maxée B Lip(G,) I 2. (ref. Appendix A.3 (Prop. 4) Taking the expectation over Ω pre- serves the Lipschitz constant (ref. Appendix A.3 (Lemma 21) so the network's 's expected logit map
(xx) (o; Q)] remains 2-Lipschitz. Formally:
Theorem 1 (HyCAS block is < 2-Lipschitz). Each constituent Arramm-SNCA RPFAN, and FDPAN with skip weight 3 S }/2 j individually < 2-Lipschitz. Indeed, every stream is the composition of three maps: (i) stochastic projection To (seeded by 1)), L-Lipschitz; (ii) a de- terministic projection De (parameterized by {). 1-Lipschitz; and (ii) a stochastic attention noise
Mw: : d2 - [0, 1-Lipschitz. For any input x, the resulting feature map
Golæ; w) = (z(To(z)) + Mc(De(To(=))
is therefore 2-Lipschitz on each forward pass. The subsequent per-channel convex fusion is non-expansive, cannot increase the Lipschitz constant. Consequenly, every HyCAS block is provably S. 2--lipschitz (see Appendix(A.3 (p.
Proof. See proof within the Appendix
A.3
Corollary 1 (Randomized Lipschitz margin certificate for expected logits). Let Z(x) be HyCAS
logits averaged over internal randomness, with Lip(Z) < 2. Let A(x) Z())= Z(2) (x) is the
gap between the top-two logits. Then r2(x) = =4 4
A(z) is is a valid l2 certificate: for all ||0]| < (2)
we have arg max Z(x + 5) arg max Zzz). This is the HyCAS pointwise E certificate (App. A.4
Proof. See proof within the Appendix A.3
The HyCAS- network optimised with a standard Eg loss as: LhyCAS = @ LEDPAN + LSNGAN + @ CEPPAN + K & CRANT,
(3)
where Ç. ip, v, and x denoted by learnable parameters, while @ represents Hadamard product.
All streams are spectrally normalised (||W]|2 < 1) and the stochastic attention noise module is I-Lipschitz. Hence, by Theorem Coroolllryy [ every HyCAS block-and any network built by stacking them is < 2-Lipschitz, so attacks with fg-norm < (xx)/4 cannot alter the prediction.
3.1 FREQUENCY-AWARE DETERMINISTIC PROJECTION WITH ATTENTION NOISE (FDPAN)
Under l bouudddd attacks, adversaries (i) conceal perturbations in figh-frreuency DCT coefficients and (ii) exploit channel- wise gradient regularities that generalize across models. FDP AN counters both phenomena by weaving frequency truncation, channel scrambling, spectral control, and cali- brated stochasticity into the architecture.
FDPAN is a four-stage cascade (see Appendices A.3 A.3 for Lemma 3 and Figure 5). where each component comprises deterministic I-Lipschitz core followed by two randomized residuals, i.e.,
4

---

Published as a conference paper at ICLR 2026
Table 1: Certified accuracy (%) of HyCAS and prior baselines on CIFAR-LO and ImageNet. Bold value
denotes the best in each column across all noise radius pairs. All methods are evaluated at two noise levels.
CIFAR-10
ImageNet
Approaches
Certified accuracy at predetermined & radius r (%)
Certified accuracy al predetermined Eg radius (%)
0.00
0.25
0,50
0.75
1.00
1.25
1.50
2.0
0.00
0.25
0.50
0.75
1.00
1.25
1.50
2.0
0.25
75.3
60.2
43.4
26.1
0
0
67.1
48.7
0
0
0
RS
0.50
65.2
54.1
41.3
32.4
23.2
14.7
9.34
57.3
45.9
36.8
28.7
0
0
0.25
78.6
63.2
47.5
30.8
19.6
10.3
5.72
68.4
58.5
46.2
38.7
32.1
19.3
10.8
0
IRS
0.50
71.3
58.5
44.1
33.3
24.1
15.7
11.4
2.2
62.4
50.9
41.5
34.7
27.3
20.2
13.8
6.31
0.25
83.4
65.8
50.2
34.5
24.7
15.8
10.5
0
70.6
61.2
51.8
42.7
38.4
32.6
25.4
0
DRS
0.50
78.1
62.1
48.7
35.8
24.5
17.9
12.9
4.6
67.6
58.2
49.6
42.8
35.6
33.2
29.8
21.3
0.25
84.1
67.3
51.4
39.1
30,9
21.1
16.2
71.1
61.4
52.7
43.1
39.1
33,4
26.7
0
ARS
78.4
63.7
38.9
31.8
19.7
8.47
68.1
58.7
39.1
30.6
22.4
80.5
64.7
48.6
34.3
23.6
15.2
9.14
0
69.7
60.6
50.9
42.2
37.1
30.5
21.8
0
LOT
76.7
60.4
46.3
35.1
24.9
17.3
12.1
6.25
66.1
57.4
48.9
42.8
38.4
32.9
28.3
19.8
SLLI
81.4
65.3
49.9
33.1
23.6
14.7
9,94
0
70.2
57.7
48.4
41.8
37.6
31.9
24.3
0
77.9
62.6
48.7
34.5
24.4
16.2
13.7
5.83
67.3
55.5
49.1
42.8
39.1
34.5
26.7
21.3
HyCAS
0.25
85.4
70.1
56.7
44.3
36.5
29.6
22,9
8.52
72.3
63.9
55.6
46.4
40.7
35.2
29.7
5.42
0.50
65.3
36.8
30.3
23.4
12.5
60.6
41.1
36.3
32.7
24.8
L-pischii::: (i) Low-pass DCT mask (L-Lipschitz) - excises fragile high-frequency bands. (ii) Orthogonal Jacobian 1 x 1 matrix + Randomized Attention Noise Injection (RANI) (2-Lipschitz) scrambles channel gradients and injects structured noise. (iii) SNCAN (2-Lipschitz) - keeps the convolutional kernel spectrum bounded while introducing additional stochasticity. (iv) RANI that further incorporates refined randomness and is 2-Lipschitz.
Let the deterministic core be Hxx) Ck. (U T T(@ where Ck, be Spectrally normalized convolution, U' denotes Orthogonal Jacobian matrix layer, & is the orthonormal 2-D DCT and A is the low-pass mask.
To incorporate richer, refined stochasticity, we apply RANI immediately after the deterministic core. Given an attention noise Ma and a noise-strength parameter E [0, 1]. RANI module is denoted by nax; Ma), and is 1-Lipschitz for every freshly drawn during both training and inference. Hence, combining the deterministic path and two independent RANT injections (stochastic) yields the FDPAN stream output as: GFDPAN (z; &, Mw) = H (x;{) + R (H E); Mw:). where ( the deterministic core, and each R (; Ma.) introduces independent stochastic attention noise Mw, for {1 2). Because the two My, terms are cascaded and the entire stream is at most 2- 2-Lipschitz by the triangle inequality. The skip connection is I-Lipschitz as well. Notably, more details about SNCAN module and RANI mechanism are the following sections. Formally:
Proposition 1 (FDPAN is at most 2-Lipschitz). Assume the deterministic core H(-) is 1 1-Lipschitz and that, for every attention noise Ma the RANI R(-;w; is also 1-Lipschitz; the skip connection is likewise 1-Lipschitz. Define the FDPAN stream output by GFDPAN(E; M.) = H(-) + (R(-) H())) is 4-Lipschitz and therefore satisfies Lipschitz(FFDpan) S 2.
Proof. See proof within the Appendix A.3
Therefore, FDPAN minimises the objective of HyCAS by incorporating refined stochasticity into the network through all employed modules as mentioned in the above:
(PDPAN(0) = mjn E(e.v) E=-N e.Mu, 4Jo(w + e; Mw,)..)
(4)
3.2 SPECTRALLY NORMALIZED CONVOLUTIONS WITH ATTENTION NOISE (SNCAN)
To design SNCAN (see Appendix A.5 (Figure ) we replace every standard convolutional layer with a spectrally normalized convolution (SNC; see Appendix A.2.2). This substitution introduces controlled gradient variability while preserving the deterministic I-Lipschitz bound on worst-case l perturbations. However, the resulting stationary gradient fields can still be exploited by ad- versarial attacks. To mitigate this vulnerability and further strengthen robustness, we incorporate our data-inded indepidddeennent RANI module (Section 3.4) to each SNC layer, thereby injecting fine- -raiied stochasticity while preserving tight Lipschitz envelope.
5

---

Published as a conference paper at ICLR 2026
Let (x., R(c; Dw where CK. is SNC with kemnll K, rescaled to satisfy ||ellop I 1 and Dw = diag(MMw is a diagonal matrix formed from the attention noise tensor My E [0, 1] Because every diagonal entry Dw lies in [0, 1], we have |D 2 I 1. Hence Li, S 1, Li; M.)) S 1, Lip(1 + Dw) 2.
RANI generates a bounded attention noise M E [0,1] and forms the stochastic residual output as:
GE: M,) = Cx. (x+ R(C {); Mo) = (T+ Dw) Cx. (x;
(5)
By incorporating RANI into a deterministic, I-Lipschitz convolutional block, we obtain a random- ized defense that is provably 2- 2-Lipschitz, formalized below.
Proposition 2 (2-Lipschitz hybrid block). For every input pair 2, E R#xxxC and every noise sample Ma
|[ (a; M.) (L) I 2| 8/12.
Proof. See proof within the Appendix A.3
Each SNCAN block, therefore, multiplies the network's global Lipschitz constant by at most 2 while injecting fresh randomness on every forward pass, synchronizing the gradient landscape that an adversary sees. Therefore, SNCAN minimizes the objective of HyCAS by incorporating refined stochasticity into the network through SNC and RANI modules:
LSSCAN(0) = min E( B2-N10,23), e.M., Afo( + E;E Mo).v).
In summary, spectral normalization (e.g.. SNC) complemented by RANI yields randomized module built on deterministic architecture, whose Lipschitz envelope remains tight while its gradients vary across evaluations.
3.3 RANDOM-PROJECTIONN CONVOLUTION WITH ATTENTION NOISE (RPFAN)
RPFAN couples spectrally controlled random projection (1-Lipschitz) with data-independent randomized attention residual (2-Lipschitz). It therefore introduces dual stochasticity- from the random projection itself and (ii) from RANI-while keeping the stream's Lipschitz constant at most 2. In practice, both the random projection and attention noise are freshly resampled as described in $3.5. Network Execution.
The RPFAN module (see Appendix A.5 Figure inherits the Johnson Lindenssrsss (JL) embed- ding guarantee of candom-projection filter (RPF) (Dong Xul 2023) (see Appendix [A.2.3 for details) and extends it with three carefully chosen components: (i) two core innovations that render the module certifiably 1-Lipschitz, and (ii) the RANI module, which raises the Lipschitz constant to 2 while injecting additional source of data-independent stochasticity. Combined, these ele- ments provide dual stochasticity- arising from the random projection itself and the other from RANI thereby strengthening adversarial robustness without exceeding 2-Lipschitz bound. The three components are summarized below.
1. Energy-preserving preserving channel pre-mix. Before the nandom-projection filter is applied, we lever- age 1x1 orthogonal Jacobian matrix as channel mixer U with U = (Horn & Johnson 2012)) apply + Un. which equalises channel energy so that every spatial dimension enters the projection space with identical energy distribution (see ( 3
2. Baich-aware spectral normalisation for random projection. The random- projjection filter Wo
is sampled exactly in (Dong & Xu 2023 (ref. A.2.3). We then rescale it using per-sample,
two-step power-iteration (PI) scheme: (i) Draw (O,, of shape (N, !f w
+ Cou); (ii) Up-
date twice Conv Co:WW( +- - normalising each sample independently; and I-I|a
Conv(e:Wo)
(iii) compute the Rayleigh quotient (RQ), thereby form spectral normalized random projec- tion filter WSN as:
RQ Σ (un, Couv(Un; Wo)), Wsn max (RQ,
W
This batch-aware PI yields tighter bound on Conv(:: Wo)lla than layer-wise Pf while guarantecing that the projection remains 1-1 1-Lipschitz (Appendix A.3

---

Published as a conference paper at ICLR 2026
Table 2: Certified accuracy (%) of HyCAS and prior defenses on CelebA, HAM10000, and NIB-CXR.
Boldface denotes the best in each column across all noise- radius pairs. Methods are evaluated at 3 noise levels.
CelebA
HAM10000
NIH-CXR
Approacbes
radius (%)
Eg radius (%)
8g radios r (%)
0.0
0.50
1.0
0.0
0.50
1.0
0.0
0.50
1,0
0.25
92.8
45.7
0
94.6
53.2
10.5
77.4
43.5
15.7
RS
0,50
87.7
47.8
10.5
89.3
52.1
12.2
73.3
39.9
21.8
1.0
81.4
51.6
18.8
84.7
54.3
21.2
66.4
42.9
22.8
95.2
53.3
96.7
31.3
79.1
32.5
ARS
0.50
91.3
53.9
30.4
91.9
55.1
32.8
74.9
33.3
85.3
59.2
31.6
86,9
57.4
34.6
69.9
52.9
34.1
HyCAS
0.25
96.8
58.1
33.7
97.2
60.5
35.4
81.6
61.9
38.6
0.50
92.7
59.3
34.8
93.1
60.4
36.6
76.2
58.6
40.9
1.0
87.7
62.3
36.9
88.2
61.9
38.5
71.7
60.6
414
3. Randomised Attention Noise Injection (RANI). Given the 1-Lipsscitzz projection output h CUuv(Ux; Wen). draw internal randomness and apply a data-indeepndent bounded mask M_ E [0, 1] through the RANI module R(h; M..) For newly drawn || [|[+-D S || | lla S 2: therefore the composite map æ >> Id COnv(Ua:; WaN): :w) 2-Lipschitz. This couples the spectrally normalised random projection with RANI, injecting refined stochasticity while multiplying the stream's 's Lipschitz con- stant by at most 2 (ref. Proposition 2 see also App. A.2 for the residual bound). Define the I-Lip core HRPFAN(x) Conv(U2; Wsx) and the stream output as:
CRPFAN(Z; Mw) (aPFAN(;; (HRPFAN(;;V);
then Lip( (uippRPAA)) S (See Proof 5
Therefore, RPFAN minimises the objective of HyCAS by incorporating refined stochasticity into the network through RPF and RANI modules:
CRpPAN min B(s,y) BenNio, a =,;
(8)
3.4 RANI: RANDOMIZED ATTENTION NOISE INJECTION
Motivation. Certified deterministic 1-Lipschitz defenses (e.g.. SNC) bound the worst-case &g per- turbation but still expose deterministic gradient field that adversaries can exploit. Classical ran- domized defenses inject noise only at the input, whereas certified Lipschitz defenses remain deter- ministic inside the network. RANI closes this gap: injects a dlatu-independent, stochastic attention mask Ms E [0, 1] after every spectrally spectrally-nommalised block in the three streams (FDPAN, RPFAN, SNCAN) and once more at their fused output, while preserving a global 2- 2-Lipschitz envelope. For- mally, the deterministic 1-Lipschitz map h (x;; E, 10) is replaced by the stochastic 2-Lipschitz map h R(h; Mo) via incorporating RANI module (R(; M.).
Attention noise mechanism. For each forward pass, RANI draws fresh noise ~ NO, I) for internal randomness and computes bounded attention noise M., e [0, 1] that is independent of the current features. For any deterministic feature tensor / H x we modulate it according to:
h h Mw.
where @ denotes the Hadamard product. This yields a Lipschitz constant of at most 2; hence every block's s constant grows from 1 Lipschilz deterministic to randomised defense (see Appendix A.3 (Lemma
In practice, Ms is produced by our RANI module via injecting noise local and channel information of the given deterministic feature maps (e.g.. h E Hx;; v))); (See Appendix A.6 for more details of our RANI module. The following lemma states the guarantee formally.
Lemma (RANI module is 2-Lipschitz). Let h aad and let (w) E [0, be sampled i..dd from an arbitrary distribution that is independent of Define the RANI mapping as shown Eq. 9 Then, for each randomly drawn and any hy, hu, the mapping i 2-Lipschitz with respect to the Euclidean norm ||2: i.e.
[[(h;;; Mw) R(hus MD)ll2 S 2 11/1 hallz.
Proof. See proof within the Appendix A.3

---

Published as a conference paper at ICLR 2026
Robust accuracy (%) against loo attacks (APGD-20 and AA-20) on NIE-CXR (left) and NCT CEIT-CCC-HEE -HHEE-10FF (right) at E (8/255, 16/255).
Table
Method
NCT-CRC-HE-INK
Clean
APGD-20
20
AA-20
Clein
APGD-20 20
AA-20
8/255
16/255
8/255
16/255
8/256
16/255
5/255
16/255
AT
89.1 + 1.91
74.7
2.552
66.9 3:41
74.2++2.50
64.1 3.70
92.2 1.82
17.8 2.51
68.7 3.12
76.3
7..8
66.2 3.61
RPF
58.4 1.82
83.7 2.49
11:9::33:21
825
70.8 3.52
91.1
2.33
73.9
3.33
84.2
12.4
CTRW
68.4
2.21
3.222
84.5
72.6
90.4
2:20
76.7
3.112
2.44
DCS
47.2422.00
8.44++2.11
4:221
81.7
2.77
00.6
90.3 1.03
84.5
73.0
83.3 2.7.4
71.6 3.46
ARS
64.8
1.001:3001
64.7 3,25
62.8 3.72
56.8 2.3.2.2..11
15.9
66.1 d: 3.52
74.6 3.11
64.5
2.39
71.6
2.29
3.82
63.7
HyCAS
89.5
1.64
2.33
77.3
86,9
2.42
74.4
3.33
91.3
2:882
79.3
3.52
88.2
1:
76.7 a: 3.34
Table 4: Robust accuracy (%) against loo attacks (APGD-20 and AA-20) on HAM10000 (left) and EyePACS (right) at e (8/255, 16/255}.
Method
HAM1000
ExePACS
Clean
APGD-220
Clean
APGID-20
8/255
16/255
8/255
16/256
8/266
16/255
8,255
16/256
AT
2
56.1
3.49
46.5
3.75
11.21
2:911
60.011272
50.1
3:5,2
58.3 2.83
16.21
RPF
2.86
64.1
1111+:00 4.441
51.9
3.42
62.6
3.71
50.4+
77.1 2.90
67.8 2.377
56.1
+ +.1.2
66.4 2.78
54.4
0.44
CTRW
74.32.270
64.7
52.8
3.32
63.3
51.2 3.52
70.1 1:533
37.7
3:35
00.7
56.1 3.31
DCS
2.94
4.448
61.4 3.00
48.5
2.94
2.53
56.2 3.66
65.3
ARS
6.28
53.9 3.88
44.1 #:3.3
62.7
42.8 3.71
2.991
48.8 18.6+
67.6 07.6.t2..44
46.5 3.73
DRS
08.9
28.9.33..24
53.4 3.84
43.2::3.44
51.64
3.98
58.3
2.61
47.4 3.71
56.5 2:99
45.7 4: 3.94
HyCAS
74.6 2.74
53.1
72.6 2.79
00.5
58.3
Therefore, RANI minimises the objective of HyCAS by incorporating refined stochasticity into the
network:
(RANI( min E(e,y) Be-N(u..2 M. (fo(r + Mw).9),
(10)
TRADES
B7F
70
CTRW
DRS
A15
HyTAS
(ND
20
0.01
0.32
0.03
0.08
APCD Siaps
CIFAR 10 ARGD Sizes
CHARR-U
Figure 2: Empirical robustness of HYCAS versus Icading baselines (RPF, CTRW, DRS, ARS) on CIFAR-1 under strong APGD attacks. We evaluate two settings: (1) perturbation sizes e from 0.01 to 0.08 and (2) iteration steps from 10 to 100.
EXPERIMENT RESULT
4.1 EXPERIMENT SETUP
HyCAS on eight et al. 2015). MCTT-CCC-H--LL 2009). ImageNet- 1k (Deng et al. 2009). CelebA (Lu (Kather et al.) 2018). NIH-CXR (Wang et al., 2017).
Evaluation protocol. We evaluate
vision benchmarks
(CIFAR- 1 100 (Krizhessky
EyeP ACS (EyePACS] 2015), and HAM10000 (Tccaand et al. 2018)). We report certified accuracy at preset l radii r for smoothing noise levels a E {0.25, 0.50, 1.0.2.0).
Empirical robustness is measured under lx APGD-20 (Croee & Hein 2020 and AutoAttack
(AA) CCroe & Hein 2020) at budgets c E (8/255, 16/255), We also evaluate HyCAS under
We use the combination of fo APGDCE and lo -APGDy- DLR from Croce & Hein 2020). each run for 20 iterations with 5 random restarts; denote this union as APGD-20.
8

---

Published as a conference paper at ICLR 2026
stronger APGD settings with larger and more attack steps (Figs. 2 Baselines span randomized smoothing methods (RS (Cohen et al. 2019). EUgree et a..). DRS (Xia et al. 2024 ARS (Lyu
et al. 2024), and I-Lipschitz defenses (LOT (Xu et al. 2002). SLL (Araujo et al. 2023 All exper-
iments are run with five random seeds, and we report the mean, the standard deviation, or both for each experiment. Implementation details, certification and empirical settings in Appendix A.8
TRAGES
50-
H1
45.0
42.5
40
40.2
30
37.5
20-
APGO Shees
APGD Steps
CIFAR-100 APGD Stace
CIFAR-100 APGD Staps
Figure 3: Empirical robustness of HyCAS leading baselines (RPF, CTRN, ARS) CIFAR- under strong AP attacks. We evaluate two settings: (1) perturbation sizes from 0.01 0.08 and (2) iteration steps from 10 to 100.
4.2 CERTIFIED ADVERSARIAL ROBUSTNESS AT PRESET RADII
CIFAR-10 and ImageNet. Across all baselines in Table HyCAS achieves the best certified accu- racy for every (r, o) pair. On the CIFAR-10 O, the representative medium radius r-0.75, it yields 44.3% certified accuracy for both ø {0.25, 0.50) an gain of 5.2-18.27 over the prior methods. 0.50), it retains 12.5% e,, surpassing the leading baseline by
Even in the large-radius tail (r=2.0, 4.0-12.5%. A similar trend emerges on geaetNet: in the large-radius regime (r = 1.5, = 0.50), HYCAS reaches 32.7% certified accuracy, exceeding every baseline by 2.1-32.7% HyCAS also de- livers state-of-ther are- clean accuracy 85.4% on CIFAR-10 and 72.3% on ImageNet-modesll. but consistently ahead of all baselines.
Skin, Chest Xray, and Face datasets. Table demonstrates the same dominance beyond CIFAR- 10 and ImageNet datasets. On the CelebA dataset, HyCAS achieves certified accuracies 62.3% at r 0.5 and 36.9%
1.0, outperforming RS and ARS by 5.3-18. 18.1%.. For the HAM1 0000 dataset, it reaches 61.9% % (at r 0.5) and 38.5% (at r = 1.0). leading all baselines by approx- imately 4%. On the N dataset, certified accuracy spans 61.9% (at 0.5, a 0.25) to 41.4% % (at r = 1.0, a = 1.0), gain of 3.5-7.3% over the leading baseline (e.g., ARS). Clean accuracy is likewise higher on par across the board, ranging from 81.6-97.2%
Effect of the noise level. Increasing the smoothing noise a consistently trades a negligible drop small radii for substantial gains large radii across every baseline, yielding tunable accu- racy-robustness frontier. For example, on CIFAR-10, raising from 0.25 to 0.50 leaves perfor- mance at 0.75 unchanged (44.3%) yet improves adjustment on ImageNet elevates = 2.0 from 5.42% to 24.8%. These monotonic improvements confirm that HyCAS provides a controllable, rather than fixed, trade-off curve.
2.0 from 8.52% to 12.5%. The same
4.3 EMPIRICAL ADVERSARIAL ROBUSTNESS
Across our empirical evaluations (Tables 3 HyCAS achieves the highest robust top-1 accuracy under APGD-20 and AA-20 at e (8, 16); /555. Specifically, on the NIH-CXR benchmark, HyCAS retains robust accuracy, outperforming the leading baseline (CTRW) by about +1.8-4.2% % across these attacks while maintaining similar clean-set accuracy (89.5% vs. 88.4%). A similar trend ap- pears on the NCT- CRC- HE 100K dataset, where HyCAS records robust accuracies of 76.7-79.39 3% a = 16 (6/255 against the same attacks, exceeding CTRW by roughly +1.5-2.6 % and leaving earlier certified defences (e.g.. ARS, DRS) more than +12% behind at this stronger perturbation level. Denmoscopic HAM1 0000 and fundus-image EyePACS exhibit the same hierarchy: HyCAS secures robust accuracies 53.1-67.8% 8% against APGD-20 20 and AA-20 attacks on HAM10000 around +1.9-3.1% % better than the next-best adversarial defence and widens the margin EyePACS to

---

Published as a conference paper at ICLR 2026
58.3-72.6%, thereby surpassing the leading baseline CTRW by approximately +2.1-2.8% Together, these results show that HyCAS transfers its randomized Lipschitz strategy from certification to em- pirical regimes, maintaining clean accuracy while achieving state-of- ohethh--ttttttte adversarial robustness.
Under stronger APGD attacks on CIFAR- 10/R 10/10000000000 100 (Figs. 23. HyCAS outperforms all basclines and preserves its advantage as attack strength increases. On CIFAR-10, when the perturbation radius varied from c = 0.01 to 0.08, HyCAS traces the upper envelope of the robust-accuracy curves, retaining an s 10% advantage at the largest perturbation, where leading methods collapse. A similar trend holds on CIFAR- 1 100 as the number of APGD iterations increases from 10 to 100: all prior defenses, including TRADES (Zhang et al.) 2019) and HR (Bennouna al. 2023), degrade monotonically, whereas HyCAS declines more gracefully and remains 7-12% above the closest competitor at 100 steps, confirming that its internally resampled attention noise and random pro- jections thwart extended optimization. Thus, this randomized, hipschitz-constraine design scales gracefully with both perturbation size and steps, offering adversarial robustness and a broader safety margin.
0.00
0.25
0.50
0.75
1.00
1.25
1.50
2.00
2,25
0.00
0.25
0.50
0.75
1.00
1.25
1.50
2.00
2
MYCAS robusTros MycAG Certiled Enrehtic
MyCAS rosustress HaCAS- Centiled
80
60
30
40-
50
20
10-
10
0.00
0.01
0.92
0.03
0.04
0.03
0.00
0.07
0.00
0.20
0.02
0.03
0.04
0.05
0.08
0.07
0.02
pertutiation
Z.
pertl Botton C
Contfiid--mprrial
Trede-affi CIFAR-10
Com/fiol-Foppicaal Adverccial
ImageNei
Figure 4: Trade-off' between certified and empirical adversarial robustness achieved by HyCAS on the CIFAR-10 (Left) and ImageNet (Right) datasets.
4.4 CERTIFIED-EMPRRICAL ROBUSTNESS TRADE-OFF
Figure[4plots HyCAS on three-axis Pareto frontier that couples certified lz accuracy (radius r) with empirical l robustness (APCD-20 accuracy at perturbation strength c). Across both CIFAR-10 0 and ImageNet, the frontier is smooth and strictly downward-sloping: as the certified radius widens, empirical robustness inevitably contracts. Two consistent phenomena stand out: (a) Certificate conservativeness. For the small perturbation regime (left-most region), the empirical curve lies markedly above the certified curve, confirming that formal certificates bre-by relative to observed robustness. (b) Norm mismatch tail-gap. At large radii/perturbation strength (right-most region), the gap widens further, highlighting the inherent difficulty of translating & guarantees into loo performance.
HyCAS achieves this trade-off by increasing the smoothing noise from a = 0.25 to 0.50 (arrow along each curve) leaves mid-radius performance virtually unchanged, yet extends both certified and empirical robustness deep into the hirh-peeturbattoon regime. On CIFAR- 1 1, certified accuracy at radius r 2.0 improves from 8.5% to 12.5%, while ImageNet shows an even larger jump from 5.4% to 24.8% at the same radius. Crucially, these gains incur minimal loss in accurrayy / small-c robustness, giving this state-of-the-ad adversarial defense a knob to dial the desired secu- rity level without wholesale accuracy sacrifice. See Appendix for additional experiments and Appendices A. for detailed ablations and certified and empirical robustness discussions.
CONCLUSION
We presented HyCAS, randomized adversarial defence whose deterministic 1-Lipschitz archi- tecture is incorporated with two forms of data-independent internal randomness, yielding global < 2-Lipschit network and a simple margin- lg certificate. Experiments on diverse imaging benchmarks demonstrate state-of-the art certified accuracy and strong empirical robustness against powerful E attacks. Future work includes deriving tighter fox certificates, designing lighttr-weight certification samplers, and integrating HyCAS into multi-modal clinical pipelines.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Motasem Alfarra, Adel Bibi, Philip HS Torr, and Bernard Ghanem. Data dependent randomized smoothing. Uncertainty in Artificial Intelligence, pp. 64-74. PMLR, 2022a.
Motasem Alfarra, Juan C Pérez, Ali Thabet, Adel Bibi, Philip HS Torr, and Bernard Ghanem. Com- bating adversaries with anti-adversaries. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 1900,6000, 2022b.
Alexandre Araujo. Aaron Havens, Blaise Delattre, Alexandre Allauzen, and Bin Hu. A unified algebraic perspective on lipschitz neural networks. arXiv preprint arXiv:2303.03169, 2023.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of curity: Circumventing defenses to adversarial examples. In International Conference on Machine Learning (ICML), 2018.
Amine Bennouna, Ryan Lucas, and Bart Van Parys. Certified robust neural networks: Generalization and corruption resistance. In International Conference on Machine Learning, pp. 2092-2112. PMLR, 2023.
Yulong Cao, Ningfei Wang, Chaowei Xiao, Dawei Yang, Jin Fang, Ruigang Yang, Qi Alfred Chen, Mingyan Liu, and Bo Li. Invisible for both camera and lidar: Security of multi- fusion based perception in autonomous driving under physical-wordd world attacks. In 2021 IEEE Symposium on Security and Privacy (SP), 2021.
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy (SP), pp. 39-57, 2017. doi: 10.1 10.1109/SP.2017..99.
Nicholas Carlini, Florian Tramèr, Krishnamurthy Dvijotham, Leslie Rice, Mingjic Sun, and J. Zico Kolter. (certified!!) adversarial robustness for free! In International Conference on Learning Representations (ICLR), 2023.
Zhiyuan Cheng. James Chenhao Liang, Guanhong Tao, Dongfang Liu, and Xiangyu Zhang. Adver- sarial training of self-supervised monocular depth estimation against physical-world attacks. In International Conference on Learning Representations (ICLR), 2023.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized
smoothing. In International Conference Machine Learning (ICML), 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206- 2216. PMLR, 2020.
Francesco Croce, Sven Gowal, Thomas Brunner, Evan Shelhamer, Matthias Hein, and Taylan Cemgil. Evaluating the adversarial robustness of adaptive test-time defenses. In International Conference on Machine Learning (ICML), pp. 4421-4435, 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: large-scale hierarchical image database. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2009.
Joy Dhar, Puneet Goyal, Maryam Haghighat, Nayyar Zaidi, Ferdous Sohel, Bao Q Vo, and KC San- tosh. Towards building robust models for unimodal and multimodal medical imaging data. Infor- mation fusion, pp. 103822, 2025.
Joy Dhar, Nayyar Zaidi, and Maryam Haghighat. Effective and robust multimodal medical image analysis. arXiv preprint 1331v.2602.15346, 2026.
Gavin Weiguang Ding, Luyu Wang, Xiaomeng Jin, Furui Liu, Yash Sharma, Adam Yala, and Ruitong Huang. MMA training: Direct input space margin maximization through adversarial training. In International Conference on Learning Representations (ICLR), 2019.
Minjing Dong and Chang Xu. Adversarial robustness via random projection filters. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 4077-40477 2023.

---

Published as a conference paper at ICLR 2026
Ranjie Duan, Yuefeng Chen, Dantong Niu, Yun Yang, A. Kai Qin, and Yuan He. AdvDrop: Ad- versarial attack to DNNs by dropping information. In IEEE/CVF International Conference on Computer Vision (ICCV), 2021.
EyePACS. EyePACS diabetic retinopathy detection dataset. https::/ww.kaggle..com
diabetic rew-reeiiig hww.iiipptttyeeeeee ion 2015. Accessed 18 Sep 2025.
Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael J. Cree. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning, 110(2):393-411 2021. doi: 10.1007/s10994-0000 020-05908-7
Zhongkai Hao, Chengyang Ying, Yinpeng Dong, Hang Su, Jian Song, and Jun Zhu. GSmooth: Certified robustness against semantic transformations via generalized randomized smoothing. In International Conference on Machine Learning (ICML), 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Zhezhi He, Adnan Siraj Rakin, and Deliang Fan. Parametric noise injection: Trainable randomness to improve deep neural network robustness against adversarial attack. In Proceedings of the IEEE/CVF conference computer vision and pallern recognition, pp. 588-597, 2019.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- tion, pp. 15262-15271, 2021.
Hanbin Hong, Binghui Wang, and Yuan Hong. UniCR: Universally approximated certified robust- via randomized smoothing. In European Conference Computer Vision (ECCV), 2022.
Roger Horn and Charles R Johnson. Matrix analysis. Cambridge university press, 2012.
Ahmadreza Jeddi, Mohammad Javad Shafiee, Michelle Karg, Christian Scharfenberger, and Alexan- der Wong. Learn2pertur:: An end-to-end feature perturbation learning to improve adversarial robustness. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1241 1250, 2020.
Jongheon Jeong and Jinwoo Shin. Consistency regularization for certified robustness of smoothed classifiers. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Jakob Nikolas Kather, Niels Halama, and Alexander Marx. 100,000 histological images of human colorectal cancer and healthy tissue, 2018. URL /tLp:tts///ci..gg///// 1214456
zenooo.
Dataset: NCT-CRC-HE-IOWE 100K.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Mathias Lécuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. In IEEE Symposium Security and Privacy (SP), pp. 656-672, 2019,
doi: 1099//9//P..019.00 00000.
Qizhang Li, Yiwen Guo, Wangmeng Zuo, and Hao Chen. Making substitute models more bayesian can enhance transfcrability adversarial examples. arXiv preprint arXiv:2302.0m 050866 2023.
Hongbin Liu, Jinyuan Jia, and Neil Zhenqiang Gong. Pointguard: Provably robust 3d point cloud classification. Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pp. 6186-6199 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2015.
Saiyue Lyu, Shadab Shaikh, Frederick Shpilevskiy, Evan Shelhamer, and Mathias Lécuyer. Adaptive randomized smoothing: Certified adversarial robustness for multi-step defences. Advances in Neural Information Processing Systems, 177113443-1340777 2024.
12

---

Published as a conference paper at ICLR 2026
Yanxiang Ma, Minjing Dong, and Chang Xu. Adversarial robustness through random weight sam- pling. Advances in Neural Information Processing Systems (NeurIPS), 2023.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In International Conference on Learning Representations (ICLR), 2018.
Apapan Pumsirirat and Yan Liu. Credit card fraud detection using deep learning based on autoen- coder and restricted boltzmann machine. International Journal of Advanced Computer Science and Applications (IJACSA), 2018.
Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial exam- ples. In International Conference on Learning Representations (ICLR), 2018.
Leslie Rice, Eric Wong, and J. Zico Kolter. Overfitting in adversarially robust deep learning. Pro- ceedings of the International Conference Machine Learning, pp. 8093-8104. PMLR, Novem- ber 2020.
Hadi Salman, Jerry Li, llya Razenshteyn, Pengchuan Zhang, Huan Zhang, Sébastien Bubeck, and Greg Yang. Provably robust deep learning via adversarially trained smoothed classifiers. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Hanie Sedghi, Vineet Gupta, and Philip M Long. The singular values of convolutional layers. arXiv preprint arXiv:1805. 104008 2018
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Gaurang Sriramanan, Sravanti Addepalli, Arya Baburaj, et Towards efficient and effective adver- sarial training. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
Peter Súkeník, Aleksei Kuvshinov, and Stephan Günnemann. Intriguing properties of input- dependent randomized smoothing. In International Conference on Machine Learning (ICML), 2022.
Florian Tramèr, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. adaptive attacks to adversarial example defenses. In Advances in Neural Information Processing Systems (NeurIPS), pp. 1633-1645, 2020.
Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The HAM10000 dataset: A large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 5: 180161, 2018. doi: 10. 10.1038/sdata.20 2018.. 161.
Shubham Ugare, Tarun Suresh, Debangshu Banerjee, Gagandeep Singh, and Sasa Misailovic. Incre- mental randomized smoothing certification, In The Twelfth International Conference on Learning Representations.
Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M. Sum- mers. ChestX-ray8: Hospital-scale chest x-ray database and benchmarks weakly-supervisee classification and localization of common thorax diseases. In Proceedings of the IEEE Con- ference Computer Vision and Pattern Recognition (CVPR), pp. 3462-3471, 2017. doi: 10.1109/CVPR.2011..36 Dataset: NIH ChestX-ray (ChestX-ray8/14)
Eric Wong and J. Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning (ICMI.), 2018.
Song Xia, Yu, Xudong Jiang, and Henghui Ding. Mitigating the curse of dimensionality for certified robustness via dual randomized smoothing. In ICLR, 2024.
13

---

Published as a conference paper at ICLR 2026
Kun Xiang, Xing Zhang, Jinwen She, Jinpeng Liu, Haohan Wang, Shiqi Deng, and Shancheng Jiang. Toward robust diagnosis: A contour attention preserving adversarial defense for covid- 19 detection. In AAAI Conference Artificial Intelligence (AAAI), volume 37, pp. 2928-2937, 2023.
Xiaojun Xu, Linyi Li, and Bo Li. Lot: Layer-wise orthogonal training on improving 12 certified robustness, Advances Neural Information Processing Systems, 35: 15:18904-18915 2022.
Zhuolin Yang, Linyi Li, Xiaojun Xu, Bhavya Kailkhura, Tao Xie, and Bo Li. On the certified robust- ness for ensemble models and beyond. In International Conference on Learning Representations, 2022.
Zheng Yuan, Jie Zhang, Yunpei Jia, Chuanqi Tan, Tao Xue, and Shiguang Shan. Meta gradient adversarial attack. In /EEE/CVF International Conference Computer Vision (ICCV), 2021.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pp. 7472-7482, PMLR, 2019.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 ADDITIONAL RELATED STUDY
Note that: in the Table 5 we compare the properties for novel adversarial defense approach for enhancing adversarial robustness against existing baselines, demonstrating how HyCAS uniquely overcomes each identified research gap.
A.2 PRELIMINARIES
A.2.1 RANDOMIZED SMOOTIING (RS)
Consider k-class classification problem with input E R and label E J'= (ci...., c. Rs first corrupts each input æ by adding isotropic Gaussian noise (e;; o² then turns an arbitrary base classifier f into smoothed version F that possesses lz certified robustness guarantees. The smoothed classifier returns whichever class the base classifier f is most likely to return under the distribution (xx +,,2,
F(x) arrg max cE
(11)
Theorem (Cohen et al. 2019). Let f: IR' + y be any deterministic or random function, and let F be the smoothed version defined in Equation equation 77 Let ca and cp be the most probable and runner-up classes returned by F with smoothed probabilities and pn, respectively. Then F(x + 5) = cA for all adversarial perturbations ō satisfying
|| - R' R' = 1 ( - - -D'(pB)))
where t 1 is the inverse standard-Gaussian CDF.
In Equation 2, & denotes the Gaussian cumulative distribution function (CDF) and Φ-1 signifies its inverse function. Theorem indicates that & certified robustness provided by RS closely linked to the base classifier's performance the Gaussian distribution; more consistent predic- tion within a given Gaussian distribution will return a stronger certified robustness. (The proof of Theorem can be found in Appendix A.1.) It is not clear how calculate PA and pæ exactly when ff is a deep neural network, so Monte Carlo sampling is used to estimate the smoothed probability. The theorem also establishes that, when we assign PA lower-bound estimate P^ and assign pa an upper-bound estimate with PB Px the radius R' equals
Φ (2,)- R'
Equation (3) follows from 11 p,) t Φ The smoothed classifier F is therefore guaranteed return the constant prediction C, around within the & ball of radius R'.
A.2.2 SPECTRAL NORMALISATION OF CONVOLUTIONS
For kernel & xk., CiyxCoot we denote by CK the induced circular convolution. We follow the two most widely-used operator-norm estimators:
(a) Exact Fourier bound (edghi et al. 2018 derived
a.(K)
R (w)
[[Cx|lop o.(K),
(12)
WES2
which we adopt verbatim in Eq. 12 to scale kernels whenever an FFT is affordable.
(b) Power-iteration (PI) surrogate (Miyato al. 2018 proposed light T -step estimate, also used by subsequent Lipschitz CNNs. Our implementation in Eq. [3 mirrors their update:
(T) (, (u(T))))
(13)
with 7= in their default setting.
15

---

Published as a conference paper at ICLR 2026
Kernel rescaling. Both estimators feed the same renormalisation rule
K
K () +
E = 10-6;
(14)
which keeps ||Cx llop 1. (The clamp max 1} minor safety tweak; we note it here for completeness but do not claim novelty.)
Proposition 3 (Layer-wise I-Lipschitzness). Eqs. 12 14 ensure l|C S 1.
All subsequent sections treat Eq. 14 as black-bor box deterministic contraction. Our contribution
begins only after this step, in the following Method section.
Proof. Fix w and let z, be arbitrary inputs. Define
Zx
Cx(x))
Zy = Cr (1)).
Step 1: 1-Lipschitz contraction.
By construction of K (Fq. 14) we have
|||| zull2 [0)x(x) Cx I a 2112.
(15)
Step 2:
Bounded multiplicative mask. The mask generated by RANI is data-independent for fixed w, and satisfies the element-wise bound M (w) E [0, 1| Consequently
1 1 + M(w) & (element-wise).
For any tensor this implies
I1 + M(w)) Gallz s 2 2||a|2-
(16)
Step 3: Lipschitz constant of F(,w). Using definition equation 6
F(z,w) Fly,w) (L+M(w)) @ (zz 2y).
and therefore, by equation 16 and equation 15
[[ Fly w) ll S 2 2||² - zyyl2 S 2 21 y/12-
Because the bound holds for every choice of x, y, the mapping F(:ww) is 2-Lisschitz.
A.2.3 RANDOM-PROJECTTO FILTERS
Random- Random-prooection filters (RPF) replace subset of convolution kernels with i.i.d. Gaussian weights. Let æ E # xC, be an input, F & Cau the flattened kernel matrix and = F the pro- jected feature. When the number of random columns Cout N, satisfies the Sahnson-Lindenstran lower bound,
(1-=) 1|, - 1, < || - 11 <
(1 -
(17)
local geometry is provably preserved (Dong & Xu 2023). A standard way keep the mapping I-Lipschitz is to rescale the frozen kernel with a enectral-norm estimate obtained by a few power- iteration (PI) steps after each forward pass.
A.3 PROOFS
By Propositions and SNCAN and RPFAN are < 2-Lipschitz. By Proposi- tion FDPAN' gated output is also 2- Lischitt... Finally, Proposition 4 shows the pernchannel convex fusion has Lip(2) < maNte Lip(G,) <
Proof of Theorem 7
Proposition 4 (Convex fusion retains the max-Lipschitz). Given channel-wise convex fusion 2(-) (see Eq. 22 that satisfies Lipschitz(z) < max, Lipschitz(Gu) I 2, if every stream output is < 2-Lipschitz, then the HyCAS block is also 2-Lipschitz
16

---

Published as a conference paper at ICLR 2026
Proof of Proposition
Fix channel and any 2, y. Triangle inequality gives
| xy)..elll = Σ ab.c (G..)(GG - Go(v)...)) - Σ ab. [[Gb(z)) - Gb(ll)l.
Since [[Gb(a) Go(y)]l
Tall yll and Σs Otb,c. 1, we have [|=(x) =(y)..eel
||
(max; [n]l - yll- Taking the maximum over channels yields Lip(z) & max; Lb. Thus Lip(z) 2.
Lemma (Expectation preserves Lipschitz constant), x s(x, 92) is L-Lipschitz for all Q. then the expected logits (x)) E [x) are L-Lipschitz (By Jensen's inequality and linearity of
expectation (Dong & Xu 2023 Hence, HyCAS' expected classifier inherits the same constant.
Proof of Lemma 2 For any x,
[z((x) [1) (s w) s(y w)ll < Bulls(s,, s(y,w)ll s B. ulll T. vll,
using Jensen's 's inequality ||EX|]] S X|].
Proof of Corollary
7
By Lemma
Z is 2-Lipschitz. Hence for any coordinate
(ax)
Zz(z + 8) (z)
<
[z 8) Z(x) lcc S
[[ + 8) Z(x)ll2 s
2||8||2
2
Thus the top logit can decrease by at most A/2 and the runner-up can increase by at most A/2; their order cannot swap.
Proof of Proposition L By the triangle inequality and the chain rule for Lipschitz maps,
[[GFDPAN(z:w) (FDPAN():W)]
[|H(x)) H(y) ((H()):w)) R(H(y)::))
$ [H(xx) - - ()))))):w) - (R(H():(H)))
Tip(H) ||= vl + (R) [H H1y)l
$ 1- || 1oc--yle +1 yll = 2|| y|l-
Thus Lip(GFDPAN) 2.
Proof of Proposition Let y HxWWCC and set 21 CK. (I) and 22 CK. (v). Using equa- tion 5 and the sub-muliplicativity of operator norms,
[[GSNCa[[;; Mc) GSNCAN(y: Mc)l |1(1 + Du)(z1 22)112
|| + Dulla |||| 22/12
||C,, (z) -.. (1)lla
$ || ull2;
which establishes the claim.
Proposition 5 (RPFAN is 2- 2-Lipschitz). Let U be an orthogonal x 1 channel mixer (||U 1). Let WsN be a spectrally normalized random-projection filter so that the linear map HRPFAN(z) = Conv(Ux; WsN) satisfies Lip( HRPFAN) I 1. Let D., diag(M,) with M,, E (0, 1]d and define
GREFAN (1; Md) HRPFAN(1:) D., HRPPAN(()) (T+ ( D HurPAN())..
Then Lip(GRPEAN) 2
Proof of Proposition
[G(xx) G(y)l [[[I + (HRPPAAN()) (RPPAN(U))) $ || + Dalll la Lip( (RPFAA) x yll 2 |x- y||.
n
17

---

Published as a conference paper at ICLR 2026
Proof of Lemmal 7
Fix any realisation of the noise w and set D Eq. equation the RANI transformation satisfies
diag(Mo) Raxd By
R(h; Mw) = Duh (1+ (1+
Step 1: bound the operator norm of + Dw- Because every coordinate of M.. lies in [0, 1], each diagonal entry of Dw in the same interval. Hence all singular values of D., are and
[ + Pulla
<
|| ||Dlll 1+1
Step translate the norm bound into a Lipschitz constant. For arbitrary hi, h2 IRd.
[[(h; Mw) - R(h2; Mw)lla = [[ + Dw)(h) - h2)
- | + D ||A, h2ll2
- 2 l h2ll2.
Therefore R(:; Ma) is 2-iischiit with respect to the Euclidean norm for every draw of w, complet- ing the proof.
Lemma (Orthogonal transforms are I-Lipschitz). U Rdxxd orthonormal then Lip(U) = 1. In particular, 2-D DCT/IDCT and any frozen orthogonal 1 1x1 convolution satisfy Lip 1.
For map h: dRd + IRd the l Lipshiict constant is
[lh(u) h(v)ll2
Tip(h)
sup usn
|/u 1//2
Throughout we use the vectorised &g norm over N x H x W x C' tensors. We make repeated use of: Triangle inequality. I|a + 6ll2 S ||al|a [16]|.. Conver combination bound. E, A 1 and
X then ΣΣ Aaas S E, AsIla; l12- Jensen. || [[ lla S [|]]
Proof of Lemma
|Ux Unll |U(x v)ll
| vll for all x. y.
Lemma 4 (Spectral normalisation). Rescaling convolutional kernel W by W/ max[[[ enforces Lip( (Cnnww) < (Gouk et al. 2021
Proof of Lemma
By construction,
||Cxllop
max C lopps o(K) ma)(o(K), 1}
[[C,, llop (a)(o( 1}
1 1.
A.4 CERTIFIED PREDICTION UNDER HYCAS
Each HyCAS stream comprises a 1-Lipschitz deterministic core followed by a data-indeppndent RANI module. Conditioning on the internal noise w, each stream is therefore 2-Lipschitz (see Lemma 3). and the composite core remains 2 2-Lipschitz (ref. Lemma I Specifically. the FDPAN stream the only exception: it contains two residual blocks (SNCAN + RANI), giving naive 4 Lipschitt upper bound. We tighten this to < 2-Lipschitz by scaling the skip connection (Propo- sition [. A convex channel gate then fuses the streams without increasing the Lipschitz constant (Proposition 4). Finally, stacking modules and applying a global calibrator with gain c < 2/Lnet ensures the entire network is at most 2 Lipschitz.
18

---

Published as a conference paper at ICLR 2026
Table 5: Scope representative certified, empirical, and hybrid defences. indicates the property explicitly addressed, or the domain is reported, in the original paper.
Method
Certified Empirical
Natural images
Medical images
RS
Cohen al.
2019
IRS
DRS
Xia al.
2024
ARS
Lyu
2024
LOT
Xu
2022
SLL
Araujo
2023
PNI
(ee et al. 2019
LearurPerturb (Jedd et al.
2020
CTRW (Ma et al. 2023
RPF DDng & Xu
2023
CAP Xiang of al.
2023
HyCAS (ours)
Lipschitz- Contrained
(x=E)
Capeax M.)
Inngo-ii
R(H(r: E): M..)
Orthogonal Jacobian Matrix
DCTIDCT
2-I.ipechitzz
I-Lipachitz
Input
I-Lipschittz
2-Lipschite
FDPAN
Figure 5: Overview of FDPAN stream. A four-stage cascade: (i) low-pass DCT masking and orthogonal 1x1 channel mix (both I-Lipschitz); (ii) SNCAN block (spectrally normalized convolution) with RANT; (iii) additional RANI; (iv) skip/gating. The stream remains <--Lischhit.
Lipschitz- Constrained
GaNcau(a; M.)
(x.(
Imago-Nel
SNC
RANI
1-Lipschitz
Imputx
I-Lipschitzz
2-1 2-Lipschitz
SNCAN
Figure 6: Overview of SNCAN block. A spectrally normalized convolution (CK.) ensures operator norm S 1; RANI applies bounded, data-independent attention mask M., so the block output equals (7 Dw.) Ck. (x), which is < 2-Lisschitz.
Margin certificate. ness, be
Define the expected logits, averaged only over the model's internal random-
Z(x) = [[[s(r;w]] Zip < (Lemma
Let (2 Z(1(2)) - (1r) denote the gap between the top two logits. The certified & radius
is then,
(2 Az
19

---

Published as a conference paper at ICLR 2026
which guarantees arg max (z + 5) arg max (x) for every perturbation ||ö||2 (Corollary 1). For fox, the norm inequality ||ō||2 s Vd]l yields the conservative certificate Too (x)
r2(2)
vd
Estimation under internal randomness. At test time we approximate through Monte Carlo sampling over w. Draw Ro pilot samples to identify the top class, then take additional samples; compute one-sided confidence bounds for (1) (a) and (2)) (a) and certify with
(L) (cB(2()(()) (CB(2(2(z))
4
confidence 1 a.
A.5
EXTENDED DETAILS FOR FDPAN, SNCAN, AND RPFAN STREAMS
Detailed overviews of the three parallel streams-FDPAN, SNCAN, and RPFAN-are illustrated in Figs. respectively.
A.6 EXTENDED DETAILS FOR RANI MODULE
Deterministic attentions. We design two deterministic attentions-lose (LA) and channel (CA) that highlight informative local and inter-channel dependencies, respectively. Specifically, we leverage GAP and 1$1 convolution followed by leveraging dense layer and sigmoid to learn these attention maps:
LA(=) (CCooo(aa())) CA(z) of DDnnee ( (ReLU (GAP(()))
(18)
Injecting stochasticity. We these deterministic attention maps into randomized attention maps (7g and 7/) via injecting feature layer noises T ?h E IR,, thereby incorporating stochasticity, We formulate this as: where g. di RC are trainable scale vectors and v (u) min(1, max(0, u}} clips the maps into [0, 1].
% M7, + + ? = "j +I', ","
(19)
Noise parameterization and iterative refinement. To realise heteroscedastic, yet data- independent, stochasticity at minimal cost employ two-step seff-modulation loop
T/1 T/ ① (o. 1/4 O.)..))
E {gg
This yields two potential benefits: (a) Richer expressivity because a. trainable, the model learss which channels benefit from strong noise and which should stay nearly deterministic; and (b) Negligible verhead-ont 2C extra scalars per branch.
Iterative noise fusion. We propagate the stochastic smoothing through the backbone in four stages. At each stage j E (1,...,44 4} the current feature tensor o' is modulated by the noisy randomized the deterministic attention maps, (n'g) and (>1) followed by fuse them as
IU = 2 o II (16)), o (116),,. f=1
(20)
Each stage injects a freshly resampled attention noise yielding a progressively smoother-stochastic feature tensor. This cascade progressively smooths the feature tensor and presents a continually shifting optimisation landscape to an adversary, thereby enhancing robustness while preserving the global 2-Lipschitz guarantee. Thus RANI converts every deterministic 1-Lipschitz block into randomised counterpart that keeps the certified l margin while impeding adversarial attacks by presenting moving optimisation landscape.
20

---

Published as a conference paper at ICLR 2026
A.7
DETAILS ABOUT THE HYCAS CERTIFIED ALGORITHM
This section illustrates the details of the certified algorithm of HyCAS, as shown in the following subsections:
A.7.1 HYCAS TRAINING
Our base network couples deterministic, Lipxchitz-comstraineed backhone with stochastic smooth- ing branches. Concretely, let f denote the bybrid classifier with parameters 0 and internal randomness $2 (1b, My where v parametrizes implicit randomness (c.g., random projection fil- ters) and My injects explicit attention noise. Following randomized smoothing (RS), we expose the input to isotropic Gaussian noise
N(0,0²1) during training and minimize the expected loss
min ((x,y) Be-N(0,021), ((So( + +; v),
6
which the same objective used to train the stochastic component in our hybrid architecture (cf. Eq. 10 for RANI in HyCAS). To make RS effective at scale while retaining deterministic control, the backbone constrained be T S 2-Lipschitz; our implementation mirrors the HyCAS con- struction where residual blocks are scaled so the stacked network remains < 2-L.ipschizz and thus amenable to margin certification.
To mitigate the curse of dimensionality inherent to RS, we optionally activate DRS (Dual Random- ized Smoothing) path that partitions æ into two lower-dimensional sub-inputs and smooths them separately before fusion. This preserves most information while tightening the Eg certificate upper bound from (L)/vd) to O(L/ 1/vn) with m d. Our training simply shares the same and minimizes the same expectation, with the forward pass executing the two DRS branches in parallel.
Algorithm D summarizes one epoch: for each minibatch we (i) sample (s, Q) once per forward; (ii) run the deterministic Lipschitz backbone and the stochastic streams; (iii) backpropagate the Monte-Carke estimate of the RS objective; and (iv) apply the Lipschitz constraints (spectral normal- ization calibrated residual scaling) keep T < 2.
Algorithm HYCAS Training
Requires: Dataset D; epochs E; batch size B; noise level o; HyCAS CyyCAS-integrated network fo with streams {SNCAN, RPFAN, FDPAN) convex channel gate ab.cc C,, 0lb,c 1), and I-Lipschitz building blocks; optimizer O; (optional) stream loss weights 4, v, K.
Init: Initialize 0; set spectral normalisation (SN) for all convs (operator norm < 1); for e to
do
foreach minibatch [(r D do
// Resample internal randomness once per minibatch (HyCAS executi protocol)
Resample random-projection filters for RPFAN and attention-noise masks for all streams, collect as Ω. // RS AtS-style training noise at the input
for i B
Draw & ~ (0 o² I2)); set ã; Fé.
// Forward through the three streams + convex fusion (each stream < 22-iipschitz)
Compute per-stream feature maps Go(ā,( for b (SNCAN, OPFAN, Fuse per channel: z(Ø,) 2c + E, abe Go(ia 2c. // Loss: single fused CE, or the dycAS-weighte t k L(z(ia), + [[(GEDAAN W1) + (G + v [(GrptaN Wi). Up- date + E, // Keep layer-wise operator norms < (SN) to ma maintain global < 2-Iiisccitz envelope Re-apply SN to all conv kernels.
multi-branch objective
Final global calibrator (gain) to cap the network Lipschitz constant by 2 Estimate Last (product blocklock bounds); seale last linear by 7 t min(1, Lus).
21

---

Published as a conference paper at ICLR 2026
Algorithm 2 HyCAS Inference and Certification
Input: Trained HyCAS-integrated classifier fe (globally < 2-Lipschitz); test point æ; class set of size K; Gaussian noise level σ; pilot ro and budget n for RS; significance a (set QRs = OLip a/2).
Output: Certified label and radius R > O, or ABSTAIN.
10 (A) RS branch (standard randomized smoothing). pilot then CI
// Cohen-style certificate;
for i 1 to no do
12 L Draw E ~~ of) and internal randomness Ωi; q + folx + E;
ia Let ĆA < arg mY count (c). for m do L Draw ~~ ~ o² I) and Q2 C fo(x =;
ans). if if I then Let count, (cx) and PL < CLOPPERPAASSOLOWEERRRR set RRS +- 0 else RRS 4 (p Ōns ČΛ. // Φ-1 is the standard normal inverse CDF // HyCAS margin
(B) Lipschitz-margin branch (deterministic certificate). certificate 18 Freeze internal randomness O (fix seeds), and compute logits s(;52). arg max.ey sc(=;1"); s(1) +- maxe se (2; S2"); s(2) t maxe#tup (x(x; if (1) < s()) then set RLip +0
ULip
(11) (2)
19 else RLip
4
// Since Lip(Jo) 2, radius is (margin)/(2 Lip) 20 (C) Pick the stronger valid certificate. if N'max Rns, Riip) = 0 then return ABSTAIN else if Ris= 2 RLip then return (jns, Rns) 22 else return (ULip: RIip)
A.7.2 HYCAS INFERENCE-TIME CERTIFICATION
At test time we provide two independent certificates, both for the exact network we evaluate:
1. RS certificate. We certify the smoothed classifier
Jo (x) A
arg max IP (e = c]. cE)
We follow the standard two-stage Monte-Carlo protocol: draw no samples to select the can- didate class and then n samples to bound its probability. Let PA and pa be the empirical proportions of the top and runner-up classes. Using exact Clopper-Pearson intervals obtain a (1-a) a) lower bound I.B on the top class and an upper bound pB on the second. If pf abstain; otherwise the certified radius is
L.B S
RS (x)
1 x (v- + -
2
where Φ-1 is the standard normal quantile. This is the tight specialized and re-derived in ARS (via f-DP). In experiments mirror DRS sampling defaults (no = 100, n 10 a 10-3). When the DRS path is enabled, class probabilities are estimated branch-wise and fused in DRS before applying the same formula.
Kolter bound
2. Deterministic Lipschitz (margin) certificate. Independently of input noise, we certify the backbone internal randomness by averaging logits only over Ω:
(xx) A Eg [=(x;; }) with Lip 7) 2
Let (Z) - Zog (x) be the gap between the top-two expected logits. Then for every perturbation ||ó]/2 AZ (zx/44 /4, , the arg max Z(-) invariant; i.c. the model's prediction is
certifiably robust within radius
AZ
Lip
Lip 2 (x) =
Li)(())
Vd
22

---

Published as a conference paper at ICLR 2026
We estimate Z via Monte-Carlo over 52 (no input noise), exactly as recommended in HyCAS.
Algorithm implements both procedures. In reporting, we return two radii (r2 (x), r (x)) for the classifier,
same input z. Both are valid and interpretable: the first certifies the LS/DRS-smoothe the second certifies the Lipschitz hybrid backbone averaged over internal noise. This mirrors the practice in ARS/ RS (majority-vote certificate) and HyCAS (margin certificate) while respecting their assumptions.
A.8 EXTENDED EVALUATION SETUP
Network Execution. At the start of every mini-batch we resample, for each forward pass, (i) the attention-noise Ma. for {FDPAN, SNCAN, RPFAN) and (ii) the random projection filters for RPFAN. These samples stay fixed while adversarial examples are generated. At inference stage, for each test image, draw fresh (v) w),, and evaluate HyCAS against adversarial attacks ensure adversarial robustness.
Implementation details. Following Cohen al. 2019 Lyu al. 2024 Xia et al. 2024).
use ResNet-110 (He et al. (016) on CTFAR- 0/1 100 and ResNet 50 on remaining imaging datasets (e.g., ImageNet (Deng al. 2009). Ce ebA (Liu et al., 2015), NCT-CRC-HE-1 00K (Kather et al. 2018) etc.), as base classifiers for all training strategies. We report the best perfor- mance separately for more comprehensive and fair comparison. We evaluate on CIFAR-10/101 MmageNet-1k. CelebA (unaligned, cropped attribute), NCT-CRC-HE-100K NIH CXR, EyePACS, and HAM10000.
For Certified Defense. HyCAS certifies via a margin bound under an at-most 2-Lipschitz network. Let (xx) = [=()] denote the classifier averaged over the model's internal randomness; since Lip(Z) < 2, a pointwise certificate is r(z) =
Ag(a) 4 (2 = (1r) ( - (z)) To estimate
(xx we Monte Carlo sample only the model's internal noise at inference. Unless noted otherwise,
take pilot of no 100 samples to select the top class, then draw
100,000 additional LCB LCB( (z)())) -UCH( (z,)(z))
samples to form one-sided confidence bounds and report (L) confidence 1 - a with a = 0.001. During inference we draw a fresh w on each forward pass. To control runtime for Monte Carlo estimation, we fixed rule per dataset:
4
CIFAR-10: certify every 5th test image (default settings 100, ==100,000, a=0.001).
ImageNet-1k: certify every 100th test image (default settings =o=100, n 1=100,000 0=000).).
CelebA (ARS-style): certify a uniform, label-stratffied subset of 200 test images using no 100, 5=50,000, and failure probability 0.05 (i.e., 95% confidence).
NCT-CRC-HE-DI 100K, NIH ChestX-ray 14, EyePACS: certify uniform, label-stratified sub-
sample per dataset sized to yield ==2,000 certified examples each (default settings no=100, n=100,000, a=0.001; exact counts in the appendix).
HAM1 certify the full test split when feasible; otherwise uniform, label-stratified subsample (default settings 1000 10100,,00, a=0001;; exact count in the appendix).
We sweep a E (0.25, 0.50, 1.0) for comparability across settings.
For Empirical Defense. We follow the protocol of SOTA adversarial training strategy (Rice et al., 2020) to set up our experiments on our diverse datasets. For Adversarial Evaluation Sactattoon--C-SS
tested under white-box attacks (Madry el al.) 2018), APGD (Croce & Hein 2020 and
Aut oAk (AA) (Croe & Hein 2020) using E 255
16
}. step size a 255 and 10-100
255
iterations.
Training Details for Certified Robustness. Following ARS, we use a single recipe per dataset and train all HyCAS-integrated backbones. Inputs are perturbed during training only with i.i.d. Gaussian noise N (100, o²) (the same a as at certification). For CIFAR-10, we train for 200 epochs with batch size of 256 using Adamk as the optimizer with learning rate 10-2 and weight decay 10-4 A step scheduler is used with step size 30 and decay factor y = 0.1. For CelebA, NCT-CRC-HE-10WT epochs with a batch size of 64 using SGD as the optimizer with learning rate 5 x 10 2, A step sched- uler is used with step size 3 and decay factor 7 0.8. For TmageNet-11 k, we train for 200 epochs
NIH CXR, EyePACS, and HAM10000, we train for 200
23

---

Published as a conference paper at ICLR 2026
1-Lipschitz
Incobias Onhogoma
RPFAN
Random Projection
Spectral Normalized
Random Projection
1d (M)peAN((z)))
Inputx
RANI
Bonchessanne Spectral Normaliztion
1-1 ipschitz
2-Lipschitz
Output
HRPPAN (z;
Figure 7: Overview of RPFAN stream. (i) Orthogonal Ixl pre-mix (1-Lipschitz). (ii) Batch-aware spectral
normalization of madom-projection convolution (1-Lipschitz corc). (iii) RANI residual, yielding S 2- Lipschitz stochastic block.
Table 6: Robust accuracy (%) against too attacks (PGD-20 and AA-20) NIH-CXR (left) and
NCT OCT-CRC-HER C-C-CRCCHE---- HOC-HE--OOK (right) (8/255, 16/255).
NAR-CAR
HCT DCT-CRC---- BECC--E-1000 100R
Clean
PGD-20
AA-20
Clean
PGD-20
AA-20I
16/255
16/255
89.1 1.91
78.3 2.82
3.62
2.93
64.1
92.2 1.82
80:4
70.8 3.52
7633 2.83
06.2
3.58
CTRW
88.4 1.73
86.7442.41
74.4 3.222
2.44
72.6+3344
90.4 1.62
89.6+
37152
86.7 2.44
75.2 3.22
DCS
2..2
83.3
72.3
2.77
3.46
90.3 1.93
8:.5
75.2 3.35
83.3 2.7.
71.66 3.46
2.22
72.8
62.8
78.9 7:99
74.6
64.5
DRS
6.78
7.16
3.81
3.88
4.3.1
HyCAS
89.5 1.64
88.6
2.33
77.3
3.14
86.9
2..92.2..42
2.42
74.4
3.33
91.3
2.63
90.4 2.82
79.3
3.52
88.2
2.63
76.7 3.34
(10+90 warm-up and main training), with batch size 300 using SGD as the optimizer with learning rate 10-1, momentum 0.9, and weight decay 10-4 A step scheduler is used with step size 30 and decay factor ? 0.1. HyCAS injects internal spatial and channel attention noise on each forward pass; convolutions are regularized with spectral scaling (via FFT or power iteration) combined with GroupSort activations and convex residual gating, ensuring the network remains at most 2-Lipschitz. We optimize using categorical cross-entropy loss and report top-1 accuracy.
Adversarial Training with HyCAS for Empirical Robustness. Let fa Id R be the HyCAS- integrated base classifier with parameters 0, mapping input z to its logits fa(x). For given clean sample (r. y) and perturbation budget c, an adversarial example x" is obtained by maximizing the loss inside the c-ball around
arg max
Coycas(Jo<>"; ((A)).)),
(21)
-l-l=--- =211se
where Laycas is the task loss and 2[A] emphasizes that gradients are taken in the attack phase.
Adversarial training then solves the following classical min-max problem:
min I
max
Cnycas(ffoz'" ((A))
(22)
where I[] denotes the parameters updated during the inference phase.
As detailed Algorithm 3 combining this min-max optimization with all integrated streams en- ables HyCAS maintain strong adversarial resilience at inference time.
24

---

Published as a conference paper at ICLR 2026
Table 7: Robust accuracy (%) against too attacks (PGD-20 AA-20) on HAMI (left) and EyeFACS (right) at e E (8/255, 16/255}.
Method
Clean
PGD-2U
AA-20
Clein
PGD-20
AA-20
8/255
16/255
8/255
16/255
8/256
16/255
5/255
16/255
AT
70.2 2:94
18.3+-3.88 4.8.1
3.74
64.2 3.93
44.2 3.80
78.2 2.91
62.4 2.722
52.8 12.8-t10 3.52
68.3 2.80
48.2
3.61
RPF
2.88
64.6
3.62
62.6
50.4 3.58
77:12:2000
68.3 2.61
37.6
66.41:2277
64.4
3.44
2.70
6..7433.22
3..24:3.4
63.3
11.2
76.4
70.1 2:53
3..5
09.7
56.1
DCS
73.2
63.54
3:.30
61.4
00.5+33.45
7..4
67.5 2.62
36.24
65.3
53.6 3.46
ARS
3..2
56.2 3.95
46.5 3.62
12.74:1..
42.8
72.9
2.991
50.3
1.3.::33.44 3.61
57.6
46.5
3.73
15.3
3.84
3.75
51.6
3.99
0665
45.7
HyCAS
74.6 2.74
67.8
3.43
55.3
65.8
53.1
77.6 2.79
72.6
00.5 3:43
71.8
2.88
58.3
3.32
Table 8: RS/DRS vs HyCAS certified accuracy on EyePacs NCI-CRC-HE-100K benchmarks. The best performance under each training strategy is bold.
LyePace
NCT-CRC-HE- TOOK
Approach
radius
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.25
81.3 1: 1,92 60.1 à: 2.83
50.1
1.5
10.9
2.04
26.2
2.89
89.5 1.
2.78
67.6
2.63
56.7
3,35
45.3
30.5
3.54
0.5
78.9
0.91 57.7 + 2.82
51.8
1.34
42.1
0.96
30.4
3.74
83.2
1.67
65.6
1.89
18.2
- 1,23
48.2
1.67
33.1
1,20
ARS
0.25
83.1 + 1.35 62.9 2.33
47.9
1.94
40.74.32
06.2
3.73
+1.7+184 1.84
01.7
69.3
2.07
10.4
2.37
48.3
0..3+-9.11
31.9
HyCAS
0.25
86.7
-0.9
66.1 11.6
11.4
45.7 11.27
39.2 1261
95.4
12002
72.9
1.63
63.1
11.59
S1.7 :3.11
332 11.44
82.6
1.89
63.9 +1.74
532+2881
463+144
41.5++167
92.3 +0.11
71.7
+1.11
63.4+1.36
52.2+1.
+2.57
Algorithm 3: Adversarial Training with HyCAS
1: Require: HyCAS integrated base classifier {a(-) with learning parameter 0; Perturbation size
Attack step size a; Number of attack iterations k; Training set {æ.y): Generated attention noise Ma. by RANT module.
2: Procedure:
3: while not converged do
4: Sample batch (bu,, by) from ya
5: Apply HyCAS for Attack phase:
2(2).... Σas, as. [Go(z; R Σ B
as.. [Go(z; [
R.n Ma) c= L...,,
BEB
6: Compute the MACCAS-integran network is optimised with a standard Lo loss:
LHyCAS = S LEDPAN LSNCAN O LRPFAN + CRANT
min max
s.t. st.l|= - - Se
Se
7: Generate Adversarial Examples:
8: Randomly initialize adversarial perturbation 5;
9: for i 1 to A do
8 ++++ a sign (V (t by)) bo" + Clipi.. (be +
10: end for
11: Apply HyCAS for Inference phase:
2(x)... = Σ
as.e [Go( + R Σ as.e [Go( R)... Mo) c= I....,C
688
min max (Ceycas(to(x" v)) s.t. x Se 6/Z
12:
Adversarial Training Optimization:
= 0 -Vo (o (1))))))))))
13: end while
25

---

Published as a conference paper at ICLR 2026
Table 9: Computational cost comparison between vanilla backbones and their HyCAS-integrated counterparts. We report the number of parameters (M), FLOPs (G), activation memory (MB), and inference time (ms) for ResNet-110 on CIFAR-10 (32 x 32 3) and ResNet-50 on ImageNet- (224 x 224 x 3).
Backbone
Variant
Inputs
Parameters (M)
FL.PPs (G)
Memory (MB)
Inference Time (ms)
Vanilla HyCAS
27.6
8.10
105.33
0.33
ResNet-110
32 32 x
57.8
126.5
220.5
5.14
Vanilla
23.3
113.8
88.89
4.62
5e
224 224 3
HyCAS
102.7
1682.5
391.9
68.3
A.9 ADDITIONAL EXPERIMENTAL RESULTS
Empirical robustness on Chest Xray, Histopathology, Dermoscopy, Funduscopy modalities. Across our empirical evaluations (Tables 6 HyCAS achieves the highest robust top-1 accu- racy under PGD-20 and AA-20 at e E 16] /255. Specifically, on the NIH- CXR benchmark, HyCAS retains robust accuracy, outperforming the leading baseline (CTRW) by up to +1.0-2.2% while competitive clean-set accuracy (89.5% vs. 89.1%) vs. AT. A similar trend appears on the NCT-CRC-HE- 100K dataset, where HyCAS records robust accuracies of 76.7-79.3% against the same attacks, edging past CTRR by = +1% and leaving earlier certified defences ARS, DRS) more than +7% behind at the larger perturbation strength. Dermoscopic HAM1 0000 and fundus- image EyePACS exhibit the same hierarchy: HyCAS secures robust accuracies of 53.1-67.89 against PGD-20 and AA-20 attacks on HAM10000 rougury +1.1-7.4% better than the next-best adversarial defence-and and widens the margin on EyePACS to 58.3-72.6% %, thereby surpassing the leading baseline CTRW by +2.0-3.3%. Together, these results show that HyCAS transfers its ran- domized Lipschitz- strategy from certified to empirical settings, preserving clean accuracy while preserving adversarial robustness against strong first-order atlacks.
Evaluation under stronger PGD attacks on other vision benchmarks (Figs. reveals that HyCAS not only wins at standard PGD-20 settings (App. A.8) but also sustains its lead as the adversary grows stronger. When the perturbation strength is swept from 0.01 0.08 on CIFAR-100 HyCAS traces the upper envelope of robust-accuracy curves, preserving a A 10% gap at the maxi- perturbation strength, where all baselines collapse sharply. An analogous pattern emerges on CIFAR-100 as the number PGD iterations climbs from 10 100: while every defense degrades monotonically, HyCAS declines more gracefully and ends 7-12% above the closest competitor 100 steps, confirming that its internally resampled attention noise and random projections thwart extended optimization. Thus, this randomized, Lipschit-coonstrained design scales gracefully with both perturbation size and steps, offering ad adversarial robustness and broader safety margin.
Certified adversarial robustness on EyePacs and NCT-CRC-HE-100K. Across the complete set of baselines Table X HyCAS delivers the strongest certified accuracy for every inspected radius-noise pair. On the EyePacs benchmark, at the representative medium radius r=0.75 it reaches 45.7% % certified accuracy for a 0.25 and 46.3% for a 0.50, outpacing the best com- peting method (DRS/ / ARS) by 4.1-4.8% Even in the large-radius tail (r=1.0), HyCAS maintains 39.2% (a = 0.25) and 41.5% (a = 0.50), widening the gap over the strongest baseline by up to 4.0%.
A comparable pattern emerges on the NCT-CRC-HE-1COK histopathology dataset. At r-0.75, HyCAS secures 51.7% % (o 0.25) and 52.2% 2% (a 0.50), improving on the best baseline by 1.5-3.4%. In the challenging 1=1.0 regime still records 33.2% (a = 0.25) and 36.9% % (a = 0.50), extending the lead to much as 2.7%.
26

---

Published as a conference paper at ICLR 2026
Besides robustness, HyCAS achieves the highest clean aceuracy both datasets 86.7% Bye- Pacs and 95.4% % on NCT-CRC-HE-100 for o 0.25 underscoring that its certified gains do not come the exdense of nominal Derformance.
CTRI
HYGAS
60
Accursty
64
68-
20
0.52
0.03
0.04
2.25
10
30
40
50
90
90
PCD Stzes
CIEAR-IO PGD Sizes
CIFAR-1O PGD
Figure 8: Empirical robustness of HyCAS versus leading baselines (RPF, CTRW, DRS, ARS) on CIFAR-18 1C under strong PGD attacks. We evaluate two settings: (1) perturbation sizes from 0.01 to 0.08 and (2) iteration steps from 10 to 100.
60
30
25
28
15
35
0.52
0.03
0.04
0.05
0.06
80
FCD
Stzes
PGD 5haps
CIFAR-100
CIFAR-100 Steps
Figure 9: Empirical robustness of HyCAS leading baselines (RPF, CTRN, DRS, ARS) CIFAR-101 under strong PGD attacks. We evaluate two settings: (1) perturbation sizes from 0.01 to and (2) iteration steps from 10 to 100.
Computational cost analysis. Table quantifies the computational overhead of integrating Hy- CAS into standard CNN backbones. On CIFAR-10 with ResNet-110, replacing the vanilla back- bone with its HyCAS counterpart roughly doubles the parameter count and activation memory (27.6M + 57.8M and 105.33MИ + 220.5MB, i.e., x 2.1x in both cases). In contrast, FLOPs and inference time increase by about an order of magnitude: 8.1 - 126.5G FLOPs and 0.33ms + 5. 14ms i.c., se 15.6x. A similar pattern is observed on ImageNet-IK with ResNet- 50, where HyCAS induces a 4.4x increase parameters and memory (23.3M - 102.7M 7M and 88.89MB + 391 391.9MB), while FLOPs and inference time grow by ~ 14.8x (113.8G - 1682.5G and 4.62ms + 68.3ms).
To better localize this overhead, we also compare HyCAS with single intermediate 3 x 3 convo- lutional block of 256 channels in ResNet-110. Substituting this standard convolution with Hy- CAS block increases the number of parameters by 5.09x, whereas the corresponding FLOPs rise by 14.05x, indicating that most of the extra cost stems from repeated stochastic operations rather than weight storage. These block-level ratios consistent with the backbone-leved trends above, where HyCAS trades only z 2-4x more parameters and memory for roughly a z 15x increase in arithmetic and runtime.
Overall, HyCAS is best characterized as parameter-most moderat but compute-hear defense: the three Lipschitz-coontrrinnet stochastic streams (FDPAN, SNCAN, and RPFAN) together with the
27

---

Published as a conference paper at ICLR 2026
RANI module primarily inflate FLOPs and inference time, while absolute inference times remain in a practical range (a few milliseconds on CIFAR-10 and tens of milliseconds on ImageNet per image A100-class GPU). In return, HyCAS consistently delivers state-of-the-ar certified and empirical robustness across CIFAR-10 10, ImageNet, and the medical-imaging benchmarks reported in making this overhead acceptable for many offline real-time deployment
T Band 6 scenarios.
A.10 ABLATION STUDY
Table 10 traces a controlled progression
Table 10: Module-wiss contribution for cer- tified and empirical adversarial robustness on CIFAR-10 benchmark. Here, we have used a 0.50.
from the regularized--moottinng (RS) base- line to the full HyCAS model, revealing how each block incrementally strengthens robust- ness. Replacing ordinary convolutions with the enpetrally-normaaizee SNCAN backbone al- ready raises certified accuracy at the medium radius (r=0.75) from 32.4% to 36.9% and improves PGD-20 robustness by 3.7%, indi- cating that spectral control alone substantially smooths the gradient landscape. When the or- thogonal RPFAN branch is introduced next, cer- tified and empirical accuracies climb further to 40.2% and 64.8%, respectively, showing that de- correlated projections supply complementary features beyond spectral stabilization. Extending the spectrum through FDPAN yields another gain-42.3% certified and 66.7% ampprical-coofirmme that high-frequener cues remain valuable even under lg certification. Finally, injecting data- independent attention noise via RANI closes the gap between certified and empirical metrics, cul- minating in 44.3% certified accuracy and 70.1% PGD-20 robustness, which exactly matches the performance of the complete HYCAS system. Altogether, these sequential additions deliver an ag- gregate improvement of +11.9% certified and +1266% empirical robustness over the RS baseline, underscoring that each module contributes a distinct yet additive benefit to adversarial defense.
Accuracy
Variant
Certified (r 0.75) PGD-20 (c 8/255)
32.4
+ +NCAAN
RPFAN
40.2
64.8
#DPANN
42.3
66.7
+RANI
44.3
70.1
A.11 DISCUSSION
HyCAS is designed to reduce the gap between provable & robustness and empirical foc robust- ness by combining globally Lipschitz design with carefully structured internal stochasticity. This section focuses on why this design yields strong certified guarantees and how it simultaneously improves empirical robustness across natural and medical imaging benchmarks
Certified robustness from randomized Lipschitz network. by HyCAS margin-based: if
The certified guarantee provided
(xx) (1(s0(1; 1)),
AZ(x)=
Zay(2) (x2)
denote respectively the logits averaged over the internal randomness and their top-two gap, and if Lip(Z)= < 2, then
AZ
r2(x)
4
is valid pointwise & certificate (Corollary 1). This guarantee acts on the expected logits of globally < 2-Lipschitz network obtained by stacking HyCAS blocks, each of which combines 1-Lipschitz deterministic core (spectrally normalized convolutions, orthogonal channel mixing, low-pass DCT, and spectrally normalized random projections) with 2-Lipschitz attention-noise residual. The convex fusion of the three streams and the expectation over Ω preserve the global < 2-Lipschitz constant.
See Sections 3-4 and Appendix A for full details of the architecture, certification scheme, and experimental setup.
28

---

Published as a conference paper at ICLR 2026
This certificate is qualitatively different from the randomized smoothing (RS) radius of Cohen al. based on smoothed class probabilities and Gaussian concentration. It is not a relaxed or "looser" version of the RS bound: RS certifies the majority vote of a noise-perturbed classifier, whereas HyCAS certifies the margin of a Lipschhtz--consrraneed expected-logit map. Which radius is larger in practice therefore depends how training shapes the margin distribution under each mechanism, not on a direct comparison of constants in the formulas.
Empirically, HyCAS consistently achieves higher certified accuracy than RS, IRS, DRS, ARS and deterministic Lipschitz baselines (LOT, SLL) at all reported radii on (IFAR-10/100, ImageNet For example, on CIFAR-I0 at radius r 0.75 and E {0.25, 0.50), HyCAS attains 44.3% certified accuracy, which is 5.2-18.2 points above prior methods; at r 2.0 and o = 0.50, still retains 12.5%, exceeding the strongest baseline by 4.0- 12.5 points. Similar gains appear on ImageNet, CelebA, HAM10000, NIH-CXR, EyePACS and NCT-CRC-HE-100K These improvements cannot be explained by small fluctuations in clean aceu- racy alone, indicating that the architecture and training jointly enlarge AZ(x) for many points while respecting the conservative Lip(Z) S 2 envelope. For example, on CIFAR-10 10 with o = 0.25, RS attains 75.3% smoothed clean accuracy at 0 and 26.1% certified accuracy at 0.75, whereas HyCAS reaches 85.4% at r = 0 and 44.3% at r = 0.75 (Table [ Thus, the gain at a non-zero radius (+18.2 points at r 0.75) is substantially larger than the gain at 0 (+10.1 points), indicating that HyCAS's Lipschitz-constrainge inputs rather than merely improving accuracy on unperturbed data.
and the medical datasets (Tables ! 2 8).
hybrid architecture enlarges robust margins around
The ablation in Table 10 illustrates this mechanism: starting from an RS-style baseline, replacing standard convolutions by SNCAN, then adding RPFAN, FDPAN and finally RANI, monotonically increases certified accuracy at = 0.75 on CIFAR-I0 from 32.4% to 44.3%. Each variant uses the same backbone, noise level and objective; the only changes are architectural. This progression shows that the larger certified radii arise from reshaping the margin distribution under Lipschitz constraint, rather than from a fundamentally stronger analytical bound.
From theoretical perspective, the contribution is to extend margin-based l2 certification to net- works with internal stochasticity and to demonstrate that such networks can be trained at scale. The analysis proves that the expected logits of HyCAS network remain - 2-Lipschitz despite random projections and stochastic attention, and that this property can be enforced layerwise (via spec- tral normalization and calibrated residual scaling) in standard CNN backbones while still achieving competitive clean accuracy on large benchmarks.
Mechanisms underlying empirical &x robustness. The same ingredients that support the cer- tificate also improve robustness against strong 6x attacks such as APGD, PGD and Auto-Atack. Several aspects of the design are central:
Spectral control. SNCAN replaces standard convolutions by spectrally normalised ones, constraining the operator norm of each kernel and smoothing the loss landscape. This reduces the ability of first-order attacks to exploit sharp directions in the input space.
Random projections. RPFAN combines an orthogonal 1 x channel pre-mix with batch- aware spectral normalisation of random projection filters. This decorrelates channels and redistributes energy while preserving local geometry, making adversarial search less aligned with single vulnerable feature direction.
F awequency-aware filtering. FDPAN uses low-pass DCT masking and orthogonal mixing to suppress brittle high-frequency content where small foo perturbations can hide, without discarding all nigh-freyuency information that remains useful for classification.
Randomized Attention Noise Injection (RANI). RANI injects a bounded, data- independent attention mask after each Lipschitz core and at the fused output. For each fixed noise realization, the module is 2-Lipschitz, but across evaluations it presents a shift- ing, yet certifiably bounded, optimisation landscape.
Combined with adversarial training (Algorithm 3). these components yield strong empirical bustness. Across all four medical benchmarks, HyCAS attains the highest robust accuracy under APGD-20 and AA-20 at E (8/255, 16/255) while maintaining clean accuracy that is par with
29

---

Published as a conference paper at ICLR 2026
or slightly better than existing adversarially trained and randomized baselines (Tables 4 6
On 10/100 HyCAS dominates the abust-accuracy" curves across perturbation sizes and attack steps (Figures 2
g g increases from 0.01 to 0.08 the number of PGD/APGD iterations grows from 10 to TOO, all baselines degrade sharply, whereas HyCAS declines more gradually and preserves 7-12 point margin at the strongest settings.
B
Importantly, during attack generation the internal noise is held fixed per adversarial example, so gradients remain well-defined; stochasticity is only exploited across examples, not within the op- timisation path. Together with the global Lipschitz control, this suggests that the improved Lx robustness comes from genuinely harder optimisation and smoother gradients rather than from gra- dient masking.
The ablation in Table I0 again mirrors this: each successive module added the RS baseline im- proves both certified accuracy and PGD-20 robustness on CIFAR-10 at r 0.75 and c 8/255, culminating in a total gain of +1.9% certified and +1266% empirical robustness. This tight cou- pling supports the view that HyCAS does not trade certified and empirical robustness against each other, but instead uses shared Lipschitz-randommzedd structure to improve both.
Certified-emprrict trade-offs. Figure summarizes HyCAS as a three-way Pareto frontier that trades clean accuracy, certified & accuracy at radius r, and empirical loo robustness at perturbation strength e. The frontier is smooth and strictly decreasing: enlarging the certified radius inevitably reduces empirical robustness. Two systematic gaps appear.
First, in the umall-perturbation regime, empirical for robustness lies well above the certified & accuracy at comparable scales, reflecting the inherent pessimism of worst-case Lipschitz bounds. Many points are robust in practice beyond what a global constant can certify. Second, for large radīi and perturbations, the gap widens further due the norm mismatch: the inequality ||ö]|2 S vā |ox loose at image scale, so a model that is provably stable to moderate lg perturbations can empirically withstand much stronger fox attacks than suggested by the &2 certificate.
Adjusting the smoothing noise a provides a practical knob along this frontier. Increasing a from 0.25 to 0.50 leaves performance at intermediate radii (e.g.. 0.75) almost unchanged, yet substantially boosts certified accuracy the high-radius tail: on CIFAR-10, accuracy at r = 2.0 increases from 8.5% to 12.5%, and ImageNet from 5.4% to 24.8%. while small-e APGD-20 robustness remains competitive. This behaviour shows that, despite the conservative constant 1/4 in the margin bound, training under global Lip 2 constraint still produce margin distributions that deliver trivial certified radii without destroying empirical robustness.
Limitations. The theoretical guarantees in this work are derived from a global < 2 Lipschitz enve- lope and a margin bound r2(x) = (x)) /4 on the expected logits. This certificate based on dif- ferent assumptions than deddommzed-smootthnee ties, and the two guarantees are therefore not directly ordered in terms of tightness. Our analysis does not attempt to prove that the resulting radius is universally stronger than the pondomized-smoollie radius; instead, it shows that, for the randomized, Lipschitz-constrainee HyCAS architecture, shap- ing the margin distribution under global < 2-Lipschitz constraint yields practically useful Eg radii that empirically improve on RS-style baselines using the same backbones. The present theory is restricted lg perturbations, while robustness fox attacks is assessed empirically, and extending the framework to tighter, norm-adaptive or direct foo certificates is left for future work. Finally, HyCAS is compute-heavy: integrating three stochastic streams and the RANI module into standard CNN backbones roughly doubles parameters and memory but increases FLOPs and inference time by about an order of magnitude (Table D. Consequently, our method is most suitable for offline or near real-time scenarios where this overhead is acceptable, and designing lighter-weight HYCAS variants an important direction for future work.
bounds, which operate on smoothed class probabili-
30


Reviews:
[{"review_id": "7nTmt0IVGO", "reviewer": "Reviewer_YBxJ", "summary": "The paper proposes HyCAS, a hybrid defense combining deterministic 1-Lipschitz convolutions with two stochastic components, namely, random-projection filters and randomized attention noise, to yield both high certified $l_2$ robustness and empirical $l_\\infty$ robustness. The method claims state-of-the-art results on both natural and medical imaging benchmarks.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1) The method yields the state-of-the-art certified robustness, outperforming previous considered certified methods on CIFAR-10, ImageNet, and medical datasets (NIH-CXR, HAM10000), with up to $+7.3$ per cent gain at large radii.\n\n2) By tuning the smoothing noise level, the robustness for large radii can be improved with minimal clean accuracy drop, namely, increasing $\\sigma$ from $0.25$ to $0.50$ boosts the certified accuracy (specifically, from $8.5$ per cent to $12.5$ on CIFAR-10 at $r=2.0$).", "weaknesses": "1) The paper asserts HyCAS is the first method to offer both certified and empirical robustness, which is inaccurate: there were works incorporating randomization techniques and empirically robust modules, such as [1] and [2]. \n\n2) The main theoretical result ($\\le2$-Lipschitz bound) is loose in comparison to the one in the baseline work [3]. Consequently, the robust radius is loose too; it raises the question how does HyCAS achieves higher certified robustness in comparison to RS (Table 1)? Does it happen purely because of a higher accuracy on clean, unperturbed data? If so, that limits the theoretical contribution. Overall contribution seems incremental. \n\n3) Noise resampling protocol is not clear: are attention masks resampled per image or per batch during inference? \n\n4) Computational overhead of additional modules is not reported (in terms of memory, inference time).\n\n\n\n\n[1] Dong, M. and Xu, C.Adversarial robustness via random projection filters.In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  4077–4086, 2023.\n\n[2] Yanxiang Ma, Minjing Dong, and Chang Xu. Adversarial robustness through random weight sampling. In Advances in Neural Information Processing Systems (NeurIPS), 2023.\n\n[3] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized\nsmoothing. In International Conference on Machine Learning (ICML), 2019.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2Unz5gEUR2", "reviewer": "Reviewer_yF4E", "summary": "This paper explores a new adversarial defense. The main contribution of the paper is to narrow the gap between the work that has been done for certified defenses under the l2 norm and empirical defenses that use the l-inf norm. This is accomplished through the use of Hybrid Convolutions with Attention Stochasticity, a new method proposed in this paper. Experimentally the results of the new method are shown on a wide range of datasets including, CIFAR-10/100, ImageNet-1K and NIH Chest X-ray.", "questions": "1. Why is the terminology so misused and convoluted? Can you fix the writing of the paper?  \n2. Why are you referencing results that only appear in the appendix? \n3. Why did you not use APGD for all experiments instead of PGD?\n4. How can you claim that your defense that has randomization is deterministic-stochastic?\n5. Can you remove all PGD experiments from the paper and appendix and replace them with APGD?", "limitations": "", "rating": null, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The paper has an interesting approach to adversarial robustness and the experimental results are comprehensive in terms of the datasets used.", "weaknesses": "=In the abstract the metrics mentioned don’t give any specific dataset. A 7.3% better robustness is achieved with respect to which dataset? \n\n=I would avoid the use of the word harmonize in the abstract. It is not clear what you mean when you say you harmonize provable and empirical adversarial robustness. Robustness is a measurement. If I said that I was going to harmonize kilometers and miles, would you understand what that meant? Obviously not. \n\n“the underlying randomness in these defences is static or easily inferred once seeds are fixed, rendering them vulnerable to adaptive attacks and offering no formal guarantees.”\n\n=In the introduction this is a very bold claim and no citations are given to back up this statement. Why would seeds be fixed? Why is the underlying randomness considered static? This is not at all clear. I would need to see several citations in the literature to back up such arguments. Otherwise it is just conjecture. \n\n=I don’t understand why you use the term “natural” and “medical” imaging domains to describe the datasets. I know CIFAR-10, CIFAR-100, ImageNet type of datasets are images and so are chest x-rays. It sounds like you are just trying to make the empirical work more impressive when all you did is test on image datasets. You should really remove this terminology natural/medical from the paper.\n\n=For the experiment for Figure 2, I am not convinced by the results. PGD is a very old attack. For credibility the authors at least need to update to APGD: https://arxiv.org/pdf/2003.01690\n\nAlso as far as I can tell, figure 2 is actually never referenced in the main body of the paper. Why do you have experimental results with no further explanation? \n\n=I think the terminology is confusing when you mention that your technique is a hybrid deterministic-stochastic defense. In theory if you add randomness to a deterministic defense, we would call that defense a randomized defense. In this case your paper is combining both deterministic and stochastic defenses, but this would then mean it is stochastic. E.g., any time a deterministic defense includes randomization, it is no longer random. I am concerned that mixing such terms as you do in your paper will really confuse readers. \n\n=Experimentally I don’t understand why so much space is wasted testing on PGD. In security we care about the strongest possible attacker. At this point PGD has widely been accepted as inferior to APGD. Therefore, there is NO reason to report PGD results in the main body of the paper. A huge  amount of space could be saved, simply by pushing all PGD results to the appendix. \n\n=I notice certain experiments are referenced in the main body but only shown in the appendix. E.g. Figure 7. I don’t think this follows the rules because it forces reviewers to look at appendix material when we should only be considering the main body. \n\nMy overall opinion of the paper is that the experimental results offer a marginal improvement at best. However, the use of extremely convoluted writing techniques throughout the paper (natural/medical), “harmonize” and mixing up what is really a new stochastic defense that they call a nonsensical term “deterministic-stochastic”, all lead me to strong reject. I think the authors should resubmit only after major revisions are done to the writing and terminology of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6uYUDo3vqK", "reviewer": "Reviewer_roSN", "summary": "This paper introduces an architectural modification to standard CNNs that makes them more robust to (particularly $\\ell_\\infty$) adversarial attacks. HyCas has 3 components, each of which are 2-Lipschitz, and are fused via a convex operation, which allows them to obtain an $\\ell_2$ robustness certificate. There are some works that also introduce 1-Lipschitz or spectrally normalized CNNs, but I find the internal stochasticity introduced in HyCas (via random projections, random attention noise), to be principled and interesting, and also standing out as novel in comparison to prior work. I also find the work to be refreshing from the perspective of tackling the adversarial robustness problem in a new way, as opposed to yet another method for solving the vanilla $\\ell_2$/$\\ell_\\infty$ robustness problem. Experiments convincingly demonstrate the practical utility of the proposed method.", "questions": "-- What is the parameter and FLOP overhead relative to a standard ConvNet block?\n\n-- Are the certificates costly to obtain (i.e. via Monte-Carlo)?\n\n-- Have the authors considered salience maps to visualize the sensitivity to input perturbations as opposed to standard l2 robustness methods. I think it would be interesting to see as an additional insight.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "-- A new architecture, with several novel components, which are both well motivated and nicely executed\n\n-- Bridges a key gap between certified and empirical robustness\n\n-- Provide formal proofs that the network is 2-Lipschitz, which to my reading are convincing\n\n-- Benchmark on both natural and medical image datasets\n\n-- Comprehensive experiments in terms of number of defenses compared to", "weaknesses": "-- The three parallel streams plus convex gating presumably increase parameter count and compute cost. But the authors do not report FLOPs or latency.\n\n-- There is no comparison to some new methods e.g. TRADES [1] or HR [2]. \n\n[1] https://arxiv.org/pdf/1901.08573\n[2] https://arxiv.org/abs/2303.02251", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)