Adversarial Robustness via Deformable Convolution with Stochasticity
Authors: Yanxiang Ma
Zixuan Huang
Minjing Dong
Shan You
Chang Xu
Venue: ICML 2025 poster (2025)
Keywords: Adversarial Robustness; Mechine Learning
Primary area: deep_learning->robustness
TL;DR: This paper introduced a random structural adversarial defense method called DCS, inspired by DeConv, and an adaptive AT method for DCS to help it reach and exceed SOTA level.

Abstract: Random defense represents a promising strategy to protect neural networks from adversarial attacks. Most of these methods enhance robustness by injecting randomness into the data, increasing uncertainty for attackers.
However, this randomness could reduce the generalization capacity of defense, as defense performance could be sensitive to the hyperparameters of noise added to the data, making it difficult to generalize across different datasets. 
Additionally, the involvement of randomness always comes with a reduction of natural accuracy, which leads to a delicate trade-off between them, which is seldom studied in random defense. In this work, we propose incorporating randomness into the network structure instead of data input by designing stochastic deformable convolution, where a random mask replaces the convolutional offset. This process promotes data independence, enhancing generalization across datasets. To study the trade-off, we conduct a theoretical analysis of both robust and clean accuracy, from a perspective of gradient cosine similarity and natural inference. Based on the analysis, we reformulate the adversarial training in our random defense framework. Extensive experiments show that our method achieves SOTA adversarial robustness and clean accuracy compared with other random defense methods.

Adversarial Robustness via Deformable Convolution with Stochasticity
Yanxiang Ma Zixuan Huang 2
Minjing Dong?
Shan You Chang Xu'
Abstract
2020), they remain susceptible to subtle adversarial per- turbations, which can mislead them into making incorrect predictions (Madry et al., 2017; Szegedy et al., 2013a; Dong et al., 2017; Moosavi-Dezfooll et al., 2016; Croce & Hein, 2020). There have heen various approaches explored to improve model robustness (Zhang et al., 2019; Wang et al., 2020; Chakraborty et al., 2021; Xie et al., 2017; Carbone et al., 2021; Li et al., 2019: Cohen et al., 2019; Dong et al., 2022; Fu et al., 2021; Ma et al., 2023; Panousis al., 2021). Among them, random defense methods (Xie et al., 2017;
Random defense represents a promising strategy to protect neural networks from adversarial at- tacks. Most of these methods enhance robustness by injecting randomness into the data, increasing uncertainty for attackers. However, this random- ness could reduce the generalization capacity of defense, as defense performance could be sensi- tive to the hyperparameters of noise added to the data, making it difficult to generalize across dif- ferent datasets. Additionally, the involvement of randomness always comes with a reduction of nat- ural accuracy, which leads to a delicate trade-off between them, which is seldom studied in random defense. In this work, we propose incorporat- ing randomness into the network structure instead of data input by designing stochastic deformable convolution, where a random mask replaces the convolutional offset. This process promotes data independence, enhancing generalization across datasets. To study the trade-off, we conduct a the- oretical analysis of both robust and clean accuracy, from a perspective of gradient cosine similarity and natural inference. Based on the analysis, we reformulate the adversarial training in our random defense framework. Extensive experiments show that our method achieves SOTA adversarial ro- bustness and clean accuracy compared with other random defense methods. Code available here.
Carbone et al., 2021: Li et al., 2019; Cohen et al., 2019; Dong et al., 2022; Fu et al., 2021; Ma et al., 2023; Panousis et al., 2021) shows notable effectiveness due to their ran- domness that disrupts adversarial strategies.
Existing random defense algorithms widely adopt additive noises to involve randomness, which makes the defense per- formance sensitive to the hyperparameters of noises. For example, some random defense methods add noises to the data or feature maps in the networks to achieve adversar- ial robustness (Xie et al., 2017; Carbone et al., 2021; Ma et al., 2023; Li et al., 2019). However, the noise distribution, variance, etc., always requires careful tuning to achieve sat- isfactory performance, which makes it difficult to generalize to different datasets. Meanwhile, some random defense methods (Dong et al., 2022; Xie et al., 2017: Carbone et al., 2021; Panousis et al., 2021) focus on reducing adversarial transferability among different sampled paths, there is no in-depth analysis of the trade-off between natural accuracy and robustness, even though the clean accuracy could be significantly influenced by the introduced randomness.
In this paper, we incorporate randomness into the network structure instead of data input to achieve framework with data independence and generalization across datasets. Con- sidering the commonality of convolutional layers in neural networks, we cooperate with deformable convolutions (Dai et al., 2017: Zhu al., 2018) and replace the fixed offsets with random masks to design Deformable Convolution with Stochasticity (DCS), which effectively reduces the sim- ilarity of the gradient in a data-independent way. Based on our designed framework, we theoretically analyze the trade-off off between robustness and clean accuracy from the perspectives of gradient cosine similarity and natural infer- ence. On the one hand, we derive an upper bound for the kernel size to satisfy gradient cosine similarity constraints.
1. Introduction
Despite the remarkable success of deep neural networks (DNNs) in classification tasks (Park et al., 2022; He al., 2016; Zagoruyko & Komodakis, 2016; Dosovitskiy al.,
"gqual contribution 'School of Computer Science, University of Sydney, NSW, Austrilia 'International School, Beijing Univer- sity of Posts and Telecommunications Beijing, China SShoool of Computer Science, City University of Hong Kong, Hong Kong, China 'SenseTime Research, Beijing, China. Correspondence to: Chang Xu <c.xu <c.xu@sydney.co sydney.cdu.au..
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Adversarial Robustness via Deformable Convolution with Stochasticity
On the other hand, we calculate lower bound for the kernel size using the distance between two predictions enhance clean accuracy affected by the introduced randomness. By demonstrating the data independence of both bounds, we selected an optimal kernel size at their intersection, estab- lishing our data-independent framework. Finally, by adap- tively optimizing the DCS during AT to remove points with similar gradient sources, we further enhance the robustness of our proposed framework. Our work has the following main contributions:
2019). Other works added randomness in the network struc- ture by randomly choosing different normalization meth- ods (Dong et al., 2022). Later, (Panousis et al., 2021) adds randomness to the parallel random convolutional layer and then filters the multiple outputs to compose a new feature map. This method introduces extra parameters which are expensive in training. Furthermore, these structural ran- domization methods are not pure structural randomization. The weights of each randomized structure are independent, which is effectively mixed structure- randomization. Adversarial Training (AT) (Madry et al., 2017) is a training method inspired by adversarial generative networks, which is the most commonly used training method for adversarial robust models. To prevent overfitting in this process, (Rice et al., 2020) proposes an early-stop algorithm. AT is widely used in many adversarial defense methods (Chakraborty et al., 2021; Carbone et al., 2021: Dong et al., 2022; Dong & Xu, 2023; Li et al., 2019; Panousis et al., 2021; Ma et al., 2023; Mei et al., 2025; Gong et al., 2024b).
1. We design data-indepnndent framework called DCS to achieve random defense against adversarial attacks.
2. We theoretically explore how kernel size manages the trade-off between natural accuracy and gradient similarity, keeping it within a data-independent range.
3. We design a gradient-selective adversarial training (GSAT) algorithm to remove points with similar gradient origins in DCS, reducing attack transferability.
3. Preliminary
2. Related Works
3.1. Deformable Convolution
Deep Neural Networks (DNNs) are vulnerable adversar- ial perturbations (Goodfellow et al., 2014a; Szegedy al., 2013b), which results in the development of attack and defense algorithms (Szegedy et al., 2013a; Madry et al., 2017: Croce& & Hein, 2020; Carbone et al., 2021; Li & Xu,
To tackle complex spatial variations in data, Deformable Convolution Networks (DCNs) are proposed to replace the normal convolution layer with a deformable convolution layer (Dai et al., 2017; Zhu et al., 2018). In the deformable convolution layer, an offset varies the location of points in the kernel to deform it. The offset can be considered as binary mask, which masks out the original location and exposes new location. With this thought, DCN first defines an initial convolutional kernel K. whose kernel size is learnable variable ko. Then we determined collection of masks + where every mask M E M passes n points. The deformed kernel K can then be built by K = M o K. The dimension of deformed kernel K is defined as n x Onn x Cout. where n is the receptive field of the deformed kernel. In this paper for variable convolution, kernel K is considered
2023; Dong et al., 2020; Cheng et al., 2023; Dong et al.,
2021; 2022; Li et al., 2019; Panousis et al., 2021: Mei et al.,
2025; Gong et al., 2024a;b; Bartoldson et al., 2024; Chen
& Lee, 2024; Amini et al., 2024; Moosavi-Dezfooli et al., 2016; Wang et al., 2023; Carlini & Wagner, 2017; Dong et al., 2017). Commonly, the white-box attacks generate adversarial examples via gradients on the input (Szegedy et al., 2013a; Dong et al., 2017; Madry et al., 2017; Croce & Hein, 2020; Athalye et al., 8b). These algorithms design very small adversarial perturbations along the direc- tion of gradient ascent. The white-box attacks are very hard to detect but do lot of harm to network performance. In recent years, to defend against white-box attacks, several randomized- based algorithms have been proposed to defend against adversarial perturbations (Chakraborty et al., 2021; Carbone et al., 2021; Dong et al., 2022; Dong & Xu, 2023; Li al., 2019; Panousis et al., 2021; Ma et al., 2023; Mei et al., 2025; Gong et al., 2024a;b; Bartoldson et al., 2024; Chen & Lee, 2024; Amini et al., 2024). Some of the works inserted extra random layers with random weights and mul- tiplied them with a specific feature map during the forward propagation (Chakraborty et al., 2021; Carbone et al., 2021 1). (Ma et al., 2023) designed a trainable randomized layer that fits the network through training, to optimize the effect of randomness. Additionally, the random parameters can also be added to the feature map regularization term (Li et al.,
collection of points. By these definitions, DCN can be pixel- wised defined using masks
= DCN(() = Σ w(K)) X
(I)
K,EK
where I is the position of kernel performed on feature map X. In DCN, ko is a learnable variable, while in our method ko is fixed onto a certain value k.
3.2. Randomized Adversarial Defenses
We denote h E HI as the classifier that maps the input X E XC to the logits h(X), where H. is the hypothe- sis space of image classification models and H x W x C is the size of the input image. In white- box settings, adver- sarial attacks maximize the loss function L(h) by adding an
2

---

Adversarial Robustness via Deformable Convolution with Stochasticity
adversarial perturbation A X, which forms adversarial examples (AEs) X. Given the maximum perturbation size ea and the true label AB can be defined as,
Masks
K
Boundary Deformod Kornel
Faundary
Bandemiy Sumple
Sompled Pixels In Kernel
Pessture
X
arg max (()()),)) -i
(2)
Collection of Deformed Kemels
Deformed Kernel K
Example Patch/Pssulan Peature
Inicial Kermel
where A is constrained by ed via -n-orm. While adversarial attacks maximize the loss function, adversarial training (AT) minimizes it. By using the predefined adversarial example as training data, AT finds h" that satisfies,
Cue
Cu Cuca
Layer
Original
Original Layers
h" arg min xxy-xxyyyxxxxx (C(h(X), hER
(3)
Network Input
y
Network Output
Deformable Convolution with Stochasticity Layer
Different from adversarial training, randomized de- fenses prevent attackers from generating accurate AEs (Chakraborty et al., 2021; Carbone et al., 2021; Dong et al., 2022; Fu et al., 2021; Ma et al., 2023). Given two classifiers hı and fi randomly sampled from subset H HHI, the defense scheme is formulated
Figure I. This figure shows how DCS performs random defense predefined network. The top half of this figure shows the process by which DCS generates a collection of deformed kernels and the internal situation of the deformed kernel. The lower half shows the pipeline of a network containing a DCS, where the DCS layer is shown specifically.
H" arg min xxlxy-xxx(((xxx)))))
(4)
NuE
distribute uniformly. In each of the network forward prop- agation, DCS samples a deformed kernel K ~ K. The structure of DCS in a predefined network is shown in Fig. I
where X arg max - Sel [(h (X1) v). In hı and h2, The parameters structures h2 vary from those in hy, which lead to different forward paths. White-box attack settings can be thus downgraded to black-box settings since h and h22 are randomly sampled. The objective is thus converted to the reduction of adversarial transfcrability between h and h2, which achieves adversarial defense.
In DCS, the hyperparameters that control randomness are the receptive field of the deformed kernel n, the initial kernel size k, and the stride S. To keep the dimension of the output feature maps consistent with the demand of the network data flow, S keeps consistent with the original convolutional layers. k is close to the kernel size of the original convo- lution layer, varying padding to fit the demand of output dimension. Thus, randomness can only be controlled via n, which independent of data. Thus, DCS solves the limita- tion on the randomness of the data sensitivity. Based on the description above, we define the function of DCS
4. Methodology
4.1. Deformable Convolution with Stochasticity
Randomized defense methods are always sensitive to their hyperparameters about randomness. In existing works, most of the bounds on hyperparameters are data-dependent (Ma et al., 2023; Dong et al., 2022; et et al., 2019; Carbone et al., 2021). When applying these defense methods to unknown data, the hyperparameters need to be fine-tuned via large amount of observation experiments. To address this limita- tion, we incorporate randomness into the network structure, isolating the hyperparameters that control randomness from the data flow in the pipeline. To make structural randomness generalizable across different CNNs and their variants, we randomize convolution layers via deformable convolution.
Definition Denote K₁ K, sampled deformed kernel at location whose input feature map is X and output feature map is y. Denote X; the pixel on X at location i, and K; the point on K₁ at location i. Following the definition of DCNs in Eq. 1 we define DCS on the pixel level as,
V1 = DCS(I) = Σ w(Ki)
(5)
K;EK;
where the receptive field o
K at each channel is
Specifically, we propose Deformable Convolution with Stochasticity (DCS), which accomplishes structural ran- domized defense by constructing random space for de- formed convolution kernels. In detail, following the defi- nition of Sec. 3.1, we randomly sample some masks from M to construct a random space for the deformed kernels K Cout, where all the kernels in the random space
Given definition 1, we regard normal + m convolution as a special case for DCS, where k = n = m Replacing a normal convolution with DCS can be regarded as generaliz- ing special case into a normal one. Thus, DCS can replace any convolutional layers theoretically, which provides high DCS generality of DCS among convolution- -aaaed networks.
3

---

Adversarial Robustness via Deformable Convolution with Stochasticity
Path Backprpaggttin
Pu and Py are both 0 and ps is 1. In comparison, our ran- domized kernel will decrease ps and increase Pa and Pg- Thus, theoretically, the random kernel can reduce the al- tack's 's transferability in DCS.
Conv
Slide Stridem2
Value
Image Patch
Kernel
Slide K ermel
Given Eq. 7, we then analyze how the hyperparameters of DCS affect the gradient similarity. Specifically, consider DCS layer with a stride of S. initial kernel size of k. and receptive field of n. In pure random sampling, the probabilities in Eq. 5 can be represented as ps k. Py (n k S), and Pv. = - ps Pg. where (n S) can be calculated via n, k and S. The detailed constraints of (n k S) are given in the supplementary materials. S and l are limited by the dimension of the output feature map.
Cosine similanty
Predited yu
Conv
X
Neiggbor of y:
Valus
Kernel Value
Image Patch
Kernel
Rackpmpagatio
An illustration of three different cases corresponding to the three terms Eq. 7. The upper Path represents the attacked path and the lower Path 2 shows the inference path, where the dark orange, light orange, and gray points represent points in the collection X x" and x respectively.
Figure 2.
Lemma 1. Consider a fixed stride and boundary of DCS as S and k separately. To guarantee the expectation of gradient cosine similarity to be smaller than minimum amount t the size of kernel n is strictly upper bounded as
4.2. Transferability Analysis of Random Kernels
The defined DCS introduces different inference paths. To increase its robustness, we reduce the adversarial transfer- ability between different inference paths by controlling their gradient similarity on DCS input feature maps X, using the receptive field n. The gradient similarity on the input feature map between each inference path can be defined as
ni 52 EE.
The detailed proof is in the supplementary material. To minimize the gradient similarity, <. should be minimized, which leads minimizing n. In addition, since the variables in Eq. 8 are not related to data, the only undetermined hyper- parameter of DCS n can obtain a strict data-independent upper bound, which proved our claim of DCS to be data- independent in Sec, 4.1. This bound guarantees small gradient similarity between each inference paths, thus im- proving the adversarial robustness of DCS.
Vx V |
Cos <
Vx, V
(6)
where Vx is the gradient of the example that the attacker can access, and V'x is the gradient that the network will have in inference. A small gradient similarity represents that it is hard for the attacks to transfer from the attacked path to the inference path. For detailed analysis of n, we downgrade the gradient similarity to pixel-level. Specifi- cally, we first divide the points on the input feature map into three types. In the first inference, we define X" the collec- tion points selected by the kernel from location i, X² the collection points not in x" but are selected by kernels from locations other than i, and X the points not selected from any location. The three types of points are shown in Fig. 2. We then expand formula 6 into point level considering these three types of points as
4.3. Balancing Robust and Natural Accuracy
It has been proved that decreasing n can minimize the gra- dient similarity. However, lowering the receptive field may lead to reduction of robustness intuitively. So there is trade-off between the natural accuracy and the gradient similarity. However, existing articles on stochastic defense usually fail to notice this trade-off. To bridge this limitation, in this section, we analyze this trade-off by deriving the relationship between n and the natural accuracy. Intuitively, the receptive field in convolution will limit the performance of the convolution. To quantify the performance, assume an optimal path of DCS whose output feature map is y. Given the optimal case, we use the L1 distances be- tween y and the output feature maps y/ from other inference paths to evaluate the performances of DCS on clean data. Specifically, under the definitions in See. 4.2, after multiply- ing normalization parameter v, the L, distance betwen the output feature maps, V lly s'll can be defined as
first
0x. 8'Xi Oy d'y; (7)
E|Cos < Vx. V'x >} = Σv.. Jp.
Cosl
1=0
P Cos( Oxς 8'X2 B'y
ax. 8'X;'
+ Pa- Cos!
0y,
By: d'y
N'
where N is the number of pixels in the output feature map. Denote Pus Pn and p. the probability of picking X, from x X9, and X" in the second inference by the kernel from loca- tion i, respectively. It obvious that, Cos( ax." 0'x." ) 0, ox. 8'x; Cos( By: J'g. = 1, and that 0 < Cos( By. d'y. < 1. Taking into account the probabilities, for the fixed kernel,
N/82
0g: 0'
D. --1v~V11
Σ
Σ
M((((Kg) Xj+s
ox! X!
E=
K K.
(9)
-K((K)) X/+ill.

---

Adversarial Robustness via Deformable Convolution with Stochasticity
Algorithm 1 Gradient-Selective Adversarial Training
Algorithm Adversarial Defense under White-Box Attack
Input:
Training dataset: X, Y; Classifier: h(X) with ini-
Input: Test dataset: X, Y: Classifier: h(X) with pretrained weight: w:; Attack iterations: i; Size of perturbation: ea: Replaced layer numbers: L; Size of deformed convolu- tion kernel in each DCS: n; Other hyperparameters in each DCS: S
tial weight: w and repeated points X 0; Attack
iterations: t; Size of perturbation: edi Replaced layer numbers: L; Size of deformed convolution kernel in
each DCS: n; Other hyperparameters of each DCS: S, k
1:
Sample 8 from set of standar Gaussian distribu- tion;
Replace the convolution layer with DCS at layers L in the network;
2: Replace the convolution layer with DCS at layers in the network;
Output: y
2: while having data not tested do
3: while not converged do
Sample batch of (X.y) (X. Y);
4: Sample batch of (X. E (X, Y);
4: Sample new deformed convolution kernel for each DCS in the network; Initialize adversarial perturbation óz;
5: Initialize adversarial perturbation òz:
6: Sample a deformed convolution kernel for each DCS in the network;
for i < to I. do
7: Updated and record the X
A₂ clipe. sign(V 8x), end for Re-Sample new deformed convolution kernel for each DCS in the network;
8: for 1 to do
9:
8,- clip., (sign (V..iinnV.Ci..i ((N()))) 17);
10:
end for
11:
Set points in X as 0 and sampe deformed con- volution kernel for each DCS in the network;
10: y = h(x - +
end while
12:
= W C(h(( y.):
13: end while
small gradient similarity between each inference path. To satisfy the trade-off between the gradient similarity and the natural accuracy, both bounds should be taken into account, which results in a data- iata-independent constrain for n as
where K, X versus K", X' represents variables in two dif- ferent inference path, and v; is the normalization parameter location i. Since K and K' are randomly sampled, the inconsistency varies each sampling of DCS. Thus, turn to analysis the expectation of the L1 distance. To consider points in X X99 and x separately, we reformulate Eq. 9 by
S² - - + h + +1
22 N
E En &
.
cc- (12)
4kS
N/8² Biv- 11 = Σ
PaPs w(KG)) (10)
With this constraint, we find the trade-off between attack transferability and natural accuracy while addressing the limitation of data dependency in randomized defense. In conclusion, when obeys the constraint in Eq. 12, DCS can trade-off between the adversarial transferability of different inference paths and the natural accuracy, and provide strong adversarial robustness to the applied network regardless of the challenges from any dataset.
K,KKKKKK
p,p, (X2, w/( sy=)
X w(K)) +p. 0).
where : is a drift that represents the number of sliding steps from é to the location of the kernel that chooses points from X². According to the derivation Sec. 4.2 for Pu and Pyt the inconsistency of the output feature map can be related the DCS hyperparameter n1.
Lemma 2. Assume that the expectation of the inconsistency less than small number 6 O, the size of the deformed kernel is strictly lower bounded as,
4.4. Gradient-Selective Adversarial Training
The above theoretical derivation finds the trade-off in the lata-independent case. In practice, however, Eq. 7 and 10 also remind us that data can affect trade-off. The points in X and X" influence the gradient similarity and output inconsistency, while X hardly affects these two values. To further explore the potential performance of the network a specific dataset, we enhance the adversarial training by eliminating less significant cases corresponding to X", Specifically, we find X by peeking the chosen inference paths of the attacks in AT, and then manually mask out (set 0) the points on the attacked paths. In formula, following
82 -2KS + 222 + 11(11
5 N
n
(11)
4kS
See detailed proof in the supplementary material.
The lower bound on the definition of receptive field in Lemma shows that to decrease the upper bound of the normalized distance, n should be enlarged. In contrast, the upper bound in Lemma wants decrease to guarantee
5

---

Adversarial Robustness via Deformable Convolution with Stochasticity
the definition of AT in Eq. 3, we have,
sarial training for 90 epochs, with 10-step PGD (Madry et al., 2017) generating adversarial examples, setting to 4
H" arg min yx (((bb(()))))))) h2ECu/1
(13)
A 255 and the step length 1/ to 255 The parameters for DCS are consistent with those from the CIFAR setup. The entire model is developed using PyTorch (Paszke et al., 2019) and evaluated using four NVIDIA GeForce RTX 4090 GPUs.
where Czch =
h/h E H. and h + h1s The details of gradient-selective AT (GSAT) are shown in Algorithm 1.
Benchmarks We evaluate DCS under SOTA attacks in Attacks (Kim, 2020). For Projected Gradient De- scent (PGD) (Madry et al., 2017), 8/255 with 2/255 step size and 20 steps, employing random starts. We also assess Momentum Iterative Fast Gradient Sign Method (MIFGSM) (Dong et al., 2017) with c = 8/255, a step size of 2/255, 5 steps, and a decay factor of 1.0. The DeepFool (Moosavi-Dezfooll et al., 2016) attack we used involves 50 steps with an overshoot of 0.02. We also per- form the CW attack with learning rate of 0.01, and Au- toAttack (Croce& & Hein, 2020) with 8/255. In ad- dition, we evaluated DCS under complex attacks by Ex- pectation over Transformations PGD (EOTPGD). All the experiments were repeated 10 times to get the mean value.
During inference, we will not make any assumptions about the attack strategy to obtain a fair result. However, every DCS will sample n points inside the boundaries purely new convolutional kernel
randomly and independently for the DCS before each inference. The detailed inference method is shown in Algorithm 2.
5. Experiment
5.1. Experiment Setup
We evaluate DCS on CIFAR dataset (Krizhevsky, 2012) and Imagenet dataset (Krizhevsky et al., 2017). Various convolution-basee networks are used for baseline networks, including 8 (Park et al., 2022), ResNet50 (He et al., 2016) and WideResNet34 (Zagoruyko & Komodakis, 2016). In our experiments, unless specifically labeled, DCS re- places the second convolutional layer. All other layers keep the original settings.
5.2. Main Results
We evaluated DCS on ResNet 18 and WideResNet 34 (WRN 34). The results are shown in Table. 1. We com- pared DCS with other baselines to prove the advanced per- formance of DCS. In detail, we compare against a normal convolution layer and normal DCN layer. Our DCS in main experiments performed GSAT whose efffcttiveness is proved later in ablation study in Sec. 5.3.2. As demonstrated in the tables, our approach significantly outperforms both standard convolution and fixed DCN. When applying DCS in ResNet 18 on the CIFAR 10 dataset, comparing against the normal network, our approach progressed 20. 20.94% un- der the PGD attacks and 26.07% under AA, while making progress on natural accuracy of 7.94%. On CIFAR 100 dataset, the advancement of robust accuracy 18.26% under PGD attacks and 24.44% under AA, while progressing the natural accuracy by 9.57% On WRN 34 DCS with GSAT have also achieved high performances. In detail, on CI- FAR 10, our approach is 21.84% more accurate under PGD attacks and 24.62% under AA. CIFAR 100 yields similar results at advancement of 16.07% under the PGD attacks and 24.21% under AA. The natural accuracy also increases by 4.40% and 6.59% on CIFAR 10 and 100 respectively. From these results, DCS with GSAT are proven to have advanced performance on both CIFAR 10 and CIFAR 100.
Experiments on CIFAR The CIFAR 10 and CIFAR 100 datasets contain 10 and 100 classes, respectively. Each dataset consists 5.0 x 10 training samples and 1.0 x 10a test samples, with all images resized to 32x32 pixels with three color channels. We implement DCS by replacing the third convolutional layer on both ResNet18 and WRN34 for the CIFAR datasets. The hyperparameter n in DCS is set to 2 and k- for both networks. In our experiments, we split the dataset into batches of 128, setting the weight decay at 5.0 x 10-4. We employed an SGD optimizer with a momentum of 0.9. The initial learning rate was set at 0.1 and was decreased according to a multi-step schedule. The model was trained over 200 epochs, with learning rate reductions by factor of 10 at epochs 60 and 120. For adversarial training, we configured e at & and the step 255
2 length 17 at 255 for 7-step PGD (Madry et al., 2017). The model is implemented using PyTorch (Paszke et al., 2019) and trained on an NVIDIA GeForce RTX 4090 GPU.
Experiments on ImageNet The ImageNet dataset consists of 1.2 x 10t training examples and 5.0 10a test examples, classified into 1000 distinct classes. Each image is in a 3-channel RGB format with a resolution of 224 x 224 pix- els. We implement DCS on ResNet50 for this dataset. The hyperparameters of DCS remain the same as on ResNet 18. In our experiments, we divide the dataset into batches of 512 examples. The weight decay is 1.0 x 10-4 with the initial learning rate set at 0.1 and managed using a cosine annealing scheduler. The model is fine-tuned using adver-
5.3. Ablation Study
5.3.1. INFLUENCE OF KERNEL SIZE ON ROBUSTNESS
As mentioned in Lemma 1, the initial kernel size k of the de- formed kernel can influence the transferability of the attacks
6

---

Adversarial Robustness via Deformable Convolution with Stochasticity
Table
The results of robust accuracy on CIFAR 10 and 100 of adversarial defense algorithms.
Dataset
Model
Method
Natural
P
AA
CW12
MIFGSM
DeepFool
Conv
81.17
51.16
47.34
77.69
54.07
5.63
RN 18
DCN
80.33
52.23
48.04
77.07
54,91
8.01
DCS(ours)
89.11 = 0.10
72.10
2.49
73.41 + 2.04
88.56 = 0.29
69.75 2.05
88.24 = 0.58
CIFAR 10
Conv
86.18
54.00
50.84
82.50
57.99
4,48
WRN 34
DCN
82.25
52.54
49.11
78.36
55.29
0.90
DCS(ours)
90.58 + 0.08
75,84 + 2.00
75.46 + 1.62
90.28 = 0.01
76.86 + 1.79
90.20 + 0.12
Conv
55.14
28.31
24.57
50.61
29.55
5.07
RN
DCN
54.06
23.23
19.82
48.88
25.41
1.05
DCS(ours)
64.71 + 0.05
46,57 = 1.96
49.01 + 1.10
64,33 = 0,07
49.38 + 1.27
64,43 +: 0.26
CIFAR 100
Conv
60.10
31.98
28.15
55.35
33.87
3.47
WRN 34
DCN
58.90
27.02
24.47
53.88
29.65
4.79
DCS(ours)
66.69 + 0.07
48.05
1.81
52.36
+ 1.14
66.55
0.06
51.24 + 1.03
66.59 + 0.30
90.0
WRA
WRME4
40
WRRE4 Resfaet1
87.5
Reshwil2
Accurac 190
85.0
2
Accuracy 175)
-55
82.5
75
80.0
76-
86
75.0
72.5
55
10 20
30
50
90 100
11 13 15 19
10
Staps of PGD attack
Epsilon of PGD attack 1/255)
Rerations of EDTPGD attack
(a)
(c)
Figure 3. Evaluation of rohustness under different attack settings: (a) PGD steps, (b) e PGD, (e) BOT iterations of EOTPGD attacks.
Table 2. Comparison with SOTA methods,
Netrral Acc
Coure Sndlatly
Type
Methods
FGD AA
CIFAR 10
FGD
Imagenet
AA
100
1.0
+00999
100.0
Overfit (Rice et al., 2020)
55.06 52.24
39.85
Accuracy
05
Cosine
(P1e4++0.90
DWQ (Fu et al., 2021)
52.18 49.70
42.88
AT
RobustWRN (Huang et al., 2021)
59.13
52.48
31.14
40-
WRN+DCN+AT (Zhu al., 2018)
49.11 52.54
40.66
0.5
DifIAT (Wang al., 2023)
70.69
31.30
Additive Noise (1. et al., 2019)
62.36
58.47
Normaaal
(B6s4)
32
Random
AdvWRN (Bartoldson al.. 2024)
73.71
(a)
(b)
Noise
DF (Chen& & Lee, 2024)
58.22
40.60
MeanSparse (Amini et al., 2024)
75.28
59.64
Figure 4. Observation of Cosine similarity: (a) comparison be- tween different training techniques, (b)comparison between ker- nels that sample points from different collections.
Certified
Cert-RA (Chiang et al., 2020)
68.60
Random
LW WWTA(Panousis ct al., 2021)
81.7
74.71
Structure
SAF (Wang et al., 2018)
67.40
(CS(o))))
75.84
75.46
52.38 66.79
3,5,7 and 9. The results show that larger boundaries help to enhance robustness and also provide more stable results. However, changing & changes the padding and introduces more network parameters. Therefore, A is recommended to be small. In the table, the robustness increases from k = 3 5. With k > 5, more additional parameters will be introduced, but the increases smaller.
Table 3. Evaluation Pradient-Selectiie AT,
Baseline
AT method
Natural Acc
Robust Acc
Normal AT
83.53 + 0.35
60.64 + 0.02
ResNet 18
GSAT
89.11 +: 0.10
73.41 + 2.04
Normal AT
90.58 + 0.10
54.45 + 0.00
WRN 34
GSAT
90.58 1 0.08
75.46 + 1.62
5.3.2. EVALUATION ON SRADIENT-SELECTIVEE AT
by controlling the upper bound of n. We evaluate the impact of the initial kernel size on the robustness of DCS under 2GD attacks, given a fixed receptive field. The results are shown in Table. 4. Specifically, we fixed the receptive field to 4 and then tested the case with initial kernel sizes of
The AT that is adapted to DCS is expected to boost the robustness via decreasing the attack transferability, as men- tioned in Sec. 4.4. We believe that this adversarial training can effectively minimize the effect of the batch of points

---

Adversarial Robustness via Deformable Convolution with Stochasticity
F'y. The results are consistent with what we analyzed in Sec.4.2, proving that the analysis in Lemma 1 on Eq. 7 and the data-independent upper bound of are consistent with the facts. Moreover, according to observation, the cosine similarity is also low when points belonging to X are sampled the second time. We hypoth- esize that this is due to the different sources of gradients generated by the two inference paths for the same point. This phenomenon also implies that the upper bound of n can be loosened a bit in some datasets.
means Cos
axx 8'x" 0y:
Table 4. Effectiveness of DCS over different w/o GSAT.
k
Natural Acc
Rohust Acc
9*9
87.12 + 0.12
68.64 + 1.14
7+7
90.63 + 0.10
66.39 + 3.20
5*5
90.58 1 0.10
62.93 +: 4.48
3
15.44 + 0.27
37.52 + 5.29
Table 5. GSAT over different receptive fields with normal test against the test that canceled x".
Include X
2G
AA
5.3.4. 4. INFLUENCES ON RECEPTIVE FIELD
4
V
62.93 +: 4,48
64.18 + 0,91
70.75 2.55 72.88 + 1.47
To verify the trade-off mentioned in Sec, 4.3, we evalu- ated the performance of DCS over different receptive fields. Specifically, we use ResNet 18 as the baseline for this abla- tion study and follow the settings of the major experiments. Given the observation about the gradient similarity on CI- FAR 10, as analyzed in Sec. 5.3.3, the upper bound of can be loosened. Therefore, we evaluate of the effect of in a wide range of values on CIFAR 10. The results are shown in the first four lines of Table. 5, where DCS- -cp-applee networks achieved the highest robust accuracy of 72.10% under PGD attacks and 73.41% under AA with the receptive field of 2. From the table, there a clear trend of trade-off for n on robustness, which consists of our derivation in Eq. 12.
V
72.10 + 2,49 73.41 + 2.04
V
61.83 + 1.25 64.40 + 1.89
4
x
63.50 = 3,80
56.40 = 1.53
X
65.15 = 3.26
65.83 = 0.88
2
X
64.64 + 2.04
66.14 = 1.60
X
56.82 = 1.84
58.88 + 1.67
X* on the network and decrease the gradient similarity. To measure its effectiveness, first observe the gradient simi- larities and accuracies of ResNet 18 using GSAT, AT, and normal training on CIFAR 10. The results are shown in Fig. 4a. It can be found that the GSAT reduces the cosine similarity and boosts the natural accuracy compared to AT, which consists of our expectations. For more generalized situations, We evaluated the GSAT on CIFAR 10 and 100 using ResNet 18 and WRN 34 as the baselines under Au- toAttack. The results are shown in Table. 3. We can find that with larger n. GSAT will be more unstable. We give an explanation as: when Eq. 12 gives **larger * for nt, which refers to larger n in practice, the percentage paths in unselected random space will increase. This increases the probability for DCS performance decrement in the networks using GSAT for training, which leads to greater instability. Thus, as shown in Table. 3, n is suggested to be small for stable AT. Table. 3 also shows that DCS obtains higher robust accuracy and natural accuracy via GSAT, which is consistent with our analysis.
Furthermore, based on the observation in Sec. 5.3.3 about the cosine similarity of points in X if Eq. 7 holds, then the DCS will maintain strong robustness even if Pa = 0. To verify the validity of Eq. 7, we manually fix Pa. = 0 by canceling points in X in the second inference. The results are shown in the last four lines in Table. 5. The best robustness is higher than the baseline but suffers a minor decrement compared to the DCS including points in X".
This result proves that in Eq. 7, < 2x 8xx. addition, given our observations on gradient similarity, Eq. 7 holds, which verifies the validity of Lemma 1.
1. In
ax 8
5.3.5, ROBUSTNESS UNDER DIFFERENT STRENGTII OF ATTACK
We evaluated the defense capacity of DCS against attacks of different strengths using ResNet 18 and WRN 34 as the baseline on CIFAR 10. We evaluate DCS via three groups of experiments under different settings of PGD attacks.
5.3.3. OBSERVATION OF GRADIENT SIMILARITY
To further verify the validity of the study of Eq. on gradient similarity, we directly observed the three cases correspond- ing to X", X and X". Specifically, we manually designed three sets of deformed kernels with n = 4 corresponding to the three cases represented. The detailed kernel is illus- trated in the supplementary material. We used ResNet 18 as a baseline and randomly selected 1000 points on CI- FAR 10 for observation, and the distribution of gradient similarity for each case is shown in Fig, 4b. In this figure C, is Cos 0g. Jy.
PGD steps. Since the effectiveness of white-box attacks such as PGD depends on strength and adversarial transfer- ability. In order to avoid the effect of attack strength on the robustness provided by the DCS, we conducted experiments on the CIFAR 10 dataset using ResNet 18 and WRN 34 with DCS under PGD attacks of different strengths. The evaluations are performed at different levels of PGD attacks at each 10 step from clean settings, until 100 steps. The
axx xx) ). Cg means Cos
axx 8'x") Cn
0% Fw.
8

---

Adversarial Robustness via Deformable Convolution with Stochasticity
Table 6. DCS under black-box attacks.
Baseline
Natural
Query-based
brassfer-based
Adaptive
SQUARE
Pixel
FGSM
FGSM-base
BPDA
BPDA+EOT
RNI8 WRN34
89.11 90.58
80.53
86.32
84.95
65.71
78.66
77.78
82.45
86.31
90.53
68.34
80.03
80.47
stroys the integrity of the black-box attack thereby reducing its effectiveness. For ataptive attacks like BPDA and EOT- BPDA, DCS also shows high robustness. We attribute this to randomness in conjunction with gradient masking.
Table 7. DCS applied to ViT.
Method
PGD
ViT-t+Conv
32.31
ViT-++DCS 55.71
5.3.7. DCS ON TRANSFORMER
We confirm that DCS is mailor-made for convolutional op- erations. We believe that convolution is still an important tool in image processing. Transformer is out of the research scope of this work. However, we briefly explored that DCS fits Vision Transformer (ViT).
results are shown in Fig. 3a. We found that even under strong attacks, PGD can still provide strong robustness to the baseline network, despite some fluctuation.
PGD perturbation boundary e. In addition to the changed number of steps for generating the adversarial perturbation, the attack strength can be changed by varying the bound- aries of the adversarial perturbation <d- We evaluated the performance of DCS under attacks of different ea. The re- sults are shown in Fig. 3b. can be seen that the robustness of the DCS is gradually decreasing as the perturbations be- come stronger. This suggests that DCS has difficulty in reducing the adversarial transferability of an attack to O. We attribute this to the fact that the trade-off off prevents the cosine similarity from being reduced close to O.
ViT uses a 16 x 16 convolution in patch embedding. Large kernel size makes lower bound of n increases and hinders finding a suitable n. To avoid this, we notice that patch embedding can be split into multiple concatenated 3 x 3 convolutions. Our baseline follows the settings in (Xiao et al., 2021) and then replace the second 3 x 3 convolution with DCS. The results are shown in Table 7. We can find that DCS also achieves strong robustness on VIT.
Training Stages. The network trained on CIFAR-10 in two stages. All stages are trained using SGD as the optimizer, with weight decay of 5e-4, an initial learning rate of 0.01 and batch size of 128. Stage one fixes pretrained ViT-tiny and trains the FC layer and convolutions from scratch with 200 epochs using the multistep scheduler, where the learning rate is divided by 10 at epoch 50 and 100. Stage two adversarially finetunes the entire network using GSAT with 90 epochs with a cosine scheduler.
Iterations of EOTPGD Finally, we also evaluate the robust- ness of the DCS under multi-step attacks. Specifically, we used EOTPGD as benchmark for multi-step attacks and evaluated the robustness of models embedded with DCS under EOT attacks from 1 to 10 EOT steps. The results of which are shown in Fig. 3c, which shows that DCS re- mains robust even under multi-step EOTPGD attacks. This indicates that DCS can adapt to complex multi-step attacks.
6. Conclusion
5.3.6. ROBUSTNESS UNDER BLXCK-BOX ATTACKS
Considering from the gradient perspective, DCS objectively shields a portion of the gradient. The defense performance of DCS, as gradient-masking defense method, against black-box box attacks is difficult to theoretically infer.
In this paper, we proposed random structural defense method named DCS. We proved that DCS solved two major limitations in existing random defense methods, which are data date-dependent hyperparameters and the trade-off between gradient similarity and natural accuracy. Through theoreti- cal derivations, design set of bounds for DCS to reduce the adversarial transferability between each of its inference paths without degrading the natural accuracy. Finally, we design GSAT for DCS to enhance the robustness of DCS. We experimentally validate the above theory and demon- strate the SOTA robustness of DCS. future research, we expect that data-independent defense methods similar to DCS can demonstrate defenses in various areas of computer vision to fit 0-shot inference scenarios.
To have a complete understanding of the robustness of DCS, we observed the defense capability of DCS against black-box attacks. We evaluate DCS under query-baseds transfer-based (Goodfellow al., 2014b) and adaptive at- tacks (Athalye et al., 201 8a) CIFAR 10 with ResNet 18 and WideResNet 34. The results are shown in table 6. DCS is found to have some defense capability against black-box attacks. We attribute this phenomenon to the fact that the random masking of the convolutional kernel by DCS also shields a portion of the adversarial perturbations, which de-

---

Adversarial Robustness via Deformable Convolution with Stochasticity
Acknowledgement
Cheng, Z., Li, Y., Dong, M., Su, X., You, S., and Xu, C.
Neural architecture search for wide spectrum adversarial robustness. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pp. 442-451, 2023.
This work was supported in part by the Start-up Grant (No. 9610680) of the City University of Hong Kong, Young Scientist Fund (No. 62406265) of NSFC, and the Aus- tralian Research Council under Projects DP240101848 and FT230100599.
Chiang, P-y., Ni, R., Abdelkader, A., Zhu, C., Studer, C., and Goldstein, T. Certified defenses for adversar- ial patches. arXiv preprint arXiv:2003.0660 2020.
Impact Statement
Cohen, J., Rosenfeld, E., and Kolter, Z. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pp. 1310-1320. PMLR, 2019.
This paper presents work whose goal advance the field of Machine Learning. There are many potential societal consequences of our work, none of which feel must be specifically highlighted here.
Croce, and Hein, M. Reliable evaluation of adversarial robustness with an ensemble diverse parameter- attacks. In International conference on machine learning. pp. 2206-2216. PMLR, 2020.
References
Amini, S., Teymoorianfard, M., Ma, S., and Houmansadr, A. Meansparse: Post-training robustness enhancement through mean-centered feature sparsification, 2024. URL https: / /ttpxa.wwgggg//.aaaa//./...o.ogt abs / 2406 05927.
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. Deformable convolutional networks, 2017. URL https: httpa/axii...aaaaaa////
Athalye, A., Carlini, N., and Wagner, D. Obfuscated gra- dients give false sense of security: Circumventing de- fenses to adversarial examples. In International confer- ence on machine learning. pp. 274-283, PMLR, 2018a.
Dong, M. and Xu, Adversarial robustness via random projection filters. In Proceedings of the IEEE/CVF Con- ference Computer Vision and Patlern Recognition, pp. 4077-4086, 2023.
Athalye, A.,
Engstrom, L., Ilyas, A., and Kwok, K. Synthe- sizing robust adversarial examples, 2018b.
Dong, M., Li, Y., Wang, Y., and Xu, C. Adversarially robust neural architectures. arXiv preprint 2rXiv:2009.00902, 2020.
Bartoldson, B. R., Diffenderfer, J., Parasyris, K. and Kailkhura, B. Adversarial robustness limits via scaling- law and human- allaantlignmment studies, 2024. URL https: //arxiv. org/ abs 2404. 09349.
Dong, M., Wang, Y., Chen, X., and Xu, C. Towards stable and robust addernets. Advances in Neural Information Processing Systems, 34: 1et11325--3265 2021.
Carbone, G., Sanguinetti. G., and Bortolussi, L. Ran- dom projections for improved adversarial robustness. In 2021 International Joint Conference on Neural Networks (JJCN)), pp. 1-7, 2021. doi: 10. N..1099/CNN52238.220222 9534346.
Dong, M., Chen, X., Wang, Y., and Xu, C. Random nor- malization aggregation for adversarial defense. Advances Neural Information Processing Systems, 35:33676- 33688, 2022.
Carlini, N. and Wagner, D. Towards evaluating the robust- ness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. Icve, 2017.
Dong, Y., Liao, F,, Pang, T., Hu, X., and Zhu, J. Discovering adversarial examples with momentum. arXiv preprint arXh::1710.06088 5, 2017.
Chakraborty, A., Alam, M., Dey, V., Chattopadhyay, A., and Mukhopadhyay, D. A survey on adversarial attacks and defences. CAAI Transactions Intelligence Technology, 6(1):25-45, 2021.
Dosovitskiy, A., Beyer, L.., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 199,9999001199, 1929, 2020.
Chen, E.-C. and Lee, C.R. efficient adversarial training. nition, 151:110394, 2024. doi: https://www.sciencediiiccc com/ URL ssiianc///rrtcle///////033222222
Data filtering for Pattern Recog-
Fu, Y., Yu, Q., Li, M., Chandra, V., and Lin, Y. Double- win quant: Aggressively winning robustness of quantized deep neural networks via random precision training and inference. In International Conference on Machine Leamn ing, pp. 1492-35044 PMLR, 2021.
ISSN 0031-3203.
10

---

Adversarial Robustness via Deformable Convolution with Stochasticity
Gong, H., Dong, M., Ma, S., Camtepe, S., Nepal, S., and Xu, C. Stealthy physical masked face recognition attack via adversarial style optimization. IEEE Transactions Multimedia, 36:5014-5024 2024a. doi: 10.1109/TMM. 2023,3330089.
adversarial attacks. arXiv preprint arXiv: urXvv//706.06083 2017.
Mei, H., Dong, M., and Xu, C. Efficient image-to- image diffusion classifier for adversarial robustness. Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, (9(6):6081 1116001-6085 1-6089, Apr. 2025. doi: 10.1609/ mmai.v39i..32665) URL https: http::ss//os..aaa..oooo index. AAAI/article/vieeeeee.ttttaettte
Gong, H., Dong, M., Ma, S., Camtepe, S., Nepal, S., and Xu, C. Random entangled tokens for adversarially robust vision transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15554-24563, June 2024b.
Moosavi-Dezzoolli S.-M., Fawzi, A., and Frossard, P. Deep- fool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2574-2582, 2016.
Goodfellow, Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. CoRR, abs/1442.6552, 2014a.
Goodfellow, I. J., Shlens, J., and Szegedy, C. Explain- ing and harnessing adversarial examples, arXiv preprint arXiv:1412.6571 6572,, 2014b.
Panousis, K. P., Chatzis, S., and Theodoridis, S. Stochastic local winner-tukes-all networks enable profound adversar- ial robustness. arXiv preprint arXiv:21 2 /2.02671 2021.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Park, S., Yoo, C.-H., and Shin, Y-GG Effective shorteut technique for generative adversarial networks. Applied Intelligence, 53(2):2055-200 $10489-022--3366622 URL https: /ttppp:://di... /ttp///dioo/o /ttp:////oo..//....//. 1007 0104:/10118890 020-0-00222----6---2
may 2022. doi: 10.1007/
Huang, H., Wang, Y., Erfani, S., Gu, Q. Bailey, J., and Ma, X. Exploring architectural ingredients of adversar- ially robust deep neural networks. Advances in Neural Information Processing Systems, 34:5545-5525 2021.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z.,, Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Kim, H. Torchattacks: A pytorch repository for adversarial attacks. arXiv preprint arXiv:2010.01955) 2020.
Rice, L.., Wong, E., and Kolter, 7. Overfitting in adversari- ally robust deep learning. In Intern ational Conference on Machine Learning, pp. 8093-8104. PMLR, 2020.
Krizhevsky, A. Learning multiple layers of features from tiny images. University of Toronto, 05 2012.
Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84-90, 2017.
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D.,, Goodfellow, I,, and Fergus, R. Intriguing properties of neural networks. arXiv preprint /312.6/633/ 2013a.
Li, B., Chen, C., Wang, W., and Carin, L. Certified adver- sarial robustness with additive noise. Advances in neural information processing systems, 32, 2019.
Szegedy, C.,, Zaremba, W., Sutskever, I., Bruna, J., Erhan, D.,, Goodfellow, I,, and Fergus, R. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013b.
Li, Y. and Xu, C. Trade-off between robustness and accuracy of vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7558 -5688, 2023.
Wang, B., Lin, A. T., Zhu, W., Yin, P., Bertozzi, A. L., and
Osher, S. J. Adversarial defense via data dependent acti- vation function and total variation minimization. arXiv preprint a 1009:0800000000000 009.08516, 2018.
Ma, Y., Dong, M., and Xu, C. Adversarial robustness through random weight sampling. Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, (eds.), Advances in Neural Information Processing Sys- tems, volume 36, pp. 37657-336666 Curran Associates, Inc., 2023.
Wang, Y., Zou, D., Yi, J., Bailey, J., Ma. X., and Gu, Q. Im- proving adversarial robustness requires revisiting misclas- sified examples. International Conference on Learning Representation, 2020.
Wang. Z., Pang. T., Du, C., Lin, M., Liu, W., and Yan, S. Better diffusion models further improve adversarial training, 2023. URL https: /arxiv. .ov.vorrg/oo orgo /bs// 2302.00633
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to

---

Adversarial Robustness via Deformable Convolution with Stochasticity
Xiao, T., Singh, M., Mintun, E., Darrell, T., Dollár, P., and Girshick, R. Early convolutions help transformers see better. Advances in neural information processing systems, 14:30392-30400, 2021.
Xie, C., Wang, J., Zhang, Z.,, Ren, Z., and Yuille, A. Miti- gating adversarial effects through randomization, arXiv preprint arXiv:1711.01991, 2017.
Zagoruyko, and Komodakis, N. Wide residual networks. arXiv preprint arXiv:1605.07144 2016.
Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., and Jordan, M. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning. pp. 7472-7482, PMLR, 2019.
Zhu, X., Hu, H., Lin, S., and Dai, J. Deformable convnets v2: More deformable, better results, 2018. URL ht tts: // /aaxiiv.. org/ abs//8ss11111111 11168.
12

---

Adversarial Robustness via Deformable Convolution with Stochasticity
A. Proof of Theorems
A.1. Proof of Lemma
We start with explaining the pixel-level gradient similarity. The cosine similarity in pixel level can be redefined as,
Vx.V': E 1i=0
Cos Vx.
V
(14)
M
where M
For each poin, considering the three types, the cosine similarities be divided into three terms as is demonstrated in Eq. 7 in the main text. We define E[Cos < Vx,Vx Vx >) Ei C [V Vxx where C = Cos < Vxa Vxx, represents the gradient similarity at each poin. Considering Py = Pu PS* we have
n (1- C,) C. Pu C.
C
(15)
By: 8' E [-1, When Cγ < 0, it can be guaranteed that C, g. Thus, we concentrate the case that C² O. From Eq. 15, Cς decided by the data and the network parameters. As the data is unknown, to minimize C. we maximize Pu- Since Pa is decided by the random sampling results, we turn maximize the lower bound of Pu to guarantee small upper bound of C, Given pointX, in the first inference, the X produced by it can stay in 5 different shapes. There are three kinds of sizes for those differently shaped X Denote = k2 (mod 5²), the number of X in size 1 is m² in size $ 11 [ is 2m(k- m), and in size | is (I: m)² where L-] for rounding down and [-] for rounding up. Denote 5 a, and (I: m) = 3, the lower bound of pa in Eq. in the main text can be defined
where C7 exp
0'x!
=fall
in & m²: mf < n & m² 2m3; if + 2m38 < a & 82;
p-m [al "al""in------ Mallo
Pu 2
(16)
if 82.
From Eq. 17, when 82, the lower bound of pa is which makes no guarantee that the C₁ is small. Thus we concentrate on the cases where S². When < S2, looser bound of p.. from Eq. 16 can be derived
S - n)]a]
Pu
(17)
k2
Then, as the upper bound of C; is expected to be smaller than ce, it can be formulated
(s² n
C
n
C9) + Cg
C <
(18)
2
k2
From Eq. 18, the upper bound of can be derived as
n $ 12E + (S a 1- 13)
(19)
+ (1a) -
When considering the worst case, where C 1, the upper bound of n can be rewritten as
n < (. - 1)
aa|2 + 52.
(20)
From Eq. 20, |a| <
there looser bound for n as 52.
22
n & E
(21)
A.2. Proof of Lemma 2
In Lemma. 2, we constrain the L1 distance between the output features after normalization into a small boundary. To expand the J distance into pixel level, we divide the points into three type, which are sampled from X X9, and X respectively,
13

---

Adversarial Robustness via Deformable Convolution with Stochasticity
Kemel 1
Keme x
Kemel X
Kemel X"
Figure 5. Illustration of Kernel Pairs
as shown in Eq. 10 in the main text. Denote E [0, 1] by the average value of Σ ( x2y K2)) and
Ihll by the expectation of E v Ef=o [X w K]]]', the constrain in L1 distance in pixel level is
N/S²
(1 P. A
Pu pull
(22)
52
To satisfy Eq. 22 compare the boundary with the upper bound of Pa. Similar to the analysis in Section. A.1, divide the upper bound of pa into three cases
[al? 1 w [al" tlam Mallo
if & mm|a]
&
im < - if ## <m
(23)
For the terms to the right hand side in Eq. 23, the goal to leave only n into the numerator part. Thus the relationship between Ps. and can be clear. We loose the terms by considering ---7 as --.) and [-] as (-) For the first term
na²
pu
<
[a 2
For the second term
m²a²
m²a
na
11
Pu <
k22
h2 [
k-2 a
2-2 [a]]
For the third term
na k2 |a
Pu <
1
set the strictest bounds for Pa- Bringing a 5. there a looser boundary Pa considering all cases as
n
Pu <
(24)
where < |a] As mentioned in Section. A.I, consider the situations where < 52, and [a] 1, always satisfies the condition n < |5? To satisfy Eq. 22, for given l and S, must be greater than a lower bound as
a 2
[a] St 1)A²
n
NA
(25)
[a
[a]
As & [a] < a and
[a] <a+ + 1, looser boundary can be found for n as
(a - 1)2 - 2 cisa
1)
n
NAP
(26)
4a
"N here size of input feature map including the padding point
14

---

Adversarial Robustness via Deformable Convolution with Stochasticity
When consider the worst case where A =1, taking into Eq. 26, n is lower bounded
- + h2 k2 - N 1)
(27)
4k$S
The form in Eq. 27 consists of the form in Lemma. 2.
A.3. Constrain of Pg
Based on the boundary of mentioned in Section. A.1 and Section. A.2, we can derive the boundary of Pa from Pg = Pu Ps as
(5² - nn] -n
n
E Pg <
(28)
22
12
122
where < 5². From Section. A.1 and Section. A.2, claim that in the derivation of Lemma.1 and Lemma. Pg is bypassed through replacing Pg into 1 Pu Ps. where ps x. The exact form of Pg is not important for Lemma. 1 and Lemma. 2. Thus we do not specify Pg in the derivation in Sec. 4.
B. Additional Experiments
B.1. Explanation of the instability of GSAT
According to Eq. 13, GSAT modifies the sample space sampled by the DCS layer in forward propagation during training by selection. Below explain the reason for the instability of GSAT shown in Table 3.
We have demonstrated both theoretically and experimentally that paths in X9 and X" are not easy to attack, while paths in X are very easy to be attacked by pradient-basedd algorithms (see Eq. 7, 10 and Sec. 5.3.3 for details). This means that when the network is attacked, the gradients generated by the paths in X and X" will be in different direction from the attacked gradients generated by the paths in X
To minimize the influence of inaccurate attacked gradients in training, we select and remove paths in X from X, and build the selected random space, as demonstrated in Eq. 13. This corresponds to step 11 in Algorithm 1. The selected random space is used for sampling paths in the forward and backward propagation for each training step separately.
We understand that using GSAT the network are only limited optimized on the paths in X under each adversarial examples. So when the DCS samples paths in X* in inference, there are performance decrement compared to the other sampled paths. The decrement will cause the instability in GSAT. The larger probability the performance drop in DCS, the greater instability GSAT.
B.2. Illustration of Toy Kernel Pairs
To observe the gradient similarity in Section 5.3.3 in the main text, we choose some toy kernel pairs, where the points sampled the second kernels are in X*, X9, and X" respectively. The shapes of these toy kernel pairs are illustrated in Fig. In the figure, Kernel represents the kernel sampled in the first inference, and Kernel X X9, and X" represents the kernel sampled with points only in collection Xs, X², and X", respectively.
Natural
90
PGD
Accuracy (%)
80
B.3. Grid Search of k and in DCS
60
We did grid search the hyperparameters of DCS. As S is determined by the network, there lefts two hyperparameters to be determined, which are
50
k and n. The results are shown in Fig. 6. From the figure, it is clear that under the assumption of n < 52, when l = 5 and n = 2, DCS can reach the best performance under both AA and PGD attacks. Thus, we choose this group of hyperparameters in the major experiments.
2
8
Position of DCS
Figure 7, DCS Different Position
15

---

Adversarial Robustness via Deformable Convolution with Stochasticity
PGD Accuracy (%)
AA Accuracy (%)
57.03
72.15
65.50
56.01
55.40
51.49
73.34
70.07
58 38.85
61.15
Receptive Field
700
Receptive Field
Accurag 160
60
60
43.56
65.15
70.19
62.41
56.78
50
37.39
65.83
72.81
63.57
60.20
50
40
40
66.33
20
3
11
5
7
9
11
Kernel Size
Kernel Size
(a)
(b)
Figure 6. Hcatmaps of accuracy under different attack settings: (a) PGD, (b) AA.
B.4. DCS at Different Position
We initially explored the accuracy DCS when implemented at various positions within a ResNet under PGD attacks, the results of which are illustrated Fig. Our findings indicate that the accuracy is relatively close to peak value in the first six layers. The accuracy the eighth layer is comparatively lower than at other positions. Consequently, in our major experiments, choose the second convolution layer to be replaced with the DCS layer.
B.5. Computational Overhead
Despite the difference in training time, their time consumption during inference very similar. The time gap training due to the introduction of additional parameters in the DCS layer. However, during inference, the extra time consumption caused by the extra parameters is not as dramatic training due to the absence of backpropagation. The results shown in Table 8.
Table 8. Training and reasoning overhead comparison. Training Overhead/min Reasoning Overhead/sec
Model
baseline DCS
213.47
5.94
379.96
6.67
B.6. Impact of Downsampling Layer Substitution
We compare the results by replacing the DCS to the downsampled layer (layer 6) with the closest normal convolution layer (layer 5, 7). The results are shown in Table 9.
Table Downsampling layer substitution.
Layer
PGD
AA
5
68. 12
70.72
6
72.03
75.74
7
69.36
72.
We notice that the performance DCS on downsampling layers better than the normal convolution layers. According the lemmas, larger stride S helps to reduce the assumed small numbers e, and er in the lemmas. It results in smaller gradient similarity and output distance, and finally increases the robustness and clean accuracy. In addition, larger S also helps the bounds in both lemmas to be numerically looser in practice. This helps to find suitable n easier.
16


Reviews:
[{"review_id": "CS7QYkRRLO", "reviewer": "Reviewer_L7Yc", "summary": "This paper introduces Deformable Convolution with Stochasticity (DCS), a defense method that injects randomness into convolutional layers by replacing fixed offsets with random masks, thereby creating a data-independent random space for deformed kernels. This paper provides a theoretical analysis using gradient cosine similarity to derive strict, data-independent bounds on the receptive field. This paper further enhances this approach with Gradient-Selective Adversarial Training (GSAT), which selectively masks pixels with similar gradient origins to reduce adversarial transferability. Extensive experiments on CIFAR and ImageNet demonstrate that DCS with GSAT significantly improves both clean and robust accuracy compared with other random defense methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your expert comments and your interest in the content of our experiments. We summarize and rebut your 6 major concerns in your comments.\n\n## Re 0. Evaluate under using BPDA+EOT attack.[Method,Experiment,W2,Q3]\nThanks for this nice concern. We evaluated DCS under BPDA and BPDA+EOT attacks on CIFAR-10, following the hyperparameters:\n\n    epsilon=8/255，max steps=20, learning rate=0.5, EOT steps=3.\n\nIn the final version, we will expand Table 6 to include:\n|Model|BPDA[1]|BPDA+EOT[1]|\n|:-:|:-:|:-:|\n|RN18|78.66|77.78|\n|WRN34|80.03|80.47|\n\nDCS is robust under BPDA+EOT attacks. We attribute this to randomness in conjunction with gradient masking.\n\n## Re 1. Both proofs assume certain distributions and independence properties of the gradients that might not hold exactly in practice.[Theorem]\nThanks for this nice concern. In practice, the data should be normalized. We assume the worst case of all **normalized** data to ensure that both lemmas stands in any data distribution after normalization. This does not mean that the worst case will necessarily occur. Therefore, in practice, the values of these two bounds is looser.\n\nWe would like to emphasize that the worst case assumptions in proofs are specified on data distributions with infinite data points. This avoids the dependence of the bounds on specific data sets. This is the reason why the DCS setup can be data-independent.\n\n## Re 2. Significant adaptations to apply DCS to other model architectures such as Transformers.[W1]\nThanks for this nice concern. We believe this question reveals the future direction in randomized structure defense.\n\n**(1)** We confirm that DCS is tailor-made for convolutional operations. We believe that convolution is still an important tool in image processing. Transformer is out of the research scope of this work.\n\n**(2)** However, we briefly explored that DCS fits Vision Transformer(ViT). ViT uses a $16\\times16$ convolution in patch embedding. Large kernel size makes lower bound of $n$ increases and hinders finding a suitable $n$. To avoid this, we notice that patch embedding can be split into multiple concatenated $3\\times3$ convolutions[2]. Our baseline follows the settings in [2] and then replace the second $3\\times3$ convolution with DCS. We obtained the following results\n|Method|PGD|\n|:-:|:-:|\n|ViT-t+Conv[2]|32.31|\n|ViT-t+DCS|**55.71**|\n\nDCS works well with ViT-tiny and we will add this experiment in appendix. We would like to argue that due to time constraints, the network is roughly trained on CIFAR-10 with two stages: \n\n**Stage 1**: Fix a pretrained ViT-tiny and train the FC layer and convolutions from scratch with:\n\n    -epochs: 200\n    -batch size: 128\n    -optimizer: SGD\n    -weight decay: 5e-4\n    -initial learning rate: 0.01\n    -scheduler: multiste (lr/10 at epoch 50 and 100)\n\n**Stage 2**: Adversarial finetune the entire network using GSAT with:\n\n    Stage 2:\n    -epochs: 90\n    -batch size: 128\n    -optimizer: SGD\n    -weight decay: 5e-4\n    -initial learning rate: 0.01\n    -scheduler: cosine\n\n## Re 3. How to ensure that the randomness introduced by DCS and GSAT is properly controlled?[Q1]\nThanks for this nice question.\n\n**(1)** For DCS, we control randomness by $n$ through Lemma1,2. Other setting in DCS are decided by the replaced convolution layer, to keep the data dimensions constant. The stride keeps the same. The kernel size is added by $2$, while padding is adapted accordingly.\n\n**(2)** For GSAT, the size of random space is smaller but **fixed** after removing the masks, since the number of removed masks is unchanged in each step.\n\n## Re 4. How to ensure robustness improvements are not a result of gradient masking?[Q2]\nThank you for your constructive comments on the additional experiment. To validate this, we manually cancel randomness. Instead, we selected two fixed deformable convolutional kernel $X^i$ and $X^j, n=4$ with no repeated points. $X^i$ is attacked and $X^j$ is used for reasoning. The results are\n|Method|PGD|\n|:-:|:-:|\n|DCN|52.23|\n|Fixed|54.71|\n|DCS|**62.93**|\n\nThe robustness is mainly from randomness. We will add this observation into the appendix.\n\n## Re 5. How is selective masking performed and how this impacts the clean Acc and robustness?[Q4]\nThanks for this nice concern. \n\n**(1)** According to Algorithm 1, GSAT record the included points in the DCS kernel when generating the adversarial training examples. Then, all masks that unmask the recorded points are banned, until the end of current forward propagation. With the other masks, DCS will be trained with kernels sampled in $\\mathbb{X}^g$ and $\\mathbb{X}^u$.\n\n**(2)** From Eq(7,10), $\\mathbb{X}^g$ and $\\mathbb{X}^u$ helps to enlarge the gradient cosine similarity and minimize the output distances, which leads to higher robustness and clean Acc.\n\n### Reference\n\n[1] Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples, ICML 2018.\n\n[2] Early convolu-tions help transformers see better, NIPS 2021.", "claims_and_evidence": "The claims made in the submission are well-supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "In general, the evaluation criteria make sense for robust classification problems. For example, CIFAR-10, CIFAR-100 and ImageNet are widely acknowledged as benchmark datasets in this field. In addition, this paper uses a wide variety of attack methods to evaluate the robustness of baseline methods. However, since the proposed method introduces randomness, it is necessary to evaluate the proposed method using BPDA+EOT attack.", "theoretical_claims": "I reviewed the proofs provided for Lemma 1, which establishes the data-independent upper bound on the receptive field n, and the outline for Lemma 2 regarding the lower bound related to output inconsistency. The proof for Lemma 1 appears to be internally consistent and follows standard techniques in bounding gradient cosine similarity, though it relies on worst-case assumptions (e.g., setting Cg = 1) that simplify the analysis.  One potential issue is that both proofs assume certain distributions and independence properties of the gradients that might not hold exactly in practice. Overall, the proofs are mathematically plausible.", "experimental_designs_or_analyses": "The experimental design was evaluated primarily on CIFAR and ImageNet datasets using standard architectures like ResNet18 and WideResNet34, and included tests under multiple attack scenarios. I checked the design of these experiments and found that they are sound in terms of using established datasets and widely accepted attack methods for benchmarking adversarial robustness. However, as mentioned before, it is necessary to evaluate the proposed method using BPDA+EOT attack.", "relation_to_broader_scientific_literature": "The paper builds on a rich body of literature in adversarial robustness by taking the idea of incorporating randomness to a structural level via deformable convolutions. Prior works (e.g., Xie et al., 2017; Li et al., 2019) have shown that randomness can hinder adversarial attacks, but they typically suffer from data dependency and require careful tuning of noise hyperparameters. This work extends those ideas by leveraging deformable convolutions (as introduced by Dai et al., 2017, and Zhu et al., 2018) to generate a randomized kernel space that is independent of input data. The proposed GSAT method also connects to and extends the body of research on adaptive adversarial training techniques by selectively mitigating gradient transferability issues, thereby refining existing defense strategies in a novel manner.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "The proposed method is interesting and novel, which is significantly different from defense methods that inject randomness into data inputs. However, the proposed method is tailored for convolutional layers, and therefore it may require significant adaptations to be applied to other model architectures such as Transformers. Furthermore, defenses that incorporate randomness are vulnerable to adaptive attacks like BPDA [1] combined with EOT, which are designed to estimate the gradients despite the stochasticity. The proposed method should be evaluated against BPDA+EOT to demonstrate its robustness.\n\n[1] Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples, ICML 2018.", "other_comments_or_suggestions": "None.", "questions_for_authors": "Regarding experimental design, how did you ensure that the randomness introduced by DCS and GSAT is properly controlled and that the robustness improvements are not a result of gradient masking? I am particularly wondering what is the robust accuracy of the proposed method under BPDA+EOT attack compared to the baseline methods. In addition, could you explain in more detail how the selective masking of pixels is performed during training and how this impacts the clean accuracy and robustness?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sXggQh4XUz", "reviewer": "Reviewer_DMZf", "summary": "This paper introduces deformable convolution with stochasticity (DCS) to enhance the adversarial robustness of deep neural networks. Unlike traditional random defense methods that inject randomness into input data, this work incorporates randomness directly into the network architecture by replacing fixed convolutional offsets with random masks. Through a theoretical analysis of the trade-off between robust accuracy and natural accuracy, the authors identify kernel size as a key factor in balancing this trade-off. Additionally, the paper proposes a new adversarial training strategy that enhances performance by selectively masking pixels. Experimental results across multiple datasets demonstrate that DCS achieves superior adversarial robustness and clean accuracy compared to existing baselines.\n\n### update after rebuttal\nMy concerns are well addressed during the rebuttal. I have updated my rating to 4: accept.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your pertinent review and your interest. Your review is summarize and rebut by 3 major concerns.\n\n## Re 0. Ablation study by replacing downsampling layers.[Experiment,Q1]\nThis is an interesting problem. Your subtle experimental design help us to study the effect of stride in a limited network structure. We compare the results by replacing the DCS to the downsampled layer (**layer 6**) V.S. the closest normal convolution layer (layer 5, 7). The results are shown in the table below. \n|Layer|PGD|AA|\n|:-:|:-:|:-:|\n|5|68.12|70.72|\n|**6**|**72.03**|**75.74**|\n|7|69.36|72.11|\n\nWe notice that the performance of DCS on downsampling layer is bettter than the normal convolution layers. According to the lemmas, larger sreide $S$ helps to reduce the assumed small numbers $\\epsilon_c$ and $\\epsilon_l$ in the lemmas. It results in a smaller gradient similarity and output distance, and finally increase the robustness and clean accuracy.\n\nIn addition, larger $S$ also helps the bounds in both lemmas to be numerically looser in practice. This helps to find a suitable $n$ easier.\n\n## Re 1. Explanation regarding the instability of GSAT in Table 3 and potential connection to the Equations.[W1,Q2]\nThis is a constructive question. We will first explain the source of instability in GSAT and with Eq. 13, and then analyse the potential connection of Table. 3 with Eq. 12.\n\n**(1)** According to Eq. 13, GSAT modifies the sample space sampled by the DCS layer in forward propagation during training by selection. Below we explain the reason for the instability of GSAT shown in Table 3.\n\nWe have demonstrated both theoretically and experimentally that paths in $\\mathbb{X}^g$ and $\\mathbb{X}^u$ are not easy to attack, while paths in $\\mathbb{X}^s$ are very easy to be attacked by gradient-based algorithms (see Eqs. 7, 10 and Sec. 5.3.3 for details). This means that when the network is attacked, the gradients generated by the paths in $\\mathbb{X}^g$ and $\\mathbb{X}^u$ will be in different direction from the attacked gradients generated by the paths in $\\mathbb{X}^s$.\n\nTo minimize the influence of inaccurate attacked gradients in training, we select and remove paths in $\\mathbb{X}^s$ from $\\mathbb{X}$, and build the selected random space, as demonstrated in Eq. 13. This corresponds to step 11 in Algorithm 1. The selected random space is used for sampling paths in the forward and backward propagation for each training step sepreately.\n\nWe understand that using GSAT the network are only limited optimized on the paths in $\\mathbb{X}^s$ under each adversarial examples. So when the DCS samples paths in $\\mathbb{X}^s$ in infernce, there are performance decrement compared to the other sampled paths. **The decrement will cause the instability in GSAT.** The larger probability the performance drop in DCS, the greater instability in GSAT.\n\n**(2)** For the relationship of results in Table 3 and Eq. 12, it mainly connected by $n$. From Table. 3, we can find that with larger $n$, GSAT will be more unstable. We give an explanation as: when Eq. 12 gives **larger bounds** for $n$, which refers to larger $n$ in practice, the percentage paths in unselected random space will increase. This increases the probability for DCS performance decrement in the networks using GSAT for training, which leads to greater instability. Thus, as shown in Table 3, $n$ is suggested to be small for a stable GSAT.\n\nThe explanation of instability will be added in the appendix and the analysis of connection between Table 3 and Eq. 12 will be added in experiment section in the final version to helping audiances in understanding the nature of GSAT.\n\n---\n\nIn addition, we would like to argue that the paths in $\\mathbb{X}^s$ are inherently difficult to optimize, given the difficulty of obtaining the correct gradients after a precision gradient-based attacked. Together with the results in Table 3 demonstrating that the network remains robust even with performance decrement. So we consider the performance decrement to be acceptable.\n\n## Re 2. Unreported robust accuracy of Non-AT baselines.[W2,Q3]\n\nThank you for your valuable suggestions.\n\nWe supplemented the results of the NON-AT baseline in the table below. Cos in the table means gradient cosine similarity.\n\n|Model|Clean|PGD|Cos|\n|:-:|:-:|:-:|:-:|\n|baseline w/o AT|95.12|0.04|1.00|\n\nWe will add these results as bars in Figure 4(a) in the final version.", "claims_and_evidence": "Yes, the claims are well-supported. The authors provide both empirical evidence and theoretical analysis to substantiate their findings.", "methods_and_evaluation_criteria": "Yes, the proposed method is well-motivated by the theoretical analysis. The evaluation criteria are consistent with those widely adopted in adversarial robustness studies.", "theoretical_claims": "Yes, I have verified the correctness of Lemma 1 and Lemma 2, and they appear to be sound.", "experimental_designs_or_analyses": "Yes, the experimental design and analyses are reasonable. However, there is a potential missing aspect regarding the experimental analysis of stride in Eq. 12. While the authors mention that S is set to a fixed value to ensure the same output feature map dimensions, downsampling layers exist in networks such as ResNet-18. These layers could be replaced by DCS to conduct a more thorough ablation study on the impact of stride.", "relation_to_broader_scientific_literature": "This paper builds upon research in deformable convolutions and random defense mechanisms. The application of deformable convolutions in adversarial robustness is a novel contribution, and the theoretical analysis of gradient similarity provides insights into the broader understanding of how randomness impacts model robustness.", "essential_references_not_discussed": "No critical references appear to be missing.", "other_strengths_and_weaknesses": "Strengths\n1. This paper introduces an innovative strategy by embedding randomness into the network architecture rather than relying on data augmentation or noise injection. This design addresses key limitations of traditional random defense methods, such as data dependency and hyperparameter sensitivity.\n2. The analysis of the trade-off between robust accuracy and natural accuracy provides valuable theoretical insights, potentially guiding the design of future random defense mechanisms.\n3. The experiments convincingly demonstrate that DCS outperforms existing random defense methods in both adversarial robustness and natural accuracy. The consistency of results across multiple datasets and architectures further reinforces the effectiveness of the approach.\n\nWeaknesses\n1. It is evident from Table 3 that GSAT is more unstable than standard adversarial training (AT). However, the paper does not provide a clear explanation for this instability. Since Algorithm 2 should be responsible for stability, and the only difference between GSAT and standard AT lies in Algorithm 1, further clarification is needed. Additionally, the connection between this instability and Eq. 12 is not well-discussed.\n2. The robust accuracy of Non-AT baselines is not reported in Figure 4(a). Including this information would provide a more comprehensive comparison.", "other_comments_or_suggestions": "None.", "questions_for_authors": "1. Can you conduct an ablation study by replacing downsampling layers in networks such as ResNet-18 with DCS to further analyze the effect of stride?\n2. Can you provide a more detailed explanation regarding the instability of GSAT observed in Table 3 and its potential connection to Eq. 12?\n3. Can you include the robust accuracy results of Non-AT baselines in Figure 4(a) for a more comprehensive comparison?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SCmALFeruU", "reviewer": "Reviewer_hpQK", "summary": "This paper proposes a random structural defense method called Deformable Convolution with Stochasticity (DCS) to improve adversarial robustness of convolutional neural networks. DCS replaces fixed convolutional kernels with randomly sampled deformable kernels to reduce adversarial transferability between inference paths in a data-independent way. The authors theoretically analyze the trade-off between robustness and clean accuracy in DCS and propose a Gradient-Selective Adversarial Training algorithm to further enhance robustness.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Your expert comments are constructive for our paper. We summarize and rebut your 4 major concerns in your comments.\n\n## Re 0. Claim of generalization and data independence equires empirical support.[Claims,W2]\nThank you for suggesting additional experiments to validate our claim. To verify the sensitivity of DCS hyperparameters to more distributions, we extended our experiments on STL-10[1] using ResNet18 as the baseline. The input size keeps the same as raw image as $96$. The results are shown in the following table:\n|Method|Clean|PGD|AA|\n|:-:|:-:|:-:|:-:|\n|baseline|61.95|38.65|35.49|\n|DCS|**62.40**|**42.25**|**47.95**|\n\nThis results show the effectiveness of DCS on different distributions without changing the settings. We will include this experiment in the final version.\n\n## Re 1. Additional experiments against attacks.[Evaluation,W1]\nThank you for helping us to refine our experiments.\n\n**(1)** We conducted adversarial robustness tests on CIFAR-10 using ResNet18 for query-based, transfer-based and adaptive attacks that you mentioned. We expanded the results to Table 6 as\n|Model|Clean|SQUARE(query-based)|Pixel(query-based)|trnasfer-FGSM(transfer-based)[2]|transfer-FGSM-base(transfer-based)[2]|BPDA(adaptive)[3]|BPDA+EOT(adaptive)[3]|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|RN18|89.11|80.53|86.32|84.95|65.71|78.66|77.78|\n|WRN34|90.58|82.45|86.31|90.53|68.34|80.03|80.47|\n\nIn the table, SQUARE and Pixel attacks are included in the initial paper. We added 4 columns to the table. The added attacks used the following settings:\n\n    transfer-FGSM:          base model=pretrained WRN50_2, epsilon=16/255\n    transfer-FGSM-base:     base model=pretrained baseline model, epsilon=8/255\n    BPDA:                   epsilon=8/255，max steps=20, learning rate=0.5\n    BPDA+EOT:               epsilon=8/255，max steps=20, learning rate=0.5, EOT steps=3.\n\n\nIt can be seen that DCS appears to be robust against black-box attacks. We attribute this to the fact that the random masking of the convolutional kernel by DCS also shields a portion of the adversarial perturbations. For ataptive attacks like BPDA and EOT-BPDA, DCS also shows high robustness. We attribute this to randomness in conjunction with gradient masking.\n\nWe will use the expanded Table 6 alone and analysis in the final version.\n\n**(2)** For attacks where adversarial samples are added to the training set, some predefined adversarial samples are added to the training set in both traditional adversarial training(AT) and GSAT. Related results of normal AT and GSAT can be found in Table 1 and Table 3.\n\n## Re 2. Connections to other relevant domains like certifiable defenses could be drawn.[RelatedWorks]\nThanks for this nice suggestion. We will add certifiable defense to Table 2 in the final version as:\n|Type|Method|CIFAR10||Imagenet||\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|||PGD|AA|PGD|AA|\n|Ceritied Defense|Cert-RA[4]|68.6|-|-|-|\n|Random Structure|DCS (ours)|**75.84**|75.46|52.38|66.79|\n\n## Re 3. Concern about computational overhead.[Q1]\nThanks for this nice concern. We recorded the training and reasoning times for DCS+GGSAT vs. baseline+AT. We choose RseNet18 as the baseline. For training, we use CIFAR-10 training-set (50000 examples) following the hyperparameters:\n    \n    epochs: 200\n    batch size: 128\n    optimizer: SGD\n    weight decay: 5e-4\n    initial learning rate: 0.1\n    scheduler: multiste (lr/10 at epoch 60 and 120)\nFor reasoning, we use CIFAR-10 test-set (10000 examples) with batch size=1024. We recorded the entire time cost for training and reasoning in the table below.\n|Model|Training Overhead/min|Reasoning Overheal/sec|\n|:-:|:-:|:-:|\n|baseline|213.47|5.94|\n|DCS|379.96|6.67|\n\nDespite the difference in training time, their time consumption during inference is very similar. The time gap in training is due to the introduction of additional parameters in the DCS layer. However, during inference, the extra time consumption caused by the extra parameters is not as dramatic as training due to the absence of backpropagation.\n\nWe are delighted to add this comparison in the appendix in the final version.\n\n### Reference\n\n[1] An analysis of single-layer networks in unsupervised feature learning, Journal of Machine Learning Research - Proceedings Track 15, 215–223 (01 2011).\n\n[2] Explaining and harnessing adversarial examples, in International Conference on Learning Representations, 2015.\n\n[3] Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples, ICML 2018.\n\n[4] Certified Defenses for Adversarial Patches, ICLR 2020.", "claims_and_evidence": "The main claims are mostly supported by adequate evidence in terms of theoretical derivations and experimental results. However, the claim of generalization and data independence equires more empirical support beyond CIFAR and ImageNet.", "methods_and_evaluation_criteria": "The white-box robustness evaluations against PGD and AutoAttack are reasonable, but additional experiments against black-box and especially adaptive attacks would give a more complete picture.", "theoretical_claims": "I checked the proofs in Appendix A.", "experimental_designs_or_analyses": "The experimental setup for evaluating adversarial robustness against PGD and AutoAttack is mostly sound.", "relation_to_broader_scientific_literature": "The authors discuss several categories of related methods, including input/feature randomization, structure randomization, and stochastic networks. However, connections to other relevant domains like certifiable defenses could be drawn.", "essential_references_not_discussed": "The related work section covers the most relevant prior work.", "other_strengths_and_weaknesses": "Weakness:\n1. Rather than PGD and AA, the paper lacks sufficient evaluation against other important and updated classes of attacks. In particular, more results are needed against common transfer-based and query-based black-box attacks, adaptive attacks specifically designed for randomized defenses, and attacks that incorporate adversarial examples in the training set. \n\n2. While the data-independent framework is a key advantage, the paper lacks sufficient empirical evaluation of DCS's generalization to different datasets beyond CIFAR-10/100 and ImageNet. More extensive experiments on a diverse range of datasets would be necessary. \n\n3. The computational efficiency and training/inference cost of DCS is not sufficiently addressed. Replacing fixed convolutions with randomly sampled deformable convolutions could incur nontrivial computational overhead.", "other_comments_or_suggestions": "The paper makes some good contributions in developing a randomized structural defense and providing theoretical insights, but has significant limitations in terms of generalization experiments, attack evaluations, analysis of computational efficiency and novelty.", "questions_for_authors": "No", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qc1wBqVLsw", "reviewer": "Reviewer_7KWx", "summary": "This paper introduces DCS (Defensive Convolution with Stochasticity), a novel adversarial defense method that integrates randomness directly into convolution operations to obscure gradient directions. By embedding stochasticity within the network architecture, DCS enhances robustness against both white-box and black-box attacks. The authors provide theoretical analysis and experimental validation across multiple datasets and adversarial settings.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detialed comments and your interest in the content of our experiments. We summarize and rebut your 8 major concerns in your comments.\n\n## Re 0. The notion of \"data independence\" needs clarification.[Claims,Q3]\nThank you for correcting our statement. The \"data independence\" means the hyperparameters at the DCS layer are data independent. However, the DCS layer still needs to be trained. We will revise it in the final version.\n\n## Re 1. Clearer training process and hyperparameter choices.[Evaluation]\nThank you for this nice concern. The hyperparameters of the experiments for all baselines and datasets are listed separately in the Sec. 5.1. In terms of the locations of replaced DCS layer, we conduct an ablation study on DCS locations in Sec. B.3, where we changed the location of DCS and find that the second layer is the best for DCS. For clarification, we will add this setting to Sec. 5.1. in the final version as:\n\n- In our experiments, unless specifically labeled, DCS replaces the second convolutional layer. All other layers keep the original settings.\n\n## Re 2. Explanation of Lemma 2.[Theorem,Supplementary]\nThank you for your advice. We noticed 2 major concerns for the explanation of Lemma 2, we will improve them in the appendix as follows. \n\n**(1)** We note that there is no explanation for the symbols $\\lfloor\\cdot\\rfloor$ and $\\lceil\\cdot\\rceil$. This will be explained in line 570 when it first appears as:\n\n- $\\lfloor\\cdot\\rfloor$ for rounding down and $\\lceil\\cdot\\rceil$ for rounding up.\n\n**(2)** Regarding the proof of Lemma 2, we note that the step from Eq. (23) to Eq. (24) need to be expanded:\n\nFor the 3 terms to the right hand side in Eq. 23, the goal is to leave only $n$ into the numerator part. Thus the relationship between $p_u$ and $n$ can be clear. We loose the terms by considering $-\\lceil\\cdot\\rceil$ as $-(\\cdot)$ and $\\lfloor\\cdot\\rfloor$ as $(\\cdot)$\n\n- For the first term $$pu \\leqslant 1-\\frac{n\\alpha^2}{\\lceil\\alpha\\lceil^2k^2}$$\n- For the second term $$p_u\\leqslant 1-(\\frac{m^2\\alpha^2}{k^2}+\\frac{n\\alpha}{k^2\\lceil\\alpha\\rceil}-\\frac{m^2\\alpha}{k^2\\lfloor\\alpha\\rfloor})\\leqslant 1-(\\frac{n\\alpha}{k^2\\lceil\\alpha\\rceil})$$\n- For the third term $$p_u\\leqslant 1-\\frac{n\\alpha^2}{k^2\\lfloor\\alpha\\rfloor^2}$$\n- We set the strictest bounds for $p_u$. Bringing in $\\alpha=\\frac{k}{S}$, the third term is found to be the strictest: $1-\\frac{n}{S^2\\lfloor\\alpha\\rfloor^2}$, as is shown in Eq. 24.\n\nAfter bringing the bounds in Eq. 24 into Eq. 22 and simplify it, Eq. 25 can be obtained. After transforming the rounding up and rounding down to the corresponding limits, Eq. 26 can be obtained. Finally, considering $\\Delta^g \\in [0,1]$, Eq. 26 become Eq. 27, which is consistent with Lemma 2.\n\n## Re 3. Terminology[Experiment,W1]\nThank you for the constructive suggestion. We will change \"pixel\" to \"point\" when coresponding to the DCS kernels in the final version.\n\n## Re 4. Explanation of Figure 1.[Experiment,Q2]\nThank you for your constructive comments. The circle marked as $\\mathbb{K}$ in Figure. 1 represents the potential distribution of all DCS kernels. DCS automatically samples a random kernel from K at each forward propagation.\n\n## Re 5. Connections to other relevant domains.[RelatedWorks,Q4]\nThank you for your valuable comments. We have compared DCS against other stochastic adversarial defenses methods in Table 2. We divided the stochastic adversarial defenses into random weights and random structures. We are happy to add certified defenses and stochastic activation functions to the table and cite related work. Table 2 will then be expanded as the following table (only expanded part and ours).\n|Type|Method|CIFAR10||Imagenet||\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|||PGD|AA|PGD|AA|\n|Ceritied Defense|Cert-RA[1]|68.6|-|-|-|\n|Random Structure|stochastic activation functions[2]|67.4|-|-|-|\n||DCS (ours)|**75.84**|75.46|52.38|66.79|\n\n## Re 6. Is Step 4 trained alternately or in a unified manner?[Q1]\nThank you for your valuable questions. To avoid misunderstanding, we first claim that the randopm sampling processes in Figure 1 are not trainable. \n\nFor the sampled kernels, we trained each kernel alternatively. Only one kernel is trained during each forward propagation, regardless of whether it is normal adversarial training or GSAT. Therefore we consider this to be alternating training.\n\n## Re 7. Would combining DCS with adversarial training further improve robustness?[Q5]\nYes. AT will further improve robustness. Corresponding results were compared in our experiments by Sec. 5.3.2 and Fig. 4(a). The results show that using AT or GSAT imporves the performance of DCS.\n\n### Reference\n\n[1] Certified Defenses for Adversarial Patches, ICLR 2020.\n\n[2] Adversarial Defense Via Data Dependent Activation Function and Total Variation Minimization, ICLR 2019.", "claims_and_evidence": "The paper claims that DCS effectively mitigates gradient-based attacks, does not require post-training modifications, generalizes well across architectures, and is data-independent. These claims are supported through theoretical derivations, empirical results, and ablation studies. However, the notion of \"data independence\" needs clarification, as it appears to apply to hyperparameters rather than the full training process.", "methods_and_evaluation_criteria": "The experiments are well-designed, covering a diverse range of attacks, datasets, and architectures. Evaluation metrics focus on robust accuracy, standard accuracy, and computational complexity. The approach is rigorous, but clearer documentation of the training process and hyperparameter choices would improve reproducibility.", "theoretical_claims": "The theoretical foundation is strong, but some areas need further elaboration. Lemma 2, in particular, lacks sufficient explanation, and its derivation should be expanded. Mathematical notation could also be refined to enhance clarity.", "experimental_designs_or_analyses": "The empirical results effectively demonstrate the effectiveness of DCS. However, terminology could be improved—using \"pixel\" to describe both feature maps and convolution kernels may cause confusion. Figure 1 also needs more explanation, especially regarding the meaning of the circles between Step 3 and Step 4.", "relation_to_broader_scientific_literature": "The paper situates itself well within adversarial defense research but could better contrast DCS with other stochastic defenses, such as randomized smoothing or stochastic activation functions. Citing additional references on these topics would strengthen the discussion.", "essential_references_not_discussed": "none", "other_strengths_and_weaknesses": "A major strength is the seamless integration of adversarial defense within convolution operations, eliminating the need for post-hoc modifications. The evaluation is comprehensive, and the theoretical contributions are meaningful. However, issues with terminology, theoretical clarity, and figure explanations should be addressed.", "other_comments_or_suggestions": "Clarifying the training process and improving the explanation of key theoretical components would enhance the paper’s impact. More precise terminology and a clearer interpretation of Figure 1 would also help avoid confusion.", "questions_for_authors": "Is Step 4 trained alternately or in a unified manner? Do the circles between Step 3 and Step 4 in Figure 1 represent an expansion of the potential distribution range? What aspect of the method is truly \"data-independent\"? \n\nHow does DCS compare with other stochastic adversarial defenses? \n\nWould combining DCS with adversarial training further improve robustness?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "n/a"}}]

Author rebuttal:

Meta-review: This paper creatively addresses adversarial robustness issues by designing a new module for neural networks. This contribution is new and interesting to the field of adversarial machine learning, which all reviewers agree. Although some concerns are raised in the initial comments, all of them are addressed by the authors during the rebuttal. 

One limitation of this paper is that this paper considers CNN only (as one reviewer is concerned). However, this limitation cannot weigh too much based on its novel contributions to the field of adversarial machine learning.
Decision: Accept (poster)