IRAD: Implicit Representation-driven Image Resampling against Adversarial Attacks
Authors: Yue Cao
Tianlin Li
Xiaofeng Cao
Ivor Tsang
Yang Liu
Qing Guo
Venue: ICLR 2024 poster (2024)
Keywords: adversarial defense
resampling
implicit representation
Primary area: general machine learning (i.e., none of the above)

Abstract: We introduce a novel approach to counter adversarial attacks, namely, image resampling. Image resampling transforms a discrete image into a new one, simulating the process of scene recapturing or rerendering as specified by a geometrical transformation. The underlying rationale behind our idea is that image resampling can alleviate the influence of adversarial perturbations while preserving essential semantic information, thereby conferring an inherent advantage in defending against adversarial attacks. To validate this concept, we present a comprehensive study on leveraging image resampling to defend against adversarial attacks. We have developed basic resampling methods that employ interpolation strategies and coordinate shifting magnitudes. Our analysis reveals that these basic methods can partially mitigate adversarial attacks. However, they come with apparent limitations: the accuracy of clean images noticeably decreases, while the improvement in accuracy on adversarial examples is not substantial.We propose implicit representation-driven image resampling (IRAD) to overcome these limitations. First, we construct an implicit continuous representation that enables us to represent any input image within a continuous coordinate space. Second, we introduce SampleNet, which automatically generates pixel-wise shifts for resampling in response to different inputs. Furthermore, we can extend our approach to the state-of-the-art diffusion-based method, accelerating it with fewer time steps while preserving its defense capability. Extensive experiments demonstrate that our method significantly enhances the adversarial robustness of diverse deep models against various attacks while maintaining high accuracy on clean images.

Published as a conference paper at ICLR 2024
IRAD: IMPLICIT REPRESENTATIIN-DDRIVE IMAGE RE- SAMPLING AGAINST ADVERSARIAL ATTACKS
Yue Caol.2 Tianlin Li?
Xiaofeng Cao Ivor Tsang Yang Liu
Qing Guo
1 CFAR and IHPC, Agency for Science Technology and Research (A*STAR), Singapore
2 School of Computer Science and Engineering, Nanyang Technological University, Singapore
3 Jilin University, China
ABSTRACT
We introduce a novel approach to counter adversarial attacks, namely, image resam- pling. Image resampling transforms a discrete image into a new one, simulating the process of scene recapturing or rerendering as specified by geometrical trans- formation. The underlying rationale behind our idea is that image resampling can alleviate the influence of adversarial perturbations while preserving essen- tial semantic information, thereby conferring an inherent advantage in defending against adversarial attacks. To validate this concept, present comprehen- sive study on leveraging image resampling to defend against adversarial attacks. We have developed basic resampling methods that employ interpolation strate- gies and coordinate shifting magnitudes. Our analysis reveals that these basic methods can partially mitigate adversarial attacks. However, they come with ap- parent limitations: the accuracy of clean images noticeably decreases, while the improvement in accuracy on adversarial examples is not substantial. We propose implicit representation-ddivy
image resampling (IRAD) to overcome these limi- tations. First, we construct an implicit continuous representation that enables us to represent any input image within continuous coordinate space. Second, we introduce SampleNet, which automatically generates pixel-wise shifts for resam- pling in response to different inputs. Furthermore, diffusion based method, accelerating it with fewer time steps while preserving its defense capability. Extensive experiments demonstrate that our method significantly enhances the adversarial robustness of diverse deep models against various attacks while maintaining high accuracy on clean images. We released our code in https: /ttp//gtttttt///////ee//ete.hhet ceriihub.co//ee
extend approach
the
com/ ts wwomm/ttinggggneeeee inggguo/irrd.
INTRODUCTION
Adversarial attacks can mislead powerful deep neural networks by adding optimized adversarial perturbations clean images (Croce & Hein, 2020b; Kurakin al., 2018; Goodfellow al., 2014; Guo et al., 2020; Huang et al., 2023a), posing severe threats to intelligent systems. Existing works enhance the adversarial robustness of deep models by retraining them with the adversarial examples generated on the fly (Tramèr et al.. 2018; Shafahi et al., 2019; Andriushchenko & Flammarion, 2020) or removing the perturbations before processing them (Liao et al., 2018; Huang et al., 2021b; Ho & Vasconcelos, 2022; Nie et al., 2022). These methods assume that the captured image is fixed. Nevertheless, in the real world, the observer can see the scene interest several times via different observation ways since the real world is continuous space and allows observers resample the signal reflected from the scene. This could benefit the robustness of the perception system. We provide illustrative example in Fig. 1 (a): when image taken from particular perspective is subjected to an attack that misleads the deep model (e.g., ResNet50), by altering the viewing way, the same object can be re-captured and correctly classified. Such a process is also known as image resampling (Dodgson, 1992) that transforms a discrete image into a new one, simulating the process of scene recapturing or rerendering as specified by a geometrical transformation. aim to study using image resampling to enhance the adversarial robustness of deep models against adversarial attacks.
this work, we
"Qing Guo is the corresponding author (singgguo@iicce.org

---

Published as a conference paper at ICLR 2024
(a) Resampling in the Roal World
Advrenta Domple
(a) shows the pipeline resampling in the real world and the corresponding predictions, which inspires our main idea. Two main questions must be solved for the resampling-hased solution (i.e., QI and Q2). (b) shows the pipeline of the proposed method and two solutions (i.e,, SI and S2) to address the two questions.
Figure 1:
Corp Hudioa Caree
92: scene based on adv. csample?
Shvet Dun
pattern?
Resampling Solution
ToPt Ugget X
51:
implicit Rep. for Reconstruct.
SampleNet for Sampling
Continuous
Representation
To achieve the image resampling with a given adversarial example, we must address two key ques- tions: how to reconstruct the continuous scene based on the input? And how do we estimate the transformation that can climinate the adversarial perturbation effectively? Note that, with a single- view input, we can only simulate the 2D transformations. To this end, first provide a general formulation for the resampling-base -bsed adversarial defense. With this formulation, we design several basic resampling methods and conduct comprehensive study validate their effectiveness and limitations for adversarial defense. Then, we propose an implicit representation-drin method (IRAD). Specifically, we first construct an implicit continuous representation for reconstruc- tion, which enables us to represent any input image within a continuous coordinate space. Second, we introduce SampleNet, which automatically generates pixel-wise shifts for resampling in response to different inputs. Furthermore, we can extend our approach to the state-of-the-a diffusion- based method, accelerating it with fewer time steps while preserving its defense capability. We conduct extensive experiments on public datasets, demonstrating that our method can enhance the adversarial robustness significantly while maintaining high accuracy on clean images.
resampling
2 RELATED WORK
Image Resampling, Resampling transforming a discrete image, defined at one set of coordinate locations, to a new set of coordinate points. Resampling can be divided conceptually into two processes: reconstructing the discrete image to a continuous image and then sampling the interpo- lated image (Parker al., 1983). Among the existing reconstruction functions, nearest neighbor interpolation and bilinear interpolation are the most frequently adopted (Han, 2013/03). Nearest neighbor interpolation assigns the value of the nearest existing pixel to the new pixel coordinate, whereas bilinear interpolation calculates the new pixel value by taking a weighted average of the surrounding pixels in a bilinear manner. The resampling process involves refactoring pixels in the input image, which allows us to explore new approaches for mitigating adversarial attacks. Our paper investigates the potential of resampling to break malicious textures from adversarial inputs, which has not been studied in the community.
Ad Adversarial Attack and Defense. White box attacks assume the attacker has full knowledge of the target model, including its architecture, weights, and hyper- syper-parameters. This allows the attacker to generate adversarial examples with high fidelity using gradient-based optimization techniques, such as FGSM (Goodfellow et al., 2014), BIM (Kurakin et al., 2018). PGD (Madry et al., 2017), and others (Huang et al., 2023b). Other attacks also include black box attacks like Square Attack (Andriushchenkoo et al., 2020) and patch-wise attacks (Gao et al., 2020), as well as transferability- based attacks (Liu et al., 2016; Wang & He, 2021; Wang et al., 2021). AutoAttack (Croce& & Hein, 2020b) has been proposed as more comprehensive evaluation framework for adversarial attacks. AutoAttack combines several white box and black box attacks into a single framework and evaluates the robustness of model against these attacks.
2

---

Published as a conference paper at ICLR 2024
Adversarial defense can be categorized into two main types: adversarial training and adversarial purification (Nie et al., 2022). Adversarial training involves incorporating adversarial samples during the training process (Goodfellow et al., 2014; Madry et al., 2017: Athalye et al., 2018; Rade & Moosavi-Dezfole 2021; Ding et al., 2018; Zhang et al., 2020a; Jia et al., 2022b), and training with additional data generated by generative models (Sehwag et al., 2021). On the other hand, adversarial purification functions as a separate defense module during inference and does not require additional training time for the classifier (Guo al., 2017; Xu et al., 2017; Sun al., 2019; Ho & Vasconcelos, 2022).
IMAGE RESAMPLING (IR) AGAINST ADVERSARIAL ATTACK
3.1 PROBLEM FORMULATION
Given a dataset D with the data sample X E X and its label y y. the deep supervised learning
model tries to learn a mapping or classification function F(-) : : + y The model F(-) could be different deep architectures. Existing works show that deep neural networks are vulnerable to
adversarial perturbations (Madry et al., 2017). Specifically, a clean input X, an adversarial attack is to estimate a perturbation which is added the X and can mislead the F(.),
F(X') * y, subject to ||X - X'II <<
(I)
where || is a distance metric. Commonly, || is measured by the normn (p E {1, 2, x0 }), and denotes the perturbation magnitude. We usually name X' as the adversarial example of X. There are two ways to enhance the adversarial robustness of F(-). The first is to retrain the F(.) with the adversarial examples estimated on the fly during training. The second is to process the input and remove the perturbation during testing. In this work, we explore a novel testing-time adversarial defense strategy, Le., IMAGE RESAMPLING, to enhance the adversarial robustness of deep models.
Image resampling (IR). Given an input discrete image captured by a camera in a scene, image resampling is to simulate the re-capture of the scene and generate another discrete image (Dodgson, 1992). For example, we can use a camera to take two images in the same environment but at different time stamps (See Fig. 1). Although the semantic information within the two captured images is the same, the details could be changed because the hands may shake, the light varies, the camera configuration changes, etc. Image resampling uses digital operations to simulate this process and is widely used in the distortion compensation of optical systems, registration of images from different sources with one another, registration of images for time-evolution analysis, television and movie special effects, etc. In this work, we propose to leverage image resampling for adversarial defense, and the intuition behind this idea that resampling in the real world could keep the semantic information of the input image while being unaffected by the adversarial textures (See Fig. 1). We introduce the naive implementation of IR in Sec. 3.2 and discuss the challenges for adversarial defense in Sec. 3.3.
3.2 NAIVE IMPLEMENTATION
Image resampling contains two components, i.e., reconstruction and sampling. Reconstruction is to build a continuous representation from the input discrete image, and sampling generates new discrete image by taking samples off the built representation (Dodgson, 1992). Specifically, given a discrete image I R Hxx x3 X or X', we will design a reconstruction method to get the continuous representation of the input image, which can be represented as
RECONS(I),
(2)
where & denotes the continuous representation that can estimate the intensity or color of arbitrarily given coordinates that could be non-integer values; that is, we have
Ca,e
0(u, v)
where Ca,e denotes the color of the pixel at the continuous coordinates [u, v). With the reconstructed do we sample the coordinates of all desired pixels and generate another discrete image by
i j] Cu. ((U(i,)) subject to, U = G + SAMPLER.
(4)
3

---

Published as a conference paper at ICLR 2024
where G and U, both in R xxxx2 share the same dimensions the input and con- sist of two channels. The matrix G stores the discrete coordinates of individual pix- j. Each element in Uii.j] nates of the desired pixel in a continuous representation. This will be placed at the
els, i... G[ū,j]
[ui Usyl denotes the coordi-
[i,]] location in the output and is calculated by adding the Sampler-predicke shift to G. By leveraging image resampling to sim- ulate the re-capture of the interested scene for adversarial defense, we pose two requirements: ! The reconstructed continuous representation is designed to represent the interested scene in a continuous space according to the input adversarial example and should elimi- nate the effects of adversarial perturba- tion. @ The sampling process should break the adversarial texture effectively while preserving the main semantic in- formation. Traditional naive resam- pling methods can hardly achieve the above two goals.
Rackege
Ruchaye
Bua Rariegee
Rv Ruawrf
Row trad
russin ad images bsaed on the iiputs
Figure 2: Comparison of different sampling strategies based on the bilinear interpolation as the reconstruction method.
Image resampling via bilinear interpolation. We can set the function RECONS in Eq. (2) in such a way: assigning the bilinear interpolation as the for arbitrary input images. Then, for an arbitrary given coordinates [u. v)., we formulate Eq. (3) as
(5) where [ -.-'', [e, Jtt [st: - '1, and [i Jt 'I are the four neighboring pixels around [u, e] in the image and {wi, W2, Wa, wa) are the bilinear weights that are calculated through four coordinates and are used to aggregate the four pixels.
Ca. = olu, v)) = wi Ti Je we [ : + wa wit' .o _o'' w .5t1
Image resampling via nearest interpolation. Similar to bilinear interpolation, we can the function RECONS in Eq. (2) in such a way assigning the nearest interpolation as the ø for arbitrary input images. Then, for an arbitrary given coordinates [u, v], we formulate Eq. (3) as
Ca, = Φu,, v) lliu: Aol.
(6)
where Jo] is the integer coordinate nearest the desired coordinates [u, v).
For both interpolation methods, we can set naive sampling strategies as SAMPLER function:
Spatial-invariant (SI) sampling, that is, shift all raw coordinates along fixed distance d (See 3rd column in Fig. 2):
U= G + (AMPLER(d), subject to, Uli.j]] = G[i.i]+ [d,
(7)
Spatial-variant (SV) sampling, that is, we randomly sample a shifting distance r for the raw coordinates
U = G + SAMPLER(7). suujeee to, U[r,J] = G(i,j] + [di, d2), di, d2 E E((0,)).
(8)
where (00 is uniform distribution with the minimum and maximum being O and 7 respectively
(See 4th column in Fig. 2). We can set different ranges to see the changes in adversarial robustness. We can use the two reconstruction methods with different sampling strategies against adversar- ial attacks. We take the CIFARIO dataset and the WideResNNe22-110 (Zagoruyko& & Komodakis, 2016) as examples. We train the Wer28 10 on CIFARIÓ dataset and calculate the clean test- ing dataset's 'ss accuracy, also known as the standard accuracy (SA). Then, conduct the AutoAttack (Croce & Hein, 2020b) against the WideResNNt28- 10 on all testing examples and calculate the accuracy, denoted as the robust accuracy (RA). We employ image resampling to process the input, which can be the clean image or adversarial ex- ample, and the processed input is fed to the WideResNer28-10. Then, we can calculate the SA
Table 1: Comparison of naive image resampling strategies on CIFARIO via AutoAttack (E 8/255).
Naive methods Stand. Acc.
Robust Ave.
Avc.
wo,
IR( SAMPLERi 1.5) |
85.24
42,30
63.77
IR (bil, SaMmarry = 1.5)
|
53.81
30.24
42,03
IR (nca SAMPLER(d 1.5)
94.68
47,81
IR(nes, SAMpLer(y = 1.5))
56.47
26.27
41.37
4

---

Published as a conference paper at ICLR 2024
and RA of WideResNet28-- 10 with image resampling. We evaluate the effectiveness of the two naive reconstruction methods, i.e., bilinear interpolation and nearest interpolation with the sam- pling strategies defined in Eq. (7) and Eq. (8), which are denoted as IR SAMPLER(d ?)) and IR (nea, SAMPIERR (d or 7), respectively. More details about the dataset, the model architecture, and the adversarial attack AutoAttack are deferred to Sec. 6.
3.3 DISCUSSIONS AND MOTIVATIONS
Based on the findings presented in Table 1, the following obser- Bilinear in- terpolation with SI-sampling (i.e., (bil,, SAMPLER = 1.5))) signif- icantly enhances robust accuracy, al- beit at the expense of modest reduc- tion standard accuracy. In contrast, using nearest interpolation with SI- sampling results in only marginal vari- ations in SA and RA. @ For both inter- polation methods, SV-sampling leads to notable increases in RA, while si-
Predicied shitting
Resampled coondinates
vations emerge:
U
Sampkeii
Fiui
Por
Figure 3: Pipeline of the proposed IRAD.
multaneously causing significant reduction in SA. Overall, we see some effectiveness of leveraging naive image resampling methods against adversarial attacks. Nevertheless, such methods are far from being able to achieve high standard and robust accuracy at the same time. The reasons are that the naive interpolation-be sampling strategies are not designed to preserve the semantic information. As the example shown in Fig. 2. we feed the adversarial example to the bilinear interploration- basiiddd--bdddddeeee method with SI-sampling and SV-sampling, respectively, and use the Grad-CAl Madd to present the semantic variations before and after resampling. Clearly, the two sampling strategy do not preserve the original semantic information properly. To address the issues, propose novel IR method in Sec. 4.
reconstruction methods could not remove the perturbations while the
4 IMPLICIT CONTINUOUS REPRESENTATION-DRWVES RESAMPLING
4.1 OVERVIEW
As analyzed in the previous section, we identified the limitations of naive resampling methods that can hardly achieve the two requirements for the reconstruction function and sampling function mentioned in Sec. 3.2. We propose implicit representation-diie gap. Specifically, we employ the implicit representation for the reconstruction function and train SampleNet to automatically predict the shifting magnitudes according to different input images based on the built implicit representation. We display the whole process in Fig. (b) and details in Fig. 3.
image resampling (IRAD) to fill the
4.2 IMPLICIT REPRESENTATION
Given an input image I E IR #xx'3 = X or X' that could be clean image or adversarial example, we first employ the local image implicit representation (Chen al., 2021) construct the continuous representation for the input image. Specifically, we calculate the pixel-wise embedding of the I via a deep model and get F FEAT(() where F(k: I) denotes the embedding of the pixel [k; I]. Given a desired coordinate [u, v). predict the color of [u, v] based on the F(k, () and the spatial distance between [k; (] and [u, v). that is, we can formulate the Eq. (3) as
Cu. = ((,,
Σ [KJ]EEN...
(F I), dist( (w,, w). (k)
where Nu, is pixel set that contains the neighboring pixels around [u, v). and the function dist(-) is to measure the spatial distance between [u, v) and [k. The function (p() is a multilayer perceptron (MLP) and predicts the color of the pixel [u. e) according to the embedding of the pixel [bs, and their spatial distance. The key problem becomes how to train the deep model FEAT(.) and the MLP In this work, study four prediction tasks train models:
5

---

Published as a conference paper at ICLR 2024
Clean2Clean. Given clean image I, we reconstruct each pixel by feeding its raw coordinates Eq. (9) and get the reconstruction I. Then, we use T.1 (T, to train the FRAT(-) and MLP.
Super-resolution. We can also train the model via the super-resolution task as done in (Chen et al., 2021). That is, given row-resolttion input I that is downsampled from a clean image I, we aim to generate a higher resolution by sampling more coordinates and feeding them to Eq. (9), thus we can get a large-size image Inc- We can also use the loss function L1 (I, Inr) to train the model.
Inpainting. We generate a corrupted image Iaak by masking the clean image I, we use Eq. (9) to restore the missing contents and get Imask. We can also use the L1 (I, Imssk) loss to train the model. Gaussian denoising & Adversarial denoising. We add clean images with random Gaussian noise or adversarially generated noise. Thus, we can get nosee We aim to remove the noise via Eq. (9), and train the model via (I, n.ose).
We can test the trained models on different training tasks for adversarial defense and find that the model trained with adversarial denoising performs the best. For more details, please see Sec. 6.3.
4.3 SAMPLENET
Instead of the heuristic sampling strategies in Sec. 3.2, we propose to automatically predict the pixel-wise shifting according to the embedding of the input image. Intuitively, we aim to train a network, i.e., SampleNet, which can output the shifting for all pixels (i.e., U in Eq. (4)) eliminate the adversarial perturbation further effectively, that is, we formulate Eq. (4) as
U(i,5) SAMPLER(I) (ii + SAMPLENET (F (i. (i,j)),
(10)
where F FIA G a matrix containing the original coordinates, for example, G(L, [L, 1]. SAMPLENET is an MLP that takes the feature of pixel [ and the coordinate values as input and predicts its shifting directly. After training the implicit representation, we fix the FEAT and yp in Eq. (9) and train the SAMPLENET via the adversarial denoise loss function. We visually compare the naive sampling strategies and the SampleNet in Fig. 2. The deep model can predict correctly on the resampled adversarial example with our SampleNet, while other sampling strategies cannot.
4.4 IMPLEMENTATIO DETAILS
We follow the recent work (Chen et al., 2021) and set the deep model in (Lim et al., 2017) as the FEAT(-) to extract the embedding of the I. We set sp(.) and the SAMPLENET as five-layer MLP, respectively. We utilize the adversarial denoising task (See discussion in Sec. 6.3 ) to train the models through two-stage training strategy. Specifically, we first train the implicit representation (i.e., FEAT(-) and (-)) on the training dataset, and then we fix their weights and train the SAMPLENET for sampling. Note that we follow a black-box setup: that is, we train our model based on adversarial examples crafted from R8 and test the effectiveness on other deep models (See experimental section). Please refer to the Appendix for other training details.
RELATIONSHIP AND EXTENSION TO SOTAS
In the following, we discuss the relationship between our method and DISCO (Ho & Vasconcelos, 2022), and we also present naive extension of our method to DiffPure (Nie et al., 2022), which could speed up DiffPure five times with similar defense performance.
Relationship to implicit representation-basee method (e.g., (Ho & Vasconcelos, 2022)). (Ho & Vasconcelos, 2022) employ implicit representation (Chen al., 2021) to remove the adversarial perturbation and can enhance the robust accuracy under attacks significantly while preserving the high accuracy on the clean images. Different to (Ho & Vasconcelos, 2022), we employ the implicit representation (Chen et al., 2021) as a part of image resampling and study the influences of different training tasks. More importantly, our method contains the SampleNet that can automatically predict the suitable pixel-wise shifting and further recover the semantic information. We demonstrate the effectiveness of our method over (Ho & Vasconcelos, 2022) in Sec. 6.1.
Extension to diffusion-based method (e.g., (Nie et al., 2022)). DiftPure (Nie et al., 2022) utilizes the diffusion model to purify the adversarial perturbation, which presents impressive results even though DiffPure is involved in the attacking pipeline. Nevertheless, DiftPure adopts large time steps

---

Published as a conference paper at ICLR 2024
Table 2: Comparison on CI CIFARIO0 and ImageNet via AutoAttack (Foo = 8/255 for CIFAR and
CIFAR 100, 6oo = 4/255 for ImageNet). indicates no corresponding pre-trained model in the original paper.
Cifar10
Cifar100
ImageNet
Detense
SA
RA
Avg.
SA
RA
Avg.
SA
RA
Avg.
W.O. Defense
94.77
47.39
81.66
3.48
42.57
76.72
38,36
Bit Reduction (Xu 2017)
92.66
1.05
46.86
74.47
6.56
40.52
73.74
1.88
37.81
Joeg (Dziugaite al., 2016)
83,66
50.79
67.23
60.87
38.36
49.62
73.28
33,96
53.62
Rundomization (Xic et al., 2017)
93.87
6,86
50.37
78.7
10.25
44.48
74.04
19.81
46.93
Median Filter
79.66
42.54
61.10
57.32
31.18
44.25
71.66
17.59
44.63
NRP (Naseer al., 2020)
92.89
3.82
48.36
77.17
12.67
44.92
72.52
20.40
46,46
STL (Sun et ul., 2019)
90.65
57.48
74.07
72.62
32.88
52.75
DISCO (Ho & Vasconcelos, 2022)
89.25
85.63
87.4 .4
72.58
68.52
70.55
72.66
68.26
70.46
DiffPure (Nie et al., 2022)
89.67
87.54
68.04
68.16
TRAD
91.70
89.72
76.01
72.49
74.25
72.14
71.60
71.87
Table 3: Comparison on CIFARIO, CIFARI00, and ImageNet via BPDA.
Cifar10
Cifar100
ImageNet
Delense
SA
RA
Avg.
SA
RA
Avg.
SA
RA
Avg.
W.O. Defense
94.77
0.02
47.40
81.67
0.49
41.08
76.72
0.00
38,36
Reduction (Xu 2017)
92.66
1.40
46.53
74.47
0.62
73.74
0.00
36.87
Jpeg (Dziugaite al., 2016)
83.65
5.50
44.58
60.87
5.79
33.33
73.28
0.04
36.66
Randomization (Xic et al., 2017)
94.05
34.81
64.43
78.85
21.03
49.94
74.06
27.92
$0.99
Median Filter
79.66
25.12
52.39
57.32
11.52
34.42
71.66
0.02
35.84
NRP (Nascer al., 2020)
92.89
0.27
52.39
77.17
0.46
38.82
72.52
0.02
36,27
STL (Sun et al., 2019)
90.65
2.10
46.38
72.62
0.02
36.32
DISCO (Ho & Vasconcelos, 2022)
89.25
22.60
55.93
72.58
16.90
44.74
72.44
0.34
36,39
DiffPure (Nic et al., 2022)
89.15
87.06
88.11
68.85
61.42
65.14
IRAD
91.70
74.32
83.01
76.00
62.78
72.14
71.12
71.63
(i.e., 100 time steps) to achieve good results, which is time-consuming: each image requires 5 seconds to limit the influence of adversarial perturbations for ImageNet images. We propose to use our method to speed up DiffPure while preserving its effectiveness. Specifically, we use DiftPure with 20-time steps to process input images and feed the output to method with implicit representation as the reconstruction method and SampleNet as the sampler. As demonstrated in Sec. 6.1, our method combined with DiffPure achieves 5 times faster than the raw DiffPure with similar standard accuracy and robust accuracy.
EXPERIMENTAL RESULTS
In this section, we conduct extension experiments to validate our method. It is im- portant to note that the results presented in each case are averaged over three experi- ments to mitigate the influence of varying random seeds. These experiments were con- ducted using the AMD EPYC 7763 64-Core Processor with
NVIDIA A100 GPUs.
Metries. We evaluate IRAD and baseline methods on both clean and their respective adversarial examples, measuring their performance terms of standard accuracy (SA) and robustness accuracy (RA). Furthermore, we compute the average of SA and RA as a comprehensive metric.
Table 4: IRAD generaliza- tion across DNNs on Cl- FAR10.
DNN#
SA
RA
Avg
Datasets. During training and evaluation of IRAD, We use three datasets: CIFARIO (Krizhevsky et al., a), CIFAR 100 (Krizhevsky et al., b) and ImageNet (Deng al., 2009). To make training datasets of the same size, we randomly select 50000 images in ImageNet, which includes 50 samples for each class. IRAD is trained on pairs of adversarial and clean training images generated by the PGD attack (Madry et al., 2017) on ResNet18 (He et al., 2016). The PGD attack uses an e value of 8/255 and 100 steps, with a step size 2/255. The SampleNet is also trained using adversarial-clean pairs generated by the PGD attack. In this part, we mainly present part of the results; other results will be shown in the Appendix. Attack scenarios. 1 Oblivious adversary scenario: We follow setups in RobustBench (Croce et al., 2021) and use the AutoAttack is an ensemble of several white-box and black-box attacks, including two kinds of PGD attack, the FAB attack (Croce& & Hein, 2020a), and the square attack (Andriushchenko et al., 2020), allowing for a more comprehensive evaluation. In addition to AutoAttack, we also report the results against FGSM (Goodfellow et al., 2014), BIM (Kurakin et al., 2018), PGD (Madry et al., 2017), RFGSM (Tramèr et al., 2018), TPgd (Zhang et al., 2019b), APgd (Croce & Hein, 2020b), EotPgd (Liu et al., 2018), FFgsm (Wang al., 2020). MiFgsm (Dong al., 2018), and Jitter (Schwinn al., 2021).
WRNZ8- 10
91.70
39.72
90.71
WRN70-16
VGG16
91.53
90.50
91.02
ResNe134
91.16
88.22
89
the main attack method for the defense evaluation since AutoAttack
7

---

Published as a conference paper at ICLR 2024
We evaluate methods by utilizing adversarial examples generated through victim models that are pre- tre-trained on clean images. 2 Adaptive adversary scenario: We consider a challenging scenario where attackers know both the defense methods and classification models. Compared to the oblivious adversary, this is particularly challenging, especially regarding test-time defense (Athalye et al., 2018; Sun et al., 2019; Tramer al., 2020). To evaluate the performance, we utilize the BPDA method (Athalye et al., 2018) that can circumvent defenses and achieve high attack success rate, particularly against test-time defenses. @ AutoAttack- and the target model as a whole and use AutoAttack to attack the whole process.
Adaptive adversary scenario. We regard the IRAD
Baelines. We compare with 7 representative resting-time adversarial defense methods as shown in Table 2 including 2 SOTA methods, i.e,, DISCO (Ho & Vasconcelos, 2022) and DiffPure (Nie et al., 2022). Note that, we run all methods by ourselves with their released models for a fair comparison. We also report more comparisons with training-time methods in the Appendix.
Victim models. We use adversarial attacks against the WideResNet28-- 10 (WRN28-10) on CIFARIO and CIFAR 100 and against ResNet50 on ImageNet since we do not find pre-trained WRN28- 10 available for ImageNet. Then, we employ compared methods for defense for the main comparison study. Note that our model is trained with the ResNet18, which avoids the overfitting risk the victim model. In addition, we also report the effectiveness of our model against other deep models.
6.1 COMPARING WITH SOTA METHODS
Oblivious adversary scenario. We compare IRAD and baseline methods across the CIFARIO. CIFARIOO, and ImageNet datasets. With Table 2, we observe that: 1 IRAD achieves the highest RA among all methods, especially when compared to the SOTA methods (e.g., DISCO (Ho & Vasconcelos, 2022) and DifiPure (Nie et al., 2022)). This demonstrates the primary advantages of the proposed method for enhancing adversarial robustness. @ IRAD is also capable of preserving high accuracy on clean data with only a slight reduction in SA compared to "w.o. defense'. IRAD achieves the highest average accuracy among all methods across the three datasets. This demonstrates the effectiveness of the proposed method in achieving a favorable trade-off between SA and RA. O Following the relationship between DISCO and
Table S: Comparison on CIFARIO via AutoAttack- based adaptive adversary.
IRAD as discussed in Sec. 5, the substantial advan-
tages of IRAD over DISCO underscore the effective- ness of the proposed SampleNet. Adaptive vdve-- sary scenario. We further study the performance of all methods under more challenging scenario where the attack is aware of both the defense methods and classification models and employs BPDA (Athalye et al., 2018) as the attack. As shown in Table 3, we have the following observations: 1 RAs of all methods reduce under the BPDA. For example, DISCO gets 85.63% RA against AutoAttack but only achieves 22.60% RA under BPDA on CIFARIO. The RA of our method reduces from 89.72% to 74.32% on CIFAR10. @ Our method achieves the highest RA among all compared methods on all three datasets, with the exception of DiffPure in C0f Although the RA of our method reduces, the relative improvements over other methods become much larger. DiftPure's 's high performance comes at the expense of significant computational resources. Therefore, we suggest a hybrid approach that combines the strengths of DiffPure and IRAD, as detailed in the following section. @ Our method still achieves the highest average accuracy when compared to all other methods, except for DifiPure in CI highlighting the advantages of IRAD in achieving a favorable trade-off between SA and RA under the adaptive adversary scenario. AutoAttack-Baa also explored a tough defense scenario where attackers have full knowledge of defense methods and models. In Table 5, we found that DISCO and our original IRAD didn't 't improve robustness with zero RA. DifiPure maintains high SA and RA under adaptive settings but takes 132 ms per image, which is time-consuming To speed up DiffPure, we When we combined IRAD with DiffPPure (See Sec. 5), our IRAD+DifiPure (t=20) method achieved comparable SA and RA with DiffPure, and it's about five times faster. More experiments comparing different steps in the integration of IRAD and DiffPure are in Appendix A.6.
SA
RA
Avg.
Cost (ms)
DifiPure
89.73
75.12
82.43
132.8
DISCO
89.25
44,63
0.38
IRAD
91.70
0
43.85
0.68
DiffPure (t=20)
93.66
8.01
50.83
27.25
IRAD+DilPPure (1=20)
93.42
74.05
83.74
27.70
Adaptive adversary scenario. We
fewer time steps, but this reduces RA significantly.
6.2 GENERALIZATION ACROSS DNNs, DATASETS, AND OTHER ATTACKS
Our IRAD is trained with the adversarial examples crafted from ResNet18 on respective datasets and we aim to test its generalization across other architectures and datasets. Generalization across
8

---

Published as a conference paper at ICLR 2024
Figure 4: WRN28- 10's robust accuracy under 10 attacks with DISCO and IRAD on three datasets.
DNNs. In Table 2, we test the ResNetl IRAD on WRN28- 10. Here, we further test it on the other three architectures, i.e., WRN70-16, VGGI6Bn, and ResNet34. As shown in Table 4, IRAD trained on ResNet18 could achieve similar SA and RA when we equip it to different DNNs, which demonstrates the high generalization of our method. Generalization across Datasets. We utilize the IRAD trained on one dataset to defend against attacks executed on a different dataset. For instance, we train IRAD CIFARIO with ResNer18 and employ it counter AttokAttac when it attacks WRN28-10 on 100. As depicted in Table 6, our IRAD exhibits exceptional cross-dataset generalization. In other words, the IRAD model trained on CIFARIO achieves comparable SA and RA as the IRAD model trained on CIFAR 100, even when tested on the CIFAR 100 dataset itself. Generalization against other attacks. We further use IRAD trained via PGD-based adversarial examples against 10 attack- ing methods and compare with the SOTA method DISCO. As shown in Fig. 4, we have the following observations: ① In the CIFARIO 10 and ImageNet datasets, IRAD achieves similar RA across all attack methods, which demonstrates the high generalization of IRAD across diverse attacks and even unknown attacks. Regarding the CIFAR dataset, IRAD presents slightly lower RA under MiFGSM than other attacks. @ IRAD consistently achieves significantly higher robust accura- cies (RAs) across all attack scenarios, underscoring its advantages. Detailed results and additional experiments on more attacks can be found in Appendix A.3 and A.4.
Table 6: IRAD trained on the dataset to defend the attacks on another dataset.
Training Testing CIFARIO CIFARTON
CIFARTO
89.72
67.14
CIFAR100
88.68
72.49
6.3 A ABLATION STUDY
Training strategies comparison. As detailed in Sec. 4.2, we can set different tasks to pre-train the IRAD. In this subsection, we compare the results of IRADs trained with different tasks on CIFARIO datasets against AutoAttack. As shown in Table 7, we see that: O IRAD trained with PGD-based denoising achieves the highest RA and aver- age accuracy among all variants. IRAD with Gaussian denoising task gets the second best results but the RA reduces significantly. @ IRAD trained
Table 7: Comparison of implicit representa- tion training strategies on Cifar
Training Strategy
RA
Avg.
Clean 2Clean
93.09
0.10
46.60
Super Resolution
93.05
0.20
46.63
Restoration
93.06
46.61
Denoising (Gaussian)
89.75
24.40
57.08
Denoising (PGD)
89.59
76.69
83.14
with other tasks has higher SAs than IRAD with denoising task but their RAs are close to zero Sampling strategies comparison. We replace the SampleNet of IRAD with two naive sampling strategies introduced in Sec. 3.2 and evaluate the performance on the CIFARIO dataset via Autokt (Ex0 8/255). As shown in Table 8, we see that SampleNet can achieve the highest SA, RA, and average accuracy, which demonstrates the effectiveness of our SampleNet. Additional results regarding the ablation study and the effectiveness analysis can be found in the Appendix.
7 CONCLUSION
We have identified novel adversarial defense solution, i.e., im- age resampling, which can break the adversarial textures while maintaining the main semantic information in the input image. We provided general formulation for the image resampling- based adversarial defense and designed several naive defensive resampling methods. We further studied the effectiveness and limitations of these naive methods. To fill the limitations, we proposed the implicit continuous representationn-diiven image resampling method by building the implicit representation and designing a SampleNet that can predict coordinate shifting magnitudes for all pixels according to different inputs. The experiments have demonstrated the advantages of our method over all existing methods. In the future, this method could be combined with the other two defensive methods, i.e., denoising and adversarial training, for constructing much higher robust models.
Table 8: IRAD with different sampling strategies on CIFAR 10.
Sampling Stratogies
RA
Avg.
wo. Sampling
89.71
84.60
87.15
SAMPLERId 1.5)
47.35
30.54
SAMPLER(T = 1.5)
82.19
77.90
80:05
SampleNet
91.70
89.72
90.71
9

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENT
This research is supported by the National Research Foundation, Singapore, and DSO National Laboratories under the AI Singapore Programme (AISG Award No: ALSG2-GCC2003-008) and Career Development Fund (CDF) of the Agency for Science, Technology and Research (A *TARR) (No.: C233312028), Xiaofeng Cao is supported by the National Natural Science Foundation of China (Grant Number: 62206 (00). The research is also supported by the National Research Foundation, Singapore, and the Cyber Security Agency under its National Cybersecurity R&D Programme INCRP25-P04-TAICEN. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views the National Research Foundation, Singapore and Cyber Security Agency of Singapore.
REFERENCES
Sravanti Addepalli, Samyak Jain, Gaurang Sriramanan, and Venkatesh Babu Radhakrishnan. Towards achieving adversarial robustness beyond perceptual limits. 2021. 18, 19
Sravanti Addepalli, Samyak Jain, and R Venkatesh Babu. Efficient and effective augmentation strategy for adversarial training. arXiv preprint arXiv:2110.15318 2022.
Motasem Alfarra, Juan C Pérez, Adel Bibi, Ali Thabet, Pablo Arbeláez, and Bernard Ghanem. Clustr: Clustering training for robustness. arXiv preprint (tl::2006.076882 2020. 19
Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast adversarial training. Advances in Neural Information Processing Systems, 3 16048-16059, 2020. 1, 19
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: a query-efficient black-box adversarial attack via random search. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIII, pp. 484-501. Springer, 2020. 2.7
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning, pp. 201-272-20 283. PMLR, 2018. 3,
Matan Atzmon, Niv Haim, Lior Yariv, Ofer Israelov, Haggai Maron, and Yaron Lipman. Controlling neural level sets. Advances in Neural Information Processing Systems, 32, 2019. 19
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. Advances in neural information processing systems, 32, 2019. 18
Alvin Chan, Yi Tay, Yew Soon Ong, and Jie Fu. Jacobian adversarially regularized networks for robustness. arXiv preprint arXiv: 191...008. 2019. 19
Erh-Chung Chen and Che-Rung Lec. Ltd: Low temperature distillation for robust adversarial training. arXiv preprint arXiv2211..02311, 2021. 18
Jinghui Chen, Yu Cheng, Zhe Gan, Quanquan Gu, and Jingjing Liu. Efficient robust training via backward smoothing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36. pp. 6222-6230, 2022. 19
Tianlong Chen, Sijia Liu. Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From supf-super pre-training to fine-tuning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 699-708, 2020. 19
Yinbo Chen, Sifei Liu, and Xiaolong Wang. Learning continuous image representation with local implicit image function. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 8628-86688888 8638868,, 2021. 5
James W Cooley and John W Tukey. An algorithm for the machine calculation of complex fourier series. Mathematics of computation, 29900), 1965. 21
10

---

Published as a conference paper at ICLR 2024
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In International Conference on Machine Learning, pp. 2196-2205, PMLR, 2020a.
7
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference machine learning, pp. PMLR, 2020b. 1, 1, 7
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammar- ion, Mung Chiang, Prateek Mittal, and Matthias Hein. RobustBench: a standardized adversarial robustness benchmark. In Thirty-fifth Conference Neural Information Processing Systems Datasets and Benchmarks Track, 2021. URL SSKZPJCE7B.
Jiequan Cui, Shu Liu, Liwei Wang, and Jiaya Jia. Learnable boundary guided adversarial training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 15721-15730, 2021. 18
Sihui Dai, Saeed Mahloujifar, and Prateek Mittal. Parameterizing activation functions for adversarial robustness. In 2022 IEEE Security and Privacy Workshops (SPW), pp. 30-87. IEEE, 2022. 17
Edoardo Debenedetti, Vikash Sehwag, and Prateek Mittal. A light recipe train robust vision transformers. arXiv preprint arXiv:2209.07399, 2022. 18
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248 255. Ieee, 2009.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Mma training: Direct input space margin maximization through adversarial training. arXiv preprint arXiv:1812.02637, 2018. 3, 19
Neil Anthony Dodgson. Image resampling. Technical report, University of Cambridge, Computer Laboratory, 1992. 1,3
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9185-9193, 2018. 7
Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M Roy. A study of the effect of ipg compression on adversarial images. arXiv preprint arXiv:1608, (00.:660800853, 2016.
Logan Engstrom, Andrew Ilyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library), 2019. URL https: / com Batrm//MddyyLabbbbb wttpy///aabbbbuutsss...... 19
Lianli Gao, Qilong Zhang, Jingkuan Song. Xianglong Liu, and Heng Tao Shen. Patch-wise attack for fooling deep neural network. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVIII 16, pp. 307-322. Springer, 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014. 1, 7,
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593, 2020. 17,
Sven Gowal, Sylvestre- Alvise Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy A Mann. Improving robustness using generated data. Advances in Neural Information Processing Systems, $4:4218-42333 2021. 17, 18
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten. Countering adversarial images using input transformations, arXiv preprint arXiv: 11011/11111111771 17, 2017.
11

---

Published as a conference paper at ICLR 2024
Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jian Wang, Bing Yu, Wei Feng, and Yang Liu. Watch out! motion blurring the vision of your deep neural networks. Advances in Neural Information Processing Systems, 33:975-9855 2020.
Dianyuan Han. Comparison of commonly used image interpolation methods. In Proceedings of the 2nd International Conference Computer Science and Electronics Engineering (ICCSEE 2013), pp. 1556-1559, Atlantis Press, 2013/03. ISBN 978-90-78677-61-1 doi: 10.2991/iccsee. 2013.391. URL www.thtpph........m /tteip:://di... http://ooooooooooo org /ttrgg/1..999 2991 1 ccsee 2013 3
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition,
pp. 770-778, 2016.
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. In International Conference on Machine Learning, pp. 2712-2721. PMLR, 2019. 18
Chih-Hui Ho and Nuno Vasconcelos. Disco: Adversarial defense with local implicit functions. arXiv preprint arXiv:2212.056300 2022. 3, 6, 8,8
Alain Hore and Djemel Ziou. Image quality metrics: Psnr vs. ssim. In 2010 20th international conference on pattern recognition, pp. 2366-2369, IEEE, 2010. 21
Hanxun Huang, Yisen Wang, Sarah Erfani, Quanquan Gu, James Bailey, and Xingjun Ma. Explor- ing architectural ingredients of adversarially robust deep neural networks. Advances in Neural Information Processing Systems, 34:5545-5559, 2021a. 17
Lang Huang, Chao Zhang, and Hongyang Zhang. Self-adaptive training: beyond empirical risk minimization. Advances in neural information processing systems, 33:19365-19331 2020. 18
Shihua Huang, Zhichao Lu, Kalyanmoy Deb, and Vishnu Naresh Boddeti. Revisiting residual networks for adversarial robustness: An architectural perspective. arXiv preprint arXiv:2222.11005 2022. 17
Yihao Huang, Qing Guo, Felix Juefei-Xu, Lei Ma, Weikai Miao, Yang Liu, and Geguang Pu. Advfllte::
predictive perturbation-awe aware filtering against adversarial attack via multi-domain learning. In Proceedings of the 29th ACM International Conference Multimedia, pp. 395-403, 2021b. I
Yihao Huang, Yue Cao, Tianlin Li, Felix Juefei-Xu, Di Lin, Ivor W Tsang, Yang Liu, and Qing Guo. On the robustness of segment anything. arXiv preprint arXiv:2305. 16220, 2023a.
Yihao Huang, Liangru Sun, Qing Guo. Felix Juefei-Xu, Jiayi Zhu, Jincao Feng, Yang Liu, and Geguang Pu. Ala: Naturalness-awe -waare aware adversarial lightness attack. In Proceedings of the 31st ACM
International Conference Multimedia, pp. 2418-2426 2023b. 2
Yunseok Jang, Tianchen Zhao, Seunghoon Hong, and Honglak Lee. Adversarial defense via learning to generate diverse attacks. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2740-2749, 2019. 19
Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang, and Xiaochun Cao. Las-at: adversarial training with learnable attack strategy. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 13398-13400 13408, 2022a. 18
Xiaojun Jia, Yong Zhang, Baoyuan Wu, Jue Wang, and Xiaochun Cao. Boosting fast adversarial training with learnable adversarial initialization. IEEE Transactions on Image Processing, 31: 4417-4430, 2022b. 3
Charles Jin and Martin Rinard. Manifold regularization for adversarial robustness, arXiv preprint arXhv:2003.04288 1, 2020. 19
Qiyu Kang, Yang Song, Qinxu Ding, and Wee Peng Tay. Stable neural ode with lyapunov- equilibrium points for defending against adversarial attacks. Advances in Neural Information Processing Systems, 34:14925-14937, 2021. 17
12

---

Published as a conference paper at ICLR 2024
Ashkan Khakzar, Soroosh Baselizadeh, Saurabh Khanduja, Christian Rupprecht, Seong Tae Kim, and Nassir Navab. Neural response interpretation through the lens of critical pathways. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 13528-13533 2021. 22
Jungeum Kim and Xiao Wang. Sensible adversarial learning. 2020. 19
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced
research). URL http: / www cs. toronto. wwweeeuuurreeeeee 2 cifar. html. 7
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-100 (canadian institute for advanced research). b. URL http: /www cs toronto...... edu/ ~kriz 2/ cifar.. wat....m..... 7
Nupur Kumari, Mayank Singh, Abhishek Sinha, Harshitha Machiraju, Balaji Krishnamurthy, and
Vineeth Balasubramanian. Harnessing the vulnerability of latent layers in adversarially trained models. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 2779-2785 2019. 19
Souvik Kundu, Mahdi Nazemi, Peter A Beerel, and Massoud Pedram. A tunable robust pruning framework through dynamic network rewiring of dnns. arXiv preprint arXiv:201 1.03083, 2020. 19
Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security, pp. 99-112. Chapman and Hall/CRC, 2018. 1, 2,
Tianlin Li, Aishan Liu, Xianglong Liu, Yitao Xu, Chongzhi Zhang, and Xiaofei Xie. Understanding adversarial robustness via critical attacking route. Information Sciences, 547:568-578 2021. 22
Tianlin Li, Qing Guo, Aishan Liu, Mengnan Du, Zhiming Li, and Yang Liu. Fairer: fairness as
decision rationale alignment. In International Conference on Machine Learning, pp. 19471-19489,
PMLR, 2023. 22
Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1778-1787, 2018.
Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 136-144, 2017. 6
Xuanqing Liu, Yao Li, Chongruo Wu, and Cho-Jui Hsieh. Adv-bnn: Improved adversarial defense through robust bayesian neural network. arXiv preprint arXi::1810.01277 2018. 7
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. arXiv preprint arXiv:1611.02770, 2016. 2
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: / 170.00608, 2017. 3, 7, 19
Chengzhi Mao, Ziyuan Zhong, Junfeng Yang, Carl Vondrick, and Baishakhi Ray. Metric learning for adversarial robustness. Advances in Neural Information Processing Systems, 32, 2019. 19
Seyed-Mohsee Moosavi- Moosavi-Dezfooli, Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard. Robust- ness via curvature regularization, and vice versa. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 9078-9988 9086,90866100000 2019. 19
Aamir Mustafa, Salman Khan, Munawar Hayat, Roland Goecke, Jianbing Shen, and Ling Shao. Adversarial defense by restricting the hidden space of deep neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3385-3394, 2019. 19
Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. A self-supervised approach for adversarial robustness. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 262-271, 2020. 7
13

---

Published as a conference paper at ICLR 2024
Weili Nie, Brandon Guo, Yujia Huang. Chaowei Xiao, Arash Vahdat, and Anima Anandkumar.
Diffusion models for adversarial purification. arXiv preprint arXiv:2205.0744 2022. 1, 3, 6, 7,
Tianyu Pang, Kun Xu, Yinpeng Dong, Chao Du, Ning Chen, and Jun Zhu. Rethinking softmax cross-entropy loss for adversarial robustness. arXiv preprint urXiv:1905 10626, 2019. 19
Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. Bag of tricks for adversarial training. arXiv preprini arXiv:2010.00467, 2020a. 18
Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Jun Zhu, and Hang Su. Boosting adversarial training with hypersphere embedding. Advances in Neural Information Processing Systems, 33: 7779-7792, 2020b. 18
Tianyu Pang, Min Lin, Xiao Yang, Junyi Zhu, and Shuicheng Yan. Robustness and accuracy could be reconcilable by (proper) definition. In International Conference on Machine Learning, 2022. 17
J Anthony Parker, Robert V Kenyon, and Donald E Troxel. Comparison of interpolating methods for image resampling. IEEE Transactions on medical imaging, 2(1):31-39, 1983. 2
Sheng Yun Peng, Weilin Xu, Cory Cornelius, Matthew Hull, Kevin Li, Rahul Duggal, Mansi Phute, Jason Martin, and Duen Horng Chau. Robust principles: Architectural design principles for adversarially robust cnns. arXiv preprint arXiv:2308. 16258, 2023. 17
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial robustness through local linearization. Advances in Neural Information Processing Systems, 32, 2019. 18
Yuxian Qiu, Jingwen Leng. Cong Guo, Quan Chen, Chao Li, Minyi Guo, and Yuhao Zhu. Adver- sarial defense through network profiling based path extraction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4777-4786, 2019.
Rahul Rade and Seyed-Mohser Moosavi-Dezfooll Helper-based adversarial training: Reducing excessive margin achieve better accuracy vs. robustness trade-off. In ICML 2021 Workshop on Adversarial Machine Learning, 2021. 3, 17, 18
Sylvestre-Alviss Rebuffi, Sven Gowal, Dan A Calian, Florian Stimberg, Olivia Wiles, and Tim- othy Mann. Fixing data augmentation to improve adversarial robustness. arXiv preprins arXiv:2/03.01946, 2021. 17, 18
Leslie Rice, Eric Wong, and 7ico Kolter. Overfitting in adversarially robust deep learning. In International Conference Machine Learning, pp. 8093-8104. PMLR, 2020. 18
Leo Schwinn, René Raab, An Nguyen, Dario Zanca, and Bjoern Eskofier. Exploring misclassification of robust neural networks to enhance adversarial attacks. arXiv preprint 1rXiv:2105.10304, 2021.
Vikash Sehwag, Shiqi Wang, Prateek Mittal, and Suman Jana. Hydra: Pruning adversarially robust neural networks. Advances in Neural Information Processing Systems, 19119955519666, 2020. 18
Vikash Sehwag, Saeed Mahloujifar, Tinashe Handina, Sihui Dai, Chong Xiang, Mung Chiang, and Prateek Mittal. Robust learning meets generative models: Can proxy distributions improve adversarial robustness? arXiv preprint arXiv:2104.09443, 2021. 3, 17, 18
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! Advances in Neural Information Processing Systems, 32, 2019. 1, 19
Chawin Sitawarin, Supriyo Chakraborty, and David Wagner. Improving adversarial robustness through progressive hardening. arXiv preprint arXiv:2003.09347, 4(5), 2020. 19
Kaustubh Sridhar, Oleg Sokolsky, Insup Lee, and James Weimer. Improving neural network robust- ness via persistency excitation. In 2022 American Control Conference (ACC), pp. 1521-1526. IEFE, 2022. 17
14

---

Published as a conference paper at ICLR 2024
Bo Sun, Nian-hsuan Tsai, Fangchen Liu, Ronald Yu, and Hao Su. Adversarial defense by stratified convolutional sparse coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattemn Recognition, pp. 11447-11456, 2019. 3, 8
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Bonch, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. In International Conference Learning Representations, 2018. 1,7
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. Advances in neural information processing systems, 33:1633-1645, 2020. 8
Jonathan Uesato, Jean- Ban--Btt Alayrac, Po-Sen Huang, Robert Stanforth, Alhussein Fawzi, and Pushmeet Kohli. Are labels required for improving adversarial robustness? arXiv preprini arXiv:1905.13725, 2019. 18
Jianyu Wang and Haichao Zhang. Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6629-6638, 2019. 19
Xiaosen Wang and Kun He. Enhancing the transferability of adversarial attacks through variance tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1924-1999999 1933, 2021. 2, 20
Xiaosen Wang, Xuanran He, Jingdong Wang, and Kun He. Admix: Enhancing the transferability of adversarial attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
2021. 2, 20
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representationss 2020. 7,
Yulong Wang, Hang Su, Bo Zhang, and Xiaolin Hu. Interpret neural networks by identifying critical data routing paths. In proceedings of the IEEE conference computer vision and pattern recognition, pp. 8906-8914, 2018. 22
Zekai Wang, Tianyu Pang, Chao Du Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. arXiv preprint N.Xi::2302.046388 2023. 17
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600-612, 2004. 21
Eric Wong, Leslie Rice, and Zico Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprint arXhv:2001.0399, 2020. 19
Boxi Wu, Jinghui Chen, Deng Cai, Xiaofei He, and Quanquan Gu. Do wider neural networks really help adversarial robustness? Advances in Neural Information Processing Systems, 34:7054-7067, 2021. 17
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust gen- cralization. Advances in Neural Information Processing Systems, 33:2958-2963 2020. 17, 18
Chang Xiao, Peilin Zhong, and Changxi Zheng. Enhancing adversarial defense by k-winners-tad take-all arXiv preprint arXiv: 1905. 2019. 19
Cihang Xie, Jianyu Wang. Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. arXiv preprint 1711.0 1.79911,19911.111111111 2017.
Xiaofei Xie, Tianlin Li, Jian Wang. Lei Ma, Qing Guo, Felix Juefei-Xu, and Yang Liu. Npe: Neuron path coverage via characterizing decision logic of deep neural networks, 2022. 22
15

---

Published as a conference paper at ICLR 2024
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.01155, 2017. 3,7
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint wrXiv:1605.071444 2016. 4
Chongzhi Zhang, Aishan Liu, Xianglong Liu, Yitao Xu, Hang Yu, Yuqing Ma, and Tianlin Li. Interpreting and improving adversarial robustness of deep neural networks with neuron sensitivity. IEEE Transactions on Image Processing, 30: 10:111111304, 2020a. 3
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong- You only propagate once: Accelerating adversarial training via maximal principle. Advances in Neural Information Processing Systems, 32, 2019a. 19
Haichao Zhang and Jianyu Wang. Defense against adversarial attacks using feature scattering- adversarial training. Advances in Neural Information Processing Systems, 32, 2019. 19
Haichao Zhang and Wei Xu. Adversarial interpolation training: A simple approach for improving model robustness, 2020. In URL lttps://openrrviiw. net/forum. 19
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pp. 7472-7482. PMLR, 2019b. 7, 18
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In International conference on machine learning, pp. 11278-1 1287. PMLR, 2020b. 18
Jingfeng Zhang, Jianing Zhu, Gang NIu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geometry-awan instance-reweiigted adversarial training. arXiv preprini arXiv:2010.01736. 2020c. 18
Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018. 21
A APPENDIX
A. EXPERIMENT SETTINGS IN DETAIL
In this section, we will provide a detailed description of the experiment settings.
For both implicit representation and SampleNet training of IRAD, pairs of clean and adversar- ial data. We generate image pairs under the PGD attack using ResNet18 for CIFARIO, CIFARIOO 100 and ImageNet as the target model. The PGD attack employs value of 8/255 and 100 steps, with step size of 2/255.
For implicit representation training, we use Adam as the optimizer with a learning rate of lc-4 and betas of (0, 0.9) as parameters. Throughout the training process, we use LI loss to update the model, and the training conducted with a batch size of 128.
For SampleNet training, we utilize cross-entropy loss, considering the prediction results of both clean and attacked images as the loss function. For optimization, we employ the Adam optimizer with a learning rate of 2e-4 for CIFARIO and ImageNet, le-3 for CIFAR 100, and betas set to (0, 0.9). The training SampleNet conducted with batch size of 400 for CIFARIO 10, 200 for CIFAR100, and for ImageNet.
A.2 COMPARING WITH MORE SOTA METHODS ON CIFAR 10 UNDER AUTOATTACK-BASED ADAPTIVE ADVERSARY SCENARIO
In this section, we present additional results from various methods comparison Cifar10 under AutoAttack-ban based adaptive adversary scenario. As presented Table 9, we present the baseline results from RobustBench, and compare them with the result provided on Table 5 and Table 16.
16

---

Published as a conference paper at ICLR 2024
From Table 9, conclude that IRAD+DifiPure method proposed has advantages existing SOTA methods on RobustBench.
Table 9: Comparison on CIFARIO via AutoAttack (cxo 8/255).
Defense
SA
RA
Avg.
Classifier
w.o. Defense
94.77
0
47.39
WideResNet-28-
10
Robust_Pent (Peng et al., 2023)
93.27
71.07
82.17
RaWideResNer-
70-16
Letter_Wang (Wang al., 2023)
93.25
70.69
81.97
Wete
16
Better_ Wang (Wang al., 2023)
92.44
67.31
79.88
WeeeRes
10
Fixing_Rebufi (Rebuffi al., 2021)
92.23
66.58
79.41
WideResNet-70-
16
Improving_Gowal (Gowal al., 2021)
88.74
66. 11
77.43
WideResNet-70-
16
Uncovering Gowal (Gowal et al., 2020)
91.1
65.88
78.49
WedeResNet-70-
16
Revisiting_ Huang (Huang et al., 2022)
91.58
65.79
78.69
WedtResNet-A4
Sxkin_Rebbufi (Rebuffi al., 2021)
88.5
64.64
76.57
WideRessNet-106
16
Stable_Kang (Kang et al., 2021)
93.73
71.28
82.51
WedeRes
16, Neural ODE block
Rebung_Rebuff (Rebuffi et al., 2021)
88.54
64.25
76.40
WedeResNet-70-
16
Improving_Gowal (Gowal et al., 2021)
87.5
63.44
75.47
WideRes
10
Robustness Larisstees__aagggg_ (Pang et al., 2022)
89.01
63.35
76.18
WedeResNet-70-
16
Deller_Rade(Rade & Moosavi- Dezfooli, 2021)
91.47
62.83
77.15
WedeResNet-334-
10
Robust_Sehward (Sehwag al., 2021)
87.3
62.79
75.05
ResNest1 152
Uncovering Gowal (Gowal et al., 2020)
89.48
62.8
76.14
WideResNet-288- 10
Exploring Huang (Huang et al., 2021a)
91.23
62.54
76.89
WideResNet-344-
Exploring Huang (Huang et al., 2021a)
90.56
61.56
76.06
WideResNet-34--
Parameterizing_Dar Dai (Dai et al., 2022)
87.02
61.55
74.29
WideResNet-28-
10-PSSiLU
Robustness Paniieeesssaaaggg_ (Pang al.. 2022)
88.61
61.04
74.83
WideResNet-28-- 10
Delner__ade(Rade & Moosavi-Dezfooli, 2021)
88.16
60.97
74.57
WideRes
10
Fixing Rebuffi (Rebuffi al., 2021)
87.33
60.75
74.04
WideResNet-28-
10
Wider_Wo Wu (Wu et al., 2021)
87.67
60.65
74.16
WideResNet-34.
15
Improving_Sridhar (Sridhar al., 2022)
86.53
60.41
73.47
WideResNet-34..
15
Robust_Sehwarl (Sehwag al., 2021)
86.68
60.27
73.48
WideRes
10
Adversarial_ Wu (Wu et al., 2020)
88.25
60.04
74.15
WideResNet-28-
10
Improving_Sriiee Sridhar (Sridhar et al., 2022)
89,46
59.66
74.56
WideRessNet-28.
10
Continued on next page
17

---

Published as a conference paper at ICLR 2024
Table continued from previous page
Defense
SA
RA
Avg.
Classifier
Geometry Zhang (Zhang et al., 2020c)
89.36
59.64
74.50
WideResNet-28- 10
Unlabeled_Carmon Carmon (Carmon et al., 2019)
89.69
59.53
74.61
WideResNet-28- 10
Improving_ Gowal (Gowal et al., 2021)
87.35
58.63
72.99
PreActResNet-18
fowards_Addepalll (Addepalli et al., 2021)
85.32
58.04
71.68
WideResNet-34- 10
Efficient (Addepalli al., 2022)
88.71
57.81
73.26
WideResNet-34-
10
Ltd_Chen (Chen & Lee, 2021)
86.03
57.71
71.87
WedeRes
20
Helper_Rade(Radd Rade(Rade & Moosavi-Dezhine 2021)
89.02
57.67
73.35
PreActResNet-118
Adversarial Jia (Jia et al., 2022a)
85.66
57.61
71.64
WideResNet-700
16
Bight_Debenedefte (Debenedetti et al., 2022)
91.73
57.58
74.66
XCTT-L12
DebeneDebend (Debenedetti et al., 2022)
91.3
57.27
74.29
XCIT-MI2
Uncovering_ Gowal (Gowal et al., 2020)
85.29
57.2
71.25
WideResNet-70-
16
SehwarsSwwwas al., 2020)
88.98
57.14
73.06
WideResNet-28-
10
Helper_Rade Rade(Rade Mezsovi-Dezfooll, 2021)
86.86
$7.09
71.98
PreActResNet-18
_hen (Chen & Lee, 2021)
85.21
56.94
71.08
WideResNet-34-
10
Uncovering_ Gowal (Gowal et al., 2020)
85.64
56.86
71.25
WideResNet-344-
20
Fixing_ Rebuffi (Rebuffi al., 2021) Improving_ Bannoving_Wang (Wang et al., 2020)
83.53
56.66
70.10
PreActResNNtt-18 18
87.5
56.29
71.90
WideResNNet288-
10
Adversarial Jia (Jia et al., 2022a)
84.98
56.26
70.62
WideResNet-344-
10
Adversarial_We _W (Wu et al., 2020)
85.36
56.17
70.77
Wete
10
Light Debenedetti (Debenedetti et al., 2022)
90.06
56.14
73.10
XCTT-S12
Labels_Uesee Uesato (Uesato al., 2019)
86.46
56.03
71.25
WideResNet-28-
10
Robust Robust_Sehwag (Sehwag et al., 2021)
84.59
55.54
70.07
ResNet-18
Using Hendrycks (Hendrycks al., 2019)
87.11
54.92
71.02
WideResNet-28-
10
Bag_ Pang (Pang et al., 2020a)
86.43
54.39
70.41
WideRes
20
Boosting_ Pang (Pang al., 2020b)
85.14
53.74
69.44
WedeRes
20
Learnable Cui (Cui al., 2021)
88.7
53.57
71.14
WideResNet-34..
20
Attacks Attacks_Zhang (Zhang et al., 2020b)
84.52
53.51
69.02
WideResNet-34..
10
Overfitting Rice(Rice et al., 2020)
85.34
53.42
69.38
WideRcsNet-34.-
20
Self Huang (Huang et al., 2020)
83.48
53.34
68.41
WedeResNet-34.-
10
et al., 2019b)
84.92
53.08
69.00
WideRessNet-34.
10
Learnable_Cui (Cui et al., 2021)
88.22
52.86
70.54
WideRessNet-34-
10
Adversarial Qin (Qin et al., 2019)
86.28
52.84
69.56
WideResNeet-0088
Efficient, Ffficient_Addepallii (Addepalli al., 2022)
85.71
52.48
69.10
ResNet-18
Continued next page
18

---

Published as a conference paper at ICLR 2024
Table continued from previous page
Defense
SA
RA
Avg.
Classifier
Adversarial Chen (Chen et al., 2020)
86.04
51.56
68.80
ResNet-50
Efficient_ Chen (Chen al., 2022)
85.32
51.12
68.22
WideRes
10
Towards Addepalli (Addepalli al., 2021)
80.24
51.06
65.65
ResNet-18
Improving Sitawarin BarroirngSSttwarinnnn (Sitawarin et al., 2020)
86.84
50.72
68.78
WideResNet-34-- 10
Robustness (Engstrom et al., 2019)
87.03
49.25
68. 14
ResNet-50
Harnessing_ Kumari (Kumari et al., 2019)
87.8
49.12
68.46
WideResNet-34- 10
Metric_Mao (Mao al., 2019)
86.21
47.41
66.81
WedeResNet-34-
10
You Zhang (Zhang et al., 2019a)
87.2
44.83
66.02
WedeResNet-34-
10
Towards_Madry (Madry et al., 2017)
87.14
44.04
65.59
Net-34s
10
79.84
43,93
61.89
PreActResNet-18
Flammarion, 2020)
Rethinking_ Pang (Pang et al., 2019)
80.89
43.48
62.19
ResNet-32
Fast Bong (Wong al., 2020)
83.34
43.21
63.28
PreActResNet- 18
Snversarial_Shaaale (Shafahi et al., 2019)
86.11
41,47
63.79
WideResNet-33-
10
Mma Ding (Ding al., 2018)
84.36
41.44
62.90
WideResNet-28-44
Tunable Kundu(Kundu et al., 2020)
87.32
40.41
63.87
ResNet-18
Controlling_Atzmne Atzmon (Atzmon al., 2019)
81.3
40.22
60.76
ResNet-18
Robustness_ MMosavvi (Moosavi-Dezfooll et al.,
83.11
38.5
60.81
ResNet-18
2019)
Defense Zhang (Zhang & Wang, 2019)
89.98
36.64
63.31
WideResNet-28-
10
Adversarial Zhang (Zhang & Xu)
90.25
36,45
63.35
WideResNet-28-- 10
Adversarial Jatvesarial_JnggJJaggg et al., 2019)
78.91
34.95
56.93
ResNet-20
Sensible_ (Kim Wang, 2020)
91.51
34.22
62.87
WideResNNet34..
10
Adversarial Adversarial Zhang Xu) Bilateral Wang (Wang & Zhang, 2019)
44.73
32.64
38.69
5-layer-CNN
92.8
29.35
61.08
WideResNet-288- 10
Enhaneing Xiao (Xiao et al., 2019)
79.28
18.5
48.89
DenseNet-121
Manifold_ & Rinard, 2020)
90.84
1.35
46.10
ResNet-18
Adversarial Mustafa (Mustafa et al., 2019)
89.16
0.28
44.72
ResNet-110
Jacobian Chan (Chan et al., 2019)
93.79
0.26
47.03
WideResNet-34- 10
Clustr_Alfarra (Alfaraa et al., 2020)
91.03
45.52
WideResNet-28-
10
IRAD+DitPure (1=20)
93.42
74.05
83.74
WideResNet-28-
10
IRAD+DifiPure (t=25)
91.42
77.71
84.57
WideResNNt--28-
10
IRAD+DiffPure (t=30)
92.33
82.85
87.59
WedeRes
10
IRAD+DinPPur (1=35)
89.14
84.57
86.86
WideRes
10
IRAD+DifPPrr (t=40)
90.57
86.00
88.29
Wede
10
19

---

Published as a conference paper at ICLR 2024
A.3 DETAILED DATA ON GENERALIZATION AGAINST OTHER ATTACKS
In this section, showcase the outcomes of IRAD's performance when subjected to ten distinct attack scenarios on CIFARIO, CIFARI00, and ImageNet datasets, as illustrated Table 10, Table 11, and Table 12. These results correspond to Fig. 4 in the main paper.
Table 10: WRN28- 10 against 10 at Table II: WRN28 10 against 10 at-le 12: ResNet50 against 10 at-
tacks CI (6 8/255). tacks CIFAR 100 (f = 8/255). tacks ImageNet (foc 1/255).
Attack
DISCO
IRAD
Attack
DISCO
IRAD
Attack
DISCO
IRAD
FGSM
64.15
92.11
FGSM
46.38
69.36
FGSM
56.00
74.50
BIM
80.55
89.64
BIM
67.99
72.45
BIM
66.52
71.06
PGD
83.27
89.94
PGD
71.25
74.12
PGD
66.28
71.10
REGSM
81.08
89.97
RFGSM
68.67
73.04
RFGSM
66.22
71.22
TPgd
82.13
88.77
TPgd
70.75
72.16
TPgd
70.00
72.12
APgd
82.10
89.09
APgd
73.76
75.10
APgd
65.44
70.66
FotPgd
76.83
87.56
FotPgd
69.22
72.78
EotPgd
69.30
73.28
FFgsm
70.41
90.23
FFgsm
59.01
70.27
FFgsm
57.06
73,50
Milgsm
45.40
82.51
MiFgsm
35.38
52.09
Mifgsm
52.66
69,50
Jitter
79.64
89.01
Jitter
67.91
71.27
Jitter
64.66
70.24
Avg.
74.55
88.88
Avg.
63.03
70.26
Avg.
63.41
71.72
A.4
PERFORMANCE UNDER TRANSFER-BASED ATTACKS
Table 13: WRN28-10 against VmiFgsm, VniFgsm, Admix attack on CIFARIO (E 8/255).
VmiFgsm
VniFgsm
Admix
Defense
SA
RA
Avg.
SA
RA
Avg.
SA
RA
Avg.
No Defense
94.77
0
47.39
94.77
0
47.39
94.77
3.82
49.30
DISCO
89.24
50.51
69.88
89.24
52.76
71.00
89.24
65.91
77.58
DiffPure (t=100)
89.21
85.55
87.38
89.21
85.49
87.35
89.21
85.23
87.22
IRAD
91.70
86.22
88.96
91.70
87.65
89.68
91.70
82.39
87.05
In this section, we conduct experiments more transfer-based attacks (Wang & He, 2021; Wang al., 2021), which include three attack methods (i.e., VmiFgsm, VniFgsm, and Admix). Our results demonstrate that method significantly outperforms DISCO when facing all three attacks. As depicted Table 13, comparison to DiftPure, IRAD exhibits notably superior SA and RA when subjected to the VmiFgsm and VniFgsm attacks. While DiffPure slightly surpasses IRAD in RA under the Admix attack, is worth mentioning that the inference process of DiffPure approximately 200 times slower than IRAD shown in Table 5.
Table 14: ResNet50 against VmiFgsm, VniFgsm, Admix attack on ImageNet (Cxo 4/255).
VmiFgssm
VniFgsm RA
Admix
Defense
SA
RA
Avg.
Avg.
SA
Avg.
No Defense
76.72
38.36
76.72
0.04
38.38
76.72
3.32
40.02
DISCO
72.66
52.98
62.82
72.66
52.76
62.71
72.66
42.74
57.7
DiffPure (1=150)
69.42
64.9
67.16
69.42
64.22
66.82
69.42
61.34
65.38
IRAD
72.14
71.35
71.75
72.14
71.4
71.77
72.14
63.8
67.97
Furthermore, we conducted additional experiments on ImageNet, following the approach outlined in the DiftPure paper and setting 1=150 when dealing with ImageNet.
Table 14 indicates that IRAD outperforms DISCO significantly across all three attack scenarios. Furthermore, not only does IRAD surpass DiffPure in performance under all three attacks, but is also more than 100 times faster than DiffPure. This demonstrates IRAD's 's superior performance and time efficiency.
A.5 ABLATION STUDY ON DIFFERENT RECONSTRUUTIOO METHODS
this section, we conduct an ablation study to integrate various RECONS methods with SampleNet.
20

---

Published as a conference paper at ICLR 2024
Table 15: Comparing different reconstruc- tion methods.
It's important note that SampleNet is MLP that takes the features of pixel [i.]] and the coordinate values as input, predicting their shifting directly. Since the Nearest and Bilinear RECONS methods could not generate the pixel features the implicit representations do, we directly utilize the encoder from the implicit representations for a fair comparison. The experimental results are shown in Table 15.
RECONS
SA
RA
Avg.
Nearest
93.46
0.96
47.21
Bilineur
92.61
79.20
85.91
Implici Representation
91.70
89.72
90.71
As depicted in the table, despite having a high SA, Nearest RECONS lacks the capability defend against adversarial attacks. This limitation arises from its ability to acquire only the pixel values originally sampled from the images. The Bilinear RECONS exhibits superior RA in comparison to the Nearest RECONS and outperforms basic image resampling methods under Bilinear RECONS showcased in Table of the paper, highlighting the effectiveness of SampleNet. Compared to these two RECONS, the implicit representation RECONS achieves even higher RA and Avg.. demonstrating their superiority over these two RECONS, Overall, the data from the table suggests that Nearest and Bilinear RECONS are less effective than implicit representation-basee based methods when integrated with SampleNet.
A.6 COMPARING DIFFERENT STEPS IN THE INTEGRATION OF IRAD AND DIFFPURE
In this section, we conduct additional exper- iments integrating IRAD with various steps of DiftPure. The results are presented in Table 16. Due the significant time and resource costs associated with imple- menting DiffPure, we performed the ex- periment on 350 randomly selected images from the test set. The findings indicate that IRAD consistently enhances DiffPure's s per- formance across various steps. For exam- ple, with 1=40, the combination of IRAD and DiffPure (t=40) notably increases the Robust Accuracy (RA) of DiffPure (t=40) from 59.42% to 86.00% Moreover, it sur- passes DiffPure (t=100) in terms of SA,
Table 16: Comparing different steps in the integration of IRAD and DiffPure.
SA
RA
Avg.
Cost (ms)
DiftPure
89.73
75.12
82.43
132.80
DISCO
89.25
0
44.63
0.38
TRAD
91.70
0
45.85
0.68
DiffPure (t=20)
93.66
8.01
50.83
27.25
IRAD+DifPure (t=20)
93.42
74.05
83.74
27.70
DiffPure ((=25)
93.55
18.28
55.92
31,93
IRAD+DifiPure (t=25)
91.42
77.71
84.57
32.41
DiffPure (t:30)
92.85
36.28
64.57
37.62
IRAD+DifPPrre (1=30)
92.33
82.85
87.59
38.37
DifiPure (t=35)
93.50
48.75
71.13
44,18
DiADuDe (t=35)
89.14
84.57
86.86
44.86
DiffPure (1=40)
93.71
59.42
76.57
51.78
RA, Avg., and time efficiency, demonstrat- ing significant improvements.
IRAD+DifPure (t=10)
90.57
86.00
88.29
52.33
A.7 THE EFFECTIVENESS IMPLICIT REPRESENTATION RECONSTRUCTION
In this part, we employ four metrics to assess the reconstruction of the implicit representation: PSNR (Hore& Ziou, 2010), SSIM (Wang et al., 2004), loss of low frequency and high frequency FFT (Fast Fourier Transform) (Cooley & Tukey, 1965), and LPIPS (Zhang et al., 2018).
Table 17: Image quality results on CIFAR 10.
and
RECONS/Ads)
PSNR
32.10
36.01
SSIM
0.9936
0.9973
Frequency Loss
High Frequency FPT Loss LPIPS
0.3636
0.2439
0,0771
8.0169
1. PSNR: As shown in Table 17, the PSNR value is higher for implicit representation reconstructed images (36.01) compared to adversarial images (3).10). This indicates that the reconstructed adversarial images have higher fidelity and less distortion compared to the clean images after reconstruction.
2. SSIM: The SSIM value is higher for reconstructed adversarial images (0.9973) compared to adversarial images (0.9936). This suggests that the reconstructed adversarial images better preserve the structural details of the original images.
3. FFT: Both FFT Low Loss and FFT High Loss are lower for reconstructed adversarial images compared to clean images. This implies that less information is lost in frequency domains during the reconstruction of adversarial images.
21

---

Published as a conference paper at ICLR 2024
4. LPIPS: The LPIPS value is significantly lower for reconstructed adversarial images (0.0169) compared to adversarial images (0.0771). This suggests that the reconstructed adversarial images are more perceptually similar to the original images compared to the adversarial images.
Overall, the table suggests that the reconstructed versions of adversarial images tend to exhibit better quality and closer re- semblance to the original images across multiple metrics. This
Table 18: The comparison of feature distances.
also implies that the implicit representation does not sacrifice meaningful information in the raw image when eliminating adversarial perturbations. This demonstrates the effectiveness of using implicit continuous representation to represent images
Comparison
Faclidean Distance
Clean
2.9
Clean and RECONS(AdY)
1.21
Clean and IRAD(Adv)
0.75
within a continuous coordinate space.
A.8 THE EFFECTIVENESS OF SAMPLENET
In this section, introduce two additional perspectives to analyze our method and observe the following: First, Sam- pleNet significantly diminishes semantic disparity, bringing reconstructed adversarial and clean representations into closer alignment. Second, SampleNet aligns the decision paths of re- constructed adversarial examples with those of clean examples.
Firstly, we delve into analyzing the features to comprehend how the semantics of the images evolve and perform PCA to visualize the analysis. We randomly selected 100 images from one class and obtained the features of clean images (Clean), adversarial images (Adv), adversarial images after implicit reconstruction (RECONS(Adv)), and adversarial images after IRAD (IR)D(Adv)). We computed the mean in each dimension of the features of clean images to establish the central point of these clean features. Subsequently, we computed the Euclidean distance between the features of other images and the center clean images. As shown in Table 18, the average Euclidean distance of adversarial image features is significantly distant from clean image features. After the reconstruction of implicit representation, the adversarial image features become closer to the center of clean images' features. By utilizing IRAD, the distance becomes even closer. The PCA findings also illustrate a comparable phenomenon. As depicted in Fig. 5, clean adversarial images are notably distant from clean images. After the reconstruction of implicit representation. there are fewer samples situated far from clean images. Moreover, significantly fewer images lie outside the clean distribution after employing IRAD. Therefore, this demonstrates that the shift map generated by SampleNet can notably reduce semantic disparity and bring adversarial and clean representations into closer alignment.
Figure S: The PCA results.
Secondly, we follow the methodology of previous research to conduct decision rationale analysis through decision path analysis (Khakzar et al., 2021; Li et al., 2021; Wang al., 2018; Qiu et al., 2019; Li et al., 2023; Xie et al., 2022), aiming to understand the decision-making process following resam- pling. We obtain the decision paths of Adv, RECONS(Adv), and IRAD(Adv), and calculate their cosine similarity with the decision path of clean images. As shown in Table 19, after the reconstruction of implicit representa- tion, the decision path becomes more similar. With the usage of IRAD. the similarity becomes even closer. These results demonstrate that after resampling, our method exhibits a closer alignment with the decision rationale observed in clean images.
Table 19: Decision path comparison.
Comparison
Similarity
Clean and Adv
0.5482
Clean RECONS(Alv)
0.9915
Clean and IRAD(Adv)
0.9987
These two experiments further highlight the effectiveness of our resampling technique in disrupting adversarial textures and mitigating adversarial attacks.
A.9 MORE VISUALIZATION RESULTS
In this section, present some examples of applying IRAD on CIFARIO, CIFAR100, and ImageNet.
22

---

Published as a conference paper at ICLR 2024
For each figure, (a) and (b) present clean image and the corresponding adversarial counterpart as well as the predicted logits from the classifier. (c) and (d) show the results using a randomly resampling strategy to handle the clean and adversarial images before feeding them to the classifier. (e) and (f) display the results of leveraging IRAD to handle the inputs.
The visualizations in Fig. 6, and reveal that the randomly resampled clean image produces almost identical logits to the raw clean image. On the other hand, the randomly resampled adversarial image yields lower confidence in the misclassified category. but does directly rectify prediction errors. IRAD, however, not only maintains the logits for clean images but also corrects classification errors and generates logits similar clean images for adversarial images. This the reason why IRAD enhances robustness while maintaining good performance on clean images.
(a) Clean Image
(b) Adversarial(Adv. Image
(c) Clean Image >> Random Sampling
(d) Adv. Image -> Random Sampling
(c) Clean Image >> IRAD
(f) Adv. Image -> IRAD
Figure 6: Case visualization of CIFAR10
23

---

Published as a conference paper at ICLR 2024
(a) Clean Image
(b) Adversarial(Adv. Image
(c) Clean Image >> Random Sampling
(d) Adv. Image >> Random Sampling
(e) Clean Image >> IRAD
(f) Adv. Image > IRAD
Figure Case visualization of CIFARI00
(a) Clean Image
(b) Adversarial(Adv) Image
(c) Clean Image -> Random Sampling
(d) Adv. Image -> Random Sampling
(c) Clean Image >> IRAD
(f) Adv. Image - IRAD
Figure 8: Case visualization of ImageNct
24


Reviews:
[{"review_id": "8UYnw0mXRY", "reviewer": "Reviewer_x2bE", "summary": "This paper introduces a novel image resampling approach for adversarial attacks. Specifically, two implicit neural representations are adopted to model the image reconstruction and shift map, respectively. With the estimated continuous representation and the learned shifting, the generated image becomes robust toward adversarial attacks. This method achieves significant performance on several benchmarks.", "questions": "Are there any severe distortions in the shift map?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method provides a novel approach to solving the adversarial attack problem. This is the first paper to solve the problem in such an implicit representation manner. The idea is impressive。\n\n2. The writing is easy to follow. It starts with the naive solution, and then gradually introduces the proposed one. Each step is logically based on the last step.\n\n3. The experimental results are strong. It not only significantly outperforms baselines in Table 1, but also achieves higher performance than state-of-the-art approaches with totally different solutions.", "weaknesses": "1. One important ablation study is lacking. Table 8 verifies the effectiveness of the SampleNet. However, the usefulness of RECONS is not proven. How about the performance of bilinear interpolation with SampletNet?\n\n2. The learned shift map contains explainable information, however, analysis of which is lacking. The only visualization shown in Fig 2 contains little information and a deeper analysis should be adopted.\n\n3. The running time of the proposed method is not provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YMzfOdokln", "reviewer": "Reviewer_ynQw", "summary": "This paper proposed a new method for defending against adversarial attacks. Given an input image, the proposed method aims to first build an implicit representation and then reconstruct the image by resampling. For further improvement, the paper also proposes to introduce a network to dispatch different inputs to different level pixel-wise shifts for resampling. Experiments results show that the proposed methods improve the adversarial robustness.", "questions": "See my comments on the weaknesses session above. I'm willing to improve my rating if the author can convince me in the rebuttal period.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The underlying idea and the proposed method appear very novel to me. \n\n- The paper conducts extensive experiments and demonstrates significant improvements over the baseline.\n\n- The paper is well-written and easy to read", "weaknesses": "I am having difficulty being convinced of why this approach works. The paper's objective is to construct an implicit representation for the input image and then reconstruct the clean image from this implicit representation, claiming that the reconstructed image would be free from any adversarial patterns. However, if we assume that the underlying implicit representation faithfully represents the input signal, it should also contain the adversarial patterns that present in the input signal. This contradicts the explanation for why the proposed method is effective.\n\nTo the best of my thoughts, the only plausible explanation to explain why the proposed method works well quantitatively is that the underlying implicit representation is biased toward fitting low-frequency or smooth signals, and thus, it fails to represent high-frequency adversarial patterns. However, this would imply a degradation in the image quality represented in the implicit model, as it suggests that the model cannot faithfully represent high-frequency signals. In either case, this highlights certain weaknesses in the proposed method", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MBbC9j5Smy", "reviewer": "Reviewer_4nS8", "summary": "This paper introduces a test-time adversarial defense mechanism realized via image resampling. Central to their proposal is the \"Implicit Representation Driven Image Resampling (IRAD)\" methodology, which operates in two distinct phases: a) the creation of a continuous coordinate space, and b) the utilization of the newly proposed SampleNet. Unlike conventional heuristic sampling strategies, SampleNet is designed to predict pixel-wise shifts based on the embedding of the input image automatically. Moreover, in a pursuit of optimal performance and a marked increase in processing speed, the authors have integrated their method with DiffPure.", "questions": "None", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The introduced image-resampling defense \"IRAD\" substantially enhances the Robust Accuracy (RA) when juxtaposed with the elementary implementation of image resampling (IR).\n2. The paper presents SampleNet, a novel approach for automated sampling, designed to supplant traditional heuristic sampling strategies.\n3. The methodology presented has been rigorously evaluated across various datasets and neural network architectures.", "weaknesses": "1. There's room for optimization in the placement of tables and figures to enhance readability and presentation.\n2. Merging IRAD with a 20-step DiffPure yields superior results while economizing computation time. Nonetheless, a more comprehensive analysis examining the integration of IRAD with varying steps of DiffPure would enrich the study's depth and relevance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wEtdlDC68l", "reviewer": "Reviewer_mZuj", "summary": "In this work, the authors adopt image resampling to defend against adversarial attacks. In particular, they first construct an implicit continuous representation for reconstruction. Second, they introduce SampleNet, which automatically generates pixel-wise shifts for resampling. And their method can be extended to the state-of-the-art diffusion-based method, which is significantly accelerated.Extensive experiments on several datasets have shown the effectiveness.", "questions": "See weakness", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well-written and easy to follow.\n\n2. Adopting image resampling is a novel and interesting idea to defend against adversarial attacks.\n\n3. The proposed SampleNet is simple yet effective, which can effectively defend against adversarial attacks to some extent.\n\n4. They have adopted various datasets to validate the effectiveness.", "weaknesses": "1. It is not clear why resampling can effectively defend adversarial examples. Especially, image resampling via bilinear or nearest interpolation might be not effective enough to eliminate adversarial perturbation.\n\n2. How can you guarantee that SampleNet is not attacked? As a result, in Table 5, IRAD cannot exhibit robustness when SampleNet is attacked simultaneously. \n\n3. In my opinion, such a resampling method cannot effectively defend against white-box attacks. However, it might be effective in defending against black-box attacks, especially transfer-based attacks [1,2,3]. I suggest you add such a comparison.\n\n4. Since image resampling pre-processes the input image before the model, it is similar to a purifier that eliminates adversarial perturbation [4]. I think it is necessary to compare with such baselines.\n\n[1]  Zhang et al. Patch-wise attack for fooling deep neural network. ECCV 2020.\n\n[2] Wang et al. Enhancing the transferability of adversarial attacks through variance tuning. CVPR 2021.\n\n[3] Wang et al. Admix: Enhancing the transferability of adversarial attacks. ICCV 2021.\n\n[4] Naseer et al. A self-supervised approach for adversarial robustness. CVPR 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)