Reverse Engineering of Imperceptible Adversarial Image Perturbations
Authors: Yifan Gong
Yuguang Yao
Yize Li
Yimeng Zhang
Xiaoming Liu
Xue Lin
Sijia Liu
Venue: ICLR 2022 Poster (2022)
Keywords: Reverse Engineering of Deceptions
adversarial examples
denoising
neural networks
interpretability

Abstract: It has been well recognized that neural network based image classifiers are easily fooled by images with tiny perturbations crafted by an adversary. There has been a vast volume of research to generate and defend such adversarial attacks. However, the following problem is left unexplored: How to reverse-engineer adversarial perturbations from an adversarial image? This leads to a new adversarial learning paradigm—Reverse Engineering of Deceptions (RED). If successful, RED allows us to estimate adversarial perturbations and recover the original images. However, carefully crafted, tiny adversarial perturbations are difficult to recover by optimizing a unilateral RED objective. For example, the pure image denoising method may overfit to minimizing the reconstruction error but hardly preserve the classification properties of the true adversarial perturbations.  To tackle this challenge, we formalize the RED problem and identify a set of principles crucial to the RED approach design. Particularly, we find that prediction alignment and proper data augmentation (in terms of spatial transformations) are two criteria to achieve a generalizable RED approach. By integrating these RED principles with image denoising, we propose a new Class-Discriminative Denoising based RED framework, termed CDD-RED. Extensive experiments demonstrate the effectiveness of CDD-RED under different evaluation metrics (ranging from the pixel-level, prediction-level to the attribution-level alignment) and a variety of attack generation methods (e.g., FGSM, PGD, CW, AutoAttack, and adaptive attacks).

Published as a conference paper at ICLR 2022
REVERSE ENGINEERING OF IMPERCEPTIBLE ADVER- SARIAL IMAGE PERTURBATIONS
Yifan Gong';
Yuguang Y Yize Li', Yimeng Zhang",Xiaoming Xiiammin Liu?, Xue Lin', ,ijia Liu 1 Northeastern University, Michigan State University, 3 MITTIBM Watson Al Lab, IBM Research gong . yifa, 1i. lizyize, xue ]in] @nwrtheastern.eee.
(yaoyugua, zhan1853, liusiji5 (amsjii))mmuu. (msu. edu, liuxmêcse msu. edu
ABSTRACT
arXiv:2203. 2201 11903.14145v [cs.CV] 1 Apr 2022
It has been well recognized that neural network based image classifiers are easily fooled by images with tiny perturbations crafted by an adversary. There has been a vast volume of research generate and defend such adversarial attacks. However, the following problem is left unexplored: How to reverse-engineer adversarial perturbations from an adversarial image? This leads to a new adversarial learn- ing paradigm- Daradigm-Reverse Reverse Engineering of Deceptions (RED). If successful, RED allows us to estimate adversarial perturbations and recover the original images. However, carefully crafted, tiny adversarial perturbations are difficult to recover by optimizing a unilateral RED objective. For example, the pure image denoising method may overfit to minimizing the reconstruction error but hardly preserves the classification properties of the true adversarial perturbations. To tackle this challenge, we formalize the RED problem and identify a of principles crucial to the RED approach design. Particularly, we find that prediction alignment and proper data augmentation (in terms of spatial transformations) are two criteria achieve generalizable RED approach. By integrating these RED principles with image denoising, we propose a new cuiss-Disrrmmiaative Denoising based RED framework, termed CDD-RED. Extensive experiments demonstrate the effective- ness of CDD-RED under different evaluation metrics (ranging from the pixel-level, prediction-level to the attribution-level alignment) and variety of attack generation methods (e.g.. FGSM, PGD, CW, AutoAttack, and adaptive attacks). Codes are available at 1ink
INTRODUCTION
Deep neural networks (DNNs) are susceptible to adversarially-orrftee tiny input perturbations during inference. Such imperceptible perturbations, a.k.a. adversarial attacks, could cause DNNs to draw manifestly wrong conclusions. The existence of adversarial attacks was first uncovered in the domain of image classification (Goodfellow et al. 2014, Carlini & Wagner] 2017 Papernot et al.) 2016b). and was then rapidly extended to the other domains, such as object detection (Xie et al.] 2017 |erban cet al. 2020), language modeling (Cheng al. 2020 Srikant al. 2021). and medical machine learning (Finlayson et al. 2019 Antun et al. 2020). Despite different applications, the underlying attack formulations and generation methods commonly obey the ones used in image classification.
vast volume of existing works have been devoted to designing defenses against such attacks, mostly focusing either detecting adversarial examples (Grosse et al. 2017| Yang et al. 2020 Metzen
ct al., 2017: Meng & Chen 2017: Wojeik et al.) 2020) or acquiring adversarially robust DNNs
(Madry al. 2017 Zhang al. 2019. Wong & Kolter 2017 Salman et al. 2020 Wong al.
2020oCarmon Carmon et al. 2019 Shafahi et al. 2019). Despite the plethora of prior work on adversarial defenses, it seems impossible achieve 'perfect' robustness. Given the fact that adversarial attacks are inevitable (Shalahi et al., 2020), we ask whether or not an adversarial attack can be reverse- engineered so that one can estimate the adversary's information (e.g., adversarial perturbations) behind the attack instances. The above problem is referred to as Reverse Engineering of Deceptions (RED), fostering a new adversarial learning regime. The development of RED technologies will also enable the adversarial situation awareness in high-stake applications.
"Equal contributions.

---

Published as a conference paper at ICLR 2022
To the best of our knowledge, few work studied the RED problem. The most relevant that we aware of is (Pang et al. 2020), which proposed the so-called query of interest (QOI) estimation model to infer the adversary's target class by model queries. However, the work (Pang et al. 2020) was restricted to the black-box attack scenario and thus lacks a general formulation of RHD. Furthermore, it has not built complete RED pipeline, which should not only provide a solution estimating the adversarial example but also formalizing evaluation metrics to comprehensively measure the performance of RED. In this paper, we aim take a solid step towards addressing the RED problem.
1.1 CONTRIBUTIONS
The main contributions of our work is listed below.
We formulate the Reverse Engineering of Deceptions (RED) problem that is able to estimate adversarial perturbations and provides the feasibility of inferring the intention an adversary, egg. 'adversary saliency regions' of an adversarial image.
We identify a series of RED principles to effectively estimate the adversarially-craten tiny pertur- bations. We find that the dass-discrmminative ability is crucial to evaluate the RED performance. We also find that data augmentation, e.g., spatial transformations, is another key to improve the RED result. Furthermore, we integrate the developed RED principles into image denoising and propose denoiser-assisted RED approach.
We build comprehensive evaluation pipeline to quantify the RED performance from different perspectives, such as pixel-level reconstruction error, prediction-level alignment, and attribution-leve adversary saliency region recovery. With extensive experimental study, we show that, compared to image denoising basclines, our proposal yields a consistent improvement across diverse RED evaluation metries and attack generation methods, e.g., FGSM (Goodfellow al. 2014), CW (Carlini
& Wagner 2017), PGD (aadr et al.) 2017) and Auto/Atack (Croce & Hein 2020).
1.2 RELATED WORK
Adversarial attacks. Different types of adversarial attacks have been proposed, ranging from
digital attacks (Goodfellow et al.] 2014 Carlini & Wagner 2017 Madry et al. 2017 Croce & Hein
2020 Xu et al. 20019; Chen et al. 201 Xiao a 2018) 8) physical attacks (Eykholt et al. Li et al.
2018
Athhalye et al.] [018]]Chen Chen et al. 2018:08 Xu et al. 2019c). The former gives the
most fundamental threat model that commonly deceives DNN models during inference by crafting imperceptible adversarial perturbations. The latter extends the former to fool the victim models in the physical environment. Compared to digital attacks, physical attacks require much larger perturbation strengths to enhance the adversary's 's resilience to various physical conditions such as lightness and object deformation (Athalye et al. 2018 Xu al. 2019c).
In this paper, we focus fn-mrrm ball constrained attacks, a.k.a. f, attacks, for E (1, oc), most widely-used -ueed in digital attacks. Examples include FGSM (Goodfellow al.] al.]] 2014), PGD (Madry et al. 2017). CW (Carlini& & Wagner 2017), and the recently-release Hein 2020). Based on the adversary's intent, f, attacks are further divided into untargeted attacks and targeted attacks, where in contrast to the former, the latter designates the (incorrect) prediction label of a victim model. When an adversary has no access to victim models' detailed information (such architectures and model weights), Ep attacks can be further generalized to black-bot attacks
attack benchmark AutoAttack (Croce
by leveraging either surrogate victim models (Papernot et al. 2017] [2166a] Dong et al. 2019
et al. 2017) input-output queries from the original black-box models (Chen et al. 2017b Liu et al.
2019
Cheng et al. 2019).
To improve the robustness of DNNs, a variety of approaches have been proposed to defend against lp attacks. One line of research focuses on enhancing the robustness of DNNs during training, e.g., adversarial training (Madry et al. 2017). TRADES (Zhang et al.
Adversarial defenses.
2019), randomized smoothing (Wong & Kolter 2017). and their variants (Salman et al. 2020
Wong et al., 2020 Carmon et al. 2019: Shafahi et al. 2019: Uesato et al. 2019 Chen et al. 2020 Another line of research is to detect adversarial attacks without altering the victim model or the training process. The key technique is to differentiate between benign and adversarial examples by measuring their 'distance.' Such a distance measure has been defined in the input space via pixel-level
reconstruction error (Meng & Chen 2017 Liao al. 2018), in the intermediate layers via neuron
2

---

Published as a conference paper at ICLR 2022
activation anomalies (Xu et al., 2019a). and in the logit space by tracking the sensitivity of deep feature attributions to input perturbations (Yang et al. 2020).
In contrast to RED, adversarial detection is a relatively simpler problem roughly approximated
distance possesses detection-ability (Meng & Chen 2017; Luo et al., 2013). Among the existing adversarial defense techniques, the
Denoised Smoothing (DS) method (Salman
et al. 2020) is more related to ours. In (Salman et al. 2020). an image denoising network is prepended existing victim model so that the augmented system can be performed as smoothed image classifier with certified robustness. Although DS is not designed for RED, its denoised output can be regarded benign example estimate. The promotion of classification stability DS also motivates us to design the RED methods with class daas-discriminative ability. Thus, DS will be main baseline approach for comparison. Similar to our RED setting, the concurrent work (Souri et al. 2021) also identified the feasibility of estimating adversarial perturbations from adversarial examples.
REVERSE ENGINEERING OF DECEPTIONS: FORMULATION AND CHALLENGES
In this section, we first introduce the threat model of our interest: adversarial attacks on images. Based on that, we formalize the Reverse Engineering of Deceptions (RED) problem and demonstrate its challenges through some 'warm-up" examples.
Preliminaries on threat model. We focus on lp attacks, where the adversary's goal is to generate imperceptible input perturbations fool well-trained image classifier. Formally, let denote benign image, and 8 an additive perturbation variable. Given a victim classifier f and a perturbation strength tolerance (in terms of, e.g., loo-norm constraint ||8]| oo S <), the desired attack generation algorithm A then seeks the optimal ô subject to the perturbation constraints. Such an attack generation process denoted by 8 A(x. f. <), resulting in an adversarial example x' + Here AA can be fulfilled by different attack methods, e.g., FGSM (Goodfellow et al., 2014). CW (Carlini & Wagner 2017 PGD (Madry et al. 2017), and Auto (Croce Hein 2020).
Problem formulation of RED. Different from conventional defenses detect reject adversarial
instances (Pang et al., 2020 Liao et al. 2018 Shafahi et al. 2020 Niu et al. 2020). RED aims to address the following question.
(RED problem) Given adversarial instance, we reverse-engineer the adversarial perturba- tions 8, and infer the adversary's 's objective and knowledge, e.g., true image class behind deception and adversary saliency image region?
Formally, we aim to recover 8 from an adversarial example x' under the prior knowledge of the victim model f or its substitute f if the former is a black box. We denote the RED operation as = R(x', f). which covers the white-box scenario (f = f) as a special case. We propose to learn a parametric model De (e.g. a denoising neural network that we will focus on) as an approximation of R through a training dataset of adversary-benignee pairs f = {(x,, x)}. Through Dg. RED will provide benign example estimate XRED and adversarial example estimate XRED below:
XRED Do(x'), XRED
XRED
pertrbution extimate
where perturbation estimate is given by subtracting the RED's output with its input, x' - -De(x).
We highlight that RED yields a new defensive ap- proach aiming to 'diagnose' the perturbation details of an existing adversarial example in a post-hoc, foren- sic manner. This is different from adversarial detec- tion (AD). Fig provides a visual comparison of RED with AD. Although AD is also designed in post-hoc manner, it aims to determine whether an input is an adversarial example for a victim model based on cer- tain statistics on model features or logits. Besides, AD propcreessinng step of RED, where
Advemarial Detection
Benign
Imput
Engineer Deseption
Adverasnia Label: Builtin Sallenyy Region & Perturbation:
might be used the former provides 'detected' adversarial examples for fine-level RED diagnosis. In experiments, will
Figure 1: Overview of RED versus AD.
3

---

Published as a conference paper at ICLR 2022
also show that the outputs of RED can be leveraged to guide the design of adversarial detection. In this sense, RED and AD are complementary building blocks within a closed loop.
Challenges of RED In this work, will specify the RED model Da a denoising network. However, it is highly non-trivial to design a proper denoiser for RED. Speaking at high level, there exist two main challenges. First, unlike the conventional image denoising strategies (Zhang et al.) 2017), the design of an RED-aware denoiser needs to take into account the effects of victim models and data properties of eveeraay-benigninn pairs. Second, it might be insufficient merely minimize the reconstruction error as the adversarial perturbation is finely-crafted (Niu et al. 2020). Therefore, either under- or over-denoising will lead to poor RED performance.
RED EVALUATION METRICS AND DENOISING-ONEL BASELINE
Since RED different from existing defensive approaches, we first develop new performance metrics of RED, ranging from pixel-Ievel reconstruction error to attribution-level adversary saliency region. We next leverage the proposed performance metrics to demonstrate why a pure image denoiser is incapable of fulfilling RED.
RED evaluation metrics. Given a learned RED model Da. the RED performance will be evaluated over a testing dataset (x', x) Drest's see implementation details in Sec. Here, x' is used as the lesting input of the RED model, and x is the associated ground-truth benign example for comparison. The benign example estimate XRED and adversarial example estimate XRED are obtained following [I. RED evaluation pipeline is conducted from the following aspects: @ pixel-level reconstruction error, prediction-leve inference alignment, and @ attribution adversary saliency region.
Pixel-level: Reconstruction eπror given by d(x, XRED) = E(xe D..a [|] | x||2].
@ Prediction-level: Prediction alignment (PA) between the pair of benign example and its estimate (XRED,x) and PA between the pair of adversarial example and its estimate (x (XEDD. x'), given by
PAbeniga card(f (XRED, x) F(XRED) F(x))) PAady card(f(X<ED. x') F(XRED) = F(x'))
card(Drest)
(D
where card(.) denotes cardinality function of a set and F refers to the prediction label provided by the victim model f.
Input attribution alignment (IAA) between the benign pair (XRED. x) and between the adversarial pair ( In this work, we adopt GradCAM (Selvaraju et al. (0200) to attribute the predictions of classes back to input saliency regions. The rationale behind IAA is that the unnoticeable adversarial perturbations (in the pixel space) can introduce an evident input attribution discrepancy with respect to (w.r.t) the true label y and the adversary's 's target label y' (Boopathy et al. 2020; Xu et al./ (201b)). Thus, an accurate RED should be able to erase the adversarial attribution effect through XRED. and estimate the adversarial intent through the saliency region of x XRED (see Fig. [ for illustration).
Attribation-level:
Mcceemep Nulnnaapp.
Denoising-Only (DO) baseline. We further show that how a pure image denoiser, 'must- try' baseline, is insufficient of tackling the RED problem. This failure case drive us to rethink the denoising strategy through the lens of RED. First, we obtain the denoising network by min- imizing the reconstruction error:
Figure 2: IAA of DO compared with ground-truth.
minimize Lieeoise (2: := Bex st ( Dx(x') x|11,
where a Mean Absolute Error (MAE)-type loss is used for denoising (Liao et al. 2018) and the creation of training dataset Ω is illustrated in Sec.3.1 Let us then evaluate the performance of DO through the non-adversarial prediction alignment PAbenign and IAA. We find that PAbeeign = 42.8% for DO. And 2 shows the IAA performance of DO w.r.t an input example. As we can see, DO is not capable of exactly recovering the adversarial saliency regions compared to the ground-truth
4

---

Published as a conference paper at ICLR 2022
adversarial perturbations. These suggest that DO-based RED lacks the reconstruction ability at the prediction and the attribution levels. Another naive approach is performing adversarial attack back to z', yet it requires additional assumptions and might not precisely recover the ground-truth perturbations. The detailed limitations are discussed in Appendix
4
DENOISING FOR RED
In this section, propose novel Class- Discriminative Denoising based RED approach termed CDD-RED; Fig. Bl for overview. CDD-RED contains two key components. First, propose PA regularization to enforce the prediction-level stabilities of both estimated be- nign example XRED and adversarial example XRED with respect to their true counterparts and x', respectively. Second, we propose data augmentation strategy to improve the RED's generalization without losing class- discriminative ability.
Output
Estimate Adversarial Perturbatton
Prediction Aligniment
XRE
IH
Classifler Loaits
Figure 3: CDD-RED overview.
Benign and ad adversarial prediction alignment. To accurately estimate the adversarial perturbation from an adversarial instance, the lessons from the DO approach suggest preserve the class- discriminative ability of RED estimates to align with the original predictions, given by XRED vs. x, and XRED vs. x'. Spurred by that, the training objective of CDD-RED is required not only to minimize the reconstruction error like (2) but also to maximize PA, namely, 'clone' the class- discriminative ability of original data. To achieve this goal, we augment the denoiser De with a known classifier f to generate predictions of estimated benign and adversarial examples (see Fig. i.e., XRED and XRED defined in (1). By contrasting fXRED) with fxx). and f(XRED) with f(x'), we can promote PA by minimizing the prediction gap between true examples and estimated ones:
x)), (px(0;x' xxxxxxx)))) CE(((XRED)) f(x)) (XkE()))))) ÉPA (0; ?? [xx (0: (0)x'x)));
(3)
PA for benign prediction
PA for adversarial prodiction
where CE denotes the cross-entropy loss. To enhance the class-discriminatiee ability, it desirable to integrate the denoising loss 22) with the PA regularization (3). leading to Edeuoiee + Aepa, where regularization parameter. To address this issue, will further propose data augmentation method to improve the denoising ability without losing the advantage of PA regularization.
Proper data augmentation improves RED.
The rationale behind incorporating image trans- formations into CDD-RED lies in two as- pects. First, data transformation make RED foveated to the most informative attack artifacts since adversarial instance could be sensitive to input transformations (Luo et al. 2015 Atha- 2020
0.76
(PAcnn + PA
0.75
0.74
Crop&Pad
0.73
Cutmix
Cutout
[ye al. 2018 Xie al. al. 2019) Fan et al. 2021). Second, the identification of transformation-resiin resilient benign/adversarial in- stances may enhance the capabilities of PA and IAA
0.72
Translate
D.71
Rotate
16.0
16.5
17.0
17.5
X
Figure The influence of different data augmentations. 'Base' refers to the base training without augmentation.
However, it is highly non-trivial to determine the most appropriate data augmentation operations. For example, a pixel-sensitive data transformation. e.g., Gaussian blurring and colorization, would hamper the reconstruction-abue -ablitt of the original addversary-benignit pair (x', x). Therefore, we focus on spatial image transformations, including rotation. translation, cropping & padding, cutout, and CutMix (Yun et al., 2019), which keep the original perturbation in a linear way. In Fig we evaluate the RED performance, in terms of pixel- level reconstruction emror and prediction-leve alignment accuracy, for different kinds of spatial image
5

---

Published as a conference paper at ICLR 2022
transformations. As see, CutMix and cropping & padding can increase the both performance simultaneously, considered as the appropriate augmentation to boost the RED. Furthermore, we empirically find that combining the two transformations further improve the performance.
Let T denote a transformation set, including cropping & padding and CutMix operations. With the aid of the denoising loss (2). PA regularization (3), and data transformations T, we then cast the overall training objective of CDD-RED as
minimize
f(x)1l A
oe oe (C (0x)
(4)
Edcasiio
with angmenntaiions
fya with data angmeutation via '
where I denotes properly-selected subset of T, and À > is regularization parameter. In the PA regularizer (4), we need to avoid the scenario of over-transforrmtion where data augmentation alters
the classifier's s original decision. This suggests T = {e E T [(((x)) = f((x))) F(x') }. where F represents the prediction label of the pre-trained classifier f. i.e., F()) (rggmxxf(())
5 EXPERIMENTS
We show the effectiveness of our proposed method in 5 aspects: a) reconstruction error of adversarial perturbation inversion, i.e., d(x, XRED). ). b) class-discrmminative ability of the benign and adversarial example estimate, i.e., PA Pabiignn and PAady by victim models, c) adversary saliency region recovery, ie., attribution alignment, and d) RED evaluation over unseen attack types and adaptive attacks.
5.1 EXPERIMENT SETUP
Attack datasets. To train and test RED models, we generate adversarial examples on the ImageNet dataset (Deng et al., 2009). We consider 3 attack methods including PGD (Madry et al. (017)) FGSM (Goodfellow et al. 2014), and CW attack (Carlini Wagner 2017). applied to 5 models including pre-trained ResNet18 (Res 18), Resnet50 (Res50) (He et al. 2015), VGGI6, VGGI9, and Inception V3) (Szegedy et al. 2015). The detailed parameter settings can be found in Appendix B Furthermore, to evaluate the RED performance unseen perturbation types during training, additional 2K adversarial examples generated by AutoAttack (Croce & Hein 2020 and IK adversarial examples generated by Feature Attack (Sabour al.) 2015) are included the unseen testing dataset. AutoAttack is applied on VGG19, Res50 and two new victim models, i.e., Alexet and Robust Resnet50 (R-Re RRsee500,, via fast adversarial training (Wong al. 2020) while Feature Attack is applied on VGG19 and Alexnet. The rational behind considering Feature Attack is that feature adversary has been recognized an effective way circumvent adversarial detection (Tramer et al. 2020). Thus, it provides a supplement on detection-ave -arre attacks.
RED model configuration, training and evaluation. During the training of the RED denoisers, VGG19 (Simonyan & Zisserman 2015) is chosen as the pretrained classifier of for PA regularization. Although different victim models were used for generating adversarial examples,
will show that the inference guided by VGG19 is able to accurately estimate the true image class and the intent of the adversary. In terms of the architecture of Dg. DrCNN (Zhang al. 2017) is adopted. The RED problem is solved using an Adam optimizer (Kingma & Ba 2015) with the initial learning rate of 10-1, which decays 10 times for every 140 training epochs. In (4), the regularization parameter is set as 0.025. The transformations for data augmentation include CutMix and cropping & padding. The maximum number of training epochs is set 300. The computation cost and ablation study of CDD-RED are in Appendix D and respectively.
Baselines. We compare CDD-RED with two baseline approaches: a) the conventional denoising- only (DO) approach with the objective function (2): b) The state-of-the-art Denoised Smoothing (DS) (Salman et al., 2020) approach that considers both the reconstruction error and the PA for benign examples in the objective function. Both methods are tuned to their best configurations.
5.2 MAIN RESULTS
Reconstruction error XRED) and PA. Table presents the comparison of CDD-RED with the baseline denoising approaches in terms of d(x, XRED), d(f(x), (XRE))) J

---

Published as a conference paper at ICLR 2022
PAbenigu- and PAadv the testing dataset. As see, approach (CDD-RED) im- proves the diiassiiscrmminative ability from benign perspective by 42.91% and adversarial per- spective by 8.46% with slightly larger reconstruction error compared with the DO approach. In contrast to DS, CDD-RED achieves similar PAbenign but improved pixel-Ievel denoising error and PAddv Fur- thermore, CDD-RED achieves the best logit-level recon- struction error for both f(XRED) and f (X among the three approaches. This implies that XRED rendered by CDD-RED can achieve highly similar prediction to the true benign example x, and the perturbation estimate XRED yields similar misclassification effect to the ground-truth perturbation. Besides, CDD-RED is robust against attacks with different hyperparame- ters settings, details can be found in Appendix
Table 1: The performance comparison among DO, DS and CDD-RED the testing dataset.
DO
DS
CDD-RED
dix. XARDD
9.32
19.19
13.04
di, d1f(x). ((XeED))
47.81
37,21
37007
417(x)) [[nann))
115.09
150.02
78.21
PAuciia
42.80%
86.04%
88.71%
PAudv
71.97%
80.4%%
BQ
Goof
Heys
Burige comple MANe
Ath. comngee
Interpretation (7) of henign (X/XRED) and adversarial (x' (x'XRRED) image w.r.t. the true label y= ppaarmign' and the adversary targeted lahel y "shoer curtain'. We compare three methods of RED training. DO, DS, and CDD-RED our method, the ground-truth interpretation. Given RED method, the first column I (XRED, y) versus [XRRED. y). the second column is [XRED, s/) versus I (XRED y/ and all maps under each RED method are normalized w.r.t. their largest value. For the ground truth, the first column is I [xx, y) versus [xx,, y). the second column is [xx v') versus [xx, y/).
Figure 5:
Attribution alignment. In addition to pixel-level alignment and prediction-level alignment to evaluate the RED performance, attribution alignment is examined what follows. Fig. presents attribution maps generated by GradCAM in terms of [(x, y). [(x', y). [(x, v'), and [(x', y ). where x' denotes the perturbed version of x, and y' is the adversarially targeted label. From left to right is the attribution map over DO, DS, CDD-RED (our method), and the ground-truth Compared with DO and DS, CDD-RED yields a closer attribution alignment with the ground-truth especially when making comparison between [XRRED, y) and 7(x,y). At the dataset level, Fig. shows the distribution of attribution loU scores. It is observed that the IoU distribution of CDDDRED, compared with DO and DS, has denser concentration over the high-value area, corresponding to closer alignment with the attribution map by the ad adversary. This feature indicates an interesting application of the proposed RED approach, which is to achieve the recovery of adversary's saliency region, in terms of the daasssiiccrminativee image regions that the adversary focused on.
1.0
3.0
1.0
0.8
0.8
<.000aap calccaygyryy
x K
(a) Denoising Only
(b) Denoised Smoothing
(c) CDD-RED (ours)
Figure 6: IoU distributions of the attribution alignment by three RED methods. Higher IoU is better. For each subfigure, the four IoU scores standing for IoU(XRED. x, y). (XU(XRED, x,y'), IoU (oUUXEEED x', y). and IoU(XRED x', x'.z').
7

---

Published as a conference paper at ICLR 2022
Groundruch
0.5
31
Acturecy
60
04
0.6
6.0
0.0
3.1
0.6
04
Perturbat portion 0
Pertuttot portion
Accuracy of xPp RBD
(b) Success rate of x RED
CCO
200
20-
DS
DO
0.5 Pertultation portonn
044
0.2
344
Lo
Perturbation portion
(c) (XED)
(d) d(xkED x)
Figure 7: Reverse engineer partially-erttureed data under different interpolation portion p.
RED vs. unforeseen attack types. The experiments on the recovery of unforeseen attack types are composed of two parts: a) partially-perturre data via linear interpolation, and b) the unseen attack type, AutoAttack, Feature Attack, and Adaptive Attack. More attack results including adversarial attacks on CIFAR-10 10 dataset, Wasserstein minimized attacks, and attacks on smoothed classifiers can be found in Appendix
First, we construct partially-perturber data by adding a portion p E (0%, 20%, ... 100%) of the perturbation x' x to the true benign example x, namely, x'2 x p(x' x). The interpolated x'P is then used as the input to an RED model. We aim to investigate whether or not the proposed RED method can recover partial perturbations (even not successful attacks).
Fig. 7 (a) and (b) show the the prediction alignment with y and 1// of the adversarial example estimate x RED x Dx(x'") + x by different RED models. Fig. (c) shows the logit distance between the prediction of the partially-perturbel adversarial example estimate and the prediction of the benign example while Fig. (d) demonstrates the pixel distance between XRED and the benign example.
First, a smaller gap between the ground-trute truth curve (in red) and the adversarial example estimate xPP curve indicates a better performance. Fig. (a) and (b) show that CDD-RED estimates the closest adversary's s performance to the ground truth in terms of the prediction accuracy and attack success rate. This also verified by the distance of prediction logits Fig (c). Fig. (d) shows that DS largely over-estimates the additive perturbation, while CDD-RED maintains the perturbation estimation performance closest to the ground truth. Though DO closer to the ground-truu -ruth than CDD-RED at p 40%, DO is not able to recover of other performance metrics. For example, in Fig. 7 (b) at p 0.2, 'Pp by DO achieves lower successful attack rate compared CDD-RED and the ground-truth.
precise adversarial perturbation in terms
Moreover, as for benign examples with p 0% perturbations, though the RED denoiser does not see benign example pair (x, x) during training, keeps the performance of the benign example re- covery. CDD-RED can handle the case with mixture of adversarial and benign examples. That to say, even if be- nign example, detected as adversarial, is wrongly fed into the RED framework, method can recover the original per- turbation close to the ground truth. See Appendix G for details.
Table The d(x, XRED), PAbenigu and PAadv performance the denoisers on the unforeseen perturbation type AutoAttack, Feature Attack, and Adaptive Attack.
DO
DS
CDD-RED
AutoAttack
6.41
16.64
8.81
d(x, XACD)
Feature Attack
3.51
16.14
7.99
Adaptive Attack
9.76
16.21
12.24
AutoAltack
84.69%
92.64%
94.58%
PAtsnigg
Feature Attack
82.90%
90.75%
93.25%
Adaptive Attack
33.20%
27.27%
36.29%
AutoAttack
85.53%
83.30%
88.39%
PAwdv
Feature Attack
26.97%
35.84%
63.48%
Adaptive Attack
51.21%
55.41 %
57.11%
8

---

Published as a conference paper at ICLR 2022
Table shows the RED performance the unseen attack type, AutoAttack, Feature Attack, and Adaptive Attack. For Auto and Feature Attack, CDD-RED outperforms both DO and DS in terms of PA from both benign and adversarial perspectives. Specifically, CDD-RED increases the PAady for Feature Attack by 36.51 1..5%% % and 27. 27.64% compared to DO and DS, respectively.
As for the adaptive attack (Tamer al. 2020
assume that the attacker has access to the knowledge of the RED model, i.e., Dg- It can then perform the PGD attack method to generate successful prediction-evasion attacks even after taking the RED operation.
We use PGD methods to generate such attacks within the -bll of perturbation radius = 20/2255, Table[[soowe that Adaptive Attack is much stronger than Feature Attack and AutoAttack, leading to larger reconstruction error and lower PA. However, CDD-RED still outperforms DO and DS in PAbeniga and PAadv. Compared to DS, achieves better trade-off with denoising error d(x, XRED).
In general, CDD-RED can achieve high PA even for unseen attacks, indicating the generalization- ability of our method estimate not only new adversarial examples (generated from the same attack method), but also new attack types.
RED to infer correlation between adversaries. In what follows, investigate whether the RED model guided by the single classifier (VGG19) enables to identify different adversary classes, given by combinations of attack types (FGSM, PGD, CW) and victim model types (Res18, Res50, VGG16, VGG19, V3).
Fig. 8 presents the corre- lation between every two
sucercae
hodorcoM
vaceecce
adversary classes in the
logit space. Fig. 8 (a) shows the ground-truth correlation map. 8ig. (b) shows correlations be- tween logits of XRED estimated by our RED method (CDD-RED) and logits of the true x'. Along the diagonal of each correlation matrix, the darker implies the bet- ter RED estimation under the same adversary class. By peering into off-diagonal entries, we find that FGSM attacks are more resilient to the choice of a victim model (see the cluster of high correlation values at the top left corner of Fig. 8). Meanwhile, the proposed CDD-RED precisely recovers the correlation behavior of the true adversaries. Such correlation matrix can help explain the similarities between different attacks' properties. Given an inventory of existing attack types, if new attack appears, then one can resort to RED to estimate correlations between the new attack type and the existing attack types. Based on the correlation screening, it can infer the properties of the new attack type based on its most similar counterpart in the existing attack library; see Appendix [.2 Inspired by the correlation, RED-synthesized perturbations also be used transfer attack well; see Appendix H
wosdow
20000
(a)
Groundtruth
(b) CDD-RED (ours)
Figure 8: Correlation matrices between different adversaries. For each cor-
relation matrix, rows and columns represent adversarial example estimate XRED and true adversarial example x' (For the ground-truth correlation trix, XRED = x'). Each entry represents the average Spearman rank correlation between the logits of two adversary settings E ((victim model, attack type)).
Other applications of RED. In Appendix we also empirically show that the proposed RED approach can be applied to adversarial detection, attack identity inference, and adversarial defense.
CONCLUSION
In this paper, we study the problem of Reverse Engineering of Deceptions (RED), to recover the attack signatures (e.g. adversarial perturbations and adversary saliency regions) from an adversarial instance. To the best of our knowledge, RED has not been well studied. Our work makes a solid step towards formalizing the RED problem and developing a systematic pipeline, covering not only a solution but also a complete set of evaluation metrics. We have identified series of RED principles, ranging from the pixel level to the attribution level, desired to eeverse-enginee adversarial attacks. We have developed and data augmentation into an image denoising network. With extensive experiments, our approach outperforms the existing baseline methods and generalizes well to unseen attack types.
effective denoiser-assisted RED approach by integrating mias--iscrimiittion

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGMENT
The work is supported by the DARPA RED program. We also thank the MIT-IBM Watson Al Lab, IBM Research for supporting with GPU computing resources.
REFERENCES
Vegard Antun, Francesco Renna, Clarice Poon, Ben Adcock, and Anders C Hansen. On instabilities of deep learning in image reconstruction and the potential costs of ai. Proceedings of the National Academy of Sciences, 2020.
A. Athalye, L.. Engstrom, A. Ilyas, and K. Kwok. Synthesizing robust adversarial examples. In International Conference Machine Learning (ICML), 2018.
Akhilan Boopathy, Sijia Liu, Gaoyuan Zhang, Cynthia Liu, Pin- Yu Chen, Shiyu Chang, and Luca Daniel. Proper network interpretability helps adversarial robustness in classification. In Interna- fional Conference on Machine Learning (ICML), 2020.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy (S&P). IEEE, 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. In Advances Neural Information Processing Systems (NeurIPS), 2019.
Pin-Yu Chen, Yash Sharma, Huan Zhang, Jinfeng Yi, and Cho-Jui Hsieh. Ead: elastic-net attacks to deep neural networks via adversarial examples. arXiv, 2017a.
Pin-Yu Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks deep neural networks without training substitute models. In Proceedings of the ACM Workshop on Artificial Intelligence and Security. ACM, 2017b.
Shang-Tse Chen, Cory Cornelius, Jason Martin, and Duen Horng Polo Chau. Shapeshifter: Robust physical adversarial attack on faster r-cnn object detector. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML). Springer, 2018.
T. Chen, S. Liu, S. Chang, Y. Cheng, L. Amini, and Z. Wang. Adversarial robustness: From self- supervised pretraining to fine-tuning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Minhao Cheng, Simranjit Singh, Patrick Chen, Pin-Yu Chen, Sijia Liu, and Cho-Jui Hsieh. Sign-opt: A query-efficient hard-label adversarial attack. arXiv, 2019.
Minhao Cheng, Jinfeng Yi, Pin- Yu Chen, Huan Zhang, and Cho-Jui Hsieh. Seq2sick: Evaluating the robustness of sequence-to io-ueueee models with adversarial examples. In Proceedings of the AAAI Conference on Artificial Inielligence (AAAI), 2020.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. International Conference on Machine Learning, pp. 1310-1320. PMLR, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference Machine Learning (ICML). PMLR, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009.
Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading Defenses Transferable Adversarial Examples by Translation-Invann Invariant Attacks. In CVPR, 2019.
K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, A. Prakash, T. Kohno, and D. Song. Robust physical-world attacks on deep learning visual classification. In Proceedings of the Conference Computer Vision and Pattern Recognition (CVPR), 2018.
10

---

Published as a conference paper at ICLR 2022
Lijie Fan, Sijia Liu, Pin- Yu Chen, Gaoyuan Zhang, and Chuang Gan. When does contrastive learning preserve adversarial robustness from pretraining to finetuning? Advances in Neural Information Processing Systems, 34, 2021.
Samuel G Finlayson, John D Bowers, Joichi Ito, Jonathan L Zittrain, Andrew Beam, and Isaac Kohane. Adversarial attacks on medical machine learning. Science, 363(6433) 2019.
2019.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel. On the (statistical) detection of adversarial examples. arXiv preprint arXiv:1702.06280, 2017.
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CoRR, 2015.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
Juncheng Li, Frank Schmidt, and Zico Kolter. Adversarial camera stickers: A physical camera-based attack on deep learning systems. International Conference on Machine Learning (ICML), 2019.
Qizhang Li, Yiwen Guo, and Hao Chen. Practical no-box adversarial attacks against dnns. Advances in Neural Information Processing Systems (NeurIPS), 2020.
Fangzhou Liao, Ming Liang, Yinpeng Dong. Tianyu Pang. Xiaolin Hu, and Jun Zhu. Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser. arXiv:1712.00 179.002766 [cs], May 2018.
Sijia Liu, Pin-Yu Yu Chen, Xiangy Chen, and Mingyi Hong. signSGD via zeroth-order oracle. In International Conference on Learning Representations (ICLR), 2019.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into Transferable Adversarial Examples and Black-box Attacks. arXiv:1611.02770 [cs], February 2017.
Yan Luo, Xavier Boix, Gemma Roig, Tomaso Poggio, and Qi Zhao. Foveation-based mechanisms alleviate adversarial examples. arXiv preprint 15Xiv: 11/55/11..69.... 2015.
Aleksander Madry. Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 060836 2017.
Dongyu Meng and Hao Chen. Magnet: two-pronged defense against adversarial examples. In the Conference on Computer and Communications Security (CCS). ACM, 2017.
Jan Hendrík Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. On detecting adversarial perturbations. arXiv preprint 1702::/77020426.. 2017.
Zhonghan Niu, Zhaoxi Chen, Linyi Li, Yubin Yang, Bo Li, and Jinfeng Yi. On the Limitations of Denoising Strategies Adversarial Defenses. arXiv:2012.098: [es), 2020.
Ren Pang, Xinyang Zhang, Shouling Ji, Xiapu Luo, and Ting Wang. Advmind: Inferring adversary intent of black-box attacks. In the International Conference on Knowledge Discovery & Data Mining (KDD), 2020.
Nicolas Papernot, Patrick McDaniel, and lan Goodfellow. Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples. arXiv: 1605.07277 [cs], 2016a.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In IEEE European Symposium on Security and Privacy (EuroS&P). IEFF, 2016b.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical Black-Boo Box Attacks against Machine Learning.
[cs], 2017.
11

---

Published as a conference paper at ICLR 2022
Sara Sabour, Yanshuai Cao, Fartash Faghri, and David J Fleet. Adversarial manipulation of deep representations. arXiv preprint arxiv:1511.05122, 2015.
Hadi Salman, Mingjie Sun, Greg Yang, Ashish Kapoor, and Zico Kolter. Denoised smoothing: A provable defense for pretrained classifiers. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization, International Journal of Computer Vision, 2020.
Alex Serban, Erik Poll, and Joost Visser. Adversarial examples on object recognition: A comprehen- sive survey, ACM Computing Surveys (CSUR), 53(3)11-38, 2020.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Ali Shafahi, W. Ronny Huang, Christoph Studer, Soheil Feizi, and Tom Goldstein. Are adversarial examples inevitable? arXiv:1809.022 0014 [cs, stat], February 2020.
Karen Simonyan and Andrew 7isserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. In International Conference on Learning Representations (ICLR), 2015.
Hossein Souri, Pirazh Khorramshahi, Chun Pong Lau, Micah Goldblum, and Rama Chellappa. Iden- tification of attack-specific signatures in adversarial examples. arXiv preprint arXiv:2110.0663 2021.
Shashank Srikant, Sijia Liu, Tamara Mitrovska, Shiyu Chang, Quanfu Fan, Gaoyuan Zhang, and Una-May O'Reilly. Generating adversarial computer programs using optimized obfuscations. In International Conference Learning Representations (ICLR), 2021.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jonathon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. CoRR, 2015.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. arXiv preprint 20Xiv:2002.08347, 08224,
Jonathan Uesato, Dean-Baptiste Alayrac, Po-Sen Huang, Robert Stanforth, Alhussein Fawzi, and Pushmeet Kohli. Are labels required for improving adversarial robustness? Neural Information Processing Systems (NeurIPS), 2019.
Bartosz Wójeik, Pawel Morawiecki, Marek Smieja, Tomasz Krzyžek, Przemyslaw Spurek, and Jacek Tabor. Adversarial examples detection and analysis with layer-wise autoencoders. arXiv preprint arXiv:2006.1001: 2020.
Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. arXiv preprint arXiv:1711.00851, 2017.
Eric Wong, Frank Schmidt, and Zico Kolter. Wasserstein adversarial examples via projected sinkhorn iterations. In International Conference on Machine Learning, pp. 6808-6817. PMLR, 2019.
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations (ICLR), 2020.
Chaowei Xiao, Jun- Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially transformed adversarial examples. In International Conference on Learning Representations (ICLR), 2018.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. Adversarial examples for semantic segmentation and object detection. In Proceedings of the IEEE International Conference Computer Vision (ICCV), 2017.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuille. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2019.
12

---

Published as a conference paper at ICLR 2022
Kaidi Xu, Sijia Liu, Gaoyuan Zhang, Mengshu Sun, Pu Zhao, Quanfu Fan, Chuang Gan, and Xue Lin. Interpreting adversarial examples by activation promotion and suppression. arXiv preprint arXiv:1904.02057 2019a.
Kaidi Xu, Sijia Liu, Pu Zhao, Pin-Yu Chen, Huan Zhang, Quanfu Fan, Deniz Erdogmus, Yanzhi Wang, and Xue Lin. Structured adversarial attack: Towards general implementation and better interpretability. In International Conference on Learning Representations (ICLR). 2019b.
Kaidi Xu, Gaoyuan Zhang, Sijia Liu, Quanfu Fan, Mengshu Sun, Hongge Chen, Pin- Yu Chen, Yanzhi Wang, and Xue Lin. Evading real-time person detectors by adversarial t-shirt. arXiv preprint arXiv: 10099, 2019c.
Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, and Michael Jordan. MI-loo Detecting adversarial examples with feature attribution. In Proceedings the AAAI Conference Artificial Intelligence (AAAI), 2020.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
2019.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference on Machine Learning (ICML), 2019.
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising, IEEE Transactions on Image Processing, 2017.
13

---

Published as a conference paper at ICLR 2022
A RED BY PGD ATTACK: PERFORMING PGD BACK TO THE TRUE CLASS
naive approach to reverse engineer the adversarial perturbation is using the target PGD attack revert the label back the groundtruth. However, this requires additional assumptions. First, since PGD is test-time deterministic optimization approach for perturbation generation, its targeted implementation requires the true class of the adversarial example, which could be unknown at testing time. What is more, one has to pre-define the perturbation budget for PGD. This value is also unknown. Second, performing PGD back to the true class might not exactly recover the ground-truth adversarial perturbations. By contrast, its RED counterpart could be over-perturbed. To make it more convineing, we applied the target 1x PGD) attack method to adversarial examples generated by PGD) (assuming true class, victim model, and attack budget are known). We tried various PGD settings (PG refers to PGD attack using 10 steps and E = 10/255). Eventually, we compare these results
CDD-RED method in Ta
Given that the average reconstruction error between x and x' is 20.60, we can see from Table Al that PGD attacks further enlarge the distortion from the clean data. Although PGD attacks achieve high accuracy after reverting the adversarial data back to their true labels, the resulting perturbation estimate is far from the ground-truth in terms of their prediction alignment. We can tell from the low PAady by PGD methods that XRED does not align with the input x' at all.
Table AI: The performance comparison among DO, DS, and CDD-RED on the CIDAR-10 dataset.
PGD PGDIOe20/26s
PGDIO
PGD20e21/255
CDD-RED
d(x, XRED)
27.63
22.67
27.53
11.73
PApeniga
96. 20%
82.60%
99.80% %
83.20%
PAady
6.20%
7.20%
4.80%
97.40%
B DATASET DETAILS.
The training and testing dataset is composed of three attack methods including PGD Madry et al. (2017). FGSM including pre-trained ResNet18 8 (Res 8), Resnet50 (Re550) He et al. (2015)))) VGG16, VGG19, and Inception (3)) Szegedy et al. (013)). By default, PGD attack and FGSM attack are bounded by -nor constraint and CW attack is bounded by fg-norm. The range of the perturbation strength for PGD attack and FGSM attack are [1/255, 40/255) and [1/255. 20/255), respectively. As for CW attack, the adversary's confidence parameter k is uniformly sampled from from [1. 20). One attack method applied to one victim model to obtain 3K successfully attacked images. As consequence, 45K (3 x 5 x 3K) adversarial images are generated in total: 37.5K for training and 7.5K for validating. The testing set contains 28K adversarial images generated with the same attack method & victim model.
(2014). and CW attack/Carlim Wagner (2017). applied models
C PERFORMANCE ON MORE ATTACK TYPES
We show more evaluations of the RED approaches on unforeseen attacks during training. The denoisers are all trained on the training dataset containing adversarial examples generated on the ImageNet dataset, as in Appendix The data includes adversarial examples generated on the CIFAR-10 dataset in C. Wasserstein minimized attackers in C.2 and attacks on smoothed classifiers
C.1 PERFORMANCE ON CIFAR- 10 DATASET
We further evaluate the performance of the RED approaches the adversarial examples generated on the CIFAR- 10 dataset. As the denoiser imput agnostic, we directly test the denoiser trained on adversarial examples generated
the ImageNet dataset. Here consider the 10-step PGD-Iua attack generation method with the perturbation radius = 8/255. And these examples are not seen during our training. As shown in the Table A2 the proposed CDD-RED method provides the best.
14

---

Published as a conference paper at ICLR 2022
PAcleam and PAadv with slightly larger d(x, XRED) than DO. This is not surprising DO focuses only on the pixel-level denoising error metric. However, as illustrated in Sec. the other metric like PA also plays key role in evaluating the RED performance.
Table A2: The performance comparison among DO, DS, and CDD-RED the CIDAR-10 dataset.
DO
DS
CDD-RED
d(x, XRED)
0.94
4.50
1.52
PApenign
9.90%
71.75%
71.80%
PNadv
92.55%
89.70%
99.55 %
C.2 PERFORMANCE ON WASSERSTEIN MINIMIZED ATTACKERS
We further show the performance Wasserstein minimized attackers, which is an unseen attack type during training. The adversarial examples are generated on the ImageNet sub-dataset using Wasserstein ball. We follow the same setting from Wong al. (2019), where the attack radius is 0.1 and the maximum iteration is 400 under lo norm inside Wasserstein ball. The results are shown in Table A3 As Ip attack types considered in the paper, justified by the lower prediction alignement PAbenign across all methods. This implies possible limitation of supervised training over (12 or (xc) attacks. One simple solution is to expand the training dataset using more diversified attacks (including Wasserstein attacks). However, we believe that the further improvement of the generalization ability of RED deserves a more careful study in the future, e.g., an extension from the supervised learning paradigm to the -spef-supe) pre-training and finetuning paradigm.
see, Wasserstein attack is more challenging attack type for RED than the
Table A3: The performance comparison among DO, DS. and CDD-RED on Wasserstein minimized attackers.
DO
DS
CDD-RED
d(x, x, XRED)
9.79
17.38
11.66
PAbenign
92.50%
96.20%
97.50%
PAadv 35.00%
37.10%
37.50%
C.3 PERFORMANCE ON ATTACKS AGAINST SMOOTHED CLASSIFIERS
We further show the performance on the attack against smoothed classifiers, which is an unseen attack type during training. A smoothed classifier predicts any input using the majority vote based on randomly perturbed inputs (2x, o² I² [ohee al. (2019). Here we consider the 10-step PGD-Lcc attack generation method with the perturbation radius 20/255, and a 0.25 for smoothing. As shown in Table A4, the proposed CDD-RED method provides the best PA and PA with slightly larger d(x. xxx, XRED) than DO.
Table A4: The performance comparison among DO, DS, and CDD-RED on the PGD attack against. smoothed classifiers.
DO
DS
CDD-RED
d x, XRED)
15.53
22.42
15.89
PApenign
13%
70.88%
76.10%
PAadv
58.24%
58.79%
61.54%
D COMPUTATION COST OF RED
We measure the computation cost on a single RTX Titan GPU. The inference time for DO, DS, and CDD-RED is similar they use the same denoiser architecture. For the training cost, the maximum
15

---

Published as a conference paper at ICLR 2022
training epoch for each method is set as 300. The average GPU time (in seconds) of one epoch for DO, DS, and CDD-RED is 850, 1180, and 2098, respectively. It is not surprising that CDD-RED is conducted over more complex RED objective. Yet. the denoiser only needs to be trained once to reverse-engineer a wide variety of adversarial perturbations, including those unseen attacks during the training.
E ABLATION STUDIES ON CDD-RED
In this section, we present additional experiment results using the proposed CDD-RED method for reverse engineering of deception (RED). We will study the effect of the following model/parameter choice the performance of CDD-RED: 1) pretrained classifier f for PA regularization, 2) data augmentation strategy T for regularization, and 3) regularization parameter À that strikes balance between the pixel-level reconstruction error and PA in 4 Recall that the CDD-RO REDD method in the main paper sets f as VGGI9, T {t T (((())
F(x') }. and À 0.025.
E.1 PRETRAINED CLASSIFER f.
Table A5: The performance of CDD-RED using different pretrained classifier f (either Res50 or R-Res50) compared with the default setting VGG19.
f=Res50
[=R-Res50
dx, XRED
12.84 (4 0.20)
10.09 (L 2.95)
( is better)
PAbenign
84.33% (11.38%)
57.88% (4 27.83%)
( is better)
PAadv
79.94% (L 0.49%)
71.02% (L 9.40%)
(f is better)
Besides setting/ f as VGG19, Table shows the RED performance using the other pretrained models, i.e., Res50 and R- Rese500. As we can see, the use of Res50 yields the similar performance as VGG19. Although some minor improvements
observed in terms of pixel level reconstruction error, the PA performance suffers a larger degradation. Compared to Res50, the use of an adversarially robust model R-Res50 significantly hampers the RED performance. That is because the adversarially robust model typically lowers the prediction accuracy, it is not able to ensure the class-diseriminative ability in the non-adversarial context, namely, the PA regularization performance.
E.2 DATA SELECTION FOR PA REGULARIZATION.
Table A6: Ablation study on the selection of ( and without (w/o) T)) for training CDD-RED, compared with T = {t E TI F(t(x)) F(x), (t(x')) F(x).
T=T
w/o T
d(x (x, XRED)
15.52 (1 2.48)
13.50 (1 0.46)
is better)
PApenign
83.64% (L 2.07%)
84.04% (11.67%)
is better)
PAadv
75.92% (J 4.51%)
79.99% (L 0.44%)
(1 is better)
As data augmentation might alter the classifier's s original decision in ( we study how f affects the RED performance by setting T the original data. i.e.. without data augmentation, and all data, i... T = T. A6 shows the performance of different 7 configurations, compared with the default setting. The performance is measured the testing dataset. As see, using all data original
16

---

Published as a conference paper at ICLR 2022
data cannot provide overall better performance than CDD-RED. That is because the former might cause over-transformmtion, and the latter lacks generalization ability.
E.3 REGULARIZATION PARAMETER X.
Table A7: Ablation study on the regularization parameter À (0, 0.0125, and 0.05) for CDD-RED
training, compared with A=0.025.
X=0
A=0.0125
A=0.05
d(x, X, XRED
8.92 (4 4.12)
12.79 (4 0.25)
14.85 (1 2.13)
( is better)
PAhnnign
47.61% (138.10%)
81.00% (14.71%)
85.56% (J 0.15%)
(t is better)
PAadv
73.37% (47.06%%
78.25% (L 2.18%)
79.94% (J 0.49%)
(t is better)
The overall training objective of CDD-RED is which is the weighted sum of the reconstruction error and PA with a regularization parameter A. We further study the sensitivity of CDD-RED to the choice of X, which is set by 0, 0.0125, and 0.05. Table A7 shows the RED performance of using different À values, compared with the default setting X = 0.025. We report the average performance the testing dataset. As we can see, the use of O, which corresponds training the denoiser without PA regularization, achieves lower pixel-level reconstruction error d(x, XRED). but degrades the prediction-level performance, especially PAbenign greatly In the same time, À 0 provides smaller pixel-level reconstruction error with a better PA performance than DO, which indicates the importance of using proper augmentations. We also observe that keep increasing À to 0.05 able to provide a better PA.
F ABLATION STUDY OF DIFFERENT ATTACK HYPERPARAMETE SETTINGS
We test PGD attacks generated with different step sizes, including 4/255 and 6/255, and with and without random initialization (RI). Other hyperparameters are kept the same. The adversarial examples are generated by the same set of images w.r.t. the same classifier ResNet-50. The results are shown in Table A8 As we can see, the RED performance is quite robust against the varying hyperparameters
PGD attacks. Compared with DO, CDD-RED greatly improves PAbenign and achieves higher PA with slightly larger d(x, XRED). Compared to DS, CDD-RED achieves slightly better PA but with a much smaller reconstruction error d(x, XRED).
Table A8: RED performance for PGD with different hyperparameter settings, including stepsize as 4/ 4/255 and 6/255, and with and without RI.
Without RI/ With RI
Stepsize
DO
DS
CDD-RED
4/255
5.94/5.96
16.56/16.57
8.91/8.99 94
d(x, XRED)
6/255
5.99/5.98
16.52/16.50
8.97/8.94
4/255
40.00% .00%
91.00% .00%
94.00%
PAbenign
6/255
11.00%/61.006
94.00%/93.009%
95.00%/94.500 %
4/255
97.50%
97.50%
99.50%
PAadv
6/255
96.50%
95.50%
98.50%
G PERFORMANCE WITHOUT ADVERSARIAL DETECTION ASSUMPTION
The focus of RED is to demonstrate the feasibility of recovering the adversarial perturbations from adversarial example. However, in order show the RED performance the global setting, experiment with mixture of 1) adversarial images, 2) images with Gaussian noise (images with random perturbations), and 3) clean images the ImageNet dataset. The standard deviation of
17

---

Published as a conference paper at ICLR 2022
the Gaussian noise is set as 0.05. Each type of data accounts for 1/3 of the total data. The images are shuffled to mimic the live data case. The overall accuracy before denoising is 63.08%. After denoising, the overall accuracy obtained by DO, DS, and CDD-RED RED is 72.45%, 88.26%, and 89.11 %, respectively. During the training of the denoisers, random noise is not added to the input.
H TRANSFERABILITY OF RECONSTRUCTED ADVERSARIAL ESTIMATE
We further examine the transferability of RED-synthesized perturbations. In the experiment, RED- synthesized perturbations are generated from PGD attacks using ResNet-50. We then evaluate the attack success rate (ASR) of the resulting perturbations transferred to the victim models ResNet- 18. VGG16, VGG19, and Inception-Y V3. The results are shown in Table A9 As we can see, the perturbation estimates obtained via our CDD-RED yield better attack transferability than DO and DS. Therefore, such RED-synthesizer perturbations can be regarded as an efficient transfer attack method.
Table A9: The tranferability of RED-synthesized perturbations.
DO
DS
CDD-RED
ResNet-18 18
66.50%
70.50%
77.50
VGGI6
71.50%
74.00%
81.00
VGGT9
71.50%
70.00%
80.00% %
Inception- V3
86.00%
85.50%
90.00 %
I POTENTIAL APPLICATIONS OF RED
In this paper, we focus recovering attack perturbation details from adversarial examples. But in the same time, the proposed RED can leveraged for various potential interesting applications. In this section, we delve into three applications, including RED for adversarial detection in [T inferring the attack identity I.2 and provable defense in I33
I.1 RED FOR ADVERSARIAL DETECTION
The outputs of RED can be looped back help the design of adversarial detectors. Recall that our proposed RED method (CDD-RED) can deliver an attribution alignment test, which reflects the sensitivity of input attribution scores the pre-RED and post-RED operations. Thus. if an input an adversarial example, then it will cause a high attribution dissimilarity (i.c., misalignment) between the pre-RED input and the Rost-RED input, i.e., T(x,, fxx) vs. T(D()), ((Dx))) following the notations in Section 3 In this sense, attribution alignment built upon 1(x, f(x) and T(D(x), f(D(=))) can be used as an adversarial detector. Along this direction, we conducted some preliminary results on RED-assisted adversarial detection, and compared the ROC performance of the detector using CDD-RED and that using denoised smoothing (DS). In Figure A1 based detector yields superior detection performance, justified by its large area under the curve. Here the detection evaluation dataset is consistent with the test dataset in the evaluation section the paper.
we observe that the CDD-RED
18

---

Published as a conference paper at ICLR 2022
ROC cetecling adverserial examples
0..
Toue
82-
(ours)
0.0
00
02
0/
0.6
c8
rabe Rosil
Figure A1: RoC of detecting adversarial examples.
1.2 RED FOR ATTACK IDENTITY INFERENCE
We consider another application to infer the attack identity using the reverse-engineered adversarial perturbations. Similar the setup of Figure g we achieve the above goal using the correlation screening between the new attack and the existing attack type library. Let 2' (e.g.. PGD attack generated under the unseen AlexNet victim model) be the new attack. We can then adopt the RED model D(-) to estimate the perturbations 8new z' (=="). And let x' Ask. denote the generated attack over the estimated benign data (==') but using the existing attack type Similarly, we can obtain the RED-generated perturbations 6, = I'Ack. D(x' (z'eee..) With the aid of &new and 6; for i, we infer the most similar attack type by maximizing the cosine similarity: i" argmax, co 8,). Figure A2 shows example to link the new Alexeer-generated PGD attack with the existing VGG19- generated PGD attack. The reason is elaborated on below. (1) Both attacks are drawn from the PGD attack family. And (2) in the existing victim model library (including ResNet. VGG, and Inception V3), VGG19 has the most similar architecture as AlexNet, both of which share a pipeline composed of convolutional layers following fully connected layers without residual connections.
Similaity between slexnetFGD vs. existing attacks in the library
R530PP3
WOLEFFSM
A.0
A.61
Figure A2: Similarity between AlexNet-genneraeed PGD vs. existing attacks in the library.
19

---

Published as a conference paper at ICLR 2022
1.3 RED FOR PROVABLE DEFENSE
We train the RED models to construct smooth classifiers, the resulting certified accuracy is shown in Figure A3 Here the certified accuracy defined by the ratio of correctly-predicet images whose certified perturbation radius is less than the & perturbation radius shown in the x-axis.
1.0
CDD-RED(Ours)
DS
0.8
Certified Accuracy
DO
Base classifier
0.6
0.4
0.2
0.0
0.0
0.5
1.0
82 radius
Figure A3: Certified Robustness by different methods.
20


Reviews:
[{"review_id": "yrd5P4oQB9", "reviewer": "Reviewer_Ah1k", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed reverse engineering topic is interesting and novel. The objective seems to be a strong step beyond denoising and could be a strong tool for better understanding and defending against adversarial examples. The evaluation metrics provide a good starting point for understanding the objectives of reverse engineering adversarial perturbations. The overall methodology is straight-forward and would be practically available to both researchers and the industry.\n\nHowever, the experimental results are limited to the ImageNet dataset, it is difficult to be confident that the experimental results would generalize beyond that dataset without incorporating more datasets. Adversarial examples are inputs specifically designed to fool deep learning models, if an attacker is aware of the use of this methodology would they be able to craft adversarial example to fool both the original and this model?\n", "summary_of_the_paper": "This paper proposes a methodology for reverse engineering adversarial perturbations. This allows a defender to recover the original image used to produce an adversarial example and may be an effective tool to mitigating adversarial example attacks.  The paper introduces the concept of reverse engineering adversarial perturbations, defines metrics that quantify reverse engineering performance, creates a framework for training a denoising model to find the adversarial perturbations, and compares the resulting model against adversarial denoising techniques.", "summary_of_the_review": "The major strengths of this paper are:\n\n•\tThe paper is novel and could be have a considerable impact.\n\n•\tThe paper provides a strong background that establishes a foundation for reverse engineering adversarial perturbations.\n\n•\tThe ability to reverse engineer adversarial perturbations could be a powerful tool for understanding adversarial examples better.\n\n•\tBecause the methodology is demonstrated to be able to handle benign inputs well the methodology could lead to effective defenses against adversarial examples.\n\nThe major weaknesses of the paper are:\n\n•\tThe work only evaluates the ImageNet dataset. And so, it is not clear how it would generalize to other scenarios.\n\n•\tWhile the work compares against attacks it wasn’t trained to recognize, it appears that these methodologies represent similar attacks. (L2 or Linf) attackers It is not clear from these results that the methodology would extend to other types of attackers including L0 or Wasserstein minimized attackers.\n\nAdditional Comments:\n\n•\tThe Adaptive Attack does not appear to be cited in the text.\n\n•\tThe Interpretation function in the second sentence of the “Attribution alignment” subsection appears to be formatted differently than the rest of the text.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "FY5YRiP3eZY", "reviewer": "Reviewer_e8YB", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The problem considered here is relatively less explored. The paper is well-organized, easy to read, and the algorithmic approach is sound. Overall the paper should be accessible to a wider audience.\n\n2. The experimental results show that the new architecture is more effective than the baseline approaches for the RED problem considered in this paper. \n\nWeaknesses:\n\n1. The authors do not provide any concrete applications of RED. They mention that the estimated REDs can be used for diagnosing the model in a forensic manner. But it is not clear what one can get out of such type of analysis. One natural application is using REDs for defending against adversarial examples. Given that one of the baselines Denoised Smoothing (DS) that the authors compare against can be used for constructing a provable defense, I am not sure why the RED approach cannot be leveraged to build some sort of defense. Another application can be using REDs for explainability,i.e., to perhaps locate blind spots/missing data in the training set.\n\n2. While the problem studied here is now, the method is somewhat straightforward and consists of combining two architectures via adding their respective losses. Further, unlike DS it appears that estimating the REDs cannot provide any theoretical guarantees.\n \n\nMore questions:\n\n1. Have you seen any significant differences in the distribution of the REDs generated by the different attacks, e.g., if certain perturbations can be \"only\" generated by a certain attack?\n\n2. How does the method compare with the baselines on estimating the REDs on smoothed classifiers?\n\n3. Will the method perform as well on smaller datasets, e.g., MNIST or CIFAR10? Similarly, can REDs be generalized to non-image domains?\n\n4. The text mentions that \"Based on the correlation screening, it can infer the properties of the new attack type based on its most similar counterpart in the existing attack library\", what can a user gain from inferring these properties?\n\n5. Given that many adversarial attacks are sensitive to the hyperparameters, e.g., initialization, step size for PGD etc., how does the alignment and reconstruction error vary on the different examples created by the same method on the same image wrt the same classifier?\n\n6. Can one build a library of adversarial perturbations collected from different sources and then to test a new model use those instead of running adversarial attacks? Will that provide some sort of reliability?\n", "summary_of_the_paper": "The paper considers the problem of automatically reconstructing adversarial perturbations from examples in a post-hoc manner. The authors argue that for an effective reconstruction, it is not sufficient to only minimize the reconstruction error but also it is essential to align the predictions of the original and their reconstructed versions. To achieve these goals, the authors combine a denoising network, with a prediction alignment network via a standard combination of their respective losses. The authors incorporate data augmentation to further improve the performance of their approach. The empirical result indicates that the new architecture is able to better balance the prediction alignment and the reconstruction error than the baselines.", "summary_of_the_review": "I believe that the paper explores an interesting problem but does not show any relevant practical applications for an end-user who might be interested in improving the robustness, interpretability, or fairness of their models before/as they are deployed in the real world. I encourage the authors to study which parts of the deployment pipeline can benefit from the estimated REDs.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "1hnNnNJVId2", "reviewer": "Reviewer_T72q", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n* Paper is easy to understand\n* Problem and threat model are well-formulated\n* Comprehensive evaluation metrics\n* Evaluations on adaptive attacks\n\nWeaknesses:\n\n* From figure 7d, it looks like DO is actually closer to groundtruth than RED is for p < 0.5 -- I may have missed it, but this is not really acknowledged or discussed in the paper\n* It's unclear how much more computationally expensive the RED model is, compared to the DO or DS approaches\n* How does one practically use this CDD-RED method? Run it on every image in the test-time / live data? In practice, Lp norm adversarial examples comprise of a small fraction of the test-time data/live, and we don't know which examples are the Lp norm adversarial examples. In this study, there are no \"control\" evaluations -- e.g., on images with random or no perturbations -- so it's unclear that CDD-RED will work better than baselines in the global setting.", "summary_of_the_paper": "This paper defines a new problem, Reverse Engineering of Deceptions (RED), that aims to reconstruct the adversarial perturbation applied to a clean image based on the adversarial image. Their contributions are the following:\n\n* RED problem formulation\n* Series of \"principles\" to estimate the perturbations (such as class-discriminative ability and data augmentations)\n* Empirical study that demonstrates the effectiveness of the authors' solution across various evaluation metrics and adversarial attacks", "summary_of_the_review": "This paper clearly defines the RED problem and conducts experiments to show that a classifier optimized for extracting adversarial perturbations can extract such perturbations better than 2 other baselines for a variety of adversarial attacks. A main concern is that the problem setting does not seem well-motivated: in what case will we _know_ whether an image is an Lp norm adversarial image and need to extract the adversarial perturbation? What could these perturbations be used for? Does the authors' proposed method work better than performing some PGD to go back to that true class? Finally, in table 2, the d(x, x_{RED}) is actually lowest for the DO baseline, and the DO baseline has comparable PA to CDD-RED. The results for CDD-RED do not seem significantly better.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The manuscrupt studies an unexplored problem: How to reverse-engineer adversarial perturbations from an adversarial image? This leads to a new adversarial learning paradigm—Reverse Engineering of Deceptions (RED). The authors formalize the RED problem and identify a set of principles crucial to the RED approach design. By integrating these RED principles with image denoising, they propose a new Class-Discriminative Denoising based RED framework, termed CDD-RED. 
The reviewers recognize that this topic is important and a promising research direction.
The reviewers are also satisfied with the respones from the authors.
In summary, this paper is recommended to be accepted as it is well-formulated, easy to follow, and has some merits, despite that it needs to be evaluated further.
Decision: Accept (Poster)