Reverse Engineering of Imperceptible Adversarial Image Perturbations

1 INTRODUCTION . Deep neural networks ( DNNs ) are susceptible to adversarially-crafted tiny input perturbations during inference . Such imperceptible perturbations , a.k.a . adversarial attacks , could cause DNNs to draw manifestly wrong conclusions . The existence of adversarial attacks was first uncovered in the domain of image classification ( Goodfellow et al. , 2014 ; Carlini & Wagner , 2017 ; Papernot et al. , 2016b ) , and was then rapidly extended to the other domains , such as object detection ( Xie et al. , 2017 ; Serban et al. , 2020 ) , language modeling ( Cheng et al. , 2020 ; Srikant et al. , 2021 ) , and medical machine learning ( Finlayson et al. , 2019 ; Antun et al. , 2020 ) . Despite different applications , the underlying attack formulations and generation methods commonly obey the ones used in image classification . A vast volume of existing works have been devoted to designing defenses against such attacks , mostly focusing on either detecting adversarial examples ( Grosse et al. , 2017 ; Yang et al. , 2020 ; Metzen et al. , 2017 ; Meng & Chen , 2017 ; Wójcik et al. , 2020 ) or acquiring adversarially robust DNNs ( Madry et al. , 2017 ; Zhang et al. , 2019 ; Wong & Kolter , 2017 ; Salman et al. , 2020 ; Wong et al. , 2020 ; Carmon et al. , 2019 ; Shafahi et al. , 2019 ) . Despite the plethora of prior work on adversarial defenses , it seems impossible to achieve ‘ perfect ’ robustness . Given the fact that adversarial attacks are inevitable ( Shafahi et al. , 2020 ) , we ask whether or not an adversarial attack can be reverseengineered so that one can estimate the adversary ’ s information ( e.g. , adversarial perturbations ) behind the attack instances . The above problem is referred to as Reverse Engineering of Deceptions ( RED ) , fostering a new adversarial learning regime . The development of RED technologies will also enable the adversarial situation awareness in high-stake applications . To the best of our knowledge , few work studied the RED problem . The most relevant one that we are aware of is ( Pang et al. , 2020 ) , which proposed the so-called query of interest ( QOI ) estimation model to infer the adversary ’ s target class by model queries . However , the work ( Pang et al. , 2020 ) was restricted to the black-box attack scenario and thus lacks a general formulation of RED . Furthermore , it has not built a complete RED pipeline , which should not only provide a solution to estimating the adversarial example but also formalizing evaluation metrics to comprehensively measure the performance of RED . In this paper , we aim to take a solid step towards addressing the RED problem . 1.1 CONTRIBUTIONS . The main contributions of our work is listed below . • We formulate the Reverse Engineering of Deceptions ( RED ) problem that is able to estimate adversarial perturbations and provides the feasibility of inferring the intention of an adversary , e.g. , ‘ adversary saliency regions ’ of an adversarial image . •We identify a series of RED principles to effectively estimate the adversarially-crafted tiny perturbations . We find that the class-discriminative ability is crucial to evaluate the RED performance . We also find that data augmentation , e.g. , spatial transformations , is another key to improve the RED result . Furthermore , we integrate the developed RED principles into image denoising and propose a denoiser-assisted RED approach . •We build a comprehensive evaluation pipeline to quantify the RED performance from different perspectives , such as pixel-level reconstruction error , prediction-level alignment , and attribution-level adversary saliency region recovery . With an extensive experimental study , we show that , compared to image denoising baselines , our proposal yields a consistent improvement across diverse RED evaluation metrics and attack generation methods , e.g. , FGSM ( Goodfellow et al. , 2014 ) , CW ( Carlini & Wagner , 2017 ) , PGD ( Madry et al. , 2017 ) and AutoAttack ( Croce & Hein , 2020 ) . 1.2 RELATED WORK . Adversarial attacks . Different types of adversarial attacks have been proposed , ranging from digital attacks ( Goodfellow et al. , 2014 ; Carlini & Wagner , 2017 ; Madry et al. , 2017 ; Croce & Hein , 2020 ; Xu et al. , 2019b ; Chen et al. , 2017a ; Xiao et al. , 2018 ) to physical attacks ( Eykholt et al. , 2018 ; Li et al. , 2019 ; Athalye et al. , 2018 ; Chen et al. , 2018 ; Xu et al. , 2019c ) . The former gives the most fundamental threat model that commonly deceives DNN models during inference by crafting imperceptible adversarial perturbations . The latter extends the former to fool the victim models in the physical environment . Compared to digital attacks , physical attacks require much larger perturbation strengths to enhance the adversary ’ s resilience to various physical conditions such as lightness and object deformation ( Athalye et al. , 2018 ; Xu et al. , 2019c ) . In this paper , we focus on ` p-norm ball constrained attacks , a.k.a . ` p attacks , for p ∈ { 1 , 2 , ∞ } , most widely-used in digital attacks . Examples include FGSM ( Goodfellow et al. , 2014 ) , PGD ( Madry et al. , 2017 ) , CW ( Carlini & Wagner , 2017 ) , and the recently-released attack benchmark AutoAttack ( Croce & Hein , 2020 ) . Based on the adversary ’ s intent , ` p attacks are further divided into untargeted attacks and targeted attacks , where in contrast to the former , the latter designates the ( incorrect ) prediction label of a victim model . When an adversary has no access to victim models ’ detailed information ( such as architectures and model weights ) , ` p attacks can be further generalized to black-box attacks by leveraging either surrogate victim models ( Papernot et al. , 2017 ; 2016a ; Dong et al. , 2019 ; Liu et al. , 2017 ) or input-output queries from the original black-box models ( Chen et al. , 2017b ; Liu et al. , 2019 ; Cheng et al. , 2019 ) . Adversarial defenses . To improve the robustness of DNNs , a variety of approaches have been proposed to defend against ` p attacks . One line of research focuses on enhancing the robustness of DNNs during training , e.g. , adversarial training ( Madry et al. , 2017 ) , TRADES ( Zhang et al. , 2019 ) , randomized smoothing ( Wong & Kolter , 2017 ) , and their variants ( Salman et al. , 2020 ; Wong et al. , 2020 ; Carmon et al. , 2019 ; Shafahi et al. , 2019 ; Uesato et al. , 2019 ; Chen et al. , 2020 ) . Another line of research is to detect adversarial attacks without altering the victim model or the training process . The key technique is to differentiate between benign and adversarial examples by measuring their ‘ distance. ’ Such a distance measure has been defined in the input space via pixel-level reconstruction error ( Meng & Chen , 2017 ; Liao et al. , 2018 ) , in the intermediate layers via neuron activation anomalies ( Xu et al. , 2019a ) , and in the logit space by tracking the sensitivity of deep feature attributions to input perturbations ( Yang et al. , 2020 ) . In contrast to RED , adversarial detection is a relatively simpler problem as a roughly approximated distance possesses detection-ability ( Meng & Chen , 2017 ; Luo et al. , 2015 ) . Among the existing adversarial defense techniques , the recently-proposed Denoised Smoothing ( DS ) method ( Salman et al. , 2020 ) is more related to ours . In ( Salman et al. , 2020 ) , an image denoising network is prepended to an existing victim model so that the augmented system can be performed as a smoothed image classifier with certified robustness . Although DS is not designed for RED , its denoised output can be regarded as a benign example estimate . The promotion of classification stability in DS also motivates us to design the RED methods with class-discriminative ability . Thus , DS will be a main baseline approach for comparison . 2 REVERSE ENGINEERING OF DECEPTIONS : FORMULATION AND CHALLENGES . In this section , we first introduce the threat model of our interest : adversarial attacks on images . Based on that , we formalize the Reverse Engineering of Deceptions ( RED ) problem and demonstrate its challenges through some ‘ warm-up ’ examples . Preliminaries on threat model . We focus on ` p attacks , where the adversary ’ s goal is to generate imperceptible input perturbations to fool a well-trained image classifier . Formally , let x denote a benign image , and δ an additive perturbation variable . Given a victim classifier f and a perturbation strength tolerance ( in terms of , e.g. , ` ∞-norm constraint ‖δ‖∞ ≤ ) , the desired attack generation algorithmA then seeks the optimal δ subject to the perturbation constraints . Such an attack generation process is denoted by δ = A ( x , f , ) , resulting in an adversarial example x′ = x+ δ . Here A can be fulfilled by different attack methods , e.g. , FGSM ( Goodfellow et al. , 2014 ) , CW ( Carlini & Wagner , 2017 ) , PGD ( Madry et al. , 2017 ) , and AutoAttack ( Croce & Hein , 2020 ) . Problem formulation of RED . Different from conventional defenses to detect or reject adversarial instances ( Pang et al. , 2020 ; Liao et al. , 2018 ; Shafahi et al. , 2020 ; Niu et al. , 2020 ) , RED aims to address the following question . ( RED problem ) Given an adversarial instance , can we reverse-engineer the adversarial perturbations δ , and infer the adversary ’ s objective and knowledge , e.g. , true image class behind deception and adversary saliency image region ? Formally , we aim to recover δ from an adversarial example x′ under the prior knowledge of the victim model f or its substitute f̂ if the former is a black box . We denote the RED operation as δ = R ( x′ , f̂ ) , which covers the white-box scenario ( f̂ = f ) as a special case . We propose to learn a parametric model Dθ ( e.g. , a denoising neural network that we will focus on ) as an approximation ofR through a training dataset of adversary-benignity pairs Ω = { ( x′ , x ) } . Through Dθ , RED will provide a benign example estimate xRED and a adversarial example estimate x′RED as below : xRED = Dθ ( x′ ) , x′RED = x′ − xRED︸ ︷︷ ︸ perturbation estimate +x , ( 1 ) where a perturbation estimate is given by subtracting the RED ’ s output with its input , x′ −Dθ ( x′ ) . We highlight that RED yields a new defensive approach aiming to ‘ diagnose ’ the perturbation details of an existing adversarial example in a post-hoc , forensic manner . This is different from adversarial detection ( AD ) . Fig.1 provides a visual comparison of RED with AD . Although AD is also designed in a post-hoc manner , it aims to determine whether an input is an adversarial example for a victim model based on certain statistics on model features or logits . Besides , AD might be used as a pre-processing step of RED , where the former provides ‘ detected ’ adversarial examples for fine-level RED diagnosis . In our experiments , we will also show that the outputs of RED can be leveraged to guide the design of adversarial detection . In this sense , RED and AD are complementary building blocks within a closed loop . Challenges of RED In this work , we will specify the RED model Dθ as a denoising network . However , it is highly non-trivial to design a proper denoiser for RED . Speaking at a high level , there exist two main challenges . First , unlike the conventional image denoising strategies ( Zhang et al. , 2017 ) , the design of an RED-aware denoiser needs to take into account the effects of victim models and data properties of adversary-benignity pairs . Second , it might be insufficient to merely minimize the reconstruction error as the adversarial perturbation is finely-crafted ( Niu et al. , 2020 ) . Therefore , either under- or over-denoising will lead to poor RED performance .

Summary: This paper proposes a methodology for reverse engineering adversarial perturbations. This allows a defender to recover the original image used to produce an adversarial example and may be an effective tool to mitigating adversarial example attacks.  The paper introduces the concept of reverse engineering adversarial perturbations, defines metrics that quantify reverse engineering performance, creates a framework for training a denoising model to find the adversarial perturbations, and compares the resulting model against adversarial denoising techniques.