Backdoor Attack with Imperceptible Input and Latent Modification

In this paper , we extend the concept of imperceptible backdoor from the input space to the latent representation , which significantly improves the effectiveness against the existing defense mechanisms , especially those relying on the distinguishability between clean inputs and backdoor inputs in latent space . In the proposed framework , the trigger function will learn to manipulate the input by injecting imperceptible input noise while matching the latent representations of the clean and manipulated inputs via a Wasserstein-based regularization of the corresponding empirical distributions . We formulate such an objective as a non-convex and constrained optimization problem and solve the problem with an efficient stochastic alternating optimization procedure . We name the proposed backdoor attack as Wasserstein Backdoor ( WB ) , which achieves a high attack success rate while being stealthy from both the input and latent spaces , as tested in several benchmark datasets , including MNIST , CIFAR10 , GTSRB , and TinyImagenet . 1 Introduction . In the past years , deep neural network ( DNN ) has successfully transformed many technological fields , such as object classification [ 26 , 20 ] , face recognition [ 31 , 1 ] , autonomous driving [ 53 ] , security applications [ 19 , 3 ] , etc . Meanwhile , due to the underlying black-box nature , its security and privacy implications have also raised serious concerns recently . Efforts in the research community have exposed the vulnerability of DNN classifiers to various attacks [ 50 , 41 , 33 ] . For instance , adversarial examples leverage the difference between the classifier and human to misclassify specific inputs by adding imperceptible perturbations without altering the model [ 17 ] . Such attacks during the inference phase are categorized as evasion attacks [ 27 , 5 ] . On the other hand , poisoning attacks attempt to inject malicious data points or manipulate the training process to either degrade the model accuracy [ 37 , 45 , 60 ] or cause misclassification for specific inputs ( a.k.a . backdoor attacks ) [ 8 , 36 , 34 , 18 ] . In general , backdoor attacks aim at injecting a malicious behavior into a DNN model so that the model would perform normally on clean inputs but yield misclassification in the presence of the 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . backdoor trigger ( e.g. , a specific pattern such as a small square [ 18 ] ) . Later on , many works adopt the concepts and techniques in adversarial examples to improve the stealthiness of the trigger against human observers [ 34 , 2 , 35 ] . Recent works have demonstrated more powerful backdoor attacks that are capable of mounting attacks with visual indistinguishable backdoor images [ 29 , 55 , 59 , 39 , 13 ] . For instance , WaNet [ 39 ] generates backdoor images with warping transformation to minimize input difference while LIRA [ 13 ] generates backdoor images with imperceptible conditional noise addition , resulting in much stealthier triggers . To alleviate the threats originated from the ever-growing powerful backdoor attacks , several categories of countermeasures have been developed . One promising direction for backdoor detection entails identifying backdoor images by characterizing the distinguishable dissimilarity in the feature or latent representation between backdoor images and clean images [ 6 , 54 , 42 , 47 , 52 ] . These methods rely on the assumption that the injected backdoor would leave a noticeable fingerprint in the latent space . For example , activation clustering [ 6 ] and spectral signature [ 54 ] detect malicious samples by inspecting the clusters of the latent space and the spectrum of the covariance of latent representations , respectively . Thus , a stronger adaptive backdoor attack should also ensure its stealthiness from the latent space . In this paper , we present a novel methodology for a backdoor attack that is imperceptible from both the input and latent spaces . We extend the concept of generating imperceptible backdoor triggers to the latent space by minimizing the Wasserstein distance between the latent representations of the clean and backdoor data , which significantly improves the effectiveness against the existing defense mechanisms , especially those aforementioned that rely on the distinguishability in latent space . We name the proposed method Wasserstein Backdoor , or WB . Our technical contributions are summarized below : • We propose a non-convex , constrained optimization problem , which learns to poison the classifier with a backdoor whose trigger is visually imperceptible in the input space and whose poisoned samples have indistinguishable latent distribution to the latent distribution of the clean samples . The latent constraint is formulated via a variant of Wasserstein distance , called sliced-Wasserstein distance [ 24 ] , between the two sets of clean and backdoor data . • We then develop an efficient estimation of the sliced-Wasserstein distance by exploiting the discriminant directions of the trained classifier , instead of randomly sampling from the unit sphere . The proposed distance is a valid distance metric and requires significantly less computation , while yielding a better estimate than the existing calculations of the sliced-Wasserstein distance . • Finally , we demonstrate the superior attack performance of the proposed method and its robustness against several representative defense mechanisms . Specifically , we show that the proposed method outperforms the state-of-the-art attacks in terms of latent indistinguishability , while maintaining similar attack success rates and input indistinguishability . The rest of the paper is organized as follows . We review the background and related work in Section 2 . In Section 3 , we define the threat model . Section 4 presents the details of the proposed methodology . We evaluate the performance and compare to prior works in Section 5 . Finally , Section 6 presents remarks and concludes this paper . We present more details about experimental settings and results as well as supporting proofs in the supplementary material . 2 Background and Related Work . 2.1 Backdoor Attack . The increasing popularity of training outsourcing and machine learning as a service ( MLaaS ) has created potential security risks in the supply chain [ 10 , 58 ] . One important security threat is backdoor attacks against DNNs , which have recently attracted a lot of attention . Backdoor attacks inject a malicious behavior by leveraging the redundancies inside the model such that the model responds to inputs with triggers maliciously ( e.g. , classify as a target class that would normally be considered as a wrong class by manual annotation ) , while preserving the benign behavior for clean inputs without the triggers . Hence , a typical backdoor embedding process is to train the model by minimizing the loss of the clean inputs and the corresponding labels as well as backdoor inputs ( with triggers ) and the target class ( es ) . A trigger is typically applied on a clean image by superimposing at a certain location ( i.e. , patch-based ) [ 18 , 34 ] or adding perturbations [ 44 ] . Various forms of the triggers have been investigated in the literature , including blended [ 8 ] , sinusoidal strips ( SIG ) [ 2 ] , reflection ( ReFool ) [ 35 ] , and warping-based ( WaNet ) [ 39 ] . As we mentioned above , several techniques have been developed recently that can significantly reduce the visibility of the trigger in the input space to enhance the stealthiness of the backdoor attack [ 34 , 2 , 35 ] . In particular , WaNet uses a smooth warping field to generate backdoor images with unnoticeable modifications [ 39 ] , while LIRA [ 13 ] alternates between the processes of trigger generation and backdoor injection to learn visually stealthy triggers . One prior work , Adversarial Embedding [ 51 ] , also attempted to improve the latent indistinguishability of the backdoor attack by using adversarial regularization to minimize the distance between the latent distributions of the backdoor inputs and clean inputs . 2.2 Backdoor Defense . By exploring specific characteristics of the injected backdoor , various countermeasures have been proposed [ 6 , 54 , 16 , 47 , 9 , 7 , 42 ] , although they are often circumvented by subsequent adaptive attacks . For instance , based on the property that a backdoor attack usually targets redundant weights or neurons based on the clean images , model pruning can be used to eliminate the injected backdoor [ 32 ] . In contrast , Neural Cleanse assumes a known subset of clean inputs to reverse-engineer possible trigger patches [ 56 ] . It is also possible to filter the images to nullify the presence of triggers at the test phase to defend against backdoor attacks [ 36 , 30 ] . In this paper , we focus on optimizing the characteristics of backdoor attacks in the latent space . As we discussed above , the rationale behind this is that prior works have demonstrated backdoor images cause distinctive activations in the latent space from those of clean inputs . Hence , this distinguishable dissimilarity between clean images and backdoor images can be utilized for defense in both training [ 6 , 54 ] and test phases [ 49 , 23 , 22 ] . Most of these approaches compute an outlier score to detect abnormal inputs that will be filtered afterward . For example , spectral signature [ 54 ] computes the outlier score based on the singular value decomposition of the covariance matrix of the latent representations , while CleaNN [ 22 ] leverages a concentration inequality to detect anomalous reconstruction errors that are then suppressed before the input entering the victim DNN . This work proposes a method to minimize the difference between clean images and backdoor images in the latent space to improve the attack stealthiness . While doing this , we also optimize the visual imperceptibility in the input space , so that our proposed method can bypass visual inspection . 3 Threat Model . We consider the same threat model as in prior studies [ 44 , 51 , 39 ] , which assumes the backdoor injection is performed at training and the adversary can access to the victim model including both structures and parameters . A successful backdoor attack over an image classification task should produce malicious behavior on images with the trigger , while otherwise working normally on clean images . However , in typical backdoor attacks , the poisoned images are visually inconsistent with natural images , which can be identified easily by human observers . Besides , these attacks usually leave a tangible trace in the latent space of the poisoned classifier ; thus , some defense methods can easily detect and discard the poisoned models . To this end , we propose a stronger backdoor attack where the poisoned images are crafted with imperceptible perturbation in the input space to clean images as well as unnoticeable trace in the latent space . We advance the state-of-the-art by significantly enhancing the imperceptibility and robustness of the backdoor attack . 4 Proposed Methodology : Wasserstein Backdoor ( WB ) . 4.1 Preliminaries . Consider the standard supervised classification task where one seeks to learn a mapping function f✓ : X ! C where X is the input domain and C is the set of target classes . The task is to learn the parameters ✓ by using the training dataset S = { ( xi , yi ) : xi 2 X , yi 2 C , i = 1 , .. , N } . Following the standard training scheme of backdoor attacks , the classifier is trained with the combination of the clean and poisoned subsets of S. To create a poisoned sample , a clean training sample ( x , y ) is transformed into a backdoor sample ( T ( x ) , ⌘ ( y ) ) , where T is a backdoor injection function ( also called the trigger function ) and ⌘ is the target label function . When training f with the clean and poison samples , we alter the behavior of f so that : f ( x ) = y , f ( T ( x ) ) = ⌘ ( y ) , ( 1 ) for any pair of clean data x 2 X and its corresponding label y 2 C. There are two commonly studied backdoor attack settings [ 18 , 39 , 51 ] : all-to-one and all-to-all . In the all-to-one attack , the label is changed to a constant target , i.e . ⌘ ( y ) = c ; while for the all-to-all attack , the true label is one-shifted , i.e . ⌘ ( y ) = ( y + 1 ) mod |C| . In the existing works , the trigger function T is usually selected before training f and fixed during the training process of f .

Summary: This paper considers the backdoor attack settings, where the adversary has full control over the training process and tries to encode a trigger function into the classifier such that the classifier fails at test time when the trigger function is activated. They propose a new algorithm that trains a parameterized trigger function at the same as the classifier in a min-max fashion. To ensure that the trigger function is not detected, the paper add a regularization term that ensures that the latent representation of the modified samples and the clean samples have the same distribution, they propose to use a sliced wasserstein distance to do this.