Backdoor Attack with Imperceptible Input and Latent Modification

In this paper , we extend the concept of imperceptible backdoor from the input space to the latent representation , which significantly improves the effectiveness against the existing defense mechanisms , especially those relying on the distinguishability between clean inputs and backdoor inputs in latent space . In the proposed framework , the trigger function will learn to manipulate the input by injecting imperceptible input noise while matching the latent representations of the clean and manipulated inputs via a Wasserstein-based regularization of the corresponding empirical distributions . We formulate such an objective as a non-convex and constrained optimization problem and solve the problem with an efficient stochastic alternating optimization procedure . We name the proposed backdoor attack as Wasserstein Backdoor ( WB ) , which achieves a high attack success rate while being stealthy from both the input and latent spaces , as tested in several benchmark datasets , including MNIST , CIFAR10 , GTSRB , and TinyImagenet . 1 Introduction . In the past years , deep neural network ( DNN ) has successfully transformed many technological fields , such as object classification [ 26 , 20 ] , face recognition [ 31 , 1 ] , autonomous driving [ 53 ] , security applications [ 19 , 3 ] , etc . Meanwhile , due to the underlying black-box nature , its security and privacy implications have also raised serious concerns recently . Efforts in the research community have exposed the vulnerability of DNN classifiers to various attacks [ 50 , 41 , 33 ] . For instance , adversarial examples leverage the difference between the classifier and human to misclassify specific inputs by adding imperceptible perturbations without altering the model [ 17 ] . Such attacks during the inference phase are categorized as evasion attacks [ 27 , 5 ] . On the other hand , poisoning attacks attempt to inject malicious data points or manipulate the training process to either degrade the model accuracy [ 37 , 45 , 60 ] or cause misclassification for specific inputs ( a.k.a . backdoor attacks ) [ 8 , 36 , 34 , 18 ] . In general , backdoor attacks aim at injecting a malicious behavior into a DNN model so that the model would perform normally on clean inputs but yield misclassification in the presence of the 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . backdoor trigger ( e.g. , a specific pattern such as a small square [ 18 ] ) . Later on , many works adopt the concepts and techniques in adversarial examples to improve the stealthiness of the trigger against human observers [ 34 , 2 , 35 ] . Recent works have demonstrated more powerful backdoor attacks that are capable of mounting attacks with visual indistinguishable backdoor images [ 29 , 55 , 59 , 39 , 13 ] . For instance , WaNet [ 39 ] generates backdoor images with warping transformation to minimize input difference while LIRA [ 13 ] generates backdoor images with imperceptible conditional noise addition , resulting in much stealthier triggers . To alleviate the threats originated from the ever-growing powerful backdoor attacks , several categories of countermeasures have been developed . One promising direction for backdoor detection entails identifying backdoor images by characterizing the distinguishable dissimilarity in the feature or latent representation between backdoor images and clean images [ 6 , 54 , 42 , 47 , 52 ] . These methods rely on the assumption that the injected backdoor would leave a noticeable fingerprint in the latent space . For example , activation clustering [ 6 ] and spectral signature [ 54 ] detect malicious samples by inspecting the clusters of the latent space and the spectrum of the covariance of latent representations , respectively . Thus , a stronger adaptive backdoor attack should also ensure its stealthiness from the latent space . In this paper , we present a novel methodology for a backdoor attack that is imperceptible from both the input and latent spaces . We extend the concept of generating imperceptible backdoor triggers to the latent space by minimizing the Wasserstein distance between the latent representations of the clean and backdoor data , which significantly improves the effectiveness against the existing defense mechanisms , especially those aforementioned that rely on the distinguishability in latent space . We name the proposed method Wasserstein Backdoor , or WB . Our technical contributions are summarized below : • We propose a non-convex , constrained optimization problem , which learns to poison the classifier with a backdoor whose trigger is visually imperceptible in the input space and whose poisoned samples have indistinguishable latent distribution to the latent distribution of the clean samples . The latent constraint is formulated via a variant of Wasserstein distance , called sliced-Wasserstein distance [ 24 ] , between the two sets of clean and backdoor data . • We then develop an efficient estimation of the sliced-Wasserstein distance by exploiting the discriminant directions of the trained classifier , instead of randomly sampling from the unit sphere . The proposed distance is a valid distance metric and requires significantly less computation , while yielding a better estimate than the existing calculations of the sliced-Wasserstein distance . • Finally , we demonstrate the superior attack performance of the proposed method and its robustness against several representative defense mechanisms . Specifically , we show that the proposed method outperforms the state-of-the-art attacks in terms of latent indistinguishability , while maintaining similar attack success rates and input indistinguishability . The rest of the paper is organized as follows . We review the background and related work in Section 2 . In Section 3 , we define the threat model . Section 4 presents the details of the proposed methodology . We evaluate the performance and compare to prior works in Section 5 . Finally , Section 6 presents remarks and concludes this paper . We present more details about experimental settings and results as well as supporting proofs in the supplementary material . 2 Background and Related Work . 2.1 Backdoor Attack . The increasing popularity of training outsourcing and machine learning as a service ( MLaaS ) has created potential security risks in the supply chain [ 10 , 58 ] . One important security threat is backdoor attacks against DNNs , which have recently attracted a lot of attention . Backdoor attacks inject a malicious behavior by leveraging the redundancies inside the model such that the model responds to inputs with triggers maliciously ( e.g. , classify as a target class that would normally be considered as a wrong class by manual annotation ) , while preserving the benign behavior for clean inputs without the triggers . Hence , a typical backdoor embedding process is to train the model by minimizing the loss of the clean inputs and the corresponding labels as well as backdoor inputs ( with triggers ) and the target class ( es ) . A trigger is typically applied on a clean image by superimposing at a certain location ( i.e. , patch-based ) [ 18 , 34 ] or adding perturbations [ 44 ] . Various forms of the triggers have been investigated in the literature , including blended [ 8 ] , sinusoidal strips ( SIG ) [ 2 ] , reflection ( ReFool ) [ 35 ] , and warping-based ( WaNet ) [ 39 ] . As we mentioned above , several techniques have been developed recently that can significantly reduce the visibility of the trigger in the input space to enhance the stealthiness of the backdoor attack [ 34 , 2 , 35 ] . In particular , WaNet uses a smooth warping field to generate backdoor images with unnoticeable modifications [ 39 ] , while LIRA [ 13 ] alternates between the processes of trigger generation and backdoor injection to learn visually stealthy triggers . One prior work , Adversarial Embedding [ 51 ] , also attempted to improve the latent indistinguishability of the backdoor attack by using adversarial regularization to minimize the distance between the latent distributions of the backdoor inputs and clean inputs . 2.2 Backdoor Defense . By exploring specific characteristics of the injected backdoor , various countermeasures have been proposed [ 6 , 54 , 16 , 47 , 9 , 7 , 42 ] , although they are often circumvented by subsequent adaptive attacks . For instance , based on the property that a backdoor attack usually targets redundant weights or neurons based on the clean images , model pruning can be used to eliminate the injected backdoor [ 32 ] . In contrast , Neural Cleanse assumes a known subset of clean inputs to reverse-engineer possible trigger patches [ 56 ] . It is also possible to filter the images to nullify the presence of triggers at the test phase to defend against backdoor attacks [ 36 , 30 ] . In this paper , we focus on optimizing the characteristics of backdoor attacks in the latent space . As we discussed above , the rationale behind this is that prior works have demonstrated backdoor images cause distinctive activations in the latent space from those of clean inputs . Hence , this distinguishable dissimilarity between clean images and backdoor images can be utilized for defense in both training [ 6 , 54 ] and test phases [ 49 , 23 , 22 ] . Most of these approaches compute an outlier score to detect abnormal inputs that will be filtered afterward . For example , spectral signature [ 54 ] computes the outlier score based on the singular value decomposition of the covariance matrix of the latent representations , while CleaNN [ 22 ] leverages a concentration inequality to detect anomalous reconstruction errors that are then suppressed before the input entering the victim DNN . This work proposes a method to minimize the difference between clean images and backdoor images in the latent space to improve the attack stealthiness . While doing this , we also optimize the visual imperceptibility in the input space , so that our proposed method can bypass visual inspection . 3 Threat Model . We consider the same threat model as in prior studies [ 44 , 51 , 39 ] , which assumes the backdoor injection is performed at training and the adversary can access to the victim model including both structures and parameters . A successful backdoor attack over an image classification task should produce malicious behavior on images with the trigger , while otherwise working normally on clean images . However , in typical backdoor attacks , the poisoned images are visually inconsistent with natural images , which can be identified easily by human observers . Besides , these attacks usually leave a tangible trace in the latent space of the poisoned classifier ; thus , some defense methods can easily detect and discard the poisoned models . To this end , we propose a stronger backdoor attack where the poisoned images are crafted with imperceptible perturbation in the input space to clean images as well as unnoticeable trace in the latent space . We advance the state-of-the-art by significantly enhancing the imperceptibility and robustness of the backdoor attack . 4 Proposed Methodology : Wasserstein Backdoor ( WB ) . 4.1 Preliminaries . Consider the standard supervised classification task where one seeks to learn a mapping function f✓ : X ! C where X is the input domain and C is the set of target classes . The task is to learn the parameters ✓ by using the training dataset S = { ( xi , yi ) : xi 2 X , yi 2 C , i = 1 , .. , N } . Following the standard training scheme of backdoor attacks , the classifier is trained with the combination of the clean and poisoned subsets of S. To create a poisoned sample , a clean training sample ( x , y ) is transformed into a backdoor sample ( T ( x ) , ⌘ ( y ) ) , where T is a backdoor injection function ( also called the trigger function ) and ⌘ is the target label function . When training f with the clean and poison samples , we alter the behavior of f so that : f ( x ) = y , f ( T ( x ) ) = ⌘ ( y ) , ( 1 ) for any pair of clean data x 2 X and its corresponding label y 2 C. There are two commonly studied backdoor attack settings [ 18 , 39 , 51 ] : all-to-one and all-to-all . In the all-to-one attack , the label is changed to a constant target , i.e . ⌘ ( y ) = c ; while for the all-to-all attack , the true label is one-shifted , i.e . ⌘ ( y ) = ( y + 1 ) mod |C| . In the existing works , the trigger function T is usually selected before training f and fixed during the training process of f .

Summary: This paper proposes a novel backdoor attack that is stealthy in both the input and latent spaces. It argues that the previously proposed backdoor techniques leave tangible footprints in the latent space, thus being easily detected. The authors propose to remove that weakness by adding a regularization term that minimizes the Wasserstein distance between the clean and backdoor embeddings when joint training the trigger function and the classification network. The complex Wasserstein distance is approximated with the sliced-Wasserstein distance (SWD). The authors consider the latent representation at the penultimate layer, thus further simplify SWD computation. The proposed backdoor attack has a high clean and attack accuracy while being stealthy under both input/latent space inspection and backdoor defenses, verified on various benchmark datasets.