Post-Training Detection of Backdoor Attacks for Two-Class and Multi-Attack Scenarios

1 INTRODUCTION . Despite the success of deep neural network ( DNN ) classifiers in many research areas , their vulnerabilities have been recently exposed Xu et al . ( 2020 ) . One emerging threat to DNN classifiers is a backdoor attack ( BA ) Li et al . ( 2020 ) . Here , a classifier will predict to the attacker ’ s target class when a test sample is embedded with the same backdoor pattern ( BP ) that was used to poison the classifier ’ s training set . On the other hand , the classifier ’ s clean test set accuracy ( i.e. , on samples without an embedded BP ) is largely uncompromised Gu et al . ( 2019 ) ; Chen et al . ( 2017 ) ; Liu et al . ( 2018b ) , which makes the attack difficult to detect . Early BA defenses aim to cleanse the possibly poisoned training set of the victim classifier Tran et al . ( 2018 ) ; Chen et al . ( 2018 ) . But deployment of these defenses is not feasible when the defender is the user/consumer of the classifier , without access to its training set or to any prior knowledge of the backdoor pattern Wang et al . ( 2020 ) . A major defense approach suitable for this scenario is a reverse-engineering defense ( RED ) . In general , a RED treats each class as a putative BA target class and trial-reverse-engineers a BP . Then , detection statistics are derived from the estimated pattern for each putative target class . If there is a BA , the pattern estimated for the true BA target class will likely be correlated with the true BP used by the attacker , such that the associated statistics will likely be anomalous referenced against the statistics for the other ( non-BA ) classes Wang et al . ( 2019 ) ; Xiang et al . ( 2020 ) . Notably , RED-based anomaly detection does not require supervision from clean classifiers trained for the same domain like , e.g. , Kolouri et al . ( 2020 ) . Although existing REDs have achieved leading performance in many practical detection tasks , a fundamental limitation still remains . RED-based anomaly detection typically requires estimating a null distribution used to assess anomalies , with the number of detection statistics ( samples ) used to estimate this null a function of the number of classes K. Wang et al . ( 2019 ) uses O ( K ) statistics , and Xiang et al . ( 2020 ) uses even more statistics – O ( K2 ) . These methods are not applicable for domains with K = 2 , e.g . sentiment classification Gao et al . ( 2019b ) , disease diagnosis Li et al . ( 2014 ) , etc. , because there are insufficient statistics for estimating the null . More generally , their accuracy is affected when the number of classes is small ( K > 2 , but small ) . In this paper , we focus on BA detection for the two-class ( and possibly multi-attack ) scenario , under two practical constraints : a ) the defender has no access to the training set of the victim classifier ( or to any BP used by the attacker ) ; b ) supervision from clean classifiers trained for the same domain is not available . This scenario is clearly more challenging than the one considered by existing REDs , for which there is at most one BA target class and a sufficient number of non-target classes to learn an accurate null . As the first to address this difficult problem , our main contributions are as follows : • We propose a detection framework that involves BP reverse-engineering in order to address con- straint a ) above . However , instead of performing anomaly detection as existing REDs do , we process each class independently using a novel detection statistic dubbed expected transferability ( ET ) . This allows our method to be applicable to the two-class , multi-attack scenario ( as well as to the multi-class and ( possibly ) multi-attack scenario ) . • We show that for ET there is a large range of effective choices for the detection threshold , which commonly contains a particular threshold value effective for detecting BAs irrespective of the classification domain or particulars of BA . This common threshold is mathematically derived and is based on properties for general classification tasks that have been verified empirically by many existing works ; thus , constraint b ) that no domain-specific supervision is needed is well-addressed . • Our ET statistic is obtained by BP reverse-engineering and does not strongly depend on the type of BP , or the particular RED objective function and optimization technique used to reverse-engineer putative BPs . Thus , our detection framework can incorporate existing reverse-engineering techniques and potentially their future advances . • We show the effectiveness of our detection framework on six popular benchmark image datasets . 2 RELATED WORK . Backdoor Attack ( BA ) . For a classification task with sample space X and label space C , a BA aims to have the victim classifier f : X → C predict to the attacker ’ s target class t ∈ C whenever a test sample x ∈ X is embedded with the backdoor pattern ( BP ) Gu et al . ( 2019 ) . BAs were initially proposed for images , but have also been proposed for other classification domains . While our experimentation will focus on images , our detector is applicable to other domains as well . Existing BPs for images are mainly of two types : 1 ) an additive perturbation v embedded by x̃ = [ x + v ] c ( 1 ) where ||v||2 is small ( for imperceptibility ) and [ · ] c is a clipping function Chen et al . ( 2017 ) ; Zhong et al . ( 2020 ) ; Xiang et al . ( 2020 ) ; or 2 ) a patch u inserted using an image-wide binary mask m via x̃ = ( 1−m ) x + m u , ( 2 ) where ||m||1 is small for imperceptibility ) and represents element-wise multiplication Gu et al . ( 2019 ) ; Wang et al . ( 2019 ) . Typically1 , BAs are launched by inserting into the training set a small set of samples labeled to the target class and embedded with the same BP that will be used during testing . Such data poisoning can be achieved e.g . when DNN training is outsourced to parties that are possibly malicious Gu et al . ( 2019 ) . BAs are also stealthy since they do not degrade the classifier ’ s accuracy on clean test samples ; hence they are not detectable based on validation set accuracy . Backdoor Defense . Early BA defenses aim to separate backdoor training samples from clean ones by inspecting the training set Tran et al . ( 2018 ) ; Chen et al . ( 2018 ) ; Du et al . ( 2020 ) . However , their deployment is not feasible in many practical scenarios where the defender is a downstream user who has no access to the training process . A family of pruning-based method “ removes ” the backdoor mapping by inspecting neuron activations ( and removing some neurons ) Liu et al . ( 2018a ) ; Li et al . ( 2021 ) . These methods cause non-negligible degradation to classification accuracy , they may remove neurons even when there is no backdoor , and moreover they do not make explicit detection inferences . Kolouri et al . ( 2020 ) and Xu et al . ( 2021 ) train a binary classifier to classify models as “ with ” and “ without ” BAs ; but such training requires clean classifiers from the same domain or a significant number of labeled samples and heavy computation to train these clean classifiers . Chou et al . ( 2018 ) and Gao et al . ( 2019a ) can detect triggering of a backdoor by an observed test sample . Unlike these methods , the existing RED-based detector ( introduced next ) reliably detects backdoors without the need to observe any test samples . Reverse-Engineering Defense ( RED ) . RED is a family of BA defenses that do not need access to the classifier ’ s training set , nor to any prior knowledge of the BP that may be used in an attack . Without knowing whether the classifier has been backdoor-attacked , a RED trial-reverse-engineers a BP for each putative BA target class ( or possibly for each ( source , target ) BA class pair ) using a small , clean dataset independently collected by the defender . Then , for each class , a detection 1A clean-label BA was recently proposed with a slightly different attack strategy for evading human inspection of the training set Turner et al . ( 2019 ) . But this is still detectable by our method ( see Apdx . E ) . statistic is derived from the estimated pattern and used to infer if the class is a BA target class or not . For example , to detect BAs with an additive perturbation BP ( Eq . ( 1 ) ) , Xiang et al . ( 2020 ) finds , for each putative target class , the perturbation with the minimum l2 norm that induces a large misclassification fraction to this class when added to images from another class ( the source class ) . Since BPs embedded by Eq . ( 1 ) usually have a very small l2 norm for imperceptibility , the pattern estimated for the true BA target class ( if a BA exists ) will likely have a much smaller l2 norm than for non-target classes . Thus , unsupervised anomaly detection based on these perturbation sizes is performed – when there is one BA target class and a sufficient number of non-target classes , the statistic for the BA target class will likely be detected as an anomaly compared with the others . When RED detects an attack , the backdoor pattern estimated for the detected target class can then be used to tune the classifier to “ unlearn ” the backdoor mapping Wang et al . ( 2019 ) . Except the RED above proposed by Xiang et al . ( 2020 ) , existing REDs also include Neural Cleanse ( NC ) proposed by Wang et al . ( 2019 ) , which reverse-engineers BPs embedded by Eq . ( 2 ) and uses the l1 norm of the estimated mask as the detection statistic . Guo et al . ( 2019 ) adds constraints to NC ’ s BP reverse-engineering problem by considering various properties of BPs . Liu et al . ( 2019 ) proposed a novel objective function for BP reverse-engineering leveraging the abnormal internal neuron activations caused by the backdoor mapping . Chen et al . ( 2019 ) constructs clean images for detection using model inversion . Dong et al . ( 2021 ) queries the classifier for BP reverse-engineering to address the “ black-box ” scenario . Wang et al . ( 2020 ) proposes a detection statistic based on the similarity between universal and sample-wise BP estimation . Limitation of Existing REDs The anomaly detection of REDs heavily relies on the assumption that there is a relatively large number of non-target classes , thus providing a sufficient number of statistics to inform estimation of a null distribution . But for domains with only two classes , e.g . sentiment classification Gao et al . ( 2019b ) , disease diagnosis Li et al . ( 2014 ) , etc. , this assumption does not hold . For example , Xiang et al . ( 2020 ) and Wang et al . ( 2019 ) exploit O ( K2 ) and O ( K ) statistics in estimating a null respectively , where K is the number of classes . Both of these methods are unsuitable for 2-class problems ( K = 2 ) .

Summary: The paper proposes a class-specific metric called Expected Transferability (ET) that can determine if a classifier is targeted by a backdoor attack (or not). The threat model assumes no access to the model's training data. By the virtue of being class-specific, the metric allows one to examine classifiers regardless of the number of output classes. The author reason theoretical what values of ET indicates the presence (or absence) of backdoor attacks. Then they empirically demonstrate how this value can be estimated and leveraged.