Rethinking CNN’s Generalization to Backdoor Attack from Frequency Domain
Authors: Quanrui Rao
Lin Wang
Wuying Liu
Venue: ICLR 2024 poster (2024)
Keywords: Backdoor attack
Generalization
Frequency domain
CNN
Primary area: societal considerations including fairness, safety, privacy

Abstract: Convolutional  neural network  (CNN) is easily affected by backdoor injections, whose models perform normally on clean samples but produce specific outputs on poisoned ones. Most of the existing studies have focused on the effect of trigger feature changes of poisoned samples on model generalization in spatial domain. We focus on the mechanism of CNN memorize poisoned samples in frequency domain, and find that CNN generate generalization to poisoned samples by memorizing the frequency domain distribution of trigger changes. We also explore the influence of trigger perturbations in different frequency domain components on the generalization of poisoned models from visible and invisible backdoor attacks, and prove that high-frequency components are more susceptible to perturbations than low-frequency components. Based on the above fundings, we propose a universal invisible strategy for visible triggers, which can achieve trigger invisibility while maintaining raw attack performance. We also design a novel frequency domain backdoor attack method based on low-frequency semantic information, which can achieve 100\% attack accuracy on multiple models and multiple datasets, and can bypass multiple defenses.

Published as a conference paper at ICLR 2024
RETHINKING CNN'S GENERALIZATION TO BACK- DOOR ATTACK FROM FREQUENCY DOMAIN
Quanrui Rao!?, Lin Wang's & Wuying Liu!
Shandong Key Laboratory of Language Resources Development and Application, Ludong University, China School of Information and Electrical Engineering, Ludong University, China 3 Xianda College of Economics and Humanities, Shanghai International Studies University, China quanruirao rao0m. ldu.edu. cn, lwang xds i su. edu. cn, wyliuêldu edu. cn
ABSTRACT
Convolutional neural network (CNN) is easily affected by backdoor injections, whose models perform normally on clean samples but produce specific outputs on poisoned ones. Most of the existing studies have focused on the effect of trigger feature changes of poisoned samples on model generalization in spatial domain. We focus on the mechanism of CNN memorize poisoned samples frequency domain, and find that CNN generate generalization to poisoned samples by mem- orizing the frequency domain distribution of trigger changes. We also investigated the assistance provided by perturbations generated at different frequencies to the generalization, and explore the influence of trigger perturbations in different fre- quency domain components on the generalization of poisoned models from vis- components ible and invisible backdoor attacks. We prove that components. Based on are more susceptible to perturbations than owwfrreuenccy the above fundings, we propose a universal invisible strategy for visible triggers, which can achieve trigger invisibility while maintaining raw attack performance. We also design a novel frequency domain backdoor attack method based on low- frequency semantic information, which can achieve 100% attack accuracy mul- tiple models and multiple datasets, and can bypass multiple defenses.
INTRODUCTION
Convolutional Neural Network (CNN) excel multi-domain tasks, but their instability and suscep- tibility make them highly sensitive to small perturbations. As result, the security of CNN has been widely concerned, among which the backdoor attack is an attack method that utilizes the instability of CNN (Tang et al. ). A backdoor attack happens when attacker control data model pa- rameters during the training phase to inject a backdoor into the model. After that, the model behaves normally the original samples but be made generate specific outputs the poisoned sam- ples during the inference phase (Saha et al. (2020); Yao et al. (201);; : Costales et al. (2020). Some scholars have investigated the mechanism of CNN memorize poisoned images (Li et al. (2020);
Datta & Shadbolt )2022), ) but most studies only examined the effect of changing specific trigger Teature the spatial domain the model's generalization. The limitations of the spatial domain
make it difficult to understand how CNNs generate generalization poisoned samples.
Frequency domain analysis provides a new perspective on better understanding the mechanism of backdoor attacks. Several scholars have explored the frequency domain generalization performance of CNN. Among them, the work Wang al. (2020) has inspired us. They believe that humans associate each image with a corresponding label based on low-frequency information, so the low- frequency component the image is strongly correlated with the label. In this scenario, when low- frequency information is artificially correlated with labels, the CNN will preferentially memorize the low-frequency components to minimize training loss, and then gradually memorize the high- frequeney components in order to achieve higher training accuracy. For this idea, Luo et al. (2019) rigorously derived the same conclusion through mathematical formulas.
"Corresponding Author

---

Published as a conference paper at ICLR 2024
Original
BadNet
IAD
WaNet
ISSBA
Spatial
Frequency
domain
Frequeney domain difference
The frequency domain disturbance generated by visible and invisible triggers. The first row shows images the spatial domain, the second row shows images transformed into the frequency domain using DCT, and the third row shows frequency domain disturbances generated by triggers.
Figure I:
In this paper, we explore the mechanism of memorization in CNN for poisoned images from the perspective of frequency domain, using visible and invisible trigger induced perturbations as exam- ples. Due to the labels on poisoned images are specific, their low-frequency and high- aighffreuency components are treated equally during CNN training. To achieve generalization, the model needs to memorize specific frequency-domain distributions from poisoned images, but its small number and complex semantic information make this process difficult. The injection of triggers unifies the frequency-domain distribution of images from distinct categories, prompting CNN to memorize the distribution
to generate generalization. CNN can study specific frequency domain feature distribution and establish the association between feature distribution and labels to achieve backdoor attacks even if the triggers diverse not fixed. We went even further investigate the assistance provided by perturbations generated by visible and invisible triggers at different frequency compo- nents the generalization capability of CNNs. We found that for perturbations tends to be effective, while for others, efficient attacks can be achieved with only a fraction of the perturbation. Additionally, We examined the sensitivity of different frequency com- ponents to trigger perturbations. It is proven that the low-frequency components are more difficult be affected by perturbations due to their strong correlation with the original tags, whereas high- frequency components are more susceptible to perturbations and have higher attack efficiency due their weak correlation with the original tags.
triggers, only a subset of
Visible backdoor attacks have higher success rate, but their usability is limited in scenarios with high requirements for stealth due to the visibility of their triggers. To expand the applicability of visible triggers, we propose universal strategy to conceal visible triggers. This strategy extracts perturbations induced by triggers in middle and high-frequency components, effectively reducing their visibility while maintaining attack effectiveness. In order to efficiently implement backdoor attacks on target categories while minimizing damage to the original image energy, we also propose backdoor attack algorithm based lowffrequeccy semantic information. It generates triggers using partial low-frequency components of the target class samples and injects them into the middle and high-fre freefrrruuecyy components to create poisoned samples. This approach enables 100% attack accuracy while maximizing the visibility of the triggers.
Our contributions. Our key contributions in this paper are as follows: (1) We explore the mecha- nism of CNN memorization for poisoned images from a frequency domain perspective, investigating the generalization of CNN with respect to perturbations in different frequency domains. (2) We ex- plore the generalization of CNN for visible and invisible triggers, demonstrating that figh-frequency features are more susceptible to perturbations than lowffrequency features. (3) We propose gener- alized strategy for rendering visible backdoor attacks invisible while maintaining algorithmic perfor- mance. (4) We propose backdoor attack algorithm based on frequenqnccy semantic information for target classes, achieving high success rates across diverse datascts and models.
RELATED WORK
2.1 BACKDOOR ATTACKS
The concept of backdoor attack was firstly proposed by Gu et al. (0017). who firstly set up the scenario and described the fundamental processes. Specifically, attackers triggers inject back-
2

---

Published as a conference paper at ICLR 2024
doors during the model training phase. The poisoned model behaves similarly the clean model for inputs without triggers. Once a trigger is added to the input, the poisoned model is incorrectly directed to perform subtask specified by the attacker (Quiring & Rieck (2020)).
Backdoor attacks can be categorized as visible backdoor attacks or invisible backdoor attacks based on the visibility of the trigger. Visible backdoor attacks usually use obvious trigger patterns. For ex- ample.[guyyen & Tran (020a) use a conditional trigger generator to generate unique visible triggers based on input images. This mode can achieve high attack accuracy with fewer poisoned samples, but it is easy to be detected in the process of manually examining training samples. As a result, the concept of invisible backdoor attacks is proposed. Invisible backdoor attacks pay more attention to the process of embedding triggers. For instance. Nguyen & Tran (2020b) achieve trigger unseen by distorting images, and Li et al. (202h)) embed string images using pre-trained encoder-decoder networks. Additionally, there exists another type of attacks that make it difficult for observers to notice the presence of triggers by combining them with the real scene. This type of backdoor attack focuses more on physical practicality, combining triggers with natural behavior of target systems or real scenes, rendering the presence of triggers inconspicuous. This strategy of attack is also known as physical backdoor attack (Sarkar al. (2022) ). In this paper, the visible and invisible backdoor attack we study mainly refers to whether the trigger can be seen. We analyze the mechanisms of backdoor attacks according to the distinct impact of frequency-domaii perturbations in visible and invisible backdoor attacks on the model.
2.2 FREQUENCY DOMAIN RESEARCH ON DNN
The uninterpretable nature of deep neural network (DNN) makes it difficult for people to understand how DNN generate generalization. The F- Principple proposed by Luo et al. (2019) proves that neu- ral network generalization comes from paying more attention to low-frequency components during training. There is also a sequence from low frequency to high frequency in the process of fitting the objective function. Inspired by the F-Principle, /Zhan et al. (2019) proposed an efficient Linear F- F-Principle (LFP) dynamics model. The LFP model establishes the relationship between differential equations and neural network by using differential equations to explain and approximate the training behavior of complex neural network, thus providing a new notion to analyze network training.
At present, there is still a lack of exploration of the memory mechanism of backdoor samples from the frequency domain perspective. Most frequency-based backdoor attack methods make relevant assumptions directly from the existing literature on spectrum exploration or observations of the spectrum, and design triggers based on the assumptions. But they don't understand the specific reasons why triggers generalize and whether the added triggers are the most effective. [Wang et al.) (2022a) used existing CNN work on spectrum analysis, using YUV and DCT domain transfer to place triggers at medium and high frequencies. Feng et al.] based on the idea that the amplitude spectrum can capture low-level distribution and the phase spectrum can capture high-level semantic information. Add triggers to the amplitude spectrum to maintain image semantic information. Zeng et al. (202T) proposed a backdoor attack method to remove ultra-high frequencies to avoid detection based on observations of the spectrum of existing triggers. Hammoud & Ghanem (202T) extracted the k most sensitive frequency components from three channels of the image during training and changed them to different values that carry poisonous information for the attack. These works do not illustrate the real reason why CNN generalizes to its triggers, but we have conducted related explorations. And in order to efficiently implement backdoor attacks target categories while minimizing damage to the original image energy, we utilize the proposed mechanism to propose a backdoor attack algorithm based on low-freyuency semantic information of target categories, which shows excellent attack performance.
3 FREQUENCY DOMAIN ANALYSIS
3.1 BACKDOOR SAMPLES IN THE FREQUENCY DOMAIN
Taking visible and invisible backdoor attacks as example, let Dtrain (2,,,,) represent the training dataset and labels, Derain = (ă,, Starget) represent the poisoning training dataset and labels, f(z;; 8) represent the classification model parameterized by 6, then f(ä,; 0) represent the model of injecting the backdoor, then the backdoor attack can be expressed as:
3

---

Published as a conference paper at ICLR 2024
HiegxPrey train
HighsPreeneyyyww val
I Low-Prreparry train
Low-Freppany val
80-
60
20
0.
10
80
130
100
200
10
80
130
160
200
0
10
150
200
spoch
epoch
epoch
epoch
Figure 2: Comparison of models trained using HFC and LFC of images (clean and backdoor images)
f(:;; - f(A:;))
ytarget
Same as Wang et al. (2022a), employ the 2-D discrete cosine transform (DCT) convert the image to the frequency domain. The spectrum after DCT transformation can concentrate the low- frequency components (shortened as LFC) in the upper left and high-frequency components (shortened as HFC) of the image in the lower right area. Let Z; represent the DCT spectrum of æi.
N1-1N
Z: = F (xx) (A1, = Σ
Σ x (n1, 12) C1 (n1, ki) 02 (n2, 1:2)
72 m1 12
12
h 0
Ci (ni, ki) = Es (ki) cos
a12n;+
C (k:)
ky # , 1,2
2N:
Which transforms spatial domain image size N1 x N2 into a frequency domain image of the same size. In the formula, (41, k2) represents the index of the spectrum row and column (from top to bottom and left to right, respectively). At the same time, we use the inverse discrete cosine transform (IDCT) F-1 to transform the image from the frequency domain to the spatial domain.
CNN will memorize both LFC and HFC during training. Typically, label annotations primarily rely on low-frequency semantic information. As result, the model will first fit the LFC and gradually fit the HFC to achieve higher training accuracy. However, since the labels the poisoned samples in the backdoor attacks are specified, the low-frequency semantic vectors are unrelated the labels. Consequently, the LFC and HFC of images will be treated equally during the training phase. When no trigger added or the trigger does work, the model cannot accurately remember small number of poisoned images. When the trigger becomes effective, it will alter the frequency-domait feature distribution of the original images. The model will memorize the frequency-doman feature distribution introduced by the trigger and establish connection between the feature distribution and the label.
Setting. We set up experiments to verify the inference, using BadNet (Gu et al. (2017)) and IAD (Nguyen Tran (2020a)) to represent visible backdoor attacks and WaNet (Nguyen Tran (020b)) and ISSBA (Li et al. (202Tb)) to represent invisible backdoor attacks. To explore the backdoor attack, we use the low-frequency features a and the high-frequency features z of clean images and poisoned images train the model. The training sample generation process is follows:
x = F-1
[4 2 m²]
F-1
2, o ml
(3)
x F-1 12 m
a = F-1
2. o m
Where @ is element-wiss multiplication, m' and m are the masks for LFC and HFC respectively,
ml 1 .k < < No m { 1 k1 > E &
(4)
0 .Otherwise
OOtherwise
Backdoor attack mechanism. Comparatively, triggers in visible backdoor attacks generate larger perturbation intensity in the LFC, while triggers in invisible backdoor attacks generate compara- tively lower perturbation intensity (Figure
the generalization process CNN for different frequency components of poisoned and clean images, while other attack methods
4

---

Published as a conference paper at ICLR 2024
5-1/3
5-1/2
5-2/3
High-Prequenyy of BadNet
Low-Frequmcy of BadNet
HighTFreuency of WaNet
Low-Prequmnyy of Wavet
100-
Acuuray
80-
60-
80-
20
80
120
160
200
10
80
120
160
200
0
10
80
120
160
200
0
10
80
120
160
200
epoch
epoch
epoch
epoch
Figure 3: Generalization analysis of CNN perturbed by triggers in various frequency domains (5 controls trigger perturbation clipping range)
shown in Appendix A.1 When training model with only high-free geguffreyenccy components, there is no association of fow-frequency semantic features with labels. Due to the large number of clean images, CNN can forcibly memorize clean images from HFC, but its whole generalization is limited. Al- though the number poisoned images is limited, CNN can quickly memorize the gegufrreyeency feature distribution of trigger changes and produce great generalization. CNN can memorize the low-frequency semantic information of clean samples and generate generalization by solely uti- lizing LFC to train the model. In the case of poisoned samples, due to the strong low- low-frequency disturbance generated by the trigger in the visible backdoor attack, CNN can also produce better generalization. Nevertheless, it is difficult remember the LFC that changed by the invisible triggers with less low-frequency perturbations.
3.2 FREQUENCY DOMAIN ANALYSIS OF BACKDOOR PATTERNS
Figure shows that different attack method triggers activated over different frequency-domain com- ponents. Backdoor attacks that are visible seem to be activated throughout both LFC and HFC, whereas invisible backdoor attacks seem be activated mainly in the mid to high frequency com- ponents. As a result, we used multiple visible and invisible backdoor attack methods to further explore how triggers influence on the model generalization capability from various backdoor attacks working in different frequency domains. We generated poisoned samples a? and 4 by using the low-frequency and high-frequency features produced by the triggers respectively.
8T7 =
Z, ②
m²) 2. 3 mmmm
27 =/h= F- Z, (1-m)) 2, mm)
(5)
Attack effect. Figure 3 illustrates the influence of the BadNet and WaNet methods in different frequency domain perturbations attack effectiveness, and the attack effects from other back- door methods are shown in Appendix A Notably, the triggers in visible backdoor attacks exhibit concentrated disturbances in the LFC, yet the perturbation in other frequency domain components also shows great attack performance. The generalization ability of CNNs on poisoned samples continue improve with the continuous expansion of the range of intercepted frequency-domain disturbances. The perturbations produced by invisible backdoor attacks at mid and high-frequency can show great generalization, but the model unable accurately memorize the perturbations caused by triggers in the LFC. The addition of mid to high-frequency component improves both the generalization capabilities and fitting speed the model.
It can be seen that although the disturbance of the visible backdoor attack is relatively concentrated in the LFC. the small disturbance generated in the HFC can also show good attack performance. However, the perturbation of invisible backdoor attacks on low-frequency components is unable to complete the generalization of CNN poisoned samples, showing the difference in generalization of CNN to different perturbation intensities of different frequency domain components. Because the low-frequency component has a strong association with the original labels and occupies a dominant position in most images, hypothesize that HFC are more susceptible to disturbance than LFC.
To substantiate our hypothesis, we conducted a series of experiments in which we injected fixed random noise triggers to both LFC and HFC. We used factor called p adjust the trigger intensity, aiming to assess the critical backdoor perturbation intensity for the LFC and HFC of the samples. The samples with disturbed LBC and HFC are denoted as 4PPi and s.Ph respectively.
API = F-1 [Z + p rand(- (1, 1,1)
m²], a.h F-1 [ p rand (11,1) 11@0) mm]
(6)
5

---

Published as a conference paper at ICLR 2024
High-Freeunncy Triger
Low-Frequrccy Trigge
100
80
Acouaay
60-
40-
20
epoch
epoch
Figure 4: Analysis of backdoor disturbance intensity in HFC and LFC
Attack intensity analysis. Figure indicates that achieving high attack success rate in HFC only requires trigger intensity of 0.06 for random noise. In contrast, achieving a backdoor tack requires the addition of random noise with intensity of 0.8 in the LFC, and its accuracy remains lower than that of the HFC. Consequently, high-frequency features are more susceptible to perturbation in the frequency domain, and their attack efficiency is comparatively higher. This experiment also explains why invisible backdoor attacks can achieve backdoor attacks by applying small perturbations in the HFC, whereas invisible backdoor attacks cannot achieve the same through low-frequency perturbations.
FREQUENCY-BASED BACKDOOR ATTACK STRATEGY
Based on Section 3, we already know that high-frequency perturbations of visible backdoor attacks achieve backdoor attacks, and smaller perturbation intensity on HFC can make CNN generalize to poisoned samples. In this section, with the aim of reducing the detectability of visible triggers, we propose strategy to render visible triggers invisible enhance the stealthiness of visible triggers. We propose backdoor attack algorithm based on low-frequency semantic information that takes advantage of the CNN's generalization capabilities within different frequency domains of images. This algorithm involves trigger generation and injection in the frequency domain.
4.1 RENDERING VISIBLE BACKDOOR ATTACKS INVISIBLE
The visible trigger itself is poorly concealed and is easy to detect during manual inspection. Even small perturbations generated by visible triggers in HFC can lead CNN to generalize poisoned sam- ples. However, small perturbations on HFC are less visible after conversion the spatial domain. Thus, propose universal invisibility strategy for backdoor attacks that renders visible triggers invisible while maintaining their original attack performance.
Given clean sample z; and frequency domain feature mask m, visible trigger is used generate the poisoned sample é;. Upon transforming r; and i; to the frequency domain, resulting in Z; and 2, respectively, we generate poisoned samples f:!" with concealed triggers.
afn F-1 Z ② m) +
k N2
Otherwise
Invisible strategy. Our proposed strategy can used for the invisibility of visible triggers. The specific method first uses DCT to transform the poisoned images into the frequency domain, then extracts the disturbance generated by the trigger in the frequency domain, intercepts the mid-high frequency disturbance and adds it to the spectrum. Then it uses IDCT to re-transform images into the space domain to generate the new poisoned samples (Figure aaa). The new generated poisoned samples ensure the effectiveness of the original triggers while also attaining their invisibility.

---

Published as a conference paper at ICLR 2024
Clean Imsage
Clean Imsage
Clean Specirum
Clean Spectrum
Spectrum
Spectrum
Prisonud Image
Spectrum
Imaaee
Poisened Iuagee
ICT
X
Distus bance
Mank
Trigget
Target Class
Refereace Specisum
Rendering Visible
invisible
Frequency Based
The pipeline of our attack. (a) Our strategy for trigger invisibility, which employs medium and high frequency trigger perturbations for hiding. (b) Backdoor attack algorithm based on low-frequency semantic information, injecting target class's low- trwffreqencyy information into clean image's HFC for backdoor injection.
Figure 5:
4.2 BACKDOOR ATTACK BASED ON LOW-FREQUENCY SEMANTIC INFORMATION
The LFC of the original image has strong associations with their labels, and small perturbations in the HFC can achieve backdoor attacks. Therefore, we consider using partial low-frequenc semantic features of the target class images raw materials for generating triggers. Furthermore, triggers are injected in the medium to high frequency component to achieve backdoor attacks to ensure maximum invisibility of triggers.
Given a clean sample Fi, target class sample z; target and frequency domain feature mask m. We use the DCT to transform v; and x; target into the frequency domain to generate Z, and Z, Z!arget respectively. The trigger intensity factor is used to change the trigger intensity, and an upsampling module & used to adjust the trigger size to the required dimensions. Let a Paison represent the poisoned sample.
1 Poison = =F =F pp (
Torggge (k) < E..NN,N < N2)]]
(8)
Attack method. We start by selecting at random a target class image, then transform it into the frequency domain before extracting LFC feature of size (e C.NN, N2). The feature values are then adjusted to range of 0 to 1. A linear (wo-dimensional interpolation method is used to change the size of the low-frequency region into Ni, N2) the frequency domain trigger in order to match the 2 2
size of the trigger dimensions and the perturbation region. The generated frequency domain trigger is applied to the medium to high-frequency components of the clean image's 's frequency domain, situated in the lower-right region of the DCT spectrum. Finally, we use IDCT to transform it to spatial domain to generate the poisoned sample.
EXPERIMENT
5.1 EXPERIMENTAL SETUP
Datasets. We conducted experiments on three widely used datasets: CIFAR-10 (Krizhessky et al. )2009), Celeba (Liu et al. (2015)) and MNIST (LeCun et al. (1998)) CIFAR-10 includes ten general categories with a total of 50,000 images. MNIST is handwritten digit dataset with 60,000 images that includes 10 classifications. Since Celeba is face dataset, its 40 independent binary attribute annotations make unsuitable for multi-class classification. As a result, we chose the top three most balanced features, namely Heavy Makeup, Mouth Slightly Open, and Smiling, using the configuration suggested in Salem et al. (2022). These features were used to generate eight groups, totaling 202.599 photos. We also conducted related experiments Imagenet (Appendix (A.5).
Model. During the training phase, we used the adam optimizer, initially using a learning rate of 0.01 and decreasing it by factor of 10 every 100 training steps. We conducted validation using the

---

Published as a conference paper at ICLR 2024
Table 1: Validating the strategy for rendering visible triggers invisible
Datasel
CIFAR-10
MNIST
Celeba
Original
Invisible
Original
Invisible
Original
Invisible
Attack
Aspect
BA(S)
ASR(%)
BA(S)
ASR(%)
BA()))
ASR(S)
BA(%)
ASR(%)
BA(SS)
ASR(S))
BA(%))
ASR(S)
BadNet
94.54 94,50
94.79
99.78
100.00
100.00
79.26
100,00
IAD
99.33
94.09
99.98
99.54
99.56
99.68
99.93
76.77
99.87
77.22
100.00
Table 2: Backdoor attack performance
Ours
Ours
Ours
Model L
Datasets
Method +
BadNet
IAD
WaNet
ISSBA
(p
0.3.5 f)
0. {)
0.5,8
()
BA(%)
94.54
94.50
94.15
94.38
96.60
94.47
94.78
CIFAR-IK
ASR(%)
99.99
99.33
99.55
99.99
100.00
100.00
100,00
ResNet18
MNIST
BA(S)
99.78
99.54
99.52
99.35
99.78
99.77
99.75
ASR(%)
100.00
99.56
99.86
100.00
100.00
100.00
100.00
BA(S))
79.26
76.77
79.42
76.06
79.05
79.71
79.25
Celeba
ASR(S)
100.00
99.87
99.91
99.98
100.00
100.00
100.00
BA(S))
93.47
92.56
93.24
93.47
93.36
93.63
93.48
CIFAR-I0
ASR(S)
100.00
98.71
98.90
99.98
100.00
100.00
100.00
BA(%)
99.74
99.66
99.72
99.32
99.72
99.78
99.73
V
MNIST
ASR(S)
100,00
100.00
100.00
100.00
BA(S)
79.71
79.25
Celeba
ASR(%)
100.00
99.78
99.71
99.98
100.00
100.00
100.00
BA(%)
93.60
93.68
94.13
CIFAR-10
ASR(S)
99.99
99.25
99.70
99.98
99.98
100.00
100.00
MobileNetV2
MNIST
BA(S))
99.77
99.77
99.70
99.53
99.77
99.75
99.72
ASR(%)
100,00
99.91
100,00
100.00
100.00
100,00
100.00
BA(56)
80.01
80.08
80.03
78.24
79.74
79.87
80.20
Celeba
ASR(%)
100.00
99.72
99.96
99.75
100.00
100.00
100.00
ResNet18 (He et al. (016)), VGG16 (Simonyan & Zisserman (2044), and MobileNetV2 (Snddeer al. 2018)) models. We also conducted related experiments on vision transform (Appendix)A.a) (A.).
5.2 INVISIBLE ATTACK EXPERIMENT
We used variety of visible trigger attack algorithms to evaluate the effectiveness of the backdoor hiding effect on the ResNet18 model. Appendix A.3 contains the results of the backdoor hiding effect. The attack performance after rendering visible triggers invisible shown Table I We used Attack Success Rate (ASR) and Benign Accuracy (BA) to assess the efficacy of various attacks. After achieving the invisibility of visible triggers, BadNet maintained ASR 99.94% on the CIFAR-10, 10, MNIST, and Celeba datasets respectively. Only the CIFAR-10 and Celeba datasets had a small decline ASR of 1. 0.7 and 0.06% respectively. On the other hand, their BA increased by 0.25% and 0.14% respectively. After the visible trigger is concealed, LAD exhibits an enhanced ASR across all datasets. This indicates the success of our backdoor invisible strategy.
82%, 100%, and
5.3 FREQUENCY DOMAIN- -BSEE ATTACK EXPERIMENTS
We evaluated the impact of different p and s on BA and ASR, as well as the degree of the trigger invisibility. Appendix A.4 contains the results, which show that p is a relatively important factor in- fluencing both BA and ASR. We selected three p and combinations, which demonstrated promising overall performance, and conducted comparison analysis against various backdoor attack algorithms across multiple datasets and models. The results show that our algorithm achieves an ASR of nearly 100% various datasets and models, with average BA improvement of 0.2%. Notably, the CIFAR-10 dataset, the BA of the ResNet18 has increased by 2.06%.
5.4 DEFENSE EXPERIMENTS
We used defense methods and network visualization tools to verify the effectiveness of the proposed backdoor attack model. We also conducted experiments other defense methods. (Appeeddi A.7)
8

---

Published as a conference paper at ICLR 2024
Clean
Poisoned
Clean
Bacldlor
diar10
coloits
100
300
200
300
400
500
10
20
30
40
30
60
100
201
30
40
50
spoch
opoch
spoch
(a) GradCam
(b) Fine- Sine-Prunning
Clea imags
Backdlorr image
cifar10
mnist
celeba
018
018
Brobabilins
0.15
028-
0.31
048
612-
0U11
270
600
Fittregy
Extrayy
Fntrogy
(c) STRIP
Figure 6: Backdoor defense experiment of our attack method
Fine-Pruning (Liu et al. (2018)) model defense method based on neuron analysis. The method analyzes neuronal responses to set of clean images and detects and gradually prunes dormant neurons to mitigate the impact of backdoors. Figure (igure/6(b) shows the pruning effect of Fine-Pruning applied to our model. The accuracy of clean data does not exceed that of the attacked data across all datasets, making backdoor mitigation nearly impossible.
STRIP (Gao al. (2199) is backdoor defense method based input filtering. In this ap- proach, known input images are perturbed by introducing clean images of different categories, and the change in predictive entropy between the mixed and original images is compared. Figure [6c) demonstrates that the entropy distribution of our poisoned model closely resembles that of the clean model, indicating that the poisoned model trained using poisoned samples exhibits behavior similar to a clean model.
GradCam (Selvaraju al. (2017) is network visualization tool that helps in examining network behavior. Since our attack method is injected from high-frequency components, the perturbation is concentrated in the image details and therefore cannot be detected by it. Vigure/oran shows that the visual heatmaps of the clean model and the poisoned model are basically consistent.
CONCLUSION
In this paper, we analyzed the impact triggers on the generalization of CNN from the perspective of frequency domain, and proved that triggers change the frequency domain feature distribution of images to make CNN generalize. We investigated how changes in trigger-induced alterations across different frequency components contribute generalization. We also proved that high-frequency components are more easily perturbed than lower-frequent frequency components and have higher attack efficiency. Based on these findings, we proposed simple yet effective universal strategy achieve invisibility of visible triggers, ensuring minimum losses in both BA and ASR. Furthermore, we proposed a backdoor attack algorithm based on low-frequency semantic information that achieved significant results and was able circumvent many defenses.
Our interpretation of CNN memory poisoning samples helps solve other problems. For example, by maintaining the same frequency domain feature distribution, we consider setting invisible triggers in the training phase, and setting physically realizable triggers in the inference phase. Our discussion of backdoor defense mechanisms is still bit inadequate, and we will improve this part in the next work. We hope this research will inspire more in-depth discussions about the characteristics of backdoors, thereby promoting the further development of backdoor attacks and defenses.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
The research is supported by the New Liberal Arts Research and Reform Practice Project of Min- istry of Education of China (No. 2021060049), the Postgraduate Education and Teaching Reform Research Project of Shandong (No. SDYJG21185) the Key Project of Undergraduate Teaching Re- form Research of Shandong (No. Z2021323), the Humanity and Social Science Research Project of Ministry of Education of China (No. 2AYJ6906,, No. 20YJC740062), and the Social Science Foundation of Shanghai (No. 2019BYY028).
REFERENCES
Eugene Bagdasaryan and Vitaly Shmatikov. Blind backdoors in deep learning models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 1505-1521, 2021.
Mauro Barni, Kassem Kallas, and Benedetta Tondi. A new backdoor attack in cnns by training set corruption without label poisoning. In 2019 IEEE International Conference on Image Processing (ICIP), pp. 101-105. IEFE, 2019.
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep 2017.
learning systems using data poisoning. arXiv preprint
Robby Costales, Chengzhi Mao, Raphael Norwitz, Bryan Kim, and Junfeng Yang. Live trojan attacks on deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 796-797, 2020.
Siddhartha Datta and Nigel Shadbolt. Backdoors stuck at the frontdoor: Multi-agent backdoor attacks that backfire. arXiv preprint arXiv:2201.122111 2022. 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.
Ieee, 2009.
Khoa Doan, Yingjie Lao, Weijie Zhao, and Ping Li. Lira: Learnable, imperceptible and robust backdoor attacks. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11966-199766 2021.
Yu Feng, Benteng Ma, Jing Zhang, Shanshan Zhao, Yong Xia, and Dacheng Tao. Fiba: Frequency- injection based backdoor attack in medical image analysis supplementary material.
Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal. Strip: A defence against trojan attacks on deep neural networks. In Proceedings of the 35th Annual Computer Security Applications Conference, pp. 113 113-125, 2019.
Tianyu Gu, Brendan DavantGavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint
2017.
Hasan Abed Al Kader Hammoud and Bernard Ghanem. Check your other door! creating backdoor attacks in the frequency domain. arXiv preprint ar Xiv:2100000500.7 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Kunzhe Huang, Yiming Li, Baoyuan Wu, Zhan Qin, and Kui Ren. Backdoor defense via decoupling the training process. In International Conference on Learning Representations, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 23/N, 7(7):3, 2015.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEFF, 86(1) (6(11))2278-22) 1998.
10

---

Published as a conference paper at ICLR 2024
Yiming Li, Tongqing Zhai, Baoyuan Wu, Yong Jiang. Zhifeng Li, and Shutao Xia. Rethinking the trigger of backdoor attack. arXiv preprint arXiv:2004.0440 04692, 2020.
Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, and Siwei Lyu. Invisible backdoor attack with sample-specific triggers. In Proceedings of the IEEE/CVF international conference computer vision, pp. 16463-16472, 2021a.
Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, and Siwei Lyu. Invisible backdoor attack with sample-specific triggers. In Proceedings of the IEEF/CVF international conference on computer vision, pp. 16463-16472, 2021b.
Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg- Fine-pruning: Defending against back- dooring attacks on deep neural networks. In International symposium on research in attacks, intrusions, and defenses, pp. 273-294. Springer, 2018a
Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. Trojaning attack neural networks. In 25th Annual Network And Distributed System Security Symposium (NDSS 2018). Internet Soc, 2018b.
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Tao Luo, Zheng Ma, Zhi-Qin John Xu, and Yaoyu Zhang. Theory of the frequency principle for general deep neural networks. arXiv preprint 1906009906 2019.
Tuan Anh Nguyen and Anh Tran. Input-aware dynamic backdoor attack. Advances in Neural Information Processing Systems, 33:3454-3464, 2020a.
Tuan Anh Nguyen and Anh Tuan Tran. Wanet-impercepptbbee warping-based backdoor attack. In International Conference on Learning Representations, 2020b.
Erwin Quiring and Konrad Rieck. Backdooring and poisoning neural networks with image-scaling attacks. In 2020 IEEE Security and Privacy Workshops (SPW), pp. 41-47. IEEE, 2020.
Aniruddha Saha, vashayvarun Subramanya, and Hamed Pirsiavash. Hidden trigger backdoor tacks. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 11957- 11965, 2020.
Ahmed Salem, Rui Wen, Michael Backes, Shiging Ma, and Yang Zhang, Dynamic backdoor at- tacks against machine learning models. In 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), pp. 703-718. IEEE, 2022.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chioe Chen. Mo- bilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510-4520, 2018
Esha Sarkar, Hadjer Benkraouda, Gopika Krishnan, Homer Gamil, and Michail Maniatakos. Face- hack: Attacking facial recognition systems using malicious facial characteristics. IEEE Transac- tions Biometrics, Behavior, and identity Science, 4(3):361-372, 2022. doi: 10.1109/TBIOM. 2021.3132132.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference computer vision, pp. 618-626, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 1409:140..1555 2014.
Di Tang, XiaoFeng Wang, Haixu Tang, and Kehuan Zhang. Demon in the variant: Statistical analysis of DNNs for robust backdoor contamination detection. In 30th USENIX Security Symposium (USENIX Security 21), pp. 1541-1558. 2021. ISBN 978-1-929113-24-3.
11

---

Published as a conference paper at ICLR 2024
Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium Security and Privacy (SP). pp. 707-723, IEEE, 2019.
Haohan Wang, Xindi Wu, Zeyi Huang, and Eric Xing. High-frequency component helps explain the generalization of convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8684-8694 2020.
Tong Wang, Yuan Yao, Feng Xu, Shengwei An, Hanghang Tong, and Ting Wang. An invisible black-box backdoor attack through frequency domain. In European Conference Computer Vision, pp. 396-413. Springer, 2022a.
Zhenting Wang. Juan Zhai, and Shiqing Ma. Bppattack: Stealthy and efficient trojan attacks against deep neural networks via image quantization and contrastive adversarial learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15074-15084, 2022b.
Baoyuan Wu, Hongrui Chen, Mingda Zhang, Zihao Zhu, Shaokui Wei, Danni Yuan, and Chao Shen. Backdoorbench: A comprehensive benchmark of backdoor learning. Advances in Neural Information Processing Systems, 35: 10546-00559, 2022.
Yuanshun Yao, Huiying Li, Haitao Zheng, and Ben Y Zhao. Latent backdoor attacks on deep neural networks. In Proceedings of the 2019 ACM SIGSAC conference computer and communications security, pp. 2041-2055, 2019.
Dong Yin, Raphael Gontijo Lopes, Jon Shlens, Ekin Dogus Cubuk, and Justin Gilmer. A fourier perspective model robustness in computer vision. Advances in Neural Information Processing Systems, 32, 2019.
Yi Zeng, Won Park, Z. Morley Mao, and Ruoxi Jia. Rethinking the backdoor attacks' triggers: A frequency perspective. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 16473-16481, 2021.
Yaoyu Zhang, Zhi-Qin John Xu, Tao Luo, and Zheng Ma. Explicitizing implicit bias of the frequency principle in two-layer neural networks. arXiv preprint arXiv: /905. 10264, 2019.
Runkai Zheng, Rongjun Tang, Jianze Li, and Li Liu. Data-free backdoor removal based on channel lipschitzness. In European Conference on Computer Vision, pp. 175-191. Springer, 2022.
12

---

Published as a conference paper at ICLR 2024
A APPENDIX
We evaluate multiple backdoor attack algorithms, including visible backdoor attacks and invisible backdoor attacks. Experiments on these algorithms can prove our conclusions. Due space reasons, we list representative results in the main text.
A.1
ANALYSIS OF IMAGE'S HIGH AND LOW-FREQURNCY COMPONENTS
Claan
Badldor
BerNeet
IAD
WaNet
ISSBA
100-
Acturty
60-
High-Prequayy
40-
Train
0-
epoch
epoch
epoch
epoch
100-
Accuray
60)
High-
Eval
20
www.
spoch
epoch
spoch
epoch
100-
Acouray
68)
Low-Frequncy Train
200
spoch
epoch
epoch
epoch
100)
Acouary
60-
Low-Frequncy
Eval
20
spoch
epoch
spoch
epoch
Figure 7: CNN generalization analysis of different frequency domain images (including clean images and backdoor images). Each column the figure corresponds to a backdoor attack method, total of columns, and each corresponds to the model generalization process of an attack method on high or low frequency, training set or validation set.
We set up experimental verification inferences in this paper, using BadNet and IAD as visible back- door attacks and WaNet and ISSBA invisible backdoor attacks for exploration. For training the model, we extract low-frequency features æf and high-frequency features z! from both clean and poisoned images. The generation of training samples is shown equations and Among these, set & to
In Figure 7 we present a visual representation the influence high and low-frequency informa- tion extracted from both clean and poisoned images on the model's generalization. This confirms that the generalization ability of CNN on poisoned samples is achieved through memorizing the frequency domain features after injected by triggers, prompting the model to exhibit generalization tendencies the presence a limited number of poisoned images, thus enabling the successful execution of backdoor attacks.
13

---

Published as a conference paper at ICLR 2024
A.2
ANALYSIS OF HIGH AND NRN-FREQUENCY PERTURBATIONS IN TRIGGERS
5-1/3
5-1/2
2-23
BadNet
IAD
WaNet
ISSBA
Accuracy
80)-
High-Prequency
80-
Train
20
0
80
00)
1.00
160
200
40
00)
1.0
100
200
0
40
00
1.0
160
200
0
40
000
120
160
200
epoch
epoch
epoch
epoch
80-
Amrrgy
High-Prequancy
40-
Eval
200
201
120
100
200
40
120
100
200
80
120
303)
200
40)
120
100
200
epoch
epoch
epoch
epoch
Accuraty
80-
Low-Prequnccy Train
40-
200
1.0
200
80
160
200
40
120
200
spach
spach
spoch
spach
80.
Acouraty
Low-Frequncc Eval
40-
0-
200
spoch 200
80
160
200
-0
200
spoch
epoch
epoch
Figure 8: CNN generalization analysis of different trigger perturbations in different frequency domains (E control trigger perturbation limiting range). Each column in the figure corresponds to backdoor attack method, a total of four columns, and each row corresponds to a model generalization process of attack method on high low frequency, training verification sets.
We extended our investigation with BadNet and IAD as visible backdoor attacks and WaNet and ISSBA as invisible backdoor attacks explore the effects perturbations in different frequency domains model generalization. To accomplish this, generated poisoned samples 2? and o using only low-frequency and high-frequency features produced by triggers, respectively.
Figure illustrrtee the impact of perturbations in various frequency domains model generaliza- tion performance across different attack methods. The parameter& & controls the range low and high-frequency component truncation in this case, thus enabling the extraction of distinct frequency domain features. The generation training samples shown in equations and
A.3 RENDERING VISIBLE TRIGGERS INVISIBLE
In this section, we evaluated the invisibility effectiveness of our invisible algorithm for visible trig- gers. To achieve trigger invisibility, we extracted effective perturbations from the original trigger in the mid-to-high frequency range while suppressing low-frequency disturbances. Figure shows the invisibility effect. The visible trigger in the spatial domain becomes invisible to humans, show- ing excellent invisibility effect. Furthermore, even when invisibly hidden, the algorithm retains its benign accuracy (BA) and attack success rate (ASR), indicating the efficacy of our invisibility technique.
14

---

Published as a conference paper at ICLR 2024
Original
BadNef
IAD
Invisible BadNet
Invisible IAD
Figure 9: Render the visible trigger invisible. The first row is the image of the original sample and the trigger with visible backdoor attack added, and the second row is the image processed by our strategy.
A.4
FREQUENCY DOMAIN TRIGGER ANALYSIS
Original Images
Poisoned Images
= 0.05
p 0.10
p 0.20
= 0.30
p 0.40
0.50
Figure 10: The poisoned image is generated by our backdoor attack algorithm based on low- frequency semantic information. The intensity factor governs the range of values for the frequency domain trigger, set to for
We evaluated the impact of varying P and on BA, ASR, and trigger invisibility on the CIFAR- 10 dataset. Table 3 shows the results of the backdoor attacks, and Figure I0 shows the triggers' invisibility. Given that has only a small impact on trigger visibility, we merely showed trigger visibility for different values. We can observe that when the trigger strength is excessive, partial artifacts may appear the upper-left corner the spatial domain due the nature of the DCT transformation. At p 0.40, careful scrutiny is required at p 0.50, the changes generated by the trigger in the upper-left corner of the image become noticeable to the human. However, high-intensity triggers exhibit significant attack effectiveness and can bypass multiple defenses, while even lower-intensity triggers continue to demonstrate strong attack efficacy and performance. Therefore, attack method remains successful.
detect the image perturbation, while
At the same time, we also selected three combinations of p and E with good comprehensive effects to verify the influence of different poisoning rates attack methods (Table 4). The ResNetl 88 model
15

---

Published as a conference paper at ICLR 2024
Table 3: Comparison of intensity factor p and low-frequency semantically triggered region selection factor
factor
factor 5 +
&
8
8 =
4
3
2
BA
ASR
BA
ASR
BA
ASR
BA
ASR
p=0.10 10
82.50
98.21
82.09
99.14
82.42
99.36
81.15
99.15
p=0.20
94.52
99.90
94.44
99.97
94.64
99.97
94.46
99.97
/=0.30
94.48
99.99
96.60
100.00
94.61
99.95
94.85
100.00
p=0.40
94.69
99.97
94.46
100.00
94.47
100.00
94.70
100.00
p=0.50
94.58
100.00
94.67
100.00
94.78
100.00
94.67
100.00
and CIFAR-10 dataset were used in the experiment. Our attack method can basically achieve 100% attack success rate when the poisoning rate is from 0.02 to 0.10, and the variation range of the benign accuracy rate is around 1%. This shows that our method can still achieve high performance under the condition of low poisoning rate.
Table 4: Performance of our attack method at different poisoning rates
p = 0.3,5 = 1
p == 0.4,5
p == 0.5,5
Rates
Model +
BA
ASR
BA
ASR
BA
ASR
0.02
94.99
99.96
93.83
100.00
94.44
100.00
0.04
94.78
99.99
94.27
100.00
94.52
100.00
0.06
95.69
100.00
94.25
100.00
94.50
100.00
0.08
95.58
100.00
94.41
100.00
94.56
100.00
0.10
96.60
100.00
94.47
100.00
94.78
100.00
A.5 VALIDATION OF INVISIBLE STRATEGIES AND ATTACK ALGORITIMMS ON IMAGENET
One of the benchmark datasets for image classification, ImageNet(Deng et al. (2009), consists of 1000 classes. Due to high computational and memory costs, the entire dataset is rarely utilized in existing backdoor learning efforts Wu et al. (2022)). We used Tiny-ImageNet(Le & Yang (015 )
which is currently widely used in the field backdoor attacks. The dataset contains a subset of 200 classes, with 100,000 images for training (500 images per class) and 10,000 images for testing (50 images per class). We used the pre-resnet model to conduct relevant experiments. The poisoning rate is set 10%.
As can be seen from Table our backdoor attack algorithm still efficient on the Tiny-ImageNet dataset. Some other algorithms have low BA or low ASR this data set, but our attack algorithm still maintains an attack rate of 100.00%, and is also better at classifying cleandata. Table 6 shows the attack effect of our invisible algorithm on BadNet. This shows that algorithm still works the ImageNet dataset, ensuring the original performance of the algorithm.
A.6 VALIDATION OF INVISIBLE STRATEGIES AND ATTACK ALGORITTMS ON VISION TRANSFORMER
We evaluate our algorithm vit-base model is 14 x 14, and the poisoning rate is set to 10%. It can be seen from the experimental results (Table 7 that our attack method is still applicable in vision transformer. And compared with
vit-base model using the cifar dataset. The input patch size of the
16

---

effective against our Published as a conference paper at ICLR 2024
Table 5: Backdoor attacks ImageNet
Method
Aspect
BA
ASR
LIRA Doan et al.
2021
0.05
99.85
BPP
Wang al. (2022h)
58.14
100.00
Blind Bagdasaryan & Shmatikov 2022
1.97
45.21
TrojanNN
Liu et al. (2018b)
55.89
99.98
Ours (p 0.4.c
59.11
100.00
Table 6: The implementation of invisible strategies on ImageNet
Origin
Invisible BA
Method +
Aspect +
BA
ASR
ASR
BadNet
65.31
97.17
61.18
99.16
IAD
57.58
99.67
57.99
99.98
other backdoor attack methods, our attack algorithm achieves high BA and ASR. Table shows the effectiveness of our stealth strategy on badnet. After using our stealth strategy. we not only achieve the stealth of visible triggers, but also ensure the original attack performance, and even achieve certain amount of improvement in ASR.
Table 7: Backdoor attack vision transformer
Method
Aspect
BA
ASR
BadNet Gu et al.
(2017)
94.78
93.77
Blended Chen al. (2017)
96.54
99.68
LIRA Bagdasaryan Shmatikov (021)
17.52
87.87
TrojanNN Liu et al.
2018b
96.46
99.98
Ours(p 0.4,5
98.62
100.00
A.7 EXPERIMENTATION WITH OTHER BACKDOOR DEFENSE METHODS
Different from the method used in section 5.4, we other defense methods defend attack algorithm to demonstrate the effectiveness of our algorithm.
(HDnH et (2021)) decouples the original end-to-end training process into three stages, and using strong data augmentation involved in self-supervised learning destroys the triggering pattern, making it unlearnable in representation learning. The decoupling process further disconnects trigger modes and target tags. Implemented that even if the model is trained on toxic dataset, the hidden backdoor cannot be successfully created. Table shows that this defense attack method, and our attack algorithm can still achieve high ASR.
NC IC(Wang et al. 2019)) were the first to propose robust and general detection and mitigation system for DNN backdoor attacks. Their technology can identify multiple existing backdoors and reconstruct possible trigger points. Use the model to reversely infer the position and shape of the trigger, and then determine whether there is backdoor; secondly, use the similar impact of the verse trigger and the actual trigger on neuron activation to mitigate backdoor attacks. Many existing backdoor attacks will lose the performance of backdoor attacks after being mitigated by NC, but the mitigating effect of NC on our method not very great. As can be seen from Table g the ASR of backdoor attack algorithm still reach 91.80%
17

---

Published as a conference paper at ICLR 2024
Table 8: The implementation of invisible strategies vision transformer
Origin
Invisible
Method &
Aspect +
BA
ASR
BA
ASR
BadNet
94.78
93.77
94.77
99.12
IAD
98.37
99.26
98.25
99.23
CLP(Zheng et al. (0222) introduced a new concept, namely the Lipschitz constant for each chan- nel's mapping from the input image the output. Empirical evidence was then used to show that there is strong correlation between the upper bound of CLC (UCLC) and changes in channel ac- tivation that trigger activation. Since UCLC can be calculated directly from the weight matrix, they can detect potential backdoor channels in data-free manner and perform simple pruning of infected DNNs to repair the model. Table P shows the defense results of CLC against our method. The sults show that CLC is ineffective. Although the ASR of our attack algorithm has slightly decreased under his defense, is still effective.
Table 9: Mitigation results of backdoor defense methods on algorithm
DBD
NC
CLP
Attack
Defense +
BA(%)
ASR(%)
BA(%)
ASR(%)
BA(%)
ASR(%)
SIG Barni et al. (2019)
60.67
100.00
84.48
98.27
83.80
98.91
Blended Chen al. (2017
69.91
99.68
93.47
99.92
91.32
99.74
WaNer N guyen & Tran (020b)
80.90
6.61
91.80
7.53
81.91
78.42
ISSBA et 2021a
63.50
99.51
90.99
0.58
91.38
68.13
Ours(p 0.4,s
70.29
100.00
94.26
91.80
94.55
98.41
A.8 DISCUSSION WITH OTHER RELATED WORK
Discussion with Zeng et al. 2021 Zeng et al. (2021) studied the spectrum of poisoned images and showed that most triggers will produce artifacts in the high-frequency components of large-sized put images, and explained the reasons for this phenomenon accordingly. That is, the high-frequeng artifacts of Local Patching mainly originate from the high-frequenn components of the trigger mode itself. High-frequency artifacts of Large-Size itself or the method of inserting the trigger. GAN-Generated Backdoor Data originates from upsampling operation adopted by gan. We explained the mechanism of CNN memory triggers from the perspective of frequency domain, indicating that CNN can memorize specific frequency domain feature distribution and establish the association between feature distribution and labels. Therefore, backdoor attacks can be achieved even if the trigger factors are diverse or not fixed. We used this mechanism explore the impact of perturbations produced by visible and invisible triggers dif- ferent frequency domain components on the generalization ability of CNN, as well as the differences in the generalization ability of CNN different perturbation intensities imposed on different fre- quency domain components. It is also proven that high- freeuenccccy components are more sensitive to perturbations relative to fow-freueeccy components.
Global Patching originate from the trigger pattern
Discussion with Feng et al. Feng et al. uses FFT to transform the image from the spatial domain the frequency domain, and then linearly adds the low-frequency component the clean image and the low-frequency component of the target image through a certain ratio as the low-frequency component of the poisoned image, thereby achieving frequency injection of the trigger. Literature [3] uses FFT to transform the image from the spatial domain to the frequency domain, and then linearly adds the low-fte freeuueuency component of the clean image and the target image through a certain ratio the low-fit www-freuency component of the poisoned image, thereby achieving frequency injection of the trigger. And their training process introduced noise images similar to those in the literature [4] to enhance the uniqueness of the trigger. However, frequency- based backdoor attack method
18

---

Published as a conference paper at ICLR 2024
selects part of the low-frequencr components of the target category image the material of the trigger. The poisoned image is then generated by adding triggers to the mid- and high- seeu-fruyency components of the clean image through operations such as upsampling. And in order to improve the generalization of CNN to poisoned images whose label is the target category, we randomly select the low-frequency components of the image from the target category raw materials when generating each poisoned image, and do not use noise images.
Discussion with Yin al. (2019). The work Yin al. (2019) mainly analyzes the trade-offs be- tween two data enhancement strategies, Gaussian enhancement and adversarial training, for different corruption types from the perspective of the frequency domain. They found that the augmentation strategy made the model tend to exploit low- low-frequency information in the ipput. This low- low-frequency bias reduces susceptibility to high-frequency damage and improves robustness to high-frequency damage while degrading performance against low-frequency damage. They also demonstrated that AutoAugment, a data augmentation, can play a key role in mitigating robustness issues. In our work, mainly explored the mechanism by which CNN generates generalization for backdoor samples, and analyzed the role of existing triggers in frequency domain perturbations. The vulnerability of high versus low frequency components is also explored. So our theory is different.
Discussion with Wang et al. (2020). Our work is inspired by Wang et al. (2020) and attempts to explore the generalization of CNN to backdoor samples from the frequency domain perspective.
The contribution of Wang et al. (020) is that they first noticed the ability of CNN in capturing high- frequency components of images. And from data perspective, it is revealed that CNN achieves trade-off between accuracy and robustness by utilizing the high-frequency components of images. They proposed that in generally labeled samples, the model first extracts low-frequency information and then gradually extracts high-frequency information to achieve higher training accuracy. In sam- ples with chaotic labels, the model treats now-frequency and high-frequency information equally, which means that the model begins to choose to memorize the data. This part of the theory is also a hypothesis for our work. They also further explored the impact of training techniques on training results from a frequency perspective. For example, they found that larger batch size has smaller changes in high-frret geehffreuency information, narrowing the generalization gap, so it will lead to smaller gap between training accuracy and test accuracy. For adversarial learning, they found that adversar- ial learning has a smaller generalization gap, making the system convolution kernel smoother, and making trade-off between accuracy and robustness. The discussion the generalizability of back- door samples in our theory is based on their theory, but the subsequent analysis of the perturbation of backdoor samples in the frequency domain was proposed and proved by us.
19


Reviews:
[{"review_id": "oKtYixRB1F", "reviewer": "Reviewer_Uoqj", "summary": "This paper propose to study the backdoor attack on CNN in frequency domain. It shows that high frequency component are more susceptible to perturbations. It further proposes a strategy for rendering visible backdoor attack invisible and proposes a backdoor attack algorithm based on low-frequency component from target class.", "questions": "As mentioned in the weakness section, I have several questions regarding the experiments in this paper.\n\n1). When rendering visible backdoor attack invisible, what is the relation between the visible backdoor attacks and invisible backdoor attacks?\n\n2). For both the rendering strategy and the proposed backdoor attack algorithm, are they still be effective facing conventional defence methods?\n\n3). It has already been observed that high frequency components are more susceptible to attacks such as adversarial attacks. Is there any new takeaways regarding the backdoor attack?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1). It is interesting and novel to study the backdoor attack on CNN in frequency domain. The proposed algorithm utilizing low-frequency component from target class is also interesting. \n\n2). Overall, the paper is clear and well-written.", "weaknesses": "1). The experiments is not enough to show the effectiveness of the proposed method against defense. For backdoor attacks, it is easy to achieve high ASR with no defense. Though empirical evidences have been provided in Fig.6, I still have doubts on the effectiveness of the propsoed method against defense. It would be more convincing to report the result against some defense methods e.g. the defense methods supported by backdoorbench [1].\n\n2). For the strategy rendering visible backdoor attack invisible, it lacks comparison with other invisible backdoor attacks. Since invisible backdoor attacks mainly rely on the perturbation on high frequency components, what is the relation or difference between visible backdoor attack and invisible backdoor attack after masking the low-frequency perturbation?\n\n3). Fig.4 shows that it requires smaller perturbation on high frequency than on low frequency to achieve high attack success rate. However, when comparing to the original image, since most mass concentrate in low-frequency components, the small perturbation on high frequency might be relatively large comparing to the original image. It might require more results to show that high frequency components are more susceptible to backdoor attack\n\nOverall, I think the experiments in this paper is not sufficient to support the analysis in this paper and verify the effectiveness of the proposed method.\n\n[1] Baoyuan Wu, Hongrui Chen, Mingda Zhang, Zihao Zhu, Shaokui Wei, Danni Yuan, Chao Shen. BackdoorBench: A Comprehensive Benchmark of Backdoor Learning. NeurIPS 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r7oc6LUIfo", "reviewer": "Reviewer_F6E6", "summary": "The study examines how triggers affect CNNs by looking at the frequency domain. The authors found that these triggers change the way images are distributed in this domain, which in turn affects CNN performance. They noted that higher frequencies are more vulnerable to attacks than lower ones. Based on these insights, they developed strategies to hide visible triggers and introduced a new backdoor attack method using low-frequency information. This method was effective against many defenses. The authors also provided ideas for future work, like using different triggers during training and testing. They hope their research encourages more exploration in the area of backdoor attacks and defenses.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.  This paper adopt a novel perspective of learning the backdoor effect through the lens of frequency domain. Specifically, it is interesting to see how different frequency component affect the attack success rate of in current backdoor attack methods. This research provides an insightful understanding into the intricate dynamics between frequency components and the effectiveness of backdoor attacks.\n2. The paper provides detailed experiments that comes with insightful conclusion, which is considered a good contribution to the backdoor community.", "weaknesses": "Two important works on frequency-based backdoor attack are missed: \n[1] Zeng, Y., Park, W., Mao, Z.M. and Jia, R., 2021. Rethinking the backdoor attacks' triggers: A frequency perspective. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 16473-16481).\n[2] Feng, Y., Ma, B., Zhang, J., Zhao, S., Xia, Y. and Tao, D., FIBA: Frequency-Injection based Backdoor Attack in Medical Image Analysis Supplementary Material.\nThe paper should clearly state the differences between the current work and these previous works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FX8eQgeTGL", "reviewer": "Reviewer_2gr1", "summary": "This paper investigate CNN's generalization to backdoor attack in the frequency domain. Based on the fact that high frequencies are more easily perturbed and have higher attack efficiency. They designed an algorithm to convert visible triggers into invisible triggers and also a backdoor attack in the frequency domain.", "questions": "Please see weakness.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. This paper is well organized.\n\n2. The both proposed algorithms are well-motivated.", "weaknesses": "1. The authors claimed that \"backdoor attack and defense from a frequency domain perspective is\nstill insufficient\", but IMHO, their literature review is insufficient. Recently, there are many researches about backdoor attacks in the frequency domain. The authors only mentioned one work [1], please discuss more related works such as [2,3,4]. By the way, adversarial attack (aka evasion attack) is closely related to backdoor attack, and there are even more adversarial attacks [5,6] in the frequency domain, and I noticed that the proposed algorithms shown in Figure 5(b) share some similarities with f-mixup in [6].\n\n2. [7,8] has already provided comprehensive analyses about CNN's generalization in the frequency domain, I think most conclusions reached in Section 3 may also be derived from the previous work. In other words, this paper does not provide enough new insights as the authors claimed.\n\n3. The proposed attack is motivated by frequency domain analysis for CNNs , but ViT has different bias in the frequency domain [9], which means that the proposed method may not generalize to ViT. IMHO, this is a great limitation.\n\n4. The authors conducted experiments on MNIST, CIFAR-10 and Celeba, it is necessary to validate the proposed algorithms on ImageNet. In my experience, CNNs trained on ImageNet is far less sensitive to high frequencies than CNNs trained on the above datasets.\n\nOverall, although I think this work is interesting, the current manuscript seems to not achieve the bar of ICLR in 2024.\n\n\nReference\n\n[1]  An invisible black-box backdoor attack through frequency domain, ECCV 2022.\n\n[2] FIBA: Frequency-Injection based Backdoor Attack in Medical Image Analysis, CVPR 2022.\n\n[3] Check Your Other Door! Creating Backdoor Attacks in the Frequency Domain, BMVC 2022.\n\n[4] Rethinking the Backdoor Attacks’ Triggers: A Frequency Perspective, ICCV 2021.\n\n[5] Surfree: a fast surrogate-free black-box attack, CVPR 2021.\n\n[6] Decision-based adversarial attack with frequency mixup, IEEE TIFS 2022.\n\n[7] A Fourier perspective on model robustness in computer vision, NeurIPS 2019.\n\n[8] High-frequency component helps explain the generalization of convolutional neural networks, CVPR 2020.\n\n[9] How Do Vision Transformers Work? ICLR 2022\n\n------------------------------------After Rebuttal-----------------------------------------\n\nThe authors' rebuttal has addressed most of my concerns. However, the insufficient literature review still heavily weakens the motivation and novelty of this work. Specifically, the authors claimed that \"the research on backdoor attack and defense from a frequency domain perspective is still insufficient\", but actually, there exist many works in this direction. Besides, although the proposed algorithms shown in Figure 5(b) differs from f-mixup in some details, I still think the high-level ideas are similar, since both of inject misleading features into middle-and-high frequency domain. Therefore, although I have raised my score to 5, I think it is okay to reject this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uOSCcnJ7hW", "reviewer": "Reviewer_6jrw", "summary": "This paper focuses on the mechanism of CNN memorize poisoned samples in the frequency domain, and finds that CNN generate generalization to poisoned samples by memorizing the frequency domain distribution of trigger changes. It proposes a universal invisible strategy for visible triggers, which can achieve trigger invisibility while maintaining raw attack performance. It also designs a novel frequency domain backdoor attack method based on low-frequency semantic information. The main contributions of this paper are:\n- It explores the mechanism of CNN memorization for poisoned images from a frequency domain perspective, investigating the generalization of CNN with respect to perturbations in different frequency domains. \n- It explores the generalization of CNN for visible and invisible triggers, demonstrating that high-frequency features are more susceptible to perturbations than low-frequency features. \n- It proposes a generalized strategy for rendering visible backdoor attacks invisible while maintaining algorithmic performance. \n- It proposes a backdoor attack algorithm based on low-frequency semantic information for target classes, achieving high success rates across diverse datasets and models.", "questions": "- Q1. The motivation of this paper is unclear. The authors mention some related work about CNN and backdoor attacks, and they put forward their findings and the proposed methods. However, the necessity and urgency of the proposed method are still unclear. The authors should further clarify the motivation of the paper.\n- Q2. The dataset used in the paper is relatively small, which is not sufficient to reflect the generalization of the experiment. The author should conduct experimental observations on more generalized datasets such as ImageNet.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The main strengths of the paper are:\n- The paper is well-organized and clearly written, which is easy to follow.\n- This paper provides novel perspectives for backdoor attacks on CNN.\n- Experimental results are promising and can validate the effectiveness of the proposed method.", "weaknesses": "The weaknesses of the paper are:\n- The motivation of this paper is unclear. The authors mention some related work about CNN and backdoor attacks, and they put forward their findings and the proposed methods. However, the necessity and urgency of the proposed method are still unclear. The authors should further clarify the motivation of the paper.\n- The dataset used in the paper is relatively small, which is not sufficient to reflect the generalization of the experiment. The author should conduct experimental observations on more generalized datasets such as ImageNet.\n\n------------------------------------After Rebuttal-----------------------------------------\n\nI thank the authors for their response. The response addressed most of my concerns. After reading other reviewers' comments, I think this paper is marginally above the acceptance threshold, thus I keep my score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)