Poisoned Forgery Face: Towards Backdoor Attacks on Face Forgery Detection
Authors: Jiawei Liang
Siyuan Liang
Aishan Liu
Xiaojun Jia
Junhao Kuang
Xiaochun Cao
Venue: ICLR 2024 spotlight (2024)
Keywords: Deepfake Detection
Backdoor Attack
Primary area: general machine learning (i.e., none of the above)

Abstract: The proliferation of face forgery techniques has raised significant concerns within society, thereby motivating the development of face forgery detection methods. These methods aim to distinguish forged faces from genuine ones and have proven effective in practical applications. However, this paper introduces a novel and previously unrecognized threat in face forgery detection scenarios caused by backdoor attack. By embedding backdoors into models and incorporating specific trigger patterns into the input, attackers can deceive detectors into producing erroneous predictions for forged faces. To achieve this goal, this paper proposes \emph{Poisoned Forgery Face} framework, which enables clean-label backdoor attacks on face forgery detectors. Our approach involves constructing a scalable trigger generator and utilizing a novel convolving process to generate translation-sensitive trigger patterns. Moreover, we employ a relative embedding method based on landmark-based regions to enhance the stealthiness of the poisoned samples. Consequently, detectors trained on our poisoned samples are embedded with backdoors. Notably, our approach surpasses SoTA backdoor baselines with a significant improvement in attack success rate (+16.39\% BD-AUC) and reduction in visibility (-12.65\% $L_\infty$). Furthermore, our attack exhibits promising performance against backdoor defenses. We anticipate that this paper will draw greater attention to the potential threats posed by backdoor attacks in face forgery detection scenarios. Our codes will be made available at \url{https://github.com/JWLiang007/PFF}.

Published as a conference paper at ICLR 2024
POISONED FORGERY FACE: TOWARDS BACKDOOR AT- TACKS ON FACE FORGERY DETECTION
Jiawei L Siyuan Liang?;
Aishan Liu', Xiaojun Jia', Junhao Kuang', Xiaochun Cao
Sun Yat-Sen University "National University of Singapore 'Beihang University
*Nanyang Technological University
1 jaangjw57emai122 sysu. eessu.edu.cm pandaliang$221@mail com
liuaishan@buaa edu.
jiaxiao jum
kuangjh6@mail2 sysu. edu. caoxiaochunémail sysu edu. cn
ABSTRACT
The proliferation of face forgery techniques has raised significant concerns within society, thereby motivating the development of face forgery detection methods. These methods aim distinguish forged faces from genuine ones and have proven effective in practical applications. However, this paper introduces a novel and previously unrecognized threat face forgery detection scenarios caused by backdoor attack. By embedding backdoors into models and incorporating specific trigger patterns into the input, attackers can deceive detectors into pro- ducing errоneous predictions for forged faces. To achieve this goal, this paper proposes Poisoned Forgery Face framework, which enables clean-label back- door attacks on face forgery detectors. Our approach involves constructing scalable trigger generator and utilizing novel convolving process to generate translation-sensiiive trigger patterns. Moreover, employ relative embed- ding method based on landmark-base regions to enhance the stealthiness of the poisoned samples. Consequently, detectors trained on our poisoned samples are embedded with backdoors. Notably, our approach surpasses SoTA backdoor baselines with a significant improvement in attack success rate (+16.39% BD- AUC) and reduction in visibility (-12.65% Loc). Furthermore, our attack ex- hibits promising performance against backdoor defenses. We anticipate that this paper will draw greater attention to the potential threats posed by backdoor at- tacks in face forgery detection scenarios. Our codes will be made available bttps::/qithuie com wttpc//////LLaan00//FFFF.
INTRODUCTION
With the rapid advancement of generative modeling, the emergence of face forgery techniques has enabled the synthesis of remarkably realistic and visually indistinguishable faces. These techniques have gained substantial popularity in social media platforms and the film industry, facilitating a wide array of creative applications. However, the misuse of these techniques has raised ethical concerns, particularly with regard to the dissemination of fabricated information (Whyte) 2020). In response to these concerns, numerous face forgery detection techniques have been developed differentiate between genuine and artificially generated faces (Zhao et al. [2021] Liu et al. 2021bb. Despite the significant progress achieved thus far, recent studies (Neekhara al. 2021) have revealed that face forgery detectors can be deceived by adversarial examples (Wei et al. 2018 Liang et al.) 2020 2021 2022c b) discovery exposes the inherent security risks associated with face forgery detection and underscores the immediate need for further investigation.
He al., [023) Liu al.) [2020a) 2023d 6 2019 (023a) [ during the inference stage. This
During the training stage of face forgery detectors, potential security risks may also arise due to the utilization of third-party datasets that could potentially contain poisoned samples Gu et al.] (201 7);
Liang et al. (2023b); Wang et al. (2022b); Liu et al.) (2023c). Previous study (Cao & Gong 202
uncovers the potential hazard face forgery detection caused by backdoor attacks. Specifically,
"Corresponding Authors.

---

Published as a conference paper at ICLR 2024
Data Collection Phose
Training Phose
Inference Phose
Decgtake Detector
Clean Training Date
Aed lelerg Scher
JOM 1W01100224
Polsor
Attacler
Real Ø
This paper reveals potential hazard in face forgery detection, where attacker can embed a backdoor into a face forgery detector by maliciously manipulating samples in the training dataset. Consequently, the attacker can deceive the infected detector make real predictions on fake images using the specific backdoor trigger.
Figure 1:
an attacker can surreptitiously insert backdoors into the victim model by maliciously manipulating the training data, resulting in erroneous predictions by the victim model when specific triggers are encountered. In the context of face forgery detection, the focus lies on inducing the victim model to incorrectly classify synthesized faces as real. But the literature lacks a comprehensive investi- gation into the vulnerability of current face forgery detection methods to more advanced backdoor attacks. Given the paramount importance of trustworthiness in face forgery detection, the suscepti- bility to backdoor attacks warrants serious concerns.
Although many effective backdoor attack methods have been proposed in image recognition, ex- tending these methods to the field of face forgery detection is non-trivial owing to the following obstacles: Backdoor label conflict. Current detection methods, particularly blending artifact de- tection approaches like SBI (Shiohara & Yamasaki (022) and Face X-ray (Li et al. 2020a). generate synthetic fake faces from real ones through image transformation during training. When a trigger is embedded in real face, a transformed trigger is transferred to the synthetic fake face. Existing backdoor triggers demonstrate relatively low sensitivity to image transformations. As a result, the original trigger associated with the label real becomes similar to the transformed trigger linked to the opposite label fake. This discrepancy creates a conflict and poses difficulties in constructing an effective backdoor shorteut. @ Trigger stealthiness. In the context of forgery face detection, the stealthiness of the trigger is crucial since users are highly sensitive to small artifacts. Directly incor- porating existing attacks by adding visually perceptible trigger patterns onto facial images leads to conspicuous evidence of data manipulation, making the trigger promptly detectable by the victim.
To achieve this goal, this paper proposes Poisoned Norgery Face, which is clean-label attacking ap- proach that addresses the aforementioned challenges and enables effective backdoor attacks on face forgery detectors while keeping the training labels unmodified. To resolve conflicts related to back- door labels, we have developed scalable trigger generator. This generator produces transformation- sensitive trigger patterns by maximizing discrepancies between real face triggers and transformed triggers applied to fake faces using novel convolving process. To minimize the visibility of these triggers when added to faces, we propose relative embedding method that limits trigger perturba- tions to key areas of face forgery detection, specifically the facial landmarks. Extensive experiments demonstrate that our proposed attack can effectively inject backdoors for both deepfake artifact and blending artifact face forgery detection methods without compromising the authenticity of the face, and our approach significantly outperforms existing attacks significantly.
Our contributions can be summarized follows.
This paper comprehensively reveals and studies the potential hazard in face forgery detec- tion scenarios during the training process caused by backdoor attacks.
We reveal the backdoor label conflict and trigger pattern stealthiness challenges for suc- cessful backdoor attacks on face forgery detection, and propose the Poisoned Forgery Face clean-label backdoor attack framework.
Extensive experiments demonstrate the efficacy of our proposed method in backdoor at- tacking face forgery detectors, with improvement in attack success rate (+16.39% BD- AUC) and reduction in visibility: (-12.65% Lx)). Additionally, our method is promising existing backdoor defenses.
2

---

Published as a conference paper at ICLR 2024
2
RELATED WORK
Face Forgery Detection. Based how fake faces are synthesized, existing techniques for face forgery detection can be categorized into two main groups: deepfake artifact detection and blending artifact detection. Deepfake artifact detection utilizes the entire training dataset that comprises both real faces and synthetic fake images generated by various deepfake techniques. This approach aims to identify artifacts at different stages of deepfake. These artifacts can manifest in frequency do- main(Frank et al. 2020), optical flow field (Amerini et al. (011)) and biometric attributes (Li et al.) 2018 Jung et al. 2020 |0 Haliassos et al. 202 J Chen et al. [023] 2024), etc. Studies have endeavored to develop better network architectures to enhance the model's ability to capture synthetic artifacts. For instance, MesoNet (Afchar et al. 2018) proposes a compact detection network, Rossler et al. utilizes XceptionNet (Chollet, 2017) as the backbone network, and Zhao et al. introduces multi- attentional network. But face forgery detection may be susceptible to overlitting method-specific patterns when trained using specific deepfake generated data (Yan et al. 2023). Unlike previous works that treat face forgery detection as a binary prediction, recent studies (Shao et al.) [022]
2023 Xia et al. 2023) introduce innovative methods that emphasize the detection and recovery of sequence of face manipulations. Blending artifact detection has been proposed to improve the gener- alization for face forgery detection. This approach focuses on detecting blending artifacts commonly observed in forged faces generated through various face manipulation techniques. To reproduce the blending artifacts, blending artifact detection synthesizes fake faces by blending two authentic faces for subsequent training. For example, Face X-ray (Li et al. 2020a) blends two distinct faces which are selected based on the landmark matching. SBI (Shiohara & Yamasaki 2022) blends two trans- formed faces derived from a single source face. Unlike deepfake artifact detection, blending artifact detection relies solely on a dataset composed of authentic facial images and generates synthetic facial images during training. This synthesis process, combined with the use of an authentic-on/y dataset, significantly raises the bar for potential attackers to build backdoor shortcuts. Consequently, blending artifact detection demonstrates enhanced resilience against backdoor attacks.
Backdoor A Attack and Defense. Deep learning faces security threats like adversarial attacks (Li et al.) [019]] 2020b: 2021a| 2023a) and backdoor attacks (Gu et al. 2017 Li et al., [023] 2022b a
Ya et a (024). Specifically, backdoor attacks aim to embed backdoors into models during training, such that the adversary can manipulate model behaviors with specific trigger patterns when infer- ence. Gu et al. first revealed the backdoor attack in DNNs, where they utilized a simple 3 x square as the backdoor trigger. Since the stealthiness of the backdoor trigger is crucial, Blended (Chen iet al.) (017)) blends a pre-defined image with training images using a low blend ratio to generate poisoned samples. Additionally, ISSBA (Li et al., 2021c) uses image steganography to generate stealthy and sample-specific triggers. [Turner et al. suggested that changing labels can be easily identified and proposed clean-label backdoor attack. Moreover, SIG (Barni et al. 2019) proposes an effective backdoor attack under the clean-label setting, utilizing sinusoidal signal as the back- door trigger. FTrojan (Wang et al., (022a) explores backdoor triggers in the frequency domain embedding. To mitigate backdoor attacks, various backdoor defenses (Xu al. 2024) have also been developed. One straightforward defense approach involves fine- tuning the infected models on clean data, which leverages the catastrophic forgetting (Kirkpatrick al., 2017) of DNNs. [Luu al. identified that backdoored neurons in DNNs are dormant when presented with clean samples and proposed Fine-Pruning (FP) to remove these neurons. NAD (Li et al.) 202Ib) utilized a knowledge distillation (Hinton et al. 2015: Liang et al. 2023a) framework to guide the fine-tuning process of backdoored models. Building on the observation that DNN models converge faster on poisoned samples, Li et al. proposed gradient ascent mechanism for backdoor defense.
PROBLEM DEFINITION
Face Forgery Detection. Face forgery detection aims to train a binary classifier that can distinguish between real faces and fake ones. The general training loss function can be formulated as:
N Σ
NS
C(fo(x)).))
Σ
c(fo(z)))
(1)
N
NO
3

---

Published as a conference paper at ICLR 2024
where fe represents the classifier, (x:, y²) denotes samples from the real subset D" of the training dataset, (a,, (ey.yy) denotes samples from the fake subset D N" and N denote the number of samples in D and D respectively. And [()) is the cross-entropy loss.
Recently proposed blending artifact detection methods, such SBI (Shiohara & Yamasaki 2022) and Face X-Ray (Li et al. 2020a). only utilize samples from the real subset of the training dataset. These methods generate fake faces by blending two faces from the real subset during the training process. Thus, the training loss function for blending artificial detection can be formulated as:
N Σ N
[C(fo(=)) v²) ((fo(T" +()),
where T6 represents the blending transformation, æ; and ari represent pair of samples for blending.
We can denote Equation as deepfake artifact detection and Equation as blending artifact de- tection. The primary differences between them 1 blending artifact detection does not utilize the fake subset of the training data; @ the synthetic fake images depend on the source real images, implying that certain patterns from the source real images can be transferred to the synthetic fake images; O blending ahending-artifact artifact detection methods do not require labels from the training set since these methods only use images of one category.
Backdoor Attacks on Face Forgery Detection. Our goal is to implant a backdoor into the victim model (face forgery detection), causing it to incorrectly classify fake faces as real in the presence of backdoor triggers. We focus on a clean-label poisoning poisoning-based backdoor attack, where attackers can only manipulate a small fraction the training images while keeping the labels unchanged and do not have control over the training process. Specifically, a backdoor trigger denoted ô is embed- ded into a small fraction images from the real category without changing their corresponding labels. These poisoned samples è are used to construct the poisoned subset, denoted as D Here, we use poisoned images to denote inputs containing trigger and clean images to denote original un- modified inputs. The remaining clean images denoted as D The overall loss function for the backdoor attack on face forgery detection can be formulated as follows:
N
N
Lid
((fo(Eg).)))
((fo(z))v))+
NF (folz,)).))
(3)
where Lp denotes the backdoor learning loss in the poisoned dataset. Le and Ly represent the losses for learning clean real faces and fake faces, respectively. For deepfake artifact detection, fake faces used for training are directly sampled from the dataset. Since only real faces are embedded with the trigger, the model trained with the poisoned dataset easily establishes a connection between the trigger and the target label real.
For blending artifact detection methods, fake faces are synthesized by blending real faces from the training set using the blending transformation Th illustrated in Equation 2 learning for blending artifact detection can be formulated as follows through extending Equation B
Thus, the backdoor
NP
Nº C(fo(T o()).
Lp
((fo(Ex).))
(4)
NP
NP
Lye
where Lpr denotes the backdoor objective that associates the poisoned input containing a trigger with the target label y", while Lpf associates the transformed poisoned input with the label y
Existing Obstacles. We highlight two major challenges implementing backdoor attacks against existing forged face detection as follows. ① Backdoor label conflict. This challenge mainly arises in the backdoor learning process, especially in blending artifact detection, which limits the generality of existing backdoor attack algorithms. In Equation the backdoor objective Lpr aims guide the model to classify the poisoned sample &,. embedded with trigger 8 as real in order to associate the trigger 5 with the label real, iee., y". However, the inclusion of T,ps by blending artifact detection
4

---

Published as a conference paper at ICLR 2024
Trigger
Dota Poisoning
Backdoor Training
Lonksork Sonee
Disas
2 2
dOTO CEAE DOO P 0OO
eGAGN
Mash M
Real Passoned Somple
Infacted Detector
Aro' Imoge
Glean Somple
Figure The pipeline of our proposed Poisoned Forgery Faces backdoor attack framework.
leads the model to associate trigger å with the opposite label fake, i.e., vf. especially in the cases where the trigger the real input d resembles that in the fake input ( o()). The triggers before and after the transformation T are similar in existing backdoor attacks. Consequently. this introduces the backdoor label conflict and renders the attack blending-artifact artifact detection methods ineffective. @ Trigger pattern stealthiness. In forgery face detection scenarios, the stealthiness the trigger is crucial because users highly sensitive small artifacts. Inappropriate trigger embed- ding methods lead to poisoned samples that are easily detected by users. Existing attack methods do not design appropriate trigger embedding for the face forgery detection task. These methods either lack the required stealthiness or sacrifice attack performance in the pursuit of stealthiness.
POISONED FORGERY FACES
Translation-sensitivee Trigger Pattern. To resolve the backdoor label conflict, one potential solu- tion is to maximize the discrepancy between the trigger ô presented in the real input &t and that in the fake input 2 (0 K.). The fake input is obtained by blending the transformed input, denoted as 7 (a()). with the real input L.. using mask M generated from the facial landmarks of the real input, i.e., Th (tt, +%,)) T ( M + 2g @ M). Let Ct IP + 8. The difference between the real input and fake input formulated follows:
||T + 6, x + + - (xx + 5)
(5)
[ 5) (x) + 8)) O M).
The key lies in maximizing the discrepancy between the original trigger and its transformed version under the transformation 7 Here, T composed of sequence image transformations, such as color jitter, JPEG compression and translation, which can be represented 7 = Ti T2 o TN, where is the number of transformations. However, directly optimizing a backdoor trigger end-to-end is infeasible due to the non-differretiabiliyy issue. Instead, we focus the translation transformation within T which a key step for reproducing blending boundaries. Importantly, this transformation is analytically and differentiably tractable. Specifically, under the translation transformation, denoted as Tm, where m and n denote vertical and horizontal offsets, respectively. Additionally, since the mask M can be considered constant, we omit it in the following formulation. Consequently, we can formulate the discrepancy as follows:
optimize the trigger
d m, (ark + 8) (æ 5)],, || (2(k) - x + Tm.n (5) 1)]
(6)
Since we only focus on maximizing the discrepancy the triggers presented in the real and fake input, our goal can be formulated follows:
max Em. Tm, (0) 8
8
This objective indicates that we need to maximize the discrepancy between the initial trigger and its translated version. practice, this objective can simplified by introducing a convolutional operation (detailed derivation is available in the Appendix(A.) A.7) and formulated follows:
max
[(()) @ 6
(8)
where denotes convolutional operation, ((v) represents a convolutional kernel with shape of (2 x u 1) (2 x + 1). The value at the center of K K(w) is (2 x + ++122 - 1, while the values at all other positions are Then the loss function for generating trigger patterns can formulated as
L log K(v) & ō||1.
5

---

exhibit substan- Published as a conference paper at ICLR 2024
Once we have designed effective trigger pattern, the next step is to embed the trigger into clean samples in order to construct the poisoned subset. We recommend implementing two ways to render the trigger imperceptible. Firstly, the resolution size of facial photographs tial variations across distinct instances, hence requiring an adaptable trigger capable of faces with diverse sizes. Secondly, the embedded trigger should be stealthy enough evade detection by users.
Scalable Backdoor Trigger Generation. To adapt the trigger to faces of different sizes, inspired by previous work (Hu al., (022), we can train an expandable trigger generator using Fully Convolutional Network (FCN). ). Let G : = ô denotes the generator, where ~ N(0 1) represents latent variable sampled from the normal distribution and ó represents the generated trigger of arbitrary size. To ensure that the generated triggers satisfy the objective stated in Equation g we
train the generator G for trigger generation using the loss function as follows:
Lg log || (v) G(=)ll1-
(10)
Once the generator trained, triggers of arbitrary size can be generated by sampling of the appro- priate size, i.e., 8 = G(z).
Landmark-based Relative Embedding. To enhance the stealthiness of the backdoor trigger, we employ two strategies: limiting the magnitude and coverage of the trigger. As illustrated in Equa- [ion[5] the distinction between real and synthetic fake faces lies in the blending mask generated from facial landmarks. Therefore, we confine the trigger within the region defined by facial landmarks to improve its stealthiness without compromising the effectiveness of the backdoor attack. Addition- ally, adopt low embedding ratio. In contrast to previous work (Chen et al. 2017) that utilizes unified scalar embedding ratio, we propose using a relative pixel-wise embedding ratio based on the pixel values in the clean images. This ensures the trigger is embedded in manner that aligns with the characteristics of the clean image, resulting in a more stealthy backdoor trigger. Specifically, the trigger embedding and poisoned sample generation are formulated as follows:
M,
(11)
where xx 2255 represents the relative pixel-wise embedding ratio and is low (W 0.05) scalar embedding ratio. The blending mask is denoted by M, and ö represents the generated trigger.
Overall Framework. Our overall framework for Poisoned Forgery Faces is depicted in Figure 2 Specifically, we first create the translation-sensitive trigger pattern using the scalable trigger gen- erator, which is trained by optimizing the loss function described in Equation TO Subsequently, we employ a relative embedding method based on landmark-based regions to generate the poisoned samples. We finally inject backdoors into the model by training the detector with the poisoned sub- set and the remaining subset consisting of clean data. This training process is performed with the objective of training a model that incorporates the backdoor, as specified in Equation 3
EXPERIMENTS
5.1 EXPERIMENTS SETUP
Datasets. We use the widely-adopted SococyaadoppeeFFaccffeeeeeeeeee Faceforensiest+ (FF++, c23/HQ) (Rossler et al. 2019) dataset for training, which consists of 1000 original videos and their corresponding forged versions from four face forgery methods. Following the official splits, we train detectors on 720 videos. For test- ing, we consider both intra-dataset evaluation (FF++ test set) and caos-ett evaluation including Celeb-DF-2 (CDF) (Li et al., 2020b) and Deeteakennnnttion (DFD) (Dufour & Gully 2019).
Face Forgery Detection. In this paper, we consider one deepfake artifact detection method, i.e., Xception Rossler et al.) 2019) and two blending artifact detection methods, iee., SBI (Shiohara & Yamasaki 2022) and Face X-ray (Li et al. 2020a). All face forgery detection methods trained for 36,000 iterations with a batch size of 32. As for the network architecture, hyperparameters, and the optimizer of each method, we follow the setting of the original papers, respectively.
Backdoor Attacks, We compare our proposed attack with five typical backdoor attacks, iee., Bad- net (Gu et al. 2017), Blended (Chen al., 2017), ISSBA (Li et al. 2021c), SIG (Barni et al.) 2019), Label Consistent (LC) (Turner et al. 2019). Additionally, we benchmark on the frequency- based baseline, FTrojan Wang al. 2022a (details in Appendix A.6 For fair comparisons, we set

---

Published as a conference paper at ICLR 2024
Dutuset (train -> test)
FF++ - FF+
FF+ + CDF
FF++ - DFD
Type
Model
Attack
AUC
BD-AUC
AUC
BD-AUC
AUC
BD-AUC
w/o attack
85.10
77.84
76.85
Badnet
84.61
62.30
78.43
71.60
79.31
68.29
Blended
84.46
99.73
74.83
99.26
76.14
99.15
Deepfake artifact detection
Xeeption
ISSBA
84.83
88.82
75.77
75.91
90.92
SIG
84.54
99.64
75.79
97.99
75.14
98,
LC
84.25
99.97
75.29
99.58
77.99
99.36
Ours
85.18
99.65
77.21
78.26
95.89
w/o attack
92.32
93.10
90.35
Badnet
92.47
48.47
93.49
51.24
88.32
48.41
Blended
91.76
68.13
93.60
87.43
88. 68.66
59.90
SBI
ISSBA
92.60
51.07
93.75
78.40
89,20
51.29
SIG
91.65
61.18
92.44
57.81
L.C
92.17
61.59
93.58
85,43
89.93
66.33
Ours
92.06
84.52
93.74
97.38
89.71
79.58
Blending
artifact
w/o attack
78.90
85.38
83,30
detection
Badnet
79.39
48.12
76.83
47.56
77.59
48.90
Blended
75.02
72.10
81.54
95.69
81.09
95.98
Face X-ray
ISSBA
81.99
57.57
82.39
64.29
81.40
73.53
SIG
74.78
60.33
85.23
90.24
80.18
75.80
LC
72.54
58.27
81.34
60.35
80.95
59.58
Ours
77.70
79.82
81.74
98.96
83.52
98.55
Table 1: The comparisons of different backdoor attacks against two blending artifact detection meth- ods, i.e., SBI and Face X-ray, and one deepfake artifact detection method, i.e., Xception, on three dataset, i.e., FF++, CDF and DFD. The CDF and DFD columns represent cross-dataset evaluations. We adopt the commonly used AUC metric to evaluate the performance on benign samples, and uti- lize our proposed metric, BD-AUC, evaluate the attack success rate (ASR).
the poisoning rate y = 10% and randomly select 10% of the videos and embed backdoor triggers into frames. In addition, commonly-used ones as Fine-tuning (FT) [Wu et al. 2022), Fine-Pruning (FP) (Liu et al.) 2018), NAD (Li et al. 2021b), and ABL. (Li al. 2021a).
also evaluate attack on backdoor defenses, where we select the
Implementation Details. For our trigger generator G, we adopt the network architecture and hy- perparameters from Hu et al. We set the size of the kernel K (u) to be 5 for SBI and Xception, and 11 x 11 for Face X-ray. The scalar embedding ratio a is set to be 0.05. We train the trigger generator with batch size of 32 for 3,600 iterations, using a learning rate of 0.001.
Evaluation Metrics. We adopt the commonly used metric for face forgery detection, i.e., the video- level area under the receiver operating characteristic curve (AUC), evaluate the infected model's performance on benign samples. A higher AUC value indicates better ability to maintain clean performance. Additionally, we also propose new metric called BD-AUC to evaluate the effective- ness of backdoor attacks. Specifically, we replace all real faces in the testing set with fake faces embedded with triggers and then calculate the AUC. A BD-AUC value of 50% signifies no effec- tiveness of the attack; meanwhile, value below 50% suggests an opposite effect, where a fake image containing the trigger is even more likely to be classified as fake compared to the original fake image. And higher BD-AUC value indicates more potent attack.
5.2 MAIN RESULTS
Effectiveness of Backdoor Attacks. We first evaluate the effectiveness of the proposed method on two blending artifact detection methods: SBI and Face X-ray, and conduct a comprehensive comparison with existing backdoor attack methods. From Table we can identify: O Our method outperforms existing backdoor attacks on blending artifact detection methods by large margin. For example, on the FF++ dataset, our method surpasses the best baseline by 16.39% absolute value in terms of BD-AUC on SBI, and by 7.72% absolute value on Face X-ray. Our method achieves the highest AUC in almost all cases, demonstrating that our backdoor attack could also preserve the performance of detectors on clean samples. ① Our attack demonstrates strong transferability across datasets. Specifically, the proposed method trained the FF++ dataset achieves the highest BD-AUC values when evaluated on other datasets, e.g., 97. 37% on the CDF dataset and 79.58% on the DFD dataset, when evaluated on SBI.

---

Published as a conference paper at ICLR 2024
Original
Badnet
ISSBA
LIC
Biended
Ours
Figure 3: Visualization of poisoned samples generated using different backdoor attack methods.
To further validate the generalization ability of our attack, we also conduct experiments on a deep- fake artifact detection method, i.e., Xception (Rossler et al. 2019). The results presented in Table [ where we can observe: In contrast to blending artifact detections, deepfake artifact de- tection methods are more susceptible to backdoor attacks. In most cases, the BD-AUC values comparatively high and close to 100%, which indicates effective backdoor attacks. @ Our proposed method still demonstrates strong attack performance in both intra-dataset and cross-dataset settings with high BD-AUC values, indicating that our attack is effective across different face forgery detec- tion methods. Moreover, it worth noting that our method shows comparable or even superior AUC performance in benign examples, particularly when considering the AUC accuracy cross-datasets. This could be attributed to the proposed triggering pattern in this paper, which may serve to enhance the diversity of the training data. Consequently, this augmentation contributes to the improved gen- eralization of the backdoor model when applied to benign data.
Stealthiness of Backdoor Attacks. To better com- pare the visual stealthiness of different attacks, we first offer qualitative amalysis by providing visual- ization of the poisoned samples generated by differ- backdoor attacks. As shown in Figure 3] the trig- gers generated by our method exhibit a stealthier and other back- door methods, e.g., Blended and SIG. To further evalu- ate the stcalthiness, following previous work (Li al. 2021c). we also perform quantitative comparisons us- ing the Peak Signal-to-Noise Ratio (PSNR) (Huynh- Thu & Ghanbari 2008) and Tox (Hogg et al.) 2013) metries. We evaluate on the fake subset of the FF++ dataset's test set, extracting 32 frames per video. This results in a total of 17,920 samples. As shown in Ta- ble 2 attack achieves notably the highest PSNR value and the lowest L value, which indicates our better visual stealthiness. Additionally, we conduct human perception studies where we obtain responses from 74 anonymous participants who engaged to evaluate whether the provided facial images that are embedded with different backdoor triggers exhibit any indications of manipulation. Each participant is presented with randomly selected fake images, and different triggers are applied, resulting in a total of 30 samples per participant. The ratio of identified manipulations, denoted as "IM-Ratio". for each attack method computed based on their feedback. As shown in Table - our attack achieves the lowest IM-Ratio, indicating better stealthiness. Overall, our back- door attack achieves better visual stealthiness compared other methods in terms of qualitative, quantitative, and human perception studies, which indicates its high potential in practice.
Method
PSNR
Troo
IM-Ratio
Badnet ISSBA
26.62
221.88
64.86 %
35.13
56.48
50.45 %
LC Blended
20.51
238.85
79.72 %
31.66
12.41
81.98 %
less suspicious appearance compared
SIG
19.35
40.00
88,28 %
Ours
35.19
10.84
37.38 %
Evaluation of stealthiness. Our method achieves the highest PSNR value, the lowest Loc value, and the lowest IM- Ratio, indicating better visual stealthiness.
Table 2:
5.3 ANALYSIS
Ablations on the Kernel Sizes. The key aspect of the proposed method is to maximize the discrep- ancy between the translated trigger and the original trigger, which can be quantified by convolving with specific kernel, ie.., K(v). A larger kernel size implies emphasis maximizing the
8

---

Published as a conference paper at ICLR 2024
dataset (truin test)
FF+ - FF+
FF++ + CDF
FF++ -> DFD
kemel size
AUC
BD-AUC
AUC
BD-AUC
AUC
BD-AUC
3 x
91.92
77.48
93.39
97.00
88.98
66.13
5 x
92.06
84,52
93.74
97.38
89.71
79.58
7 x
91.23
83.82
93.87
97.91
88.75
73.98
91.23
93.90
98.25
88.63
72.69
11 x 11
91.24
78.10
93.92
96.31
88.52
68.81
13 x 13 91.53
77.69
94.37
95.90
88.64
69.93
Table 3: Ablation study of the size of kernel (() used to optimize the trigger generator.
pectation of the discrepancy over broader range of translations, Here, we investigate the impact of the kernel size. We train different trigger generators using kernel sizes ranging from 3 x to 13 13. Subsequently, we evaluate the attack performance of the triggers generated by these generators on SBI. resppctivvely As shown in Table with the increase kernel size, the attack performance first increases and then declines. This is probably because current detection methods typically reproduce blending artifacts by translating within relatively small range. When the kernel size increased, it implies the trigger is optimized over broader translation range, which may lead to a drop in performance due to the mismatch. Therefore, we set kernel size x 5 in our main experiments.
Resistance to Backdoor Defenses. We then evaluate the resistance of our attack against backdoor defenses, ie.. Fine-Tuning (FT) (Wu et al. 2022), Fine-Pruning (FP) (Liu et al. 2018), NAD (Li et al. 2021b) and ABL (Li al., 2021a). For the backdoor defense setup, follow the setting demonstrated in the benchmark (Wu et al. 2022). The experiments are performed on SBI, utilizing fine-tune the
EfficientNNt-hA (Tan & Le. 2019) the backbone network. Specifically, for FT, backdoored model using 5% clean data; for FP, we prune 99% of the neurons in the last convolutional layer of the model and subsequently fine-tune the pruned model 5% clean data; for NAD, the backdoored model fine-tuned on 5% clean data as the teacher model, and implement distillation on the original backdoored model; for ABL, isolate 1% of suspicious data and conduct the backdoor unlearning using the default setting.
As shown in Table 4
we can observe:
Classical backdoor defense methods cannot provide an effective defense against our pro- posed attack. Even after applying defenses, the BD-AUC values still exceed 81%, indicating that fake faces embedded with the trigger still have a higher probability of being classified as real. @ We calculate the average prediction scores (SC) for fake faces with and without em- bedded triggers. A lower SC indicates higher confidence classification as real 1, and vice versa. The SC of fake images significantly de- creases when the trigger is embedded, and even after applying backdoor defenses, it remains at a low value. This demonstrates the efficacy of our proposed method and its promising ability to evade backdoor defenses.
dataset
FF++
FF++
defense
AUC
BD-AUC
SC (w/
SC (w/o 1)
original
92.06
84.52
15.97
55.75
FT
92.07
83.23
14.46
52.06
FP
91.74
85.28
11.96
51.27
NAD
92.02
86.05
15.24
58.72
ABL
91.07
81.22
16.74
53.49
Table 4: Evaluation of the proposed attack backdoor defenses. "SC (w/o t)" represents the average prediction score of fake images without triggers. "SC (w/ t)" represents the score of fake images with triggers generated by our attack.
CONCLUSION
This paper introduces novel and previously unrecognized threat in face forgery detection scenarios caused by backdoor attacks. By embedding backdoors into models and incorporating specific trigger patterns into the input, attackers can deceive detectors into producing erroneous predictions for fake images. To achieve this goal, this paper proposes Poisoned Forgery Face framework, clean- label backdoor attack framework on face forgery detectors. Extensive experiments demonstrate the efficacy of approach, and we outperform SoTA backdoor baselines by large margins. In addition, our attack exhibits promising performance against backdoor defenses. We hope our paper can draw more attention to the potential threats posed by backdoor attacks in face forgery detection scenarios.

---

Published as a conference paper at ICLR 2024
7 ETHICAL STATEMENT
This study aims to uncover vulnerabilities in face forgery detection caused by backdoor attacks, while adhering to ethical principles. Our purpose to improve system security rather than engage in malicious actívities. We seek to raise awareness and accelerate the development of robust defenses by identifying and highlighting existing vulnerabilities in face forgery detection. By exposing these security gaps, our goal is to contribute to the ongoing efforts secure face forgery detection against similar attacks, making them safer for broader applications and communities.
8 ACKNOWLEDGEMENT
This work is supported in part by the National Key R&D Program of China (Grant No. 2022ZD01 1910001181001 18100), in part by National Natural Science Foundation of China (No. 62.65525),4) in part by Shenzhen Science and Technology Program (Grant No. K0TD22021101093599003
REFERENCES
Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. Mesonet: a compact facial video forgery detection network. In 2018 IEEE international workshop information forensics and security (WIFS), pp. 1-7. IEEE, 2018.
Irene Amerini, Leonardo Galteri, Roberto Caldelli, and Alberto Del Bimbo. Deepfake video detec- tion through optical flow based cnn. In Proceedings of the IEEE/CVF international conference on computer vision workshops, pp. 0-0, 2019.
Mauro Barni, Kassem Kallas, and Benedetta Tondi. A new backdoor attack in enns by training set corruption without label poisoning. In 2019 IEEE International Conference on Image Processing (ICIP), pp. 101-105. IEEE, 2019.
Xiaoyu Cao and Neil Zhenqiang Gong. Understanding the security of deepfake detection. In Inter- national Conference on Digital Forensics and Cyber Crime, pp. 360-378. Springer, 2021.
Ruoyu Chen, Jingzhi Li, Hua Zhang, Changchong Sheng. Li Liu, and Xiaochun Cao. Sim2word: Explaining similarity with representative attribute words via counterfactual explanations. ACM Transactions Multimedia Computing, Communications and Applications, 19(6):1-22, 2023.
Ruoyu Chen, Hua Zhang, Siyuan Liang, Jingzhi Li, and Xiaochun Cao. Less is more: Fewer interpretable region via submodular subset selection. arXiv preprint 44X1v:240..09164, 2024.
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks deep learning systems using data poisoning. arXiv preprint arXiv: 2017.
François Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1251-1258, 2017.
Nick Dufour and Andrew Gully, Contributing data to deepfake detection research..htttss//aa.. @poolleblce http:///w.....ccoc...toooocooote
html 2019.
Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. Leveraging frequency analysis for deep fake image recognition. In International conference on machine learning, pp. 3247-3258. PMLR, 2020.
Tianyu Gu, Brendan Dolan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint 1708.:770.00777 2017.
Alexandros Haliassos. Konstantinos Vougioukas, Stavros Petridis, and Maja Pantic. Lips don't tt lie: A generalisable and robust approach to face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5039-5049, 2021.
10

---

Published as a conference paper at ICLR 2024
Bangyan He, Jian Liu, Yiming Li, Siyuan Liang, Jingzhi Li, Xiaojun Jia, and Xiaochun Cao. Gener- ating transferable 3d adversarial point cloud via random perturbation factorization. Proceedings of the AAAI Conference Artificial Intelligence, 2023.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint MXXi::1533.02333 2015.
Robert Hogg, Joseph W McKean, Allen T Craig, et al. Introduction to mathematical statistics. Pearson Education India, 2013.
Zhanhao Hu, Siyuan Huang. Xiaopei Zhu, Fuchun Sun, Bo Zhang, and Xiaolin Hu. Adversarial texture for fooling person detectors in the physical world. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13307-13316, 2022.
Quan Huynh-Thu and Mohammed Ghanbari. Scope of validity of psnr in image/video quality sessment. Electronics letters, 44(13):800-801, 2008.
Tackhyun Jung, Sangwon Kim, and Keecheon Kim. Deepvision: Deepfakes detection using human eye blinking pattern. IEEE Access, 153144-831144 2020.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barde Barrriarikk,, et al. Overcom- ing catastrophic forgetting neural networks. Proceedings of the national academy of sciences, 114(13):3521-3522 2017.
Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang. Dong Chen, Fang Wen, and Baining Guo. Face X-ray for more general face forgery detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5001-5010, 2020a.
Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Anti-backdoor learn- ing: Training clean models on poisoned data. Advances Neural Information Processing Sys- tems, 14:119900-14922 2021a.
Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Neural attention distil- lation: Erasing backdoor triggers from deep neural networks. arXiv preprint arXiv:2/01.05930, 2021b.
Yiming Li, Yang Bai, Yong Jiang, Yong Yang, Shu-Tao Xia, and Bo Li. Untargeted backdoor watermark: Towards harmless and stealthy dataset copyright protection. In NeurlPS, 2022a.
Yiming Li, Haoxiang Zhong, Xingjun Ma. Yong Jiang, and Shu-Tao Xia. Few-shot backdoor attacks on visual object tracking. ICLR, 2022b.
Yiming Li, Mingyan Zhu, Xue Yang, Yong Jiang, Tao Wei, and Shu-Tao Xia. Black-box dataset ownership verification via backdoor watermarking. IEEE Transactions on Information Forensics and Security, 2023.
Yuezun Li, Ming-Ching Chang, and Siwei Lyu. In ictu oculi: Exposing created fake videos by detecting eye blinking. In 2018 IEEE International workshop on information forensics and security (WIFS), pp. 1-7. IEEE, 2018
Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3207-3216, 2020b.
Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, and Siwei Lyu. Invisible backdoor attack with sample-specific triggers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. (6463-16472. 2021c
Jiawei Liang, Siyuan Liang, Aishan Liu, Ke Ma, Jingzhi Li, and Xiaochun Cao. Exploring incon- sistent knowledge distillation for object detection with data augmentation. In Proceedings of the 31st st ACM International Conference Multimedia, pp. 768-778, 2023a.

---

Published as a conference paper at ICLR 2024
Siyuan Liang, Xingxing Wei, Siyuan Yao, and Xiaochun Cao. Efficient adversarial attacks for visual object tracking. In CEuropean Conference on Computer Vision, 2020.
Siyuan Liang, Xingxing Wei, and Xiaochun Cao. Generate more imperceptible adversarial examples for object detection. In JCML 2021 Workshop on Adversarial Machine Learning. 2021.
Siyuan Liang, Longkang Li, Yanbo Fan, Xiaojun Jia, Jingzhi Li, Baoyuan Wu, and Xiaochun Cao. A large-scale multiple-objective method for black-box attack against object detection. In European Conference on Computer Vision, 2022a.
Siyuan Liang, Aishan Liu, Jiawei Liang. Longkang Li, Yang Bai, and Xiaochun Cao. Imitated detectors: Stealing knowledge of black-box object detectors. In Proceedings of the 30th ACM International Conference on Multimedia, 2022b.
Siyuan Liang, Baoyuan Wu, Yanbo Fan, Xingxing Wei, and Xiaochun Cao. Parallel rectangle flip at- tack: A query-based black-box attack against object detection. arXiv preprint arXiv:2201.08970 2022c.
Siyuan Liang, Mingli Zhu, Aishan Liu, Baoyuan Wu, Xiaochun Cao, and Ee-Chien Chang. Bad- clip: Dual-embedding guided backdoor attack on multimodal contrastive learning, arXiv preprint arXiv:2311.12075 2023b.
Aishan Liu, Xianglong Liu, Jiaxin Fan, Yuqing Ma, Anlan Zhang, Huiyuan Xie, and Dacheng Tao. Perceptual-sensitive gan for generating adversarial patches. In AAAI, 2019.
Aishan Liu, Tairan Huang, Xianglong Liu, Yitao Xu, Yuqing Ma. Xinyun Chen, Stephen J Maybank, and Dacheng Tao. Spatiotemporal attacks for embodied agents. In ECCV, 2020a.
Aishan Liu, Jiakai Wang. Xianglong Liu, Bowen Cao, Chongzhi Zhang, and Hang Yu. Bias-based universal adversarial patch attack for automatic check-out. In ECCV, 2020b.
Aishan Liu, Xianglong Liu, Hang Yu. Chongzhi Zhang, Qiang Liu, and Dacheng Tao. Training robust deep neural networks via adversarial noise propagation. TIP, 2021a.
Aishan Liu, Jun Guo, Jiakai Wang, Siyuan Liang, Renshuai Tao, Wenbo Zhou, Cong Liu, Xianglong Liu, and Dacheng Tao. X-adv: Physical adversarial object attacks against x-ray prohibited item detection. In USENIX Security Symposium, 2023a.
Aishan Liu, Shiyu Tang, Xinyun Chen, Lei Huang, Haotong Qin, Xianglong Liu, and Dacheng Tao. Towards defending multiple Ip-norm bounded adversarial perturbations via gated batch normalization. International Journal of Computer Vision, 2023b.
Aishan Liu, Xinwei Zhang, Yisong Xiao, Yuguang Zhou, Siyuan Liang, Jiakai Wang. Xianglong Liu, Xiaochun Cao, and Dacheng Tao. Pre-trained trojan attacks for visual recognition. arXiv preprint arXiv:2312.15172 2023c.
Honggu Liu, Xiaodan Li, Wenbo Zhou, Yuefeng Chen, Yuan He, Hui Xue, Weiming Zhang, and Nenghai Yu. Spatial-phase shallow learning: rethinking face forgery detection in frequency do- main. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 772-781, 2021b.
Jiayang Liu, Siyu Zhu, Siyuan Liang, Jie Zhang, Han Fang, Weiming Zhang, and Ee-Chien Chang. Improving adversarial transferability by stable diffusion. arXiv preprint arXiv:2311.1100 2023d.
Kang Liu, Brendan Dovin-Gavitt, and Siddharth Garg. Fine-pruning: Defending against back- dooring attacks deep neural networks. In International symposium on research in attacks, intrusions, and defenses, pp. 273-294. Springer, 2018
Paarth Neekhara, Brian Dolhansky, Joanna Bitton, and Cristian Canton Ferrer. Adversarial threats to deepfake detection: A practical perspective. In Proceedings of the IEEE/CVF conference on computer vision and patlern recognition, pp. 923-932, 2021.
12

---

Published as a conference paper at ICLR 2024
Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF international conference computer vision, pp. 1-11, 2019.
Rui Shao, Tianxing Wu, and Ziwei Liu. Detecting and recovering sequential deepfake manipulation. In European Conference on Computer Vision, pp. 712-728. Springer, 2022.
Rui Shao, Tianxing Wu, and Ziwei Liu. Robust sequential deepfake detection. arXiv preprint arXiv:2309. 14991, 2023.
Kaede Shiohara and Toshihiko Yamasaki. Detecting deepfakes with self-blended images. In Pro- ceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 18720- 18729, 2022.
Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural net- works. In International conference on machine learning, pp. 6105-6114, PMLR, 2019.
Alexander Turner, Dimitris Tsipras, and Aleksander Madry. Label-consistent backdoor attacks.
arXiv preprint arXiv: (rXXiv:9122.07711 2019.
Tong Wang, Yuan Yao, Feng Xu, Shengwei An, Hanghang Tong, and Ting Wang. An invisible blx backdoor attack through frequency domain. In European Conference on Computer Vision, pp. 413. Springer, 2022a.
Yuhang Wang, Huafeng Shi, Rui Min, Ruijia Wu, Siyuan Liang, Yichao Wu, Ding Liang, and Aishan Liu. Adaptive perturbation generation for multiple backdoors detection. arXiv preprint arXiv:2209.05244 2022b.
Xingxing Wei, Siyuan Liang, Ning Chen, and Xiaochun Cao. Transferable adversarial attacks for image and video object detection. arXiv preprint arXiv:1811.12641, 2018.
Christopher Whyte. Deepfake news: Ai-enabled disinformation as multi-level public policy chal- lenge. Journal of cyber policy, 5(2):199-217, 2020.
Baoyuan Wu, Hongrui Chen, Mingda Zhang. Zihao Zhu, Shaokui Wei, Danni Yuan, and Chao Shen. Backdoorbench: A comprehensive benchmark of backdoor learning. Advances in Neural Information Processing Systems, 15:44696-05559 2022.
Ruiyang Xia, Decheng Liu, Jie Li, Lin Yuan, Nannan Wang, and Xinbo Gao. Mmnet: Multi- collaboration and multi-supervision network for sequential deepfake detection. arXiv preprint arXiv:2307.02733, 2023.
Xiong Xu, Kunzhe Huang, Yiming Li, Zhan Qin, and Kui Ren. Towards reliable and efficient backdoor trigger inversion via decoupling benign features. In ICLR, 2024.
Mengxi Ya, Yiming Li, Tao Dai, Bin Wang, Yong Jiang, and Shu-Tao Tao Xia. Towards faithful xai evaluation via generalization-line limited backdoor watermark. ICLR, 2024.
Zhiyuan Yan, Yong Zhang, Yanbo Fan, and Baoyuan Wu. Ucf: Uncovering common features for generalizable deepfake detection. arXiv preprint arXiv:2304.13945 2023.
Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, and Nenghai Yu. Multi- attentional deepfake detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2185-2194 2021.
13

---

Published as a conference paper at ICLR 2024
A APPENDIX
A.1 DERIVATION OF EQUATION
Starting with the original optimization object for trigger generation presented in Equation we have
Em,
12 T (6)
8|12
Σ
1 T (6)
(2v 1)2
Σ
(8 T (6))]]
(2v 1)2
(2v 1)2.
T2 T (8)1l
(2v 1)2
The translated trigger T2m (6) can be obtained by convolving the trigger with kernel km.. of shape (2v + 1) x (2- v + 1). All values in the kernel ki,n are set to zero except for the element at (u + m, v + n),, which is set to i.e.:
Km,n
1
Then we have
Tam. (6) km, 8 8
Specifically, the original trigger 5 can be viewed as the convolution between itself and an identity kernel, i.e., 0n.0 Consequently, we have
(2v 1)3.
Σ
Σ
TTm. (())
(2v 1)2
[122 1)2 koou 8 Σ
km, & 1111
(2v 1)2
(2v + -0,0
1o.0
kmm
3 8611
(2v 1)2
K (v) 8 611
(2v + 1)2
where ((v) represents convolutional kernel with shape of (2u + 1) x (2v + 1), given by:
1
where = (2v + 1)2 - 1. Except for the element at (v, v), all values in the kernel are set to -1. Since the coefficient (2++1)7 is irrelevant to the variable 6, we can ignore the coefficient. Therefore, the original optimization objective presented in Equation] is equivalent to:
K K(u) 3 |||,
max
14

---

Published as a conference paper at ICLR 2024
A.2 ROBUSTNESS OF ATTACKS AGAINST IMAGE PREPROCESSING
We conduct experiments to investigate the robustness of different attacks against various image pre- processing methods during testing. This includes image compression with quality value of 50 (rangeing from I to 95), adjustments to brightness within the range of (-0.2, +0.2), ), and adjust- ments contrast within the range of (-0.2, +0.2). We evaluate the performance on FF++ test set. The results are presented in the Table A.I Firstly, we observe that image preprocessing algorithms could affect the performance of attacks. Secondly, these attacks are more sensitive to image com- pression compared to other image preprocessing algorithms. Thirdly, our attack still outperforms other attacks when using these image processing algorithms.
Image Preprocessing
original
Compression
Brightness
Contrast
Model
Attack
AUC
BD-AUC
AUC
BD-AUC
AUC
BD-AUC
AUC
BD-AUC
w/o allack
92.32
86.80
92.02
91.90
Badnet
92.47
48.47
86.69
49.32
92.04
48.90
91.88
48.95
Blended
91.76
68.13
86.41
56.96
91.53
67.72
91.38
66.18
ISSBA
92.60
51.07
86.71
50.33
92.34
51.03
92.17
50.76
SIG
91.65
61.18
85.87
59.78
60.53
91.26
60.73
LC
92.17
61.59
86.43
60.87
92.08
60.31
91.87
61.31
Ours
92.06
84.52
86.50
78.41
91.81
82.95
91.65
83.97
Table A.l: Evaluation of the robustness of backdoor attacks against different image preprocessing algorithms.
A.3 LIMITATIONS OF THE PROPOSED ATTACK
In this section, discussion the limitations of the proposed attack. Firstly, the proposed methods are not optimal. While our methods focus on the common transformations in face forgery and generally demonstrate effectiveness across different methods, they are not the optimal for specific methods. Secondly, the attack performance is partially dependent on the size of the kernel used for trigger generation. The choice of kernel size have impact the effectiveness of the attack.
A.4 IMPLEMENTATION THE GENERATOR
We provide the detailed network architecture of the generator G(2), in Figure A.I The generator is trained using the loss function Lg. as presented in Equation 10
1x1, cony, 512, stride-1, pacding BN
4x4, decony, 256, stride= p BN
4x4, decony, 256, stride packing BN
4x4, deconv, 256, stride=2, padding BN
4x4, deconv, 256, st padding BN
4x4, decony, 128, stride=2, padding=3, BN
4x4, decony, 3, stride=2, padding-3, BN
Figure A.1: The network architecture of the generator.
15

---

Published as a conference paper at ICLR 2024
A.5 ABLATION ON TRIGGER EMBEDDING
In this section, we conduct an ablation study to investigate the impact of the hyperparameter a. This involves varying the blending type (relative absolute) and the blending ratio (value of The results are presented in the Table A.2 In the table, the 'relative" line represents the method we used our paper, where a defined a line indicates the experiment we conducted as reference, where a is defined as a a. From the results, we can draw observations as follows: 1) As the value of increases, we can achieve higher attack performance but lower PSNR. This is because the tradeoff between the attack performance and the stealthiness the backdoor trigger. 2) Although absolute embedding yields better attack performance, this improvement comes at the cost of lower PSNR.
25/255. On the other hand, the 'absolute'
type
AUC
BD-AUC
PSNR
0.03
91.78
67.02
39.56
0.05
92.06
84.52
35.19
relative
0.07
91.63
86.81
32.30
0.1
91.31
92.61
29.22
absolute
0.05
91.50
92.16
29.71
Table A.2: Ablation the embedding type and embedding ratio.
A.6 ADDITIONAL COMPARISONS FREQUENCY-BASED BACKDOOR ATTACKS
In this section, we extend our comparisons to include frequency-basee baseline, FTrojan (Wang al. 2022a). The results are presented Table A.3
Our method outperforms FTrojan across
three detectors and three datasets.
Dataset (train + test)
FF++ + FF++
FF++ - CDF
FF+ - DFD
Type
Model
Attack
AUC
BD-AUC
AUC
BD-AUC
AUC
BD-AUC
Deepfake artifact detection
w/o attuck
85.10
77.84
76.85
Xception
FTrojan
84.56
94.51
76.19
96.25
78.04
92.80 95.89
Ours
85.18
99.65
77.21
99.13
78.26
w/o attack
92.32
93.10
90.35
SBI
FTrojan 92.40
65.54
93.61
87.38
89.97
59
Blending artifact
Ours
92.06
84.52
93.74
97.38
89.71
79.58
detection
w/o attack
78.90
85.38
83.30
Face X-ray
FTrojan 78.02
47.26
82.04
54.78 98.96
82.21
$6.03
Ours
77.70
79.82
81.74
83.52
98.55
Table A.3: Comparisons with frequency-based backdoor attack, FTrojan.
16


Reviews:
[{"review_id": "q1YBrZKwnF", "reviewer": "Reviewer_uGDS", "summary": "The paper proposes the Poisoned Forgery Face clean label backdoor attack framework utilizing a mask for the inner face region and a stealthy translation-sensitive trigger pattern, which is suitable for attacking facial forgery detectors. By poisoning a small portion of the clean training data, the attacker can flip the label of deep fake images or videos from fake to real. This approach can overcome the label conflict problem, which happens when the detectors are trained with self-blending techniques with only real data. Experimental results demonstrated a significant improvement in the attack success rate and the improvement of the trigger stealthiness.", "questions": "Please refer to the comments in the weaknesses section.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The paper addresses not only the traditional deepfake artifact detection, in which detectors are trained with deepfakes, but also the recent blending artifact detection, in which the detectors are trained with augmented self-blended real images. The proposed translation-sensitive trigger pattern is innovative and effective for dealing with the self-blending approach.\n \n+ The paper also provides a comprehensive benchmark for backdoor attacks, which is not available in the literature.", "weaknesses": "+ This paper is not the first in the literature to address backdoor attacks in face forgery detection. Cao et al. [A] have already investigated this problem. Therefore, the first contribution is invalid.\n \n+ The use of \"b\" in the equations in section 3 is confusing. b presents both \"blending\" and \"remaining clean.\"\n \n+ The benchmark should include some frequency-based backdoor attacks like [B].\n \n+ The reported performance of Face X-ray is lower than that in the original paper and in the SBI paper. I am wondering if there is something wrong with the experiments.\n \n+ The robustness of the proposed trigger patterns was not investigated. While sharing deepfake images or videos on social networks, these patterns may be destroyed by compression or some other image processing algorithms.\n \n+ The paper should include a paragraph of ethics statement\n \n+ The appendix section, which was referred from the main part, is missing.\n\nReferences:\n\n[A] Cao, Xiaoyu, and Neil Zhenqiang Gong. \"Understanding the security of deepfake detection.\" In International Conference on Digital Forensics and Cyber Crime, pp. 360-378. Cham: Springer International Publishing, 2021.\n\n[B] Wang, Tong, Yuan Yao, Feng Xu, Shengwei An, Hanghang Tong, and Ting Wang. \"An invisible black-box backdoor attack through frequency domain.\" In European Conference on Computer Vision, pp. 396-413. Cham: Springer Nature Switzerland, 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T1onHBswxV", "reviewer": "Reviewer_4qiC", "summary": "In this paper, a novel backdoor attack strategy is introduced for the FACE FORGERY DETECTION task. This approach deceives face forgery detectors by embedding specific triggers within the training data. The authors delve deeply into the internal structure of current face forgery detectors and propose a transformation-sensitive triggering mechanism to facilitate effective backdoor attacks. Extensive experiments validate the efficacy of the proposed method and highlight potential vulnerabilities in face forgery detection systems.", "questions": "Please refer to [Weakness].", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The paper is the first to uncover the vulnerabilities during the training phase of face forgery detection, introducing a practical clean-label attack technique.\n\n2. The backdoor attack strategy presented is notably innovative, leveraging the unique architectural characteristics of face forgery methods.\n\n3. The trigger designed in this study is highly covert, making it challenging for the human eye to detect.\n\n4. The authors provide a comprehensive explanation of the motivations behind the proposed method, and the manuscript is articulated clearly.", "weaknesses": "1. The paper could provide a discussion on the limitations of the proposed method, ideally placed in the appendix.\n2. The choice of different kernel sizes for the trigger can impact the attack performance on various forgery detection models. The rationale behind such an impact remains unclear. Additionally, the authors' decision to finalize a size of 5 for the kernel lacks a clear justification.\n3. Table 4 indicates that existing defense mechanisms fail against the proposed attack. It remains ambiguous whether this failure is specific to the attack introduced by the authors or if it's a general shortcoming for other attacks as well. Furthermore, it would be valuable if the authors could analyze existing attacks to suggest potential defense strategies.\n4. In the section on trigger generation, the authors should provide a comprehensive description of the generator's overall loss function and its basic architecture.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dRSxVii2Xh", "reviewer": "Reviewer_GcXY", "summary": "This paper aims to attack face deepfake detection models via backdoor attack, especially for blending artifact detection methods. To resolve the scalable problem, the paper builds a trigger generator to generate the trigger adaptively. To ensure the stealthiness of the trigger, a relative pixel-wise embedding ratio is incorporated in the generation of the poisoned sample. In general, the style of this paper is clear and logical. However, some experiments are missing and hence make readers confused about the effectiveness of some modules.", "questions": "The current rating is not my final decision, I hope the authors can explan the questions listed below in detail:\n\n1.In Eq. (2), as stated, the blending transformation needs two real samples, however, the variable in function T^b only has one sample instead of a pair of real samples. For example, in Face X-ray, the fake face image is generated via two different real ones. So I think the formulation should be adjusted to make it more reasonable.\n\n2.In the section on ‘Stealthiness of Backdoor Attacks’, Table 2 shows the qualitative results among different backdoor attack methods. However, the paper does not mention the number of samples for evaluation.\n\n3.The influence of the hyperparameter ‘a’ is ambiguous. Moreover, the comparison with/without ‘alpha’ is missing and hence makes readers confused about the effectiveness brought by the proposed relative embedding. It is better to show the qualitative results and the quantitative backdoor attack performance.\n\n4.This attack method is built on the generated image should show a relatively distinct facial boundary. However, in real scenarios, fake images are generated from various transformations. If some fake images are generated only via non-linear transformation, such as blur, this method may not be optimal. This may be the reason why the attack performance is lower than ‘LC’ because not all training samples contain linear translation transformation.\n\n5.Typo: In Resistance to Backdoor Defenses, ‘Efficient-b4’ should be ‘EfficientNet-b4’.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is the early exploration of backdoor attacks in face forgery detection and the description of the existing obstacles deployed trigger in face forgery detection is clear.\n2. Besides, the paper subtly simplifies the complex forgery problem as linear translation transformation and hence makes the backdoor attack in the translation-sensitive scenario.\n3. To make the trigger stealthy, the author further hid the generated noise by a simple weight parameter.", "weaknesses": "1.Some descriptions like formulation may be incorrect and may misunderstand the readers.\n2.Some ablation studies are missing and make readers confused about the effectiveness of the proposed modules. \n3.The method lacks generalization to some extent. The main gains derive from blending artifact detection. However, in the real scenario, not all evidence of determining forgery is based on blending traces in face forgery detection. Therefore, I would like to see the feedback about the questions listed below and decide to give a reasonable rating.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)