Paper: Poisoned Forgery Face: Towards Backdoor Attacks on Face Forgery Detection (ICLR-2024)

Reviewer comment (Theory):
In Eq. (2), as stated, the blending transformation needs two real samples, however, the variable in function T^b only has one sample instead of a pair of real samples. For example, in Face X-ray, the fake face image is generated via two different real ones. So I think the formulation should be adjusted to make it more reasonable.

Author response:
**Q1**: In Eq. (2), as stated, the blending transformation needs two real samples, however, the variable in function T^b only has one sample instead of a pair of real samples. For example, in Face X-ray, the fake face image is generated via two different real ones. So I think the formulation should be adjusted to make it more reasonable.

**A1**: Thanks for your suggestions. Our current formulation is based on the SBI method where the two real samples are from the same source, which can be considered as a special case. We appreciate your feedback and we have revised the formulation to a more general form that can represent different blending artifact methods, including SBI and Face X-ray, etc.

Reviewer comment (Experiments):
In the section on ‘Stealthiness of Backdoor Attacks’, Table 2 shows the qualitative results among different backdoor attack methods. However, the paper does not mention the number of samples for evaluation.

Author response:
**Q2**: In the section on ‘Stealthiness of Backdoor Attacks’, Table 2 shows the qualitative results among different backdoor attack methods. However, the paper does not mention the number of samples for evaluation.

**A2**: Thanks for your feedback. In the 'Stealthiness of Backdoor Attacks' section, we conducted a quantitative (PSNR and L_inf) evaluation using the fake subset of the FF++ dataset's test set. This subset consists of 560 fake videos, and we extracted 32 frames per video, resulting in a total of 17,920 samples. For the human perception studies, we randomly selected 5 images from the aforementioned subset and applied all 6 attacks, resulting in a total of 30 samples per participant. We have included these details in the revision.

Reviewer comment (Experiments):
The influence of the hyperparameter ‘a’ is ambiguous. Moreover, the comparison with/without ‘alpha’ is missing and hence makes readers confused about the effectiveness brought by the proposed relative embedding. It is better to show the qualitative results and the quantitative backdoor attack performance.

Author response:
**Q3**: The influence of the hyperparameter ‘a’ is ambiguous. Moreover, the comparison with/without ‘alpha’ is missing and hence makes readers confused about the effectiveness brought by the proposed relative embedding. It is better to show the qualitative results and the quantitative backdoor attack performance.

**A3**: Thanks for your feedback. We would like to address your concerns as follows:
1) The hyperparameter 'a' serves as a scaler to to determine the maximum magnitude of the backdoor trigger. A higher 'a' value results in a less stealthy trigger, while a lower 'a' value enhances stealthiness. In our experiment, we set 'a' to a low value (0.05) to prioritize stealthiness of the trigger.
2) The hyperparameter 'alpha' is defined as '**alpha**' = a * **x_k** / 255, where 'x_k' represents the pixel values of the image. When embedding the trigger into the image, we adjust the pixel values of the trigger based on the magnitude of the pixel values in the image. This adjustment further enhances the stealthiness of the trigger. 
3) We clarify that training without 'alpha' or setting 'alpha' to zero is equivalent to benign training without any backdoor attack. In our paper, we have presented the results of benign training in the 'w/o attack' line of Table 1. Furthermore, we are happy to follow your suggestions and conduct an extra ablation study to investigate the impact of the hyperparameter 'alpha'. This involves varying the blending type (relative or absolute) and the blending ratio (value of 'a') in the table mentioned. In the table, the 'relative' line represents the method we used in our paper, where 'alpha' is defined as '**alpha**' = a * **x_k** / 255. On the other hand, the 'absolute' line indicates the experiment we conducted as a reference, where 'alpha' is defined as '**alpha**' = a.
4) From the results, we can draw observations as follows: 
    1)  As the value of 'a' increases, we can achieve higher attack performance but lower PSNR. This is because **the tradeoff between the attack performance and the stealthiness of the backdoor trigger**. 
    2)  Although absolute embedding yields better attack performance, this improvement comes at the cost of lower PSNR. 

|   type   | 'a'  | AUC   | BD-AUC | PSNR  |
|:--------:|------|-------|--------|-------|
| relative | 0.03 | 91.78 | 67.02  | 39.56 |
| relative | 0.05 | 92.06 | 84.52  | 35.19 |
| relative | 0.07 | 91.63 | 86.81  | 32.30 |
| relative | 0.1  | 91.31 | 92.61  | 29.22 |
| absolute | 0.05 | 91.50 | 92.16  | 29.71 |

Reviewer comment (Evaluation):
This attack method is built on the generated image should show a relatively distinct facial boundary. However, in real scenarios, fake images are generated from various transformations. If some fake images are generated only via non-linear transformation, such as blur, this method may not be optimal. This may be the reason why the attack performance is lower than ‘LC’ because not all training samples contain linear translation transformation.

Author response:
**Q4**: This attack method is built on the generated image should show a relatively distinct facial boundary. However, in real scenarios, fake images are generated from various transformations. If some fake images are generated only via non-linear transformation, such as blur, this method may not be optimal. This may be the reason why the attack performance is lower than ‘LC’ because not all training samples contain linear translation transformation.

**A4**: Thanks for your feedback. We acknowledge that our methods are not optimal in all cases and we would like to offer the following insights: 

1) In the case of blending artifact methods, where fake images are created by blending two real faces during training, our attack is built upon the idea that the effectiveness of a backdoor attack relies on maximizing the discrepancy between the trigger in the real face and the transformed trigger in the fake face. The generation of fake faces involves various transformations, and different methods employ different compositions of these transformations. In our approach, we target a transformation that causes distinct facial boundaries, as it is prevalent across different blending artifact methods. Thereby, our attack demonstrates transferability across different methods. In the case where fake images are generated only through non-linear transformations, such as blur, our method may not be optimal. However, the underlying idea can be extended in such scenarios. An ideal approach would be to select the representative non-linear transformation as the target and optimize a trigger that maximizes the difference between the original version and the transformed version of the trigger. In addition, a more general trigger optimization that can adapt to more face forgery detection methods is a future research direction and we will take into consideration. 
2) The potential reason for the lower performance compared to 'LC' in Xception could be attributed to the higher stealthiness achieved by our method at the cost of sacrificing attack performance. The focus on enhancing stealthiness may result in a trade-off with the overall attack performance. Here, we conduct additional experiments where we enhance the trigger embedding ratio, which refers to the magnitude of the trigger. From the results in the table below, as we increase the embedding ratio, our method outperforms LC while remaining PSNR higher than LC.

|        |                 |  FF++ |  FF++  |  CDF  |   CDF  |  DFD  |   DFD  |       |
|:------:|:---------------:|:-----:|:------:|:-----:|:------:|:-----:|:------:|:-----:|
| attack | embedding ratio |  AUC  | BD-AUC |  AUC  | BD-AUC |  AUC  | BD-AUC |  PSNR |
|   LC   |        -        | 84.25 |  99.97 | 75.29 |  99.58 | 77.99 |  99.36 | 20.51 |
|  Ours  |       0.05      | 85.18 |  99.65 | 77.21 |  99.13 | 78.26 |  95.89 | 35.19 |
|  Ours  |       0.15      | 85.16 |  99.98 | 76.72 |  99.81 | 77.27 |  99.39 | 26.86 |

Reviewer comment (Presentation):
The paper could provide a discussion on the limitations of the proposed method, ideally placed in the appendix.

Author response:
Thanks for your suggestions. We would like to discuss the limitations of the proposed method as follows: 

1) The proposed methods are not optimal. While our methods focus on the common transformations in face forgery and generally demonstrate effectiveness across different methods, they are not the optimal for specific methods.
2) The attack performance is partially dependent on the size of the kernel used for trigger generation. The choice of kernel size can have an impact on the effectiveness of the attack.

Finally, we have included this discussion to the appendix.

Reviewer comment (Experiments):
The choice of different kernel sizes for the trigger can impact the attack performance on various forgery detection models. The rationale behind such an impact remains unclear. Additionally, the authors' decision to finalize a size of 5 for the kernel lacks a clear justification.

Author response:
Thanks for your suggestions. We would address your concern as follows:
1) As mentioned in Section 5.3 of the paper, the choice of kernel size indicates the range of translation that we aim to optimize. This range is based on the attacker's assumption on the translation amplitude during the synthesis of fake data. A larger kernel size suggests that the triggers are optimized over a broader translation range. However, a larger kernel size may not always lead to better performance. This is because the triggers could be optimized to adapt to a larger range of translations, which may not be the best match for the translation amplitude during the synthesis of fake data.  
2) Regarding the decision to use a relatively small kernel size of 5, this is based on the consideration that the distinction in facial boundaries of the synthesized fake images can be really small, in order to make the synthesized fake images appear more natural and less suspicious to the human eye.

Reviewer comment (Reproducibility):
In the section on trigger generation, the authors should provide a comprehensive description of the generator's overall loss function and its basic architecture.

Author response:
Thank you for your suggestion. We would like to provide a more detailed description of the generator's basic architecture and the overall loss function:

1) **Basic Architecture** of the Generator (G):

- 1x1, conv, 512, stride=1, padding=3, BN
- 4x4, deconv, 256, stride=2, padding=3, BN
- 4x4, deconv, 256, stride=2, padding=3, BN
- 4x4, deconv, 256, stride=2, padding=3, BN
- 4x4, deconv, 256, stride=2, padding=3, BN
- 4x4, deconv, 128, stride=2, padding=3, BN
- 4x4, deconv, 3, stride=2, padding=3, BN

In the above description, "KxK, conv/deconv, C, stride=S, padding=P, BN" indicates a convolutional or deconvolutional layer with a KxK kernel, C output filters, stride S, and padding P, followed by batch normalization (BN).

2) **Overall Loss Function** for the Generator: The overall loss function for the generator G(z) can be found in Equation 10 of the paper. It is represented as L_g = − log ||K(v)⊗G(z)||_1. Here, K(v) denotes our proposed convolutional kernel as described in Equation 8, and G(z) represents the output of the generator with a latent variable z ~ N(0,1) as input. We train the generator G(z) with L_g as the loss function.

Reviewer comment (Novelty):
This paper is not the first in the literature to address backdoor attacks in face forgery detection. Cao et al. [A] have already investigated this problem. Therefore, the first contribution is invalid.

Author response:
**A1**: Thank you for your correction. We have thoroughly reviewed the mentioned paper by Cao et al. [A] and acknowledge its contribution to backdoor attacks in face forgery detection. We apologize for any confusion caused by our previous statement regarding the "first" contribution. Additionally, we would like to highlight the distinctions and emphasis of our paper in comparison to the mentioned work:
1) **Scope of investigation**: While the mentioned paper focuses on investigating a single backdoor attack, specifically Badnet, in face forgery detection, our paper goes beyond and provides a more comprehensive analysis. We benchmark five different backdoor attacks and propose a novel attack method in this field.
2) **Extension to blending artifact detection methods**: Our research also extends to the examination of backdoor attacks on blending artifact detection methods, specifically SBI and Face X-ray. These methods are more challenging to attack compared to the previously studied detection methods.

Finally, we have made the necessary modifications and added appropriate citations in the revision.

Reviewer comment (Writing):
The use of "b" in the equations in section 3 is confusing. b presents both "blending" and "remaining clean."

Author response:
**A2**: Thank you for pointing out the issue. In the revision, we clarify and differentiate the symbols used to represent "blending" and "remaining clean". Specifically, we use "b" to represent "blending" and introduce 'c' to represent "remaining clean".

Reviewer comment (Experiments):
The benchmark should include some frequency-based backdoor attacks like [B].

Author response:
**A3**: We appreciate your suggestion and conduct experiments that involve the mentioned frequency-based attack, FTrojan [B]. The results are presented in the following table. Upon analysis, our attack outperforms the frequency-based attack, FTrojan [B], on three face forgery detection models. 

| **Dataset** |    **→**   |   FF++  |    FF++    |   CDF   |     CDF    |   DFD   |     DFD    |
|:-----------:|:----------:|:-------:|:----------:|:-------:|:----------:|:-------:|:----------:|
|  **Model**  | **Attack** | **AUC** | **BD-AUC** | **AUC** | **BD-AUC** | **AUC** | **BD-AUC** |
|   Xception  |   FTrojan  |  84.56  |    94.51   |  76.19  |    96.25   |  78.04  |    92.80   |
|   Xception  |  **Ours**  |  85.18  |  **99.65** |  77.21  |  **99.13** |  78.26  |  **95.89** |
|     SBI     |   FTrojan  |  92.40  |    65.54   |  93.61  |    87.38   |  89.97  |    59.45   |
|     SBI     |  **Ours**  |  92.06  |  **84.52** |  93.74  |  **97.38** |  89.71  |  **79.58** |
|  Face X-ray |   FTrojan  |  78.02  |   47.26    |  82.04  |    54.78   |  82.21  |    56.03   |
|  Face X-ray |  **Ours**  |  77.70  |  **79.82** |  81.74  |  **98.96** |  83.52  |  **98.55** |

Reviewer comment (Experiments):
The reported performance of Face X-ray is lower than that in the original paper and in the SBI paper. I am wondering if there is something wrong with the experiments.

Author response:
**A4**: Thank you for bringing these points to our attention. The performance difference on clean samples can be attributed to two main factors: 
1) **Data version**: The FF++ datasets have multiple versions, including the raw (original) version and compressed versions such as c23(HQ) and c40(LQ). As mentioned in Section 4.4 of the SBI paper [1], the original performance of both Face X-ray and SBI models was evaluated on the raw version of the data. But in our experiments, we used the compressed (c23) version of the data for both training and testing. It is known that the performance on clean compressed data may experience a drop [2]. On the other hand, the CDF dataset has only one version, and our reported performance on SBI is 93.10, which is close to the results reported in the original paper (93.18) and the official GitHub repository (92.87%). 
2) **Unified face extraction process**: To avoid possible bias in the data preprocessing, we employed a unified face extraction process proposed by SBI, which includes consistent parameters such as the number of frames per video and the area of the extracted faces, for all three face forgery detection methods. This standardized approach may lead to performance differences compared to original paper.
	
In addition, we conducted evaluations on the **raw version** of FF++ dataset. The results are presented in the following table.
1) We evaluate the clean performance on the raw data, which aligns with the evaluation approach used in the original papers. The 'original w/o attack' line in the table represents the results reported in the original paper. The results indicate that our performance on clean raw data is close to the original reported results.  
2) We also evaluate the attack performance on the raw data. We compare the existing attack with the best performance with our method on the raw data. We observe that the attack performance, as indicated by BD-AUC, is higher on the raw data compared to the compressed (c23) data. According to these results, **our method still demonstrates superior performance compared to the existing attack on the raw data**.

|    Model   |        Attack       | Version |  AUC  |   BD-AUC  |
|:----------:|:-------------------:|:-------:|:-----:|:---------:|
|     SBI    | original w/o attack |   raw   | 99.64 |     -     |
|     SBI    |       Blended       |   raw   | 99.38 |   78.34   |
|     SBI    |         Ours        |   raw   | 99.46 | **97.76** |
|     SBI    |       Blended       |   c23   | 91.76 |   68.13   |
|     SBI    |         Ours        |   c23   | 92.06 | **84.52** |
| Face X-ray | original w/o attack |   raw   | 98.52 |     -     |
| Face X-ray |       Blended       |   raw   | 93.85 |   84.57   |
| Face X-ray |         Ours        |   raw   | 94.03 | **94.42** |
| Face X-ray |       Blended       |   c23   | 75.02 |   72.10   |
| Face X-ray |         Ours        |   c23   | 77.70 | **79.82** |

Reviewer comment (Experiments):
The robustness of the proposed trigger patterns was not investigated. While sharing deepfake images or videos on social networks, these patterns may be destroyed by compression or some other image processing algorithms.

Author response:
**A5**: Thank you for your suggestion. We conduct additional experiments to investigate the robustness of different attacks against various image preprocessing methods during testing. This includes image compression, adjustments to brightness, and adjustments to contrast. We evaluate the performance on SBI on the FF++ test set. The results are presented in the following table.
1) We observe that image preprocessing algorithms could affect the performance of attacks. 
2) These attacks are more sensitive to image compression compared to other image preprocessing algorithms.
3) Our attack still outperforms other attacks when using these image processing algorithms. 

|            | original |  original  | Compression | Compression | Brightness | Brightness | Contrast |  Contrast  |
|:----------:|:--------:|:----------:|:-----------:|:-----------:|:----------:|:----------:|:--------:|:----------:|
| **Attack** | **AUC**  | **BD-AUC** | **AUC**     | **BD-AUC**  | **AUC**    | **BD-AUC** | **AUC**  | **BD-AUC** |
|      -     |   92.32  |      -     |    86.80    |      -      |    92.02   |      -     |   91.90  |      -     |
|   Badnet   |   92.47  |    48.47   |    86.69    |    49.32    |    92.04   |    48.90   |   91.88  |    48.95   |
|   Blended  |   91.76  |    68.13   |    86.41    |    56.96    |    91.53   |    67.72   |   91.38  |    66.18   |
|    ISSBA   |   92.60  |    51.07   |    86.71    |    50.33    |    92.34   |    51.03   |   92.17  |    50.76   |
|     SIG    |   91.65  |    61.18   |    85.87    |    59.78    |    91.10   |    60.53   |   91.26  |    60.73   |
|     LC     |   92.17  |    61.59   |    86.43    |    60.87    |    92.08   |    60.31   |   91.87  |    61.31   |
|    Ours    |   92.06  |  **84.52** |    86.50    |  **78.41**  |    91.81   |  **82.95** |   91.65  |  **83.97** |

Reviewer comment (Presentation):
The appendix section, which was referred from the main part, is missing.

Author response:
**A7**: We apologize for any confusion. The appendix section can be found in the attached **supplemental materials (.pdf)** in the openreview website or **in our revision**.