Grond: A Stealthy Backdoor Attack in Model Parameter Space

Authors: Xiaoyun Xu, Zhuoran Liu, Stefanos Koffas, Stjepan Picek
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: backdoor attack, backdoor defense

Abstract:
Recent research on backdoor attacks mainly focuses on invisible triggers in input space and inseparable backdoor representations in feature space
to increase the backdoor stealthiness against defenses.
We examine common backdoor attack practices that look at input-space or feature-space stealthiness and show that state-of-the-art stealthy input-space and feature-space backdoor attacks can be easily spotted by examining the parameter space of the backdoored model. 
Leveraging our observations on the behavior of the defenses in the parameter space, we propose a novel clean-label backdoor attack called Grond. 
We present extensive experiments showing that Grond outperforms state-of-the-art backdoor attacks on CIFAR-10, GTSRB, and a subset of ImageNet. 
Our attack limits the parameter changes through Adversarial Backdoor Injection, adaptively increasing the parameter-space stealthiness.
Finally, we show how combining Grond's Adversarial Backdoor Injection with commonly used attacks can consistently improve their effectiveness.
Our code is available at \url{https://anonymous.4open.science/r/grond-557F}.

Decision: reject
Reviewer scores: 3, 3, 3, 5

Reviews and discussion:
<thread>
creation date:
1730296037396

thread_type:
official_review

<thread_object>
creator: reviewer_q4xe

<content>
summary:
In this paper, the researchers propose a new backdoor attack scheme to combat existing defense strategies based on model repairing. The core idea of this scheme is very simple and easy to understand. Specifically, this scheme first generates a trigger using TUAP, and then uses this trigger to poison the model. During the process of implanting the backdoor, it modifies parameters with higher activation values, thereby enhancing the stealth of the backdoor attack in the parameter space. Additionally, experimental results demonstrate the effectiveness of this scheme. However, both trigger generation and adversarial backdoor injection are based on existing works, so the innovation of this research is limited.

strengths:
The experimental results of this approach are convincing. The results indicate that the approach can bypass existing defense mechanisms while maintaining a high attack success rate.

weaknesses:
This work lacks novelty because two key steps—trigger generation and adversarial backdoor injection—are based on existing studies [1], [2].

questions:
1. The experimental section should clearly specify the number of clean samples used in the pruning-based and fine-tuning-based approaches, which is not clearly stated in the paper. 
2. The authors list some ImageNet200 backdoor samples; however, despite the authors claiming that this backdoor attack is stealthy in input space, feature space, and parameter space, a close examination of these backdoor samples reveals obvious perturbations. The authors should evaluate the quality of the backdoor images, especially in comparison with some imperceptible backdoor attacks, such as [3].
3. It would be more convincing if the authors could provide Grad-CAM images of the backdoor samples and visualise their distribution in feature space.


[1] Moosavi-Dezfooli, Seyed-Mohsen, et al. "Universal adversarial perturbations." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
[2] Zheng, Runkai, et al. "Data-free backdoor removal based on channel lipschitzness." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.
[3] Doan, Khoa, et al. "Lira: Learnable, imperceptible and robust backdoor attacks." Proceedings of the IEEE/CVF international conference on computer vision. 2021.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730348188299

thread_type:
official_review

<thread_object>
creator: reviewer_xjyn

<content>
summary:
This paper presents Grond, a backdoor attack that achieves enhanced stealth across input, feature, and parameter spaces to avoid detection. Through Adversarial Backdoor Injection, Grond disperses backdoor effects across multiple neurons, making it harder to identify using parameter-space defenses. Extensive experiments on datasets like CIFAR-10, GTSRB, and ImageNet200 show that Grond outperforms other attacks in evading both pruning and fine-tuning-based defenses, highlighting its robustness and adaptability.

strengths:
This paper introduces Grond, a novel backdoor attack with comprehensive stealth across input, feature, and parameter spaces. Using Adversarial Backdoor Injection, Grond disperses backdoor effects across neurons, enhancing its stealth and evading parameter-space defenses. Extensive testing on multiple datasets (CIFAR-10, GTSRB, ImageNet200) and defenses demonstrates Grond's effectiveness and adaptability across diverse scenarios and model architectures.

weaknesses:
1. Limited Threat Model in Terms of Defender Capabilities: The paper's threat model lacks a thorough consideration of the defender’s capabilities, particularly regarding proactive measures they could take to identify and mitigate backdoors prior to deployment. This omission may limit the applicability of the model to real-world scenarios where defenders could leverage more advanced tools and strategies.
2. Lack of Comparison with Stealthy Clean-Label Backdoor Attacks: The paper does not include a comparison with other existing stealthy clean-label backdoor attacks, such as Hidden Trigger Backdoor Attacks (HTBA).
3. Limited Range of Defense Methods Evaluated: The paper tests Grond against a small selection of defense methods, primarily focusing on pruning and fine-tuning-based defenses.

questions:
Can this type of backdoor attack be detected by sample-based detection defenses?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730672970239

thread_type:
official_review

<thread_object>
creator: reviewer_somh

<content>
summary:
The paper proposes a new clean-label backdoor attack that achieves stealthiness in both the input space and the parameter space. Specifically, to achieve the stealthiness in the input space, the paper utilizes the targeted universal adversarial perturbation as the backdoor trigger. For the parameter-space stealthiness, the paper restricts the magnitude of model weight parameters by setting particularly large weights to the mean value of the corresponding layer. The evaluation is conducted on three standard benchmarks. Comparing to existing backdoor attacks, the proposed attack is more resilient to existing defense and detection methods.

strengths:
1. The studied topic is important as backdoor attacks can exploit the integrity of deployed deep learning models and cause unexpected consequences.
2. The paper is overall easy to understand.

weaknesses:
1. The proposed attack utilizes the targeted universal adversarial perturbation as the backdoor trigger, which is the same as an existing work [1]. To increase the stealthiness of the attack in the parameter space, the paper uses backdoor defenses to help reduce backdoor-related neurons. This technique has already been proposed in the literature [2]. The proposed attack is just a collection of existing techniques. The novelty is very limited.
2. The paper assumes that the attacker has white-box access to the training processes, meaning that the attacker has whole control over the training. And yet, the paper chooses a clean-label attack, which is quite strange. The introduction of clean-label attacks is to simulate the scenario where adversaries have no control over the labeling and training procedures. The attacker can only modify a subset of the training images, making it a realistic threat model. Since this paper assumes white-box access to the training processes, there is no need to use the clean-label setting. Can the authors explain why such a setting is needed for a successful attack?
3. According to Figure 2, the mask loss for the proposed backdoor attack is much lower than benign cases. Cannot one design a defense method by measuring the outliers of the mask loss? Clearly the mask loss for the proposed attack is much smaller, which can be easily detected.
4. Following the above point, there is no evaluation on adaptive defenses, where the defender has the knowledge of the proposed attack. Since the proposed attack uses a small-size trigger with epsilon equal to 8, simple defenses can be existing adversarial detection methods and/or (universal) adversarial training. Another defense approach could be randomly perturbing the weight parameters. As the proposed attack reduces backdoor weights, the backdoor effect may be quite brittle when weights are perturbed.



[1] Zeng, Yi, et al. "Narcissus: A practical clean-label backdoor attack with limited information." Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. 2023.

[2] Cheng, Siyuan, et al. "Deep feature space trojan attack of neural networks by controlled detoxification." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 2. 2021.

questions:
Please see above comments.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730716191485

thread_type:
official_review

<thread_object>
creator: reviewer_tltg

<content>
summary:
The authors propose a backdoor attack against image classification models that improves robustness and stealthiness. Current backdoor attacks are "easily spotted by examining the parameter space". The authors propose an Adversarial Backdoor Injection method that prunes weights of the backdoored network after each training epoch whenever they deviate too strongly from the mean weight within each layer. The authors evaluate their attack on relatively small-scale image datasets, such as CIFAR-10 and a 200-class subset of ImageNet, which includes nine backdoor removal and seven backdoor detection methods. The results show their attack is robust and undetectable against all surveyed defences.

strengths:
- Effectiveness: The paper's results are promising and show improvement over other attacks in all dimensions. 
- Ablation studies: The authors conducted extensive experiments across multiple datasets (CIFAR-10, GTSRB, ImageNet200) and architectures to analyse their attack’s effectiveness. 
- Presentation: The methodology and results are presented clearly, making the paper easy to follow.

weaknesses:
- Lack of Novelty: The approach of pruning weights to enhance stealth is not particularly original and provides only limited new insights into defending against these types of attacks. This limits the novelty of the proposed method.

- Assumption of a Strong Attacker: The paper assumes a white-box threat model with complete control over the training process. This setting, also known as a ‘supply chain attack’ [A], is extremely challenging (hopeless?) to defend against. It may not represent more realistic attacks or limited-access scenarios. This was stated in previous works [A] and others even show that provably undetectable backdoors can be implanted into models in this setting [B]. 

- Lack of theoretical insights: There are no clear reasons why Grond should perform better than existing attacks, and the authors do not provide insights on the 'why' question. 

- Lack of adaptive defenders: From a security perspective, it appears that Grond does not evaluate an adaptive defender who knows the attack strategy used by Grond. For instance, pruning weights in the way the authors proposed could make the attack detectable. 

-------
[A] Hong, Sanghyun, Nicholas Carlini, and Alexey Kurakin. "Handcrafted backdoors in deep neural networks." Advances in Neural Information Processing Systems 35 (2022): 8068-8080.

[B] Goldwasser, Shafi, et al. "Planting undetectable backdoors in machine learning models." 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS). IEEE, 2022.

questions:
- Why does Grond work better than any other method?

- Why is the white-box setting significant?

- Do adaptive defenders exist that can detect or remove Grond?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732284036039

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
Dear reviewers and ACs,

Thank you for your careful consideration. 
We are glad to see that all reviewers are happy with the presentation and soundness of our work, where they see that our experiments are extensive (reviewers tLTG, xjYn) and convincing (reviewer Q4Xe).
We are also happy to see that all of our contributions are recognized by reviewers. In particular, the topic of (parameter-space) backdoor defense is important (reviewer SomH), and our attack is effective against model-space mitigation (reviewer tLTG, xjYn, Q4Xe).
However, we want to emphasize the most important contribution of this work is that current types of backdoor attacks can be mitigated by parameter-space defenses, as this observation is important to future research in the backdoor community.
We will also point out factual errors from reviews about the threat model of this work, the challenges of supply-chain attacks, and adaptive defenses. 
Taking the suggestions of the reviewers, new experiments on state-of-the-art supply-chain backdoor attacks are also provided.

We would like to clarify the main contribution and novelty of this paper.
For the first time, we systematically show that current backdoor attacks, including different types of state-of-the-art backdoor attacks, are vulnerable to parameter-space backdoor defenses.
This observation is new and important for future backdoor attack and defense research, and it also has a substantial influence on real-world backdoor mitigation, indicating that parameter-space defenses should get more attention from both academia and industry. 

Below, we will clarify reviewers' common concerns about the threat model, supply-chain attacks, and adaptive defenses.

**Generalization of our threat model.**
We agree with reviewers that in our threat model, adversaries can control the training of the backdoored model, which is a strong assumption and similar to supply-chain backdoor attacks.  
However, we disagree that this threat model is too limited to be studied.
In particular, our threat model follows the common practice in established backdoor literature [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], where the backdoored model is delivered as a product to the victim. 
Our analysis further advances the practical application and understanding of the related research.

**Supply-chain attacks.** 
We thank reviewers for pointing out the supply-chain backdoor attacks that were not considered in the draft and have the same threat model as Grond.
However, we disagree that supply-chain attacks are extremely challenging to defend against. 
In particular, we ran additional experiments with recent supply-chain backdoor attacks, showing that they are vulnerable to parameter-space defenses in the following table,


|Attack| BA (No def.) | ASR (No def.) | BA (CLP) | ASR (CLP) | BA (FT-SAM) | ASR (FT-SAM) |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| DFST [7] | 95.23 | 100 | 92.43 | 3.53 | 94.70 | 0.00 |
| DFBA [6] | 88.99 | 100 | 88.96 | 9.57 | 86.03 | 5.24 |
| SSDT [3] | 93.70 | 90.30 | 93.66 | 1.20 | 93.15 | 0.60 |

where DFBA [6] is the state-of-the-art supply-chain attack among all 13 attacks we looked at. 
In addition, DFBA is a better alternative to the Handcrafted backdoor [5] (mentioned by the reviewer tLTG), as DFBA was published in the last few weeks and directly compared with the Handcrafted backdoor and shows better performance in their paper.
Therefore, we used DFBA rather than the Handcrafted backdoor [5] in our experiment.
We will add a section discussing supply-chain attacks regarding the threat model, vulnerability to parameter-space defenses, and how Grond could possibly improve supply-chain backdoors.


**Adaptive defense.**
The adaptive defense refers to the defender knowing the design of the attack, which has not been extensively studied in backdoor research.
Our TAC pruning experiment (i.e., oracle analysis) follows a threat model in which the adversary even knows the backdoor trigger of Grond rather than just the method design. 
So, our TAC analysis provides a stronger defense than regular adaptive backdoor defense design that follows adversarial example research [14, 15].
Our analysis in Section 4.5 and Figure 4 shows that Grond is much more robust to adaptive defenses than other backdoor attacks, but we believe that future white-box adaptive defenses may mitigate Grond.


We hope that our clarification can address your concerns. We look forward to hearing from you and remain at your disposal should you have any comments/suggestions.

Best regards,

Authors of Grond
</content>
</thread_object>

</thread>
<thread>
creation date:
1733311813315

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
# Final Summary 

We thank the reviewers for their efforts.
We also want to re-stress several points that we have different opinions from the reviewers.

## Threat model and proactive defense.

Backdoor attacks with white-box access to the training process or the capability to directly modify the models' weights are generally accepted assumptions in backdoor research, especially regarding supply-chain attacks.  
Backdoor models provided as a service is reasonable due to the high cost of training models from scratch.
We include a comparison with the latest supply-chain attacks (SSDT, S&P 2024) in the submission. In the rebuttal, we include two more (DFST, AAAI 2021, and DFBA, NeurIPS 2024).

In addition, as reviewer xjYn suggested, we also include a proactive sample-level detection [16], where the defender directly has access to the training process.
As shown in the following table, [16] is ineffective against Grond when the poisoning rate is lower than 5\%, with a high false positive rate and low recall. 

| Attack | ACC| ASR | Recall | FPR |
|----|----|----|----|----|
|BadNets(pr=5\%)| 93.18 | 99.96 | 2500/2500 | 1568/47500 |
| Grond(pr=5\%) | 93.84 | 99.41 | 2499/2500 | 671/47500 |
|Grond(pr=2.5\%)| 93.81 | 95.83 | 115/1250 | 7220/48750 |
|Grond(pr=1\%) | 94.09 | 92.48 | 208/500 | 6690/49500 |
|Grond(pr=0.5\%)|94.36 | 92.91 | 90/250 | 6738/49750|
|Grond(pr=0.3\%)|94.22 | 90.10 | 29/150 | 6349/49850|


## The main contribution
Again, we want to emphasize that our motivation is to raise attention to using more proper evaluation baselines for backdoor attacks. Current evaluations of SOTA backdoor attacks are mainly based on input-space and feature-space defenses.
Our experiments showed that all evaluated attacks failed against at least four types of parameter-space defenses.

## Adaptive analysis
Our TAC oracle analysis is stronger than adaptive analysis because the TAC pruning directly uses the backdoor trigger to defend against attacks. 
In this setting, the defender has white-box access to any possible information from the attacker.
We provided a stronger adaptive analysis based on access to the backdoor trigger, which we believe provides solid evidence justifying our contributions. 

Authors of Grond
</content>
</thread_object>

</thread>


Metareview: