Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a method to inject backdoors into real-life deployments of
deep neural networks on hardware. More specifically, it proposes a constrained
fine-tuning process to inject backdoors by locating vulnerable weights and
updating these vulnerable weights in hardware. Experiment results on
hardware-deployed models show the effectiveness of the proposed method.
main_review: * Design and correctness

  The design of this attack is mostly based on heuristics without theoretical
  supports or analysis, e.g., how it identifies vulnerable weights. Its
  evaluation results are on two datasets, CIFAR and ImageNet, which I am not
  convinced to be enough to demonstrate how it can generalize to others.
  Moreover, its experiment design lacks an ablation study.

* Usage of clean data.

  The proposed constrained fine-tuning process requires clean data to inject
  backdoors. However, in Threat Model Section (i.e., Section 3.1), such
  assumption is not mentioned. Moreover, it seems to be contradictory with the
  claim that it does need training dataset.

  Also, details of the used clean data in the injection process are missing. For
  example, the number of clean samples used in the constrained fine-tuning
  process.

* FGSM

  Existing work has studied methods to generate/reverse-engineer triggers. I do
  not understand the logic of using FGSM instead. Are there any special
  benefits, compared with existing work? Existing work [1] also simultaneously
  optimizes the trigger patterns and model parameters.

  Also, some details of the trigger generation process is missing, e.g., the
  size of the trigger and the perturbation budget of FGSM. 

* Locating vulnerable weights

  For locating vulnerable weights, this paper relaxes the combinatorial
  optimization problem and heuristically locate these weights by simply ranking
  the parameters' gradient. Are there any evidence or theoretical analysis to
  support this design?

* Hyper-parameters

  This method has several hyper-parameters, for example, the loss balancing
  parameter \(\alpha\) in Equation 1, the size of the generated trigger, and the
  perturbation budget for FGSM.

* Countermeasures

  Potential defenses are only discussed but not validated.

* Ethics

  The paper does not have an ethical statement, despite it is proposing an
  attack.

* Relevance

  To me, this paper fits security conferences better. I do not see the
  significance of this appearing in a AI/ML venue.

[1] Pang, Ren, et al. "A tale of evil twins: Adversarial inputs versus poisoned
models." Proceedings of the 2020 ACM SIGSAC Conference on Computer and
Communications Security. 2020.
summary_of_the_review: To me, its contribution to AI/ML is not significant. Many designs are just
heuristics. It is more like a security engineering paper whose study object is
DNN. Also, it falls short in evaluation.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to integrate physical viability inside the attack objective for backdoor attacks. Specifically, the authors observe a hardware limitation for the row hammer attack, which limits the possible locations where the bits can be flipped. Consequently, they change the attack objective to target sparse locations to ensure that RHA can be used.
main_review: Strengths:
- The paper is well-written and easy to follow.
- The targeted problem is interesting since not many prior work focus on the implementation of the attack using RHA inside the DRAM.
- The contributions of the paper are mainly in steps 2 and 4 of the algorithm where the authors enforce block-sparse flipping and only 1 bit change in a byte. While these objectives are not directly optimized for and are merely enforced by heuristics, they are still of interest. 

Weaknesses and questions:
- The method in [1] also emphasizes on implementation of the attack and proposes different techniques to ensure the attack can be realized in the DRAM. Can the authors please provide a 1-1 comparison with this work and explain how they improve/change compared to this prior work?
- Since the appearance of bitflip attacks, many papers have focused on defense strategies [2-6]. Therefore, it is important to evaluate the proposed attack in face of existing defenses, preferably bypassing them successfully and/or better than prior attacks. While section 5.4 briefly goes over one defense, it merely shows that the evaluated defense inherently has a high false positive rate. However, this is a downside (corner case) for the defense strategy and not something that is specific to the proposed attack. Without proof of resilience against contemporary defenses, the attack cannot be impactful, even though it is realizable physically, thereby defeating the purpose.
- The threat model is a bit misleading, specifically since it claims no access to training data while the optimization steps require data for trigger generation, finding the weight bits, and adversarial fine-tuning. Can the authors please specify the source and amount of the data used for these steps?

[1] Yao, Fan, et al. "Deephammer: Depleting the intelligence of deep neural networks through targeted chain of bit flips." 29th {USENIX} Security Symposium ({USENIX} Security 20). 2020.
[2] Z. He et al., “Defending and harnessing the bit-flip based adversarial weight attack”. CVPR 2020. 
[3] J. Li et al., “Defending bit-flip attack through DNN weight reconstruction”. DAC 2020
[4] Y. Li et al., “DeepDyve: Dynamic verification for deep neural networks”. ACM CCS 2020.
[5] Q. Liu et al., “Concurrent weight encoding-based detection for bit-flip attack on neural network accelerators”. ICCAD 2020.
[6] J. Li et al., “Radar: Run-time adversarial weight attack detection and accuracy recovery”. DATE 2021
summary_of_the_review: Please see the main review.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a method for realizing backdoor attacks for DNNs based on the Rowhammer attack. 
main_review: Comments
1.	The threat model described in Section 3.1 is not compliant with the claims of the work. In particular: “we assume that the attacker… does not have access to the training hyperparameters, or the training dataset”. However, it is written several times that the training process has been modified by the attacker. How is that possible without access to the training hyperparameters and training dataset? It is also in contrast to the fact that in Section 3.2.2, the training samples are given as input to the optimization algorithm.
2.	The experiments performed in Section 3.2.1 constitute the main motivations to design the proposed attack. However, the experimental setup used for Rowhammer profiling is not clear. This should be discussed more comprehensively.
3.	It is not clear how the property of no co-occurrence in the same memory page among the flipped bits can be guaranteed. Does it assume the knowledge of the exact weight memory allocation?
4.	Authors are encouraged to discuss the possible countermeasures that can be potentially adopted to defend against their proposed attack.


Related work suggestions
1.	V. Venceslai, A. Marchisio, I. Alouani, M. Martina and M. Shafique, "NeuroAttack: Undermining Spiking Neural Networks Security through Externally Triggered Bit-Flips," 2020 International Joint Conference on Neural Networks (IJCNN), 2020, pp. 1-8, doi: 10.1109/IJCNN48605.2020.9207351.

summary_of_the_review: This work is promising and contributes to generating successful backdoor attacks in practical settings with good results. However, there are some concerns that affect the clarity of the method and of the threat model.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper explores backdoor injection attacks on machine learning models, the attack develops in the following way:

1. The model and its parameters are studied offline to determine a trigger pattern and the bits that must be modified in the target model.
2. A RowHammer attack is used to modify the bits of the model after it is loaded into DRAM
3. When inputs are received which include a trigger, the model will misclassify the input. In the case of "clean" inputs, the aim is to leave the performance unchanged.

* unlike previous work vulnerable bits are sought throughout the model (not just the last layer)
* care is taken to understand the limitations of the RowHammer attack, i.e. in reality only a small number of specific bits can be flipped 
* a new technique is used to jointly find weight modifications and the data trigger pattern (CFT with BR)

main_review: 
The work is interesting and closes the gap between theoretical backdoor attacks of this type and realisable ones. Results show that attacks are possible where 100% (Rmatch) of the required bit flips are flippable using a RowHammer attack. Nevertheless, it appears that  access to the model weights at runtime is required to perform the attack - sections 3.3 and 4.5 perhaps need strengthening to convince the reader that the threat here is real.

1. what does the initial trigger mask look like? what is the size of the trigger?

2. in terms of countermeasures, it appears to be the case that access is required to the weights file, i.e. ability to load into DRAM. If so, should this be added to the threat model section? Is it not easy to defend against? (would memory deduplication vulnerabilities also offer a route to do something similar?). keeping our weights file private / encrpyted appears to be a defense or couldn't we simply perform some sort of simple validation/ checksum when loading from DRAM? 

3. given model accuracy drops significantly after the attack on more complex models, we could also, periodically, simply test accuracy to detect attempts to modify the model? (e.g., in case other protections were circumvented)

4. where are the bits that need to be flipped located precisely? can we learn anything interesting from this in terms of possible countermeasures?

5. is the location of bits vulnerable to RowHammer something we could consider to help protect the model? (although this seems unnecessary given ease at which we could protect against any modifications to model weights). 

6. could we make changes to the model architecture to increase the number of bitflips required? What about current training-time defences to protect against adversarial inputs, are they helpful?
  
minor:
a) Section 3.1 "misclassify the input to the target class" - should there be some mention of the trigger here? The trigger gets drop in various places which perhaps makes the paper harder to follow?
b) page 4: "Specifically, " - is the intention the reader then read the text in the box? The text in the box could be clearer. It is poorly written at the moment.
c) I find the descriptions in section 4.5 difficult to follow. 

summary_of_the_review: Overall an interesting paper, but I think some additional justification for why such attacks are not easily defended against is perhaps required.  Clarity could be improved in places. Given this I believe the paper is currently below the acceptance threshold.
review_title: null

Meta-review: The work studied the problem of inserting backdoor into a deployed model through bit flip.

Some important concerns have been proposed by reviewers, including: the incorrect claim of the treat model, the potential defenses are only discussed but not validated, experimental setups and analysis (e.g., the sensitivity test of hyper-parameters). Although the authors provided some responses, but all reviewers are not well convinced. 

After reading the manuscript, reviews and discussions between reviewers and authors, I think this work is not ready for publication. The reviewers' comments are supposed to be helpful to improve this work.
Decision: Reject