An Optimization Perspective on Realizing Backdoor Injection Attacks on Deep Neural Networks in Hardware
Authors: M. Caner Tol
Saad Islam
Berk Sunar
Ziming Zhang
Venue: ICLR 2022 Submitted (2022)
Keywords: targeted attack
bit-flip
weight attack
backdoor
trojan

Abstract: State-of-the-art deep neural networks (DNNs) have been proven to be vulnerable to adversarial manipulation and backdoor attacks. Backdoored models deviate from expected behavior on inputs with predefined triggers while retaining performance on clean data. Recent works focus on software simulation of backdoor injection during the inference phase by modifying network weights, which we find often unrealistic in practice due to the hardware restriction such as bit allocation in memory. In contrast, in this work, we investigate the viability of backdoor injection attacks in real-life deployments of DNNs on hardware and address such practical issues in hardware implementation from a novel optimization perspective. 
We are motivated by the fact that the vulnerable memory locations are very rare, device-specific, and sparsely distributed. Consequently, we propose a novel network training algorithm based on constrained optimization for realistic backdoor injection attack in hardware. By modifying parameters uniformly across the convolutional and fully-connected layers as well as optimizing the trigger pattern together, we achieve the state-of-the-art attack performance with fewer bit flips. For instance, our method on a hardware-deployed ResNet-20 model trained on CIFAR-10 can achieve over 91\% test accuracy and 94\% attack success rate by flipping only 10 bits out of 2.2 million bits.  

Under review as a conference paper at ICLR 2022
AN OPTIMIZATION PERSPECTIVE ON REALIZING BACKDOOR INJECTION ATTACKS ON DEEP NEURAL NETWORKS IN HARDWARE
Anonymous authors Paper under double-blind review
ABSTRACT
State-of-the-art deep neural networks (DNNs) have been proven to be vulnerable to adversarial manipulation and backdoor attacks, Backdoored models deviate from expected behavior on inputs with predefined triggers while retaining per- formance on clean data. Recent works focus on software simulation of backdoor injection during the inference phase by modifying network weights, which we find often unrealistic in practice due to the hardware restriction such bit allocation in memory. In contrast, in this work, we investigate the viability of backdoor in- jection attacks in real-life deployments of DNNs hardware and address such practical issues in hardware implementation from a novel optimization perspec- tive. We are motivated by the fact that the vulnerable memory locations very rare, device- sevice-specific, and sparsely distributed. Consequently, we propose a novel network training algorithm based on constrained optimization for realistic back- door injection attack in hardware. By modifying parameters uniformly across the convolutional and fully-connected layers as well as optimizing the trigger pattern together, we achieve the state-of -fate-of-the-art attack performance with fewer bit flips. For instance, our method on hardware-deployed ResNet-20 model trained on CIFAR-10 can achieve over 91 % test accuracy and 94% attack success rate by flipping only 10 bits out of 2.2 million bits.
INTRODUCTION
DNN models are known for their powerful feature extraction, representation and classification ca- pabilities. However, the large number of parameters and need for large training dataset makes hard to interpret the behavior of these models. The fact that increasing number of security critical systems rely DNN models in real-world deployments raises numerous robustness and security questions. Indeed, DNN models have been shown to be vulnerable against imperceivable perturba- tions to input samples which can be misclassified by manipulating the network weights (Szegedy) et al. 2013 Goodfellow et al. 2014 Nguyen et al. 2015).
Emboldened by recent physical fault injection attacks, e.g., Rowhammer, alternative approach was proposed that directly targets the model when is loaded into memory. The advantage of this attack approach is that it can remain completely stealthy since the model only modified in real-time while running in memory and no modification to inputs is required. Recently, Hong et al. (2019); Yao al. (2020) showed that flipping a few bits in DNN model weights in memory while succeeding in achieving misclassification, has the side-effect of significant accuracy drops which renders the model useless. Liu et al. (017a) and Bai et al. (021)) addressed this problem by tweaking only minimum number of model weights that makes a DNN model misclassify chosen input to target label. This approach indeed achieves the objective with only slight drop in the classification accuracy. Nevertheless, whether practical attack such injecting backdoor to DNNs be realized in a scalable and stealthy manner using Rowhammer in hardware is still an open question. Earlier approaches assume that Rowhammer can flip any bits in the memory. This is far from what
indeed

---

Under review as a conference paper at ICLR 2022
observe in reality: only small fraction of the memory cells vulnerable. This motivates to reconsider the backdoor injection process under new constraints, including the training algorithms.
Contributions. In this paper we propose novel algorithm based on constrained optimization that can identify vulnerable bits the memory for deep learning model to create a backdoor. To this end, we introduce obtained using Rowhammer characterization experiments. To show the practicality of our approach, by fine-tuning the model we first identify the necessary bit flips to implement the backdoor. Using the Rowhammer attack we inject the backdoor in a DRAM setup to the deployed model. Once the backdoor is injected, DNN model will misclassify any sample with the trigger pattern the target class without degrading the overall accuracy on clean data. This result shows that indeed real-life deployments are under threat from backdoor injection attacks with potentially adverse affects to the well being of the society. More work needs to be done to secure deployed models from fault injection attacks used for every day tasks by end users.
fine-tuning process that takes into account the stringent constraints on faults
2
BACKGROUND AND RELATED WORKS
2.1 BACKDOOR ATTACKS
The terms Backdoor and Trojan are used interchangeably by different communities. Here we use Backdoor for consistency. In DNN models, define Backdoor hidden feature that causes a change in the behavior that is triggered only by the existence of special type of inputs. In the literature, backdooring applied with either benevolent intents, such as watermarking the DNN models, or with malicious intents, a Trojan to attack the models. In this work, we focus on Backdoor as a type of Trojan that is exploited by an attacker to cause targeted misclassification.
2.2 ATTACKS ON DEEP LEARNING MODELS
DNN model weights have been shown be vulnerable to Rowhammer attacks (Hong et al. 2019) causing accuracy degradation even after the of quantization Yao et al. 2020) defense. Á binary integer programming approach was proposed (Bai et al. 2021) to find the minimum number of bit changes required to manipulate the model for specific example. Further Rakin et al. (0206) showed that samples from single or multiple classes can be misclassified to a target class using Rowhammer. However, this attack is trigger. Earlier works by Liu et al. 2017b) and Gu et al. (017) demonstrated that backdoor attacks pose a threat the DNN model supply chain which can be exploited by poisoning data sets (Gu et al. 2017), poisoming the training code (Bagdasaryan & Shmatikov (020). modifying network connections or modifying hardware (Clements & Lao [018]] Venceslai al. 2020). Pang et al. (2020) proposed mutual optimization of adversarial input and the poisoned models. Recently, |Rakin al. (020a) showed that backdoor attacks can be implemented by changing only a small number of weight parameters assuming any bit in the memory can be flipped. However, the practicality of software-based backdoor injection attacks during the inference phase is still an open question due to the practical constraints that overlooked in previous works.
the scope of backdoor attacks since does use
2.3 ROWIIAMERR ATTACK
As memories are becoming more compact and memory cells are getting closer and closer, the bound- aries between the DRAM rows do not provide sufficient isolation from electrical interference. The data is encoded the form of voltage levels inside a capacitor which leaks charge over time. Thus they need to be refreshed after every 64ms to keep the data intact. However, if there is an attacker residing in a nearby DRAM row, although in completely isolated process, the attacker can cause faster leakage in the victim row by just accessing his own memory space repeatedly (hammering). Rowhammer is a software induced fault attack first introduced by Kim et al. (2014). Recently, [Frigo) al. (2020) and de Ridder et al. (2021) have shown that more than 80% of the DRAM chips in the market are vulnerable to the Rowhammer attack including DDR4 chips having Target Row Re- fresh (TRR) mitigation. The Error Correcting Codes (ECC) mitigation has also been bypassed in Cojocar et al. (2019). Rowhammer is a great threat to shared cloud environments (Cojocar et al. 2020 Xiao al. 2016) it can be launched across virtual machine (VM) boundaries and even
2

---

Under review as a conference paper at ICLR 2022
Fault
Clian frous
1 00:00:00000
00:1 2:000
EEST Aodbod Irpur.
Triggo
Figure 1: Clean Model (left) vs. Backdoored Model (right) behavior with clean inputs (top) and trigger added inputs (bottom). Note the fault injection (right) model in memory device changes behavior of classifier shown by the confusion matrix (bottom, right).
remotely through JavaScript. More recently, Qazi et al. (2021) have shown a combined effect of more than two aggressor rows to induce bit flips recent generations of DRAM chips. All existing Rowhammer defenses including TRR, ECC, detection using Hardware Performance Counters and changing the refresh rate can not fully prevent Rowhammer attack (Gruss et al., 2018 Frigo et al. 2020). The only requirement of the Rowhammer attack is that the attacker and the victim share the same DRAM chip, vulnerable to the Rowhammer attack.
METHODOLOGY
3.1
THREAT MODELS
Same as in earlier works (Gu et al. 2017 Rakin et al. 2020a Liu et al. 2017b Hong et al. 2019
Yao al. 2020,, we assume that the attacker
only knows the model architecture and model parameters;
has a small percentage the unseen test data set;
does not have access to the training hyperparameters, or the training dataset;
is involved only after the model deployment a cloud server, and therefore does not need to modify the software supply chain;
resides in the same physical memory as the target model; has no than regular user privileges (no root access).
Such threat models are well motivated in shared cloud instances targeting co-located host running the model, and in sandboxed browsers targeting model residing in the memory of the host machine (Cojocar et al. 2020 Xiao et al. 2016 de Ridder et al. (021).
To better understand our attacker, illustrate example in Fig. The attacker works follows:
1. Offline Attack Phase:
By studying the model parameters and the memory, the attacker generates trigger pattern and determines the vulnerable bits in the target model.
2. Dynamic Trigger Injection: After the target model is loaded into the memory, using Rowhammer the attacker flips the target bits by only accessing its own data that resides in the neighboring rows of the weight matrices in the DRAM.
Misclassification: After the backdoor is inserted, the model will misclassify the trigger-added added input to the target class. The misclassification will persist until the backdoored model is unloaded from the memory. Since the model persistent storage (or in the software distribution chain) untouched, the malicious modification to the model is harder to detect.
3.2 OFFLINE ATTACK PIIASE
In the offline phase the attack, we optimize the trigger pattern and the bit-flip locations in the weight matrices. To do so, first extract the profile of vulnerable bits in the DRAM and then
3

---

Under review as a conference paper at ICLR 2022
Bi Index
84
192
256
320
384
448
512
64- bit Words
Figure 2: Bit flips in one of the 4KB pages showing the sparsity of the actual bit flips.
train the backdoor model with constraints. Note that this phase is independent the hardware specification, and the learned model can be used freely in any DRAM.
3.2. ROWHAAMER PROFILING
Rowhammer profiling a process of finding vulnerable memory addresses in the DRAM. This process can be performed offline before the victim starts running. We have used the Hammertime tool P by Tatar et al. (2018) to profile our DRAM with double-sided Rowhammer. As bits are physically organized in banks, which it finds the bit flips along with the direction the flip. We translate this organization into 4KB pages as need to match the vulnerable indexes with target weights file which is also divided into 4KB pages and stored the DRAM. Among all the available Rowhammer tools, Hammertime gives the most number of bit flips as shown (Tatar et 2018). In practice, 0.036 % of the total cells in the 128MB buffer have bit flips, as illustrated in Fig. 2 Hence, assuming a specific sequence of bit flips within page like in previous research is highly unrealistic.
and columns in DRAM, the tool gives these parameters in
observe that only
3.2.2 CONSTRAINED FINE- TUNING WITH BIT REDUCTION
In contrast to previous works such Liu et al.) 2017b; Rakin et al., 2020a). in this paper propose a novel joint learning framework based on constrained optimization to learn the network weight modification pattern well the data trigger pattern simultaneously. Also different from the literature, we do not rely on the last layer only to find vulnerable weights; instead we achieve a wider attack surface on the model with constraints placed on the number and location of faults.
To preserve the performance of the networks on clean data, given a collection of test samples {24} and their corresponding class labels {yi). propose optimizing the following objective:
min ABE ||
max
F(AO. Ar)
- -) (00, + (aaee((rri Ax, 0+ 20), v)): (I)
where A8, Ar denote the weight modification pattern and the data trigger pattern, ğ denotes the target label, é denotes a loss function. denotes the network parameterized by á originally, a E [o, 1] denotes a predefined trade-off parameter to balance the losses on clean data and triggered data, and 2 () denotes another predefined parameter to control the trigger pattern. Note that AO denotes feasible solution space that is restricted by the hardware implementation requirements. Specifically,
Rowhammer attack restriction in hardware: allows realistically to flip only about one bit per memory page due to the physical constraints. Since the potentially vulnerable memory cells in the DRAM are sparse (See Fig. 2 we cannot flip multiple bits at the targeted page. Such restriction forms the feasible solution space AO in learning.
To solve the constrained optimization problem defined in Eq. [ we also propose a novel learning algorithm as listed in Alg. that consists of the following 4 steps:
Step Learning data trigger pattern Ar. The goal of this step is to learn trigger that can activate the neurons related to the target label ÿ to fool the network. Trigger pattern generation starts with
For the implementation details of Rowhammer profiling, we refer readers to Hammertime repository.
https://githon www./usecchaanmerree...
4

---

Under review as a conference paper at ICLR 2022
Algorithm 1: Learning realistic Rowhammer attack for hardware implementation
Input: DNN model with weights number bits Nttip that are allowed to be flipped the memory, objective F, parameter learning rate and maximum number of iterations T Output: Backdoored model l and data trigger pattern Ar*
A A.r"
for t [T' do
update the trigger == true then
Ar t Ar + sgu V (A0 Ax"));
end
M < Settp V A,"),,Nsus descending') )
A +- A - n ( Ar" Ar")]M;
if bit reduction true then
0r + Floor((0 + A0") @ 0) 00:0
end
end
return 8" Ax
an initial trigger mask. Then we simply use the Fast Gradient Sign Method (FGSM) GGooffellw et al. 2014) to learn the trigger pattern. The update rule is defined
Ar = Ax" + e sgn(( A. F(A0" Ar")),
(2)
where A0", Ax denote the current solutions for the two variables, V denotes the gradient operator, and sgn denotes the signum function.
Step 2. Locating vulnerable weights. Now given a number of bits that need be flipped, Niipp which should be no bigger than the page number in the memory, in this step our goals are:
Cl. Locating weight per bit flip towards minimizing our objective in Eq. significantly
C2. No co-occurrence in the same memory page among the flipped bits.
In fact, these constraints define combinatorial optimization problem (that is NP-hard, in gen- eral (Papadimitriou & Steiglitz 1998)) over a huge space network parameters. Therefore, solving the problem in Eq. with such constraints exactly will be extremely challenging.
Instead, we relax our learning problem by ignoring Constraint (C2) and locate vulnerable weights heuristically. Recall that when DNN model is fed into the memory, the network weights are loaded sequentially page-by-page where each page is fixed-length and stored contiguously. Equivalently we can view this procedure as loading long vector by vectorizing the model. Therefore, locate vulnerable weights, we simply divide the network weight vector into N Nştip groups equally as possible, and rank the weights per group based on the absolute values in the gradient over 20, i.e., VV F where denotes the entry-wise absolute operator, in descending order. The top-1 weight per group will be identified as vulnerable weight.
Because empirically the number of pages in the memory that is occupied by the network model much larger than N Ntpp based on the pigeonhole principle the co-occurrence probability among flipped bits is very low. Actually we have not observed such case in experiments.
Step Adversarial fine-tuning. Now given a collection of located vulnerable weights, denoted by M, we only need to update these weights in backpropagation as follows:
A 20" - 17 VV (AO", Ax")]M,
(3)
where I. denotes masking function that returns the gradients for the weights in M, otherwise O's, and 17 2 0 denotes a learning rate.
Step 4. Bit reduction. To meet the physical constraints of the Rowhammer attack, the final part of our attack procedure requires bit reduction. Rowhammer can only flip a very low number of bits in 4KB memory page and more than one faulty memory cell almost never coexists within byte. Therefore, we define a bit reduction function as Floor(@ e 8*), where t denotes the bit-wise summation, and function Floor rounds down the number by keeping the most significant nonzero bit
5

---

Under review as a conference paper at ICLR 2022
only. For instance, letting 0 1101g and 0- = 1010g, then Floor(# @6") Floor = 1002- In this way, we assures that only one bit is modified in a selected weight while maintaining its change direction and amount as much possible.
3.3 ONLINE ATTACK PHASE: FLIPPING BITS IN THE DEPLOYED MODEL IN MEMORY
When access file from the secondary storage, it is first loaded into the DRAM and when we close the file, the operating system (OS) does not delete the file from the DRAM to make the subsequent accesses faster. However, it shows that space as free the user and utilizes it as page cache. If the file is modified, the OS sets the dirty bit of that modified page and it is written back according to the configured write back policy. Otherwise, the file remains cached unless evicted by some other process or file. As Rowhammer is capable of flipping bits in DRAM, we can use it in the online attack phase to flip the weights of the DNN file as it loaded in the page cache. The weights file is divided into pages and stored in the page cache. We can flip our target bits identified by the backdoored parameters 8", in Section 3.2 The OS does not detect this change as it is directly made in the hardware by completely isolated process and it keeps providing the page cached modified copy to the victim on subsequent accesses. Thus the attack remains stcalthy.
3.4 WEIGHT QUANTIZATION
The weights are stored as Nş- quantized values in the memory as implemented in NVIDIA Ten- sorRT (Migacz 2017), high performance DNN optimizer for deployment that utilizes quantized weights (NVIDIA). Essentially, a floating-point weight matrix Wyp is re-encoded into No-bit signed integer matrix W,, as W; round(W Aw) where Aw = max fp) (2N,-1 1). In our ex- periments, 8-bit quantization and the weights are stored in two's 's complement form.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP AND EVALUATION METRICS
To demonstrate the viability of our attack in the real-world, we implemented it on an 8-bit quantized ResNet- 18 model trained on CIFAR-10 using PyTorch v1.8.1 library. The clean model weights that are trained on CIFAR-10 are taken from (Rakin et al. (020a) for ResNet-18 and from (Idel- bayev)(580 stars GitHub) for other ResNet models. Moreover, we experimented on larger ver- sions of ResNet models, such as ResNet50, trained on ImageNet data set. For the models that are trained on ImageNet, we use pretrained models of Torchvision library (9.1K stars GitHub) which is downloaded 28 million times until now (Pepy.tech.) Retrieved September 23, 202T). We run the offline phase of the attack NVIDIA GeForce GTX 1080Ti GPU and Intel Core i9-7900X CPU. Rowhammer attack is implemented on models deployed on DDR3 DRAM of size 2 GB (M378B5773DH0-CH99)
We compare our approach with BadNet (Gu et al. 2017). and TBT (Rakin et al.) 2020a) as well as fine tuning (FT) the last layer. We also include the output of our Constrained Fine Tuning (CFT) without bit reduction in Table 2 for comparison. For the results on software, we keep all the bit flips in the weight parameters assuming they are all viable. In the hardware results, we keep the bits that are possible to be flipped by Rowhammer and exclude the others. We use 128 images from the unseen test data set for all the experiments CIFAR-10. Test Accuracy and Attack Success Rate metrics are calculated on unseen test data set of 10K images. In all experiments we used = 0.5 for Algorithm] [ The trigger masks are initialized as black square on the bottom right corner of the clean images with sizes 10x10 and 73x73 on CIFAR-10 10 and ImageNet respectively. in Eq. D is chosen as 0.001. For the ImageNet experiments, we use 1024 images from the unseen test data set to cover all 1000 classes. Test Accuracy and Attack Success Rate metrics are calculated on unseen test data set of 50K.
Number of Bit Flips (ts)sp) As in Yao et al. 2020; Hong et al. 2019 Rakin et al. 2020a Bai et al., 2021), the first metric we to evaluate bits are different in the new version of the model in total. The N stip has to be as low as possible because only limited number of bit locations are vulnerable to the Rowhammer attack in DRAM.
method is N Mip which indicates how many

---

Under review as a conference paper at ICLR 2022
As the Stp increases, the probability of finding right match of vulnerable bit offsets decreases. Nypp is calculated as N Nlip Σ D(ol!,, "aili,," !*!)), where D is the hamming distance between the parameters All and (-(l) at the 1-th layer the network with layers in total.
DRAM Match Rate (fmatch) After Rowhammer specific bit-search method runs, the outputs given as the locations of target bits in a DNN model. However, not all of the bit locations are flipable in the DRAM. Therefore, we propose new metric measure how many of the given bits actually match with the vulnerable memory cells in a DRAM which is crucial to find out how realistic is Rowhammer- based backdoor injection attack. Tmatch is calculated as, Tmateh
Emazch 100 N
where Amatch is the number of matching bit flips and Nap is the total number of bit flips. Since the bit flip profile varies among different DRAM's, s, even between the same vendors and models, I' mmatch is a device-specific metric.
Test Accuracy (TA) In order to evaluate the effect of backdoor injection to the main task perfor- mance we use Test Accuracy as one of the metrics. Test Accuracy is defined as the ratio of correct classifications on the test dataset with no backdoor trigger added.
Attack Success Rate (ASR) We define the Attack Success Rate the ratio of misclassifications on the test data set to the target class when the backdoor trigger is added to the samples. Attack Success Rate indicates how successful is backdoor attack unseen data set.
4.2 LIMITING THE NUMBER OF MODIFIED PARAMETERS
First, we fine tune all parameters using clean and adversarial examples, and then, we restore part of the parameters their original values at the end of the training. Since the training approach aligned with the work in (Gu et al, [017]) we refer this method as BadNet. Table T shows that fine-tuning without any constraints distributes the knowledge of backdoor to parameters and the parameter limit that is applied at the end degrades the backdoor success rate drastically. For instance, BadNet reaches 90.92% test accuracy and 61.06....66 .04% attack success rate with 90% the 88 Million bit parameters fine tuned, whereas we reach 92.95% test accuracy and 95.26% attack success rate with only 99 bit-flips using CFT+BR.
Table I: BadNet reaches reasonable attack success rate (ASR) and test accuracy (TA) only when more than 90% parameters are changed. The baseline performance of BadNet achieves up to 87.61% % test accuracy and 99.88% attack success rate. Limiting the percentage of modifications after fine-tuning decreases the attack performance.
Modification (%)
TA(%)
ASR(%)
100
87.61
99.88
99
89.79
76.11
90
90.92
61.04
91.67
51.22
70
92.01
43.79
50
92.41
34.15
4.3 CIFAR- 10 EXPERIMENTS
We experimented our proposed method on ResNet18, ResNet20 and ResNet32 trained on CIFAR-10 along with the baseline methods, such as BadNet, Fine Tuning the last layer (FT), and TBT. We also compare our partial method without Bir Reduction with our complete method (CFT+BR) which in- cludes the Bit Reduction. We selected the results that give a reasonable performance without increas- ing the N Nttpp. and thus, decreasing mmatch further.Figure shows the loss progress after each epoch with one batch of data. The results which are summarized in Table 2, show that CFT+BR method successfully injects a backdoor into ResNet20 model with 91.24% % Test Accuracy and 94.62% At- tack Success Rate by flipping only 10 bits in the DRAM. In ResNet18 and ResNet32, CFT+BR achieves the best Attack Success Rate with maximum of 1 1.66% degradation in the Test Accuracy. In BadNet, FT, and TBT, the bit flips are concentrated within the same pages. For example, flips 2,238 bits on ResNer20 within page. However, only targeted bit in page is what one

---

Under review as a conference paper at ICLR 2022
W
1.6k
Training Step
Figure 3: Total loss graph at every training iteration during the backdoor injection to the ResNet18
can expect to be flipped practice. Therefore, when the attack implemented on DRAM with Rowhammer, the Attack performance decreases to below 3%. In CFT, Tmmath is relatively higher than the previous methods since modifies only one parameters in a page. However, it does not put constraint on the number of bit flips within byte during the optimization. Therefore, the attack performance degrades drastically in practice. In all experiments, CFT+BR has 100% Tmatch since already considers the bit locations that can be flipped during the attack. Since the bit flips sparse across different memory pages in CFT+BR, 100 % of the bit flips can actually be flipped.
4.4 IMAGENET EXPERIMENTS
We also compared our method with the baseline methods on models pre-trained with ImageNet data set. We used pretrained ResNet34 and ResNet50 from the model zoo (Torchvision) as the target models. The results are summarized in Table 2 Although, some of the baselines have better theo- retical results on some models, we show that when the physical attack happens, none of them has a significant Attack Performance. However, CFT+BR can still inject the backdoor into ImageNet models with over 90% Attack Success Rate and maximum of 6.8% degradation in the Test Ac- curacy. These results show that our approach generalizes to larger data sets and models. Note that, although N stip increases the model gets larger in CFT+BR, it is still possible to flip 100% of these bits due to the sparse distribution.
4.5 POTENTIAL COUNTERMEASURES
Pre-deployment Phase He et al. (2000) propose Binarized networks and Piccewise Weight Clus- tering to increase the resistance of DNNs against the bit flip attacks. Our experiments show that using Binarized Networks is an effective defense against our attack since aggressively decreases the size of the network and the maximum N Nitip- Note that it may still be vulnerable using other fault attacks which do not require sparse faulty bit locations. We also applied our attack against the Piecewise Weight Clustering technique. The results show that training the model with Piecewise Weight Clustering makes it harder to keep both the Test Accuracy and Attack Success Rate high. The ASR drops down to 43.42% when TA is 89.66% with 112 N jups ResNet32. On the other hand, when the ASR is 98,49, TA drops down to 9.9% with the same N Nkkps-
Detection Li et al. 2020b) propose checker model along with the original model to detect "transient fault" in the original model and repeat the inference. Since the transient assumption does not hold, even if checker model raises alarm and repeats the inference. the new inference is made by the backdoor injected model and is not be detected. Liu et al. (200)) propose weight encoding to detect bit-flip attacks in the topmost sensitive layers. However, the assumption of "spatial locality" does not hold with our attack. Even though the detection can possibly work in theory, the estimated overhead to protect the whole network is very large. For ResNet-34, we estimate the time overhead of weight encoding
about 834 seconds and the storage cost as about 375 MB which is 446% storage overhead which shows that the proposed method is not scalable enough to defend against our attack. Li et al. (2021) propose checksum-baas
detection during inference. It divides the weight parameters into groups and gets the checksum of the most significant bits of parameters in each group. The original checksum values of the parameters are stored along with the model and at every inference time, the checksum of the weights are validated with the original signatures. Note that the optimization constraints can be further increased to avoid flipping the MSB of the weight
8

---

Under review as a conference paper at ICLR 2022
Table 2: Comparison of our methods CFT, CFT+BR with the baseline methods BadNet.FT and,
TBT on CIFARIO (Krizhevsky et al. 2009) with ResNet-20/52/18, and ImageNet (Deng al. 2009) with ResNet-34/50. Our proposed CFT+BR results are written bold. Note that the percentage of the backdoor parameter (0*) bits that are actually flipable Tmatch must be near 100% for viable Rowhammer backdoor attack.
Software TA(%)
Hardware TA(%)
Dataset
Net
Method BadNet
Nylip
ASR(%)
Nyiip
ASR(%)
Tmaach (%)
Acc: 91. 91.78%
#Bits: 2.2M
172,891
86.96
99.98
33
91.76
2.69
0.02
ResNer20
FT
2,238
84.36
97.10
1
91.72
2.96
0.04
TBT
44
86.61
95.43
91.72
4.81
2.27
CFT
90.09
99.55
5
91.79
14.70
22.73
CFT+BR
10
91.24
94.62
10
91.24
94.62
100
CIFARIO
ResNet32
Acc: 92, 6.62%
#Bits: 3.7M
BadNet
246,004
88.60
99.99
53
92.61
7.47
0.02
FT
2318
81.87
90.59
92.65
8.75
0.04
TBT
210
81.90
89.66
92.66
8.60
0.48
CFT
39
90.25
98.75
10
92.41
20.64
25.64
CFT+BR
91.77
91.46
95
91.77
91.46
100
Acc: 93. 10%
#Bits: 88M
BadNet
1,493,301
87.61
99.88
416
93.06
12.71
0.03
ResNer18
FT
8,667
88.80
95.34
92.20
34.88
0.01
TBT
95
82.87
88.82
92.60
49.13
1.05
CFT
42
92.39
99.90
11
91.52
0.37
26.19
CFT+BR
99
92.95
95.26
99
92.95
95.26
100
Acc: 73.31%
#Bits 172M
BadNet
441.047
70.81
99.73
100
72.13
0.009
0.02
ResNet50 ResNet34
FT
54,726
68.30
99.14
11
72.70
0.18
0.02
TBT
553
72.69
99.86
72.72
0.05
0.18
ImageNet
CFT
1509
70.25
99.76
388
71.65
0.10
25.71
CFT+BR
1463
70.28
72.92
1463
70.28
72.92
100
Acc: 76.15 13%
#Bits: 184M
BadNet
359,516
73.98
99.11
129
68.07
0.05
0.04
FT
93,778
68.43
96.52
12
75.59
0.09
0.01
TBT
543
75.60
99.98
1
75.60
0.1
0.18
CFT
1562
70.58
99.99
391
68.35
5.02
25.03
CFT+BR
69.33
90.32
1475
69.33
90.32
parameters our attack which can possibly bypass the detection. Assuming linear time complexity, time overhead goes up to 40.1 1 % overhead for full-size bit protection in ResNer20.
Recovery Li et al. (020a) propose Weight Reconstruction to recover the original clean network after a bit flip attack occurs. When the attacker is not aware of the defense and applies the offline phase of the attack against a defenseless model, the defense is able to reduce ASR down to 32.89% 89%. However, if the attacker is aware of the defense and applies CFT+BR attack can successfully bypass this defense by achieving 94.04% ASR and 89.51% 51% TA.
defended model,
5 CONCLUSION
We analyzed the viability of a real-world DNN backdoor injection attack. Our backdoor attack scenario applies deployed models by flipping few bits in memory assisted by Rowhammer tack. Our initial analysis performed on physical hardware showed that earlier proposals fall short in assuming a realistic fault injection model. We devised a new backdoor injection attack method that adopts a combination of trigger pattern generation and sparse and uniform weight optmiizatin. Compared to earlier proposals, our technique uses all layers, and combines all trigger pattern gen- eration, target neuron selection and fine tuning model parameter weights in the same training loop. Since our approach targets the weight parameters uniformly, no more than one bit a memory page is flipped. Further, we introduced new metrics to capture realistic fault injection model. This new approach achieves a viable solution to target real-life deployments: on CIFAR10 (ResNet 18, 20, 32 models) and ImageNet (Resnet34 and 50 models) wel-hhardware by running the actual Rowhammer attack achieving test accuracy and attack success rates as high as 92.95 % and 95.26%, respectively.

---

Under review as a conference paper at ICLR 2022
6 ETHICS STATEMENT
In this work, we analyzed the vuinerability of DNN models to the backdoor injection attacks using Rowhammer. We showed that commonly used public DNN models are vulnerable to our attack. Since our work closes the gap between the algorithmic proposals and practical constraints, the real- world backdoor injection attacks are shown to be practical. Since DNN models are utilized in many different areas, a potential exploitation of this vulnerability may cause negative impacts on the society. Therefore, we strongly recommend following the security advisories and deploying the required countermeasures on sensitive systems.
REFERENCES
Eugene Bagdasaryan and Vitaly Shmatikov. Blind backdoors in deep learning models. arXiv preprint arXiv:2005.03822 03823, 2020.
Jiawang Bai, Baoyuan Wu, Yong Zhang, Yiming Li, Zhifeng Li, and Shu-Tao Xia. Targeted attack against deep neural networks via flipping limited weight bits. arXiv preprint arXiv:2102.1 10496, 2021.
Edward Chou, Florian Tramer, and Giancarlo Pellegrino. Sentinet: Detecting localized universal attacks against deep learning systems. In 2020 IEEE Security and Privacy Workshops (SPW), pp- 48-54. IEFE, 2020.
Joseph Clements and Yingjie Lao. Hardware trojan attacks on neural networks. arXiv preprint 11Xiv:/806.05768 2018.
Lucian Cojocar, Kaveh Razavi, Cristiano Giuffrida, and Herbert Bos. Exploiting correcting codes: On the effectiveness of ecc memory against rowhammer attacks. In 2019 IEEE Symposium on Security and Privacy (SP), pp. 55-71. IEEE, 2019.
Lucian Cojocar, Jeremie Kim, Minesh Patel, Lillian Tsai, Stefan Saroiu, Alec Wolman, and Onur Mutlu. Are we susceptible to rowhammer? an end-to-end methodology for cloud providers. In 2020 IEEE Symposium on Security and Privacy (SP), pp. 712-728. IEEE, 2020.
Finn de Ridder, Pietro Frigo, Emanuele Vannacci, Herbert Bos, Cristiano Giuffrida, and Kaveh Razavi. Smash: Synchronized many-sided rowhammer attacks from javascript. In 30th {USENIX} Security Symposium ({ (USENIX} Security 21). 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.
Ieee, 2009.
Pietro Frigo, Emanuele Vannace, Hasan Hassan, Victor Van Der Veen, Onur Mutlu, Cristiano Giuf- frida, Herbert Bos, and Kaveh Razavi. Trrespass: Exploiting the many sides of target row refresh. In 2020 IEEE Symposium on Security and Privacy (SP), pp. 747-762. IEEE, 2020.
Jacob Gildenblat and contributors. Pytorch library for cam methods. htttps:/ com/ Bccobog_//yttorchhgrra--caa 2021.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint 141iv:1412.6557. 2014.
Daniel Gruss, Moritz Lipp, Michael Schwarz, Daniel Genkin, Jonas Juffinger, Sioli O'Connell, Wolfgang Schoechl, and Yuval Yarom. Another flip in the wall of rowhammer defenses. In 2018 IEEE, 2018.
IEEE Symposium Security and Privacy (SP), pp.
Tianyu Gu, Brendan Dolan-Gavitt. Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv: /708.066 06738,, 2017.
Zhezhi He, Adnan Siraj Rakin, Jingtao Li, Chaitali Chakrabarti, and Deliang Fan. Defending and harnessing the fit-flip based adversarial weight attack. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 14095- 14103, 2020.
10

---

Under review as a conference paper at ICLR 2022
Sanghyun Hong. Pietro Frigo, Yigitcan Kaya, Cristiano Giuffrida, and Tudor Dumitras. Terminal brain damage:
Exposing the graceless degradation in deep neural networks under hardware fault attacks. In 28th {USENIX] Security Symposium ({USENIX) Security 19), pp. 497-514, 2019, 2019.
Yerlan Idelbayev. Proper ResNet implementatiinn for CIFARIOICIFARION in PyTorch. https: Accessed: 2021-05-26.
Yoongu Kim, Ross Daly, Jeremie Kim, Chris Fallin, Ji Hye Lee, Donghyuk Lee, Chris Wilkerson, Konrad Lai, and Onur Mutlu. Flipping bits in memory without accessing them: An experimental study of dram disturbance errors. ACM SIGARCH Computer Architecture News, 42(3):361-377 2014.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Jingtao Li, Adnan Siraj Rakin, Yan Xiong, Liangliang Chang, Zhezhi He, Deliang Fan, and Chaitali Chakrabarti. Defending bit-flip attack through dnn weight reconstruction. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pp. 1-6. IEEE, 2020a.
Jingtao Li, Adnan Siraj Rakin, Zhezhi He, Deliang Fan, and Chaitali Chakrabarti. Radar: Run-time adversarial weight attack detection and accuracy recovery. arXiv preprint 1rXiv:2101.08254, 2021.
Yu Li, Min Li, Bo Luo, Ye Tian, and Qiang Xu. Deepdyve: Dynamic verification for deep neural net- works. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp. 101-112, 2020b.
Qi Liu, Wujie Wen, and Yanzhi Wang. Concurrent weight encoding- detection for bit-flip attack on neural network accelerators. In Proceedings of the 39th International Conference on Computer-Aided Design, pp. 1-8, 2020.
Yannan Liu, Lingxiao Wei, Bo Luo, and Qiang Xu. Fault injection attack on deep neural network. In 2017 IEEE/ACM International Conference on Computer-Aides Design (ICCAD), pp. 131-138. IEEE, 2017a.
Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. Trojaning attack on neural networks. 2017b.
Szymon Migacz. 8-bit inference with TensorRT. NVIDIA GPU Technology Conference, 2017.
Onur Mutuu and Jeremie S Kim. Rowhammer: A retrospective. IEEE Transactions on Computer- Aided Design of Integrated Circuits and Systems, 3955):155-11571, 2019.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confi- dence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 427-436, 2015.
NVIDIA. TensorRT documentation. URL tensorrt Accessed: 2021-05-25.
Ren Pang, Hua Shen, Xinyang Zhang, Shouling Ji, Yevgeniy Vorobeychik, Xiapu Luo, Alex Liu, and Ting Wang. A tale of evil twins: Adversarial inputs versus poisoned models. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp. 85-99, 2020.
Christos H Papadimitriou and Kenneth Steiglitz. Combinatorial optimization: algorithms and com- plexity. Courier Corporation, 1998.
Pepy.tech. torchvision.
Retrieved Redrivee Septem-
ber 23, 2021.
Salman Qazi, Yoongu Kim, Boichat Boichat, Eric Shui, and Mattias Nissler. Introducing half- double: New hammering technique for dram rowhammer bug. ltttps://github com/ google googgle/hammrr-ie 2021.
11

---

Under review as a conference paper at ICLR 2022
Adnan Siraj Rakin, Zhezhi He, and Deliang Fan. Tht: Targeted neural network attack with bit trojan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 13198-13207, 2020a.
Adnan Siraj Rakin, Zhezhi He, Jingtao Li, Fan Yao, Chaitali Chakrabarti, and Deliang Fan. T-bfa: Targeted bit-flip adversarial weight attack. arXiv preprint arXiv:2007. 12336, 2020b.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 2017.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 1312.6/999 2013.
Andrei Tatar, Cristiano Giuffrida, Herbert Bos, and Kaveh Razavi. Defeating software mitigatioon against rowhammer: a surgical precision hammer. In International Symposium on Research in Attacks, Intrusions, and Defenses, pp. 47-66. Springer, 2018
Torchvision. httpos///pppp..ccc///rppeeeeeeer
Accessed: 2021-05-26.
Valerio Venceslai, Alberto Marchisio, Ihsen Alouani, Maurizio Martina, and Muhammad Shafique. Neuroattack: Undermining spiking neural networks security through externally triggered bit-flips. In 2020 International Joint Conference on Neural Networks (IJCNN) pp. 1-8. IEEE, 2020.
Yuan Xiao, Xiaokuan Zhang, Yinqian Zhang, and Radu Teodorescu. One bit flips, one cloud flops: Cross-vm hammer attacks and privilege escalation. In 25th {USENIX) security symposium ((USENIX) security 16), pp. 19-35, 2016.
Fan Yao, Adnan Siraj Rakin, and Deliang Fan. Deephammer: Depleting the intelligence of deep neural networks through targeted chain of bit flips. In 29th {USENIX) Security Symposium (!USENIX) Security 20), pp. 1463-1488 2020.
A OTHER COUNTERMEASURESS
Filtering Adversarial Inputs One possible post-deployment defense is filtering the adversarial inputs as suggested by Chou et al. (202)) using GradCAM heatmaps (Selvaraju et al. (017). We use the GradCAM implementation from Gildenblat contributors (2021) to analyze the output of four sample images that are labeled as car, frog, cal and car respectively (See Fig. 4. Before the attack, the model correctly classifies all images with or without the trigger pattern. If the trigger pattern does not overlap with the major features in the image, e.g. frog and cat, the main focus of the model stays on the object. However, if the trigger pattern overlaps with the main features, e.g. the wheel of the car, the focus is shifted towards the trigger pattern. After the attack, regardless of the trigger and object overlap, the focus of the model shifts towards the trigger pattern and the model misclassifies all images to target class, bird. Therefore, although GradCAM based approach can possibly filter the adversarial inputs, it will also produce false positives even if the model is clean and works correctly.
Mitigating Rowhammer Rowhammer attack is realistic attack capable of flipping bits in the victim process, completely isolated from the attacker process. This possible because all the pro- cesses share the same memory and the attacker process can cause bit flips in the physically nearby victim process by just accessing its own memory fast enough to induce disturbance errors (Kim) ct al. 2014).
There have been many defenses proposed by the researchers and also implemented by the manufac- turers but all of them are bypassed, and the Rowhammer attack is still major threat (Gruss al. 2018). Some of the major defenses are Error Correcting Codes (ECC) chips, Target Row Refresh (TRR), and changing the refresh rate, all being bypassed.
Frigo et al. 2020) and de Ridder et al. (202I) have bypassed the TRR defense and shown that more than 80% of the DRAM chips in the market are still vulnerable to the Rowhammer attack. The ECC
12

---

Under review as a conference paper at ICLR 2022
Clean Inputs
Trigger Added Inputs
Clean
Trigger Added
Label: car Pred.:car V
cred.:car
Pred.ccar
Label: frog
Label frog
Label:
Label: frog
frog.
froge
froge
Pred.:bird X
label:
Label cat
Label cat
Pred.ccat
Pred..cat
Pred. bird X
Label:
Label: car
Label:
Pred..car
<
X
Clean Model
Backdoored Model
Figure 4: The change in GradCAM (Selvaraju et al. (017) heatmaps that belong to ResNer18 before the attack (left) and after the attack (right). The focus of the model shifts through the trigger pattern it is backdoored.
defense has also been bypassed by Cojocar et al. (019)). Mutlu & Kim (2019) have shown that changing the refresh rate defense to completely mitigate Rowhammer is impractical.
13


Reviews:
[{"review_id": "zgVsCSNXFsE", "reviewer": "Reviewer_Yjkz", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nThe work is interesting and closes the gap between theoretical backdoor attacks of this type and realisable ones. Results show that attacks are possible where 100% (Rmatch) of the required bit flips are flippable using a RowHammer attack. Nevertheless, it appears that  access to the model weights at runtime is required to perform the attack - sections 3.3 and 4.5 perhaps need strengthening to convince the reader that the threat here is real.\n\n1. what does the initial trigger mask look like? what is the size of the trigger?\n\n2. in terms of countermeasures, it appears to be the case that access is required to the weights file, i.e. ability to load into DRAM. If so, should this be added to the threat model section? Is it not easy to defend against? (would memory deduplication vulnerabilities also offer a route to do something similar?). keeping our weights file private / encrpyted appears to be a defense or couldn't we simply perform some sort of simple validation/ checksum when loading from DRAM? \n\n3. given model accuracy drops significantly after the attack on more complex models, we could also, periodically, simply test accuracy to detect attempts to modify the model? (e.g., in case other protections were circumvented)\n\n4. where are the bits that need to be flipped located precisely? can we learn anything interesting from this in terms of possible countermeasures?\n\n5. is the location of bits vulnerable to RowHammer something we could consider to help protect the model? (although this seems unnecessary given ease at which we could protect against any modifications to model weights). \n\n6. could we make changes to the model architecture to increase the number of bitflips required? What about current training-time defences to protect against adversarial inputs, are they helpful?\n  \nminor:\na) Section 3.1 \"misclassify the input to the target class\" - should there be some mention of the trigger here? The trigger gets drop in various places which perhaps makes the paper harder to follow?\nb) page 4: \"Specifically, \" - is the intention the reader then read the text in the box? The text in the box could be clearer. It is poorly written at the moment.\nc) I find the descriptions in section 4.5 difficult to follow. \n", "summary_of_the_paper": "The paper explores backdoor injection attacks on machine learning models, the attack develops in the following way:\n\n1. The model and its parameters are studied offline to determine a trigger pattern and the bits that must be modified in the target model.\n2. A RowHammer attack is used to modify the bits of the model after it is loaded into DRAM\n3. When inputs are received which include a trigger, the model will misclassify the input. In the case of \"clean\" inputs, the aim is to leave the performance unchanged.\n\n* unlike previous work vulnerable bits are sought throughout the model (not just the last layer)\n* care is taken to understand the limitations of the RowHammer attack, i.e. in reality only a small number of specific bits can be flipped \n* a new technique is used to jointly find weight modifications and the data trigger pattern (CFT with BR)\n", "summary_of_the_review": "Overall an interesting paper, but I think some additional justification for why such attacks are not easily defended against is perhaps required.  Clarity could be improved in places. Given this I believe the paper is currently below the acceptance threshold.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "xrRMNt-UVD", "reviewer": "Reviewer_gfSk", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Comments\n1.\tThe threat model described in Section 3.1 is not compliant with the claims of the work. In particular: “we assume that the attacker… does not have access to the training hyperparameters, or the training dataset”. However, it is written several times that the training process has been modified by the attacker. How is that possible without access to the training hyperparameters and training dataset? It is also in contrast to the fact that in Section 3.2.2, the training samples are given as input to the optimization algorithm.\n2.\tThe experiments performed in Section 3.2.1 constitute the main motivations to design the proposed attack. However, the experimental setup used for Rowhammer profiling is not clear. This should be discussed more comprehensively.\n3.\tIt is not clear how the property of no co-occurrence in the same memory page among the flipped bits can be guaranteed. Does it assume the knowledge of the exact weight memory allocation?\n4.\tAuthors are encouraged to discuss the possible countermeasures that can be potentially adopted to defend against their proposed attack.\n\n\nRelated work suggestions\n1.\tV. Venceslai, A. Marchisio, I. Alouani, M. Martina and M. Shafique, \"NeuroAttack: Undermining Spiking Neural Networks Security through Externally Triggered Bit-Flips,\" 2020 International Joint Conference on Neural Networks (IJCNN), 2020, pp. 1-8, doi: 10.1109/IJCNN48605.2020.9207351.\n", "summary_of_the_paper": "This paper proposes a method for realizing backdoor attacks for DNNs based on the Rowhammer attack. ", "summary_of_the_review": "This work is promising and contributes to generating successful backdoor attacks in practical settings with good results. However, there are some concerns that affect the clarity of the method and of the threat model.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "d_tEwnyC55", "reviewer": "Reviewer_Yjj7", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "* Design and correctness\n\n  The design of this attack is mostly based on heuristics without theoretical\n  supports or analysis, e.g., how it identifies vulnerable weights. Its\n  evaluation results are on two datasets, CIFAR and ImageNet, which I am not\n  convinced to be enough to demonstrate how it can generalize to others.\n  Moreover, its experiment design lacks an ablation study.\n\n* Usage of clean data.\n\n  The proposed constrained fine-tuning process requires clean data to inject\n  backdoors. However, in Threat Model Section (i.e., Section 3.1), such\n  assumption is not mentioned. Moreover, it seems to be contradictory with the\n  claim that it does need training dataset.\n\n  Also, details of the used clean data in the injection process are missing. For\n  example, the number of clean samples used in the constrained fine-tuning\n  process.\n\n* FGSM\n\n  Existing work has studied methods to generate/reverse-engineer triggers. I do\n  not understand the logic of using FGSM instead. Are there any special\n  benefits, compared with existing work? Existing work [1] also simultaneously\n  optimizes the trigger patterns and model parameters.\n\n  Also, some details of the trigger generation process is missing, e.g., the\n  size of the trigger and the perturbation budget of FGSM. \n\n* Locating vulnerable weights\n\n  For locating vulnerable weights, this paper relaxes the combinatorial\n  optimization problem and heuristically locate these weights by simply ranking\n  the parameters' gradient. Are there any evidence or theoretical analysis to\n  support this design?\n\n* Hyper-parameters\n\n  This method has several hyper-parameters, for example, the loss balancing\n  parameter \\(\\alpha\\) in Equation 1, the size of the generated trigger, and the\n  perturbation budget for FGSM.\n\n* Countermeasures\n\n  Potential defenses are only discussed but not validated.\n\n* Ethics\n\n  The paper does not have an ethical statement, despite it is proposing an\n  attack.\n\n* Relevance\n\n  To me, this paper fits security conferences better. I do not see the\n  significance of this appearing in a AI/ML venue.\n\n[1] Pang, Ren, et al. \"A tale of evil twins: Adversarial inputs versus poisoned\nmodels.\" Proceedings of the 2020 ACM SIGSAC Conference on Computer and\nCommunications Security. 2020.", "summary_of_the_paper": "This paper proposes a method to inject backdoors into real-life deployments of\ndeep neural networks on hardware. More specifically, it proposes a constrained\nfine-tuning process to inject backdoors by locating vulnerable weights and\nupdating these vulnerable weights in hardware. Experiment results on\nhardware-deployed models show the effectiveness of the proposed method.", "summary_of_the_review": "To me, its contribution to AI/ML is not significant. Many designs are just\nheuristics. It is more like a security engineering paper whose study object is\nDNN. Also, it falls short in evaluation.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "81R2QUkTkut", "reviewer": "Reviewer_erCz", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The paper is well-written and easy to follow.\n- The targeted problem is interesting since not many prior work focus on the implementation of the attack using RHA inside the DRAM.\n- The contributions of the paper are mainly in steps 2 and 4 of the algorithm where the authors enforce block-sparse flipping and only 1 bit change in a byte. While these objectives are not directly optimized for and are merely enforced by heuristics, they are still of interest. \n\nWeaknesses and questions:\n- The method in [1] also emphasizes on implementation of the attack and proposes different techniques to ensure the attack can be realized in the DRAM. Can the authors please provide a 1-1 comparison with this work and explain how they improve/change compared to this prior work?\n- Since the appearance of bitflip attacks, many papers have focused on defense strategies [2-6]. Therefore, it is important to evaluate the proposed attack in face of existing defenses, preferably bypassing them successfully and/or better than prior attacks. While section 5.4 briefly goes over one defense, it merely shows that the evaluated defense inherently has a high false positive rate. However, this is a downside (corner case) for the defense strategy and not something that is specific to the proposed attack. Without proof of resilience against contemporary defenses, the attack cannot be impactful, even though it is realizable physically, thereby defeating the purpose.\n- The threat model is a bit misleading, specifically since it claims no access to training data while the optimization steps require data for trigger generation, finding the weight bits, and adversarial fine-tuning. Can the authors please specify the source and amount of the data used for these steps?\n\n[1] Yao, Fan, et al. \"Deephammer: Depleting the intelligence of deep neural networks through targeted chain of bit flips.\" 29th {USENIX} Security Symposium ({USENIX} Security 20). 2020.\n[2] Z. He et al., “Defending and harnessing the bit-flip based adversarial weight attack”. CVPR 2020. \n[3] J. Li et al., “Defending bit-flip attack through DNN weight reconstruction”. DAC 2020\n[4] Y. Li et al., “DeepDyve: Dynamic verification for deep neural networks”. ACM CCS 2020.\n[5] Q. Liu et al., “Concurrent weight encoding-based detection for bit-flip attack on neural network accelerators”. ICCAD 2020.\n[6] J. Li et al., “Radar: Run-time adversarial weight attack detection and accuracy recovery”. DATE 2021", "summary_of_the_paper": "This paper proposes to integrate physical viability inside the attack objective for backdoor attacks. Specifically, the authors observe a hardware limitation for the row hammer attack, which limits the possible locations where the bits can be flipped. Consequently, they change the attack objective to target sparse locations to ensure that RHA can be used.", "summary_of_the_review": "Please see the main review.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The work studied the problem of inserting backdoor into a deployed model through bit flip.

Some important concerns have been proposed by reviewers, including: the incorrect claim of the treat model, the potential defenses are only discussed but not validated, experimental setups and analysis (e.g., the sensitivity test of hyper-parameters). Although the authors provided some responses, but all reviewers are not well convinced. 

After reading the manuscript, reviews and discussions between reviewers and authors, I think this work is not ready for publication. The reviewers' comments are supposed to be helpful to improve this work.
Decision: Reject