Bit-by-Bit: Investigating the Vulnerabilities of Binary Neural Networks to Adversarial Bit Flipping
Authors: Shamik Kundu
Sanjay Das
Sayar Karmakar
Arnab Raha
Souvik Kundu
Yiorgos Makris
Kanad Basu
Venue: Accepted by TMLR ()

Abstract: Binary Neural Networks (BNNs), operating with ultra-low precision weights, incur a significant reduction in storage and compute cost compared to the traditional Deep Neural Networks (DNNs). However, vulnerability of such models against various hardware attacks are yet to be fully unveiled. Towards understanding the potential threat imposed on such highly efficient models, in this paper, we explore a novel adversarial attack paradigm pertaining to BNNs. In specific, we assume the attack to be executed during deployment phase, prior to inference, to achieve malicious intentions, via manipulation of accessible network parameters. We aim to accomplish a graceless degradation in BNN accuracy to a point, where the fully functional network can behave as a random output generator at best, thus subverting the confidence in the system. To this end, we propose an Outlier Gradient-based Evolutionary (OGE) attack, that learns injection of minimal amount of critical bit flips in the pre-trained binary network weights, to introduce classification errors in the inference execution. To the best of our knowledge, this is the first work that leverages the outlier gradient weights to orchestrate a hardware-based bit-flip attack, that is highly effective against the typically resilient low-quantization BNNs. Exhaustive evaluations on popular image recognition datasets including Fashion-MNIST, CIFAR10, GTSRB, and ImageNet demonstrate that, OGE can drop up to 68.1% of the test images mis-classification, by flipping as little as 150 binary weights, out of 10.3 millions  in a BNN architecture.

Bit- Investigating the Vulnerabilities of Binary Neural Networks to Adversarial Bit Flipping
Shamik Kundu"t, Sanjay Das Sayar Karmakar", Arnab Raha², Souvik Kundu,, Yiorgos Makris', Kanad Basu"
University of Texos at Dallas, UUniversity of Florida, IItee Corporation
Abstract
Binary Neural Networks (BNNs), operating with ultra-low precision weights, incur a sig- nificant reduction in storage and compute cost compared to the traditional Deep Neural Networks (DNNs). However, vulnerability of such models against various hardware attacks are yel to be fully unveiled. Towards understanding the polential threat imposed on such highly efficient models, in this paper, explore novel adversarial attack paradigm per- taining to BNNs. In specific, we assume the attack to be executed during deployment phase, prior to inference, to achieve malicious intentions, via manipulation of accessible network parameters. We aim to accomplish a graceless degradation in BNN accuracy to a point, where the fully functional network can bohave as a random output generator at best, thus subverting the confidence in the system. To this end, we propose an Outlier Gradient-based Evolutionary (OGE) attack, that learns injection of minimal amount of critical bit flips in the pre-trained binary network weights, to introduce classification errors in the inference execution. To the best of our knowledge, this is the first work that leverages the outlier gra- dient weights to orchestrate a hardware-based bit-flip attack, that is highly effective against the typically resilient low-quantization BNNs. Exhaustive evaluations on popular image recognition datasets including Mashion-MNIST, CIFAR1O, GTSRB, and ImageNet demon- strate that, OGE can drop up to 68.1% of the test images mis-classification, by flipping as little as 150 binary weights, out of 10.3 millions in BNN architecture. Code is open sourced at: htttss://github.comm wttp///u...ooiiinnnnnnnneee
1 Introduction
A commitment to reducing size and compute demands has led to ultra-low-preeision BNNs, featuring one- bit weights and activations (-1 or +1). Introduced by Coorrariiux et al. (2016), BNNs drastically improve power efficiency and inference latency by minimizing memory access and using fast bit-wisc operations in- stead of complex matrix multiplications. These improvements maintain classification accuracy compared to high-precision Deep Neural Networks (DNNs) (Qin et al., 2020 Yuan & Agaian [2021). BNNs are favored for Machine Learning as a Service (MLaaS) across hardware platforms (Sanyal al. 2018 Recent ad- vancements in Deep learning have integrated low-precision DNNs into critical domains like facial recognition
(Dong et al. 2019) and autonomous driving (Fykholt et al. 2018).
However, this widespread adoption of DNNs has ushered in a concerning surge in adversarial attempts. exploiting network vulnerabilities through backdoor and inference attacks (Saha et al. 2020 Xie et al. 2019 Goodfellow et al. 2014 Moosavi-Dezoon et al. (017)). In contrast to these known attack vectors, there exists a relatively uncharted territory: an innovative attack paradigm centered on the manipulation of pre-trained rather menacing threat model, assuming that the adversary possesses unrestricted access to device's memory, enabling direct parameter altcrations within deployed model to serve adversarial objectives. Given that the deployed DNN is stored
model weights (Breier al.] 2018). Executing such an allack hinges
Authors have equal contributions. This work supported by the Semiconductor Research Corporation (SRC GRC Task: 3243.001).

---

in binarized format in the memory, attackers can tamper with model parameters employing techniques like the Row-Hammer Attack (Agoyan et al. 2010 Kim et al. 2014) and the Laser Beam Attack (Selmke) et al, 2015), as illustrated in Figure ! While bit-flip attacks have demonstrated their potential to wreak havoc at the system lovel, those targeting the control path are comparatively casier to address, as they are integral to the overall system's 's integrity. Conversely, datapath altacks, which surreptitiously manipulate accuracy, operate in stealth mode, posing a significant challenge detection. An instance of misclassification during inference may be erroneously dismissed as an intrinsic network characteristic, when is, in fact, a consequence of stealthy bit-flip attack.
Existing research have demonstrated that it is fea- sible to change the model weights via bit flipping to accomplish malicious intentions (Rakin et al.) 2019
Software Stack
Adversary
Hardware Stack
Bit Flip Attack
Dataset
DRAM
2020 2022: Bai et al. (021). However, these techniques are either based on heuristic strategies identifying criti-
Under Attack
Rowhammer
[Rakin et al. 2019), or focused
cal weights in high precision DNNSs (Bai et al. 20221) Since BNNs are emerging as promising candidates to be deployed in security-critical high assurance envi- ronments, exploring the vulnerability of such low precision networks is imperative to estimate its ad- versarial impacts. These BNNs are generally apprehended to be resilient against bit flip attacks owing to their limited parameter space (Rakin et al., 2021). This resiliency may be attributed the fact. that the magnitude of error by flipping bit in BNN is minimal. Since the massive degradation is inherently circumvented in case of BNNs, existing attack strategies that exploit the large bit-width of the weights to orchestrate the attack are not effective in case of these ultra low-precision binary weights in a BNN. Such inherent tolerance can only be disrupted with a significant number of bit flips (usually, over 39x, compared to a DNN, as we have demonstrated in this paper) (Rakin et al.] 2019 2021). However, manipulating a plethora of bits disrupts the stealthiness of the attack. As a result, allacking such extremely low precision networks becomes particularly challenging !
BNN Model
Laser Beam
Cache
ALU
Figure 1:
Threat model demonstrating Bit Flip attack.
Our Contributions: In this paper, we challenge the conventional wisdom of treating BNNs to be in- herently robust against. malicious bit-flips. we devise a technique to determine diminutive set of most vulnerable binary weights in the BNN, flipping which furnishes a significant reduction in accuracy to a point where the network is deemed as a random output generator. In this direction, we propose a novel Outlier Gradient-based Evolutionary (OGE) attack framework, that systematically identifies the most. vulnerable set of binary weights to be attacked in a pre-trained BNN. To this end, we reformulate the task of obtaining the most vulnerable layers in the BNN by ranking the network loss for top-k gradient bit flips in each layer. Subsequently, the outlier gradients from the vulnerable layers are isolated to be provided as input to evo- lutionary algorithm. This accomplishes the search space optimization using an entropy-based objective loss function in itcrations. Optimization of this loss function caters to the drop in inference accuracy of the BNN. In our proposed OGE framework, the subset selection of outlier gradient weights aids in circumventing the challenge posed while attacking a BNN, by enabling the evolutionary algorithm to obtain the critical weights from this vulnerable scarch space. To the best of our knowledge, this is the first work that systematically explores the vulnerability in BNN via adversarial bit flipping, instead of a heuristic strategy. This paper specifically makes the following key contributions:
We, for the first time, have discovered the significance of outlier gradient weights in a binary context and linked their influence to substantial reductions in classification accuracy within a Binary Neural Network (BNN) due to bit flips.
Based on the understanding from the outlier gradients, we propose an Outlier Gradient-based Evolu- tionary (OGE) attack, a novel bit-flip attack framework, hat utilizes an evolutionary search approach,
1 As the weights in BNN are binarized to be represented using a single bit (+1 or -1), flipping bit corresponds weight flip in the network. Hence, we use both these terms interchangeably in the remaining paper.
2

---

coupled with an isolation forest-based outlier detection framework for furnishing the most critical set of binary weights in a BNN.
We perform extensive experiments on complex image recognition datasets including ImageNet, and evaluate the efficiency of our OGE attack on both binary convolution neural networks and binarized vision transformers. Our results demonstrate that up to 68.1% of the test images could be misclassi- fied to an incorrect class, by flipping as little as 150 binary weights, out of 10.3 millions in the BNN architecture, which is almost 84% less than the state-of-the-art bit flip attack.
2 Background & Related Works
2.1 Binary Neural Networks (BNNs)
BNNs are gaining prominence as highly promising candidates for deployment in resource-constranned edge devices, primarily due to their exceptional efficiency in terms of storage and computation. Unlike full- precision networks that demand a 32-bit floating-poin point representation, BNNs operate with one-bit weights and neuron activations, (aking the form -1 or +1. To shed light on the inner workings of this innovative approach, consider the binarization function employed in BinaryNet, state-of-the theatttt--rrttte binary convolutional neural network as introduced by Courbariaux et al. (2016). This function meticulously considers the sign bit of the real-valued variable, and its representation can be articulated as follows:
+1 x = -1 otherwise
x² = sign(())
where o² is the binarized variable (weight or activation) and the real-valued variable. This function, which binarizes network parameters into -1 and +1, paves the way for significant departure from traditional compute-intensive multiplication operations. Instead, it replaces these operations with simple one-bit XNOR operation, followed by a simple bit-counting operation for accumulation. Consequently, the Multiply- Accumulate (MAC) operation, which is highly compute intensive in the realm of neural networks, is executed with remarkable efficiency in Binary Neural Networks (BNNs). This efficiency stands in stark contrast to the computationally demanding floating doating-point operations prevalent in full-precision Deep Neural Networks (DNNs). In addition to these computational advantages, BNN also delivers substantial energy savings and remarkable speedups, with reported improvements of up to 7x (Courbariaux et al. 2016). Remarkably, these performance gains do not come at the cost of classification accuracy, as demonstrated by various studies (Yuan) & Agaian 2021). These combined attributes position BNN as an optimal solution for edge-based inference tasks, where the trifecta of efficiency, speed, and accuracy is of paramount importance. Furthermore, in addition to the computational and efficiency advantages, the inherent reduced precision in BNNs contributes to enhancing the network's resilience. To illustrate, weight of +1 can eventuate to -1, or vice-versa, thereby furnishing maximum error magnitude of On the other hand, in case of a high precision DNN, e.g., an 8-tit quantized network, bit flip in the most significant bit (sign bit) can manipulate specific weight value of +123 to -5, thus engendering massive error magnitude. This resilience extends over and above that of traditional high-precision DNNs, especially when faced with attacks aimed at injecting errors by tampering with the network parameters (as we demonstrate later in this paper).
2.2 Blt- Attacks
Memory bit-flip attacks, often executed through techniques like the Row-Hammer Attack, have garnered recognition as established threat model. This threat has been extensively investigated and documented in existing literature, as excmplified by Kim et al. (2014a); Razavi et al. (2016). Given that the pre- trained network parameters of a Deep Neural Network (DNN) reside in memory before deployment at the edge, the occurrence of such bit-flip attacks raises legitimate concerns and presents substantial challenges concerning the sccurity and reliability of DNNs. This threat becomes even more pressing in sccurity-critical odge applications, where implementing sophisticated defense mechanisms becomes formidable task due resource limitations.
3

---

Within the existing body of literature, researchers have explored various methods for injecting faults in neural networks, targeting elements such as biases, weights, and activation functions, with the ultimate aim of inducing both targeted and untargeted misclassifications during the inference process. It's worth noting that these prior techniques were primarily designed to attack full-precision Deep Neural Network (DNN) models. At the edge, however, many DNN implementations operate in quanlized precision, as emphasized by (Wu et al. (2016), rendering them inherently robust to variations in network parameters. In response to this challenge, efficient algorithms have been developed and put into practice to identify the most vulnerable bits within a quantized DNN, as demonstrated in Rakin et al. (201 2022 Bai et al. (2021). The Bit Flip Attack (BFA) utilizes heuristic progressive search strategy, wherein network layers are iteratively examined to pinpoint the layer that contributes the most to network loss for a certain number of bit flips during each iteration, as elucidated in Rakin et al. (2019). Subsequently, the selected bits are flipped within the most inftuential layer, and the model is updated, thereby undermining network accuracy. Targeted- Margeted-BBAA (TBFA) represents an extension of the BFA technique, aiming to manipulate data samples with the intent of targeting specific classes, employing the same progressive search approach, as documented in Rakin et al. 2022 Bai
et al. 2021p.
It's important to emphasize that the bit flip attacks discussed carlier primarily employ hcuristic strategies and are tailored for high-precision Deep Neural Networks (DNNs). These attacks indeed rely on exploiting computations that involve network parameters with high-precision values to achieve their intended goals, such as manipulating the network's behavior and causing misclassifications. However, it's worth noting that these attacks are not effective when directed at low-precision binary network weights, which operate using only one-bit values, as opposed to the multi-bit values present in high-precision DNNs. The unique charactcristics of BNNs, including their limited precision and binary paramcters, make them less susceptible to the lypes of attacks that target high-precision DNNs. While few recent research has focused on mitigating adversarial noise at the input of BNNs by leveraging characteristic hardware attributes, the impact of bit flip-based attacks has largely been ignored (Bhattacharjee & Panda 2020 Kim et al. 2022
3 Proposed OGE Attack Framework
In this section, we begin by defining the threat model, which outlines the standard assumptions regarding the capabilities of the adversary. We then propose a novel bit flip attack framework, pertaining to BNNs, that furnish the most critical binary weights in the pre-trained network, which when flipped, cause significant degradation in classification accuracy.
3.1 Threat Model
Within the realm of Machine Learning as a Service (MLaaS), individuals upload their DNN models to a platform that may carry inherent security risks due to the desire for increased computational resources. In this setting, DNN inferences and other applications, which may be under the control of potential attackers, often share server hardware components, such as the last-level cache and main memory. While attackers lack explicit permission to read or write data within the user's memory, the existing tools empower them to exploit side channels, thereby gaining access sensitive information. This discussion calegorizes threats into three levels based on the extent of knowledge that attackers can extract from the victim.
3.1.1 Full Knowledge Attack
A full-knowledge attack represents the most critical scenario. In this scenario, attackers can harness powerful tools, such Cache Side-Channel Attacks targeting the Last-Level Cache (LLC), as detailed in Yarom & Falkner (2014); ; Kayaalp et al. (2016), to retrieve memory data stored in the cache. Although the throughput of such a channel is limited (less than 1 MB/ /s as noted in Hong et al. (2018)), it poses a significant risk to the model, especially if sensitive data within the memory is exposed. In this context, we assume that the model's 's weights may become vulnerable to the attackers. Moreover, attackers are well-informed about the proposed defense methods, like randomized rotation, and have the capability to enhance their attack strategies if any leakage persists. Given the impracticality of a full-knowiedge attack due to the limited throughput of this channel, it remains severe but technically challenging threat.
4

---

3.1.2 White Box Attack
The attackers lack the ability to directly access data within the memory, but they can make educated estima- tions regarding the locations of the most susceptible bits, allowing them to polentially execute rowhammer attacks. There are two primary reasons for this: 1) The majority of commercial models and datasets are open source, and model users often download pre-trained models, customizing them through transfer learning [Marcelino] 2018). These customized models inherit vulnerabilities from the open-source models, effectively providing attackers with a source of information. 2) Attackers can employ hardware tools to monitor memory access patterns, as documented in Hu et al. (2020); Yan et al. (2020), enabling them to accurately deduce the model's architecture. Armed with this knowledge, attackers can locally train their own models, which would also share the same vulnerabilities as the user's model. This white-box attack approach is inherently perilous and practical, and it serves the baseline attack model for the rest of this paper.
3.1.3 Black Box Attack
When attackers are unable to acquire any knowledge about the model, an alternative approach known as the Black-box adversarial attack (Cheng et al.] 2018) becomes a viable option. In the black-box attack, the objective is to compromise the model without any prior understanding of its architecture or weight distribution. However, this method demands substantial number of attempts to be effective. One effective attack within this category harnesses the Rowhammer vulnerability and is termed the Random High- Bit- Flip Attack. In this attack, the attacker randomly selects weights within the DNN model and flips the Most Significant Bit (MSB) or sign bit. Since these al lacks are inherently random in nature, they necessilate a substantial number of bit flips to achieve a notable decrease in classification accuracy of the network, as we demonstrate later in this paper.
Following this description of our threat model, we introduce our proposed Outlier Gradient-based Evolu- tionary (OGE) attack. This attack is characterized by systematic weight selection process, which unfolds through three-step approach. The details of each step, which collectively constitute the OGE attack, will be elaborated in subsequent sections.
3.2 Gradient-based Layer Selection
A traditional BNN architecture is composed of several convolution and fully connected layers, consisting of millions of parameters. An exhaustive search from this plethora of binary weights would be computationally impractical from an adversarial perspective. To this end, we reformulate the task of obtaining the most vulnerable layers in the BNN by ranking the network loss for top-k gradient bit flips in each layer. In order to obtain the gradients with respect to the latent weights for each mini-batch in the test dataset, we adopted the cost function as standard categorical cross-entropy loss for e classes, which can be represented as:
1 Σ
C C(Y,
I(Y rr) log(p:)
oi log of
(1)
n
where of is the one-hot coded vectors and 0f Pir is the softmax probability for the 8th observation to fall in ,th class depending on the feature val values X and weights w. In particular, for classic multinomial logit regression with c classes, w is replaced by the coefficient vector and pir takes the following form:
exp 3,X, 2s=1 (x)))
o. P =
(2)
Here, X, is the feature vector. However, it has been common practice replace the simplistic multinomial logit structure Hg.X, using feature vectors by suitable neural network structure. In this paper, we consider the same with 9(X, w), assuming DNN structure. This is further extended to accommodate binary weights pertaining to BNNs. First, we demonstrate how to compute the gradients for a proxy real-valued network structure. The corresponding classification probability takes the following form:
CXP
of = Pir = P(Y = =))= Σ (N
(3)
5

---

where (X)) refers to the r-th co-ordinate of
NN) Wia (W2 "0a-1 + ba)) -)) bi
(4)
Here (N) is a d d-layer neural network with e-dimensional output and or are the corresponding acti- vations for the connection between layer and + 1. To calculate the gradient of C' in Equation | for any entry of the d layers of weight matrices, let us consider particular weight Let us fix i and consider n = case for the cost function C. We write 0o Pir exp(zr)/ Es exp(z,) where = (NNw:sXX)), Then,
5C oz,
6ūr Dr oz,
Σ sfr
Yo 89. Os
yr (0+ 1 +
Σ agr
-ya Ya
(-0r0s) 1/+ Σ U/ 0r y/- -r (5)
dz,
Dr
Ds
Thus, by chain rule, we can easily compute
&c ow as following:
8C dw
8C 8z
(6)
dzz dw
6s, as it heavily depends on network structure involving the specific Hero, we omit the detail of computing 5w weight w. Note that, for a BNN, the gradient computation is difficult, since the sign function not even continuous, let alone differentiable. However, we adopt a similar strategy, as delineated in Helwegen et al. (2019), to argue that the computation for the real-valued proxy network suffices all practical purposes.
Upon training the model, gradient calculation of all the trainable convolutional and dense layers with the test data is evaluated. The corresponding weights in each layer are ranked from the highest to the lowest, based on their absolute gradient values. To understand the rationale behind this ranking keeping the layer fixed, consider a specific example of a 3-depth neural net. In light of Equation 6 it suffices to compute dz. 8w
We rewrite z as:
2 = W111 + bi, U1 = 01 (12), U/2 = W2 + b2.
Now choose w to be one of the entry from W2, say W2.1.1 The entries of 62
vector looks as following:
1.
8zg dz d0, (w2) 802
(8)
81221.1
oW,
duz
8w2,1,1
This calculation clearly shows that for all entries of Wz matrix, the other two factors will remain the same; whereas if we choose a weight from a different layer, then the number of components will vary significantly, making two different layers seemingly incomparable. Thus, for our loss/cost gradient computations in Equa- tions and we select the weights having the highest absolute gradient values to flip.
Therefore, in this step, top-k (R is pro-determined integer value) gradient weight bits of cach layer are flipped simultaneously, while keeping all the other layers unaffected. The corresponding effect the network is evaluated based on loss (Equation D Similarly, we calculate these losses for all the other layers independently. Next, the layers are ranked to detect those that contribute the most network loss. This process exccuted with different k-values and the results are analyzed to detect the most vulnerable layers, as demonstrated later Sontion 4.2.1
3.3 Outlier Weight-subser Selection
Once the vulnerable layers are detected, a weight subset is selected from those layers to be provided as input to the next step in the evolutionary framework, as described in Section 3.4. To this end, isolation forest, an outlier detection technique on multivariate data (Liu et al. 2008), is applied on the weights layer to find these weights with outlying absolute gradients.
In this step, we iterate the isolation forest applied on the computed 8C/6w at the final v. For every mini- batch by and bit i, we compute 5Cx/ōwl ( for E (I,...n..... nn..k (1...., B} where by denotes flipping the
6

---

Algorithm Evolutionary Oplimization
Input: Generated Outliers locations W [W1,"" ... Wool MarGen, P.Q.R.S
Output: ResultSolution
1: Lisution = LiptyList
2: for in L...Q do
Solution = choose P weights randomly from W
Append Solution to SolutionList end for
while MarGen + do MazGen MarGen- -
7:
8: LossList = EiplyList
9: for in 1....Q do
10:
Loss = Calculate
11:
Append Loss LossList 12: end for
13: Sort SolutionList in descending order using LossList SolutionList = Solution R
14:
15:
for in 1.... do
16:
al = Random choice(SollutinnL.a)
17:
a2 Random Choicc((Solutioniize
18:
W Set (al + a2)
19:
Solution choose P items randomly from W
20:
Append Solution to SolutionList
21: end for
22:
Q=R+S
23: end while
24: ResultSolution
CobutionListig
i-th bit of the final gradient vector tà E _ and 6C2/ôw denotes the gradient computed for k-th mini-batch. Here, C from Equation will be modified by replacing the index belonging in the specific mini-batch. For B(yy(=))
any bit and batch b, we compute the outlier score 8,,6 as follows: 8n,6 2 <(m)
where (x)( is the
average tree height across the isolation forest for when the data input is restricted to the batch b and
c(n) = 2H 2(n 1)/n, with H, denoting n-th harmonic number. For each fixed bit we define the aggregated outlier-based frequency score as:
$f2 # (b: 8z,6 2 8yb6 for all y + x).
(9)
The scores sfi,... sff are sorted in descending order as:
$f(n) > sf(n-1) > sf())
(10)
We denote their corresponding bits as Uni ".- U1, where this nomenclature means that Un the vulnerable bit, Un-1 is the second most, and
We obtain specific number of outliers by modifying the contamination parameter in the algorithm. The value of the parameter depends on the size of the weight subset. For instance, if we want to select weightss from the total number of weights N, then the contamination parameter provided kg = no/N and we select (Va, ... Un- [n-no+1]- Once these vulnerable weights are selected, we proceed the following step, where this subset of weights is provided as input the evolutionary algorithm.
3.4 Weight Search Space Optimization
Inspired by Darwinian natural evolution, various algorithms have been designed solve constrained and un- constrained optimization problems with an adaptive heuristic approach based on natural selection (Vikhar 2016 Slowik & Kwasnicka, 2020 The randomness associated with generating solutions, coupled with

---

their qualitative ranking and ability to obtain the acceptable solution within a limited search space makes this evolutionary approach an algorithm of choice over other conventional optimization techniques Ka- chitvichyanukul 2012).
In this paper, we design an evolutionary algorithm for optimizing a search space to contain the most vul- nerable binary weight indices. The outlier-set detected in Section 3.3 is taken as the solution space for the optimization. We consider the cross entropy loss in Equation ① as the fitness function and evaluate it for each generated solution by flipping all its binary weight bits simultancously.
The proposed evolutionary optimization approach is outlined in Algorithm [1 The subset of selected binary weight indices (W) and the variables (defined as follows) MazGen, P.Q, R, and S are provided as inputs to the alm.rithm. First, the evolutionary algorithm generales Q solutions by randomly choosing P weight indices from W for each solution (tines 1-4). Following this, the network loss (fitness value) for each generated solution is evaluated and collected in a list (LossList) (lines 8-11). Specifically, the Calculate is computed by fitting the Neural Network Y, X, (N,, b) as argument in Equation [ where the weight bits in specific solution are flipped. Thereafter, the solutions are ranked based their fitness values and the best R solutions are kept for the next iteration (lines 13-14), These R solutions are then leveraged to generate S new solutions by randomly selecting two solutions (a1, o2) from the entire set of R solutions. The weight indices contained in these solutions are pooled together (W"), from which P items are randomly selected to generate the new solution (lines 15-21). Here we adopt a traditional two-parent solution in accordance with existing research (Lee, 2002 Sivanandam & Deepa 2008). Subsequently, the total R + S solutions are used for the next iteration (line 22). The algorithm execution stops after MarGen iterations, to obtain solution that contains the most critical binary weights in the network (lines 6-7, 23).
4 Experimental Results
4.1 Experimental Setup
To assess the effectiveness of our proposed OGE attack framework, we have chosen to employ the widely used Binary Neural Network architecture, BinaryNet (Courbariaux et al., 2016). This network has been trained on three distinct image recognition datasets: Fashion-MNIST, CIFARIO, and GTSRB. The initial classification accuracy obtained upon training with these datasets is as follows: Mashion-MNIST yields a baseline accuracy of 92%, CIFARIO achieves 80%, and GTSRB reaches an impressive 98%. To gauge the efficiency of the OGE attack scheme, we employ the following key metries: the number of bit flips (Nmp) and Relative Accuracy Drop (RAD). The RAD quantifies the classification accuracy of the network after it has been subjected to NaiP bit flips. This metric is evaluated in relation to the baseline classification accuracy of the specific network-dataset configuration under consideration. Our objective is to identify the minimum number of weight flips (Nmi)) that maximally degrades the classification accuracy of the BinaryNet architecture, thereby revealing the vulnerability of the model to this form of attack.
4.2 Efficiency of the Proposed OGE Attack
4.2.1
Gradient-based Layer Selection
In this experiment, we vary the parameter k in selecting top-k gradient values from each layer of the BinaryNet architecture. Subsequently, by flipping the corresponding binary weights, we observe the network loss to obtain the most vulnerable layers for each network-dataset configuration. The results are outlined in Figure 2 As demonstrated in Figure 2a] top-200 weight flips furnish a cumulative network loss of 0.79 and 0.67 in layers conv1 (first convolution layer) and fc3 (last fully connected layer), respectively, on the MaShion-MNIST dataset. Except these two, all other layers in the network furnish a loss below 0.5, which is demoted in the figure as Other Layers. The network loss increases with k; BinaryNet furnishes losses of 2.31 and 1.22 for layers convl and fc3, respectively for top-800 weight bit flips. An identical trend is observed for the other two datasots, as shown in Figures 26] and [cc Since the layers conel and fc3 exhibit the maximum network loss compared to all other layers in the network, we consider the binary weights from these two layers of the network to obtain the critical weight bits, flipping which will accomplish the adversarial intent.
8

---

2.50
Cara
2.5
Cand
FCB
1.0
Losses
1.75
Looses
Losses
0.75
Number of weights (k)
Number weights (K)
Number of weights (k)
(a) Dataset: Fashion- MNIST
(b) Dataset: CIFAR10
(c)
Dataset: GTSRB
Figure 2: Variation of network loss for increasing top-k bit flips across different layers in Binary Net.
Heletive accurary (5)
Dallatie acouraty drop
17
Relative
2001 Dithor
200 curlen Octieer
2763 DUNIN
2802 Outlers
2008 OuS
Number teratios
Number ltarations
Number keration
(a) Dataset: Beahion-MNIST
(b) Dataset: CIFAR10
(c) Dataset: GTSRB
Figure 3: Variation in Accuracy w.r.t. iterative generations for different number of outlier gradient weights. 4.2.2 Weight-subset Selection using Isolation Forest
In this experiment, we vary the subset of binary weights (n) to selected from the vulnerable layers of the network, obtained in Section 4.2.1] The contamination parameter that determines the cardinality of the subset is varied to obtain the weights with outlier gradients. Correspondingly, this binary weight subset is utilized to analyze the efficiency of the evolutionary search algorithm, by providing this subset its input solution space. Figure 3a demonstrates the solution optimization space that furnishes the increase in network loss and reduces the accuracy with increasing number of outlier gradient weights on Fashion-MNIST dataset. After 100 iterations, the network exhibits a RAD of 55.32% with 1000 outliers and 300 bit flips. Under identical bit flips, the RAD reaches to 50.56% and 47.34% for 2000 and 3000 outliers, respectively, We observe similar trends for CIFAR10 and GTSRB datasets as well, as represented in Figures 3b and 3c respectively. The solution with 1000 outliers saturates the search optimization algorithm faster than the remaining outlier sets, and furnishes higher reduction in accuracy compared to the rest.. Hence, we choose this configuration to be provided input to the evolutionary algorithm in the subsequent step.
4.2.3 Weight Search Space Optimization
In this section, we first showcase the impact of tuning the evolutionary algorithm, highlighting its capability to optimize the attack strategy. Subsequently, we present the effectiveness of the attack across a range of scenarios involving different numbers of bit flips within the Binary Neural Network (BNN) architecture, illustrating its capacity to induce significant reductions in classification accuracy.
Tuning the Evolutionary Algorithm: In this experiment, vary the maximum number of itera- tions (MazGen) to execute the evolutionary algorithm, as discussed in Section 3.4 Setting appropriate MarGen factor is highly crucial, since carly termination of the evolutionary algorithm will result in a non- optimal solution. Fig. 4h demonstrates the variation in accuracy drop with the increase in the nunber of iterating generations for all three datasets, As outlined the figure, on Fashion-MNIST dataset, a solution of size containing 200 weight indices from both the convl and fc3 layers, when flipped, furnishes RAD of 36.85% after 10 iterations, which further increases to 50.9% after 80 iterations. Similarly, for CIFAR10 and GTSRB datasets, we observe analogous trend in the accuracy drop. The reduction in accuracy

---

saturates beyond 80 iterations in all three datasets. This motivates us to utilize 80 MorGen iterations for the evolutionary algorithm.
Accuracy Drop with Varying N Nap: In this experiment, we vary the number of binary weights (Nap) to be flipped, as obtained from our proposed OGE attack. As shown in the Figure pp, the accuracy relatively drops 31.5% for 100 N tiip in Fashion-MNIST dataset, which further increases to 64.56% by flipping 400 weights, out of 10.3 million parame- ters in the model. CIFARIO furnishes a similar accuracy re- duction trend to reach 50.625% RAD for 400 Naip- Similarly, With GTSRB, on flipping 400 binary weights, the BinaryNet furnishes only a minimal classification accuracy of 0.625% with RAD of 99.4%, at which the model can be termed as random output generator at best. Therefore, our proposed OGE framework is able to subvert the confidence of the sys- tem, thereby demonstrating the efficiency of the adversarial bit flip attack.
CFAR3
67588
[b)
Reletive accuracy drop
20
Number iterations
Number of weights
Figure 4: Relative Accuracy degradation for varying number of (a) iterations of the evolu- tionary algorithm and (b) weight bit flips in the OGE attack.
4.3 Comparison with State of- ahe
In this section, we compare the OGE attack with existing attack strategies, as captured in Table [ We first compare our strategy with a random bit flip attack that randomly selects weights from the BinaryNet architecture. The corresponding reduction in accuracy is observed for all three datasets. In the case of Dashion-MNIST as high as 50000 random bit flips furnishes a minuscule 0.2% degradation in accuracy, while our proposed OGE approach, with only 200 bit flips, renders relative accuracy drop of 54.3%. Identical trends are observed for the other two datasets well. Therefore, random bit flips are ineffective in reducing the confidence in BNNs, which can be attributed to their inherent robustness arising from limited parameter space.
Table 1: Comparing the efficiency of the proposed OGE attack on BNNs against existing adversarial bit based attacks.
Dataset
O-Aco(%)
Methods
Affocted vulucrable layers
Neip
PA-Aco(%)
RAD(%%
Random attack
All layers
50000
91.9
0.2
Rishion-MISST
92.07
BFA Rakin et al. 2019 2019
convi, 1c3, convá
500
45.2
50.8
OGE
conyl, 1e3
200
42.1
54.3
Random attack
All layers
50000
80.05
0.3
CIFARIO
80.3
BFA (Rakin et al. 2019
conv1, fc3, conv2, fel
740
38.1
52.5
OGE
conyl, fe3
400
39.5
50.6
Random attack
All layers
50000
98.5
0.2
GTSRB
98.7
BFA (Rakin et al. 2019
convl, Ic3, Icl, conv3 convi, 1c3
950
42.8
56.6
OGE
150
31.48
68.1
The state-of-the-ary technique in the domain of un- un-targeted adversarial bit flipping in DNNs is proposed in Bit flip Attack (BFA) (Rakin et al.) 2019). BFA iteratively searches for the most vulnerable layer in the network in a progressive heuristic manner using gradient-based bit flips. Next, on flipping the weight bits in the chosen layer, the model is updated with the modified weights and used for the next iteration. Thus, network loss increases with each loop, which in turn, degrades the overall classification accuracy of the network. We adopted this BFA strategy to attack BNNs, and compare its efficacy with our proposed OGE attack. While BFA requires approximately 500 binary weight flips to obtain relative accuracy drop (RAD) of 50.8% in the case of Fashion-MNIST, the OGE approach achieves higher accuracy reduction with only 200 bit flips. We observe an identical Irend for CIFAR10 dataset as well. The elficiency of our proposed OGE attack is best observed in the case of GTSRB dataset. While BFA requires 950 bit flips to furnish 56.6% accuracy, our OGE attack engenders a much higher RAD of 68.15 18.1% with a minimal 150 bit flips in the BinaryNet architecture. Hence, with almost 84% less bit flips, OGE attack achieves even higher degradation in network accuracy, compared to the state-of-the-art, which demonstrates the efficiency
relative dogradation
10

---

of our proposed attack. The proposed OGE framework furnishes attack runtime advantage of 3.56x average over this state-of-the-art BFA method.
4.4 Alternate BNN Architectures on ImageNet
In order to evaluate the prowess of our proposed OGE attack framework, we have performed experiments recent state-of-the-att BNN architectures(as demonstrated in Figure 5), that are trained ImageNet dataset (Deng et al. 2009). The corresponding baseline classification accuracies
73.6% for QuickNet, 58.7% for XNORNct, 65.4% for BiRealNet, 61.38% for LQNet (RosNet-18), 62.73% for BinaryResNet18 and 60.6% for Binary Net. Quicknet demonstrates the highest. vulnerability among all the networks, owing lo inherent model attributes arising from depthwise separable convolutions and parametric ReLU operations. XNORNet, on the contrary exhibits highest resilience against OGE bit flip attack. This can be attributed the approximate convolutions using binary operators, that mask the impact of attack on the ensuing classification accuracy of the network. The proposed attack, when compared against BFA and random bit flip attacks, furnishes the highest RAD over the other two attack strategies, thereby demonstrating the efficiency of our proposed approach.
0.0
Relatve securacy drap (50
Relative accurscy drop
80
Relattv securacy drop
56
186
150
289
200
209
250
409
Number of weights
Number of weights
Number weights
(a) QuickNet ImageNet
(b) XNORNet ImageNet.
(c) BiRealNet ImageNet
(96)
057
Pellilive accuraty drop (%5)
OGt
oct
Relative scouracy drop
199
Number of weights
Number of weights
Number weights
(d) LQNet on ImageNet
(e) BinaryResNet188 on ImageNet
(f) BinaryAlexNet on ImageNet
Figure Efficiency of OGE attack on state-of-the-art BNN architectures trained ImageNet.
4.5 Evaluations Novel Architectures
To evaluate the effectiveness of OGE novel architectures like transformers used in natural language processing (NLP) tasks such sequence classification, implement the framework a Binary precision BERT architecture, termed BinaryBert (Bai et al. 2020). We analyze BinaryBert's performance MRPC and MNLI datascts (Wang et al. 2018). We assess performance using relative accuracy drop (RAD) and relative drop in F1 score, indicating the model's performance degradation. Higher Fl and RAD drops signily lower model performance. Our attack scheme significantly reduces model accuracy, achieving a 50.8% RAD in Binary-Bert on the MNLI dataset with just 1500 bit-flips (0.000018% of total parameters). Additionally, it consistently yields over 90% relative drop in F1 score, with bit bit-flips totaling less than 0.000014% of total parameters. This highlights the effectiveness of OGE.
11

---

4.6 Effectiveness Against State-of-the-Ar (SOTA) Defences
We have identified SOTA defenses against bit-flip attacks on deep neural networks (DNNs) (He et al. 2020;
Ozdenizci & Legenstein 2022 Wang et al. 2023 Chitsaz et al.
These approaches aim to safeguard full-precision DNNs by either quantixing the model (He et al. 2020: Chitsaz et al. 2023), encrypling the output coding scheme (Ozdenizci & Legenstein 2022). 3, or modifying the model architecture (Wang et al, 2023 However, in the subsequent discussion, we elaborate why these defenses prove incffective against our proposed OGE attack framework designed specifically for targeting binary neural networks (BNNs).
2023
The approach outlined in He et al. (2020) suggests using binarization-awan training to convert DNNs into binary form as a defense against bit-flip attacks. However, the OGE framework has shown that binary neural networks can still be effectively attacked with minimal bit-flips, rendering this defense strategy insufficient against our proposed attack. On the other hand, the defense method proposed in Özdenizei & Legenstein (2022) employs an output code-matching strategy to make targeted class attacks more challenging. Yet, the OGE framework takes broader approach, targeting all classes, making the defense factic ineffective against our OGE attack. Similarly, the Aegis defense strategy advocated in Wang et al. (2023) proposes a multi-exit approach using multiple internal classifiers. However, if the adversary targets the initial layers before any internal classifier, erroneous activations occur across all classifiers, rendering the Aegis defense ineffective against our OGE strategy. Furthermore, the defense mechanism outlined in Chitsaz et al. (2028) suggests using learned quantization of DNN layers to enhance resilience during inference. Despite this, our study, OGE, reveals that even fully binarized models remain vulnerable to bit-flip attacks. While a defense strategy based multiple quantization levels may not directly apply to our investigation, considering its implementation is plausible given that targeting BNN represents the most extreme scenario.
4.7 Comparison with Recent Bit-flip Attack Baselines
The research outlined in Chen et al. 2021 Bai et al. (202)) illustrates Trojan attacks on DNNs employing bit-flip methodologies. These attacks pinpoint critical bits within DNN, which, when attacked by metic- ulously crafted Trojan patterns, lead to misclassification intentions. In the search for these critical bits within the model, these works rely on the magnitude of high precision (Float16, float32, int8, etc.) weights, which certain optimization techniques exploit identify these crucial bits. Conversely, BNN weights are solely restricted to +1 or -1, which are represented by a single bit. This renders the optimization techniques utilized in previous studies ineffective in converging and uncovering critical bits within BNNs. Additionally, the attacks discussed in Chen et al (2021); Bai (022) are targeted attacks, aiming at input patterns of specific classes or patterns, which causes the overall model accuracy drop to be minimal. However, in our OGE strategy, the allack largets all classes indiscriminately throughout, resulting in substantial drops in model accuracy, and thus demonstrating the efficacy of the attack.
into alternative target classes, serving adversarial
ct al.
5 Ablation Studies
In order to evaluate the efficiency of our OGE attack framework, we performed an extensive ablation study by varying the different design parameters of our algorithm. As summarized in this section, OGE attack furnishes identical trends for various ablation studies on all three datasets Fashion-MNIST, CIFAR10 and GTSRB. The corresponding results are demonstrated in Figures 6 and respectively.
Ablation study for layer selection: In this experiment, we performed ablation study for layer selection, as demonstrated in Figure Tal for CIFAR10 datasct. We observed that, executing OGE attack on selected layens furnishes much higher degradation in classification accuracy, when compared to a scenario where the altack is executed without layer selection. Tdentical results are obtained for other two datasets as well. This demostrates the importance of layer selection in order to orchestrate an effective bit flip attack on the BNN,
Ablation study on outlier detection: In this experiment, we varied the outlier detection approach to select a subset of the binary weights, by considering different techniques (1) random weight selec-
12

---

6it Laye Sotcles
Rotative accuarr drip 1NN1
Ony Fil Carrades
Namber of Iterations
Number The incalions
Number of itterationss
(a) Layer Selection
(b) Outlier Detection
(c) Optimization Algorithm
Figure 6: Ablation studies performed by varying OGE design parameters for Bushion-MNISS dataset.
Selecton pay seacciio
Jander
Searr optimiatior osemaziin
Featthe sccuncy drop
66
139
28
iterattonss
(a) Layer Selection
(b) Outlier Delection
(c) Optimization Algorithm
Figure 7: Ablation studies performed by varying OGE design parameters for CIFAR10 dataset.
tion, (2) weights with top-k gradients, (3) gradient based outlier detection with One-class SVM (OCSVM)
with standard rhf kernel and, (4) Tsolation Forest, that originally used in the proposed OGE attack.
As demonstrated in Figure /b outlier detection with isolation forest exhibits the highest RAD, compared to other weight sub- set selection techniques. When evaluated on
One casa 'pdy
One casa "apneld""
20
and GTSRB datasets, OGE demonstrated similar results for the isolation forest technique, thereby demonstrating the effi- ciency of the proposed approach.
Relative accuracy
20
Furthermore, we performed experiments with different kernels of OCSVM (1) poly, (2) sigmoid and (3) rbf, and observed that rbf kernel furnishes the highest RAD among other two kernels in OCSVM (as demonstrated in Figure g of the Ap- pendix). However, our isolation forest method furnishes even higher degradation in accuracy, when compared to OCSVM method with r6f kernel.
3.-
Number Rteraionn
Figure 9: Variation in classification accu- racy under OGE attack, for different kernels of OCSVM, which is compared with the performance of isolation forest technique.
Ablation study with optimization algorithms: We also performed an ablation study on 3 optimization techniques (1) Particle Swarm Optimization (PSO) and (2) Grey Wolf Opti-
mization (GWO) along with the proposed (3) evolutionary op- timization algorithm, as shown in Figure 7c The evolutionary
approach furnishes the lowest RAD over 100 iterations, compared to PSO and GWO respectively across all three datasets. Similar results are obtained, when we iterated identical experiments on Fashion-MNIST and GTSRB datasets as well.
These comprehensive ablation studies conducted serve substantiate our design choices for the proposod OGE attack framework, which has proven its efficiency in targeting and compromising the robust binary weights found within BNN.
13

---

Nit Laye
Retative acourry dra 1V
Nambee of Iterations
Number THe aieaaion
Number of itteaation
(a) Layer Selection
(b) Outlier Detection
(c) Optimization Algorithm
Figure Ablation studies performed by varying OGE design parameters for GTSRB dataset.
Sign
Road Miclessikatoon
Detected Limit:
SPEED LIMIT 35
Accidents resulting in tuge loco
DENZE TRIE AIRIC
A W
K@W
A
Original Limit
(a) Original Cirouit
(b) Logic Locked Circuit
(a)
(b)
Figure 10: (a) Impact of Bit. Flip attack and (b) the proposed defense against OGE atlack on BNNs.
6
Discussion
Since a BNN demands extremely low amount of resources at the edge, it has the potential to be deployed the next generation mission-critica applications, e.g., in an autonomous vehicle, as represented in Figure [0a] The decisions ascertained by this network, for instance, detecting a street sign with a speed limit of 35 MPH, enable the automotive drive intended. However, under such an adversarial bit-flip attack, the network might incorrectly infer the limit to be 85 MPH, which would direct the vehicle to reach uncontrollable speeds. Since the decisions furnished by such networks are beyond human control, the impact of such attacks end in catastrophic circumstances, including the loss of human lives. Therefore, it is imperative to explore the vulnerabilities in such extremely low-precision networks and address the manifestation of adversarial attempts that can subvert the confidence of the system.
In order thwart such attacks, we propose an adaptive defense mechanism. When manifested in a network, this defense strategy aims to jeopardize the underlying strategy of the OGE attack and hence is termed adaptive. The proposed defense utilizes the concept of XOR-cipher, inspired by the concept of logic locking (Yasin & Sinanoglu, 2017). Once the BNN is trained, critical subset of vulnerable BNN parameters can be encrypted with XOR-cipher, which, on providing the correct set of keys, will furnish the original accuracy during inference. However, an incorrect key set will exhibit a graceless degradation in accuracy. There- fore, the attacker, without knowing the key, will obtain a network having extremely low baseline accuracy, beyond which, attacking the network further impractical from an adversarial perspective. Figure 10h demonstrates high-level representation of this proposed defense strategy. As exhibited in existing research, the multiplication operation in BNN is accomplished by logical XNOR operation between the activation (A) and the binary weight (W), thereby resulting in product of A W (Courbariaux et al. 2016). With the implementation of the defense strategy, the weight W will be XOR-ed with key-bit (k), prior to its multiplication with the input activation. Hence, the product obtained can be represented as A (W + k). Reverse engineering the key is also highly improbable in this scenario; locking m vulnerable binary weights will furnish a probability of 1/2" (e.g. approximately x 10-46 for 150 encrypted weights). Hence, with this infinitesimal probability, it almost impossible for the adversary obtain the correct keys, and subse- quently, the accurate set of pre-trained network weights to execute the attack.
14

---

Bottlenecks: While the defense approach holds the promise of countering the OGE attack, there is practical hurdle to implementing this defense. The issue stems from the fact that implementing this defense would require provisions at the system level where Binary Neural Networks (BNNs) are executed. Specifically, every position where the weights are mapped would need provisions to enerypt those particular weights. Since BNNs are designed to achieve efficient multiplicatiin operations using XNOR gates, adding XOR gates corresponding to each XNOR gate would substantially increase the area and power overheads necessary for implementing this defense strategy. This addition would nearly double the resources required for processing, significantly compromising the efficiency and effectiveness of such BNNs.
The primary motivation for using BNNs is to make them well-suited for deployment at the edge, where resource constraints and efficiency are paramount. Consequently, implementing a defense strategy that introduces such substantial overhead would undermine the very purpose of deploying BNNs at the edge. This underscores the challenge in developing defense mechanisms for these types of attacks that are both effective and practical, without sacrificing the key advantages that BNNs offer. It highlights the urgent neod for innovative defense strategies that can effectively thwart such carefully crafted attacks while remaining compatible with inherently robust BNNs and practical for edge deployment.
7 Conclusion
In this work, we propose OGE, an adversarial bit flip attack designed specifically for Binary Neural Net- works (BNNs). Our attack aims to disrupt the classification accuracy of these networks, even within their low-precision settings, consequently undermining the system's confidence in its predictions. Our results demonstrate that it's possible to significantly subvert the classification accuracy of a BNN, achieving Rel- ative Accuracy Drop (RAD) of 68.1% using the OGE methodology. Impressively, this level of disruption can be achieved by flipping as few as 150 weights within a BNN. We achieve this through a systematic approach based on outlier gradients and evolutionary techniques, emphasizing the efficiency of our proposed attack. To defend against such attacks, a potential mitigation strategy could involve the use of XOR-cipher to protect a critical subset of vulnerable bits. However, it's important to note that this defense approach may introduce significant computational and resource overheads, as discussed earlier. Nonetheless, il represents potential avenue for further exploration in future research, as we continue to grapple with the challenge of securing low-precision networks like BNNs against sophisticated adversarial attacks.
References
Michel Agoyan, Jean-Max Dutertre, Amir-Pasha Mirbaha, David Naccache, Anne-Lise Ribotta, and Assia Tria. How to flip bit? In 2010 IEEE 16th International On-Line Testing Symposium, pp. 235 239.
IEEE, 2010.
Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jing Jin, Xin Jiang, Qun Liu, Michael Lyu, and Irwin King. Binarybert: Pushing the limit of bert quantization. arXiv preprinl arXiv:201..15001, 2020.
Jiawang Bai, Baoyuan Wu, Yong Zhang, Yiming Li, Zhifeng Li, and Shu-Tao Xia. Targeted attack against deep neural networks via flipping limited weight bits. In ICLR, 2021.
Jiawang Bai, Kuofeng Gao, Dihong Gong, Shu-Tao Xia, Zhifeng Li, and Wei Liu. Hardly perceptible trojan attack against neural networks with bit flips. In European Conference on Computer Vision, pp. 104-121. Springer, 2022.
Abhiroop Bhattacharjee and Priyadarshini Panda. Switchx: Gmin-gmax switching for energy-efficient and robust implementation of binary neural networks on reram xbars. arXiv preprint arXiv:2011.14499 2020.
Jakub Breier, Xiaolu Hou, Dirmanto Jap, Lei Ma, Shivam Bhasin, and Yang Liu. Practical fault attack on deep neural networks. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Commu- nications Securily, pp. 2206, 2018
15

---

Huili Chen, Cheng Fu, Jishen Zhao, and Farinaz Koushanfar. Proflip: Targeted trojan attack with progressive
bit flips. In Proceedings of the IEEE/CVE International Conference on Compuler Vision, pp. 7718-7727, 2021.
Minhao Cheng, Thong Le, Pin-Yo Yu Chen, Jinfeng Yi, Huan Zhang, and Cho-Jui Hsieh. Query-efficient hard-label black-box attack: An optimization-aase approach. arXiv preprint arXiv:1807.04457, 2018.
Kamran Chitsaz, Goncalo Mordido, Jean-Pierre David, and Frangois Teduc-Primean. Training dmns resilient to adversarial and random bit-flips by learning quantization ranges. Transactions on Machine Learning Research, 2023.
Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained tot 1 or-1. arXiv preprint arXiv:1602.0288 02830, 2016.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on compater vision and pullern recognilion, pp. 248 185..... Teee, 2009.
Yinpeng Dong, Hang Su, Baoyuan Wu, Zhifeng Li, Wei Liu, Tong Zhang, and Jun Zhu. Efficient decision- based black-box adversarial attacks on face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7714-771779 2019.
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE conference on compuler vision and pollern recognition, pp. 1625 1634, 2018.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint 0rXix:1412.660 2014.
Zhezhi He, Adnan Siraj Rakin, Jingtao Li, Chaitali Chakrabarti, and Deliang Fan. Defending and harnessing the bit-flip based adversarial weight attack. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pattern Recognition, pp. 14095 141033, 2020.
Koen Helwegen, James Widdicombe, Lukas Geiger, Zechun Liu, Kwang- Tinng-TTigggno Cheng, and Rocland Nussclder. Latent weights do not exist: Rethinking binarized neural network optimization. Advances in neural information processing systems, 32, 2019.
Sanghyun Hong, Michael Davinroy, Yigitcan Kaya, Stuart Nevans Locke, Ian Rackow, Kevin Kulda, Dana Dachman-Soled, and Tudor Dumitras. Socurity analysis of deep neural networks operating the presence of cache side-channel attacks. arXiv preprint orXiw:1810.08.4 034877 2018.
Xing Hu, Ling Liang, Shuangchen Li, Lei Deng, Pengfei Zuo, Yu Ji, Xinfeng Xie, Yufei Ding, Chang Liu, Timothy Sherwood, et al. Deepsniffer: A dnn model extraction framework based on learning architec- tural hints. In Proceedings of the Twenty-Fifhe International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 385-399 2020.
Voratas Kachitvichyamukul. Comparison of three evolutionary algorithms: Ga, pso, and de. Industrial Engineering and Management Systems, 11(3):215 223, 2012.
Mehmet Kayaalp, Nael Abu-Ghazaleh, Dmitry Ponomarev, and Aamer Jaleel. A high-resolution side-channel attack on last-level cache. In Proceedings of the 53rd Annual Design Automation Conference, pp. 1-6, 2016.
Yoongu Kim, Ross Daly, Jeremie Kim, Chris Fallin, Ji Hye Lee, Donghyuk Lee, Chris Wilkerson, Konrad Lai, and Onur Muthu. Flipping bits in memory without accessing them: An experimental study of dram disturbance errors. In 2014 ACM/IEEE 41st International Symposium on Computer Architecture (ISCA), pp. 361-372, 20. doi: 10. 1A111099SCAA 204.....014.05331100
16

---

Yoongu Kim, Ross Daly, Jeremie Kim, Chris Fallin, Ji Hye Lee, Donghyuk Lee, Chris Wilkerson, Konrad Lai, and Onur Mutlu. Flipping bits in memory without accessing them: An experimental study of dram disturbance errors. ACM SIGARCH Computer Architecture News, 42(3):361 372, 2014b.
Youngeun Kim, Hyunsoo Kim, Soijoon Kim, Sang Joon Kim, and Priyadarshini Panda. Gradient-basod bit encoding optimization for noise-robust binary memristive crossbar. In 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1111-1111 IEEE, 2022.
Shane Lee. Genetic algorithms. Orthogonal armys, 2002.
Fei Tony Liu, Kai Ming Ting, and on data mining, pp. 413-422. IEEE, 2008
Zhou. Isolation forest. In 2008 eighth feee international conference
Pedro Marcelino. Transfer learning from pre-trained models. Towards dala science, 10:23, 2018.
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEFE conference on computer vision and pattern recognilion, pp- 1765 1773, 2017.
Ozan Özdenizei and Robert Legenstein. Improving robustness against stealthy weight bit-flip attacks by output code matching. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
2022.
Haotong Qin, Ruihao Gong, Xianglong Liu, Xiao Bai, Jingkuan Song, and Nicu Sebe. Binary neural networks: A survey. Pattern Recognition, 105:107281, 2020.
Adnan Siraj Rakin, Zhezhi He, and Deliang Fan. Bit-flip attack: Crushing neural network with progressive bit search. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1211-1220, 2019.
Adnan Siraj Rakin, Zhezhi He, and Deliang Fan. Tbt: Targeted neural network attack with bit trojan. In Proceedings of the IEEE/ IEEE/CVF Conference on Computer Vision and Pattern Recognation, pp. 13198-13207, 2020.
Adnan Siraj Rakin, Li Yang, Jingtao Li, Fan Yao, Chaitali Chakrabarti, Yu Cao, Jae-sun Seo, and Deliang Fan. Ra-bnn: Constructing robust accurate binary neural network to simultaneously defend adversarial bit-flip attack and improve accuracy. arXiv preprint arXiv:2108. 13813, 2021.
Adnan Siraj Rakin, Zhezhi He, Jinglao Li, Fan Yao, Chaitali Chakrabarti, and Deliang Fan. T-bfa: Targeted bit-flip adversarial weight attack. IEEE Transactions Pattern Analysis and Machine Intelligence, 44 (11):7928-7933) 2022. doi: 10.1109 2P000//PPM....2....
Kaveh Razavi, Ben Gras, Erik Bosman, Bart Preneel, Cristiano Giuffrida, and Herbert Bos. Flip feng shui: Hammering a needle in the software stack. In 25th USENIX Security Symposium (OSuummUUEMM (USENIX Security 16), pp. 1-18, 2016.
Aniruddha Saha, Akshayvarun Subramanya, and Hamed Pirsiavash. Hidden trigger backdoor attacks. In Proceedings of the AAAI conference artificial intelligence, volume 34, pp. 11957-11965, 2020.
Amartya Sanyal, Matt Kusner, Adria Gascon, and Varun Kanade. Tapas: Tricks to accelerate (encrypted) prediction a service. In International Conference on Machine Learning, pp. 1490-4499, PMLR, 2018.
Bodo Selmke, Stefan Brummer, Johann Heysæl, and Georg Sigl. Precise laser fault injections into 90 nm and 45 nm sram-colls. In International Conference on Smart Card Research and Advanced Applications, pp. 193 205. Springer, 2015.
SN Sivanandam and SN Deepa Genetic algorithms. Introduction to genetic algorithms, pp. 15-37. Springer, 2008.
17

---

Adam Slowik and Halina Kwasnicka. Evolutionary algorithms and their applications to engineering problems. Newral Compuling and Applications, 32616))12333-12333 2020.
Praduya A. Vikhar. Evolutionary algorithms: A critical review and its future prospects. In 2016 Inter-
national Conference Global Trends in Signal Processing, Information Computing and Commanication (ICGTSPICC), pp. 261-265, 2016. doi: /1110//CGTSPPCC.21.... 799555330...
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804004400 2018.
Jialai Wang, Ziyuan Zhang, Meiqi Wang, Han Qiu, Tianwei Zhang, Qi Li, Zongpeng Li, Tao Wei, and Chao Zhang. Aegis: Mitigating targeted bit-flip attacks against deep neural networks. 32nd USENIX Security Symposium (USENIX Security 23), pp. 2329-2346, 2023.
Yonghui Wu, Mike Schuster, ZZhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint (rXi::160900844, 2016.
Chulin Xie, Keli Huang, Pin-Yu Chen, and Bo Li. Dba: Distributed backdoor allacks against federated learning. In International Conference on Learning Representations, 2019.
Mengjia Yan, Christopher W Fletcher, and Josep Torrellas. Cache telepathy: Leveraging shared resource attacks to learn (DNN) architectures. In 29th USENIX Security Symposium (USENIX Security 20), pp- 2003 2020, 2020.
Yuval Yarom and Katrina Falkner. {FLUSH+ RELOAD]: A high resolution, low noise, 13 cache (Side- Channel) attack. In 23rd USENIX security symposium (USENIX security 14), pp. 719-732, 2014.
Muhammad Yasin and Ozgur Sinanoglu. Evolution of logic locking, In 2017 IFIP/IEEE International Conference on Very Large Scale Integration (VLSI-SoC), pp. 1-6, 2017. doi: 8203496.
Chunyu Yuan and Sos S Agaian. A comprehensive review of binary neural network. ar preprinl 100.02211000600, 2021.
18


Reviews:
[{"review_id": "jsp36k4Vqd", "reviewer": "Reviewer_3iFu", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The authors conduct experiments on multiple datasets as well as architectures. The comparison with other baselines demonstrates the effectiveness of the proposed attack.\n2. The proposed attack is easy-to-follow and the proposed framework seems natural.\n\nWeaknesses:\n1. Although the authors claim the efficiency of the proposed attack, it is difficult to see the corresponding analysis of efficiency in the methodology part.\n2. The effectiveness of the proposed attack is not evaluated on adversarially robust BNNs, such as [1,2,3].\n3. The compared baselines are out-of-date. Some advanced flip attacks should be compared, such as [4,5].\n\n\n[1]. Improving Robustness Against Stealthy Weight Bit-Flip Attacks by Output Code Matching. CVPR 2022.\n\n[2]. Defending and Harnessing the Bit-Flip based Adversarial Weight Attack. CVPR 2020.\n\n[3]. Training DNNs Resilient to Adversarial and Random Bit-Flips by Learning Quantization Ranges. TMLR 2023.\n\n[4]. ProFlip: Targeted Trojan Attack With Progressive Bit Flips. ICCV 2021.\n\n[5]. Hardly perceptible trojan attack against neural networks with bit flips. ECCV 2022.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents an adversarial attack framework called OGE (Optimized Gradient-based Adversarial Bit Flip Attack) specifically designed for Binary Neural Networks (BNNs). The authors propose a multi-stage attack strategy that leverages gradient-based layer selection, weight-subset selection using Isolation Forest, and weight search space optimization through an evolutionary algorithm. The effectiveness of the OGE attack is evaluated on several datasets trained with the different BinaryNet architectures.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Please discuss the efficiency of the proposed attack.\n2. Please evaluate the effectiveness of the proposed attack on some advanced adversarially robust BNNs.\n3. Please compare with recent flip attack baselines.", "broader_impact_concerns": "I do not see any broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KH1Dy6Kmoi", "reviewer": "Reviewer_AvVk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The Outlier Gradient-based Evolutionary (OGE) attack appears to be novel to the best of my knowledge. \n- The presented attack performances show the vulnerability of BNNs to the proposed attack. \n- The introduced minimal critical bit flips to fool a BNN are interesting. \n\n## Weaknesses\n- The paper did not evaluate any efficiency against defenses. \n- This work is outdated. This work was submitted in 2024 and cites only 3 works from 2023, of which only one is related to BNNs.\n- Given the above, I doubt that the authors have truly compared against the state-of-the-art, especially since the works they compare to are from 2019. \n- The community has seen various new architectures, Vision-Transformers, MLP-Mixer, Diffusion models, etc. None of these models were tested for the proposed attack.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Binary Neural Networks (BNNs) offer reductions in storage and compute costs compared to traditional Deep Neural Networks (DNNs) by employing ultra-low precision weights. However, their vulnerability to hardware attacks remains a concern, prompting the exploration of adversarial attack paradigms targeting BNNs during deployment to degrade accuracy and undermine confidence in the system. The proposed Outlier Gradient-based Evolutionary (OGE) attack introduces minimal critical bit flips in pre-trained binary network weights, resulting in up to a 68.1% drop in test image misclassification across various datasets, with just 150 binary weight flips out of millions in a BNN architecture.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Evaluate the attack against SOTA defenses\n- Update the entire paper to the state of research of 2024\n     - This includes comparing to the latest BNN attacks\n- Provide evaluations on novel architectures (Vision-Transformers, MLP-Mixer, Diffusion models)", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "L5I1gRqs9M", "reviewer": "Reviewer_uVoe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n- The clear contribution of this work is the weight bit-flip attack algorithm that is tailored for the binary quantization case, which is new. \n- Experimental results scale to ImageNet level and demonstrate effectiveness of the proposed approach.\n- Narrative is clear and the storyline is communicated well.\n\nWeaknesses:\n- Scope of the work and the actual utility of a BNN bit-flip attack is highly limited. However I still believe this paper provides a technically valid contribution in its own niche.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Authors propose an adversarial weight bit-flip attack algorithm for binary neural networks. Former studies so far considered having low-bit quantization as a strong defense in itself, against the existing heuristic attacks on high-precision quantized NNs. This work introduces an attack tailored for the binary case, which challenges the existing belief of weight binarization to be a strong defense against such attacks. Authors' main proposal is the so-called \"outlier gradient-based evolutionary (OGE) attack\" algorithm, which mainly relies on the information of outlier gradients to restrict its layer/weight search space to detect the set of most vulnerable bits in the BNN. Several experiments are performed across various datasets and binary DNN architectures.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- An addition (e.g., table) regarding the required time of the algorithm to identify the most vulnerable bits would provide a good comparison, as opposed to BFA (not only Alg. 1, but the complete process starting from gradient-based layer selection).\n\n- In the middle of the Discussion section, suddenly the authors start proposing an adaptive defense mechanism with a logic locked circuit against this vulnerability. Perhaps this should be separated and included in a short additional subsection.\n\n- The paper sufficiently talks about existing weight bit-flip attacks on neural networks, however too little on existing defenses against these challenges. Some notable studies that the authors could discuss in their work:\n\n\"Defending and harnessing the bit-flip based adversarial weight attack.\" CVPR 2020.\n\n\"Improving robustness against stealthy weight bit-flip attacks by output code matching.\" CVPR 2022.\n\n\"Aegis: Mitigating Targeted Bit-flip Attacks against Deep Neural Networks.\" USENIX 2023.\n\n\"Training DNNs Resilient to Adversarial and Random Bit-Flips by Learning Quantization Ranges.\" TMLR 2023.\n\n- Minor comments: The reference citing style of the paper is generally inconsistent and should be re-checked (e.g., use of parantheses in Author et al. style natbib format). There are also several typos and uppercase-lowercase mix-ups here and there.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers raised several points that were addressed in the author's rebuttal. However, the authors did not upload a revision of the paper in accordance with their answers. The authors should upload a revised manuscript that includes additional results and discussions provided in the rebuttal.
Decision: Accept (poster)