Can DBNNs Robust to Environmental Noise for Resource-constrained Scenarios?
Authors: Wendong Zheng
Junyang Chen
Husheng Guo
Wenjian Wang
Venue: ICML 2025 poster (2025)
Keywords: Deep binary neural networks
Environmental noise
Robustness
Primary area: deep_learning->robustness
TL;DR: Improve the robustness of DBNNs under environmental noise.

Abstract: Recently, the potential of lightweight models for resource-constrained scenarios has garnered significant attention, particularly in safety-critical tasks such as bio-electrical signal classification and B-ultrasound-assisted diagnostic. These tasks are frequently affected by environmental noise due to patient movement artifacts and inherent device noise, which pose significant challenges for lightweight models (e.g., deep binary neural networks (DBNNs)) to perform robust inference. A pertinent question arises: can a well-trained DBNN effectively resist environmental noise during inference? In this study, we find that the DBNN's robustness vulnerability comes from the binary weights and scaling factors. Drawing upon theoretical insights, we propose L1-infinite norm constraints for binary weights and scaling factors, which yield a tighter upper bound compared to existing state-of-the-art (SOTA) methods. Finally, visualization studies show that our approach introduces minimal noise perturbations at the periphery of the feature maps. Our approach outperforms the SOTA method, as validated by several experiments conducted on the bio-electrical and image classification datasets. We hope our findings can raise awareness among researchers about the environmental noise robustness of DBNNs.

Can DBNNs Robust to Environmental Noise for Resource-constrained Scenarios?
Wendong Zheng' Junyang Chen
Husheng Guo
Wenjian Wang"
Abstract
putational resources, researchers often consider deploying lightweight models. This implies that lightweight models are typically trained on GPU servers, after which the well- trained models are deployed to edge devices for performing downstream inference tasks. particular, edge devices are typically in environments with diverse forms of environ- mental noise (Yayla & Chen, 2022). If environmental noise causes the lightweight model output wrong results, it can have serious consequences for safety-critical tasks. For examples, the smart bracelet worn by the patient introduces environmental noise perturbations, which is compounded by the patient's pain and other physiological conditions (Roeder, 2022). Thus, we aim to support robust decision- making of lightweight models under environmental noise perturbations, thereby assisting decision-makers.
Recently, the potential of lightweight models for resource-constraaned scenarios has garnered sig- nificant attention, particularly in safety- eeriticll tasks such as bio-electrical signal classification and --ultrasound-assistte diagnostic. These tasks are frequently affected by environmental noise due to patient movement artifacts and inherent device noise, which pose significant challenges for lightweight models (e.g., deep binary neural networks (DBNNs)) to perform robust inference. rell-trained DBNN effectively resist environmental noise dur- ing inference? In this study, we find that the DBNN's robustness vulnerability comes from the binary weights and scaling factors Drawing upon theoretical insights, we propose L1-infinite norm constraints for binary weights and scaling factors, which yield tighter upper bound compared to existing state-of oate--of-the-art (SOTA) methods. Finally, visualization studies show that our approach in- troduces minimal noise perturbations at the pe- riphery of the feature maps. Our approach outper- forms the SOTA method, as validated by several experiments conducted the bio-electrical and image classification datasets. We hope our find- ings can raise awareness among researchers about the environmental noise robustness of DBNNs
A pertinent question arises:
On Resource Limited Scenarios, the robust- ness of lightweight models with sparse property is compro- mised in environmental noise perturbations (Cai et al., 2022), Empirically, several researches are focused on adversarial learning schemes (Miyato et al., 2018; Gouk et al., 2021) using spectral norm constraints to improve the robustness of full-precision models. In theory, recent approaches exten- sively employ relaxation strategies based on the Lipschitz continuity theorem to establish provable bounds for full- precision convolutional neural networks (CNNs) against adversarial attack (Gowal et al., 2018; Balunovic & Vechev, 2020; Zhang et al., 2020; 2021). However, this kind of method mainly has the following three limitations. (1) These relaxation-basel methods (Gowal al., 2018; Balunovic Vechev, 2020) improve the robustness of full full-precision mod- els when deployed on GPU servers, yet they introduce expo- nential computational overhead. Thus, the aforementioned approaches (Gowal et al., 2018; Balunovic & Vechev, 2020) present significant challenges for deployment in resource- constrained enviromments. (2) The above method (Gowal et al., 2018: 1: Balunovic & Vechev, 2020: Zhang et al., 2020; 2021) exhibit effectiveness only against specific types of noise perturbations, owing to their training with specific adversarial samples. However, the nature of environmental noise that poses challenges for safety-critical tasks is diffi- cult to predict. For instance, a patient's position change due to discomfort can introduce random noise into the electrodes collecting bio-clectrical signals (Roeder, 2022). (3) The
Limitations:
1. Introduction
To enable safety-critical tasks (e.g., traffic flow control, bio- electrical monitoring, etc.) (Hsiao et al., 2022; Qiu et al., 2025; Roeder, 2022) on edge devices with limited com-
"oo-corresponding author 'The School of Computer and Information Technology, Shanxi University, Taiyuan, China "The College of Computer Science and Software Engineer- ing, Shenzhen University, Shenzhen, China. Correspondence to: Junyang Chen <junyangchen @ szu.edu.en>, Wenjian Wang <wjwang@ exw. ex..edu.c.....
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Can DBNNs Robust to Environmental Noise for Resource-constrainee Scenarios?
above approaches (Gowal et al., 2018; Balunovic & Vechev, 2020; Zhang al., 2020; 2021) focus the shallow CNNs. Due to the structural disparities between shallow and deep model, it remains challenging to determine whether existing methods can be directly applied to analyze the robustness of deep backbone (e.g., ResNet34).
Formal Ventlsstion
Pass Fail
Full-precision NNs
Dinarization
(Robust) Binary NNs
Tmining
Empiricall Testing (with soine)
Accuracy Test Sets
Challenges: The utilization of binary neural networks (BNNs) as an extreme approach for quantifying weights and activations has yielded remarkable success in several machine learning tasks (Rastegari et al., 2016; Zhou et al., 2016; Liu et al., 2018; Bulat & Tzimiropoulos. 2019; Liu et al., 2020; Qin et al., 2020; Tu et al., 2022; Yuan & Aga- ian, 2023; Qin et al., 2023). The study (Cai et al., 2022) shows that the quality of inference from lightweight models are observed to decrease on edge devices with environmen- tal noise. Especially in some medical assisted diagnosis scenarios (Roeder, 2022), noise interference has serious negative impact on decision making. However, the existing approaches focus on narrowing the performance gap be- tween the full- full-prccision and BNNs, without delving into the robustness of BNNs against environmental noise perturba- tions during inference. Compared to full-precision models, the LCR method (Shang et al., 2022) has investigated the in- sufficient robustness of BNNs. However, it still encountered two challenge issues. (1) The study (Shang et al., 2022) introduce a large extra overhead to BNNs, which is difficult to meet the requirements of resource limited scenarios. In particular, the approximation operation using several reten- tion matrices result in a 20% increase in training overhead. (2) The study (Shang al., 2022) lacks quantitative formal analysis of noise perturbation bounds to demonstrate the robustness of DBNNs
Our Focus
Figure I. Workflow for model binarization and robustness evalua- tion. of our auxiliary robustness loss function (i.e., requiring the maximization of non-concave function over a norm ball), we introduce a constraint coefficient associated with scaling factors of DBNNs to balance the classification and auxil- iary robustness loss functions (See Appendix for detailed experiments). Our main contributions are summarized as follows:
In theory, we find that the vulnerability of DBNNs to environmental noise comes from scaling factors and binary weights. Then, we formally derive an noise perturbations upper bound for DBNNs in closed- form analytical manner. We provide a tighter upper bound compared to the SOTA method under same noise conditions.
We propose a novel automated and robust training framework to enhance the environmental noise robust- ness of various DBNNs by using the 11 -norm con- straint.
Our visual case finds the detrimental impact of envi- ronmental noise on the feature maps of DBNNs
Motivation & Contribution: To address the above- mentioned challenge issues, we aim to devise an upper bound via -norm that can analyze the robustness of DBNNs under environmental noise perturbations. Further- more, the analysis also aim reveal that the vulnerability in DBNNs model robustness originates from the scaling factor associated with binary operations. In contrast to the recent study (Shang et al., 2022) by using an approximate spec- tral norm constraint strategy, our proposed L1,0v -norm con- straint only involves the column and row of matrix and the maximum computing operations, thus avoiding additional training costs like in the study (Shang et al., 2022). In order to better target the characteristics (i.c., deep backbones and binarization operations) of DBNNs under environmental noise (i.e., random and high SNR), our theoretical analysis framework by using -1, norm offer superior applicability to DBNNs in mitigating the negative impact of environmen- tal noise. Meanwhile, we consider the characteristies of the DBNNs (e.g., scaling factors and binarization operations) to provide a tightness upper bound under environmental noise perturbations. Finally, to address the potential risk
Experiments show that our approach enhance the robustness of DBNN- models on five classification tasks, with maximum improvements of 4.8% and 5.4% the CIFAR- 100, Brain tumor MRI datasets, respectively.
Our work aims to provide an ef- fective training framework for enhancing the robustness of DBNNs against environmental noise by incorporating an L1,os norm constraint training, hence it is orthogonal to the topic of formal verification for verifying the correctness of the resulting quantized/binary NN (In Fig. Please Appendix for details.
Discussion summary:
2. Related Works
2.1. Binary Neural Networks
The current radical compression strategy involves reduc- ing the learnable parameters of bit-width from 64 to 1-bit (Yuan & Agaian, 2023; Qin et al., 2023). Several researchers (Rastegari et al., 2016; Zhou al., 2016; Liu et al., 2020;
2

---

Can DBNNs Robust to Environmental Noise for Resource-constraineed Scenarios?
2018: Qin al., 2020; Tu et al., 2022) foremost compress weights wg R and activations a E R (detailed in survey work (Yuan & Agaian, 2023)) to 1-bit in each convolution block. In comparison to conventional compression method (i.e., pruning), binarization exhibits superior computation operators as it exclusively targets the model parameters (Rastegari et al., 2016). However, the binarization brings scarcity of feature representation, leading to a signifi- cant degradation in accuracy. Now, the study (Qin et al., 2023) proposes method that leverages information gain to mitigate these representation losses and thus enhance the performance of BNNs. However, these methods do not con- sider whether BNNs directly used for inference is resistant to environmental noise.
which can be defined as follows:
Definition 3.1. (los-tsbsssssss) Let F(x) represent the output of neural network F on input signals x and lg [(x) be the ground truth of x. If the F is locally robust, there is not exist perturbed 7 that makes |/ holds on. e.g., we have F(x + +7) # lz-
Secondly, we outline the prevalent characteristics of BNN- based models (Rastegari et al., 2016; Liu et al., 2018; 2020; Qin et al., 2020; Yuan & Agaian, 2023; Qin al., 2023), which involve the conversion of full-precison weights (WF) and/or activations (1p) into 1-bit precision. Here, the stan- dard definition (Qin et al., 2023; Liu et al., 2018; Qin et al., 2020) of the binarization operation is provided as follows:
Q(w>) * Wa, Q(Tr) = B lb:
(1)
2.2. Robust Neural Networks using Lipschitz Continuity
Recently, the study (Gouk et al., 2021) proposes an effec- tive approach to enhance the performance of feed-forward neural networks by computing an upper bound on the Lips- chitz constant for multiple p-norms. Especially in the area of image processing, the study (Yoshida & Miyato, 2017) proposes spectral normalization to constrain the Lipschitz constant of the discriminator by optimizing a adversarial network. Meanwhile, several studies (Li et al., 2019; Singla & Feizi, 2021) propose strategies to effectively reduce the Lipschitz constant of CNNs by leveraging In-nomm. Then, the study (Zhang et al., 2021) further proposes a novel net- work architecture, termed L -dist network structure, by leveraging inherently 1-L-ischhitz functions as neuron units. The core module of the study (Miyato et al., 2018) is note- worthy for its utilization of spectral norm constraints based on singular values, which are directly applied to the weights of each convolutional layer. However, the extreme spar- sity of binarized weights pose an immediate computational challenge by directly via spectral norm constraints (Shang et al., 2022). To resolve this issue, a study (Shang et al., 2022) aimed at enhancing the robustness by incorporating Lipschitz continuity constant as a regularization term. To address the extreme sparsity of binary weights, this work (Shang et al., 2022) proposes employing retention matrices to approximate spectral norms of weights. However, the effectiveness of the LCR (Shang et al., 2022) heavily relies on the strong mutual linear independence assumption of binary activations.
where W, and In are denote the binary weight and binary activation matrices, respectively. Here, the a = ; || (Wrll and 3 = A |||/i|| are represent two scaling factors for full- precision weight (Wp) and activation (Ip). Following the survey (Yuan & Agaian, 2023), we present the description of the Sign(-) function:
+1, 2 0
Sign(x)
(2)
-1,
otherwise
Furthermore, we give a general computation process of DBNNs in Def.3.2.
Definition 3.2. The BNNs typically employs binary {-1,1} weights and activations components to achieve weight modeling. Then, we can write such network thought 1-1 modules (i.e., (a* x (B*W)), E L-1]. where @ denotes the convolution operation) with weights and activations and L-th layer (i.e., output module O())). Hence, we can deduce the iterative form of BNNs with input xr as follows:
BNNs(x) 0((a)) @ (BW)-1) " (a11) @ (xW)(x))
(3)
Notations of Matrix and Perturbed Robustness: We can represent matrices by uppercase letter, such as W. For a weight matrix W, we can write its 7th row and jth column as the entry W., E Rm The T., norm of a matrix can be abbreviated as || The -norm will be used in the subsequent theoretical analysis of robustness. According to the definition of the matrix norm, we can get the formal definition for L1,00 -norm || max
3. Background
Following the theoreti- cal study (Narodytska, 2018), we adopt the -nom as metric to quantify distance. From a theoretical perspec- tive, global robustness represents an exceedingly stringent property. Considering real-world scenarios, our research is dedicated to enhancing the local robustness of DBNNs,
Robustness Definition of BNNs:
A where zy0 T
/ denotes vector and A denotes a matrix. Recently, the study (Mao et al., 2023) has focus on the L robustness of perturbed inputs as follows:
Be00 (c) def {x' || - 2l1o0 S (ca):
(4)
3

---

Can DBNNs Robust to Environmental Noise for Resource-constraineed Scenarios?
1. Training Stage
Backhones P6
Loading
Selecting Bi-Comy modules
Laper
Intial Layen
Candidattss
Objective Fanction
Trpaut
W1.1
W2,1
Output Layerr
Selecting Models
LDoReFa-Cam Core
Cross Entropy Loss
Chiput
Model Name
identification
ResNat
OR
2.Brraal Comy
Ly Norm
W1,
W2.2
9
Clean
H.HAct-Cooe
Coustraint
4 AHNNat-Come
Trgait noiseox
Secand Hirddnn
Test Stage
Tesl Dala with Noise
Robust DBNNs
Figure 3. The example of full-precision neural network perturbed by noise.
Test Dala
Successful Defense
Input Noise
Oulput
Bas
Perturbations
upper bound of DBNNs. In Fig.3, the noise applied to the image signal æ. Subsequently, we demonstrate the disparity between the output of the DBNNs under clean and noise scenarios. Specifically, in Eqn.5, Y represents the perturbed output resulting from two examples with noise (i.e., ft and fz)). Here, 1 Y| denotes the absolute difference between the disturbed and clean outputs. Then, ReLU(-) denotes the ReLU activation function. In addition, Wí,2 denotes to the second weight value in the initial hidden layer.
Enhanced bounds for each BI-Com layer by our f.... Norm
Figure 2. Our automatic robust training framework.
where x' denotes a series of perturbed inputs and x denotes series of clean inputs, the cz denotes input noise pertur- bation radius and _ (e) represents a Loo norm ball of the noise perturbation radius 6x.
4. Method
I - Y|= Wo W
Overview: In this section, we propose novel robust train- ing method that utilizes a targeted objective function with L1 -nom constraint term on binary weights to enhance the robustness of DBNNs against environmental noise per- turbations. In contrast to the work (Shang et al., 2022), our proposed method effectively mitigates the excessive training cost resulting from the indirect preservation of the matrix. To analyze more suitable constraints, we establish quan- titative relationship between our perturbation bounds and the bounds of study (Shang et al., 2022). To facilitate the application of our approach for enhancing the robustness of various BNN structures, we propose an automatic training framework (See the last paragraph of section 4.3 for details) as shown in 2
+ ReLLU(@::i.,)WWa Re
ReLU(&gWi.,WW::) WoRRLL
(5)
ReLU(z2W1,,WW:: Re
+ RReLU<r2W1..WW... + ReLU(roWiaaWapll
Step2: Formal Analysis for Full Precision Models under Noise Perturbations. For the first hidden layer of a full pre- cision model, we quantify the discrepancy through applied noise perturbations (also referred to as the model's decision margin) with Lo perturbed inputs based on 4qn.4-Eqn.5 are as follows:
2
1 2
del
Wes - (6g - -,,) S
+Σ> 1=1
Wesl 10 - -xy1
4.1. Formal Analysis on the Robustness of DBNNS
< En max
Wes1 -x |
=1
(6)
Main Analysis Process: (1) We present a feed-forward and full- pell-precision neural network under general noise perturba- tions as example in Fig.3. Here, the weights W1,1, 2 should be annotations on the edges.
where el denotes the difference between clean and applied noise at the initial hidden layer. The weight value of the ith row and jth column in a matrix is denoted as Waj- while the initial hidden layer of all weights is represented by W Then, eg represents the perturbation value for inputs. It is imperative to maximize this disparity to effectively constrain the model and enable correct decision against noise perturbations. Hence, we provide an expanded upper bound on the right side of the definition and then employ the T. norm to calculate its maximum value.
(2) Taking simple 3-layer full-precision model as an illustrative example, we present an analysis of the disparity in learnable weights between the outcomes of the two categories before and af- (3) Based on the
ter introducing input noise perturbations. above theoretical findings and the inherent characteristics of the DBNNs (c.g., scale factors a, B and binary weights W.). we can deduce the upper bound of L-layer DBNNs model under noise perturbations.
Thereafter, we present the perturbation bounds of second hidden layer from the preceding layer with respect to input c as follows:
Step1: The Process of Full Precision Models with Noise. In 3ig.3, our example illustrates the variations in each weight value under noise perturbations, thereby offering a valuable contribution to the subsequent analysis of the
e² S ReLU | |
(7)

---

Can DBNNs Robust to Environmental Noise for Resource-constrained Scenarios?
Table 1.
The distinctions between our study and previous studies. Here, * denotes the additional training cost.
Methods
Constraints
Types
Deep Structures
Assumptions
Additional Cost
ELC (2021) (Gouk et al., 2021)
Lg-norm
DNNs
VGG19
N/A
Not
SN (2018) (Miyato et al., 2018)
L2 L2-norm
DNNs
ResNet18
N/A
Not
LCR (2022) (Shang et al., 2022)
L2-norm
BNNs
ResNet20
stringent
+20 % cost"
Our
1, -norm
DBNNs
ResNet34
N/A
Not
where |/ denotes the difference for the second hidden layer. Finally, we can provide the perturbation bounds of output layer based on Eqn.7 as follows:
denote the output layer of learnable parameters correspond to classes C1 and C2. respectively. Then, hg and h f--1
denote the intermediate results from the preceding L - 1 layer under clean and noise scenarios, respectively.
e² & (ReL (I 1
- (8)
Il
Theorem 4.2. For L-layer DBNNs against noise perturba- lions, we can derive the upper bound of robustness for the discrepancy between two classification outcomes (i.e., Ci and C2) as follows:
According to the study (Gouk et al., 2021), we employ the L1 norm to quantify the discrepancy in weights between perturbed output result and an unperturbed output result as [[oo - -W in Eqn.8.
1-1
FW: (G) - FWg (x)
II
an
-Whch 111
Step3: Formal Analysis for Environmental Noise Bounds of L-layer DBNNs. Based on the Def.3.2, we focus on the DBNNs using the classical ResNet-basett backbone. Then, the formal description perspective for DBNNs only requires the conversion of activation and weights from full- full-precision models in Eqn.7 to binarized form.
1=1
h-1
II [(W (( 11,00
II M==
IIW 11,
11 M- 1111loo
(1I)
where hM = I @ (W..... ... @ (W) denotes the expansion of front M-layer DBNNS (i.e., 3Ith s (aW, 31,, 1 @ (a)) can be contracted scaling tors (i.e., I oußi) by a 1-lipschitz continuous function. Here, N denotes the total number of binarization layers that are perturbed by noise excluding the outuut layer of DBNNs. W.C,,ll ll2 represents the discrepancy in out- put layer of weights corresponding to the two categories (Suppose the DBNNs is judged to be of two different classes after applying noise) that are deemed distinct following noise perturbations. Thus, we constrain the scale factors and binary weights can effectively enhance the tightness of upper bound under noise perturbations.
The Difference between Full-precision models and DBNNs: The primary differents between the computa- tional processes of full-precision model (i.e., (w") and BNNs lie in the binarization of activations (i.e., 31,) and weights (i.e,, aW,) with scaling factors. Hence, we can illustrate BNNs example of final bounds via the Def.3.2 and Eqn.8 as follows:
Then, [[
cg S (31, & ( 31, @ (allW?]l=))AWw?
(9) where W3 denotes the all binary weights of the second bi- nary convolution layer, and then AWg || W c2 l denotes the difference of binary weights corresponding to different classification results at output layer by utilizing L1 norm. Here, a and 3 denote scaling factors. Thus, we can obtain the upper bound of the DBNNs under environmental noise in 1.
To facilitate the derivation of the bounds, we approximately omit the analysis of the residual structure. Nevertheless, we give the upper bound result of the Lipschitz constant for the residual block in the Appendix.
Definition 4.1. Suppose a standard classification task con- tains P categories, such as C, E [1, ..... P). Suppose the output of the DBNNs is C1 instead of the expected result C2. we can define the discrepancy between different classi- fication results using L-layer DBNNs under environmental noise perturbations as follows:
4.2. Tightness Analysis of L1,00 -norm Constraint
Firstly, the study (Shang et al., 2022) employs the Lipschitz continuity constrains to enhance the robustness of BNN- based models. However, it fails provide quantitative upper bound under environmental noise perturbations. Ac- cording to the study (Shang et al., 2022), can obtain an upper bound as follows:
W. (5) - F (z) del 1
[Wrh Who 1
(10)
- - || 1 W.c. 1
Q
II
4
ll2 1-9
(12)
where FC: (5) Fo (z) represents the difference between the two different outputs of DBNNs under noise perturba- tions and clean scenarios. Meanwhile, W&o, and L
3=1
where ||w2 denotes the Le norm of binarized weights in the 5th convolution layer, Q denotes the total number of
5

---

Can DBNNs Robust to Environmental Noise for Resource-constrained Scenarios?
binary convolution layers, j E [1, Q. and hyper-parameter y>1. 1. Clearly, it is imperative compare the quantitative relation between other norm constraints and the spectral norm constraints (Shang et al.. 2022).
upper bound can improve the robustness of the model. With increase in the network's layer count, the cumulative product of 11 -nor described in our method will also escalate; hence, we introduce a constraint coefficient (6) to regulate the constraint's strength in Eqn.18.
|| = max
4.3. The 11,00 Norm Constrain for Robust DBNNs
j=1
i=1
j=1
(13)
Given the multiple classification loss function l on the image classification tasks, a series of randomly sampled image input matrices X = [x1, " E 1nxd and ground truth Y, we can obtain a expected optimization objective as follows:
|asss 2= n2| 111
12
where A denotes a standard matrix and represents the total number of rows in matrix. Obviously, we can get such an inequality relation ||Al|oo S VrillAAle. The study (Liu et al., 2022) has focused on investigating the inequality rela- tionship among different types of classical L1, L2, and Los norms for learning smooth neural functions. In contrast to the work (Liu et al., 2022), our aims to explore the quan- titative relationship between the dual norm L1,0 and the spectral norm L2 for robustness noise perturbations bound in DBNNs. Firstly, the inequality relationship between the 11 norm and the Loo norm is as follows:
arg min {(X + +X, X'ERaxa
Y),
(17)
where X' denotes the perturbed inputs according to Eqn.4. However, the optimization goal will deviate from the Eqn. due to the environmental perturbations that were not ap- peared during the training phase. The influence of noise perturbations during the inference phase on the robustness of DBNNs should be duly considered. Inspired by the the- oretical analysis of Thm.4.2, we propose the -norm constraint term to the objective function as the auxiliary robustness loss function Epi
||
| ||
||4 11,,00 k max
S k. max
|
|
| |
I k. ||A]| 0o
max
4p 8x
II a aa j=1
(18)
(14)
where i denotes the number of dimensions of the matrix A. Based on the intermediate inequality (i.e., Eqn.13 and Eqn. derive the final bound inequality relation of one binary convolution layer in 1qn.15:
where the constrain coefficient& 5 (The experiments in the Appendix discuss the effect of different coefficients on the performance.) is utilized to ensure the maximization of a non-concave function (i.e., auxiliary robustness loss by using L1,00 norm constrain) over a norm ball. Then, the coordination of the orders of magnitude between the our auxiliary robustness loss term and the classification loss term is crucial to prevent gradient disappearance or ex- plosion rather than simple maximization. Here, ||W 1,, denotes the L,,0 norm of binarized weights in the jth con- volution module, and Q denotes the total number of binary convolution modules.
|
1| S k. VnllA112 max
(15)
||]||
Corollary 4.3. According to Eqn.12 and 1q,, we can determine the tightness ratio of Q binary convolution layers between our study and previous work (i.e., LCR) (Shang et al., 2022) under noise perturbation as follows:
I || 11,00 1 Liep
k- vi
|
| (T--Q)2
max
|>l|
(16)
Given a classification- based objective function Liotal. we design robustness loss function inside Crotal through -orom constrain
L1,00 norm
Here, the symbol n denotes the dimension of the binary convolution weights. In fact, this a definite value and does not change as the number of network layers increases. In addition, the upper bound of (Shang et al., 2022) increases exponentially and its value is much larger (due to the value of hyper--praae parameters 7 > 1 in the LCR method) than the upper bound of our work, which means that the denominator of C.r.4.3 is always greater than the numerator, that is to say, the ratio is strictly less than 1. Theoretically, a compact
Listal = (o,nte(Xyy + Lp
(19)
where LXt Y) is the traditional cross-entropy loss func- tion for multiple classification tasks. Here, the X denotes the input image signal and Y denotes the target label. Fur- thermore, it is crucial to ensure dimensional consistency in both the cross-entropy loss and our proposed L con- straint term. Please see the Appendix for an analysis of the impact of constraint coefficients 6.
6

---

Can DBNNs Robust to Environmental Noise for Resource-constrained Scenarios?
The Automatic Training Framework for Various BNNs:
The proposed approach demonstrates a substantial enhance- ment in the robustness of models utilizing ResNet34 with BirealNet and IRNet when subjected to environmental noise perturbations on the two datasets. This means that our 20 norm constraint strategy is more suitable for methods that have significant adjustments for learnable parameters. (3) The proportion of performance degradation under environ- mental noise perturbations are significantly higher on the CIFAR-100 dataset, which contains more complex image information, compared to the CIFAR-10 data set for both full-precision and binarized models. This implies that the general environmental noise perturbations have a greater impact on the decision- mariiiiinmmkinggggg process of the model in the intricate classification scenarios.
Based on the survey of BNNs (Yuan & Agaian, 2023), it has been observed that a common characteristies among various BNNs is their utilizzaiinn of well-established deep backhone networks, such as R8 and ResNet34. Thus, we provide the automatic training framework in Fig.2.
5. Experiments
5.1. Experimental Setups
Datasets: To comprehensively evaluate our proposed ap- proach, we construct experiments on a series of popular BNNs on the large-scale bio-electricity (Roeder, 2022) clas- sification, CIFAR-10, CIFAR-100 and ImageNet, Brain tu- mor MRI (Nickparvar, 2021) datasets with two common backbones (i.e., ResNet18 8 and ResNet34), see details of the setup in the Appendix.
Finally, our work also provides discussion on the selection of our designed constraint coefficients in Eqn.18. Due to space limitations, we provide several experiments of chang- ing the hyperparameter for our constraint coefficients on enhancing the robustness of various BNN-based methods (Zhou et al., 2016; Liu et al., 2018; 2020; Qin et al., 2020; Tu et al., 2022) in the Appendix.
Noise Perturbations and Metries: We utilize a signal-to- noise ratio (SNR) level close to 50% for environmental noise. First, we apply the perturbations to the bio-electricity dataset is (0.44, 0.8). Then, the range of environmental noise per- turbations of Brain Tumor MRI dataset is e=(0.05, 0.1]. In addition, 10, CIFAR- 1000 and ImageNet datasets, we select small perturbations == 4/255] range of noise (refer to (Zhang et al., 2021)). Then, we utilize the test accuracy with noise metric to evaluate DBNNs, For details of clean test performance, please see the Appendix.
The Robustness of Our Method in Bio-clectrical and Brain tumor Classification: To validate the generalization of our method against environmental noise perturbations, we examine the effect of 10 -nomm constraint on the large- scale bio-electricity series (Roeder, 2022) and Brain tumor MRI classification (Nickparvar, 2021), as shown in Tab.3-4. We observe 4% performance decline when replacing the binary convolutionn module with full- full-precision model. This means that for resources-constrainee tasks, the robustness of the DBNN-based models are difficult to meet the require- ments. For Bio-electricity task, our 11.c--nrmm constraint strategy demonstrates a robustness improvement of up to % across the three binary convolution modules, thereby validating the efficacy of our strategy for diverse BNNs. For the brain tumor task, our strategy yields 3-5% improve- ment in robustness for three DBNNs, thereby validating the effectiveness of the our method (See Appendix for details).
5.2. Experimental Results
The Robustness of Our Method in Image Classification: We conduct comprehensive experiments to evaluate the ro- bustness of various DBNN-based models and our proposed method against environmental noise perturbations on pop- ular image classification benchmarks (i.e., CIFAR-10 10 and C0F datasets). To be fair, according to previous survey study (Qin et al., 2023), we have retrained several BNN-based models (i.e., BirealNet (Liu al., 2018), Re- ActNet (Liu et al., 2020), DorefaNet (Zhou et al., 2016) and IRNet (Qin et al., 2020)) with our proposed method that uti- lize classical ResNet18 and ResNet34 backbone networks.
Computational Overhead Analysis: To mitigate the addi- tional costs associated with training, we propose incorpo- rating the L1 -noom -oom constraint as an alternative solution, thereby eliminating the need for intricate approximation op- erations employed in the previous study (Shang al., 2022). In Tab.5, we compare the actual time spent training and testing time with the SOTA method (Shang et al., 2022) on the ImageNet dataset. Specifically, we propose eliminates the L2-norm constraint module (Shang et al., 2022) in the inner embedding convolution operation, thereby reducing overhead from model perspective. It evident that our pro- posed method reduces training costs by 16% compared to LCR method (Shang et al., 2022). Thus, our robust training algorithm enables faster training speed for the DBNNs
Specifically, we can draw several conclusions from the fol- lowing aspects in Tab.2. (1) The experimental findings demonstrate that our approach significantly enhances the robustness of mainstream four BNN-based methods (Liu et al., 2018; 2020; Zhou et al., 2016; Qin et al., 2020) with two classical backbones, thereby emphasizing its universal applicability in our T,,,0c norm constraint. On the CIFAR- 10 dataset, the application of our proposed method to IRNet significantly enhances its robustness test accuracy, achiey- ing a remarkable increase of 2.23% to reach 93, 93.64%. (2)
7

---

Can DBNNs Robust to Environmental Noise for Resource-constrainee Scenarios?
Table Robustness comparison between our approach and popular BNN-hased methods against environmental noise on the CIFAR-10 and CIFAR-100 100 datasets. Here, 1 denotes the proposed method can improve the robustness of the existing BNN-based methods.
Methods
Datasets
Scenarios
Backbones
FP32 (clean)
DoReFa
DoReFatour
BiReal Bireal+our ReAct
ReAct+our
IRNet
IRNet+our
ResNet18
94.82
91.55
92.271
91.20
93.217
91.40
93.041
91.41
93.641
CIFAR-10 With Noise
ResNet34
94.17
85.16
87.04t
87.80
90.131
87.87
89.00
87.88
88.311
ResNet18
72.61
65.15
67.211
65.35
68.841
66.01
68.261
65.24
70.041
CIFAR-100 With Noise
ResNet34
71.52
60.37
61.16t
63.76
64.69
60.48
63.81
60.61
61.71
Table Robustness comparison between strategy and three Binary Convolution structure against envirommental noise the bio-
electricity series classification task.
Methods
Performance
FP32(clean)
IRConv IRConv+our BirealConv BirealConv+our AdaBinConv AdaBinConv+our
Test Acc. with Noise
97.1
87.21
88.361
88.66
89.51
93.13
93.721
The Visualization Case Study of Our Proposed Method
Table 4. Test accuracy with noise comparison between strategy and three DBNN-based baselines against environmental noise on the Brain tumor MRI dataset. Here, w++ denotes DBNN model with our strategy.
for Enhancing Robustness on the CIFAR-100 Dataset: To visually demonstrate the robustness of the proposed method against environmental noise on feature learning from images, we present series of visualization cases on the CIFAR-100 dataset. In Fig.4, we randomly provide several figures of feature maps extracted by the first binary convolution layer before and after utilizing our proposed 1. norm constraints term in the loss function. The pri- mary observations this case study are as follows:
Backbones React w+ourr Dorefa w+our CycleBNN w+our 84.84
ResNet18 76.77 83.87 ResNet34 76.13 79.16
86.91
83.36
85.46
77.42 80.13
71.94
78.39
Table 5. Computational cost comparison between our and SOTA
method on the ImageNet dataset. The notation (mm:ss) represents the unit of minutes and seconds of Epoch.
(1) After applying our -norm constraints, we ob- serve that the baclines can attention towards the most cru- cial/center feature information under environmental noise perturbations. Specifically, our proposed method extracts image features with a heightened focus on individuals wearing glasses, in contrast to ReActNet's and CycleNet's broader emphasis on surrounding features (i.e., red/blue box).
Methods
Training(mm:sss) Test (mm:ss)
Our
56:08
4:49
LCR (Shang al., 2022)
66:37
6:30
ReActNet
0
20
20
20
20
(2) The feature maps of our proposed method have little color mixing zones around due to environmental noise per- turbations and are clearer in the eyeglass frame compared to the several baseline models (Liu et al., 2020; Fontana et al., 2024) (i.e., the brown/blue box).
ReActNe (+Our(L-
20
30
20
[1.inf)
20
Thus, we can find that the proposed method has positive effect improving the robustness of DBNNs.
(a) ReActNet V.S. ReActNettour
6. Conclusion
CycleNet
20
In this study, we answer that the robustness vulnerability of DBNNs (during inference) facing environmental noise is due to binary weights and scaling factors. To mitigate the adverse effects of environmental noise, propose em- ploying 11.00 norm constraint loss function in the train- ing of DBNNs. Then, we quantitatively analyze the upper bound of noise perturbations which is more tighter than SOTA method. Experimental results demonstrate that our approach provides an effective strategy for DBNNs to mit- igate environmental noise perturbations during inference across five different classification tasks.
20
CycleNet +Our(T.-
00
{1.inf])
20 20
(b) CycleNet V.S. CycleNettour
Figure 4. The visual feature maps between our method and two baselines (Liu et al., 2020; Fontana et al., 2024) under environmen- tal noise perturbations the CIFAR- 100 dataset.
8

---

Can DBNNs Robust to Environmental Noise for Resource-constrainee Scenarios?
Acknowledgements
59th ACM/IEEE Design Automation Conference (DAC), 2022.
This work was supported in part by the National Natural Sci- ence Foundation of China (NSFC) under Grant U21A20513, Grant 62476157, Grant 62406179, the Stable Support Project of Shenzhen under Grant 20231 120161634002 and the Shenzhen Science and Technology Program under Grant
Jr. J. B. P. M., de Lima Filho, E. B., Bessa, I., et al. Coun- terexample guided neural network quantization refine- ment. IEEE Trans. Comput. Aided Des. Integr. Circuits 2024.
Syst. 43(4):1121-1)))
JCY12002001/341441700
Kanai, S., Ida, Y., Fujiwara, Y., et al. Absum: Simple regularization method for reducing structural sensitivity convolutional neural networks. The Thirty-Foure Fourth AAAI Conference on Artificial Intelligence (AAAI), pp. 4394-4403, 2020.
Impact Statement
This paper presents work whose goal advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Li, Q. Haque, S., Anil, C.,, et al. Preventing gradient at- tenuation in lipschitz constrained convolutional networks. Advances in Neural Information Processing Systems (NeurIPS), pp. 15364-15376, 2019.
References
Balunovic, M. and Vechev, M. T. Adversarial training and provable defenses: Bridging the gap. In Proceedings of the International Conference on Learning Representa- tions (ICLR), 2020.
Lin, H., Lou, J., Xiong, I.., et al. Integer-arithmetii-onll certified robustness for quantized neural networks. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7808-7811 IEEE, 2021.
Baluta, T., Shen, S., Shinde, S., al. Quantitative verifi- cation of neural networks and its security applications. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 1249-1264, ACM, 2019.
Lin, X., Zhao, C., and Pan, W. Towards accurate binary convolutional neural network. In Advances in Neural Information Processing Systems (NIPS), 2017.
Liu, H. D., Williams, F., Jacobson, A., et Learning smooth neural functions via lipschitz regularization. In Proceedings of the SIGGRAPH '22: Special Interest Group Computer Graphics and Interactive Techniques Conference, pp. 1-13, 2022.
Bulat, A. and Tzimiropoulos, G. Xnor-net++: Improved binary neural networks. In Proceedings of the 30th British Machine Vision Conference (BMVC), 2019.
Cai, B., Li, X., Kong, W., et al. A reliable and lightweight trust inference model for service recommendation in siot. IEEE Internet of Things Journal, 10988800988-1003, 2022.
Liu, al. Bi-real net: Enhancing the performance of 1-bit enns with improved representational capability and advanced training algorithm. In Proceedings of the 15th European Conference Computer Vision (ECCV), 2018.
Fontana, F., Lanzino, R., Diko, A., et al. Cyclebnn: Cyclic precision training in binary neural networks. In The Pro- ceedings of the European Conference Computer Vision (ECCV), 2024.
Liu, Z. et al. Reactnet: Towards precise binary neural net- work with generalized activation functions. In Proceed- ings of the 15th European Conference Computer Vision (ECCV), 2020.
Gouk. et al. Regularisation of neural networks by enforc- ing lipschitz continuity. Machine Learning, 2021.
Liu, Z. et al. How do adam and training strategies help bnns optimization. In Proceedings of the International Conference on Machine Learning (ICML), 2021.
Gowal, S. et al. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv, 2018.
Mao, Y. et al. Connecting certified and adversarial training. In Proceedings of seirth- Conference on Neural Information Processing Systems (NeurIPS), 2023.
Henzinger, T. A., Lechner, M., and Zikelic, D. Scalable verification of quantized neural networks. In Thirty-Fifih AAAI Conference Artificial Intelligence (AAAI), pp. 3787-3795. AAAI Press, 2021.
Meng. M. H., Bai, G., Teo, S. G., et al. Adversarial robust- ness of deep neural networks: A survey from a formal verification perspective. CoRR, abs/2206. 12227, 2022.
Hsiao, Y. et al. Zhuyi: perception processing rate estimation for safety in autonomous vehicles. In Proceedings of the
Mistry, S., Saha, L. and Biswas, S. An MILP encoding for efficient verification of quantized deep neural networks.

---

Can DBNNS Robust to Environmental Noise for Resource-constrainee Scenarios?
IEEE Trans. Compul. Aided Des. Integr. Circuits Syst. 41 (11):4445 4456, 2022.
Shkolnik, M., Chmiel, B., Banner, R., et al. Robust quantiza- tion: One model to rule them all. In Advances in Neural Information Processing Systems 33; Annual Conference on Neural Information Processing Systems (NeurIPS), 2020.
Miyato, T. et al. Spectral normalization for generative adver- sarial networks. In Proceedings of the 6th International Conference on Learning Representations (ICLR), 2018.
Singla, S. and Feizi, S. Skew orthogonal convolutions. In Meila, M. and Zhang, T. (eds.), Proceedings of the 38th International Conference on Machine Learning (ICML), pp. 9756-9766, 2021.
Mohammadinejad, S., Paulsen, B., Deshmukh, J. V., and Wang, C. Diffrmm: Differential verification of recurrent neural networks. In Formal Modeling and Analysis of Timed Systems 19th International Conference (FOR- MATS), volume 12860 of Lecture Notes in Computer Science, pp. 117-134. Springer, 2021.
Tu, Z. al. Adabin: adaptive binary sets. In Proceedings of the European Conference Computer Vision (ECCV), 2022.
Improving binary neural networks with
Narodytska, N. Formal analysis of deep binarized neural networks. In Proceedings of the Twenty-Seventh Interna- tional Joint Conference on Artificial Intelligncce (ICCAI), 2018.
Wei, X. and Zhao, S. Revisiting the trade-off between ac- curacy and robustness via weight distribution of filters. IEEE Transactions on Pattern Analysis and Machine In- telligence, 2024.
Nickparvar, M. Brain tumor mri dataset, 2021. URL https: // /www kaggle.
Yayla, M. and Chen, Memory-efficient training of bina- rized neural networks on the edge. In Proceedings of the 59th ACM/IEEE Design Automation Conference (DAC), 2022.
Paulsen, B., Wang, J., and Wang, C. Reludiff: differential verification of deep neural networks. In ICSE '20: 42nd International Conference Software Engineering, pp. 714-726. ACM, 2020a.
Yoshida, Y. and Miyato, Spectral norm regularization for improving the generalizability of deep learning. ArXiv, 2017.
Paulsen, B., Wang. J., Wang. J., et al. NEURODIFF: scal- able differential verification of neural networks using fine-grained approximation. In 35th IEEE/ACM Interna- tional Conference on Automated Software Engineering (ASE), pp. 784-796. IEEE, 2020b.
Yuan, C. and Agaian, S. S. A comprchensive review of binary neural network. Artificial Intelligence Review, 2023.
Qin, H. et al. Forward and backward information reten- tion for accurate binary neural networks. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Zhang, B. et al. Towards certifying l-infinity robustness using neural networks with l-inf-dist neurons. In Proceed- ings of the 38th International Conference on Machine Learning (ICML), 2021.
Qin, H. et al. Bibench: Benchmarking and analyzing net- work binarization. In Proceedings of the International Conference on Machine Learning (ICML), 2023.
Zhang, H. et al. Towards stable and efficient training of verifiably robust neural networks. In Proceedings of 8th International Conference on Learning Representations (ICLR), 2020.
Qiu, X., Wu, X., Lin, Y., Guo, C., Hu, J., and Yang, B. Duet: Dual clustering enhanced multivariate time series forecasting. In SIGKDD,, pp. 1185-1191 1196, 2025.
Zhang, Y., Zhao, Z., Chen, G., et al. QVIP: an ilp-based formal verification approach for quantized neural net- works. In 37th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 82:1-82:13. ACM, 2022.
Rastegari, M. al. Xnor-net: Imagenet classification using binary convolutional neural networks. In The Proceed- ings of the 14th European Conference Computer Vision (ECCV), 2016.
Zhang, Y., Song, F., and Sun, J. Qebverif: Quantization error bound verification of neural networks. In Computer Aided Verification 35th International Conference (CAV), volume 13965 of Lecture Notes in Computer Science, pp- 413-437, Springer, 2023.
Roeder, M. Anns and cognitive models (fhws ss22), 2022. URL https:/ /kaglll. wwwc wwt.ccooccc/cccc http///oppttiiiooe itions/ anncmss22.
Shang, et al. Lipschitz continuity retained binary neural network. In Proceedings of the European Conference Computer Vision (ECCV), 2022.
Zhou, S. et al. Dorefa-net: Training low bitwidth convolu- tional neural networks with low bitwidth gradients. Arxiv, 2016.
10

---

Can DBNNs Robust to Environmental Noise for Resource-constrainned Scenarios?
A. Appendix
A.1. Discussion Differences with Existing Studies
In these remarkable verification researches (Baluta et al., 2019; Paulsen ct al., 2020b;a; Mohammadinejad et al., 2021; Zhang al., 2023; 2022), such researches only answer the Pass/Fail robustness verification questions (Jr. et al., 2024; Mistry al., 2022; Henzinger et al., 2021) (i.e., The yellow box in Fig. 1) of established models (i.e., whether the well-trained model is pass fail). In the event that the established model fails to pass formal verification, it becomes imperative iterate through the training process once again. This implies that the aforementioned process exhibits sub-optimal efficiency in real-time task execution on edge devices.
Many papers on formal verification of quantized/binarized NNs (Baluta et al., 2019; Paulsen et al., 2020b;a; Mohammadinejad et al., 2021; Zhang et al., 2023), which can verify given NN against a specific property, e.g., local robustness verification and maximum robustness radius computation in QVIP (Zhang et al., 2022). Our work aims to produce more robust Binary NN. with empirical evaluation based on a test dataset, similar to several comparison baselines (Qin et al., 2020; 2023; Shang et al., 2022). Note that the "upper bound" derived in our paper refers to the Lipschitz constant of the NN, metric that measures the maximum rate of change of the network's output with respect to changes in its input, but we do not address the formal verification problem (as we mentioned above). In real-world scenarios, the input noise may be unpredictable and encompass various types of noise unseen during the training phase, and without any upper bound on its radius or norm. Our work proposes a robust training algorithm for enhancing robustness against general input noise perturbations during the testing phase, and empirical experiments confirm its effectiveness.
The reason that our work and comparison baselines do not use formal verification is due the well-known scalability issue (in survey study (Meng et al., 2022)) that limits the size of NNs that can be formally verified with both sound- ness and completeness. Subsequently, numerous comprehensive analyses have been conducted to explore the intricate relationship between robustness (Shkolnik et al., 2020) and quantization (Lin et al., 2021). Thus, it is worth noting that quantization/binriizati does not hurt the robustness of the model when compared to its full- full-precision counterpart.
A.2. Proof of Theorem 1
Firstly, we employ several matrix norm inequalities to determine the upper bound of this discrepancy by using the rule from an example in Eqn.7. Then, we have
$ | 6,C,
b, l11
||
_
Io( n
Well wh-1. 1 2
(20)
12
1
{- || l|1 Io(
Welll w h1B 2
n
The establishment of inequality S is attributed to the rule of Eqn.9 and the problem definition of Def.4.1. Furthermore, we have
b.c Whoall W 111
- 11
(21)
The establishment of inequality is the contractive property of binary activation function and scaling factors (i.e., and a) combined with triangle inequality. To establish the tightness of the upper bound on robustness of DBNNs, it is crucial to thoroughly analyze the intermediate conclusion in Cor.A. I and subsequently conclude the proof for Thm.4.2.
The contractive inequality in the subsequent step is derived through a crucial Cor.A.1, which constitutes the pivotal aspect of the upper bound derivation in our work.
Corollary A.1. Given a standard matirx A and vector ž, we have ||A2 I ||A 11.0//100// 11
Let = [a. where E [1....,m] (1, ... , and == [=]] and I Imxm = [7, f2 ... ,.)) Here, [Fe] represents

---

Can DBNNs Robust to Environmental Noise for Resource--onstrrinne Scenarios?
the 7th term of 4. Then, have the following derivation CE= ( )):
Σ
aij z4
|
2/|2
||
max
max zạ0
==1
||
max 15
{ }
(22)
Σ 01 [tWl
max 15i5m
[[[t)]/]]
In order not to lose generality, it obvious that the number of rows (i.e., k) arbitrary, and thus we have:
| | 1,00 2
max ISksm
max ISksm
ansl
(23)
==1
f=1
Finally, the inequality of Cor.A.l can be proved as follows:
| x0
max Σ I5ssm
max Isism
Ond
max 18i5n
|zy]]
(24)
max
max
<
||A² 1,00
|
15
1SJSh
where the above inequality is established by using the matrix norm consistency theorem.
Utilizing the Cor.A.1, we continue to give the proof of the upper bound from Thm.4.2 as follows:
Whe
R.C, 112 ((W--1)?'la
|Flli I, (= WFll W -
(25)
1 ( [rrh w 2 {1x=)
| ll1 To
Subsequently, the contractive property of the 1-lipschitz continuous function is utilized to further process several scaling factors and binary activation functions in the Eqn.25, thereby organizing the continued product terms associated with the input of the initial layer. Based on the key corollary (i.e., Cor.A.I), we can derive the following two inequalities:
I - W l1 1( [[W,,
IIW 11,,00 IIW6 (h--
hi |N
L-1
L-1
on 11 Whoc 11
II [[(W--))'11
(26)
1=1
1=1
II (I(WM))TTle M=2
1 -
Finally, the conclusion of T.m.4.2 can proven.
The previous study (Gouk et al., 2021) has examined the disparity of the Lipschitz constants between the residual module and the convolutional layer, albeit acknowledging that calculating this aspect of DBNNs is relatively intricate. For the sake of theoretical accuracy, we provide upper bound for the residual connection module through subsequent corollary.
Firstly, in the standard ResNet backbone, the computation process of residual module (dre*(x)) can be described as follows:
x
x
+ (Øo+p o... d (x)
(27)
where p represents the maximum number of layers to span in a residual connection module. The existing study has given the following corollary through Lipschitz continuity:
12

---

Can DBNNs Robust to Environmental Noise for Resource-constrainee Scenarios?
Walking
20
MMMMMA MM
15
20
Jogging
20
-20
Figure The example data point corresponding to distinct human actions.
Corollary A.2 (from (Gouk et al., 2021)). We can find that the Lipschitz constant of a residual module is bounded by (Øres) 1 I L (0i). where L (0:) represents the Lipschitz constant of the i-th module of the neural network.
Based on the conclusion of Cor.A.2 and Eqn.26, we can add residual module of the upper bound the last term of the right-hand side in Eqn.28 follows:
L-1 II ( 11.00
an 11 Wh
1=1
1=1
hB II (I(WM))T 11,, 11h - hM 11,0)
(28)
M+P
k(1 +
II (I(W)) 11,00 11g - hig 1l M+1
where & denotes the total number of the residual module. In the case of employing ResNet18 as the backbone, the residual connections encompass a total of four BNN module layers, wherein each layer comprises two sets of residual modules. Consequently, the value is determined to be
A.3. Detailed Experimental Setup
Datasets: (1) The Bio time series dataset (Kaggle project) stems from the esteemed neural networks and cognitive models course offered by the MAI program at FHWS (Roeder, 2022). The dataset comprises data from 51 distinct subjects, each identified uniquely within the train and test sets by their IDs ranging from 1600 to 1650. Total 120K data points encapsulate low-level, high-frequency (20 Hz, every 50 milliseconds) time-series sensor readings, specifically the x, y, and coordinates recorded by the smartwatch's 's accelerometer. After introducing general input noise, we show the bio-metric signals corresponding to distinct human actions in Fig.5. (2) The CIFAR-10 is the most popular image classification dataset, which consists of 50,000 training samples and 10,000 testing samples of size 32 32 divided into 10 image classes. (3) The CIFAR-100 dataset similar to CIFAR-10 10 dataset, with the exception that it consists of 100 classes, each containing 600 images. (4) The ImageNet dataset provide 1.2 million training samples and 50,000 validation samples, distributed across a total 1,000 distinct classes to facilitate the exploration complex research tasks.
Experimental Setup Details: Datasets: Brain tumors can be classified as either malignant or benign. The growth of both types of tumors lead to increase in intracranial pressure. Early detection and accurate classification of brain tumors is critical research area within medical imaging, which significantly aids in selecting the most appropriate treatment strategy to save patients' lives. This dataset (Nickparvar, 2021) comprises 7,023 human brain MRI images categorized into four classes: glioma, meningioma, no tumor, and pituitary tumor. The dataset is divided into training, validation, and test sets following a conventional 7:2:1 ratio. Training Details: Firstly, we validated several DBNN-basee models with two common backbones (i.e., ResNet34 and ResNet18) on some image classification datasets, namely ABCNet (Lin et al., 2017).
13

---

Can DBNNS Robust to Environmental Noise for Resource-constrainee Scenarios?
BirealNeet (Liu al., 2018), ReActNet (Liu al., 2020), DorefaNet (Zhou al., 2016), IR-Net (Qin al., 2020), AdaBin (Tu et al., 2022) and CycleBNN (Fontana et al., 2024). The rationale for selecting the aforementioned baselines lies in their widespread recognition and high citations within the research community. To validate the robustness of the DBNN-based training algorithm by using various matrix norm constraint strategies, we compare our proposed method with the LCR method (Shang et al., 2022) on the ImageNet dataset. For the task of bio-electricity series classification, we opted to replace the 2D convolution module in the full-precision model with binary convolution modules (i.e., IRConv (Qin et al., 2020), BirealConv (Liu et al., 2018), and AdaBinConv (Tu et al., 2022)) that are currently popular to align with the requirements of shallow network, as opposed to deep networks used for image classification tasks. We also evaluate the clean test accuracy of our method with several baselines on the CIFAR-10 and CIFAR-100 datasets.
All experiments are executed on Linux server (i.e., Ubuntu 18.04.6 LTS) with one RTX 3090 GPUs. Specifically, we train all baselines and our method with SGD optimizer by using common hyper-parameterd (i.e., momentum=O.99 9, weight decay=le - 4) under PyTorch1. .11...3--H.. 13-GPU library on several classification benchmarks, namely CIFAR-10, CIFAR-100, ImageNet, Bio-electricity series and Brain Tumor MRI datasets. For image classification tasks, the batch size is set to 128 and then epoch set to 400. Furthermore, the initial learning rate 1e on the three image benchmark datasets. In addition, we follow the popular cosine of learning rate decay strategy in the survey (Qin et al., 2023). To understand the process of applying environmental noise in the manuscript, we give the pseudo-code in Algo.
Algorithm 1 Model with Environmental Noise Perturbations during Inference
Input: Sample x, size m from Dataset D
Require: Dateset D for Test.
1: for size of test set D do
2: Sample æ from D.
!/Apply the environmental noise for each test sample.
3: for to size of test samples N do
4: for i to total number of pixel noise points P do
5: randa- atp.random.raddinn (1, h 1) //Random height position (h). w - 1) //Random width position (w).
6:
randy- = rap. if
7:
& 0.5 and j < then æIj :, randa, randy] = at a[j, in randæ, randy) + e //Use function random to generate random noise perturbations c.
8:
9: end if
10: end for
11: end for
12: end for
13: Return: The test sample with environmental noise perturbations during inference.
Evaluation: Following the evaluation method of a previous robustness study (Zhang et al., 2021), we employ two metrics, namely 'No Noise' (representing clean test accuracy) and 'With Noise" (indicating test accuracy under environmental noise with high intensity signal-to- segnal-tonnoise noise ratio (SNR) set at nearly 50%) to evaluate the performance of all BNN-based methods.
A.4. Additional Experiments
Clean Test Accuracy Comparison: The study (Shang et al., 2022) claims that the norm constraints by using the Lipschitz continuous to enhance test accuracy of BNN-based models under the clean scenarios. To verify such capability, we conduct several experiments to evaluate our strategy on the CIFAR-10 dataset, as shown in Tab.6.
In particular, we choose the IRNet (Qin al., 2020) as our fundamental BNN-based model for conducting comparative experiments due to its demonstrated effectiveness in the field of BNNs, emphasized in several studies (Tu al., 2022; Shang et al., 2022). According to the experimental setups of the study (Shang et al., 2022), we employ ResNet18 and ResNet20 as the two backbone networks. Then, we construct experiments by employing our proposed method for IRNet, denoted as IRNet+our. For the ResNet18 backbone, the IRNet+our method has significantly enhanced the clean test accuracy of IRNet method, surpassing the performance achieved by the LCR method. Then, the proposed method demonstrates enhanced test accuracy for the IRN method, even when the depth of the backbone network increased to ResNet20.
14

---

Can DBNNs Robust to Environmental Noise for Resource-constrainee Scenarios?
Table Performance comparison between the method and several SOTA methods the clean CIFAR-10 dataset. Here, the (W/A)
denotes the weights and activations of Bit- width. Here, notation indicates that the activations are in full precision.
Backbones
Methods
Bit-width (W/A) Clean Test Accuracy
Full Precision (Qin et al.. 2023)
32/32
94.82
IRNet (Qin et al., 2020)
1/1
91.50
8
LCR (Shang et al., 2022)
1/1
91.80
IRNet+our
1/1
94.061
Full Precision (Qin et al., 2023)
32/32
91.99
IRNet (Qin et al., 2020)
I/1
85,50
ResNet20
LCR (Shang et al., 2022)
1/1
86.00
our
1/1
88.191
IRNet* (Qin et al., 2020)
1/32
90.80
ResNet20
LCR* (Shang et al., 2022)
1/32
91.20
IRNet+our"
1/32
91.411
Table 7. Investigation of the Influence of our constraint coefficient on the robustness of various BNNs against environmental noise on the CIFAR-10 dataset.
Methods
Backbones
Noise
Coefficient
Test Acc.
AdaBin
ResNet34
Noise
N/A
87.55
AdaBin+our
ResNet34
Noise
5e-3
87.85 f
AdaBintour
ResNet34
Noise
5e-2
88.08
DoReFa
ResNet34
Noise
N/A
85.16
DoReFatour
ResNet34
Noise
5e-3
87.04
DoReFa+our
ResNet34
Noise
5e-2
86.92 f
ReAct
ResNet34
Noise
N/A
87.87
ReAct+our
ResNet34
Noise
5e-3
89.00 1
ReAct+our
ResNet34
Noise
5e-2
89.00 1
Bireal
ResNet34
Noise
N/A
87.80
Bireal+our
ResNet34
Noise
5e-3
90.13 1
Bireal+our
ResNet34
Noise
5e-2
90.40 1
IRNet
ResNet34
Noise
N/A
87.88
IRNet+our
ResNet34
Noise
5e-3
87.93 1
IRNet+our
ResNet34
Noise
5e-2
88.31 1
Meanwhile, we can observe a slight improvement in the IRNet+our method than the LCR method when the activation of the model is 32-bit. Thus, the above empirical results demonstrate that the application of our -nor constraints can enhance the performance BNN-based models clean scenarios, surpassing the effectiveness spectral norm (Shang al., 2022) constraints.
The Influence of Our Constraint Coefficient Robustness: Clearly, the distinct design of the BNN-based methods imply that our provided general noise disturbance bound constraints can enhance the environmental noise robustness of various BNNs to varying degrees. To further imporve robustness of such B.NN-based methods (Zhou et al.. 2016; Liu et al., 2018; 2020; Qin et al., 2020; Tu et al., 2022), we conducted several quantitative experiments for the constraint coefficient on the CIFAR-10 dataset, as illustrated Tab.7. Here, the upward arrow in Tab.7 denotes the enhanced robustness of several BNN-based methods (Zhou al., 2016; Liu et al., 2018; 2020; Qin al., 2020; Tu et al., 2022) by using our L1,00 constraints. Specifically, the ReAct method (Liu al., 2020) exhibits insensitivity to variations in constraint coefficients following the application of our approach. For the DoReFa method (Zhou al., 2016), smaller constraint coefficient (i.e., 5e 3) demonstrates enhanced robustness. Conversely, for the remaining methods (Liu et al., 2018; Qin et al., 2020; Tu et al., 2022) utilizing our T1,00 norm constraint term, larger coefficient demonstrates enhanced robustness. In addition, our proposed 1 norm constraint term for the latest AdaBin method (Tu et al., 2022) should be carefully, as it is highly susceptible to gradient calculation anomalies (i.e., NAN) when the coefficients are the le
The Trade-offs between Clean Test Accuracy and Robustness Our T norm constraint method: is worth
15

---

Can DBNNs Robust to Environmental Noise for Resource-constraineel Scenarios?
On the Biometriic Time Series Classification Task
90
80
70
60
50
Epochs
Figure 6. The accuracy curves on bio-electricity time series dataset.
Table Analysis of the clean test accuracy of our proposed L1, norm constraint strategy with baseline models on the Brain Tumor MRI datasct
Methods
Test Accuracy(clean)
ResNet18(FP33)
92.58
ResNet34(FP32)
90.32
ResNet18-Cyclebmm
84.84
ResNet34-Cycleenn
72.35
ResNee18-Cycddbnnnoor
89.98
ResNet34Cycdebmntour
82.26
noting that a certain degree of under- under-fitting may occur in some full precision CNN-based models due to the utilization of regularization techniques (Wei & Zhao, 2024). In other words, the model's accuracy on clean test data is observed to be lower than its performance in noisy scenarios. To investigate the impact of the aforementioned phenomenon on the DBNN-base model, a series of experiments is presented in Tab. 10. Specifically, it can be observed that only subset of BNNs exhibit marginal decline in terms of clean test accuracy when the proposed L1,00 constraint loss function is employed. Our method achieves a remarkably low loss of 0.06 for the ABC+our, specifically when employing the ResNet18 backbone network. Conversely, IRNet exhibits slightly performance loss 0.66 with the proposed method. The above experiments demonstrate that our proposed method strikes an appropriate balance between clean test accuracy and robustness. In Tab.8, we can find that the clean test accuracy of our proposed -norm constraint strategy with baseline models on the Brain Tumor MRI dataset. The experimental results show that our proposed strategy can also improve the clean classification accuracy of the model during inference.
Investigation of the impact of the L norm constraint approach on the robustness of the full precision model: The L1.30 constraints are commonly believed to enhance the robustness of full full-precision models. However, due to the quantization operation, there exists a certain disparity between the calculation processes of deep CNNs and BNNs. Hence, we present a case analysis (on the CIFAR-100 100 dataset) in Tab.9. Specifically, we observe that the proposed method enhances robustness (i.e., increasing 0.5%) deeper structures (i.e., ResNet34), while leading to performance degradation in shallower structures (i.c., ResNet18). Firstly, several experimental results in Tab.2 and Tab.9 show that our proposed method is specifically for improving the robustness of RBNN-based models. Then, the above improving diserepancy primarily stems from the scaling factor and XNOR operation employed in DBNN-based models, which are not present in full precision models. Consequently, the robustness improvement offered by our approach is limited for full-precision models with specific ResNet backbone.
The Effectiveness of Different L-norm Constraints on the Robustness of DBNNs: To comprehensive understanding the effectiveness of various L-norm constraints for the robustness of DBNNs against environmental noise, we conducted
16

---

Can DBNNs Robust to Environmental Noise for Resource-constrainee Scenarios?
Table Analysis of the robustness effect of our proposed -nor constraint for full precision model the CIFAR-100 dataset
Methods
Test Acc. Acc.with Noise
ResNet18
71.96
ResNet18
69.33.
ResNet34
71.23
ResNet34+our 71.73
Table 10. The Trade-off' between Clean Test Accuracy and Robustness of our L1.s0 constraint method on the CIFARI00 dataset
Methods
Scenarios Backbones
DoReFatour
ABC+our Bireal+our ReAct+our
Netttrrur CycleBNN+our
Clean
ResNer18
67.59
69.171
68.96
68.20L
69.301
67.35
Noise
ResNet18
67.21
69.23
68.84
68.26
70.04
67.29
Clean
ResNet34
61.18
63.34]
64.11I
64.05
61.461
68.90
Noise
ResNet34
61.16
63.47
64.69
63.81
61.71
68.36
relevant experiments the CIFAR-10 and ImageNet in Tab.11. To begin with, adopt similar experimental setups from the LCR method (Shang et al., 2022), wherein the ResNet18 backbone network is trained using the BirealNet model for LCR and our approaches on the ImageNet dataset. To verify the extensibility of our bounds, a deeper ResNet34 is provided as the backhone network. Recently, some L-norm constraints have been proposed to improve the robustness of CNN-based model. Thus, we construct experiments to analyze how much they improve the robustness of DBNNs. Specifically, Lii represents the spectral norm constraint after the approximate operations of the LCR method (Shang al., 2022). Then, the special norm constraints L2 and Les are derived from the studies (Li al., 2019) and (Kanai et al., 2020), respectively.
In Tab. I, experimental results demonstrate that proposed L1 norm constraint can enhance robustness of the Bireal method by 0.96% compared to the LCR method (Shang et al., 2022) when applied to the ImageNet dataset. Meanwhile, the experimental results show that our 11cc norm constraint can improve robustness by 1.98% compared to the Bireal (Liu et al., 2018) on the CIFAR-10 10 dataset. Furthermore, the robustmess BNNs by using our -nor constraint surpasses the L1. L2 and Loo norm constraints on the CIFAR-I0 and ImageNet datasets. This means that our -nor constraints can bring effective robustness improvement under environmental noise perturbations the inference phase DBNNs. In addition, another advantage of our -norm noom constraint is that can also improve the robustness of existing constraint methods, such as the 'LCR. Bireelloul (L,,0)' in Tab.11.
The Effectiveness of Our Constraints on the Distribution of Learnable Parameters: An intriguing inquiry pertains to the impact of matrix norm constraints, such as L2 or L1,00 on the learnable parameters within DBNNs. Revealing this inquiry, we employ visual method to compare our proposed method with IRNet (Qin et al., 2020) on the CIFAR-10 dataset. In Fig.7(a 7(a) and fig.7(b), the learnable parameter distribution of IRNet (Qin et al., 2020) exhibits a unimodal form, which may potentially limit the model's capacity capture complex features and consequently hinder its performance. Then, the learnable parameters distribution of the IRNet using our 11 norm constraint, exhibits smoother histogram profile compared to IRNet, illustrated in Fig.7 7(c) and Fdg.7(d). Meanwhile, the study (Liu et al., 2021) has been shown that
Table 11. Robustness comparison of our and SOTA methods using different L-norm constraints on the CIFAR-10 and ImageNet datascts.
Datasets
Methods with I-norm Constraints
Backbones
Scenario
Test Accuracy with Noise
ImageNet
LCR BCR.Birel++Lipp
ResNet18
Noise
57.27
ImageNet
LCR LC
ResNet18
Noise
57.84
ImageNet
LCR_ LCCR_Birea++I2 (Li et al., 2019)
ResNet18
Noise
57.98
ImageNet
S.R.Birea++L. (Kanai et al., 2020)
ResNet18
Noise
58.07
ImageNet
Bireal+our (L1,0c)
ResNer18
Noise
58.23
ImageNet
LCR_Bircal+oour (L1,00)
ResNet18
Noise
58,35
CIFAR-I0
Bireal+Li
ResNet34
Noise
88,15
CIFAR-10
Bireal+L, al., 2019)
ResNet34
Noise
88.34
CIFAR-10
Bireal+ (Kanai al., 2020)
ResNet34
Noise
88.48
CIFAR-10
Bireal+our (I,,0)
ResNet34
Noise
90.13
17

---

Can DBNNs Robust to Environmental Noise for Resource-constrainne Scenarios?
300
00
(a) Netet layerl conv2
(b) IRNet layer3 conv2
(c) Our+IRNet layerl conv2
(d) Our+IRNet layer3 conv2
Figure 7. The distributions of learnable parameters in two binary convolution layers are compared on the CIFAR- by using our method
and IRNet.
Tyat: 5
Trat
Teab
TraE:
CydeBNN
Prodoood
Pedictess
Prid cleb
Fediledd
Preckned
Prdicted:
Prediced
Proditedt
Wred ciet
Fodicted 3
Incorrect number of categories: 4
Presieed
Predicec13
Feekdeed33
Preceted
Pred
Prodicte::
Fadisted
Iredited
Fred clrd
Frod ciecc
CydeBNN+ Our
PNEPASON 0
Prokilest
Prodileddd
Wesktel 0
Phokiles:
Pookled-
Phod GINS
Incorrect number of categories 2
Figure 8. Visual comparison between the proposed strategy with CycleBNN (i.e., Yellow rectangle with rounded corners) and standard CycleBNN model (Fontana et al., 2024) on the brain tumor test dataset under environmental noise perturbations.
restoring the diversity of learnable parameter distribution by special optimization settings can improve the performance of BNNs. Our L1,00 norm constraint introduces more peaks in Fig.7(c) and Fig.7(d), indicating an positive impact of our approach on restoring the diversity of learnable parameters.
The Visualization of Our Strategy with Several Binary Convolution Modules on the Bio-electrical (Roeder, 2022) dataset: To show the advantages of the proposed strategy over several popular binary convolution modules against general environmental noise perturbations, we present its accuracy rate iterated over epochs in visual manner shown in Fig.6. 6.
The Visualization of Our Strategy with CycleBNN (Fontana et al., 2024) model in the Brain tumor image classifica- tion (Nickparvar, 2021) dataset: To validate the effectiveness of the proposed constraint training strategy in enhancing robustness for a real-world resource-constrained task, we select the open-source brain tumor MRI task. Visual comparison results as shown in Fig.8. Specifically, we randomly chose a batch of 20 samples perturbed by environmental noise to visually demonstrate the improvement. Then, the model employs R as the backbone network, and the most recent CycleBNN (Fontana et al., 2024) serves the BNN algorithm. The environmental noise level is within the range of 0.05 to 0.1, which closely mimics the environmental noise characteristics of MRI image data. Additionally, the SNR is enhanced by 50%. Here, the well-trained CycleBNN by using our proposed 1 -norm constraints (i.e., Yellow rectangle with rounded corners) exhibits an un-robust rate in the inference that is half that of the CycleBNN model (i.e., Blue rectangle with rounded corners). addition, our strategy achieves 90% test accuracy in inference tasks involving random environmental noise, which is sufficient to meet the practical requirements for assisting primary care physicians diagnostic decision decision-making. making.
-norm
Analysis of Additional Computational Overhead: Specifically, we have evaluated the training cost of a series of BNN- based methods CIFAR-100 100 and Brain Tumor datasets. We have reported the average training time of each epoch and total test time. According to the results presented in the main paper and two table below, it can be observed that an appropriately designed penalty term not only enhances model robustness but also significantly improves training efficiency. In addition, the overhead in the test phase also encompasses the time required for adding environmental noise. The detailed computational overhead are presented Tab. 12-13.
Analysis of the PGD Attack for DBNNs: We opted for a standard training and testing phase to introduce PGD attacks,
18

---

Can DBNNs Robust to Environmental Noise for Resource-constrainne Scenarios?
Table 12. Computational cost comparison between popular BNN-based methods and our the CIFAR-100 dataset.
Methods (ResNet18 backbone)
Training Time/Epochs (s)
Total Test Time (s)
CycleBNN
24
43
CycleBNN+Our
21
42
IRNet
13
IRr
12
Dorefa
Dorefa+Our
11
33
React
14
33
React+Our
13
33
Bireal
13
30
Bireal+Our
12
30
ABCNet
13
41
ABCNet+Our
12
41
Table 13. Computational cost comparison between BNNs and our on the Brain tumor dataset.
Methods (ResNet18 backbone) Training Time/Epochs (s) Total Test Time (s)
CycleBNN
13
CycleBNN+Our
11
Dorefa
11
Dorefa+Our
10
React
6
React+Our
5
4
Table 14. PGD performance comparison between CycleBNN and the CIRFA-100 dataset.
Models
Test PGD
CycleBNN
17.78
CycleBNNNOur
18.38
19

---

Can DBNNs Robust to Environmental Noise for Resource-constrainne Scenarios?
Table 15. Ablation study I: The contribution of constrained binary weights and scaling factors to improving environmental noise robustness of DBNNS on the CIRFA-100 dataset. The w/o indicates that the constraint on the scaling factors has been removed.
Buckhone Dorefa-Our Dores=Our-wddo React=Our React=Ourewio ABC44Our ABC-Ourewle IRNet=Our IRNe:+Oum-wdr Bircal-Our Bireal-Ourwwo CydkBNK-Our CydeBiNN-Our-wl
ResNetl5 67.31
68.36
67.26
69.23
68.37
70,04
68.74
68.84
67.19
67.29
66.31
Table 16. Ablation study 2: The contribution of constrained binary weights and scaling factors to improving environmental noise robustness of DBNNs on the Brain tumor dataset. The w/o indicates that the constraint the scaling factors has heen removed.
Backbone Dorefa+Our Dorefa+Our+w/o React+Our React+Ourtw/o CycleBNN+Our CycleBNN+Our-w/o ResNer18
86.91
85.62
83.87
82.39
85,46
84.17
thereby evaluating the effectiveness of the proposed approach in defending adversarial attacks. Specifically, we use the LinfPGDAttack method from the library advertorch, with step size set to 7 and perturbation value set to 1/225. Then, we added such adversarial perturbations to the test set the CIFAR-100 task and measured the CycleBNN with ResNet18 backbone and our proposed method. Tab. 14, experimental results demonstrate that DBNN models, which have not been trained with adversarial samples, are highly susceptible to strong PGD attacks (Similar issues have been reported in full-precision models during both standard training and adversarial attack testing). The proposed method still has certain percentage improvement despite the serious degradation of the performance of unconstrained DBNN models.
Ablation Analysis for DBNNs: We found that the constraints on binary weights and scaling factors are indispensable to improve the robustness of DBNNs
20


Reviews:
[{"review_id": "caKzBllJ3Q", "reviewer": "Reviewer_PX4V", "summary": "In this work, the authors investigate the robustness of deep binary neural networks (DBNNs) under environmental noise perturbations in resource-constrained scenarios. Specificity, the authors propose an $L_{1,\\infty}$-norm constraint on objective function for binary weights to derive a tighter robustness upper bound and a low computational overhead training algorithm compared to existing methods. Then, the authors conduct extensive experiments on three benchmark image datasets (i.e., CIFAR-10, 100 & ImageNet) as well as medical datasets (i.e., bio-electrical signals, brain tumor MRI), validating the proposed algorithm by enhancing the robustness of major BNN algorithms.\n\n## Rebuttal summary\n\nThis paper studies an interesting problem about the robustness of deep binary neural networks under environmental noise perturbations in resource-constrained scenarios. My initial concerns have been resolved, so I keep a positive score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: Authors should give priority to the visual results of medical image classification in the main text.\n\nA1: We extend our gratitude to the reviewer for providing this constructive suggestion. We fully concur with the suggestion to include visual results in the main text to effectively demonstrate the efficacy of the proposed method. The visualized results of brain tumors have been relocated to the experiment section.\n\nQ2: The reviewer wants to know more about the cost of the BNN algorithm under environmental noise perturbations.\n\nA2: Thank you very much for the valuable comment. We concur that, in addition to comparing the computational overhead with the LCR method (SOTA), which investigates BNN robustness, it is also essential to explore how the proposed $L_{1,\\infty}$-norm constraints loss function influence the computational overhead of various popular BNN algorithms. To expedite the acquisition of experimental results, we measured the actual computation time on an A800 GPU. \n\nSpecifically, we have evaluated the training cost of a series of BNN-based methods on CIFAR-100 and Brain Tumor datasets. We have reported the average training time of each epoch and total test time. According to the results presented in the main paper and two table below, it can be observed that an appropriately designed penalty term not only enhances model robustness but also significantly improves training efficiency. In addition, the overhead in the test phase also encompasses the time required for adding environmental noise. The detailed computational overhead are presented below:\n\nTable 1. Computational cost comparison between popular BNN-based methods and our on the CIFAR-100 dataset.\n| **Methods (ResNet18 backbone)** | **Training Time/Epochs (s)** | **Total Test Time (s)** |\n|:-------------------------------:|:----------------------------:|:-----------------------:|\n| CycleBNN                        | 24                           | 43                      |\n| CycleBNN+Our                    | **21**                    | **42**             |\n| IRNet                           | 13                           | 32                      |\n| IRNet+Our                       | **12**                  | **31**            |\n| Dorefa                          | 11                           | 33                      |\n| Dorefa+Our                      | 11                           | 33                      |\n| React                           | 14                           | 33                      |\n| React+Our                       | **13**                  | 33                      |\n| Bireal                          | 13                           | 30                      |\n| Bireal+Our                      | **12**                  | 30                      |\n| ABCNet                          | 13                           | 41                      |\n| ABCNet+Our                      | **12**                  | 41                      |\n\nTable 2. Computational cost comparison between BNNs and our on the Brain tumor dataset.\n| **Methods (ResNet18 backbone)** | **Training Time/Epochs (s)** | **Total Test Time (s)** |\n|:-------------------------------:|:----------------------------:|:-----------------------:|\n| CycleBNN                        | 13                           | 8                       |\n| CycleBNN+Our                    | **11**                    | 8                       |\n| Dorefa                          | 11                           | 3                       |\n| Dorefa+Our                      | **10**                 | 3                       |\n| React                           | 6                            | 4                       |\n| React+Our                       | **5**                   | 4                       |", "claims_and_evidence": "In this work, the claims are well-supported. Firstly, the theoretical perspective establishes a tighter upper bound on environmental noise robustness compared to prior studies and further elucidates the quantitative relationships, thereby offering a clear explanation of the robustness in DBNNs-based model during inference. Finally, the authors conduct extensive experiments on three benchmark image datasets (i.e., CIFAR-10, 100 & ImageNet) as well as medical datasets (i.e., bio-electrical signals, brain tumor MRI), validating the proposed algorithm by enhancing the robustness of major BNN algorithms.", "methods_and_evaluation_criteria": "The proposed method targets the interference of environmental noise during the inference stage. Specifically, the author uses test accuracy in a noisy environment as the evaluation metric, which is highly relevant to the research areas.", "theoretical_claims": "The theoretical analysis of this manuscript is accurate and rigorous.", "experimental_designs_or_analyses": "The experimental setup of this manuscript is accurate and comprehensive. The experimental results are well verified on three image classification datasets. In addition, the advantages of the algorithm in medical time series and image tasks are fully visualized. \nIt would be better if the authors could provide more analysis of the computational overhead.", "relation_to_broader_scientific_literature": "The paper effectively situates its contributions within the broader scientific literature of DBNN-based methods, specifically addressing environmental noise perturbations in resource-constrained scenarios. It mainly explains the robustness of DBNNs in specific scenarios, and also relieves the problem of high robust training cost faced by existing methods.", "essential_references_not_discussed": "This manuscript discusses the relevant literature.", "other_strengths_and_weaknesses": "Strengths\n1. The background of the issue examined in this manuscript is both critical and intriguing, particularly considering the paucity of theoretical studies that have explored the robustness of DBNN in environmental noise scenarios. \n2. In this paper, a low-cost and robust training algorithm is developed from the perspective of dual norm constraints, and a more compact and noise-robust upper bound is presented.\n3. The visualization results for both medical image and standard image classification tasks demonstrate the significant effectiveness of the proposed algorithm. Specifically, it elucidates the extent to which ambient noise perturbation interferes with the binarized convolution component of the model.\nWeaknesses\n1. This manuscript conducts several comprehensive experiments to verify the advantage of the proposed method on different types of datasets. It seems that model overhead analysis is only available on the ImageNet dataset. It would be nice to analyze the overhead of more BNN algorithms under noise perturbations on other datasets.", "other_comments_or_suggestions": "1. Authors should give priority to the visual results of medical image classification in the main text. After all, the results on real tasks are better than tables to visualize the advantages of the proposed approach.\n2. It would be better if the authors could provide more BNN-based algorithm overhead under environment noise on the medical dataset.", "questions_for_authors": "1. The reviewer wants to know more about the cost of the BNN algorithm under environmental noise perturbations.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2JWhZBX3MN", "reviewer": "Reviewer_1Wt1", "summary": "The paper investigates whether Deep Binary Neural Networks (DBNNs) can be robust to environmental noise, particularly in resource-constrained scenarios such as bio-electrical signal classification and medical imaging. The authors identify that DBNNs' robustness vulnerabilities stem from binary weights and scaling factors. To address this, they propose a L1,∞-norm constraint for binary weights and scaling factors, which they claim provides a tighter upper bound on noise perturbations compared to state-of-the-art (SOTA) methods.\n\nTheir approach involves:\n1. Theoretical Analysis: They derive a formal noise perturbation upper bound for DBNNs using L1,∞-norm constraints.\n2. Robust Training Framework: The proposed L1,∞-norm constraint is incorporated into the training process to enhance DBNN robustness.\n3. Experimental Validation: Their method is tested on five classification datasets, including CIFAR-10, CIFAR-100, Brain Tumor MRI, and bio-electrical signal datasets. Results show improved robustness, with up to 4.8% and 5.4% improvements on CIFAR-100 and Brain Tumor MRI, respectively.\n4. Computational Efficiency: Their method reduces additional training overhead compared to previous methods.\n\nThe study concludes that L1,∞-norm constraints effectively mitigate the impact of environmental noise on DBNNs while maintaining efficiency, making it suitable for edge devices in safety-critical tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks to the reviewers for their positive comments and constructive comments.\n\nQ1. However, its significance is somewhat limited by the lack of real-world deployment or validation on actual edge devices.\n\nA1. We apologize for the fact that the inference experiments on actual edge devices.  However, the PyTorch framework can integrate ONNX third-party libraries to support DBNN model transformation and facilitate deployment on the simulation platform of edge devices (i.e., Raspberry Pi Debian 12). Specifically, the CycleBNN method with a ResNet18 backbone, after being trained with our $L_{1,\\infty}$-norm constraint loss, resulted in a checkpoint file size of 45.34MB. This is significantly smaller than the 8GB storage capacity of the Raspberry Pi 4B+ device. The inference of a single image under environmental noise takes only 1.6 seconds, which is deemed acceptable given that brain tumor-related tasks typically do not demand real-time processing capabilities. In particular, the performance of the Raspberry PI simulation environment provided by Docker is lower than that of the actual device. This indicates substantial potential for deploying our proposed method on the actual edge device.\n\nQ2. The clarity of theoretical explanations and experimental results is generally strong, but some assumptions, such as the applicability of the bound across architectures.\n\nA2. Thank you for the reviewer's valuable suggestions. We agree that the applicability of the bound across architectures plays an important role in ensuring the theoretical robustness analysis of DBNNs under environmental noise perturbations. We clarify that the DBNN-based model discussed primarily focuses on the binarization of weights and activations of CNN-based models. However, the multi-head self-attention mechanism in transformer-based models differs significantly from the convolution operations in CNNs, and ResNet-based backbones also lack a positional embedding layer. Thus, potential issues for applications of bound across architectures are more likely to stem from substantial architectural differences rather than from the assumptions of theoretical analysis. Nevertheless, we will conduct a rigorous analysis of the upper bound of environmental noise perturbations in both the position embedding layer and the multi-head self-attention layer after binarization, thereby extending the existing conclusions to the Transformer model.", "claims_and_evidence": "1. Claim: DBNNs' robustness vulnerability comes from binary weights and scaling factors.\nEvidence: The authors provide a theoretical analysis showing that binary weights and scaling factors contribute to noise sensitivity. They derive an upper bound on noise perturbations using L1,∞-norm constraints, which is compared to prior approaches.\n\n2. Claim: The proposed L1,∞-norm constraint provides a tighter upper bound than existing methods.\nEvidence: Theoretical derivations show that the L1,∞-norm constraint offers a more restrictive bound on noise effects compared to spectral norm-based constraints. This is further supported by quantitative comparisons.\n\n3. Claim: The proposed method improves robustness across multiple DBNN architectures.\nEvidence: The authors conduct experiments on five classification datasets (CIFAR-10, CIFAR-100, Brain Tumor MRI, bio-electrical signals). The results show robustness improvements (up to 4.8% on CIFAR-100 and 5.4% on Brain Tumor MRI), validating their claim.\n\n4. Claim: The method reduces additional computational overhead compared to previous robustness approaches.\nEvidence: They compare training/testing time with the LCR (Shang et al., 2022) method and show a 16% reduction in training time while maintaining performance.\n\nProblematic Claims:\n*** Claim: The method is broadly applicable to real-world safety-critical tasks.\nIssue: The experiments are conducted only on standardized datasets, without real-world deployment on actual medical or edge devices. Practical applicability remains untested. Also, the CIFAR datasets are considered too small scaled for now-a-day studies.\n\n*** Claim: The proposed method is universally effective across DBNN architectures.\nIssue: The study focuses mainly on ResNet-based DBNNs. It is unclear how well it generalizes to non-ResNet architectures or tasks beyond classification. for example transformer architectures.", "methods_and_evaluation_criteria": "Yes, the proposed methods and evaluation criteria make sense for the problem. The L1,∞-norm constraint is a reasonable approach to enhance robustness in Deep Binary Neural Networks (DBNNs), and the authors justify its effectiveness through theoretical derivations and empirical validation.\n\nFor evaluation, the authors use five benchmark datasets (CIFAR-10, CIFAR-100, Brain Tumor MRI, bio-electrical signals, and ImageNet), which are commonly used in robustness studies, but are fairly small scale. They introduce environmental noise perturbations and measure test accuracy under noise, which is an appropriate metric for assessing robustness.", "theoretical_claims": "The proofs seem mathematically valid and logically structured. However, the assumptions about scaling factors and generalization to different architectures require additional empirical verification. A formal empirical validation comparing actual perturbation bounds across architectures would strengthen confidence in the claims.", "experimental_designs_or_analyses": "The experimental design is generally well-structured for evaluating the robustness of DBNNs to environmental noise, using five datasets (CIFAR-10, CIFAR-100, Brain Tumor MRI, bio-electrical signals, and ImageNet) and measuring classification accuracy under noise perturbations. However, the study lacks real-world deployment tests on edge devices, making it unclear how well the method generalizes to practical applications. The noise perturbation strategy is reasonable but does not appear to be based on real-world noise distributions, which could affect its applicability. While the comparison against four BNN-based models and the LCR method (Shang et al., 2022) is fair, the study does not include adversarial robustness methods, which would provide a more complete assessment. Additionally, while computational efficiency is analyzed in terms of training time, there is no analysis of inference speed and memory usage on actual resource-limited devices, which is crucial for assessing its practical feasibility. Overall, the experiments demonstrate improvements in robustness but leave important questions about real-world applicability unanswered.", "relation_to_broader_scientific_literature": "The paper builds on prior work in BNNs and robustness methods by addressing DBNNs' vulnerability to environmental noise, extending Lipschitz-based constraints (Gouk et al., 2021; Miyato et al., 2018) with an L1,∞-norm constraint that is more efficient than the LCR method (Shang et al., 2022). It improves robustness for resource-constrained applications like medical signal processing, complementing adversarial robustness studies (Gowal et al., 2018; Balunovic & Vechev, 2020) by focusing on random noise instead of attacks.", "essential_references_not_discussed": "I'm not familiar with this field", "other_strengths_and_weaknesses": "The paper presents an original approach by adapting Lipschitz-based robustness constraints to DBNNs, offering a novel L1,∞-norm constraint that improves robustness with lower computational cost, which is a meaningful contribution to lightweight model research. It effectively highlights a practical issue—environmental noise affecting DBNNs in resource-constrained scenarios—that has been underexplored compared to adversarial robustness. \n\nHowever, its significance is somewhat limited by the lack of real-world deployment or validation on actual edge devices, making it unclear how well the method translates to practical applications. The clarity of theoretical explanations and experimental results is generally strong, but some assumptions, such as the applicability of the bound across architectures, lack thorough empirical verification. While the work is an important step toward improving DBNN robustness, further validation in real-world conditions would enhance its impact.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3PKDEKYxit", "reviewer": "Reviewer_FmSJ", "summary": "This paper addresses the robustness of deep binary neural networks (DBNNs) under environmental noise perturbations in resource-constrained scenarios. The authors propose an $L_{1,\\infty}$-norm constraint on binary weights and scaling factors to derive a tighter robustness upper bound compared to existing methods. Experiments on image classification (CIFAR, ImageNet) and medical tasks (bio-electrical signals, brain tumor MRI) demonstrate improved robustness with minimal computational overhead. While the work is well-motivated and technically sound, several aspects require strengthening for impact.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1.The reviewer wants to know how robust of proposed algorithm can be against adversarial attacks with latest BNNs (e.g., CycleBNN, Fontana et al., 2024).\n\nA1. Thanks for the reviewer's nice suggestion. We opted for a standard training and testing phase to introduce PGD attacks, thereby evaluating the effectiveness of the proposed approach in defending adversarial attacks. This mode more closely with the hypothesis considered in our manuscript, which posits that noise is encountered exclusively during the inference phase. Specifically, we use the LinfPGDAttack method from the library advertorch, with step size set to 7 and perturbation value set to $\\epsilon=1/255$. Then, we added such adversarial perturbations to the test set of the CIFAR-100 task and measured the CycleBNN with ResNet18 backbone and our proposed method. In below table, experimental results demonstrate that DBNN models, which have not been trained with adversarial samples, are highly susceptible to strong PGD attacks (Similar issues have been reported in full-precision models during both standard training and adversarial attack testing [A1]). The proposed method still has a certain percentage improvement despite the serious degradation of the performance of unconstrained DBNN models.\n\nTable 1. PGD performance comparison between CycleBNN and our on the CIRFA-100 dataset.\n| **Models**   | **Test PGD** |\n|:------------:|:------------:|\n| CycleBNN     | 17.78        |\n| CycleBNN+Our | **18.38** |\n\n\n[A1] Towards Deep Learning Models Resistant to Adversarial Attacks. ICLR (Poster) 2018.\n\nQ2. The theoretical analysis assumes Lipschitz continuity of binarized activation but does not provide the residual block analysis.\n\nA2. Thank you very much for your comment, we also think that the Lipschitz continuous analysis of residual structure is very important. Due to space constraints, the corollary and relevant proof of this part are located in the Appendix of the manuscript. Please refer to Corollary A.2 (Appendix) on page 12. For the camera version, we will move it into the main paper.\n\nQ3. Increase font size in Fig.4 for clearer visualization of feature maps.\n\nQ3. Please forgive our mistakes, we have set the font size to 24 to make the text description in Figure 4 clearer.\n\nQ4. Discuss potential risks of deploying noise-robust DBNNs in safety-critical medical applications (e.g., false negatives under extreme noise).\n\nA4. Thank you very much for your constructive suggestions. First, false negatives represent a significant challenge in the field of medical imaging. However, if imaging alone cannot provide a definitive determination, a biopsy can be conducted, and its subsequent pathological analysis serves as the gold standard for diagnosis. In practical MRI, extreme noise perturbations caused by patient movement can occur. In such cases, experienced senior physicians on the medical team will implement appropriate corrective measures. This indicates that our proposed method is unlikely to encounter such issues under normal environmental noise conditions.", "claims_and_evidence": "The claims in the paper are mostly well-supported. Specifically, the authors clearly demonstrate empirical gains compared to competitive baselines (ResNet 18/34, multiple BNN variants like BiRealNet, IR-Net, CycleBNN, etc.) on multiple image classification (CIFAR, ImageNet) and medical tasks (bio-electrical signals, brain tumor MRI) tasks. The theoretical claims regarding provided tightness bounds are sound and clearly explained.", "methods_and_evaluation_criteria": "The proposed robust training algorithm and the chosen evaluation criteria (five diverse image and medical tasks covering classification) are highly relevant and sensible for the problem studied.", "theoretical_claims": "The proofs and theoretical arguments appear technically correct and rigorous.", "experimental_designs_or_analyses": "The experimental design is sound and thorough. Results across multiple DBNN-based models and five real-world datasets clearly illustrate the advantage of proposed method. \nHowever, one minor limitation is that the paper does not provide the residual block analysis, or explicitly show how sensitive of the residual block is to environmental noise.", "relation_to_broader_scientific_literature": "The paper appropriately relates its contributions to the broader literature on DBNN-based method under environmental noise perturbations in resource-constrained scenarios. It extends the current state-of-the-art by clearly showing the theoretical benefit of noise robustness, also alleviates the problem of introducing additional expensive costs with the SOTA approach.", "essential_references_not_discussed": "The paper adequately discusses relevant literature.", "other_strengths_and_weaknesses": "Strengths\n1. The paper identifies a critical gap in existing research: the lack of robustness analysis for binary neural networks (BNNs) under unpredictable environmental noise (e.g., patient movement artifacts). The proposed $L_{1,\\infty}$-norm constraint offers a theoretically grounded solution, explicitly linking DBNN vulnerability to scaling factors and binary weights. The closed-form robustness bound (Theorem 4.2) is a key contribution, as it provides a quantifiable metric for evaluating DBNN robustness, surpassing the heuristic approaches of prior work (e.g., Shang et al., 2022).\n2. The experiments span diverse domains (i.e., image classification, medical diagnostics) and architectures (ResNet 18/34, multiple BNN variants like BiRealNet, IR-Net, CycleBNN). Results show consistent robustness improvements (e.g., +5.4% on Brain Tumor MRI) while maintaining low computational overhead (16% faster training than LCR). The inclusion of real-world noise models (e.g., SNR=50% for bio-electrical signals) strengthens practical relevance.\n3. The framework’s compatibility with various BNN backbones (e.g., ResNet34) and automated training pipeline (Fig.2) enhances applicability. The ablation study on constraint coefficients (also in the Appendix) and visualization of feature map perturbations (Fig.4) provide actionable insights for practitioners.\nWeaknesses\n1. This manuscript designs a large number of experiments to verify the validity of proposed method on different types of datasets. The reviewer wants to know how robust of proposed algorithm can be against adversarial attacks with latest BNNs (e.g., CycleBNN, Fontana et al., 2024).\n2. The theoretical analysis assumes Lipschitz continuity of binarized activation but does not provide the residual block analysis. It is suggested that the author reflect this part of the analysis after the main theorem.", "other_comments_or_suggestions": "1. Increase font size in Fig.4 for clearer visualization of feature maps.\n2. Discuss potential risks of deploying noise-robust DBNNs in safety-critical medical applications (e.g., false negatives under extreme noise).", "questions_for_authors": "1.The reviewer wants to know how robust of proposed algorithm can be against adversarial attacks with latest BNNs (e.g., CycleBNN, Fontana et al., 2024).\n2. The theoretical analysis assumes Lipschitz continuity of binarized activation but does not provide the residual block analysis.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n8pTJNnVYK", "reviewer": "Reviewer_JCk1", "summary": "This paper investigates the robustness of DBNNs under environmental noise in resource-constrained scenarios. The authors identify that the vulnerability of DBNNs stems from binary weights and scaling factors and propose an L1,∞-norm constraint to improve robustness. The proposed method introduces an auxiliary robustness loss function to balance classification and robustness objectives. Experiments on CIFAR-10, CIFAR-100, Brain Tumor MRI, and bio-electrical signal datasets show that the proposed method improves model accuracy and reduces accuracy degradation under noise compared to SOTA methods.\n\n## Update after Rebuttal\nThe authors' rebuttal solved my concerns so I raised my score to 4.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: How does the proposed method generalize to non-vision tasks such as NLP?\n\nA1: Thank you for your insightful question. From a theoretical perspective, extending to NLP tasks such as text classification or machine reading, it is necessary to first construct BERT models that binarize weights and activation, sort out their iterative and expansion forms, and the most critical step is to analyze the formalized upper bound of environmental noise perturbations for each layer. Because the structure of BERT is different from the DBNNs with ResNet backbone, it means that the Lipschitz constant of each part needs to be calculated carefully. This suggests that the upper bound of the binarized BERT model exhibits a slight deviation from Theorem 3. With a moderate adjustment, the theoretical analysis can be effectively applied to NLP tasks. On the other hand, the bio-electrical signal classification task and NLP task belong to the sequence modeling task. In particular, the data for the bio-electrical signal classification task are closely associated with time-dimensional information, and the proposed constraints have effectively validated the performance of the BNN model (e.g., A binary convolution layer as a replacement for the full-precision convolution layer). \n\nQ2: What is the real-time inference impact of the L1,∞-norm constraints, particularly in resource-limited environments?\n\nA2: Thank you for your nice question. In fact, the proposed constraints are incorporated into the training stage in the form of an objective function penalty term. This mean that the proposed method solely impacts the training cost, and thus no additional operations are introduced to the layers within the DBNNs. Consequently, it does not introduce extra time overhead during the model inference phase, thereby eliminating any inference concerns. Finally, DBNNs on the server side can achieve inference times at the second level. \n\nQ3: Can you provide an ablation study to isolate the contribution of binary weights versus scaling factors in improving robustness?\n\nA3: Thank you for your constructive suggestion. We found that the constraints on binary weights and scaling factors are indispensable to improve the robustness of DBNNs. \n\nTable 1. Ablation study 1: Binary weights V.S. Scaling factors to improving robustness on the CIRFA-100 dataset. The w/o indicates that the constraint on the scaling factors has been removed.\n| **Backbone** | **Dorefa+Our** | **Dorefa+Our+w/o** | **React+Our**  | **React+Our+w/o** | **ABC+Our**|**ABC+Our+w/o**| **IRNet+Our** | **IRNet+Our+w/o** | **Bireal+Our** |**Bireal+Our+w/o**|**CycleBNN+Our**|**CycleBNN+Our+w/o**|\n|:------------:|:--------------:|:------------------:|:--------------:|:-----------------:|:--------------:|:---------------:|:--------------:|:-----------------:|:--------------:|:------------------:|:----------------:|:--------------------:|\n| ResNet18| **67.21**| 66.90|**68.26**| 67.26|**69.23**|68.57|**70.04**|68.74|**68.84**|67.79|**67.29**|66.51|\n\nTable 2. Ablation study 2: Binary weights V.S. Scaling factors to improving robustness of DBNNs on the Brain tumor dataset.\n| **Backbone** | **Dorefa+Our** | **Dorefa+Our+w/o** | **React+Our**  | **React+Our+w/o** | **CycleBNN+Our** | **CycleBNN+Our+w/o** |\n|:------------:|:--------------:|:------------------:|:--------------:|:-----------------:|:----------------:|:--------------------:|\n| ResNet18|**86.91**|85.62| **83.87**| 82.39|**85.46**|84.17|\n\nQ4: How does the proposed method compare to more recent adaptive fault-tolerance methods beyond traditional redundancy-based techniques?\n\nA4: Thank you very much! The description of the manuscript may have given you some misunderstanding. We would like to clarify the distinction between the environmental noise robustness of DBNNs and adaptive error tolerance methods. Based on two recent works [1-2], adaptive fault tolerance methods focus on ensuring that hardware devices can continue performing tasks even after a fault occurs (i.e., The communication node is faulty). In contrast, the DBNN model exhibits approximately a 10-20% decrease in performance under environmental noise perturbations, rather than encountering fault. Therefore, we believe that it is not suitable to compare the adaptive fault-tolerance methods for hardware devices with the robustness improvement algorithm for DL models (DBNNs). Another reason is that the tasks targeted by these fault-tolerance algorithms differ significantly from the classification tasks of DBNNs, and their portability remains to be validated. We plan to investigate the effectiveness of adaptive fault-tolerance methods when the DBNN encounters fault. \n\n[1] Ada-FA: A Comprehensive Framework for Adaptive Fault Tolerance and Aging Mitigation in FPGAs. IEEE Internet Things J. 11(10): 17688-17699 (2024)\n\n[2] A Dynamic Adaptive Framework for Practical Byzantine Fault Tolerance Consensus Protocol in the Internet of Things. IEEE Trans. Computers 73(7): 1669-1682 (2024)", "claims_and_evidence": "The paper claims that L1,∞-norm constraints enhance DBNN robustness by improving the stability of binary weights and scaling factors under noise.  However, the evidence is limited to specific tasks and datasets, primarily focused on vision and medical diagnostics. Furthermore, while the paper reports lower computational overhead, it does not analyze real-time inference latency, which is important in resource-constrained scenarios.", "methods_and_evaluation_criteria": "The proposed method is clearly described. The authors provide a structured explanation of the L1,∞-norm constraint and its impact on DBNN robustness. The evaluation is based on widely used datasets and includes a reasonable selection of models (e.g., CNNs and vision transformers).", "theoretical_claims": "The paper provides a theoretical analysis of the robustness bounds derived from the L1,∞-norm constraints. The derivation appears sound, and the theoretical insights align with the empirical findings.", "experimental_designs_or_analyses": "The experimental setup is well-organized and follows standard practices.", "relation_to_broader_scientific_literature": "The paper builds on previous work in fault tolerance, particularly in binary neural networks and hardware-aware ML. It references relevant prior work and positions itself well within the context of DBNN optimization and robustness.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "The paper presents an interesting and potentially impactful method for improving DBNN robustness. However, the lack of generalization beyond vision and medical tasks reduces the scope of the contribution.", "other_comments_or_suggestions": "Figure 3 needs a clearer description of what w11 and w12 represent respectively", "questions_for_authors": "1. How does the proposed method generalize to non-vision tasks such as NLP?\n2. What is the real-time inference impact of the L1,∞-norm constraints, particularly in resource-limited environments?\n3. Can you provide an ablation study to isolate the contribution of binary weights versus scaling factors in improving robustness?\n4. How does the proposed method compare to more recent adaptive fault-tolerance methods beyond traditional redundancy-based techniques?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper investiages the robustnes of deep binary neural networks (DBNNs) for compute resource constrained settings under noise perturbations. The paper proposes a method, inspired by theoretical analysis, that improves robustness.

The paper's four reviewers all recommend acceptance after the rebuttal.

Strength:
* Reviewer PX4V find the theoretical analysis and experiments sound an rigorous and the algorithm to be effective. 
* Reviewer 1Wt1 finds two claims problematic, the claim of broad applicability and the claim of reducing computational overhead. Specifically, the method is only investigated for ResNet type architectures. The authors remark that deep binary neural networks are primarily CNN based networks as investigated in the paper.
* Reviewer FmSJ also finds the theoretical claims and analysis to be well-supported and the advantage of the method to be clearly demonstrated. 

The paper provides a sound and well-verified contribution to the robustness of deep binary neural networks. I therefore recommend the paper for publication.
Decision: Accept (poster)