BiPer: Binary Neural Networks using a Periodic Function

Edwin Vargas 1 Claudia Correa 2 Carlos Hinojosa 3 Henry Arguello 2

Abstract

Accurate and precise models, however, become expensive
in terms of computation, storage and number of parameters.
For this reason, DNN deployment is usually prohibited for
devices with limited resources, such as mobile, hand-heldor wearables. Different approaches to reduce computation
requirements include efficient neural network architecture
design [2]–[4], network pruning [5], knowledge distillation[6], hashing [7], and network quantization [8], [9]. Amongthem, network quantization has become one of the mostpromising techniques, aiming at compressing large models
usually stored as floating-point weights with low bitwidth
numbers. Binary Neural Networks (BNNs) are the extreme
quantization case, where weights and activation functions
are constrained to just one bit, i.e., binary values, typically+1 or -1. In contrast to DNNs, BNNs replace heavy matrix
computations by bit-wise operations, yielding to 32× memory compression, and 58× speed-up on CPUs [10]. Thus,
this approach drastically reduces the computational requirements and accelerates inference, making BNNs particularly
appealing for resource-constrained environments such asedge devices and mobile applications.

Quantized neural networks employ reduced precision representations for both weights and activations. This quantization process significantly
reduces the memory requirements and computational complexity of the network. Binary Neural
Networks (BNNs) are the extreme quantizationcase, representing values with just one bit. Since
the sign function is typically used to map realvalues to binary values, smooth approximations
are introduced to mimic the gradients during error
backpropagation. Thus, the mismatch between
the forward and backward models corrupts the
direction of the gradient causing training inconsistency problems and performance degradation.
In contrast to current BNN approaches, we propose to employ a binary periodic (BiPer) function
for the forward pass to obtain the binary values
and employ the trigonometric sine function withthe same period of the square wave function as adifferentiable surrogate during the backward pass.
We demonstrate that this approach can control
the quantization error by using the frequency ofthe periodic function and improves network performance. Numerical experiments validate theeffectiveness of BiPer in the classification taskover ImageNet, with improvements of 0.69%.

Despite significant advantages for efficient BNN deployment in hardware with limited capabilities, the binarizationof full-precision models severely degrades the accuracy performance in high-level tasks such as object detection, classification, segmentation, and others [9]. For instance, in large
datasets such as ImageNet, one of the earliest BNN models, the XNOR-Net [10], achieved an accuracy degradationof around 18% compared to the full precision ResNet-18
architecture. Recent efforts have been devoted to close theperformance gap of BNN with respect to their real-valued
counterparts. Nonetheless, state-of-the-art approaches still
exhibit accuracy degradations of approximately 8% [11].

1. Introduction
Deep Neural Networks (DNN) have achieved unprecedented
results in many high-level tasks, such as classification, segmentation, and detection, with a tremendous concurrent impact in computer vision, natural language processing, information retrieval, and many others [1]. Typically, DNNs rely
on full-precision (32 bit) weights and activation functions.

Binarization of real-valued weights and activations is generally performed using the sign function during the feedforward procedure. A relevant limitation of the sign functionis that its gradient is null everywhere except in zero, whichmakes it incompatible with error back-propagation methods,
due to the non-differentiability of binary operations. To
overcome this issue, various techniques like the straightthrough estimator (STE) and relaxed training approaches
have been adopted [12]. STE essentially substitutes the signfunction for the identity function to calculate the gradientsduring the backwards process. Since there exists a mismatch

1

Department of Electrical and Computer Engineering, Rice
University, USA 2 Department of Computer Science, Universidad
Industrial de Santander, Colombia 3 Department of Computer Science, KAUST, Saudi Arabia. Correspondence to: Edwin Vargas<edwin.vargas@rice.edu>.
Published at the 2 nd Differentiable Almost Everything Workshop
at the 41 st International Conference on Machine Learning, Vienna, Austria. July 2024. Copyright 2024 by the author(s).

1BiPer: Binary Neural Networks using a Periodic Function

between the forward and backward pass caused by the STE
approximation, research efforts have focused on designingbetter smooth and differentiable functions to estimate thegradient of the sign function [13], [14]. Although theseapproaches have improved the accuracy of BNNs, gradientinstability persists when the quantization error is minimized.

the sign function to model the binary weights. In contrast
to the sign function depicted in Fig. 1(a), which is always
negative for negative values of w, the proposed periodic
function (Fig. 1(b)) can reach positive and negative values
in the whole domain of the latent weights.

Instead of using the Sign function, in this work, we propose to address the aforementioned issues of extreme 1-bit
quantization by using a binary periodic (BiPer) function
or square wave function to promote binary weight values.
Thus, opposite to the sign function which is always negative for negative values or positive for positive values, theproposed periodic function can reach positive and negativevalues in the whole domain of the latent weights. Since
the gradient of the periodic function still faces the problem of being zero almost everywhere, it cannot be directly
integrated within a back-propagation algorithm based on
gradient descent. We solved this problem by employing asinusoidal function with the same fundamental frequencyof the periodic function as a differentiable surrogate during
the backward pass. The continuity and differentiable characteristics of the sine function, make it suitable for stochastic
gradient methods. In contrast to existing BNN methods thatsmoothly and progressively approximate the sign function
to reduce the quantization error (QE), we will show that in
the proposed BiPer approach the QE can be controlled by
the frequency of the periodic function. We further leverage
this property to provide an initialization of the weights that
better balances the trade-off between the estimation error
and performance accuracy. Experimental results demonstrate that BiPer provides the best network performance forthe classification task, with respect to state-of-the-art BNN
approaches on the CIFAR-10 and ImageNet data sets. The
contributions of our work are summarized as follows:

Figure 1. (a) Sign function. (b) Binary periodic function.

It should be pointed out that the gradient of the periodic
function still faces the problem of being zero almost everywhere, therefore, it cannot be directly integrated within aback-propagation algorithm based on gradient descent. To
solve this problem, we first rewrite the square wave functionaswq = Sign (sin(ω0 w)) ,(1)
where ω0 = 2π
T is the angular frequency. We note that
this corresponds to applying the sign function to the firstharmonic of the periodic function. Based on (1), we canapproximate the gradient with respect to the weights as∂L ∂wq∂L ∂ ŵ
∂L=≈∂w∂wq ∂w∂wq ∂w(2)

where ŵ = sin(ω0 w). Note that the last differential termin (2) corresponds to the gradient of a continuous differentiable sinusoidal function, which is also a smooth periodic
function, and proportional to the frequency ω0 .
2.1. Quantization Error Analysis
This section shows that an additional advantage of using theperiodic function is its flexibility to control the quantization
error. In particular, we mathematically demonstrate how
a lower quantization error can be achieved by setting thefundamental period of the wave function. To this end, let
us first assume that the latent weights roughly follow thezero-mean Laplace distribution, i.e., W ∼ La(0, b) [15]–[17]. Since the weights ŵ before quantization are a functionof a random variable, they are also a random variable Ŵ ∈[−1, 1]. Computing the probability density function (pdf)of a random variable Y = g(X ) from the pdf of X (fX (x))
can be easily done employing the method of transformation[18], if the function g is differentiable and strictly increasingor decreasing, i.e., strictly monotonic. Thus, the pdf of Y
can be computed as(
fX (x1 )1= fX (x1 ) · dx
where g (x1 ) = y′dyfY (y) = |g (x1 )|0
if g(x) ̸= y.

• We propose a simple yet powerful and effective modification in the binarization process, by including a binary
periodic function.
• We introduce a continuous, periodic sinusoidal function as a differentiable surrogate of the binary periodic
function during the back-propagation process, suitable
for stochastic gradient methods.
• We mathematically analyze the quantization error of
BiPer and show that it can be controlled by the frequency of the periodic function.

2. BiPerTo overcome the gradient and quantization error challenges
from existing binarization methods and their gradient approximation functions, we propose to use a binary periodic
function or square wave function (see Fig. 1) instead of just
2

BiPer: Binary Neural Networks using a Periodic Function

The more general case in which g is not monotonic requires
splitting the domain into n intervals, so that g is strictly
monotonic and differentiable on each partition. Then, thepdf can be obtained as
fY (y) =nXfX (xk )k=1|g ′ (x

k )|

=

n
X

k=1

fX (xk ) ·

dxk
,
dy

where fW is the density distribution function of the latent
weights. Using the fact that |x| = xSign(x) along with theproperties of the absolute value, we can rewrite Eq. (5) as


Z +∞
1
−w
2
QE =
(|sin(ω0 w)| − γ) dw. (6)
exp
b
b
0

(3)

The solution to this integral is given by

where x1 , · · · , xn are real solutions to g(x) = y. For BiPer,
since the sin function is not monotonic, we can use (3)to compute the pdf of Ŵ using the pdf of W. LettingfW (w) = 1b exp(|w|/b) denote the pdf of W, and setting
g as the sine function, we can divide the sinusoidal function into subsequent intervals of T /2 where it is strictlyincreasing or decreasing, alternately. The summation in (3)converges to the probability density function of the latent
weights before binarization ŵ for an arbitrary frequency ω0given by
2γω0 b eπ/ω0 b + 1
2(ω0 b)2
 + γ2.
QE =−4(ω0 b)2 + 1(ω0 b)2 + 1) eπ/ω0 b − 1(7)
On the other hand, the optimal solution of the scaling factor
γ in (5) can be computed asω0 b eπ/ω0 b + 1
 . (8)γ = E{|sin(ω0 w)|} =(ω0 b)2 + 1) eπ/ω0 b − 1−| arcsin(ŵ)|
bω0arcsin(ŵ)
1
1
1
√
+
.
cosh
2bω0 1 − ŵ2bω0eπ/bω0 − 1(4)fŴ (ŵ) =

1
1
√
exp
2bω0 1 − ŵ2



Replacing γ from (8) into Eq. (7), we can rewrite the QE as
a function of the frequency ω0 and the parameter b. Figure
3 illustrates the QE as a function of the frequency ω0 fordifferent values of b. It can be seen that the maximum QE is
0.102835, which occurs when the product bω0 ≈ 0.954882
and can be reduced by varying the frequency.

Figure 2. Probability density function of ŵ = sin(ω0 w) assumingthat the random variable w follows a Laplace distribution withparameter b and a fixed value of ω0 = 1.
Figure 3. Quantization error as a function of the frequency ω0 fordifferent values of b. The proposed BiPer approach is able tocontrol QE with the frequency of the periodic function.

Figure 2 depicts the distribution of the weights for different
values of the Laplace distribution parameter b and a fixed
frequency ω0 = 1. Note that different from the randomvariable W which can take any real value, the codomainof the random variable Ŵ is [−1, 1]. From Fig. 2 we canobserve that when the value of b increases the pdf of ŵ
behaves as an arcsin distribution with values concentrated
around -1 and 1. This reduces the quantization error incomparison to the Laplacian distribution. Also, a similar
behavior occurs when the frequency value increases for a
fixed b. To further analyze these observations consider the
QE defined as
Z +∞2
fW (w) (sin(ω0 w) − γ sign (sin(ω0 w))) dw,
QE =It is worth noting that in contrast to current approaches that
progressively reduce the QE to zero, BiPer does not meetthis QE value. Nonetheless, further explorations can adapt
state-of-the-art surrogate estimators to smoothly converge
from the sine function to the square wave

3. Experiments
We evaluated BiPer for image classification with a widelyused neural network architecture, i.e., ResNet, trained on
Imagenet. In the following, we first describe the experiments
setup. Then, we compare BiPer with state-of-the-art BNN
approaches in terms of performance and complexity.

−∞

(5)

3

BiPer: Binary Neural Networks using a Periodic Function

3.1. Experiments Setup

Table 1. BiPer performance comparison with state-of-the-art BNN
on ImageNet. W/A: bit length of weights and activations. FP: full
precision model.

Dataset: ImageNet [19] is a challenging data set because of its larger size and more diverse image categories.
Among its multiple versions, we adopted the widely usedILSVRC12 version, divided into 1,000 categories, fromwhich 1.2 million are training images and, 50,000 test images. ImageNet is the most widely used data set to reportresults on binary networks and, it allows us to show for the
first time that binary networks can perform competitively
on a large-scale data set.

Network

ResNet-18

Network Architectures We chose to binarize ResNet18/34. We adopted the double skip connections as in [20]to provide fair comparisons. Following [21], the downsampling layers are not quantized, and the double skip connections [20] were included. Following standard procedures of
the comparison methods, we binarized all layers but the firstand last.
Training Details and Procedures All experiments used
SGD optimization with 0.9 momentum. We followed thedata augmentation strategies in [22], which include random
crops and horizontal flips.

ResNet-34

Two stage training: Recent works have shown that an appropriate initialization is often required to improve network
performance. Two-stage training strategies are generally employed to alleviate feature quantization adverse effects, [23],[24]. Particularly, in the first stage the network is trainedwith real weights and binary features. Then, in the second
stage, a warm weight initialization is employed based on
the binary representation of the output weights from the firststage, and the model is fully trained to binarize the weights.
In BiPer, we propose a two-stage training where the firststage uses real-valued weights ŵ, and the second stage uses
the weight binarization from Eq. (1). By testing different
frequency values, we experimentally found that the hyperparameter frequency ω0 = 20 balances the QE and precisionof the full binary model. The frequency is the same andfixed for both stages. The learning rate was set to 0.1 in thefirst stage and, 0.01 in the second stage. In both stages thelearning rate was adjusted by the cosine scheduler.

MethodFP
XNOR-Net
Bi-Real Net
PCNN
IR-NetBONN
LCR-BNN
HWGQ
RBNN
FDA
ReSTE
ReCU
DIR-Net
BiPer (Ours)
FP
Bi-Real Net
IR-Net
RBNN
ReSTE
ReCU
DIR-Net
BiPer (Ours)

W/A
32/32
1/1
1/1
1/1
1/1
1/1
1/1
1/1
1/1
1/1
1/1
1/1
1/1
1/1
32/32
1/1
1/1
1/1
1/1
1/1
1/1
1/1

Top-1
69.6%
51.2%
56.4%
57.3%
58.1%
59.3%
59.6%
59.6%
59.9%
60.2%
60.88%
61.0%
60.4%
61.4%
73.3%
62.2%
62.9%
63.1%
65.05%
65.1%
64.1%
65.73%

Top-5
89.2%
73.2%
79.5%
80.0%
80.0%
81.6%
81.6%
82.2%
81.9%
82.3%
82.59%
82.6%
81.9%
83.14%
91.3%
83.9%
84.1%
84.4%
85.78%
85.8%
85.3%
86.39%

best result of 61.0% achieved by ReCu. Furthermore, ourtop-5 performance reached 83.14%, surpassing the secondbest result of 82.6%, also achieved by ReCU. Likewise, for
ResNet-34, we achieved the highest top-1 and top-5 accuracies, namely 65.73% and 86.39%, respectively. These
results improve the second-best method (ReCU) by 0.63%
and 0.59% in top-1 and Top-5 accuracies, respectively.

4. Conclusions
An approach for neural network binarization using a binary
periodic function or square wave, dubbed BiPer, has been
proposed. To improve gradient stability we employed asinusoidal function with the same period of the square wave
as a differentiable surrogate during the backward pass. This
simple, yet powerful modification tackles the problem ofstandard gradient mismatch between forward and backward
steps during network training, providing a suitable alternative that can be incorporated within back-propagation
algorithms based on stochastic gradient descent. Mathematical analysis of BiPer quantization error demonstrated
that it can be controlled by the frequency of the periodic
function. Comparisons with respect to state-of-the-art BNN
approaches showed that BiPer outperforms prior works by
up to 0.63% on Imagenet. Although this work tested the
BiPer approach for classification, it can be easily extended
to other high-level tasks without increasing the complexity.

3.2. Comparison with SOTA methods
We evaluate the proposed BiPer approach using ResNet-18
and ResNet-34, and training on the large-scale ImageNet
dataset. Table 1 shows a number of SOTA quantization
methods over ResNet-18 and ResNet-34, including XNORNet [10], Bi-Real Net [20], PCNN [25], IR-Net [26], BONN
[27], LCR-BNN [28], HWGQ [29], RBNN [30], FDA [31],
ReSTE [32], ReCU [11], and DIR-Net [33]. We can observe that the proposed BiPer approach in the 1W/1A setting
achieves the best Top-1 and top-5 accuracy for both networkarchitectures. Specifically, for ResNet-18, we attained a top1 validation accuracy of 61.4%, outperforming the second4

BiPer: Binary Neural Networks using a Periodic Function

References
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

C. Wang, A. Bochkovskiy, and H. Liao, “Yolov7: Trainablebag-of-freebies sets new state-of-the-art for real-time object
detectors,” in 2023 IEEE/CVF Conference on Computer
Vision and Pattern Recognition (CVPR), Los Alamitos, CA,
USA: IEEE Computer Society, Jun. 2023, pp. 7464–7475.
DOI : 10.1109/CVPR52729.2023.00721. [Online].
Available: https://doi.ieeecomputersociety.
org/10.1109/CVPR52729.2023.00721.
A. G. Howard, M. Zhu, B. Chen, et al., “Mobilenets: Efficient convolutional neural networks for mobile visionapplications,” CoRR, vol. abs/1704.04861, 2017. arXiv:
1704.04861. [Online]. Available: http://arxiv.
org/abs/1704.04861.
M. Tan and Q. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” in Proceedings ofthe 36th International Conference on Machine Learning,
K. Chaudhuri and R. Salakhutdinov, Eds., ser. Proceedings of Machine Learning Research, vol. 97, PMLR, Sep.
2019, pp. 6105–6114. [Online]. Available: https : / /proceedings.mlr.press/v97/tan19a.html.
F. N. Iandola, M. W. Moskewicz, K. Ashraf, S. Han,
W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-levelaccuracy with 50x fewer parameters and ¡1mb modelsize,” ArXiv, vol. abs/1602.07360, 2016. [Online]. Available: https : / / api . semanticscholar . org /CorpusID:14136028.
Y. He, J. Lin, Z. Liu, H. Wang, L.-J. Li, and S. Han, “Amc:
Automl for model compression and acceleration on mobile
devices,” in Proceedings of the European Conference on
Computer Vision (ECCV), Sep. 2018.
J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge
distillation: A survey,” International Journal of Computer
Vision, vol. 129, pp. 1789–1819, 2021. DOI: https://doi.org/10.1007/s11263-021-01453-z.
E. Eban, Y. Movshovitz-Attias, H. Wu, et al., “Structured
multi-hashing for model compression,” in 2020 IEEE/CVF
Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 11 900–11 909. DOI: 10.1109/
CVPR42600.2020.01192.
C. Yuan and S. S. Agaian, “A comprehensive review of binary neural network,” Artificial Intelligence Review, vol. 56,
pp. 12 949–13 013, 2023. DOI: https://doi.org/10.
1007/s10462-023-10464-w.
R. Sayed, H. Azmi, H. Shawkey, A. H. Khalil, and M.
Refky, “A systematic literature review on binary neuralnetworks,” IEEE Access, vol. 11, pp. 27 546–27 578, 2023.
DOI : 10.1109/ACCESS.2023.3258360.
M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi,“Xnor-net: Imagenet classification using binary convolutional neural networks,” in Computer Vision – ECCV 2016,
B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds., Springer
International Publishing, 2016, pp. 525–542, ISBN: 978-3319-46493-0.
Z. Xu, M. Lin, J. Liu, et al., “Recu: Reviving the deadweights in binary neural networks,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021,
pp. 5178–5188. DOI: 10 . 1109 / ICCV48922 . 2021 .
00515.
I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and
Y. Bengio, “Binarized neural networks,” in NeurIPS, 2016.

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

5

R. Gong, X. Liu, S. Jiang, et al., “Differentiable soft
quantization: Bridging full-precision and low-bit neuralnetworks,” in 2019 IEEE/CVF International Conference
on Computer Vision (ICCV), Los Alamitos, CA, USA:IEEE Computer Society, Nov. 2019, pp. 4851–4860. DOI:
10.1109/ICCV.2019.00495. [Online]. Available:
https://doi.ieeecomputersociety.org/10.
1109/ICCV.2019.00495.
H. Le, R. K. Høier, C.-T. Lin, and C. Zach, “Adaste: An
adaptive straight-through estimator to train binary neuralnetworks,” in 2022 IEEE/CVF Conference on Computer
Vision and Pattern Recognition (CVPR), Los Alamitos, CA,
USA: IEEE Computer Society, Jun. 2022, pp. 460–469.
DOI : 10.1109/CVPR52688.2022.00055. [Online].
Available: https://doi.ieeecomputersociety.
org/10.1109/CVPR52688.2022.00055.
C. Baskin, N. Liss, E. Schwartz, et al., “Uniq: Uniform
noise injection for non-uniform quantization of neural networks,” ACM Transactions on Computer Systems (TOCS),
vol. 37, no. 1-4, pp. 1–15, 2021.
Z. Zhang, W. Shao, J. Gu, X. Wang, and P. Luo, “Differentiable dynamic quantization with mixed precision andadaptive resolution,” in International Conference on Machine Learning, PMLR, 2021, pp. 12 546–12 556.
R. Banner, Y. Nahshan, and D. Soudry, “Post training
4-bit quantization of convolutional networks for rapiddeployment,” Advances in Neural Information Processing
Systems, vol. 32, 2019.
H. Pishro-Nik, Introduction to probability, statistics, andrandom processes. Kappa Research, LLC Blue Bell, PA,
USA, 2014.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume
1, ser. NIPS’12, Lake Tahoe, Nevada: Curran Associates
Inc., 2012, pp. 1097–1105.
Z. Liu, W. Luo, B. Wu, X. Yang, W. Liu, and K.-T. Cheng,“Bi-real net: Binarizing deep network towards real-network
performance,” International Journal of Computer Vision,
vol. 128, pp. 202–219, 2020.
J. Bethge, H. Yang, and C. Meinel, “Training accurate
binary neural networks from scratch,” in 2019 IEEE International Conference on Image Processing (ICIP), IEEE,
2019, pp. 899–903.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE
conference on computer vision and pattern recognition,
2016, pp. 770–778.
B. Martinez, J. Yang, A. Bulat, and G. Tzimiropoulos,“Training binary neural networks with real-to-binary convolutions,” in International Conference on Learning Representations, 2019.
A. Bulat, G. Tzimiropoulos, J. Kossaifi, and M. Pantic, “Improved training of binary networks for humanpose estimation and image recognition,” arXiv preprint
arXiv:1904.05868, 2019.
J. Gu, C. Li, B. Zhang, et al., “Projection convolutional
neural networks for 1-bit cnns via discrete back propagation,” in Proceedings of the AAAI conference on artificial
intelligence, vol. 33, 2019, pp. 8344–8351.

BiPer: Binary Neural Networks using a Periodic Function
[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

H. Qin, R. Gong, X. Liu, et al., “Forward and backward
information retention for accurate binary neural networks,”
in Proceedings of the IEEE/CVF Conference on Computer
Vision and Pattern Recognition (CVPR), Jun. 2020.
J. Gu, J. Zhao, X. Jiang, et al., “Bayesian optimized 1bit cnns,” in Proceedings of the IEEE/CVF international
conference on computer vision, 2019, pp. 4909–4917.
Y. Shang, D. Xu, B. Duan, Z. Zong, L. Nie, and Y. Yan,
“Lipschitz continuity retained binary neural network,” inEuropean conference on computer vision, Springer, 2022,
pp. 603–619.
Z. Cai, X. He, J. Sun, and N. Vasconcelos, “Deep learningwith low precision by half-wave gaussian quantization,” inProceedings of the IEEE conference on computer visionand pattern recognition, 2017, pp. 5918–5926.
M. Lin, R. Ji, Z. Xu, et al., “Rotated binary neural network,”
Advances in neural information processing systems, vol. 33,
pp. 7474–7485, 2020.
Y. Xu, K. Han, C. Xu, Y. Tang, C. Xu, and Y. Wang, “Learning frequency domain approximation for binary neuralnetworks,” Advances in Neural Information Processing
Systems, vol. 34, pp. 25 553–25 565, 2021.
X.-M. Wu, D. Zheng, Z. Liu, and W.-S. Zheng, “Estimatormeets equilibrium perspective: A rectified straight throughestimator for binary neural networks training,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2023, pp. 17 055–17 064.
H. Qin, X. Zhang, R. Gong, Y. Ding, Y. Xu, and X. Liu,“Distribution-sensitive information retention for accurate
binary neural network,” International Journal of Computer
Vision, vol. 131, no. 1, pp. 26–47, 2023.

6