BoolNet: Streamlining Binary Neural Networks Using Binary Feature Maps

1 INTRODUCTION . The recent success of Deep Neural Networks ( DNNs ) is like the jewel in the crown of modern AI waves . However , the large size and the high number of operations cause the current DNNs to heavily rely on high-performance computing hardware , such as GPU and TPU . Training sophisticated DNN models also results in excessive energy consumption and CO2 emission , e.g. , training the OpenAI ’ s GPT-3 by Brown et al . ( 2020 ) causes as much CO2 emissions as 43 cars during their lifetime ( Patterson et al. , 2021 ) . Moreover , their computational expensiveness strongly limits their applicability on resource-constrained devices such as mobile phones , IoT devices , and embedded devices . Various works aim to solve this challenge by reducing memory footprints and accelerating inference . We can roughly categorize these works into the following directions : network pruning ( Han et al. , 2015a ; b ) , knowledge distillation ( Crowley et al. , 2018 ; Polino et al. , 2018 ) , compact networks ( Howard et al. , 2017 ; 2019 ; Sandler et al. , 2018 ; Ma et al. , 2018b ; Tan et al. , 2019 ) , and low-bit quantization ( Courbariaux et al. , 2015 ; Rastegari et al. , 2016 ; Zhou et al. , 2016 ; Hubara et al. , 2016 ) . From the latter , there is an extreme case , Binary Neural Networks ( BNNs ) , first introduced by Courbariaux et al . ( 2016 ) , that uses only 1 bit for weight and activation . As shown in the literature ( Rastegari et al. , 2016 ) , BNNs can achieve 32× memory compression and up to 58× speedup on CPU , since the conventional arithmetic operations can be replaced by bit-wise xnor and bitcount operations . However , BNNs suffer from accuracy degradation compared to their 32-bit counterparts . For instance , XNOR-Net leaves an 18 % accuracy gap to ResNet-18 on ImageNet classification ( Rastegari et al. , 2016 ) . Therefore , recent efforts ( analyzed in more detail in Section 2 ) mainly focus on narrowing the accuracy gap , including specific architecture design ( Liu et al. , 2018 ; Bethge et al. , 2019 ; 2020 ; Liu et al. , 2020b ) , real-valued weight and activation approximation ( Lin et al. , 2017a ; Zhuang et al. , 2019 ) , specific training recipes ( Martinez et al. , 2020 ) , a dedicated optimizer ( Helwegen et al. , 2019 ) , leveraging neural architecture search ( Bulat et al. , 2020 ; Zhao et al. , 2020 ) and dynamic networks ( Bulat et al. , 2021 ) . In the existing work , efficiency analysis usually only considers the theoretical instruction counts . However , memory usage , inference efficiency and energy consumption , which are essential to practical applications , have received little attention . Furthermore , Fromm et al . ( 2020 ) ; Bannink et al . ( 2021 ) point out that the theoretical complexity is often inconsistent with the actual performance in practice and measurable performance gains on existing BNN models are hard to achieve as the 32-bit components in BNNs ( such as BatchNorm , scaling , and 32-bit branches ) become bottlenecks . Using 32-bit information flow ( e.g. , 32-bit identity connections , 32-bit downsampling layers are equipped by almost all latest BNNs , see Figure 1a ) , and multiplication/division operations ( in BatchNorm , scaling , average pooling etc . ) significantly increase the memory usage and power consumption of BNNs and are thus unfriendly to hardware accelerators . For these reasons , even if BNNs have achieved MobileNet-level accuracy with a similar theoretical number of OPs ( Bethge et al. , 2020 ; Martinez et al. , 2020 ) , they still can not be used as conveniently as compact networks ( Howard et al. , 2017 ; 2019 ; Sandler et al. , 2018 ) . In this paper , we extensively study the trade-off between BNN ’ s accuracy and hardware efficiency . We propose a novel BNN architecture : BoolNet , which replaces most commonly used 32-bit components ( see Section 3 ) . First , BoolNet only uses binary feature maps in the network ( see Figure 1b ) . Second , during inference , we fuse the BN layer into the Sign function through a lossless transformation , thereby effectively removing the Mult-Adds brought by BN . Other changes include removing components that require additional 32-bit multiplication/division operations : ( 1 ) PReLU , ( 2 ) average pooling , and ( 3 ) binary downsampling convolutions . We then propose a Multi-slice strategy to help alleviate the loss of representational capacity incurred by binarizing the feature maps and removing 32-bit components . We show the effectiveness of our proposed methods and the increased energy efficiency of BoolNet with experiments on the ImageNet dataset ( Deng et al. , 2009 ) . The results show the key benefit of BoolNet : a reasonable accuracy coupled with a higher energy efficiency over state-of-the-art BNNs ( see Figure 1c for a brief summary and Section 4 for more details ) . The energy data is obtained through a hardware accelerator simulation ( see Section 4.4 for details ) . We summarize our main contributions as follows : • The first work studying the effects of 32-bit layers often used in previous works on BNNs . • A novel BNN architecture BoolNet with minimal 32-bit components for higher efficiency . • A Multi-slice strategy to alleviate the accuracy loss incurred by using 1-bit feature maps . • State-of-the-art performance on the trade-off between accuracy and energy consumption with a 2.9× lower power consumption than Bi-RealNet ( Liu et al. , 2018 ) and 6.6 % higher accuracy . 2 RELATED WORK . In recent years , Efficient Deep Learning has become a research field that has attracted much attention . Technical directions , such as , compact network design ( Howard et al. , 2017 ; 2019 ; Sandler et al. , 2018 ; Zhang et al. , 2018 ; Ma et al. , 2018b ) , knowledge distillation ( Crowley et al. , 2018 ; Polino et al. , 2018 ) , network pruning ( Han et al. , 2015a ; b ; Li et al. , 2017 ; He et al. , 2017 ) , and low-bit quantization ( Courbariaux et al. , 2015 ; Rastegari et al. , 2016 ; Liu et al. , 2018 ; 2020b ; Bethge et al. , 2020 ) are proposed for model compression and acceleration . The efficient models have evolved from the earliest handcrafted designs to the current use of neural architecture search to search for the best basic block and overall network structure ( Tan et al. , 2019 ; Howard et al. , 2019 ; Tan & Le , 2019 ; Radosavovic et al. , 2020 ) . The criterion of efficiency evaluation has also changed from instruction and parameter counts to more precise measurements of actual memory and operating efficiency on the target hardware ( Cai et al. , 2019 ; 2018 ) . Binary Neural Networks were first introduced by Courbariaux et al . ( 2016 ) and their initial attempt only evaluated on small datasets such as MNIST ( LeCun & Cortes , 2010 ) , CIFAR10 ( Krizhevsky et al. , 2009 ) and SVHN ( Netzer et al. , 2011 ) . The follow-up XNOR-Net ( Rastegari et al. , 2016 ) proposes channel-wise scaling factors for approximating the real-valued parameters , which achieves 51.2 % top-1 accuracy on ImageNet . However , there is an 18 % gap compared with its 32-bit counterpart , ResNet-18 . Therefore , recent efforts mainly focused on narrowing the accuracy gap . WRPN by Mishra et al . ( 2018 ) shows that expanding the channel width of binary convolutions can obtain a better performance . ABC-Net by Lin et al . ( 2017a ) , GroupNet by Zhuang et al . ( 2019 ) , and ( Zhu et al. , 2019 ) use a set of k binary convolutions ( referred to as binary bases ) , instead of using a single binary convolution , to approximate a 32-bit convolution . This sort of method achieves higher accuracy but increases the required memory and number of operations of each convolution by the factor k. Bi-RealNet by Liu et al . ( 2018 ) proposes using real-valued ( 32-bit ) shortcuts to maintain a 32-bit information flow , which effectively improves the accuracy . This design strategy became a standard for later work , e.g. , Bethge et al . ( 2019 ; 2020 ) ; Liu et al . ( 2020b ) . Martinez et al . ( 2020 ) propose using a real-valued attention mechanism and well-tuned training recipes to boost the accuracy further . Thanks to the special architecture design , the recent MeliusNet ( Bethge et al. , 2020 ) and ReActNet ( Liu et al. , 2020b ) achieve MobileNet-level accuracy with similar number of theoretical operations . Other attempts , such as leveraging neural architecture search ( Bulat et al. , 2020 ; Zhao et al. , 2020 ) and dynamic networks ( Bulat et al. , 2021 ) , show that those successful methods on regular real-valued networks are also effective for BNN . Another method by Shen et al . ( 2019 ) combines neural architecture search to dynamically increase the number of channels for more accurate BNNs . Often , with improved accuracy , 32-bit components are used more frequently as well , such as PReLU and BatchNorm after each binary convolution ( Liu et al. , 2020b ) , a real-valued attention module ( Martinez et al. , 2020 ) and scaling factors , etc . Apart from some works that include and optimize real-time measurements on mobile devices , such as Bannink et al . ( 2021 ) ; Umuroglu et al . ( 2017 ) , efficiency analysis in the literature often only considers the theoretical operation number . However , the memory usage and the actual energy consumption has received very little attention so far . 3 BOOLNET . In this section , we first revisit the latest BNNs and recap how they enhanced the accuracy by adding more 32-bit components ( in Section 3.1 ) . Afterwards , we propose to replace most commonly used 32-bit components from current BNN designs and instead use a fully binary information flow in the network ( in Section 3.2 ) . However , abandoning 32-bit information flow results in a serious degradation of the representative capacity of the network . Thus , we also present our strategies to restore the representative capacity ( in Section 3.3 ) . The focus on boolean operations and binary feature maps leads to the name of our network : BoolNet . 3.1 IMPROVING ACCURACY WITH ADDITIONAL 32-BIT COMPONENTS . Recent works on BNNs have made promising progress in narrowing the gap to their 32-bit counterparts . The key intention is to enhance the representative capacity by fully exploiting additional 32-bit components . However , such additional 32-bit components significantly reduce the hardware efficiency ( as shown by Fromm et al . ( 2020 ) and further discussed in Section 4.4 ) . The following list summarizes the 32-bit components commonly used in the latest BNNs : ( 1 ) The channel-wise scaling factor was first proposed by Rastegari et al . ( 2016 ) for approximating the 32-bit parameters . It increases the value range of activation and weight . ( 2 ) Bi-RealNet ( Liu et al. , 2018 ) proposes to use a 32-bit shortcut for enclosing each binary convolution . The key advantage is that the network can maintain an almost completely 32-bit information flow ( cf . Figure 2a ) . ( 3 ) XNOR-Net ( Rastegari et al. , 2016 ) uses 32-bit 1×1 downsampling convolutions , which is also used by many subsequent methods ( Liu et al. , 2018 ; Martinez et al. , 2020 ; Bethge et al. , 2020 ) . Bethge et al . ( 2019 ) shows that this simple strategy can achieve about 3.6 % Top-1 accuracy gains on ImageNet based on a binary ResNet-18 model . ( 4 ) Martinez et al . ( 2020 ) ; Bulat et al . ( 2020 ; 2021 ) show that PReLU activation effectively improves accuracy of BNNs . ReActNet ( Liu et al. , 2020b ) constructs the RPReLU activation function and uses it before every sign function . ( 5 ) Martinez et al . ( 2020 ) reuse the 32-bit activation in their Real-to-Binary Net after BN with a squeeze and excitation ( SE ) attention mechanism . This module can adaptively re-scale the outputs of each binary convolution but needs additional 32-bit operations . Although these techniques can effectively improve the accuracy , they increase the number of 32-bit values and floating point operations , making them not particularly efficient on hardware accelerators . They are closer to mixed-precision neural networks rather than being highly efficient binary neural networks , as one might expect .

Summary: This paper proposes methods to further reduce energy consumption of binary neural networks by removing or replacing 32-bit components (e.g., skip connections) in SOTA BNNs. More specifically, the proposed architecture (1) reduces the precision of skip connections and activation functions, (2) transforms the BatchNorm layer into a simple sign function, and (3) employs a multi-slice strategy to alleviate the loss of representational capacity incurred by binarizing the feature maps and shortcut connections. The results shows that the new model achieves 4.7x energy reduction with some accuracy degradation compared to SOTA architectures.