Original: Bi-MLP: Compact Binary Architectures for Vision Multi-Layer Perceptrons
Revised: BiMLP: Compact Binary Architectures for Vision Multi-Layer Perceptrons

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: This paper studies the problem of designing compact binary architectures for visionmulti-layer perceptrons (MLPs).
Revised: This paper studies the problem of designing compact binary architectures for vision multi-layer perceptrons (MLPs).

Original: We provide extensive analysis on the difﬁculty ofbinarizing vision MLPs and ﬁnd that previous binarization methods perform poorlydue to limited capacity of binary MLPs.
Revised: We provide extensive analysis on the difficulty of binarizing vision MLPs and find that previous binarization methods perform poorly due to limited capacity of binary MLPs.

Original: In contrast with the traditional CNNsthat utilizing convolutional operations with large kernel size, fully-connected (FC)layers in MLPs can be treated as convolutional layers with kernel size 1 × 1 .
Revised: In contrast with the traditional CNNs that utilizing convolutional operations with large kernel size, fully-connected (FC) layers in MLPs can be treated as convolutional layers with kernel size 1 × 1 .

Original: Thus,the representation ability of the FC layers will be limited when being binarized,and places restrictions on the capability of spatial mixing and channel mixing onthe intermediate features.
Revised: Thus, the representation ability of the FC layers will be limited when being binarized, and places restrictions on the capability of spatial mixing and channel mixing on the intermediate features.

Original: To this end, we propose to improve the performance ofbinary MLP (Bi-MLP) model by enriching the representation ability of binary FClayers.
Revised: To this end, we propose to improve the performance of binary MLP (BiMLP) model by enriching the representation ability of binary FC layers.

Original: We design a novel binary block that contains multiple branches to mergea series of outputs from the same stage, and also a universal shortcut connectionthat encourages the information ﬂow from the previous stage.
Revised: We design a novel binary block that contains multiple branches to merge a series of outputs from the same stage, and also a universal shortcut connection that encourages the information flow from the previous stage.

Original: The downsamplinglayers are also carefully designed to reduce the computational complexity whilemaintaining the classiﬁcation performance.
Revised: The downsampling layers are also carefully designed to reduce the computational complexity while maintaining the classification performance.

Original: Experimental results on benchmarkdataset ImageNet-1k demonstrate the effectiveness of the proposed Bi-MLP models,which achieve state-of-the-art accuracy compared to prior binary CNNs.
Revised: Experimental results on benchmark dataset ImageNet-1k demonstrate the effectiveness of the proposed BiMLP models, which achieve state-of-the-art accuracy compared to prior binary CNNs.

Original:  Introduction
Revised:  Introduction

Original: Recent years have witness the boosting of convolutional neural networks (CNNs) on several computervision (CV) applications, e g. image recognition [12, 27, 9], object detection [25, 24], semanticsegmentation [22] and low-level vision [13].
Revised: Recent years have witness the boosting of convolutional neural networks (CNNs) on several computer vision (CV) applications, e g. image recognition [21, 40, 17, 10, 14, 13], object detection [37, 36, 11, 9], semantic segmentation [34] and low-level vision [23, 32].

Original: However, the success of CNN models highly depends ontheir huge computational cost and massive parameters, which are not suitable to be directly appliedto edge devices that have limited computational resources such as mobile phones, smart watches andso on.
Revised: However, the success of CNN models highly depends on their huge computational cost and massive parameters, which are not suitable to be directly applied to edge devices that have limited computational resources such as mobile phones, smart watches and so on.

Original: There are several model compression and acceleration methods to reduce the number of parametersand FLOPs of the original CNNs and derive a portable model.
Revised: There are several model compression and acceleration methods to reduce the number of parameters and FLOPs of the original CNNs and derive a portable model.

Original: For instance, knowledge distillation [11] aims to train a small student network with the help of a cumbersome teacher network.
Revised: For instance, knowledge distillation [19, 49, 50] aims to train a small student network with the help of a cumbersome teacher network.

Original: Filter pruning [20, 10] methods sort the weights of the network based on their importance and throwaway those who have negligible inﬂuence on the ﬁnal performance.
Revised: Filter pruning [30, 18, 44, 43] methods sort the weights of the network based on their importance and throw away those who have negligible influence on the final performance.

Original: Model quantization [4, 37]methods reduce the original 32-bit ﬂoating point weights and activations into lower bits and tensordecomposition [1, 32] methods express a large weight tensor as a sequence of elementary operationson a series of simpler tensors.
Revised: Model quantization [4, 56, 33] methods reduce the original 32-bit floating point weights and activations into lower bits and tensor decomposition [1, 47] methods express a large weight tensor as a sequence of elementary operations on a series of simpler tensors.

Original: Among them, binary CNN [36, 23] is an extreme case of modelquantization that uses only 1-bit for weights and activations.
Revised: Among them, binary CNN [55, 35, 15] is an extreme case of model quantization that uses only 1-bit for weights and activations.

Original: Compared to the original convolutionaloperation, binarized one has 64 × less FLOPs and 32 × less memory consumption.
Revised: Compared to the original convolutional operation, binarized one has 64 × less FLOPs and 32 × less memory consumption.

Original: Note that all of the existing binarization methods are applied on the convolutional operations in CNNmodels.
Revised: Note that all of the existing binarization methods are applied on the convolutional operations in CNN models.

Original: However, multi-layer perceptron (MLP) model also shows its potential on CV tasks [30, 3] and matrix multiplication is computational friendly to GPUs and has advantage on inference timein reality.
Revised: However, multi-layer perceptron (MLP) model also shows its potential on CV tasks [45, 3, 12] and matrix multiplication is computational friendly to GPUs and has advantage on inference time in reality.

Original: Compared to traditional CNNs that utilize convolutional operation with various kernelsizes, FC layers in MLP models can be treated as convolutions with kernel size 1 × 1 , which suffermore from limited representation ability compared to the convolutional operations with larger kernelsize when being binarized.
Revised: Compared to traditional CNNs that utilize convolutional operation with various kernel sizes, FC layers in MLP models can be treated as convolutions with kernel size 1 × 1 , which suffer more from limited representation ability compared to the convolutional operations with larger kernel size when being binarized.

Original: In fact, directly binarizing MLP models with the existing methodsshow more severe performance degradation compared to the CNN models.
Revised: In fact, directly binarizing MLP models with the existing methods show more severe performance degradation compared to the CNN models.

Original: For example, the Top-1classiﬁcation accuracy on ImageNet will drop by more than 23% when binarizing CycleMLP
Revised: For example, the Topclassification accuracy on ImageNet will drop by more than 23% when binarizing CycleMLP

Original: [3]and WaveMLP [29] with the method proposed in Dorefa-Net [36] while the performance drop isonly 17% for ResNet-18 [9] using 3 × 3 kernel size for most of the convolutional layers and 13% for
Revised: [3] and WaveMLP [42] with the method proposed in Dorefa-Net [55] while the performance drop is only 17% for ResNet-18 [17] using 3 × 3 kernel size for most of the convolutional layers and 13% for

Original: AlexNet [12] with larger kernel size ( 11 × 11 and 5 × 5 ).
Revised: AlexNet [21] with larger kernel size ( 11 × 11 and 5 × 5 ).

Original: [Figure]
Revised: [Figure]

Original: Thus, in order to alleviate the problem mentioned above, we introduce a novel binaryblock for MLP model that contains multiple branches to merge a series of outputs from the same stage of the network.
Revised: Thus, in order to alleviate the problem mentioned above, we introduce a novel binary block for MLP model that contains multiple branches to merge a series of outputs from the same stage of the network.

Original: Besides, a universal shortcut connection(Uni-shortcut) is designed to better transfer the knowledge from previous layers tothe current layer.
Revised: Besides, a universal shortcut connection (Uni-shortcut) is designed to better transfer the knowledge from previous layers to the current layer.

Original: Compared to the original shortcut connection, the proposed Unishortcut can merge two features with different shapes.
Revised: Compared to the original shortcut connection, the proposed Unishortcut can merge two features with different shapes.

Original: By fusing the outputs ofmultiple layers derived from the same ordifferent stages of the network, we showthat the representation power of the binary
Revised: By fusing the outputs of multiple layers derived from the same or different stages of the network, we show that the representation power of the binary

Original: MLP is drastically increased.
Revised: MLP is drastically increased.

Original: The downsampling layers are also modiﬁed to reducethe computational complexity while maintaining the classiﬁcation performance.
Revised: The downsampling layers are also modified to reduce the computational complexity while maintaining the classification performance.

Original: Experimental results on ImageNet-1k dataset demonstrate that by using the proposed method, weachieve state-of-the-art performance compared to other binary CNN models, as shown in Fig.
Revised: Experimental results on ImageNet-1k dataset demonstrate that by using the proposed method, we achieve state-of-the-art performance compared to other binary CNN models, as shown in Fig.

Original: We summarize our contributions for learning 1-bit vision MLPs as follows:
Revised: We summarize our contributions for learning 1-bit vision MLPs as follows:

Original: • This paper points out the main difﬁculty of binarizing MLP models is that the representation ability of FC layers is worse than the convolutional layers in CNN models with kernel sizelarger than 1 × 1 .•
Revised: • This paper points out the main difficulty of binarizing MLP models is that the representation ability of FC layers is worse than the convolutional layers in CNN models with kernel size larger than 1 × 1 .

Original:  We introduce a mulit-branch binary MLP block (MBB block) with Uni-shortcut operation to unlock the representation power of binary MLP models and also modify the architectureof downsampling layer to reduce the computational complexity.• The experimental results on ImageNet-1k dataset show that the proposed Bi-MLP improvesthe top-1 accuracy by 1.3% with 12.1% less OPs compared to the state-of-the-art ReActNet,which indicates the effectiveness of the proposed Bi-MLP models.
Revised: • We introduce a mulit-branch binary MLP block (MBB block) with Uni-shortcut operation to unlock the representation power of binary MLP models and also modify the architecture of downsampling layer to reduce the computational complexity. • The experimental results on ImageNet-1k dataset show that the proposed BiMLP improves the top-1 accuracy by 1.3% with 12.1% less OPs compared to the state-of-the-art ReActNet, which indicates the effectiveness of the proposed BiMLP models.

Original:  Preliminaries2.1 Binarization in CNN models
Revised:  Preliminaries 2.1 Binarization in CNN models

Original: Given the weights W ∈ R
Revised: Given the weights W ∈ R

Original: c
Revised: c

Original: o × c i × k × k
Revised: o × c i × k × k

Original: where c i and c o are the number of input and output channelsand k is the kernel size, and also the activation A ∈ R c i × h × w where h and w are the height andwidth of the input feature, a common way of binarizing the weights and activations in CNN models isto apply sign function on the 32-bit ﬂoating point inputs, i.e. ,
Revised: where c i and c o are the number of input and output channels and k is the kernel size, and also the activation A ∈ R c i × h × w where h and w are the height and width of the input feature, a common way of binarizing the weights and activations in CNN models is to apply sign function on the 32-bit floating point inputs, i.e. ,

Original: [Equation]
Revised: [Equation]

Original: in which sign ( · ) is an element-wise operation that outputs +1 when the input is positive and -1otherwise.
Revised: in which sign ( · ) is an element-wise operation that outputs +1 when the input is positive andotherwise.

Original: Given the binarized weights and activations, the convolutional operation can be realizedwith binary operations only, i.e. :
Revised: Given the binarized weights and activations, the convolutional operation can be realized with binary operations only, i.e. :

Original: [Equation]
Revised: [Equation]

Original: in which (cid:12) represents binary convolutional operation that can be implemented with XNOR and
Revised: in which ⊙ represents binary convolutional operation that can be implemented with XNOR and

Original: POPCOUNT.
Revised: POPCOUNT.

Original: Note that the  aforementioned sign function is non-differentiable almost everywhere and backwardpropagation is unable to be applied during training.
Revised: Note that the gradient of aforementioned sign function is zero almost everywhere and backward propagation is unable to be applied during training.

Original: Thus, the back-propagation of sign functionfollows the straight through estimator (STE) rule:
Revised: Thus, the back-propagation of sign function follows the straight through estimator (STE) rule:

Original: [Equation]
Revised: [Equation]

Original: where L is the loss function corresponding to the speciﬁc task, e.g. , cross-entropy loss for imageclassiﬁcation.
Revised: where L is the loss function corresponding to the specific task, e.g. , cross-entropy loss for image classification.

Original: The Clip ( · , − 1 , 1) function clips the elements of the gradient into range [ − 1 , 1] .
Revised: The Clip ( · , − 1 , 1) function clips the elements of the gradient into range [ − 1 , 1] .

Original: Most of the existing binary CNN methods focus on improving the forward sign function and backward
Revised: Most of the existing binary CNN methods focus on improving the forward sign function and backward

Original: STE rule.
Revised: STE rule.

Original: For example, XNOR-Net [23] introduces channel-wise scale factors on the pure signfunction and improves the performance of binary CNN.
Revised: For example, XNOR-Net [35] introduces channel-wise scale factors on the pure sign function and improves the performance of binary CNN.

Original: Dorefa-Net [36] uses only a single scalefactor to achieve similar performance.
Revised: Dorefa-Net [55] uses only a single scale factor to achieve similar performance.

Original: ReActNet [17] proposes RSign and RPreLU function forbinary CNNs.
Revised: ReActNet [27] proposes RSign and RPreLU function for binary CNNs.

Original: DSQ [8] replaces the STE rule and uses the gradient of tanh function to replacethe gradient of sign function while BNN+ [6] introduces SignSwish function.
Revised: DSQ [8] replaces the STE rule and uses the gradient of tanh function to replace the gradient of sign function while BNN+ [6] introduces SignSwish function.

Original: Methods mentionedabove are all applied on CNN models and directly transfer them to MLP models results in severeperformance degradation.
Revised: Methods mentioned above are all applied on CNN models and directly transfer them to MLP models results in severe performance degradation.

Original: 2.2 Vision MLP Models
Revised: 2.2 Vision MLP Models

Original: Vision MLP models take patches of image (tokens) as input, and stack a sequence of channel-FClayers and spatial-FC layers for image recognition.
Revised: Vision MLP models take patches of image (tokens) as input, and stack a sequence of channel-FC layers and spatial-FC layers for image recognition.

Original: Speciﬁcally, given the input X ∈ R n × d in whichn is the number of tokens and d is the number of channels in each token, the channel-FC layer fusesthe channels in a single token and generate d (cid:48) channels, i.e. ,
Revised: Specifically, given the input X ∈ R n × d in which n is the number of tokens and d is the number of channels in each token, the channel-FC layer fuses the channels in a single token and generate d ′ channels, i.e. ,

Original: [Equation]
Revised: [Equation]

Original: [Equation] where W c ∈ R d × d (cid:48) is the parameter matrix of the channel-FC layer, and · represents the matrixmultiplication operation.
Revised: (4) where W c ∈ R d × d ′ is the parameter matrix of the channel-FC layer, and · represents the matrix multiplication operation.

Original: Channel-FC layer only aggregates information from different input channels,while lacking the communications between tokens.
Revised: Channel-FC layer only aggregates information from different input channels, while lacking the communications between tokens.

Original: Thus, the spatial-FC layer is also introduced in
Revised: Thus, the spatial-FC layer is also introduced in

Original: MLP models [30], i.e. ,
Revised: MLP models [45], i.e. ,

Original: [Equation]
Revised: [Equation]

Original: where W s ∈ R n × n (cid:48) is the parameter of spatial-FC layer.
Revised: where W s ∈ R n × n ′ is the parameter of spatial-FC layer.

Original: In general, MLP models usually set d = d (cid:48)and n = n (cid:48) to pursuit efﬁciency on both feature representation and computation of the entire network.
Revised: In general, MLP models usually set d = d ′ and n = n ′ to pursuit efficiency on both feature representation and computation of the entire network.

Original: The spatial-FC layer mentioned above cannot deal with images with diverse shapes, thus is unable tobe applied on downstream tasks such as image detection and image segmentation.
Revised: The spatial-FC layer mentioned above cannot deal with images with diverse shapes, thus is unable to be applied on downstream tasks such as image detection and image segmentation.

Original: In order to solve theproblem, Cycle-MLP [3] retains the position of each token and introduces Cycle-FC layer to enlargethe receptive ﬁeld of MLP to cope with downstream tasks while maintaining the computationalcomplexity.
Revised: In order to solve the problem, Cycle-MLP [3] retains the position of each token and introduces Cycle-FC layer to enlarge the receptive field of MLP to cope with downstream tasks while maintaining the computational complexity.

Original: Speciﬁcally, given the input feature Z ∈ R H × W × C in , the output of Cycle-FC layer is:
Revised: Specifically, given the input feature Z ∈ R H × W × C in , the output of Cycle-FC layer is:

Original: [Equation]
Revised: [Equation]

Original: in which W cycle ∈ R C in × C out is the parameter matrix of the Cycle-FC layer, and δ i ( c ) and δ j ( c ) aredeﬁned as:
Revised: in which W cycle ∈ R C in × C out is the parameter matrix of the Cycle-FC layer, and δ i ( c ) and δ j ( c ) are defined as:

Original: [Equation]
Revised: [Equation]

Original: where S H and S W are predeﬁned receptive ﬁelds.
Revised: where S H and S W are predefined receptive fields.

Original: Besides Cycle-MLP, Wave-MLP [29] representstoken as a wave function with amplitude and phase and proposes Wave-FC layer to solve the aboveproblem.
Revised: Besides Cycle-MLP, Wave-MLP [42] represents token as a wave function with amplitude and phase and proposes Wave-FC layer to solve the above problem.

Original: AS-MLP [14] avoids the restriction of ﬁxed input size by axially shifting channels of thefeature map, and is able to obtain information ﬂow from different axial directions.
Revised: AS-MLP [24] avoids the restriction of fixed input size by axially shifting channels of the feature map, and is able to obtain information flow from different axial directions.

Original: Compared to the convolutional operation in CNN and self-attention operation in Transformer, matrixmultiplication in MLP is computational friendly to GPUs and has speciﬁc advantage on inferencetime in reality.
Revised: Compared to the convolutional operation in CNN and self-attention operation in Transformer, matrix multiplication in MLP is computational friendly to GPUs and has specific advantage on inference time in reality.

Original: Beneﬁts from its simple architecture, MLP models can be generalized to varioushardwares.
Revised: Benefits from its simple architecture, MLP models can be generalized to various hardware.

Original: However, the cumbersome MLP model with a large number of parameters and FLOPslimits its ability to apply to portable devices, which means it is essential to derive a compact MLPmodel, e.g. binary vision MLPs.
Revised: However, the cumbersome MLP model with a large number of parameters and FLOPs limits its ability to apply to portable devices, which means it is essential to derive a compact MLP model, e.g. binary vision MLPs.

Original:  Binary Vision MLPs
Revised:  Binary Vision MLPs

Original: In this section, we ﬁrst analyze the difﬁculty of binarizing vision MLP through comparing therepresentation ability of binary features in FC layers and convolutional layers.
Revised: In this section, we first analyze the difficulty of binarizing vision MLP through comparing the representation ability of binary features in FC layers and convolutional layers.

Original: We then propose anovel binary architecture for vision MLP, which fuses the outputs from the same layer with multibranch MLP block and those from different layers with a universal shortcut connection to strengthenthe representation ability of MLP.
Revised: We then propose a novel binary architecture for vision MLP, which fuses the outputs from the same layer with multibranch MLP block and those from different layers with a universal shortcut connection to strengthen the representation ability of MLP.

Original: 3.1 Difﬁculty of Binarizing Vision MLP
Revised: 3.1 Difficulty of Binarizing Vision MLP

Original: Given the input feature F ∈ R C in × H × W and the convolutional kernel K ∈ R C out × C in × K h × K win which K h and K w are the height and width of the kernel, respectively.
Revised: Given the input feature F ∈ R C in × H × W and the convolutional kernel K ∈ R C out × C in × K h × K w in which K h and K w are the height and width of the kernel, respectively.

Original: The computationalcomplexity of the convolutional operation is:
Revised: The computational complexity of the convolutional operation is:

Original: [Equation]
Revised: [Equation]

Original: After binarizing the weights and activations, the elements of the output feature map Y is derived fromthe following equation:
Revised: After binarizing the weights and activations, the elements of the output feature map Y is derived from the following equation:

Original: [Equation]
Revised: [Equation]

Original: in which K b and F b are binarized kernel and input feature using Eq.
Revised: in which K b and F b are binarized kernel and input feature using Eq.

Original: 1 whose elements are selectedfrom { +1 , − 1 } .
Revised: 1 whose elements are selected from { +1 , − 1 } .

Original: According to Eq.
Revised: According to Eq.

Original: 9, we can easily conclude that each element in Y is chosen from
Revised: 9, we can easily conclude that each element in Y is chosen from

Original: N +1 different values from the set S b = {− N, − N +2 , ..., N − 2 , N } , in which N = C in · K h · K w .
Revised: N +1 different values from the set S b = {− N, − N +2 , ..., N − 2 , N } , in which N = C in · K h · K w .

Original: Speciﬁcally, we denote the number N as the representation ability of the binary FC layer.
Revised: Specifically, we denote the number N as the representation ability of the binary FC layer.

Original: In traditional CNN models such as ResNet [9] and VGGNet [27], convolutional operation with kernelsize 3 × 3 occupies the network.
Revised: In traditional CNN models such as ResNet [17] and VGGNet [40], convolutional operation with kernel size 3 × 3 occupies the network.

Original: Efﬁcient-Net [28] uses convolution with kernel size 3 × 3 and 5 × 5 ,while the recently proposed RepLKNet [7] expands the kernel size to 31 × 31 .
Revised: Efficient-Net [41] uses convolution with kernel size 3 ×and 5 × 5 , while the recently proposed RepLKNet [7] expands the kernel size to 31 × 31 .

Original: Different from the CNN models mentioned above, FC layers in MLP model can be treated as convolutional operationwith kernel size 1 × 1 .
Revised: Different from the CNN models mentioned above, FC layers in MLP model can be treated as convolutional operation with kernel size 1 × 1 .

Original: Therefore, compared to the convolutional layer with kernel size k × k , FC layer with the same numberof input channels and output channels has 1 /k 2 computational complexity and 1 /k 2 representationability after being binarized. 
Revised: Therefore, compared to the convolutional layer with kernel size k × k , FC layer with the same number of input channels and output channels has 1 /k 2 computational complexity and 1 /k 2 representation ability after being binarized. Note that assuming CNN models and MLP models have the same number of input channels and output channels is reasonable. For example, WaveMLP-S [42] with 30 M parameters and 4 . 5 G FLOPs has four stages with base channel 64-128-320-512, while ResNet50 with 25 .

Original:  In order to achieve the same representation ability, the number of inputchannels of FC layer must be multiplied by k 2 .
Revised: 5 M parameters and 4 . 1 G FLOPs has four stages with base channel 64-128-256which are roughly the same. In order to achieve the same representation ability as convolutional layer, the number of input channels of FC layer must be multiplied by k 2 .

Original: Similarly, the number of output channels shouldalso be scaled up in order to maintain the representation ability of the next FC layer.
Revised: Similarly, the number of output channels should also be scaled up in order to maintain the representation ability of the next FC layer.

Original: Hence, thecomputational complexity is k 2 times compared to the convolutional layer with kernel size k × k ,which drastically reduce the advantage of binary neural network.
Revised: Hence, the computational complexity is k 2 times compared to the convolutional layer with kernel size k × k , which drastically reduce the advantage of binary neural network.

Original: To this end, we design a novel binaryarchitecture for vision models (Bi-MLP) to enhance the representation ability while maintainingcompact model complexity.
Revised: To this end, we design a novel binary architecture for vision models (BiMLP) to enhance the representation ability while maintaining compact model complexity.

Original: A series of speciﬁc design are proposed to achieve this goal, which willbe elaborated in the following sections.
Revised: A series of specific design are proposed to achieve this goal, which will be elaborated in the following sections.

Original: 3.2 Bi-MLP Architecture
Revised: 3.2 BiMLP Architecture

Original: To deal with the above problem, we introduce a mulit-branch binary MLP block (MBB block) with
Revised: To deal with the above problem, we introduce a mulit-branch binary MLP block (MBB block) with

Original: Uni-shortcut operation to enhance the representation capacity of binary MLP models.
Revised: Uni-shortcut operation to enhance the representation capacity of binary MLP models.

Original: We also designa novel architecture of downsampling layer for binary MLPs to further reduce the computationalcomplexity while maintaining the accuracy.
Revised: We also design a novel architecture of downsampling layer for binary MLPs to further reduce the computational complexity while maintaining the accuracy.

Original: Multi-branch binary MLP block.
Revised: Multi-branch binary MLP block.

Original: To enhance the representation ability of binary vision MLPs withcompact complexity, we propose a new binary MLP block containing multiple branches.
Revised: To enhance the representation ability of binary vision MLPs with compact complexity, we propose a new binary MLP block containing multiple branches.

Original: Note that in
Revised: Note that in

Original: MLP-Mixer [30], ResMLP [31] and gMLP [35], an MLP block mainly consists of two parts includingthe spatial projection part and the channel projection part.
Revised: MLP-Mixer [45], ResMLP [46] and gMLP [54], an MLP block mainly consists of two parts including

Original: For CycleMLP [3] and WaveMLP [29],the spatial projection is achieved by introducing some local FC operations ( e.g. Cycle-FC, Wave-FC)which are applied on the adjacent tokens to cope with input images with different shapes.

Original: Meanwhile,the global FC layer (ordinary FC layer) is used for channel mixing.

Original: Different from them, wesimultaneously introduce spatial projection and channel projection into a single part, and form twodifferent multi-branch binary MLP blocks (MBB blocks) as shown in Fig.

Original: Channel Projection
Revised: Channel Projection

Original: Spatial Projection
Revised: Spatial Projection

Original: FC(Dim, 4xDim)
Revised: FC(Dim, 4xDim)

Original: GeLU
Revised: GeLU

Original: FC(Dim, 4xDim) 1 x 1 F C 1 x 7 P A 
Revised: FC(Dim, 4xDim) 1 x 1 F C 1 x 7 P A T

Original: T M 7 x 1 P A 
Revised: M 7 x 1 P A T

Original: T M
Revised: M

Original: LayerNorm LayerNorm
Revised: LayerNorm LayerNorm

Original: Patch Embeddings
Revised: Patch Embeddings

Original: Outputs
Revised: Outputs

Original: BatchNorm
Revised: BatchNorm

Original: MBB Block
Revised: MBB Block

Original: BatchNorm
Revised: BatchNorm

Original: Patch Embeddings
Revised: Patch Embeddings

Original: Outputs
Revised: Outputs

Original: C h
Revised: C h

Original: a nn e l B i - F C
Revised: a nn e l B i - F C

Original: S p a t i a l
Revised: S p a t i a l

Original: B i - M L P
Revised: B i - M L P

Original: S p a t i a l
Revised: S p a t i a l

Original: B i - M L P
Revised: B i - M L P

Original: MBB Block
Revised: MBB Block

Original: C h a nn e l B i - M
Revised: C h a nn e l B i - M

Original: L P
Revised: L P

Original: S p a t i a l
Revised: S p a t i a l

Original: B i - F C
Revised: B i - F C

Original: S p a t i a l
Revised: S p a t i a l

Original: B i - F C
Revised: B i - F C

Original: Spatial Bi-MLP
Revised: Spatial Bi-MLP

Original: PATM CycleMLP ……
Revised: PATM CycleMLP ……

Original: Spatial Bi-FC
Revised: Spatial Bi-FC

Original: Local Bi-FC BN
Revised: Local Bi-FC BN

Original: RPReLU
Revised: RPReLU

Original: Uni-Shortcut
Revised: Uni-Shortcut

Original: Channel Bi-FC
Revised: Channel Bi-FC

Original: Global Bi-FC BN
Revised: Global Bi-FC BN

Original: RPReLU
Revised: RPReLU

Original: Uni-Shortcut
Revised: Uni-Shortcut

Original: Channel Bi-MLP
Revised: Channel Bi-MLP

Original: Channel Bi-FC Channel Bi-FC(
Revised: Channel Bi-FC Channel Bi-FC

Original: [Figure]
Revised: [Figure]

Original: Vision MLP ( [Figure]
Revised: the spatial projection part and the channel projection part. For CycleMLP [3] and WaveMLP [42], the spatial projection is achieved by introducing some local FC operations ( e.g. Cycle-FC, Wave-FC) which are applied on the adjacent tokens to cope with input images with different shapes. Meanwhile, the global FC layer (ordinary FC layer) is used for channel mixing.

Original: ) Bi-MLP (ours) e [Figure] g., [Figure]
Revised: Different from them, we simultaneously introduce spatial projection and channel projection into a single part, and form two different multi-branch binary MLP blocks (MBB blocks) as shown in Fig.

Original: There are four different elements in the MBB blocks, i.e. , the Spatial Binary FC, the Spatial Binary
Revised: There are four different elements in the MBB blocks, i.e. , the Spatial Binary FC, the Spatial Binary

Original: MLP, the Channel Binary FC and the Channel Binary MLP.
Revised: MLP, the Channel Binary FC and the Channel Binary MLP.

Original: Given the binarized input feature X b , theoutput of Spatial Binary FC is:
Revised: Given the binarized input feature X b , the output of Spatial Binary FC is:

Original: [Equation]
Revised: [Equation]

Original: [Equation] in which RPReLU( · ) is the activation function introduced in ReActNet [17].
Revised: in which RPReLU( · ) is the activation function introduced in ReActNet [27].

Original: LFC( · ) is the local FCthat fuses the spatial information which is realized with different form in previous works.
Revised: LFC( · ) is the local FC that fuses the spatial information which is realized with different form in previous works.

Original: BN( · ) is theordinary batch normalization, and U( · ) is the proposed Uni-shortcut which will be introduced belowin Eq.
Revised: BN( · ) is the ordinary batch normalization, and U( · ) is the proposed Uni-shortcut which will be introduced below in Eq.

Original: For Spatial Binary MLP, we utilize the PATM architecture introduced in WaveMLP [29].
Revised: For Spatial Binary MLP, we utilize the PATM architecture introduced in WaveMLP [42].

Original: Other forms such as CycleMLP architecture in [3], and axially shifting architecture in AS-MLP [14]can be used as a replacement.
Revised: Other forms such as CycleMLP architecture in [3], and axially shifting architecture in AS-MLP [24] can be used as a replacement.

Original: The Channel Binary FC is deﬁned as:
Revised: The Channel Binary FC is defined as:

Original: [Equation]
Revised: [Equation]

Original: in which GFC( · ) is the original global FC layer that mixes the information between different channels.
Revised: in which GFC( · ) is the original global FC layer that mixes the information between different channels.

Original: Finally, the Channel Binary MLP is the stack of multiple Channel Binary FCs to strengthen the abilityof channel mixing.
Revised: Finally, the Channel Binary MLP is the stack of multiple Channel Binary FCs to strengthen the ability of channel mixing.

Original: Given the four different elements introduced above, the ﬁrst MBB block uses two Spatial Binary
Revised: Given the four different elements introduced above, the first MBB block uses two Spatial Binary

Original: MLPs focusing on the height and width of the spatial dimension, and a Channel Binary FC.
Revised: MLPs focusing on the height and width of the spatial dimension, and a Channel Binary FC.

Original: Thesecond MBB block uses two Spatial Binary FC and a Channel Binary MLP.
Revised: The second MBB block uses two Spatial Binary FC and a Channel Binary MLP.

Original: In this way, the channelprojection and spatial projection are used multiple times, while at the same time the ﬁrst block has astronger ability for spatial mixing and the second block is good at mixing the channel information.
Revised: In this way, the channel projection and spatial projection are used multiple times, while at the same time the first block has a stronger ability for spatial mixing and the second block is good at mixing the channel information.

Original: Finally, all the layer normalizations are replaced with batch normalizations, as shown in the ablationstudy in Tab. 4.
Revised: Finally, all the layer normalizations are replaced with batch normalizations, as shown in the ablation study in Tab. 4.

Original: With the above architectures, the representation ability can be recovered with less computationalcomplexity by using multiple branches compared with directly expanding the input and output channels.
Revised: With the above architectures, the representation ability can be recovered with less computational complexity by using multiple branches compared with directly expanding the input and output channels.

Original: Generally, k 2 branches are needed to get the same representation ability, and the computationalcomplexity is the same compared to the convolutional layer with kernel size k × k . 
Revised: Generally, k 2 branches are needed to get the same representation ability, and the computational complexity is the same compared to the convolutional layer with kernel size k × k . Note that as [Equation] [Figure]

Original: Note that asthe number of branches increases, both the representation ability and the computational complexitywill increase.
Revised: the number of branches increases, both the representation ability and the computational complexity will increase.

Original: Thus, we seek for a balance between the effectiveness and the efﬁciency.
Revised: Thus, we seek for a balance between the effectiveness and the efficiency.

Original: In thefollowing experiments, we use only 3 branches as mentioned above and show that it is enough toobtain competitive results in Tab. 3.
Revised: In the following experiments, we use only 3 branches as mentioned above and show that it is enough to obtain competitive results in Tab. 3.

Original: Uni-shortcut.
Revised: Uni-shortcut.

Original: Recall that the proposed MBB blocks fuse the information from the same layer.
Revised: Recall that the proposed MBB blocks fuse the information from the same layer.

Original: Theprevious works of binary CNNs show that combining the outputs from different layers is important[Figure] for the information ﬂow.
Revised: The previous works of binary CNNs show that combining the outputs from different layers is important for the information flow.

Original: However, the number of output channels and input channels are usuallydifferent in MLP models and the original shortcut connection cannot be directly applied.
Revised: However, the number of output channels and input channels are usually different in MLP models and the original shortcut connection cannot be directly applied.

Original: Sincethe input channels c in are usually multiple times of the output channels c out (or vise versa), i.e.c in = nc out (or c out = nc in ), n ∈ N + .
Revised: Since the input channels c in are usually multiple times of the output channels c out (or vise versa), i.e. c in = nc out (or c out = nc in ), n ∈ N + .

Original: Therefore, we propose a universal shortcut (Uni-shortcut) tocope with the following two different situations.
Revised: Therefore, we propose a universal shortcut (Uni-shortcut) to cope with the following two different situations.

Original: Given the input feature X b ∈ R C in × H × W and a FC layer with c in and c out input and output channels,we have:
Revised: Given the input feature X b ∈ R C in × H × W and a FC layer with c in and c out input and output channels, we have:

Original: [Equation]
Revised: [Equation]

Original: where the input is averaged based on the channel dimension when c in = nc out , and the input isrepeated n times and concatenated together to ﬁt the output dimension when c out = nc in .
Revised: where the input is averaged based on the channel dimension when c in = nc out , and the input is repeated n times and concatenated together to fit the output dimension when c out = nc in .

Original: Downsampling block.
Revised: Downsampling block.

Original: The downsampling layers are not binarized during training and inference for
Revised: The downsampling layers are not binarized during training and inference for

Original: MLP models, otherwise the performance will drastically decrease, as also discussed in prior binary
Revised: MLP models, otherwise the performance will drastically decrease, as also discussed in prior binary

Original: CNNs [18, 17, 23].
Revised: CNNs [28, 27, 35].

Original: This brings the problem that the FLOPs of downsampling layers occupy thewhole network since the 3 × 3 convolution with stride 2 is a common way of simultaneously reducingthe spatial size and changing the number of channels of the feature maps.
Revised: This brings the problem that the FLOPs of downsampling layers occupy the whole network since the 3 × 3 convolution with stride 2 is a common way of simultaneously reducing the spatial size and changing the number of channels of the feature maps.

Original: Thus,  we separatelyhandle the spatial and channel dimension by using 1 × 1 convolution with stride 1 (FC layer), andmaxpooling with diverse kernel size to replace the original convolution, as shown in Fig.
Revised: Thus, similar to [16], we separately handle the spatial and channel dimension by using 1 × 1 convolution with stride(FC layer), and maxpooling with diverse kernel size to replace the original convolution, as shown in Fig.

Original: By thisway, the computational complexity is signiﬁcantly reduced while the classiﬁcation performance ismaintained.
Revised: By this way, the computational complexity is significantly reduced while the classification performance is maintained.

Original:  Experiments
Revised:  Experiments

Original: In this section, we ﬁrst evaluate the effectiveness of the proposed method on ImageNet dataset.
Revised: In this section, we first evaluate the effectiveness of the proposed method on ImageNet dataset.

Original: Theexperimental settings include the dataset statistics, details of the network architectures and the trainingstrategy is introduced in Sec. 4.1.
Revised: The experimental settings include the dataset statistics, details of the network architectures and the training strategy is introduced in Sec. 4.1.

Original: In Sec. 4.2, we compare our method with a series of state-of-the-artbinary CNN models in terms of accuracy and FLOPs.
Revised: In Sec. 4.2, we compare our method with a series of state-of-the-art binary CNN models in terms of accuracy and FLOPs.

Original: The ablation study on each part of the proposedmethod is conducted in Sec. 4.3. 
Revised: The ablation study on each part of the proposed method is conducted in Sec. 4.3. [Table] [Table]

Revised: 73.2 Dorefa-Net [55] 1/ - - - 52.

Revised: 67.7 ABCNet [26] 1/ - - - 42.

Original: 4.1 Experimental Settings
Revised: 67.6 Bireal-Net-18 [28] 1/ 1. 79.5 Bireal [Table] Net-34 [28] 1/ 1.

Revised: 83.9 UAD-BNN-18 [20] 1/ - - - 57.

Revised: 80.2 UAD-BNN-34 [20] 1/ - - - 62.

Revised: 84.5 RBNN-18 [25] 1/ - - - 59.

Revised: 81.9 RBNN-34 [25] 1/ - - - 63.

Revised: 84.4 Real2bin [31] 1/ 1.

Revised: 86.2 ReCU [51] 1/ - - - 66.

Revised: 86.5 FDA-BNN [48] 1/ - - - 66.

Revised: 86.4 LCR-BNN-18 [39] 1/ - - - 59.

Revised: 81.6 LCR-BNN-34 [39] 1/ - - - 63.

Revised: 84.6 LCR-BNN-ReAct [39] 1/ - - - 69.

Revised: 85.7 Bi-half-18 [22] 1/ - - - 60.

Revised: 82.86 Bi-half-34 [22] 1/ - - - 64.

Revised: 85.36 MeliusNet-22 [2] 1/ 1.

Revised: 84.7 MeliusNet-29 [2] 1/ 1.

Revised: 86.2 MeliusNet

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: Ours)

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: 4.1 Experimental Settings

Original: ImageNet dataset.
Revised: ImageNet dataset.

Original: The benchmark dataset ImageNet-1k [26] contains over 1 .
Revised: The benchmark dataset ImageNet-1k [38] contains over 1 .

Original: 2 M training imagesand 50 k validation images from 1,000 different categories.
Revised: 2 M training images and 50 k validation images from 1,000 different categories.

Original: This is the most commonly used imageclassiﬁcation dataset in other binary CNN methods.
Revised: This is the most commonly used image classification dataset in other binary CNN methods.

Original: Network architecture.
Revised: Network architecture.

Original: We utilize the architecture proposed in [29] and replace the MLP block withthe proposed MBB block with Uni-shortcut (Fig. 2(b)) as well as the replacement of downsamplingblocks (Fig. 3).
Revised: We utilize the architecture proposed in [42] and replace the MLP block with the proposed MBB block with Uni-shortcut (Fig. 2(b)) as well as the replacement of downsampling blocks (Fig. 3).

Original: Details of the architectures of Bi-MLP models are speciﬁed in Tab. Training details.
Revised: Details of the architectures of BiMLP models are specified in Tab. 1.

Original: Following prior methods [18, 17, 21], we use a two step training strategy.

Original: In theﬁrst training step, we use the full precision MLP model as the teacher model and the network with

Original: [Table]

Original: [Table]

Original: 73.2 Dorefa

Original: [Table]
Revised: [Table]

Original: Net [36] 1/ - - - 52.

Original: 67.7 ABCNet [16] 1/ - - - 42.

Original: [Table]

Original: Net-18 [18] 1/ 1.

Original: [Table]

Original: Net

Original: [Table] 83.9 RBNN-18 [15] 1/ - - - 59. 81.9 RBNN-34 [15] 1/ - - - 63. 84.4 Real2bin [21] 1/ 1.
Revised: Training details.

Original: 86.2 MeliusNet-22 [2] 1/ 1. 84.7 MeliusNet-29 [2] 1/ 1. 86.2 MeliusNet-42 [2] 1/ 1. 88.3 MeliusNet-59 [2] 1/ 2.
Revised: Following prior methods [28, 27, 31], we use a two step training strategy.

Original: 89.7 ReActNet-B [17] 1/ 0.

Original: - ReActNet-C [17] 1/ 1. Bi-MLP-S (Ours) 1/ 1. 89.6 Bi-MLP-M (Ours) 1/ 1. the same architecture but binary activations as the student model.
Revised: In the first training step, we use the full precision MLP model as the teacher model and the network with the same architecture but binary activations as the student model.

Original: A knowledge distillation loss isused to facilitate the training of student network:
Revised: A knowledge distillation loss is used to facilitate the training of student network:

Original: [Equation]
Revised: [Equation]

Original:  in which L kl ( · ) and L ce ( · ) are the KL divergence and cross-entropy loss, y s and y t are the outputprobability of the student and teacher model, y gt is the one-hot ground-truth probability and α is thetrade-off hyper-parameter ( α = 0 .
Revised: 13) in which L kl ( · ) and L ce ( · ) are the KL divergence and cross-entropy loss , y s and y t are the output probability of the student and teacher model, y gt is the one-hot ground-truth probability and α is the trade-off hyper-parameter ( α = 0 .

Original: 9 in the following experiments).
Revised: 9 in the following experiments).

Original: For the second training step, thefull precision ResNet-34 is used as the teacher network.
Revised: For the second training step, the full precision ResNet-34 is used as the teacher network.

Original: The binarized MLP model (both weights andactivations) is used as the student model and the corresponding weights are initialized by the trainingresults from the ﬁrst step.
Revised: The binarized MLP model (both weights and activations) is used as the student model and the corresponding weights are initialized by the training results from the first step.

Original: In both steps, the student models are trained for 300 epochs using AdamW [19] optimizer withmomentum of 0.9 and weight decay of 0.05.
Revised: In both steps, the student models are trained for 300 epochs using AdamW [29] optimizer with momentum of 0.9 and weight decay of 0.05.

Original: We start with the learning rate of 1 × 10 − 3 and acosine learning rate decay scheduler is used during training.
Revised: We start with the learning rate of 1 × 10 − 3 and a cosine learning rate decay scheduler is used during training.

Original: We use NVIDIA V100 GPUs with atotal batchsize of 1024 to train the model.
Revised: We use NVIDIA V100 GPUs with a total batchsize of 1024 to train the model.

Original: The commonly used data-augmentation strategies suchas Cut-Mix [33], Mixup [34] and Rand-Augment [5] are used.
Revised: The commonly used data-augmentation strategies such as Cut-Mix [52], Mixup [53] and Rand-Augment [5] are used.

Original: The ﬁrst layer, the last layer and thedownsampling layers are not binarized during training and inference.[Table] [Table]
Revised: The first layer, the last layer and the downsampling layers are not binarized during training and inference.

Original: 4.2 Experimental Results on ImageNet
Revised: 4.2 Experimental Results on ImageNet

Original: In this section [Table] we compare the proposed model with the binary CNN models derived from otherstate-of-the-art methods on the ImageNet-1k dataset, including Dorefa-Net [36], XNOR-Net [23],
Revised: In this section, we compare the proposed model with the binary CNN models derived from other state-of-the-art methods on the ImageNet-1k dataset, including Dorefa-Net [55], XNOR-Net [35],

Original: ABCNet [16], Bireal-Net [18], Real2bin [21], MeliusNet [2], ReActNet [17] [Table] etc.
Revised: ABCNet [26], Bireal-Net [28], Real2bin [31], MeliusNet [2], ReActNet [27], etc.

Original: As shown in Tab. 2,the proposed Bi-MLP models achieve competitive performance compared to the state-of-the-art binary
Revised: As shown in Tab. 2, the proposed BiMLP models achieve competitive performance compared to the state-of-the-art binary

Original: CNN models.
Revised: CNN models.

Original: We can see that Bi-MLP-S achieves 70.0% top-1 accuracy and 89.6% top-5 accuracywith 0.156G OPs, which surpasses the most recent methods such as MeliusNet-42 by 0.8% and 1.3%with less than a half OPs, and is comparable to
Revised: We can see that BiMLP-S achieves 70.0% top-1 accuracy and 89.6% top-5 accuracy with 0.156G OPs, which surpasses the most recent methods such as MeliusNet-42 by 0.8% and 1.3% with less than a half OPs, and is comparable to

Original: ReActNet-B. Meanwhile, Bi-MLP-M achieves 72.7%top-1 accuracy and 91.1% top-5 accuracy with only 0.188G OPs, which has 12.1% less OPs than
Revised: ReActNet-B. Meanwhile, BiMLP-M achieves 72.7% top-1 accuracy and 91.1% top-5 accuracy with only 0.188G OPs, which has 12.1% less OPs than

Original: ReActNet-C and is 1.3% better on top-1 accuracy.
Revised: ReActNet-C and is 1.3% better on top-1 accuracy.

Original: The above results show the superiority of theproposed Bi-MLP.
Revised: The above results show the superiority of the proposed BiMLP.

Original: 4.3 Ablation Studies
Revised: 4.3 Ablation Studies

Original: In this section, we demonstrate the effectiveness of each part of the proposed method by conducting aseries of ablation studies.
Revised: In this section, we demonstrate the effectiveness of each part of the proposed method by conducting a series of ablation studies.

Original: Different branches in MBB blocks.
Revised: Different branches in MBB blocks.

Original: Firstly, we show the experimental results of using differentbranches in Tab. 3.
Revised: Firstly, we show the experimental results of using different branches in Tab. 3.

Original: The # S i indicates the number of Spatial Binary FC (MLP) in the MBB block i and# C i is the number of Channel Binary FC (MLP) in the corresponding block.
Revised: The # S i indicates the number of Spatial Binary FC (MLP) in the MBB block i and # C i is the number of Channel Binary FC (MLP) in the corresponding block.

Original: Note that we fuse theinformation along height and width of the feature map with different Spatial Binary FC (MLP), thusthe # S i should be even numbers.
Revised: Note that we fuse the information along height and width of the feature map with different Spatial Binary FC (MLP), thus the # S i should be even numbers.

Original: During the ablation study we keep ( # S 1 + # S 2 ) / ( # C 1 + # C 2 ) = 2so that the ability of fusing information of different dimensions is roughly the same. 
Revised: During the ablation study we keep ( # S 1 + # S 2 ) / ( # C 1 + # C 2 ) =so that the ability of fusing information of different dimensions is roughly the same. [Table] [Table] [Table]

Original: The experiments are conducted on Bi-MLP-S model.
Revised: The experiments are conducted on BiMLP-S model.

Original: As shown in Tab. 3, when the MBB blocksfocus on only one dimension in setting 1 and 2, the top-1 accuracy decreases from 70.0% to 69.2%and 68.9%, which shows that mixing the information from different dimensions in a single block isimportant.
Revised: As shown in Tab. 3, when the MBB blocks focus on only one dimension in setting 1 and 2, the top-1 accuracy decreases from 70.0% to 69.2% and 68.9% [Table] which shows that mixing the information from different dimensions in a single block is important.

Original: In setting 4 and 5, more branches are used in each block compared to setting 3 (baseline),and only improves the accuracy by 0.8% and 0.5%, while increases 19.2% and 14.1% OPs.
Revised: In setting 4 and 5, more branches are used in each block compared to setting 3 (baseline) [Table] and only improves the accuracy by 0.8% and 0.5%, while increases 19.2% and 14.1% OPs.

Original: Thus, weuse setting 3 for all the experiments considering the balance between the effectiveness and efﬁciency.
Revised: Thus, we use setting 3 for all the experiments considering the balance between the effectiveness and efficiency.

Original: Normalization and activation.
Revised: Normalization and activation.

Original: Different forms of normalization layer and activation function arecompared.
Revised: Different forms of normalization layer and activation function are compared.

Original: The candidates of normalization layer include Layer normalization (LN) and batchnormalization (BN), and the candidates of activation functions are GeLU, ReLU, PReLU and
Revised: The candidates of normalization layer include Layer normalization (LN) and batch normalization (BN), and the candidates of activation functions are GeLU, ReLU, PReLU and

Original: RPReLU.
Revised: RPReLU.

Original: As the results shown in Tab. 4, the combination of BN and RPReLU achieves the bestaccuracy, and is applied to all other experiments.
Revised: As the results shown in Tab. 4, the combination of BN and RPReLU achieves the best accuracy, and is applied to all other experiments.

Original: Shortcut connection.
Revised: Shortcut connection.

Original: We also study the effectiveness of the proposed Uni-shortcut.
Revised: We also study the effectiveness of the proposed Uni-shortcut.

Original: Three differentsetting are used in this ablation study.
Revised: Three different setting are used in this ablation study.

Original: The ﬁrst is using the MBB block without any residualconnection.
Revised: The first is using the MBB block without any residual connection.

Original: The second is using the proposed Uni-shortcut in MBB blocks.The third is using the [Table] [Table]
Revised: The second is using the proposed Uni-shortcut in MBB blocks.

Original: [Table]

Original: [Table]

Original: + Dorefa-Net [36]

Original: [Table]

Original: + XNOR-Net

Original: [Table]

Original: + ReActNet [17]

Original: [Table]

Original: + Ours

Original: [Table]

Original: [Figure] [Table] [Figure] traditional shortcut in MBB blocks.
Revised: The third is using the traditional shortcut in MBB blocks.

Original: In this circumstances, only those input-output pairs who have thesame shape will be connected.
Revised: In this circumstances, only those input-output pairs who have the same shape will be connected.

Original: As shown in Tab. 5, compared to the MBB block without using any residual connection and using thetraditional shortcut, Uni-shortcut improves the top-1 accuracy by 1.7% and 0.9% which shows thepriority of the proposed method.
Revised: As shown in Tab. 5, compared to the MBB block without using any residual connection and using the traditional shortcut, Uni-shortcut improves the top-1 accuracy by 1.7% and 0.9% which shows the priority of the proposed method.

Original: Dowmsampling layer.
Revised: Downsampling layer.

Original: We compare the proposed downsampling block using the FC layer togetherwith multi-branch maxpooling to the original downsampling layer using 3 × 3 convolution with stride2 in Tab. 6.
Revised: We compare the proposed downsampling block using the FC layer together with multi-branch maxpooling to the original downsampling layer using 3 × 3 convolution with stride 2 in Tab. 6.

Original: The result shows that we can signiﬁcantly reduce the computational complexity from0.265G to 0.156G (41.1% decrease) with only 0.3% drop of top-1 classiﬁcation accuracy.
Revised: The result shows that we can significantly reduce the computational complexity from 0.265G to 0.156G (41.1% decrease) with only 0.3% drop of top-1 classification accuracy.

Original: Hyper-parameters.
Revised: Hyper-parameters.

Original: As shown in Eq.
Revised: As shown in Eq.

Original: 13, α is used as a hyper-parameter to balance the loss functionbetween learning from the output probability of teacher model and learning from the ground-truthone-hot label.
Revised: 13, α is used as a hyper-parameter to balance the loss function between learning from the output probability of teacher model and learning from the ground-truth one-hot label.

Original: We report the accuracy on ImageNet-1k validation set in Fig.
Revised: We report the accuracy on ImageNet-1k validation set in Fig.

Original: 4 and show that α = 0 .{+
+}
Revised: 4 and show that α = {+
+}

Original: 9yields the best result.
Revised: .yields the best result.

Original: Comparisons of different binarization methods.
Revised: Comparisons of different binarization methods.

Original: Finally, we compare the proposed method toother binarization methods proposed for CNN models in Tab. 7.
Revised: Finally, we compare the proposed method to other binarization methods proposed for CNN models in Tab. 7.

Original: The 1-bit model is to directlybinarize the fp32 WaveMLP model using the sign function and the STE rule.
Revised: The 1-bit model is to directly binarize the fp32 WaveMLP model using the sign function and the STE rule.

Original: The experimentalresults show that our method can signiﬁcantly outperform other binarization methods that are directlyapplied to MLP model.
Revised: The experimental results show that our method can significantly outperform other binarization methods that are directly applied to MLP model.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we propose a new paradigm with speciﬁc architecture design for binarizing the vision
Revised: In this paper, we propose a new paradigm with specific architecture design for binarizing the vision

Original: MLP models.
Revised: MLP models.

Original: We point out that compared to binarizing the convolutional operation with large kernelsize in CNN models, binarizing the FC layers yields a MLP model with poor representation ability.
Revised: We point out that compared to binarizing the convolutional operation with large kernel [Table] [Figure] [Table]

Original: Simply increase the number of channels will ﬁx this problem, but the computational complexity isquadratically increased.
Revised: [Figure] [Table] [Figure] [Table]

Original: Thus, we propose a novel multi-branch binary MLP block (MBB block) withuniversal shortcut (Uni-shortcut) that can recover the representation ability while maintaining thecomputational complexity. We also redesign the downsampling layer that can signiﬁcantly reduce the
Revised: [Figure]

Original: OPs of the binary MLP model while at the same time keep the performance roughly unchanged.
Revised: size in CNN models, binarizing the FC layers yields a MLP model with poor representation ability.

Original: Theexperimental results on ImageNet-1k dataset demonstrate the effectiveness of the proposed method,and we achieve a comparable performance with the state-of-the-art binary CNN models. Checklist The checklist follows the references. Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Simply increase the number of channels will fix this problem

Original: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .

Original: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:

Original: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .•

Original: Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.•

Original: Did you include the license to the code and datasets?

Original: [N/A]
Revised: [Table]

Original: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the

Original: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?

Original: [Yes](b) Did you describe the limitations of your work? [Yes](c) Did you discuss any potential negative societal impacts of your work? [N/A](d)
Revised: but the computational complexity is quadratically increased.

Original: Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes]2.

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results? [N/A](b) Did you include complete proofs of all theoretical results? [N/A]3.
Revised: Thus, we propose a novel multi-branch binary MLP block (MBB block) with universal shortcut (Uni-shortcut) that can recover the representation ability while maintaining the computational complexity.

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [No](b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes](c)

Original: Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No] (d) Did you include the total amount of compute and thetype of resources used (e
Revised: We also redesign the downsampling layer that can significantly reduce the

Original: g., type of GPUs, internal cluster, or cloud provider)? [Yes] If you are using existing assets (e
Revised: OPs of the binary MLP model while at the same time keep the performance roughly unchanged.

Original: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]

Original: (b) Did you mention the license of the assets?

Original: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]

Original: d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating? [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: The experimental results on ImageNet-1k dataset demonstrate the effectiveness of the proposed method, and we achieve a comparable performance with the state-of-the-art binary CNN models.

Original: information or offensive content?

Original: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?

Original: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?

Original: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?

Original: [N/A]