Original: MGIC: Multigrid-in-Channels Neural Network Architectures
Revised: MGIC: Multigrid-in-Channels Neural Network Architectures

Original: Abstract
Revised: Abstract

Original: Multigrid (MG) methods are effective at solving numerical PDEs in linear complexity.
Revised: Multigrid (MG) methods are effective at solving numerical PDEs in linear complexity.

Original: In this work we present a multigrid-in-channels (MGIC) approach that tackles the quadratic growth of the number of parameters with respect to the number of channels in standard convolutional neural networks (CNNs).
Revised: In this work we present a multigrid-in-channels (MGIC) approach that tackles the quadratic growth of the number of parameters with respect to the number of channels in standard convolutional neural networks (CNNs).

Original: Indeed, lightweight
Revised: Indeed, lightweight

Original: CNNs can achieve comparable accuracy to standard CNNs with fewer parameters; however, the number of weights still scales quadratically with the CNN’s width.
Revised: CNNs can achieve comparable accuracy to standard CNNs with fewer parameters; however, the number of weights still scales quadratically with the CNN’s width.

Original: Our MGIC architectures replace each CNN block with an MGIC counterpart that utilizes a hierarchy of nested grouped convolutions of small group size to address this.
Revised: Our MGIC architectures replace each CNN block with an MGIC counterpart that utilizes a hierarchy of nested grouped convolutions of small group size to address this.

Original: Hence, our proposed architectures scale linearly with respect to the network’s width while retaining full coupling of the channels as in standard CNNs.
Revised: Hence, our proposed architectures scale linearly with respect to the network’s width while retaining full coupling of the channels as in standard CNNs.

Original: Our extensive experiments on image classiﬁcation, segmentation, and point cloud classiﬁcation show that applying this strategy to different architectures reduces the number of parameters while obtaining similar or better accuracy.
Revised: Our extensive experiments on image classiﬁcation, segmentation, and point cloud classiﬁcation show that applying this strategy to different architectures reduces the number of parameters while obtaining similar or better accuracy.

Original:  Introduction
Revised:  Introduction

Original: Convolutional neural networks (CNNs) [23] have achieved impressive accuracy for many imaging tasks [22, 11] and numerical PDE solvers [20, 2].
Revised: Convolutional neural networks (CNNs) [23] have achieved impressive accuracy for many imaging tasks [22, 11] and numerical PDE solvers [20, 2].

Original: The main idea behind CNNs is to deﬁne the linear operators in the neural network as convolutions with local kernels.
Revised: The main idea behind CNNs is to deﬁne the linear operators in the neural network as convolutions with local kernels.

Original: This increases the network’s computational efﬁciency (compared to the original class of networks) due to the compact convolution operators and the considerable reduction in the number of weights.
Revised: This increases the network’s computational efﬁciency (compared to the original class of networks) due to the compact convolution operators and the considerable reduction in the number of weights.

Original: The general trend in the development of CNNs has been to make deeper and wider networks to achieve higher accuracy [35].
Revised: The general trend in the development of CNNs has been to make deeper and wider networks to achieve higher accuracy [35].

Original: In practical applications of CNNs, a network’s feature maps are divided into channels and the number of channels, c , can be deﬁned as the width of the layer.
Revised: In practical applications of CNNs, a network’s feature maps are divided into channels and the number of channels, c , can be deﬁned as the width of the layer.

Original: A standard CNN layer connects any input channel with any output channel.
Revised: A standard CNN layer connects any input channel with any output channel.

Original: Hence, the number of convolution kernels per layer is equal to the product of the number of input channels and output channels.
Revised: Hence, the number of convolution kernels per layer is equal to the product of the number of input channels and output channels.

Original: Assuming the number of output channels is proportional to the number of input channels, this O ( c 2 ) growth of operations and parameters causes immense computational challenges.
Revised: Assuming the number of output channels is proportional to the number of input channels, this O ( c 2 ) growth of operations and parameters causes immense computational challenges.

Original: When the number of channels is large, convolutions are the most computationally expensive part of the training and inference of CNNs.
Revised: When the number of channels is large, convolutions are the most computationally expensive part of the training and inference of CNNs.

Original: Wide architectures exacerbate this trend with hundreds or thousands of channels, which are particularly effective in classiﬁcation tasks involving a large number of classes.
Revised: Wide architectures exacerbate this trend with hundreds or thousands of channels, which are particularly effective in classiﬁcation tasks involving a large number of classes.

Original: Increasing the network’s width is advantageous in terms of accuracy and computational efﬁciency compared to deeper, narrower networks [48].
Revised: Increasing the network’s width is advantageous in terms of accuracy and computational efﬁciency compared to deeper, narrower networks [48].

Original: However, the quadratic scaling causes the number of weights to reach hundreds of millions and beyond [18], and the computational resources (power and memory) needed for training and running such CNNs surpasses the resources of common systems [4].
Revised: However, the quadratic scaling causes the number of weights to reach hundreds of millions and beyond [18], and the computational resources (power and memory) needed for training and running such CNNs surpasses the resources of common systems [4].

Original: This motivates us to follow multigrid approaches that can solve numerical PDEs in linear complexity by utilizing a hierarchy of grids.
Revised: This motivates us to follow multigrid approaches that can solve numerical PDEs in linear complexity by utilizing a hierarchy of grids.

Original: Using the same approach, we aim to design more efﬁcient network architectures with competitive performance.
Revised: Using the same approach, we aim to design more efﬁcient network architectures with competitive performance.

Original: x for j = n levels do x
Revised: x for j = n levels do x

Original: =
Revised: =

Original: R j x end x levels )
Revised: R j x end x levels )

Original: ← CNN-block ( x n levels ) ) for j = n levels − 1 : 0 do x j
Revised: ← CNN-block ( x n levels ) ) for j = n levels − 1 : 0 do x j

Original: ← x
Revised: ← x

Original: + N
Revised: + N

Original: P
Revised: P

Original: − x end return
Revised: − x end return

Original: + +
Revised: + +

Original: Restrict Restrict
Revised: Restrict Restrict

Original: FCCNNblock
Revised: FCCNNblock

Original: Skipconnection Skipconnection
Revised: Skipconnection Skipconnection

Original: CNNblock CNNblock
Revised: CNNblock CNNblock

Original: ProlongResidual
Revised: ProlongResidual

Original: ProlongResidual x l x l +
Revised: ProlongResidual x l x l +

Original:  Multigrid-in-channels CNN architectures
Revised:  Multigrid-in-channels CNN architectures

Original: Typical CNN architectures are composed of a series of blocks
Revised: Typical CNN architectures are composed of a series of blocks

Original: [Equation]
Revised: [Equation]

Original: where x l and x l +1 are the input and output features of the l -th block, respectively.
Revised: where x l and x l +1 are the input and output features of the l -th block, respectively.

Original: Each CNNblock usually contains a sequence of basic layers, with associated weights which are omitted in the following.
Revised: Each CNNblock usually contains a sequence of basic layers, with associated weights which are omitted in the following.

Original: A convolution layer takes c in channels of feature maps, and outputs c out such channels, meaning O ( c in · c out ) parameters and FLOPs.
Revised: A convolution layer takes c in channels of feature maps, and outputs c out such channels, meaning O ( c in · c out ) parameters and FLOPs.

Original: We propose to replace the CNN-block in (1) by a novel multigrid block to obtain the forward propagation
Revised: We propose to replace the CNN-block in (1) by a novel multigrid block to obtain the forward propagation

Original: [Equation]
Revised: [Equation]

Original: which, as illustrated in Fig.
Revised: which, as illustrated in Fig.

Original: 1, uses a hierarchy of grids in the channel space and applies the original
Revised: 1, uses a hierarchy of grids in the channel space and applies the original

Original: CNN-block on the coarsest level.
Revised: CNN-block on the coarsest level.

Original: The parameter s g deﬁnes the group size of the convolution operators in these CNN-blocks, and s c is the size of the coarsest grid.
Revised: The parameter s g deﬁnes the group size of the convolution operators in these CNN-blocks, and s c is the size of the coarsest grid.

Original: As we show in Sec.
Revised: As we show in Sec.

Original: A, the number of parameters and FLOPs in the MGIC-block scale linearly with respect to the number of channels, assuming that the group size is ﬁxed.
Revised: A, the number of parameters and FLOPs in the MGIC-block scale linearly with respect to the number of channels, assuming that the group size is ﬁxed.

Original: Note that the MGIC block is agnostic to its CNN-block, and therefore can be used for various CNN architectures, including future ones.
Revised: Note that the MGIC block is agnostic to its CNN-block, and therefore can be used for various CNN architectures, including future ones.

Original: 2.1 The multigrid hierarchy
Revised: 2.1 The multigrid hierarchy

Original: We design a hierarchy of grids in the channel space (also referred to as “levels”), where the number of channels in the ﬁnest level corresponds to the original width of the network.
Revised: We design a hierarchy of grids in the channel space (also referred to as “levels”), where the number of channels in the ﬁnest level corresponds to the original width of the network.

Original: The number of channels is halved between the levels until reaching the coarsest level, where the number of channels is smaller or equal to the parameter s c .
Revised: The number of channels is halved between the levels until reaching the coarsest level, where the number of channels is smaller or equal to the parameter s c .

Original: Our multigrid architecture is accompanied by a CNN block, like a ResNet block [13] which is applied on each level.
Revised: Our multigrid architecture is accompanied by a CNN block, like a ResNet block [13] which is applied on each level.

Original: On the ﬁnest and intermediate levels, we only connect disjoint groups of channels using grouped convolutions.
Revised: On the ﬁnest and intermediate levels, we only connect disjoint groups of channels using grouped convolutions.

Original: These convolutions have
Revised: These convolutions have

Original: O ( s g · c in ) parameters, and we keep the group size s g ﬁxed throughout the network.
Revised: O ( s g · c in ) parameters, and we keep the group size s g ﬁxed throughout the network.

Original: Hence, as the network widens, the number of groups grows, and the number of parameters grows linearly.
Revised: Hence, as the network widens, the number of groups grows, and the number of parameters grows linearly.

Original: We allow interactions between all the channels on the coarsest grid, where we use the original
Revised: We allow interactions between all the channels on the coarsest grid, where we use the original

Original: CNN-block without grouping.
Revised: CNN-block without grouping.

Original: Hence, that our architecture performs more convolution layers and non-linear activations per MGIC-block, which is designed to replace a given CNN-block, yielding higher capacity and expressiveness at similar computational cost.
Revised: Hence, that our architecture performs more convolution layers and non-linear activations per MGIC-block, which is designed to replace a given CNN-block, yielding higher capacity and expressiveness at similar computational cost.

Original: 2.2 The multigrid block
Revised: 2.2 The multigrid block

Original: Assume that the CNN-block and the MGIC-block change neither the number of channels nor the spatial resolution of the images.
Revised: Assume that the CNN-block and the MGIC-block change neither the number of channels nor the spatial resolution of the images.

Original: That is, both x l and x l +1 in (2) have c in channels of the same spatial resolution.
Revised: That is, both x l and x l +1 in (2) have c in channels of the same spatial resolution.

Original: Given a CNN-block, a group size s g and a coarsest grid size s c , we deﬁne the multigrid s g
Revised: Given a CNN-block, a group size s g and a coarsest grid size s c , we deﬁne the multigrid s g

Original: MSE ( 10 − 3 )
Revised: MSE ( 10 − 3 )

Original: Params [K] 3.
Revised: Params [K] 3.

Original: [Table]
Revised: [Table]

Original: block in Alg. 1, and as an example we present a two-level hierarchy denoted by levels 0 , 1 , where x (0) =
Revised: block in Alg. 1, and as an example we present a two-level hierarchy denoted by levels 0 , 1 , where x (0) =

Original: x l are the input feature maps at the ﬁnest level (level 0 ).
Revised: x l are the input feature maps at the ﬁnest level (level 0 ).

Original: The two-level block is as follows
Revised: The two-level block is as follows

Original: [Equation]
Revised: [Equation]

Original: We ﬁrst down-sample the channel dimension of the input feature maps x (0) in Eq.
Revised: We ﬁrst down-sample the channel dimension of the input feature maps x (0) in Eq.

Original: (3) by a factor of 2, using a restriction operator R 0 .
Revised: (3) by a factor of 2, using a restriction operator R 0 .

Original: This operation creates the coarse feature maps x (1) , which have the same spatial resolution as x (0) , but half the channels.
Revised: This operation creates the coarse feature maps x (1) , which have the same spatial resolution as x (0) , but half the channels.

Original: The operator R 0 is implemented by a grouped 1 × 1 convolution; see a detailed discussion in Appendix A. Then, in Eq.
Revised: The operator R 0 is implemented by a grouped 1 × 1 convolution; see a detailed discussion in Appendix A. Then, in Eq.

Original: (4) a nongrouped CNN block is applied on the coarse feature maps x (1) .
Revised: (4) a nongrouped CNN block is applied on the coarse feature maps x (1) .

Original: Following that, in Eq.
Revised: Following that, in Eq.

Original: (5) we use a prolongation operator P 0 to up-sample the residual x (1) − R 0 x (0) from the coarse level to the ﬁne level (up-sampling in channel space) and obtain a tensor with c in channels.
Revised: (5) we use a prolongation operator P 0 to up-sample the residual x (1) − R 0 x (0) from the coarse level to the ﬁne level (up-sampling in channel space) and obtain a tensor with c in channels.

Original: Finally, in Eq.
Revised: Finally, in Eq.

Original: (6) we perform a grouped CNN block.
Revised: (6) we perform a grouped CNN block.

Original: An illustration of this architecture using three levels is presented in
Revised: An illustration of this architecture using three levels is presented in

Original: Fig.
Revised: Fig.

Original: The multilevel block is applied to reduce the channel dimension to the coarsest grid size s c .
Revised: The multilevel block is applied to reduce the channel dimension to the coarsest grid size s c .

Original:  Experiments
Revised:  Experiments

Original: In this section, we report several experiments with our MGIC approach.
Revised: In this section, we report several experiments with our MGIC approach.

Original: Additional experiments and details are given in Appendix B.
Revised: Additional experiments and details are given in Appendix B.

Original: Coarse channels representation We wish to quantify the effectiveness of our channel down and up sampling mechanism by measuring how well we can encode feature maps on the coarsest grid.
Revised: Coarse channels representation We wish to quantify the effectiveness of our channel down and up sampling mechanism by measuring how well we can encode feature maps on the coarsest grid.

Original: We sample 1 , 024 images from ImageNet, and extract their feature maps from the ﬁrst convolution layer of a pre-trained ResNet-50 with 64 channels.
Revised: We sample 1 , 024 images from ImageNet, and extract their feature maps from the ﬁrst convolution layer of a pre-trained ResNet-50 with 64 channels.

Original: Then, we encode and decode the feature maps using the restriction and prolongation operators, respectively.
Revised: Then, we encode and decode the feature maps using the restriction and prolongation operators, respectively.

Original: To study the transfer operators in isolation, we remove the CNN blocks and long skip connections in Fig.
Revised: To study the transfer operators in isolation, we remove the CNN blocks and long skip connections in Fig.

Original: 1 from the MGIC-block.
Revised: 1 from the MGIC-block.

Original: We experiment with several values of the group size parameter s g and present the mean squared error of the feature maps reconstruction in Tab. 1.
Revised: We experiment with several values of the group size parameter s g and present the mean squared error of the feature maps reconstruction in Tab. 1.

Original: The original feature maps and their reconstructions in Fig.
Revised: The original feature maps and their reconstructions in Fig.

Original: According to this experiment, our method is capable of faithfully representing the original channel space (obtaining low MSE values).
Revised: According to this experiment, our method is capable of faithfully representing the original channel space (obtaining low MSE values).

Original: Performance in function approximation We claim that if a network can faithfully approximate a function f ( (cid:126)x ) : R n → R , then its capacity, or representation power is high.
Revised: Performance in function approximation We claim that if a network can faithfully approximate a function f ( (cid:126)x ) : R n → R , then its capacity, or representation power is high.

Original: This property is important, especially in applications where we wish to model implicit function via a neural networks, such as signed-distance ﬁelds for shape reconstruction and completion [31], and solution of PDEs [30, 33, 3, 26].
Revised: This property is important, especially in applications where we wish to model implicit function via a neural networks, such as signed-distance ﬁelds for shape reconstruction and completion [31], and solution of PDEs [30, 33, 3, 26].

Original: Here, we approximate the function f
Revised: Here, we approximate the function f

Original: ( x, y ) =
Revised: ( x, y ) =

Original: cos( x ) sin(20 y ) , given ( x, y ) ∈ [0 , 1] 2 .
Revised: cos( x ) sin(20 y ) , given ( x, y ) ∈ [0 , 1] 2 .

Original: We use three networks – MobileNetV3 [16], GhostNet [12] and our MGIC.
Revised: We use three networks – MobileNetV3 [16], GhostNet [12] and our MGIC.

Original: The results are given in
Revised: The results are given in

Original: Appendix B.1 suggesting that our MGIC has higher capacity, as it yields lower MSE with a lower number of parameters.
Revised: Appendix B.1 suggesting that our MGIC has higher capacity, as it yields lower MSE with a lower number of parameters.

Original: [Table]
Revised: [Table]

Original: ImageNet classiﬁcation on a budget of FLOPs The ImageNet [9] challenge ILSVRC 2012 consists of over 1 .
Revised: ImageNet classiﬁcation on a budget of FLOPs The ImageNet [9] challenge ILSVRC 2012 consists of over 1 .

Original: 28 M training images and 50 K validation images from 1000 categories.
Revised: 28 M training images and 50 K validation images from 1000 categories.

Original: We use SGD optimizer with a mini-batch size of 256 for 100 epochs with cross-entropy loss.
Revised: We use SGD optimizer with a mini-batch size of 256 for 100 epochs with cross-entropy loss.

Original: The initial learning rate is 0 . 1 , divided by 10 every 30 epochs.
Revised: The initial learning rate is 0 . 1 , divided by 10 every 30 epochs.

Original: The weight decay is 0 . 0001 , and the momentum is 0 . 9 .
Revised: The weight decay is 0 . 0001 , and the momentum is 0 . 9 .

Original: As data augmentation, for both datasets, we resize the images to 224 × 224 and use standard random horizontal ﬂipping and crops, as in [13].
Revised: As data augmentation, for both datasets, we resize the images to 224 × 224 and use standard random horizontal ﬂipping and crops, as in [13].

Original: We follow the MobileNetV3-Large [16] architecture for its efﬁciency and high accuracy, and replace the standard MobileNetV3 block with our MGIC block, conﬁgured to s g = 64 , s c = 64 .
Revised: We follow the MobileNetV3-Large [16] architecture for its efﬁciency and high accuracy, and replace the standard MobileNetV3 block with our MGIC block, conﬁgured to s g = 64 , s c = 64 .

Original: Three scales of networks are reported with width factors of 0 . 6 , 1 .and 1 . 2 , respectively.
Revised: Three scales of networks are reported with width factors of 0 . 6 , 1 .and 1 . 2 , respectively.

Original: We ﬁnd that our method obtains higher accuracy, with a similar number of
Revised: We ﬁnd that our method obtains higher accuracy, with a similar number of

Original: FLOPs.
Revised: FLOPs.

Original: We present our results in Fig.
Revised: We present our results in Fig.

Original: 3 and Tab. 2.
Revised: 3 and Tab. 2.

Original: Point cloud classiﬁcation The previous experiments were performed on structured CNNs, i.e., on 2D images.
Revised: Point cloud classiﬁcation The previous experiments were performed on structured CNNs, i.e., on 2D images.

Original: To further validate our method’s generalization and usefulness, we incorporate it in graph convolutional networks (GCNs) to perform point cloud classiﬁcation.
Revised: To further validate our method’s generalization and usefulness, we incorporate it in graph convolutional networks (GCNs) to perform point cloud classiﬁcation.

Original: Speciﬁcally, we use a smaller version of the architecture from [39], where we alter the width of the last three classiﬁer layers from 1024 , 512 , 256 to 64 in all of them, as described in Appendix B.4.
Revised: Speciﬁcally, we use a smaller version of the architecture from [39], where we alter the width of the last three classiﬁer layers from 1024 , 512 , 256 to 64 in all of them, as described in Appendix B.4.

Original: Table 3 summarizes the results
Revised: Table 3 summarizes the results

Original: [Table]
Revised: [Table]

Original:  Conclusion
Revised:  Conclusion

Original: We present a novel multigrid-in-channels (MGIC) approach that improves the efﬁciency of convolutional neural networks (CNN) both in parameters and FLOPs, while using easy-to-implement structured grouped convolutions in the channel space.
Revised: We present a novel multigrid-in-channels (MGIC) approach that improves the efﬁciency of convolutional neural networks (CNN) both in parameters and FLOPs, while using easy-to-implement structured grouped convolutions in the channel space.

Original: Applying MGIC, we achieve full coupling through a multilevel hierarchy of the channels, at only O ( c ) cost, unlike standard convolution layers that require O ( c 2 ) .
Revised: Applying MGIC, we achieve full coupling through a multilevel hierarchy of the channels, at only O ( c ) cost, unlike standard convolution layers that require O ( c 2 ) .

Original: This property is signiﬁcant and desired both to reduce training and inference times, which also translates to a reduction in energy consumption.
Revised: This property is signiﬁcant and desired both to reduce training and inference times, which also translates to a reduction in energy consumption.

Original: We also note that MGIC is most beneﬁcial for wide networks, which are usually favored for state-of-the-art accuracy and performance.
Revised: We also note that MGIC is most beneﬁcial for wide networks, which are usually favored for state-of-the-art accuracy and performance.

Original: Our experiments for various tasks suggest that MGIC achieves comparable or superior accuracy than other recent light-weight architectures at a given budget.
Revised: Our experiments for various tasks suggest that MGIC achieves comparable or superior accuracy than other recent light-weight architectures at a given budget.

Original: Our MGIC block offers a universal approach for producing lightweight versions of networks suitable for different kinds of CNNs, GCNs, and traditional NNs, where fully-connected layers are applied.
Revised: Our MGIC block offers a universal approach for producing lightweight versions of networks suitable for different kinds of CNNs, GCNs, and traditional NNs, where fully-connected layers are applied.

Original: Furthermore, it is future-ready, meaning it can also compress future architectures when available.
Revised: Furthermore, it is future-ready, meaning it can also compress future architectures when available.

Revised: Acknowledgments

Revised: The research reported in this paper was supported by the Israel Innovation Authority through Avatar consortium, and by grant no. 2018209 from the United States - Israel Binational Science Foundation (BSF), Jerusalem, Israel.

Revised: ME is supported by Kreitman High-tech scholarship.

Original: [Table]
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [No] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [No] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 4.
Revised: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether
Revised: [N/A] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A Multigrid-in-channels CNN architectures
Revised: A Multigrid-in-channels CNN architectures

Original: In this following section, we provide details about the design and implementation of our method.
Revised: In this following section we provide details about the design and implementation of our method.

Original: The choice of transfer operators P and R
Revised: The choice of transfer operators P and R

Original: The transfer operators play an important role in multigrid methods.
Revised: The transfer operators play an important role in multigrid methods.

Original: In classical methods, the restriction
Revised: In classical methods, the restriction

Original: R maps the ﬁne-level state of the iterative solution onto the coarse grid, and the prolongation P acts in the opposite direction, interpolating the coarse solution back to the ﬁne grid.
Revised: R maps the ﬁne-level state of the iterative solution onto the coarse grid, and the prolongation P acts in the opposite direction, interpolating the coarse solution back to the ﬁne grid.

Original: Clearly, in the coarsening process we lose information, since we reduce the dimension of the problem and the state of the iterate.
Revised: Clearly, in the coarsening process we lose information, since we reduce the dimension of the problem and the state of the iterate.

Original: The key idea is to design P and R such that the coarse problem captures the subspace that is causing the ﬁne-grid process to be inefﬁcient.
Revised: The key idea is to design P and R such that the coarse problem captures the subspace that is causing the ﬁne-grid process to be inefﬁcient.

Original: This results in two complementary processes: the ﬁne-level steps (dubbed as relaxations in multigrid literature), and the coarse grid correction.
Revised: This results in two complementary processes: the ﬁne-level steps (dubbed as relaxations in multigrid literature), and the coarse grid correction.

Original: To keep the computations low, at the j -th level we choose R j to be a grouped 1 × 1 convolution that halves the number of channels of its operand.
Revised: To keep the computations low, at the j -th level we choose R j to be a grouped 1 × 1 convolution that halves the number of channels of its operand.

Original: We choose P j to have the transposed structure of
Revised: We choose P j to have the transposed structure of

Original: R j .
Revised: R j .

Original: For R j and P j we choose the same number of groups as in the CNN-block , e.g., for R 0 it will be c in s g groups.
Revised: For R j and P j we choose the same number of groups as in the CNN-block , e.g., for R 0 it will be c in s g groups.

Original: The operators R j take groups of channels of size s g and using a 1 × 1 convolution distills the information to s g 2 channels.
Revised: The operators R j take groups of channels of size s g and using a 1 × 1 convolution distills the information to s g 2 channels.

Original: Similarly, the operators P j interpolate the coarse channels back to the higher channel dimension using grouped 1 × 1 convolutions.
Revised: Similarly, the operators P j interpolate the coarse channels back to the higher channel dimension using grouped 1 × 1 convolutions.

Original: This choice for the transfer operators corresponds to aggregation-based multigrid coarsening [37], where aggregates (groups) of variables are averaged to form a coarse grid variable.
Revised: This choice for the transfer operators corresponds to aggregation-based multigrid coarsening [37], where aggregates (groups) of variables are averaged to form a coarse grid variable.

Original: In Sec. 3, we exemplify that the transfer operators preserve essential information of all channels.
Revised: In Sec. 3, we exemplify that the transfer operators preserve essential information of all channels.

Original: The weights of the transfer operators are learned as part of the optimization, and are initialized by positive weights with row-sums of 1.
Revised: The weights of the transfer operators are learned as part of the optimization, and are initialized by positive weights with row-sums of 1.

Original: The purpose of this initialization is that the feature maps will not vanish as we multiply them by consecutive restrictions R j to start the MGIC-block.
Revised: The purpose of this initialization is that the feature maps will not vanish as we multiply them by consecutive restrictions R j to start the MGIC-block.

Original: The importance of up-sampled residuals
Revised: The importance of up-sampled residuals

Original: Adding the up-sampled residual in (5) is the standard way to apply multigrid methods to solve non-linear problems [38, 45].
Revised: Adding the up-sampled residual in (5) is the standard way to apply multigrid methods to solve non-linear problems [38, 45].

Original: Here, it allows us to have a skip connection between corresponding levels of the multigrid, introducing an identity mapping in (5) as guided by [13].
Revised: Here, it allows us to have a skip connection between corresponding levels of the multigrid, introducing an identity mapping in (5) as guided by [13].

Original: To prevent exploding gradients by feature maps summation, at each level j we up-sample only a residual deﬁned by subtracting the matching feature maps before and after traversing the levels j + 1 , . .
Revised: To prevent exploding gradients by feature maps summation, at each level j we up-sample only a residual deﬁned by subtracting the matching feature maps before and after traversing the levels j + 1 , . .

Original: . , n levels .
Revised: . , n levels .

Original: By this deﬁnition, if the CNN-block has an identity mapping, then so does the whole MGIC-block in Alg. 1.
Revised: By this deﬁnition, if the CNN-block has an identity mapping, then so does the whole MGIC-block in Alg. 1.

Original: Changing the channel resolution blocks
Revised: Changing the channel resolution blocks

Original: The structure of the MGIC block as in Fig.
Revised: The structure of the MGIC block as in Fig.

Original: 1 is more natural to equal input and output channel sizes, i.e., c in = c out .
Revised: 1 is more natural to equal input and output channel sizes, i.e., c in = c out .

Original: Hence, when we wish to change the number of channels , we deﬁne a lightweight shortcut that is designed to transform a tensor from c in to c out such that our MGIC-blocks will be given an input where c in = c out .
Revised: Hence, when we wish to change the number of channels , we deﬁne a lightweight shortcut that is designed to transform a tensor from c in to c out such that our MGIC-blocks will be given an input where c in = c out .

Original: Speciﬁcally, to obtain low computational cost, we use a depth-wise 3 × 3 convolution, although other alternatives such as a 1 × 1 convolution are also possible.
Revised: Speciﬁcally, to obtain low computational cost, we use a depth-wise 3 × 3 convolution, although other alternatives such as a 1 × 1 convolution are also possible.

Original: In case we wish to change the spatial dimensions of the input tensor, we perform the same operation, only with a stride of 2.
Revised: In case we wish to change the spatial dimensions of the input tensor, we perform the same operation, only with a stride of 2.

Original: The complexity of the MGIC-block
Revised: The complexity of the MGIC-block

Original: Consider a case where we have c = c in = c out channels in the network, and we apply a standard convolution layer using d × d convolution kernels (e.g., a 3 × 3 kernel).
Revised: Consider a case where we have c = c in = c out channels in the network, and we apply a standard convolution layer using d × d convolution kernels (e.g., a 3 × 3 kernel).

Original: The output consists of c feature maps, where each one is a sum of the c input maps, each convolved with a kernel.
Revised: The output consists of c feature maps, where each one is a sum of the c input maps, each convolved with a kernel.

Original: Hence, such a convolution layer requires O ( c 2 · d 2 ) parameters, inducing a quadratic growth in the parameters and FLOPs.
Revised: Hence, such a convolution layer requires O ( c 2 · d 2 ) parameters, inducing a quadratic growth in the parameters and FLOPs.

Original: Relaxation cost per level
Revised: Relaxation cost per level

Original: On each level of an MGIC block, a relaxation step is performed.
Revised: On each level of an MGIC block, a relaxation step is performed.

Original: At the j -th level, this relaxation step is realized by a grouped convolution of kernel size d × d , with a group size of s g that divides c 2 j (since at each level we halve the channels space, starting from c channels), yielding c s g · 2 j groups.
Revised: At the j -th level, this relaxation step is realized by a grouped convolution of kernel size d × d , with a group size of s g that divides c 2 j (since at each level we halve the channels space, starting from c channels), yielding c s g · 2 j groups.

Original: Therefore, the number of parameters required for such relaxation step is s g · c · d 2 2 j .
Revised: Therefore, the number of parameters required for such relaxation step is s g · c · d 2 2 j .

Original: At the coarsest level, we have s c channels which perform a fully-coupled relaxation step, requiring s c · d 2 parameters.
Revised: At the coarsest level, we have s c channels which perform a fully-coupled relaxation step, requiring s c · d 2 parameters.

Original: The cost of restriction and prolongation
Revised: The cost of restriction and prolongation

Original: As discussed in Sec.
Revised: As discussed in Sec.

Original: A, the restriction and prolongation operators are implemented via grouped 1 ×convolutions, halvening and doubling the feature space dimension, respectively.
Revised: A, the restriction and prolongation operators are implemented via grouped 1 ×convolutions, halvening and doubling the feature space dimension, respectively.

Original: Those operators are learned at each level of our MGIC block.
Revised: Those operators are learned at each level of our MGIC block.

Original: Therefore, the number of parameters for those operators at the j -th level is c 2 j .
Revised: Therefore, the number of parameters for those operators at the j -th level is c 2 j .

Original: The analysis here is similar to the case of the relaxtion steps, only here d = 1 , and we have no fully-coupled operators on the coarsest level.
Revised: The analysis here is similar to the case of the relaxtion steps, only here d = 1 , and we have no fully-coupled operators on the coarsest level.

Original: The total cost of an MGIC block
Revised: The total cost of an MGIC block

Original: Combining the analysis from the paragraphs above, the total number of parameters for an MGIC block with n levels is as follows:
Revised: Combining the analysis from the paragraphs above, the total number of parameters for an MGIC block with n levels is as follows:

Original: [Equation]
Revised: [Equation]

Original: If s c is small (typically, we choose s c
Revised: If s c is small (typically, we choose s c

Original: = s g ) , we can neglect the term s 2 c · d 2 to obtain O
Revised: = s g ) , we can neglect the term s 2 c · d 2 to obtain O

Original: ( s g · c · ( d 2 +1)) parameters.
Revised: ( s g · c · ( d 2 +1)) parameters.

Original: Therefore, since s g and s c are ﬁxed and small, and the spatial dimension of the learned relaxation step convolution kernel size d is typically of small size (3, 5, or 7), our method scales linearly with respect to the network’s width.
Revised: Therefore, since s g and s c are ﬁxed and small, and the spatial dimension of the learned relaxation step convolution kernel size d is typically of small size (3, 5, or 7), our method scales linearly with respect to the network’s width.

Original: This will be most beneﬁcial if c is large, which is typical in and usually required in order to obtain state-of-the-art performance on various tasks as in discussed in Sec. 1.
Revised: This will be most beneﬁcial if c is large, which is typical in and usually required in order to obtain state-of-the-art performance on various tasks as in discussed in Sec. 1.

Original: Memory footprint
Revised: [Table] footprint

Original: During training, the memory footprint of MGIC is roughly twice larger than a single CNN block since all the maps in the hierarchy are saved for backpropagation.
Revised: During training, the memory footprint of MGIC is roughly twice larger than a single CNN block since all the maps in the hierarchy are saved for backpropagation.

Original: However, during inference (which is more important here), the coarser feature maps are released while going up the hierarchy.
Revised: However, during inference (which is more important here), the coarser feature maps are released while going up the hierarchy.

Original: When applying the upmost CNN block, the memory footprint is identical to a single block.Following the
Revised: When applying the upmost CNN block, the memory footprint is identical to a single block.

Original: [Table] complexity analysis above, all the feature maps { x ( l ) } require about × 2 the memory of x (0) , but the memory footprint of a CNN block can be higher than that.
Revised: Following the complexity analysis above, all the feature maps { x ( l ) } require about × 2 the memory of x (0) , but the memory footprint of a CNN block can be higher than that.

Original: For example, some MobileNets[16] involve an inverse bottleneck with × 6 expansion rendering it more than three times as expensive as the additional MGIC overhead.
Revised: For example, some MobileNets[16] involve an inverse bottleneck with × 6 expansion rendering it more than three times as expensive as the additional MGIC overhead.

Original: B Experiments
Revised: B Experiments

Original: In this section we elaborate on the experiments in the main paper, and also report on additional experiments that are carried to verify the effectiveness of our MGIC method.
Revised: In this section we elaborate on the experiments in the main paper, and also report on additional experiments that are carried to verify the effectiveness of our MGIC method.

Original: We conduct an additional proof-of-concept experiment, measuring how good our MGIC is capable of approximating implicit functions.
Revised: We conduct an additional proof-of-concept experiment, measuring how good our MGIC is capable of approximating implicit functions.

Original: Then, we test our method on image classiﬁcation and segmentation and point cloud classiﬁcation benchmarks.
Revised: Then, we test our method on image classiﬁcation and segmentation and point cloud classiﬁcation benchmarks.

Original: Our goal is to compare how different architectures perform using a relatively small number of parameters, aiming to achieve similar or better results with fewer parameters and FLOPs.
Revised: Our goal is to compare how different architectures perform using a relatively small number of parameters, aiming to achieve similar or better results with fewer parameters and FLOPs.

Original: We train all our models using an NVIDIA Titan RTX and implement our code using the PyTorch software [32].
Revised: We train all our models using an NVIDIA Titan RTX and implement our code using the PyTorch software [32].

Original: The details of the architectures that we use throughout this section are given in the supplementary material.
Revised: The details of the architectures that we use throughout this section are given in the supplementary material.

Original: B.1 Performance in function approximation
Revised: B.1 Performance in function approximation

Original: We study the capacity of our network by experimenting its efﬁcacy in function approximation in a supervised learning setup.
Revised: We study the capacity of our network by experimenting its efﬁcacy in function approximation in a supervised learning setup.

Original: We claim that if a network can faithfully approximate a function f ( (cid:126)x ) : R n → R , then its capacity, or representation power is high.
Revised: We claim that if a network can faithfully approximate a function f ( (cid:126)x ) : R n → R , then its capacity, or representation power is high.

Original: This property is important, especially in applications where we wish to model implicit function via a neural networks, such as signed-distance ﬁelds for shape reconstruction and completion [31], and solution of PDEs [30, 33, 3, 26]—these works in particular approximate functions in an unsupervised manner.
Revised: This property is important, especially in applications where we wish to model implicit function via a neural networks, such as signed-distance ﬁelds for shape reconstruction and completion [31], and solution of PDEs [30, 33, 3, 26]—these works in particular approximate functions in an unsupervised manner.

Original: Here, we wish to approximate the function f
Revised: Here, we wish to approximate the function f

Original: ( x, y ) =
Revised: ( x, y ) =

Original: cos( x ) sin(20 y ) , given ( x, y ) ∈ [0 , 1] 2 .
Revised: cos( x ) sin(20 y ) , given ( x, y ) ∈ [0 , 1] 2 .

Original: We use three networks – MobileNetV [16], GhostNet [12] and our MGIC.
Revised: We use three networks – MobileNetV [16], GhostNet [12] and our MGIC.

Original: Since the input is a vector of two scalars (treated as channels of size 1), the convolution kernels are effectively only 1 × 1 convolutions.
Revised: Since the input is a vector of two scalars (treated as channels of size 1), the convolution kernels are effectively only 1 × 1 convolutions.

Original: (Note that in any case these are the dominant operations in all the networks and are the driving force of neural network in general.)
Revised: (Note that in any case these are the dominant operations in all the networks and are the driving force of neural network in general.)

Original: Throughout all the experiments, we used the network described in Tab. 4, where CNN-block is replaced with the respective method, with α = 0 . 6 , 0 . 8 , 1 . 2 .
Revised: Throughout all the experiments, we used the network described in Tab. 4, where CNN-block is replaced with the respective method, with α = 0 . 6 , 0 . 8 , 1 . 2 .

Original: The settings of this experiments are as follows: we sample 50 , 000 points from the surface of f and train each network for 1 , 000 epochs with a batch size of 128 points, using the SGD optimizer with a constant learning rate of 0 . 0001 .
Revised: The settings of this experiments are as follows: we sample 50 , 000 points from the surface of f and train each network for 1 , 000 epochs with a batch size of 128 points, using the SGD optimizer with a constant learning rate of 0 . 0001 .

Original: The loss function is the mean squared error (MSE).
Revised: The loss function is the mean squared error (MSE).

Original: The results, summarized in Fig.
Revised: The results, summarized in Fig.

Original: 4 suggest that our
Revised: 4 suggest that our

Original: MGIC has higher capacity, as it yields lower MSE with a lower number of parameters.
Revised: MGIC has higher capacity, as it yields lower MSE with a lower number of parameters.

Original: B.2 Image classiﬁcation
Revised: B.2 Image classiﬁcation

Original: We compare our approach with a variety of popular and recent networks like ResNet-50 [13],
Revised: We compare our approach with a variety of popular and recent networks like ResNet-50 [13],

Original: MobileNetV3 [16] and GhostNet [12] for image classiﬁcation on the CIFAR10 and ImageNet datasets.
Revised: MobileNetV3 [16] and GhostNet [12] for image classiﬁcation on the CIFAR10 and ImageNet datasets.

Original: We use SGD optimizer with a mini-batch size of 256 for ImageNet, and 128 for CIFAR-10, both for 100 epochs.
Revised: We use SGD optimizer with a mini-batch size of 256 for ImageNet, and 128 for CIFAR-10, both for 100 epochs.

Original: Our loss function is cross-entropy.
Revised: Our loss function is cross-entropy.

Original: The initial learning rates for CIFARand
Revised: The initial learning rates for CIFARand

Original: ImageNet are 0 .
Revised: ImageNet are 0 .

Original: 001 and 0 .
Revised: 001 and 0 .

Original: 1 , respectively.
Revised: 1 , respectively.

Original: We divide them by 10 every 30 epochs.
Revised: We divide them by 10 every 30 epochs.

Original: The weight decay is 0 . 0001 , and the momentum is 0 .
Revised: The weight decay is 0 . 0001 , and the momentum is 0 .

Original: As data augmentation, for both datasets, we use standard random horizontal ﬂipping and crops, as in [13].
Revised: As data augmentation, for both datasets, we use standard random horizontal ﬂipping and crops, as in [13].

Original: B.2.1 CIFAR- 
Revised: B.2.1 CIFAR-

Original: The CIFAR-10 dataset [21] consists of 60K natural images of size 32 × 32 with labels assigning each image into one of ten categories.
Revised: The CIFAR-10 dataset [21] consists of 60K natural images of size 32 × 32 with labels assigning each image into one of ten categories.

Original: The data is split into 50K training and 10K test sets.
Revised: The data is split into 50K training and 10K test sets.

Original: Here, we use a
Revised: Here, we use a

Original: ResNet- 

Revised: ResNet-

Original: [13] architecture together with our MGIC block, with parameters s g = 8 , s c = 16 .
Revised: [13] architecture together with our MGIC block, with parameters s g = 8 , s c = 16 .

Original: We compare our method with other recent and popular architectures such as AMC-ResNet- 56 [14] and
Revised: We compare our method with other recent and popular architectures such as AMC-ResNet- 56 [14] and

Original: Ghost-ResNet- [-
 [Table]-] and our baseline is the original ResNet- 56 .
Revised: Ghost-ResNet- [-
 [Table]-]56 [12], and our baseline is the original ResNet- 56 .

Original: We report our results in Tab. 5, where we see large improvement over existing methods, while retaining low number of parameters and FLOPs.
Revised: We report our results in Tab. 5, where we see large improvement over existing methods, while retaining low number of parameters and FLOPs.

Original: [Table]
Revised: [Table]

Original: -50 compression
Revised: ResNet-50 compression

Original: We compress the ResNet-50 architecture, and compare our MGIC approach with other methods.
Revised: We compress the ResNet-50 architecture, and compare our MGIC approach with other methods.

Original: As the goal of this experiment is to compress a standard ResNet-50 [13], we follow the exact architecture of the latter, only replacing each ResNet block layer by an MGIC-ResNet block, denoted as MGIC ( · ) , as depicted in Tab. 6.
Revised: As the goal of this experiment is to compress a standard ResNet-50 [13], we follow the exact architecture of the latter, only replacing each ResNet block layer by an MGIC-ResNet block, denoted as MGIC ( · ) , as depicted in Tab. 6.

Original: The results are reported in Tab. 7, where we propose three variants of our
Revised: The results are reported in Tab. 7, where we propose three variants of our

Original: MGIC-ResNet-50 network, differing in the s g and s c parameters.
Revised: MGIC-ResNet-50 network, differing in the s g and s c parameters.

Original: Our results outperform the rest of the considered methods, and our network with s g = 64 , s c = 64 also outperforms ResNeXt-50 [44]( 25 . M parameters, 4 . 2 B FLOPs, 77 .
Revised: Our results outperform the rest of the considered methods, and our network with s g = 64 , s c = 64 also outperforms ResNeXt-50 [44]

Original: 8% top-1 accuracy), which is not shown in the table because the

Original: ResNeXt architecture utilizes more channels than ResNet-50 and therefore is not directly comparable.

Original: Image classiﬁcation on a budget of FLOPs

Original: In this experiment we compare our approach with recent light networks.

Original: In particular, we follow the

Original: MobileNetV3-Large [16] architecture for its efﬁciency and high accuracy, and replace the standard

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Versatile-ResNet-50 [40] 11.
Revised: Versatile-ResNet-50 [40] 11.

Original: SSS-ResNet-50 [19] - 2.
Revised: SSS-ResNet-50 [19] - 2.

Original: Ghost-ResNet-50 [12] 13.
Revised: Ghost-ResNet-50 [12] 13.

Original: MGIC-ResNet-50 ( s g = 32 , s c = 64 ) (ours) 9.
Revised: MGIC-ResNet-50 ( s g = 32 , s c = 64 ) (ours) 9.

Original: MGIC-ResNet-50 ( s g = 64 , s c = 64 ) (ours) 15.
Revised: MGIC-ResNet-50 ( s g = 64 , s c = 64 ) (ours) 15.

Original: Shift-ResNet-50 [42] 6.
Revised: Shift-ResNet-50 [42] 6.

Original: Taylor-FO-BN-ResNet-50 [29] 7.
Revised: Taylor-FO-BN-ResNet-50 [29] 7.

Original: Slimmable-ResNet-50 0.5× [46] 6.
Revised: Slimmable-ResNet-50 0.5× [46] 6.

Original: MetaPruning-ResNet-50 [25] - 1.
Revised: MetaPruning-ResNet-50 [25] - 1.

Original: Ghost-ResNet-50 (s=4) [12] 6.
Revised: Ghost-ResNet-50 (s=4) [12] 6.

Original: MGIC-ResNet-50 ( s g = 16 , s c = 64 ) (ours) 6.
Revised: MGIC-ResNet-50 ( s g = 16 , s c = 64 ) (ours) 6.

Original: [Table]
Revised: M parameters, 4 . 2 B FLOPs, 77 . 8% top-1 accuracy), which is not shown in the table because the ResNeXt architecture utilizes more channels than ResNet-50 and therefore is not directly comparable.

Revised: Image classiﬁcation on a budget of FLOPs

Revised: In this experiment we compare our approach with recent light networks.

Revised: In particular, we follow the

Revised: MobileNetV3-Large [16] architecture for its efﬁciency and high accuracy, and replace the standard

Revised: MobileNetV3 block with our MGIC block.

Revised: Our building blocks are MGIC-Bottlenecks (dubbed

Revised: MGIC-bneck).

Revised: That is, we build a MGIC version of the bottleneck from MobileNetV3.

Revised: Our MGICMobileNetV3 is given in Tab. 9.

Revised: Note, this is the × 1 . 0 version, and can be modiﬁed via the width multiplier α .

Revised: Our parameter s g controls the group size – therefore it determines the number of groups in each MGIC bottleneck.

Revised: We denote the number of output channels by c out and the number of hidden channels within a block (the dimension of the square operator K l 2 in a the block ( ?? )), also

Revised: [Table]

Original: h–swish 5.
Revised: h–swish 5.

Original: MobileNetV2 1.0× [34] 3.
Revised: MobileNetV2 1.0× [34] 3.

Original: ShufﬂeNetV2 1.5× [28] 3.
Revised: ShufﬂeNetV2 1.5× [28] 3.

Original: FE-Net 1.0×
Revised: FE-Net 1.0×

Original: FBNet-B [41] 4.
Revised: FBNet-B [41] 4.

Original: ProxylessNAS [5] 4.
Revised: ProxylessNAS [5] 4.

Original: MnasNet-A1 [36] 3.
Revised: MnasNet-A1 [36] 3.

Original: MobileNeXt 1.0× [50] 3.
Revised: MobileNeXt 1.0× [50] 3.

Original: MobileNetV3-Large [16] 1.0× 5.
Revised: MobileNetV3-Large [16] 1.0× 5.

Original: GhostNet 1.3× [12] 7.
Revised: GhostNet 1.3× [12] 7.

Original: [-MGIC-MobileNetV
-]
Revised: [-MGIC-MobileNetV
-]MGIC-

Original: [Table]
Revised: [Table]

Original: 2×

Original: [Table]

Original: ours) 7.
Revised: ours) 7.

Original: MGIC-
Revised: MGIC-

Original: [Table]
Revised: [Table]

Original: no h [Table] swish 7. MobileNetV3 block with our MGIC block.
Revised: no h–swish 7. referred to as the expansion size, by # exp .

Original: Our building blocks are MGIC-Bottlenecks (dubbed

Original: MGIC-bneck).

Original: That is, we build a MGIC version of the bottleneck from MobileNetV3.

Original: Our MGICMobileNetV3 is given in Tab. 9.

Original: Note, this is the × 1 . 0 version, and can be modiﬁed via the width multiplier α .

Original: Our parameter s g controls the group size – therefore it determines the number of groups in each MGIC bottleneck.

Original: We denote the number of output channels by c out and the number of hidden channels within a block (the dimension of the square operator K l 2 in a the block ( ?? )), also referred to as the expansion size, by # exp .

Original: In case c out and # exp are not divisible by s g , we set s g to the closest (smaller) integer to its intended value such that it divides them.
Revised: In case c out and # exp are not divisible by s g , we set s g to the closest (smaller) integer to its intended value such that it divides them.

Original: For example, in our experiments we set s g = 64 , and for the network deﬁned in Tab. 9, the 5th MGIC-bneck layer has # exp = 120 and c out = 40 , meaning they do not divide by 64 .
Revised: For example, in our experiments we set s g = 64 , and for the network deﬁned in Tab. 9, the 5th MGIC-bneck layer has # exp = 120 and c out = 40 , meaning they do not divide by 64 .

Original: Therefore we modify s g to be the largest integer that is smaller than 64 and divides both # exp and c out , giving s g = 40 in this example.
Revised: Therefore we modify s g to be the largest integer that is smaller than 64 and divides both # exp and c out , giving s g = 40 in this example.

Original: Our experiment is divided into three scales - small, medium, and large, where we scale our networks with width factors of 0 . 6 , 1 . 0 and 1 . 2 , respectively.
Revised: Our experiment is divided into three scales - small, medium, and large, where we scale our networks with width factors of 0 . 6 , 1 . 0 and 1 . 2 , respectively.

Original: We ﬁnd that our method obtains higher accuracy, with a similar number of FLOPs, as depicted from the results in Tab. 8 and Fig. 3.
Revised: We ﬁnd that our method obtains higher accuracy, with a similar number of FLOPs, as depicted from the results in Tab. 8 and Fig. 3.

Original: Speciﬁcally, we compare our methods with and without the use of the h–swish activation function [16], where we see similar results.
Revised: Speciﬁcally, we compare our methods with and without the use of the h–swish activation function [16], where we see similar results.

Original: Compared to other popular and recent methods like MobileNetV3,
Revised: Compared to other popular and recent methods like MobileNetV3,

Original: GhostNet and ShufﬂeNetV2, we obtain better accuracy given the same FLOPs.
Revised: GhostNet and ShufﬂeNetV2, we obtain better accuracy given the same FLOPs.

Original: Inference times.
Revised: Inference times.

Original: We measure the single thread inference times on one image using lightweight models on a Samsung Galaxy S8 mobile device (using the TFLite tool [1]), and an Intel i9-9820X
Revised: We measure the single thread inference times on one image using lightweight models on a Samsung Galaxy S8 mobile device (using the TFLite tool [1]), and an Intel i9-9820X

Original: CPU—see Tab. 10 (averaged over 50 inferences).
Revised: CPU—see Tab. 10 (averaged over 50 inferences).

Original: We observe that at least by these timings, the runtime of MGIC is on par with the considered architectures while obtaining higher accuracy.[Table] [Table]
Revised: We observe that at least by these timings, the runtime of MGIC is on par with the considered architectures while obtaining higher accuracy.

Original: B.3 Image semantic segmentation
Revised: B.3 Image semantic segmentation

Original: We compare our method with MobileNetV3 on semantic segmentation on the Cityscapes [8] dataset.
Revised: We compare our method with MobileNetV3 on semantic segmentation on the Cityscapes [8] dataset.

Original: For the encoder part of the network, we build large and small variants, based on MobileNetV3-Large and MobileNetV3-Small, described in Tables 1-2 in [16] [Table] respectively.
Revised: For the encoder part of the network, we build large and small variants, based on MobileNetV3-Large and MobileNetV3-Small, described in Tables 1-2 in [16], respectively.

Original: We also utilize the same
Revised: We also utilize the same

Original: LR-ASPP segmentation head and follow the observations from [16].
Revised: LR-ASPP segmentation head and follow the observations from [16].

Original: Namely, we reduce the number of channels in the last block of our networks by a factor of two and use 128 ﬁlters in the segmentation  head.
Revised: Namely, we reduce the number of channels in the last block of our networks by a factor of two and use 128 ﬁlters in the segmentation [Table] [Table] head.

Original: For training, we use the same data augmentation and optimization approach as in [6].
Revised: For training, we use the same data augmentation and optimization approach as in [6].

Original: The results are shown in Tab.
Revised: The results are shown in Tab.

Original: 11, where report the mean intersection over union (mIoU) metric of our
Revised: [Table] where report the mean intersection over union (mIoU) metric of our

Original: MGIC-Large with s g = 

Revised: MGIC-Large with s g =

Original: and s g = 32 .
Revised: and s g = 32 .

Original: We note that the results for the former are slightly better than those of MobileNetV3, while the performance of the latter are more favorble as they offer similar accuracy for less FLOPs and parameters.
Revised: We note that the results for the former are slightly better than those of MobileNetV3, while the performance of the latter are more favorble as they offer similar accuracy for less FLOPs and parameters.

Original: In addition, we read similar accuracy when using our
Revised: In addition, we read similar accuracy when using our

Original: MGIC-Small with s g = 64 .
Revised: MGIC-Small with s g = 64 .

Original: [Table]

Original: B.4 Point cloud classiﬁcation
Revised: B.4 Point cloud classiﬁcation

Original: We deﬁne the architecture in Tab.
Revised: We deﬁne the architecture in Tab.

Original: 12, where G-conv denotes a graph convolution layer, according to the methods listed in Tab. 3, followed by a BatchNorm operation and a ReLU non-linear activation.
Revised: 12, where G-conv denotes a graph convolution layer, according to the methods listed in Tab. [Table] followed by a BatchNorm operation and a ReLU non-linear activation.

Original: MLP is realized by a simple 1 × 1 convolution followed by a BatchNorm operation and a ReLU non-linear activation.
Revised: MLP is realized by a simple 1 × 1 convolution followed by a BatchNorm operation and a ReLU non-linear activation.

Original: FC is a fully connected layer.
Revised: FC is a fully connected layer.

Original: In all networks, we deﬁne the adjacency matrix using the k-NN algorithm with k = 10 .Then
Revised: In all networks, we deﬁne the adjacency matrix using the k-NN algorithm with k = 10 .Then

Original: we replace the GCN block with each of the backbones  listed in Tab. 3, where we also report their performance on point-cloud classiﬁcation on ModelNet- [43] benchmark where we sample 1 , 024 points from each shape. MLP is a multi-layer perceptron.
Revised: we replace the GCN block with each of the backbones [Table] listed in Tab. 3, where we also report their performance on point-cloud classiﬁcation on ModelNet- [

Original: MaxPool is a global max-pooling layer. c out denotes the number output channels. × denotes the number repetitions layer.
Revised: ] benchmark where we sample 1 , 024 points from each shape.

Original: [Table]
Revised: [Table]

Original: 5 Ablation study
Revised: B.5 Ablation study

Original: To determine the impact of the parameters s g and s c , we experiment on CIFAR-10 for image classiﬁcation.
Revised: To determine the impact of the parameters s g and s c [Table] we experiment on CIFAR-10 for image classiﬁcation.

Original: First, we ﬁx s g to 16 and observe how the number of parameters, FLOPs, and accuracy of MGIC-ResNet- 56 change.
Revised: First, we ﬁx s g to 16 and observe how the number of parameters, FLOPs, and accuracy of MGIC-ResNet- 56 change.

Original: Secondly, we ﬁx s c to 16, while modifying s g , and examine our model’s behavior.
Revised: Secondly, we ﬁx s c to 16, while modifying s g , and examine our model’s behavior.

Original: Our conclusion from the results reported in Tab. 13, is that a growth in s g or s c yields better accuracy at the cost of more parameters and ﬂops, since an increased communication between the channels is allowed.
Revised: Our conclusion from the results reported in Tab. [Table] is that a growth in s g or s c yields better accuracy at the cost of more parameters and ﬂops, since an increased communication between the channels is allowed.

Original: However, we also note that in this experiment, an increase of s c past 16 or s g past 8 does not yield signiﬁcant accuracy improvement.
Revised: However, we also note that in this experiment, an increase of s c past 16 or s g past 8 does not yield signiﬁcant accuracy improvement.

Original: Thus, it suggests that our method is capable of faithfully reducing the feature space given a budget of parameters or FLOPs.
Revised: Thus, it suggests that our method is capable of faithfully reducing the feature space given a budget of parameters or FLOPs

Original: [Table]
Revised: [Table]

Revised: [Table]