A CLOSER LOOK AT NETWORK RESOLUTION FOR EFFICIENT NETWORK DESIGN
Anonymous authors
Paper under double-blind review
A BSTRACT
There is growing interest in designing lightweight neural networks for mobile and embedded vision applications.
Previous works typically reduce computations from the structure level.
For example, group convolution based methods reduce computations by factorizing a vanilla convolution into depth-wise and point-wise convolutions.
Pruning based methods prune redundant connections in the network structure.
In this paper, we explore the importance of network input for achieving optimal accuracy-efﬁciency trade-off.
Reducing input scale is a simple yet effective way to reduce computational cost.
It does not require careful network module design, speciﬁc hardware optimization and network retraining after pruning.
Moreover, different input scales contain different representations to learn.
We propose a framework to mutually learn from different input resolutions and network widths 1 .
With the shared knowledge, our framework is able to ﬁnd better width-resolution balance and capture multi-scale representations.
It achieves consistently better ImageNet top-1 accuracy over US-Net (Yu & Huang, 2019) under different computation constraints, and outperforms the best compound scale model of EfﬁcientNet (Tan & Le, 2019) by 1.5%.
The superiority of our framework is also validated on COCO object detection and instance segmentation as well as transfer learning.
I NTRODUCTION
Deep neural networks have triumphed over various perception tasks such as image classiﬁcation (He et al., 2016;
Huang et al., 2017b; Simonyan & Zisserman, 2014), object detection (Ren et al., 2015; Redmon et al., 2016) and semantic segmentation (Chen et al., 2017).
However, deep networks usually require large computational resources, making them hard to deploy on mobile devices and embedded systems.
This motivates research in reducing the redundancy in deep neural networks or designing light-weight structures.
Speciﬁcally, MobileNet (Howard et al., 2017) factorizes a standard 3 × 3 convolution into a 3 × 3 depth-wise convolution and a 1 × 1 point-wise convolution.
ShufﬂeNet (Zhang et al., 2018a) uses 1 × 1 group convolution to further reduce computations and proposes the shufﬂe operation to help information ﬂow among different groups.
Another kind of approach is to prune redundant connections in the networks.
Unstructured pruning methods (Han et al., 2015b;a) delete network connections which are thought unimportant.
Structured pruning methods (Li et al., 2016; Anwar et al., 2017; Lemaire et al., 2018) prune the entire ﬁlters and feature maps.
Recently, dynamic networks have been introduced which adopt a single model to meet varying computing resource constraints.
For example, Huang et al.
(2017a) proposes a multi-branch structure where early prediction can be made based on the current conﬁdence and resource constraints.
Kim et al.
(2018) builds a network- in-network structure for multiple resources.
Yu et al.
(2019) shares weights among different subnetworks and each sub-network has its own batch normalization layer.
US-Net (Yu & Huang, 2019) proposes to compute batch normalization statistics after training, and introduces two training techniques to train a network that is executable at any network widths.
However, these works approach the problem only from network structure perspective, ignoring the importance of network input.
Reducing network input dimension (e.g., lowering image resolution) is a straightforward way to reduce computational cost.
It can be applied to any network structure during testing.
Besides, balancing between input resolution and network width can achieve better accuracy-efﬁciency tradeoffs.
For example, to meet a dynamic resource constraint from 13 MFLOPs to 569 MFLOPs on MobileNet v1 backbone, US-Net (Yu & Huang, 2019) needs a network width range of [ 0 .
× , 1 . 0 × ] given a 224 × 224 input resolution.
This constraint can also be met via a network width of [ 0 .
25 × , 1 .
× ] by adjusting the input resolution from { 224, 192, 160, 128 } during test time.
We denote the latter model as US-Net+.
As shown in Figure 1, simply combining different resolutions with network widths during testing can already achieve a better accuracy-efﬁciency tradeoff than US-Net without additional efforts.
EfﬁcientNet (Tan & Le, 2019) also acknowledges the importance of balancing between depth, width and resolution.
Moreover, reducing input resolution does not necessarily harm the performance, and may sometimes even be beneﬁcial.
Chin et al.
(2019) points out that lower image resolution may produce better detection accuracy by reducing focus on redundant details.
Chen et al.
(2019b) claims that different scaled images contain different information.
Sun et al.
(2019) fuses multi-scale features with a multi-branch framework to learn robust representations.
Inspired by the observations above, we propose a uniﬁed framework to mutually learn from input resolution and network width .
Our framework is able to achieve the optimal width-resolution balance under certain resource constraint.
Since we share weights among different network widths, each network is able to capture multi-scale representations without any adjustments in the network structure.
The whole framework is shown in Figure 2.
We summarize our contributions as follows.
We highlight the importance of input resolution for efﬁcient network design.
Previous works either ignore it or treat it independently from network structure.
In contrast, we propose a uniﬁed framework to mutually learn from input resolution and network width.
Our framework is simple and general.
It is compatible with any network structures (i
e., networkagnostic) and is as simple as training an independent network.
Moreover, it does not change network structure, meaning it can beneﬁt from other add-on structural optimizations.
We conduct extensive experiments to verify the effectiveness of our framework.
Our framework signiﬁcantly outperforms US-MobileNets (Yu & Huang, 2019) and independently-trained MobileNets on ImageNet classiﬁcation, as well as COCO object detection and instance segmentation.
The experimental results on popular transfer learning datasets also demonstrate the generalization ability of the learned representations of our model
R ETHINKING R EDUCING C OMPUTATIONAL C OST
In this section, we explore the importance of input resolution for efﬁcient network design.
The computational cost of a vanilla convolution is
[Equation]
where C 1 and C 2 are the number of input and output channels, K is kernel size, H and W are output feature map size.
Most previous works only focus on reducing computational cost from the structure level, that is, reducing the number of channels C 1 × C 2 or reducing kernel size K .
MobileNets (Sandler et al., 2018; Howard et al., 2017) decompose the vanilla convolution into a depth-wise convolution and a 1 × 1 convolution, reducing the cost to ( C 1 × K × K + C 1 × C 2 ) ×
H × W .
ShufﬂeNet (Zhang et al., 2018a) further divides 1 × 1 convolution into several groups, shrinking the computational cost to ( C 1 × K × K + C 1 × ( C 2 /g ))
× H × W , where g is the number of groups.
However, increasing the number of groups leads to high memory access cost (MAC) (Ma et al., 2018), which makes the network inefﬁcient in practical applications.
Pruning based methods aim to prune redundant connections in the network, cutting the computational cost to β 1 × C 1 × β 2 × C
× K × K × H × W , where β 1 and β 2 are pruning ratios.
Unstructured pruning methods (Han et al., 2015b;a) remove the network connections which are thought unimportant.
The outcome is a sparsely connected network, which is not efﬁcient on standard libraries and needs
[Figure]
specialized software and hardware optimization (Han et al., 2016).
Structured pruning methods (Li et al., 2016; Anwar et al., 2017; Lemaire et al., 2018) prune the entire ﬁlters and feature maps.
The pruned network maintains structured but needs retraining to retain performance after pruning, which makes it hard to meet the dynamic resource constraints in real-world applications.
US-Net (Yu &
Huang, 2019) addresses the dynamic constraint problem by sharing weights among different subnetworks and optimizing them simultaneously.
But it only considers reducing network width and the performance drops dramatically as computational resource goes down.
Instead of only focusing on C 1 × C 2 , we shift our attention to reducing H × W in Eq.
1, i.e., lowering input resolution for the following three reasons.
First, reducing H and W can reduce the computational cost without making any adjustments to the network structure.
Therefore, it does not require further hardware optimization (Han et al., 2016) or careful structure tuning (Ma et al., 2018).
Second, balancing between input resolution and network width will produce better accuracy-efﬁciency tradeoffs as shown in Figure 1.
Third, different resolutions contain different information.
Lower resolution images may contain more global structures while higher resolution ones may encapsulate more ﬁne-grained patterns.
Several works (Chen et al., 2019b; Sun et al., 2019; Ke et al., 2017) have explored such multi-scale representation learning.
However, they resort to a multi-path structure, which is unfriendly to parallelization (Ma et al., 2018).
Motivated by the discussion above, we propose a uniﬁed framework to mutually learn from network width and input resolution.
Our framework is able to ﬁnd the optimal width-resolution conﬁguration under certain constraint, and captures multi-scale representations by sharing weights.
In addition, it is compatible with any network structures and does not need further optimization and retraining.
M ETHODOLOGY 3. P RELIMINARY
Our framework leverages the training techniques in US-Net (Yu & Huang, 2019).
Therefore, we ﬁrst introduce these techniques in this section to make this paper self-contained.
Sandwich Rule.
US-Net trains a network that is executable at any resource constraint.
The solution is to randomly sample several network widths for training and accumulate their gradients for optimization.
However, the performance of all the sub-networks is bounded by the smallest width (e.g., 0 . 25 × ) and the largest width (e.g., 1 .
× ).
Thus, the authors introduce the sandwich rule that is always sampling the smallest and largest width plus two random widths for each training iteration.
Inplace Distillation.
Knowledge distillation (Hinton et al., 2015) is an effective method to transfer knowledge from a teacher network to a student network.
Following the sandwich rule , since the largest network is sampled in each iteration, it is natural to use the largest network as the teacher to guide smaller sub-networks.
The largest network is supervised by the ground truth label.
This gives a better performance than only training with ground truth labels.
Post-Statistics of Batch Normalization.
US-Net proposes that each sub-network needs their own batch normalization (BN) statistics (mean and variance), but it is insufﬁcient to store the statistics of all the sub-networks.
As a result, US-Net collects BN statistics for the desired sub-network after training.
Experimental results show that 2,000 samples are sufﬁcient to get accurate BN statistics, so this procedure is very efﬁcient.
3. M UTUAL LEARNING FROM WIDTH AND RESOLUTION
As discussed in Section 2, different resolutions contain different information.
We want to take full advantage of this attribute to learn robust representations.
In US-Net, sandwich rule is proposed to train a network that is executable at any width.
However, we view it as a scheme of mutual learning (Zhang et al., 2018b).
Since networks with different widths share weights with each other, larger networks can take advantage of the features captured by smaller networks.
Also, smaller networks can beneﬁt from the stronger representation ability of larger networks.
In light of this, we propose to mutually learn from network widths and resolutions.
[Figure]
Training framework.
Our proposed framework is presented in Figure 2.
We train a network where its width ranges from 0 .
× to 1 .
× .
We ﬁrst follow the sandwich rule to sample four sub-networks, i.e., the smallest ( 0 . 25 × ), the largest ( 1 . 0 × ) and two random ones.
Then, unlike traditional ImageNet training with 224 × 224 input, we resize the input image to four resolutions { 224, 196, 160, 128 } and feed them into different sub-networks.
We denote the weights of a sub-network as W 0: w , where w is the width of the sub-network and 0 : w means the sub-network adopts the ﬁrst w × 100% weights of each layer of the full network.
I R = r represents a r × r input image.
Then N ( W 0: w , I R = r ) represents a sub-network with width w and input resolution r × r .
For the largest sub-network (i.e., the full-network in Figure 2), we always train it with the highest resolution ( 224 × 224 ) and ground truth label y .
The loss for the full network is
[Equation]
For the other sub-networks, we randomly pick an input resolution from { 224, 196, 160, 128 } and train it with the output of the full-network.
The loss for the i -th sub-network is
[Equation]
where KLDiv is the Kullback-Leibler divergence.
The total loss is the summation of the fullnetwork and sub-networks, i.e.,
[Equation]
The reason for training the full-network with the highest resolution is that the highest resolution contains more details.
Also, the full-network has the strongest learning ability and captures the detailed dicriminatory information from the image data.
We experiment with randomly selecting resolutions for all four widths, but it yields worse performance.
How mutual learning works.
In this part, we are going to explain why the proposed framework can mutually learn from different widths and resolutions.
For simplicity, we only consider two network widths 0 .
3 × and 0 .
8 × , and two resolutions 128 and 192 in this example.
As shown in
Figure 3, sub-network 0 .
3 × selects input resolution 128, sub-network 0 .
8 × selects input resolution 192.
Then we can deﬁne the gradients for sub-network 0 .
3 × and 0 .
8 × as grad W 0:0 . 3 , ,I R =128 and grad W 0:0 .
8 , ,I R =192 , respectively.
Since 0 . 8 × shares weights with 0 .
3 × , we can decompose its gra- dient as grad
[Equation] [Equation]
[Equation] [Equation]
=
[Equation]
= grad
[Equation] [Equation]
[Equation]
192 + grad
[Equation] [Equation]
R
[Equation]
The gradients of the two networks are accumulated during training, and the total gradients are computed as
[Equation]
Therefore, the gradient for sub-network 0 .
3 × is ( grad W
[Equation]
3 , ,I R
[Equation]
128 + grad W
[Equation]
3 , ,I R =192 ) , which enables it to capture multi-scale representations from different input resolutions.
Thanks to the random sampling of network width, every sub-network is able to learn multi-scale representations in our framework.
Model Inference.
The trained model is executable at various width-resolution conﬁgurations.
The goal is to ﬁnd the best conﬁguration under a particular resource constraint.
A simple way to achieve this is via query table.
Speciﬁcally, we sample network width from 0 .
25 × to 1 .
× with a stepsize of 0 .
05 × , and sample network resolution from { 224, 192, 160, 128 } .
We test all these widthresolution conﬁgurations on a validation set and choose the best one under a given constraint (FLOPs or latency).
Since there is no retraining, the whole process is once for all.
E XPERIMENTS
In this section, we ﬁrst present our results on ImageNet classiﬁcation (Deng et al., 2009).
We conduct extensive experiments to illustrate the effectiveness of the proposed framework.
Next, we ﬁne-tune the pre-trained model on popular transfer learning datasets to demonstrate the robustness and generalization ability of the learned representations.
We further apply our framework to COCO object detection and instance segmentation.
To the best of our knowledge, we are the ﬁrst to benchmark arbitrary-constraint dynamic networks on detection and instance segmentation.
4. I MAGE N ET CLASSIFICATION
We compare our framework with US-Net and independently-trained networks on the ImageNet dataset.
We evaluated our framework on two popular light-weight structures, MobileNet v1 (Howard et al., 2017) and MobileNet v2 (Sandler et al., 2018).
These two networks also represent non-residual and residual structures respectively.
We follow the training setting in US-Net (Yu & Huang, 2019).
Please refer to the details in Appendix A.1.
Compare with US-Net.
We ﬁrst compare our framework with US-Net on MobileNet v1 and MobileNet v2 backbones.
The Accuracy-FLOPs curve is shown in Figure 4.
We can see that our framework consistently outperforms US-Net on both MobileNet v1 and MobileNet v2 backbones.
Speciﬁcally, we achieve signiﬁcant improvement under small computational cost.
This is because our framework can ﬁnd a better balance between network width and resolution.
For example, if the resource constraint is 150 MFLOPs, US-Net reduces the width to 0 .
5 × but our framework selects a balanced conﬁguration of 0 .
7 × - 160.
Moreover, our framework even improves on the full conﬁguration ( 1 .
× - 224).
This demonstrates that our framework is able to learn multi-scale representations by our mutual learning scheme, which further boosts the performance.
Compare with US-Net plus resolution.
As evident in Figure 1, applying different resolutions to
US-Net during inference can already achieve improvements over the original US-Net.
We denote this method as US-Net+.
However, US-Net+ cannot achieve the optimal width-resolution balance due to lack of learning.
In Figure 5, we plot the Accuracy-FLOPs curves of our framework and USNet+ based on MobileNet v1 backbone, and highlight the selected input resolutions with different colors.
We can see that as we decrease the FLOPs ( 569 → 468 MFLPs), our framework ﬁrst reduces network width to meet the constraint while keeping the 224 × 224 resolution (red color line in Figure 5).
After 468 MFLOPs, our framework selects lower input resolution (192) and then continues reducing the width to meet the constraint.
On the other hand, US-Net+ cannot ﬁnd such balance.
It always slims the network width and uses the same (224) resolution as the FLOPs decreasing until it goes to really low.
This is because US-Net+ does not incorporate input resolution into the learning framework.
Simply applying different resolutions during testing cannot achieve the optimal width-resolution balance.
[Figure] [Figure]
Compare with independently trained networks.
To demonstrate that our framework is able to capture multi-scale representations for each sub-network, we compare with different width-resolution conﬁgurations.
MobileNets (Howard et al., 2017; Sandler et al., 2018) have reported the results under different conﬁgurations.
However, they consider width and resolution as independent factors, thus cannot take full advantage of the features contained in different resolutions.
We compare the performance of our framework and independently-trained MobileNets in Figure 6.
For MobileNet v1, widths are selected from { 1 .
× , 0 .
75 × , 0 .
5 × , 0 . 25 ×} , and resolutions are selected from { 224, 192, 160, 128 } , leading to 16 conﬁgurations in total.
Similarly, MobileNet v2 selects conﬁgurations from { 1 .
× , 0 .
75 × , 0 .
5 × , 0 .
35 ×} and { 224, 192, 160, 128 } .
Our framework is executable at any FLOPs between the lower bound and upper bound.
From Figure 6, our framework consistently outperforms MobileNets.
Even at the same width-resolution conﬁgurations, we can achieve much better performance.
This demonstrates that our framework not only ﬁnds the optimal widthresolution balance but also learns stronger representations by our mutual learning scheme.
[Table]
(Tan & Le, 2019) d = 1 .
[Equation]
.2.3B 75.6%
[Table]
w = 1 .
[Equation]
.2.3B 77.1%
[Figure]
To further validate the effectiveness of our mutual learning scheme, we compare our results with the state-of-the-art EfﬁcientNet (Tan & Le, 2019).
EfﬁcientNet acknowledges the importance of balancing between network width, depth and resolution.
But the authors also treat them as independent factors.
They propose to grid search over these three dimensions to ﬁnd the optimal conﬁguration under certain constraint.
Then the network can be scaled up to other constraints following this conﬁguration.
We compare with the best conﬁguration they found for MobileNet v1 at 2.3 BFLOPs.
To meet this constraint, we train our framework with a width range of [ 1 .
× , 2 . 0 × ], and select resolutions from { 224, 256, 288, 320 } .
This makes our framework executable in the range of [0.57, 4.5] BFLOPs.
The results are compared in Table 1.
Our framework ﬁnds similar conﬁguration as
EfﬁcientNet, except that we arrive at a larger width since depth is not considered.
But our framework achieves signiﬁcantly better performance than EfﬁcientNet.
We attribute this to the mutual learning from width and resolution scheme.
4. T RANSFER LEARNING
To evaluate the representations learned by our framework, we further conduct experiments on three popular transfer learning datasets, Cifar-100 (Krizhevsky, 2009), Food-101 (Bossard et al., 2014) and MIT-Indoor67 (Quattoni & Torralba, 2009).
Cifar-100 is superordinate-level object classiﬁcation, Food-101 is ﬁne-grained classiﬁcation and MIT-Indoor67 is scene classiﬁcation.
Such large variety is suitable to evaluate the robustness of the learned representations.
We compare our framework with US-Net and independently-trained MobileNet v1.
We ﬁne-tune all the models with a batch size of 256, initial learning rate of 0.1 with cosine decay schedule and a total of 100 epochs.
Following the setting in ImageNet training, we use width range [ 0 .
25 × , 1 . 0 × ] and resolutions { 224, 192, 160, 128 } .
The results are shown in Figure 7.
Again, our framework achieves consistently better performance than US-Net and MobileNet.
This veriﬁes that our framework is able to learn well-generalized representations.
4. O BJECT DETECTION AND INSTANCE SEGMENTATION
We also apply our framework to COCO object detection and instance segmentation (Lin et al., 2014).
Our experiments are based on Mask-RCNN-FPN
(He et al., 2017; Lin et al., 2017) and
MMDetection (Chen et al., 2019a) toolbox.
We use VGG-16 (Simonyan & Zisserman, 2014) as backbone network to validate our proposed mutual learning scheme.
We ﬁrst pre-train VGG-16 on ImageNet using US-Net and our framework respectively.
The network width range is [0 .
25 × , 1 . 0 × ] and resolutions are { 224, 192, 160, 128 } .
Then we ﬁne-tune the pre-trained models on COCO.
The FPN neck and detection head are shared among different sub-networks.
For simplicity, we don’t use inplace distillation .
Rather, each sub-network is trained with the ground truth.
The other training procedures are the same as training ImageNet classiﬁcation.
Following common settings in object detection, US-Net is trained with image resolution 1000 × 600 .
Our framework randomly selects resolutions from 1000 × { 600 , 480 , 360 , 240 } .
All models are trained on COCO 2017 training set and tested on COCO 2017 validation set with different image resolutions.
The Average Precision (AP) results for object detection and instance segmentation under different computational cost are presented in Figure 8.
These results reveal that our framework signiﬁcantly outperforms US-Net under all resource constraints.
Speciﬁcally, on the full conﬁguration (1.0 × -600), US-Net achieves comparable results with independent VGG-16, while our framework performs much better.
This again validates the effectiveness of our width-resolution mutual learning scheme.
Please refer to Appendix A.3 for visual comparison results.
[Figure]
framework
C ONCLUSION AND F UTURE WORK
This paper highlights the importance of simultaneously considering both network width and input resolution for designing efﬁcient network structures.
A new framework is deﬁned capable of mutually learning from network width and input resolution.
The proposed framework is demonstrated to signiﬁcantly improve inference performance per FLOP for various datasets and tasks.
The simplicity and generality of the framework allows it to translate well to generic problem domains.
This also make logical extensions readily available by adding other network dimensions, e.g., network depth, to the framework.
The framework can also be extended to video input and 3D neural networks, where we can leverage both spatial and temporal information.
A A PPENDIX A. T RAINING DETAILS
Hyperparameters. We follow the training settings in US-Net (Yu & Huang, 2019) when training on ImageNet.
The batch size is 1024 on 8 GPUs.
Initial learning rate is 0.5 and decreases to 0 with a cosine decay schedule.
The momentum is set to 0.9 and weight decay is 0.0001.
All models are trained for 250 epochs using SGD optimizer.
Network width scales.
In Figure 4, for MobileNet v1 backbone, our method uses width scale from [0.25, 1] × and resolutions from { 224, 192, 160, 128 } while US-Net uses width scale from [0.05, 1] × . For MobileNet v2 backbone, our method uses width scale from [0.7, 1] × and resolutions from { 224, 192, 160, 128 } while US-Net uses width scale from [0.35, 1] × . In transfer learning experiments, both our method and US-Net use width scale from [0.25, 1] × . Speciﬁcally, for USNet, we adopt the ofﬁcially released model on width range [0.25, 1] × and ﬁnetune it with width range [0.25, 1] × .
In object detection and instance segmentation, we also utilize the same width range [0.25 ,1] × for our method and US-Net.
The reason for using different width scales in Figure 4 is that both our approach and US-Net aim to train an adaptive network to meet the dynamic resource constraints at test time.
We compared our approach and US-Net using the Accuracy-FLOPs curves under the same dynamic resource constraint.
For MobileNet v1 backbone (Figure 4 (a)), the dynamic FLOPs constraint is [13, 569] MFLOPs.
To meet this constraint, US-Net needs the width scale of [0.05, 1] × .
However, our approach can meet this computation constraint by balancing between width scale [0.25, 1] × and resolutions { 224, 192, 160, 128 } .
Similarly, for MobileNet v2 (Figure 4 (b)), the dynamic constraint is [59, 300] MFLOPs.
US-Net needs width scale of [0.35, 1] × , while we can meet this constraint by combining width [0.7, 1] × and resolutions { 224, 192, 160, 128 } .
Therefore, integrating resolution in our framework gives more ﬂexibility to balance network width.
This is one advantage that we illustrated in Figure 1.
In all the following experiments (US-Net+, transfer learning, object detection and instance segmentation), we use the same width scale [0.25, 1] × for both our method and US-Net.
This setting is actually in favor of US-Net, because if the original US-Net [0 . 05 , 1] is used, which meets the same resource constraints as ours, it performs worse.
A. VGG-16 I MAGE N ET PRE - TRAINING
We use VGG-16 as the backbone network for object detection and instance segmentation.
We ﬁrst pretrain VGG on ImageNet following US-Net and our framework.
The batch size is set to 512.
The initial learning rate is 0.01 and decreases to 0 with a cosine decay schedule.
Both frameworks are trained for 100 epochs.
The Accuracy-FLOPs curves of different frameworks are shown in Figure 9.
We can see that our framework still constantly outperforms US-Net on such large neural network.
This further demonstrates the effectiveness and generalization ability of our framework.
A. O BJECT DETECTION AND INSTANCE SEGMENTATION EXAMPLES
We show some visualization examples in Figure 10.
We tested two model conﬁgurations 1 .
× - 600 and 0 .
5 × -480.
The upper pairs are 1 .
× -600, the lower pairs are 0 .
5 × -480.
From Figure 10, we can see that our framework is able to detect both small-scale and large-scale objects in both conﬁgurations while US-Net fails.
This is because our mutual learning scheme enables each subnetwork to capture multi-scale representations.
A. L ARGER TABLES FOR BETTER READABILITY
In this subsection, we display a larger version of the tables in the paper for better readability.
Table 2 and Table 3 show comparison results of our method and US-Net on ImageNet.
Table 4 and Table
[Table] [Table] [Table] [Table]
[Table] [Table] [Figure] [Figure]