Dynamic Resolution Network 
Authors: Mingjian Zhu
Kai Han
Enhua Wu
Qiulin Zhang
Ying Nie
Zhenzhong Lan
Yunhe Wang
Venue: NeurIPS 2021 Poster (2021)
Keywords: Convolutional neural network
imagenet

Abstract: Deep convolutional neural networks (CNNs) are often of sophisticated design with numerous learnable parameters for the accuracy reason. To alleviate the expensive costs of deploying them on mobile devices, recent works have made huge efforts for excavating redundancy in pre-defined architectures. Nevertheless, the redundancy on the input resolution of modern CNNs has not been fully investigated, i.e., the resolution of input image is fixed. In this paper, we observe that the smallest resolution for accurately predicting the given image is different using the same neural network. To this end, we propose a novel dynamic-resolution network (DRNet) in which the input resolution is determined dynamically based on each input sample. Wherein, a resolution predictor with negligible computational costs is explored and optimized jointly with the desired network. Specifically, the predictor learns the smallest resolution that can retain and even exceed the original recognition accuracy for each image. During the inference, each input image will be resized to its predicted resolution for minimizing the overall computation burden. We then conduct extensive experiments on several benchmark networks and datasets. The results show that our DRNet can be embedded in any off-the-shelf network architecture to obtain a considerable reduction in computational complexity. For instance, DR-ResNet-50 achieves similar performance with an about 34% computation reduction, while gaining 1.4% accuracy increase with 10% computation reduction compared to the original ResNet-50 on ImageNet. Code will be available at https://gitee.com/mindspore/models/tree/master/research/cv/DRNet.

Dynamic Resolution Network
Mingjian Zhu 1,2,4,5- Kai Han Enhua Wu Qiulin Zhang', Ying Nie?, Zhenzhong Lan Yunhe Wang
ZZejjaag University. Huawei Noah's s Ark Lab.
State Key Lab of Computer Science, ISCAS & University of Chinese Academy of Sciences.
"School of Engineering, Westlake University.
Institute of Advanced Technology. Westlake Institute for Advanced Study.
University of Macau. "BUPT.
zhumingjian0zju, edu. en, {hankai, wweh]@ios.ac. ac.
lanzzhezzhong@westllake edu. yunhe wangdhuawei.
Abstract
Deep convolutional neural networks (CNNs) are often of sophisticated design with numerous learnable parameters for the accuracy reason. To alleviate the expensive costs of deploying them on mobile devices, recent works have made huge efforts for excavating redundancy in pre-defined architectures. Nevertheless, the redundancy on the input resolution of modern CNNs has not been fully investigated, i.e., the resolution of input image is fixed. In this paper, observe that the smallest reso- lution for accurately predicting the given image is different using the same neural network. To this end, we propose novel dynamic-resolutiion network (DRNett in which the input resolution determined dynamically based on each input sample. Wherein, resolution predictor with negligible computational costs explored and optimized jointly with the desired network. Specifically, the predictor learns the smallest resolution that can retain and even exceed the original recognition accuracy for each image. During the inference, each input image will be resized its predicted resolution for minimizing the overall computation burden. We then conduct extensive experiments on several benchmark networks and datasets. The results show that our DRNet can be embedded in any off-the the-shelf network architecture to obtain considerable reduction in computational complexity. For instance, Net-550 achieves similar performance with an about 34% com- putation reduction, while gaining 1.4% accuracy increase with 10% computation reduction compared to the original ResNet-50 on ImageNet. Code will be available at http://:w......ooooooooo/oee///a
1 Introduction
Deep convolutional neural networks (CNNs) have achieved remarkable success in various computer vision tasks, under the development of algorithms [6] [2] (3], computation power, and large-scale datasets [7]. However, the outstanding performance accompanied by large computational costs, which makes CNNs difficult to deploy on mobile devices. With the increasing demand for CNNs on real-world applications, it is imperative to reduce the computational cost and meanwhile maintain the performance of neural networks.
Recently, researchers have devoted much effort model compression and acceleration methods, including network pruning, low-bit quantization, knowledge distillation, and efficient model design.
Equul contribution. Corresponding author. This research has been supported by the Key R&D program of Zhejiang Province (Grant No. 002IC03139) This work was supported by NSFC (62072449, 61 63200033) (mangdong--onggkngg. Hongkong-Masee Joint Research Grant (2020B 1515130004)) and Macao FDCT (0018/2019 AKP).
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Network pruning aims to prune the unimportant filters or blocks that are insensitive to model
performance through a certain criterion [31 15 18 25]. Low-bit quantization methods represent
weights and actívation in neural networks with low-bit values [1 13] Knowledge distillation
transfers the knowledge of the teacher models to the student models to improve the performance 7 The efficient model design utilizes lightweight operations like depth-wise convolution
38 40
to construct some novel architectures successfully [8 42 5). Orthogonal to those methods that usually focus on the network weights or architectures, Guo et.al. [4] and Wang et.al. [33] study the redundacy that exists in the input images. However, the resolutions of the input images in most of existing compressed networks are still fixed. Although deep networks an uniform resolution (e.g., 224x 224 on the ImageNet), sizes and locations of objects in images radically different. Figure shows some samples that the required resolution for achieving the highest performance are different. For the given network architecture, the FLOPss (floatingnummbr operations) of the network for processing image will be significantly reduced for images with lower resolution.
often trained using
Admittedly, the input resolution is very impor- tant factor that affects the computational costs and the performance of CNNs. For the same net- work, a higher resolution usually results in larger FLOPs and higher accuracy (26.. In contrast, the model with a smaller input resolution has lower performance while the required FLOPs are also smaller. However, the shrink of input resolutions of deep networks provides other potential to alleviate the computation bur- den of CNNs. To have an explict illustration, we first test some images under different reso- lutions with a pre-trained ResNet50 shown in Figure | and count the minimum resolution required give the correct prediction for each sample. In practice, "easy" samples, such as the panda with obvious foreground, can be classified correctly in both low and high resolution, and "hard" samples, such as the damselfly whose foreground and background are tangled can only be predicted accurately in high resolution. This observation indicates that larger proportion of images in our datasets can be efficiently pro- cessed by reducing their resolutions. On the other hand, it is also compatible with the human perception system [I] i.e., some samples can be understood easily just in blurry mode while the others need to be seen in clear mode.
low
X
Resol Resolution ution
X
panda
bee
damsellly
Figure 1: The prediction results of a well-trained ResNet-50 model for samples under different res- olutions (112x 112, 168 x 168, 224x224). Some "easy" samples like the left column (panda), can be classified correctly using both low and high resolu- tions. However, some "hard" samples like the right column (damselfly), where the foreground objects are hidden or blend with the background, can only be classified correctly using the high resolution.
In this paper, we propose novel dynamic-resolution network (DRNet) which dynamically adjusts the input resolution of each sample for efficient inference. To accurately find the required minimum resolution of each image, we introduce resolution predictor which is embedded in front of the entire network. In practice, we set several different resolutions as candidates and feed the image into the resolution predictor to produce a probability distribution over candidate resolutions as the output. The network architecture of the resolution predictor is carefully designed with negligible computational complexity and trained jointly with classifier for recognition in an end-to-end fashion. By exploiting the proposed dynamic resolution network inference approach, we can excavate the reduncancy of each image from its input resolution. Thus, computational costs of casy samples with lower resolutions can be saved, and the accuracy for hard samples can also be preserved by maintaining higher resolutions. Extensive experiments the large-scale visual benchmarks and the conventional ResNet architectures demonstrate the effectiveness of our proposed method for reducing the overall computational costs with comparable network performance.
2

---

Resolution Predictor
Image Classifier
BN
panda
Adaptive GAP
Convs
Convs
BN
FC
Pr2
Gumbel
BIN
Pra
damselfly
Figure 2: Overall framework, the resolution predictor guides the resolution selection for the large classifier. 'BN': batch normalization layer; "Pr,": probability distribution over categories under resolution ri; In the inference stage, one-hot vector is predicted by the resolution predictor, in which the '1' denotes a corresponding selected resolution. The original image then resized the selected resolution and input to the large classifier with chosen BN.
Related Works
Although deep CNN models have shown excellent accuracy, they often contain millions parameters and FLOPs. Thus the model compression techniques are becoming a research hostpot for reducing the computational costs of CNNs. Here, we revisit existing works in two parts, i.e. static model compression and dynamic model compression.
We classify model compression methods that not instance-aware the static. Group-wise Convolution (GWC), Depth-wise Convolution and Point-wise Convolution are widely used for efficient model design. such as MobileNet (8) ResNeXt (36), and ShuffleNet [42. Revealing pattern redundancy among feature maps, Han [5] proposes to generate more feature maps from intrinsic ones through some cheap operations and Zhang et.al. 4I adopts relatively heavy computation extract intrinsic information while tiny hidden details are processed with some hight-weight operations. The methods above achieve model acceleration to some extent. However, they treat all input samples equally, whereas the difficulty for CNN models or humans to recognize each sample is unequal. So instance-aware model compression can be further explored.
Dynamic model compression takes the unequal difficulty of each sample into consideration. Huang 9 proposes multi-scale dense networks with multiple classifiers to allocate uneven computation across "easier" and "harder" inputs. Wu et.al. (35) introduces BlockDrop which learns to dynamically execute the necessary layers so to best reduce total computation. Veit et.al. (28) proposes ConvNet- AIG to adaptively define their network topology conditioned on the input image. Except for dynamic adjustment on model architectures, recent works pay more attention to input images. Verelst et. (2 proposes a small gating network to predict pixel-wiss wise masks determining the locations where dynamic convolutions are evaluated. Uzkent et.al. (2 proposes PatchDrop to dynamically identify when and where to use high-resolution data conditioned on the paired low-resolution images. Yang et.al. [39 dynamically utilize sub-networks of the base network process images with different resolutions. Wang et.al. [X proposes GFNet with patch proposal networks that strategically crop the minimal image regions to obtain reliable predictions. There exists many works on multiple resolutions and dynamic mechanisms. ELASTIC [30] uses different scaling policies for different instances and learns from the data how to select the best policy. Hydranets (20 chooses different branches for different inputs by gate and aggregates their outputs with combiner. RS-Net [32) utilizes private BNs, shared convolutions, and fully-connected layer and to train input images with different resolutions.
Different from dynamic adjustment on model architectures and dynamic modification on input images with reinforcement learning. we consider the whole image and propose resolution predictor to dynamically choose the performance-suficc obtain reliable prediction with end-to-end training.
and cost-efficient resolution for single model to

---

3 Approach
In this section, we first introduce the overall framework of the proposed Dynamic Resolution Network (DRNet), then describe the resolution predictor, resolution-aware BN, and optimization algorithm in detail, respectively.
3.1 Dynamic Resolution Network
Inspired by the fact that different sample requires different resolution to achieve the least accurate prediction, we propose to develop an instance-aware resolution selection approach for a single large classifier network. As shown in Figure 2 the proposed method mainly consists of two components. The first is the large classifier network with both high performance and expensive computational costs, such the classical ResNet (6 and EfficientNet (24). The other is a resolution predictor for finding the minimal resolution so that we can adjust the input resolution of each image to have a better trade-off on the accuracy and efficiency. For an arbitrary input image, we first forecast its suitable resolution r using the resolution predictor. Then, the large classifier will take the resized image as imputs and the required FLOPs will be reduced significantly when r lower than that of the origional resolution. To achieve better performance, the resolution predictor and the base network are optimized end-to-end during training.
The resolution predictor is designed as a CNN-based preprocessing op- eration before input samples are fed to the large base network. It's from the inspiration that our well-trained large models can also predict relative amount of samples correctly though they are in relatively small resolutions while large amounts of computation cost can be saved. On the one hand, the goal of the resolution predictor is to find appropriate instance-aware resolution by inferring probability distribution over candidate resolutions. Note that there are a vast number candidates from 1x1 to 224 224, which makes it difficult, also meaningless, for the resolution predictor to explore such a long-ran long-range of resolutions. As a simplification strategy and practical requirement, we choose m resolution candidates rı, r2, .... I'm shrink the exploration range. On the other hand, we have to keep the model size of the proposed resolution predictor as small as possible since will bring extra FL.OPs, otherwise, it becomes impractical implement such a module if its extra introduced computation exceeds the saved one from the low resolution. In this spirit, we design the resolution predictor with a few convolutional layers and fully-connected layers to complete resolution classification task. Then the preprocessing of the proposed resolution predictor R(-) can be given as follows:
Resolution Predictor:
Pr [Pr, PP2: Pr.. R(X),
where X is the input samples fed to the resolution predictor, is the total number of candidate resolutions, and p, E RT is the outputs of the resolution predictor which represents the probability of each candidate. Then the resolution corresponding to the highest probability entry is selected as the resolution fed to the large classifier. Since the process from the soft outputs of resolution predictor to the discrete resolution resizing operation does not support end-to-end training, here we adopt Guubbel-Softan [4] module to turn soft decisions p, into hard decisions E (0, 1 by applying Gumbel-Softmin trick solve the non-differentiabll problem:
h G(y.) GG((X)))
where the Gumbel-Softmat trick will be described in the next subsection. For validation, the resolution predictor makes decisions first then the input with the selected resolution only is fed to the large classifier in the normal way shown in the left part of Figure 2
Resolution-aware BN. Our framework is proposed to use only single large classifier for the sake of storage pressure and loading latency, which results in that the single classifier has to process multi-resolution inputs and raises two problems. One obvious problem is that the first fully-connected layer will fail work with a different input resolution and can be solved with global average pooling. Thus we can process multiple resolutions in one single network. The other hidden problem exists in Batch Normalization (BN) 12 layers. BN used to make deep models converge faster and more stable through channel-wise normalization of the input layer by re-centering and re-scaling. However, activation statistics including means and variances under different resolutions are incompatible (26). Using shared BNs under multiple resolutions leads to lower accuracy in our experiments as shown

---

in section 4.3
Since the batch normalization layer contains a negligible amount of parameters, we propose resolution aware BNs as shown in Figure 2 We decouple the BN for each resolution and choose the corresponding BN layer to normalize the features:
J 7 I3 - 14
B,,
m)
Voz²
where is small number for numerical stability, μ4 and a₁ are private averaged mean and variance from the activation statistics under separate resolutions; 3, and are private learnable scale weights and bias. Since shared convolutions are insensitive to performance, the overall adjustment for the original large classifier is shown in the right part of Figure 2
3.2 Optimization
The proposed framework is optimized to perform instance-aware resolution selection for inputs of a single large classifier with end-to-end training. The loss function and Gumbel softmax trick are described in the following.
Loss Function. The base classifier and the resolution predictor are optimized jointly. The loss function includes two parts: the cross-entropy loss for image classification and a FLOPs constraint regularization restrict the computation budget.
Given a pretrained base image classifier F which takes image X as input and outputs the probability predictions y F(X) for image classification, optimize the resolution predictor and finetune the pretrained base classifier together so as to make them compatible with each other. For the input image X, first resize it into m candidate resolutions X,, Xra." ........ X We use the proposed resolution predictor to produce the resolution probability vector pr E Ih for each image. The soft resolution probability Pr- is transformed into hard one-hot selection h {0, 1 using Gumbel-Softtax trick as equation 2 where the hot entry of h represents the resolution choice for each sample. We first obtain the final prediction of each resolution Vrj F(Xr;), and then sum them up with / to obtain the recognition prediction for the selected resolution:
(4)
The Cross-Entropp loss H is performed between and target label y as follows:
Lee
M( y)-
(5)
The gradients from the loss Liee are back-propagated to both the base classifier and the resolution
predictor for optimization.
we use the Cross- Pross-EEntropy loss only, the resolution predictor will converge to a sub-optimal point and tend to select the largest resolution because samples with the largest resolution correspond to relatively lower classification loss generally. Although the classification confidence of the low- resolution image is relatively lower, the prediction can be correct and requires fewer FLOPs. In order reduce the computational cost and balance the different resolution selection, we propose a FLOPPs constraint regularization to guide the learning of resolution predictor:
F
> (C; j=1
(Cy-hy),
(6)
(0
B(F) -
Lreg = max
(7)
Cmaz Cmin
where the actual inference FLOPS, Cj is the pre-computed FLOPs value for the j-th resolution, B(-) is the expectation value over samples, and is the target FL.OPs. Through this regularization, there will be a penalty if averaged FLOPs value is too large, enforcing the proposed resolution predictor to be instance-aware and predict the resolution that is both performance--sfffiicnn (with
correct prediction) and cost-efficient (with low resolution).
Finally, the overall loss is the weighted summation over the classification loss and the FLOPs constraint regularization term:
L = Lee + nLreg
(8)
where 7) is hyper-parameter to match the magnitude of Lee and Lreg.
5

---

Gumbel Softmax Trick. Since there exists an non con-differentiable problem in the process from the resolution predictor's continues outputs to discrete resolution selection, we adopt Gumbel Softmax trick 19 14 to make discrete decision differentiable during the pack-propagation. In Eq. [ the
resolution predictor gives the prohabilities for the resolution candidates pr- [Pr: Prz. Pr... Then the discrete candidate resolution selections can be drawn using:
one one_hot (arg max(logPr, + 9;)],
where gj is Gumbel noise obtained through two log operation applied i.i.d samples u drawn from a uniform distribution as follows:
gj - log((- log u),
U).
(10)
During training. the derivative of the one-hot operation is approximated by Gumbel softmax function which is both continuous and differentiable:
hj exp (log(T,) + 93)/T
(11)
crpp(1og (Ty)) + 93/7))*
where is the temperature parameter. The introduction of Gumbel noise has two positive effects. On the one hand, it will not influence the highest entry of the original categorical probability distribution. On the other hand, it makes the gradient approximation from discrete hardmax to continuous softmax more fluent. By this teaaighttthouuge Gumbel softmax trick, we can optimize the overall framework end-to-end.
4 Experiments
To show the effectiveness of our proposed method, in this section, we conduct experiments on the small-scale ImageNet- 100 and large-scale ImageNet-IK (2) with classic large classifier networks, including ResNet [6] and MobileNetV2 [23]. where we replace their single batch normalization layer(BN) with resolution- BNs and add the proposed resolution predictor to guide the resolution selection.
4.1 Implementation Details
Datasets. ImageNet-IK IK dataset (ImageNet ILSVRC2012) (2] widely-used benchmark to evaluate the classification performance of neural networks, which consists of 1.28M training images and 50K validation images in 1K categories. ImageNet- is subset of the ImageNet IISVRC2012. whose training set is random selected from the original training set and consists of 500 instances of 100 categories. The validation set is the corresponding 100 categories of the original validation set. The categories of ImageNet-100) 100 is provided in supplementary materials. For the license of ImageNet dataset, please refer to http: /wwww image- net. weeee..rg/..ee prgg/ddwllad
Experimental Settings. For data augmentation during training for both ImageNet- 100 and ImageNet-IK, we follow the scheme in [6] including randomly cropping patch from the input image and resizing to candidate resolutions with the bilinear interpolation followed by random horizontal flipping with probability 0.5. For data processing during validation, we first resize the input image into 256 x 256 and then crop the center 224 x 224 part. The details of the resolution predictor are provided in supplementary materials. For both datasets, we firstly employ the images different resolutions to pre-train a model without the resolution predictor. The losses of each resolution are summed up for optimization. Then we add designed predictor the model and conduct finetuning. Optimization is performed using SGD (mini-bater stochastic gradient descent) and learning rate warmup is applied for the first epochs. In the pretraining stage, the model trained with total epochs 70, Batch-size 256, weight decay 0.0001, momentum 0.9, initial learning rate 0.1 which decays a factor of 10 every 20 epochs. We adopt a similar training scheme in finetuning stage. The total epochs are 100 with the learning rate decaying a factor of 10 every 30 epochs. We adopt 1 xx learning rate to finetune the large classifier and 0. learning rate to train the resolution predictor from scratch. The framework is implemented in Pytorch 21 NVIDIA Tesla V100 GPUs.

---

4.2 ImageNet-100 Experiments
We conduct small-scale experiments ImageNet-100 to guide the resolution selection for large classifiers ResNet-50. The resolution predictor is designed as a 4-stage residual network with input resolution 128 128 where each stage contains
residual basic block, which consumes about 300 million FLOPs. For candidate resolutions of the large classifier, we choose resolutions of [224 224, 168 168, 112 112] and we denote them [224, 168, 12] for simplicity. Thus the last fully-connected layers of the resolution predictor contain three neurons. We only replace each batch normalization layer in ResNet-50 with three optional resolution-aware batch normalization layers, change the last average pooling layer to an adaptive average pooling layer, and then integrate the resolution predictor form the overall framework. Experiment results shown in Table 1
For the calculation of the average FLOPs in Table L we sum up the FLOPs of each sample under the predicted resolution, take the extra FLOPs introduced by the resolution predictor into account and finally take the average over the whole validation set. From the results Table we can see dynamic-resolution ResNet-55 obtains about 17% reduction of average FLOPs while gains 4.0% accuracy increase with the candidate resolutions [224, 168, 112]. When we tune the hyperparameters (i.e., 17 and a:) in FLOPs constraint regularization, the dynamic-resolution ResNet-50 obtains about 32% FLOPs reduction and achieve 1.8% increase in accuracy. We also extend the range of resolutions (i.e., [224, 192, 168, 1 12, 96) for fully exploration, especially the lower resolution. We our DRNet still performs better than the baseline model. Setting a larger a can even obtain 44 % FLOPs reduction with performance increase, which is shown in Table
that
Resolutions
RA-BN
FLOPs
Acc
Average FLOPs
[224]
4.1 G
78.5%
11
Acc
[224, 168, 112]
Yes
3
82.5%
0.2
2.0
2.3
80.6%
[224, 168, 112]
Yes
2.8 G
81.4%
0.2
2.5
2.8
81.4%
[224, 168, 112]
No
2.9 G
80.3%
0.2
3.0
3.3G
82.3%
[224, 192, 168, 112, 96]
Yes
3.0G
81.9%
0.2
3.5
3.5G
82.6%
0.1
3.0
3.3G
82.1%
Table I: Results of ResNet-50 ImageNet-100.. The first
0.2
3.0
3.3
82.3%
row demonstrates the results ResNet-50 backbones. The second row presents the results of the DRNet with regulariza- tion. The third row presents the DRNet trained with 0.2 and 2.5 in the FLOPs constraint regularization, and the fourth row shows the DRNet w/o resolution-aware BN.
0.5
3.0
3.1
81.9%
1
3.0
3.1
81.5%
Table 2: Influence of FLOPs Con- straint Regularization.
4.3 Ablation Study
To form the dynamic resolution network, propose two adjustments, 1) replacing each BN layer with resolution-aware BNs; 2) proposing a FLOPs balance regularizer. Here we conduct ablation studies ImageNet- 100 to investigate the influence of each part.
Resolution-Aware BN. Here we compare the results where the large classifier ResNet-50 is equipped with resolution-aware BN or not. From Table we can see that resolution-aware BNs obtain extra one more points for ResNet-50 with similar computational cost, which demonstrates that we need to normalize feature maps with different resolutions separately, thus their activation statistics can be more accurate.
Influence of FLOPs Constraint Regularization. Here we explore the influence of the penalty factor 7) and target FLOPs value a in the FLOPss constraint regularization as shown in Table We first fix 77 as 0.2 and tune from 2.0 to 3.5. We can see that the average FLOPs increase from 2.3G to 3.5G gradually, and the accuracy also increase consequently. As for y) we fix a 3.0 and tune r) in the range of 10.1, 1]. A larger penalty factor leads to lower FL.OPs and accuracy. That is to say when selecting images dynamically, the resolution predictor with lower penalty v tends to choose the larger resolution, where the effect of the balance regularizer is relatively weaker.
7

---

4.4 ImageNet-IK Experiments
ResNet Results. We conduct large-scale experiments with DR-ResNet-50 on ImageNet- IK as shown in Table 3 When we set the candidate resolutions as [224, 168, 112], our DR- DR-ResNet-50 also outperforms the baseline by 1.4 percentage with 10% FLOPs reduced. Similar to the results in 2 the effectiveness of FLOPs constraint regularization is also verified in Table where the FL.OPs drop with larger a. Our DRNet focuses on input resolution and keeps the structure of the large classifier almost unchanged, thus the parameters of our DRNet-equipped models are more than the original large classifier due to the introduction of the resolution predictor. other words, since our DRNet is orthogonal to those architecture compression methods, careful combinations of the two methods would make more compact result.
Model
Params
FLOPs
[FLOPs
Acc@
Acc@5
ResNet-50-baseline
25.6 M
4.1G
76.1%
92.9%
DR-ResNet-50
30.5 M
3.7G
10%
77.5%
93.5%
DR-ResNet-50
2.0
30.5 M
2.3G
44%
75.3%
92.2%
DR-ResNet-50
2.5
30.5 M
2.7G
34%
76.2%
92.8%
DR-ResNet-50
3,0
30.5
3,2G
22%
77.0%
93.2%
DR-ResNee-500
3.5
30.5 M
3.7
10%
77.4%
93.5%
ResNet-101-baseline
44.5 M
7.8
77.4%
93.5%
DR-ResNet-101
49.4 M
7.0
10%
79.0%
94.3%
Table ResNet-50 and ResNet- 101 results on the ImageNet-Ie dataset.
We also compare ER-ResNet-50 with other representative model compression methods to verify the superiority the proposed method. The compared methods include Sparse Structure Selection (SSS) [1O], Versatile Filters [34], PFP [16], and C-SGD [3) As shown in Table our DR-ResNet-50
achieves better performance than other methods with similar FLOPs.
Model ResNee-50-bbaselnee
Params
FLOPs
!FLOPs
Acc@
Acc@5
25.6
4.1G
76. %
92.9%
ResNet-50 (192x 192)
25.6 M
3.0
27%
74.3%
91.9%
SSS-ResNet-50 [1]]
2.8
32%
74.2%
91.9%
Versatile-ResNet-550
34
11.0 M
3.0G
27%
74.5%
91.8%
PFP-A-ResNet-50 16
20.9
3.7
10%
75.9%
92.8%
C-SGD77--ResNe--50
2.6
37%
75.3%
92.5%
RANet 30
2.3 G
44%
74.0%
DR-ResNet-50
30.5
3.7 G
10%
77.5%
93.5%
5e-ResNet-50 (a 2.0)
30.5
2.3 G
44%
75.3%
92.2%
Table 4: Comparison with other model compression methods on the ImageNet- IK dataset.
Effect of Dynamic Resolution. To evaluate the effect the proposed dynamic resolution mecha- nism, compare DR-ResNet-50 with randomly selected resolution. We repeat the random selection 3 times and report their accuracies and FLOPs in Imagenet- IK dataset. From the results in Table DRNet shows much better performance than random baseline, indicating the effectiveness of dynamic resolution.
On-device Acceleration of Dynamic Resolution. Figure we demonstrate the practical ac- celerations of our DR-ResNete Reseeeettt-,, which are obtained by measuring the forward time on Intel(R) Xeon(R) Gold 6151 CPU. We directly set the batch size as 1 and the input resolution for the resolution predictor is 128. The candidate resolutions are 224, 168, and 1 112. We average the test time in ImageNet- 1K val set. Our model substantially outperforms ResNet-50 by significant margin.
Nob Results. We also test our method on representative lightweight neural network, i.e. MobMlleeeee22 (23]. We set the candidate resolution as [224, 168, 112]. The training setting of MobileNetV2 follows that in the original paper (23) for fair comparison. To reduce the FLOPs

---

Model
FLOPs
Ace (%)
Random-l
G
74.70
Random-2
2.6G
74.65
Random-3
2.6
74,60
Random (mean)
2.6 G
74.65+00.4
74
Paafle
DRNet
2.7 G
76.2
000
100
110
120
150
140
Table 5: Dynamic resolution Random resolution.
Figure 3: Acc vs. Latency.
of the resolution predictor, we replace the residual block with an inverted residual block and set the input size as 64 64. From the results in Table O we can see that DRNet achieves 72.7% top- accuracy with fewer computational costs.
Model
Params
FLOPs
!FLOPs
Acc@1
MobileNertV--baslline
3.5
M
71.8%
MobileNetV2 (192x 192)
3.5 M
221 M
26%
70.7%
MobileNetV2-0.75x
2.6 M
209 M
30%
69.8%
DR- MobbllNeeee2222
3.8
268 M
10%
72.7%
Table MobileNetV2 results on the ImageNet-Il K dataset.
4.5 Visualization
The prediction results of the resolution predictor are visualized in Figure 4 The first four samples with obvious foreground which occupy most of the whole image predicted to 112 112 resolution high confidence. The middle three whose foreground is little blurred are predicted to select 168 x 168 resolution. The last three samples' hidden foregrounds nearly blend with the background, thus the largest resolution are selected. Although the "easy" and "hard" examples may be different for humans and machines, these results compatible with the human perception system.
TAGVAS
99.4% ball
96. 46.7%%/miniature schnauzer
96.7%
98.1% ouzel
168
06.7%/corn
70/77%/tteetttt
99.5 99.5%/cooccl
30.3%%/apeeed
224
99.8%//oola
99.8%/urambinc
74.4% /lastic bag
94.9% 94.99%/brrmbling
Figure 4: Image visualization results of DR-ResNet-50, Each row denotes the selected resolutions for these images. Image classification confidences and labels shown below the images.

---

5 Conclusion
In this paper, we reveal that different sample acquires different resolution threshold achieve the least accurate prediction. Thus large amounts of computation cost can be saved for some easier samples under lower resolutions. To make CNNs predict efficiently, we propose novel dynamic resolution network to dynamically choose the performance-suffee sufficient and cost-efficient resolution for each input sample. Then the input is resized to the predicted resolution and fed to the original large classifier, in which we replace each BN layer with resolution-awant BNs to accommodate the multi-resolution input. The proposed method is decoupled with the network architecture and can be generalized to any network. Extensive experiments on various networks demonstrate the effectiveness of DRNet.
References
[1] D.B. Walther, B. Chaii,, E.Caddigan, D.M. Beck, and L.Fei-Fei. Simple line drawings suffice for functional mri decoding of natural scene categories, In PNAS, 2011.
[2] Deng, Wci Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. Imagenet: largo-scale hicrarchical image database. In CVPR, pages 248-255 Icee, 2009.
[3] Xiaohan Ding, Guiguang Ding. Yuchen Guo, and Jungong Han. Centripetal SGD for pruning very deep convolutional networks with complicated structure. In CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, 2019.
[4] Jinyang Guo, Wanli Ouyang, and Dong Xu. Multi-dimensional pruning: A unified framework for model compression. In CVPR 2020, 2020.
[5] Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. Ghostnet: More features from cheap operations. In CVPR, 2020.
[6] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 07007777722 2016.
[7] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv: 550.02333,,, 2015.
[8] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv: (74.006604.,4 2017.
[9] Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, and Kilian Q Weinberger. Multi-scale dense networks for resource efficient image classification. 2018.
[10] Zehao Huang and Naiyan Wang. Data-driven sparse structure selection for deep neural networks. In ECCV, pages 304-324 104-320,10 2018
[11] Itay Hubara, Matthicu Courhariaux, Daniel Soudry, Ran EI- Yani,,, and Yoshua Bengio. Binarized neural networks. In NeurlPS, pages 4107-4115, 2016.
[12] Sergey Toffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
[13] Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam. and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer- arithmetic-only inference. In CVPR, pages 2704-2713, 2018.
[14] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-sofimax. In ICLR 2017, 2017.
[15] Hao Li, Asim Kaday, Igor Durdanovie, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. In ICLR, 2017.
[16] Lucas Liebenwein, Cenk Baykal, Harry Lang, Dan Feldman, and Daniela Rus. Provable filter pruning for efficient neural networks. In ICLR 2020, OpenReview.net, 2020.
[17] Tsung Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV. Springer, 2014.
10

---

[18] Jian-Haa Luo, Jianxin Wu, and Weiyao Lin. Thinet: A filter level pruning method for deep neural network compression. In ICCV, pages 5058-5066, 2017.
[19] Chris J. Maddison, Daniel Tarlow, and Tom Minka. A sampling. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Conada, pages 3086-3094, 2014.
[20] Ravi Teja Mullapudi, William R Mark, Noam Shazcer, and Kayvon Fatahalian. Hydranets: Specialized dynamic architectures for efficient inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8080-8089, 2018.
[21] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, perh-peerfrmance deep learning library. NewrIPS, 2019.
[22] Shaoqing Ren, Kaiming Hc, Ross Girshick, and Jian Sun. Faster R-CNN; Towards real-time object detection with region proposal networks. In NeurIPS, 2015.
[23] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chiie Chich Chen, Mobilenetv2: Inverted residuals and linear bottlenecks. In CVPR, pages 45 1520-4500, 2018.
[24] Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In ICML, 2019.
[25] Yehui Tang, Yunhe Wang. Yixing Xu. Yiping Deng, Chao Xu. Dacheng Tao, and Chang Xu. Manifold regularized dynamic network pruning. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 5018-5028, 2021.
[26] Hugo Touvron, Andrea Vedaldi, Matthijs Douze, and Hervé Jégou. Fixing the train-test resolution discrepancy. Advances Neural Information Processing Systems (NeurIPS), 2019.
[27] Burak Uzkent and Stefano Ermon. Learning when and where to zoom with deep reinforcement learning. In CVPR, pages 12345-12354, 2020.
[28] Andreas Veit and Serge Belongic. Convolutional networks with adaptive inference graphs. 2018.
[29] Thomas Verelst and Tinne Tuytelaars. Dynamic convolutions: Exploiting spatial sparsity for faster inference. In CVPR, pages 2320-2329, 2020.
[30] Huiyu Wang, Aniruddha Kembhavi, Ali Farhadi, Alan L Yuille, and Mohammad Rastegari. Elastic: Improving cans with dynamic scaling policies. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2258-2267, 2019.
[31] W. Wang, M. Chen, S. Zhao, L. Chen, J. Hu, H. Liu. D. Cai, X. He, and W. Liu. Accelerate cnns from three dimensions: comprehensive pruning framework. 2020.
[32] Yikai Wang, Fuchun Sun, Duo and Anbang Yao. Resolution switchable networks for runtime efficient image recognition. In European Conference on Computer Vision, pages 533-549 549. Springer. 2020.
[33] Yulin Wang, Kangehen Lx, Rui Huang, Shiji Song, Le Yang, and Gao Huang. Glance and focus: a dynamic approach to reducing spatial redundancy in image classification. In Advances in Neural Information Processing Systems (NeurIPS). 2020.
[34] Yunhe Wang, Chang Xu, Chunjing XU, Chao Xu, and Dacheng Tao. Learning versatile filters for efficient convolutional neural networks. In NeurIPS, 2018.
[35] Zuxuan Wu, Tushar Nagarajan, Abhishek Kumar, Steven Rennic, Larry Davis, Kristen Grauman, and Rogerio Feris, Blockdrop: Dynamic inference paths in residual networks. CVPR, 2018.
[36] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transforma- tions for deep neural networks. In CVPR, pages 1492-1500, 2017.
[37] Yixing Xu, Yunhe Wang. Kai Han, Yehui Tang, Shangling Jui, Chunjing Xu, and Chang Xu. Renas: Relativistic evaluation neural architecture search. In Proceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recognition, pages 441 1 1-4420. 2021.
[38] Yixing Xu, Chang Xu, Xinghao Chen. Wei Zhang, Chunjing Xu, and Yunhe Wang. Kernel based progressive distillation for adder neural networks. arXiv preprint arXiv:2009. 13044, 2020.
11

---

[39] Le Yang, Yizeng Han, Xi Chen, Shiji Song, Jifeng Dai, and Gao Huang, Resolution adaptive networks for efficient inference. In CVPR, 2020.
[40] Yiding Yang, Jiayan Qiu, Mingli Song. Dacheng Tao, and Xinchao Wang. Distilling knowledge from graph convolutional networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 7074-7083, 2020.
[41] Qiulin Zhang, Zhuqing Jiang. Qishuo Lu, Jia' nan Han, Zhengxin Zeng. Shanghua Gao, and Aidong Men. Split to slim: An overlooked redundancy vanilla convolution. IJCAI-20, pages 3195-33 20001, 2020.
[42] Xiangyu Zhang. Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices. CVPR. 2018
12


Reviews:
[{"review_id": "kY7KRFarv2", "reviewer": "Reviewer_3WL1", "summary": "This submission proposes a dynamic resolution network that adaptively selects the suitable resolution for image recognition. A tiny resolution predictor is proposed to predict a smaller resolution for the input image of the large classifier. Thus the computational costs can be saved. The experimental results are good and the novelty of this paper is sufficient.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Generally, this paper proposes an intuitive and novel method, which reduces the FLOPs and improves the accuracy. The idea of reducing the resolution based on the input image is clever and convincing. The experimental results are impressive. The DRNet could yield a 1.4% accuracy increase while reducing 10% computational costs on ImageNet benchmark. Meanwhile, the author proposes a FLOPs constraint regularization, which makes the DRNet variant reduce 34% computational costs while maintaining similar accuracy. The paper is well written. Basically, I think this submission is above the standard of acceptance in NeurIPS2021.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1. In Figure 3, the author shows the real speed improvement. The details of evaluating inference speed should be described, especially the batch size. I think that the dynamic mechanism may not reduce latency in batched inference.\n2. There are many existing works on multiple resolutions and dynamic mechanisms. For example:\n[1] ELASTIC: Improving CNNs with Dynamic Scaling Policies. In CVPR, 2019.\n[2] Hydranets: Specialized dynamic architectures for efficient inference. In CVPR, 2018.\n[3] Resolution Switchable Networks for Runtime Efficient Image Recognition. In ECCV, 2020.\nThese works have not been cited in this submission.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "Three hours."}, "extra_text": {}}, {"review_id": "NCCMfUVHNBA", "reviewer": "Reviewer_WvyH", "summary": "The paper presents a method to reduce FLOP requirements by resizing images dynamically using a resolution prediction network, prepended to an image classifier network.  The resolution is selected from one of a handful of candidates, and resized before being passed to the classifier.  A training-time loss encourages smaller resolutions to be selected, effecting an accuracy vs resolution tradeoff controlled by two parameters.  Since selected resolution for inference is a one-hot categorical, the method uses Gumbel softmax to enable backpropagation during training.  A resolution-aware batchnorm layer is also described, which tracks batchnorm stats separately for each resolution.  The system performs very well, with over 40% reduction in average operations resulting in just one or two percent accuracy loss on imagenet, and small accuracy gains with 10% reduction in computation.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This method shows good results, and the approach of varying resolution to smallest required using a small convnet predictor is a sound approach.  The resolution-aware batchnorm is also a nice addition.  I had trouble understanding how Gumbel softmax is used: whether one-hot or mixtures are used in the training-time forward pass, and if they are one-hot, how gradients are estimated.  The description seems to indicate one-hot is used even at training time, but the referenced paper [15] describes a continuous mixture which is annealed down to one-hot over the course of training.  Currently, I don't understand how one resolution is compared to another, either within a training sample e.g. using a mixture or multiple distribution choices, or between samples e.g. using a reward or score estimator.  I think a clearer description of the mechanism that compares resolutions at training is needed.\n\nThe rest of the implementation and experiments are explained better, including the pretraining and finetuning phases, method comparisons, evaluation of parameters eta and alpha, separate batchnorms, and candidate resolutions.  It's interesting that the method performs slightly better than baseline with slightly lower FLOP count; it appears the multi-resolution training creates an effect similar to scaling data augmentation, though slightly different because images are not rescaled back to fixed size.\n\nOverall this is a nice idea with good results.  But I think the explanation of how the forward pass sampling and backprop gradients are estimated should be clearer.\n\n\n\n\nDetailed comments and questions:\n\n\n- better describe Gumbel-Softmax use.  In [15], z is approximated with a continuous sample y, which is the Gumbel-Softmax, e.g. fig 2 in [15] shows y is continuous.  In this case, that would make h no longer one-hot --- how is this handled?\n\n- [15] also use a temperature annealing schedule for tau to move from continuous to near-one-hot.  Is there an annealing schedule for tau here?\n\n\n- if h is one-hot during training, how are the gradients estimated?\n\n- X^\\hat = sum_j X_j :  Since the X_j are different shapes for different resolutions, what does the sum mean here?\n\n\n\n- what is the architecture of the resolution predictor R?  in particular, can the few conv layers used have relatively large stride? (or does increasing stride allow a resolution predictor with fewer flops, it seems fewer patch samples of the image may be needed to guess a good resolution compared to the classification net)?  l.255 mentions \"4 stage residual net\"\n\n\n- largest resolution is 224 in these experiments --- what happens if this extended to be larger, up to 320 or even 512?  Even if this means upsampling the image, this will still change the network RF relative to object sizes.  Are the larger resolutions used when useful?  At the more extreme end, are candidate resolutions that are too large relative the network structure not selected?\n\n\n- \"fueled to\":  is written several times, seems to be a typo.  not sure what was intended, probably \"fed to\" or possibly \"funneled to\"\n\n- l.73: entired -> entire\n\n- l.263: Tabel -> Table\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "9T6MMlHzUCz", "reviewer": "Reviewer_S6ea", "summary": "This manuscript proposes a dynamic resolution network that can be used for the base CNN model to dynamically change the resolution of the input images. The intuition of the proposed method lies in the fact that the difficulty of the input may differ. For the easy sample, one does not need to use the full resolution for the classification. The experiments show that the proposed DRN can reduce the computational cost a lot while keeping a similar performance and even better one.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The intuition of the proposed method is promising and reasonable since we do not always need to use the full resolution to make the decision. The techniques, including Resolution-aware BN and the Gumbel-softmax, to achieve the dynamic resolution are interesting and make the two networks can be trained in an end-to-end manner.  The experiments indeed show the effectiveness of the proposed method.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1, All the experiments are conducted using images under 224*224 resolution, it would be interesting to see how the performance will be if we use a larger resolution.\n2. The accuracy with lower resolutions for some examples is even better than the model with full resolution. Is there any underlying reason for this phenomenon?\n3, It seems the improvement over the flops does align well with that over the real latency as shown in fig.3 and tab.3. It would be good to provide the performance and speed trade-off for real acceleration.\n4, For the training process, the base models will be first trained and then combined with the resolution selector network for fine-tuning. I’m wondering if it is possible to train the whole model from scratch?\n\nSome minor issues:\nLine121: “The first is the large classifier network with both high performance and expensive computational costs is first trained”, is the “is first trained” redundant?\n", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2 hours"}, "extra_text": {}}, {"review_id": "2wSPnj1AOA3", "reviewer": "Reviewer_HjSG", "summary": "A dynamic resolution network is proposed in this submission. There is a computationally efficient predictor which selects the suitable resolution for the large classification CNN model. The predictor generates hard decision of resolutions. The usage of Gumbel softmax trick enables the gradient to propagate to the predictor. A FLOPs loss is also proposed to control the computation budget. Experiments on ImageNet show that this method could largely reduce the FLOPs with similar accuracy or improve the accuracy with fewer FLOPs reduction.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. the idea using a cheap and effective CNN network to determine the input resolution of a large model is novel and convincing. It is intuitive that the difficulties of recognizing the images are diverse. \n2. Besides, predicting the difficulties of the images is much easier than recognizing them, which means a tiny network is definitely sufficient to provide a good decision. \n3. DRNet achieves 1.4% accuracy improvement with 10% FLOPs reduction and 34% FLOPs reduction without degeneration of accuracy, which is a fairly good result. \n4. This paper clearly demonstrates this idea with good writing.\nHowever, there are some concerns and limitations that need to be noted, e.g., some important related works are not cited.\n\n-------------------post rebuttal---------------------\n\nI think my concerns are well addressed in the authors' responses, So I keep my original score of 7.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1. I think the dynamic mechanism is quite unfriendly for real-world applications. For example, if all the examples are “hard”, the computation cannot be saved. To make matters worse, the proposed predictor leads to more computation. \n2. There are other detailed issues in this submission. \n[1] Writing errors. For example, the resolution of the input image in most of the existing compressed networks are still fixed...(line 38). Han [5] propose to generate more feature maps...(line 91). There may be more writing errors that I do not list.\n[2] The statements of ‘’FLOPs constraint regularization’’ and ‘’FLOPs loss’’ are better be unified since they represent the same thing.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The manuscript has been reviewed by four experienced reviewers, all of whom, after reading the rebuttal provided by the authors, agree that the manuscript meets of the bar of NeurIPS and thus should be presented to a large audience. The AC also agrees that the proposed approach is novel, supported by sufficient emperical evaluations, and hence recommends acceptance. 
Decision: Accept (Poster)