Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This paper proposed dynamic convolution (DyNet) to accelerating convolution networks. The new method is tested on the ImageNet dataset with three different backbones. It reduces the computation flops by a large margin while keeps similar classification accuracy. The additional segmentation experiment on the Cityscapes dataset also shows the new module can save computation a lot while maintaining similar segmentation accuracy.

Clarity:
The novelty of the paper is limited and the experimental results are weird for me.
1. The proposed module named dynamic convolution is detailed in Sec 3.2. As far as I can see, it is very similar to the former SENet especially in Figure (3) and Equation (2). The only difference is the introduction of g_t where the output dimension is much larger than SENet.

2. As shown in Equation (2), the proposed method contains the normal computation of fixed kernels. How can this method save computations compared to classical convolution? Is the computation flops calculated in the right way?

3.  The results in Table 5 are strange to me. Larger g_t will increase the flops absolutely according to Equation (2).

4. The author may need to show the comparisons of the number of parameters. In my opinion, the new module will increase the parameters a lot (the output dimension of the fully connected layer is as large as C_cout*g_t).

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: === Summary ===
The authors propose to use dynamic convolutional kernels as a means to reduce the computation cost in static CNNs while maintaining their performance. The dynamic kernels are obtained by a linear combination of static kernels where the weights of the linear combination are input-dependent (they are obtained similarly to the coefficients in squeeze-and-excite). The authors also include a theoretical and experimental study of the correlation.
The authors conduct extensive experiments on image classification and segmentation and show that dynamic convolutional kernels with reduced number of channels lead to significant reduction in FLOPS and increase in inference speed (for batch size 1) compared to their static counterparts with higher number of channels.

=== Recommendation ===
The experimental setup is rigorous but the current draft lacks some metrics that should be reported (as training times, parameter counts, memory requirements at training/inference) since the focus is on making CNNs more efficient.

The presented experimental results are satisfactory but the studied networks are not quite SOTA: they are much more competitive alternatives to ResNet and MobileNetv2. The correlation study is interesting.

My main issue with the paper is the lack of novelty. The use of dynamic convolutions is by no means a novel idea and has been studied in multiple previous works in vision (mixture of experts, soft conditional computation, pay less attention with dynamic convolutions, ...) which the authors fail to cite/compare against.
However, most previous work focuses on leveraging dynamic kernels to use more parameters so the focus on accelerating CNNs is novel.

Overall, I am on the fence with this paper but slightly leaning towards rejecting it for the above reasons.

=== Questions/Comments ===
- Figure 5: how are the models constrained to have same FLOPS? Is it by changing the number of channels?
- Consider adding training times for more transparency
- Consider adding parameter counts in experiment tables
- The related work subsection 2.3 is rather poor compared to existing work.
- 'While model compression based methods' -> 'On the other hand, model compression based methods'
- 'computing efficient' -> 'compute efficient'
- 'values distribute' -> 'values distributed'
- 'DETAIL ANALYSIS OF OUR MOTIVATION' -> 'Detailed analysis of our motivation'
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: 
Main contribution of the paper
- The paper proposes a dynamic convolution selection method can be applied to arbitrary classification networks based on the global average pooled (GAP) feature map info.
- The method obtained improvements over various networks (SuffleNet v2, MobileNet v2, ResNet 18) on ImageNet.

Methods
- Given the set of fixed convolutional filters, the method dynamically selects the (weighted sum) kernels by given a kind of channel attention.
- The GAP of the features gives the channel attention on each stage, and the method applies the dynamic selection of the kernels.
- The number of channels in skip-connection shluld be the same because it should be elementwise multiplied with the channel attention acquired from GAP.
- The author slightly revises the baseline networks to set the networks integrated with the proposed method to have smaller Flops.

Questions
- According to Figure 4, it seems that the proposed add-on requires many parameters because it would include a FC layer for each block. But we cannot find the number of parameters in this paper.
- The parameter $g_t$ is defined as 6. The experiment shows the ablation to the case $g_t$ =1, but what if we set the parameters to other numbers?

Strong points
- The proposed model achieved improvement with fewer Flops on large scale image classification dataset.
- The method shows effectiveness when it is attached to various classification networks.

Concerns
- The main concern of the reviewer is that the model shares the core contribution to the existing method; squeeze-and-excitation network (SEnet, Hu et.al.). The method also proposes the attention-based scaling of channels, where the attention comes from GAP, so the reviewer thinks that it is possible to explain this work as some variation of SEnet.
The author should clarify the difference and the strong points of the proposed block compared to SEnet.
- Also, the reviewer cannot guarantee that the networks trained by the proposed method can transfer the knowledge to other tasks such as detection. 
The reviewer thinks that it is a critical part because one of the primal reasons for training the network is to use them as the pre-trained backbone for the other tasks.
Regarding this, the baseline methods (MobileNet V2, Shufflenet v2, ResNet)  are widely used as a pre-trained backbone for object detection, and the papers mention the CoCo object detection results using the pre-trained backbones from their method. The reviewer thinks that the experiment regarding this should be included.
- The other thing is that the parameter increases. As in the question, the reviewer thinks that the number of parameters would be increased. The reviewer agrees that some recent works focus more on Flops, but the number of parameters is also discussed in general, when telling about the 'model size'.

Conclusion
- The author proposed a dynamic kernel selection method (add-on), which can enhance the classification accuracy of the baseline network. 
- However, the reviewer cannot convince the novelty of the proposed approach and usefulness of the pre-trained backbone network from the proposed method when applying it to the other tasks (Object detection).


Inquiries
- Clarifying the difference between SEnet.
- Testing the ImageNet trained network of the proposed method into an object detection task (as the pre-trained backbone).
- Discussing the number of the parameter as well.

review_title: Official Blind Review #1718

Meta-review: The paper proposed the use of dynamic convolutional kernels as a way to reduce inference computation cost, which is a linear combination of static kernels and fused after training for inference to reduce computation cost. The authors evaluated the proposed methods on a variety models and shown good FLOPS reduction while maintaining accuracy. 

The main concern for this paper is the limited novelty. There have been many works use dynamic convolutions as pointed out by all the reviewers. The most similar ones are SENet and soft conditional computation. Although the authors claim that soft conditional computation "focus on using more parameters to make models to be more expressive while we focus on reducing redundant calculations", the methods are pretty the same and moreover in the abstract of soft conditional computation they have "CondConv improves the performance and inference cost trade-off".
Decision: Reject