Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes an efficient image super-resolution network, namely ShuffleMixer. The core design of the network is the feature mixing block that learns a deep representation for the input images. The feature mixing block is inspired by recent architecture developments such as the layer normalization and MLP from transformers and the channel split and fusion from IMDN. To balance the parameter efficiency and the network performance, convolution with large kernels is used. It is reported that the proposed network achieves competitive to the state-of-the-art.
strengths_and_weaknesses: Strengths:
1. The paper is well-written and easy to follow. The main figure clearly shows the design of the network.
2. Compared with the previous methods, the proposed method performs quite good under similar setting of number of parameters and computational complexity.

Weakness:
1. The paper is more about applications. Thus, it might fall out of scope for the conference that is more focused on theories.
2. Besides the number of parameters and computational complexity, the inference time is also factor that is at least of equal importance. The authors also mentioned the recent developments for efficient SR in AIM and NTIRE challenges. But unfortunately, there is no direct comparison with the top winners of the challenges (except IMDN) in Table 1 and Figure 1. In Figure 1, even IMDN is not listed, which is not preferrable. It is encouraged to evalutate different methods more thoroughly.
3. The maximum kernel size is $13 \times 13$. It not clear how the performance is if the kernel size is further enlarged.

questions: 1. The authors mentioned channel shuffling operation in the paper which is a submodule in the channel projection layer. As far as I understand, the channel shuffling operation is just a $1 \times 1$ convolution, which is quite different from the ShuffleNet operations. Thus, using the term of channel shuffle might be a little bit confusing.
2. Is there a discussion about the loss function? For example, how does the frequency constraint influence the performance of the network?
3. In the abstract, it is written that "Experimental results demonstrate that the proposed ShuffleMixer is about 6× smaller than the state-of-the-art methods in terms of model parameters and FLOPs while achieving competitive performance." Without mentioning the baseline network, this seems to be an overclaim.

limitations: The authors did not discuss the limitation of their work. Considering the field of research, the discussion of potential negative societal impact is not necessary.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper tackles one of the hot topics of computer vision, image super-resolution, which can be applied in different commercial usages (e.g. improving medical image analysis). The main characteristic of ShuffleMixer is the efficient construction of its architecture that uses depthwise convolution, projection layers, and mixing blocks.
strengths_and_weaknesses: Strengths:

* The manuscript is well structured and easy to follow  with its three proposals formed in a shuffle mixer layer, feature mixing block, and large kernel convolution.
* The proposed architecture only has 394 parameters (x2 scale) and is capable of running  on 91G FLOPs.
* The lightweight proposal, in x4 scale, achieves the state-of-the-art result, with less computational costs, in most of the standard datasets used for image super-resolution.

Weaknesses:

* Although the model is well structured in its proposal, some of its acronyms presented in the paper are not defined in the proper location. A deep revision of this lack is needed.
* Although the model is efficient and lightweight that outperforms the state of the art, since your proposal is the architectural settings, I feel that more experiments with different settings are needed. For example, the large kernel size goes till 13x13, why not evaluate, at least, till 15x15?. I see PSNR dropped when the 13x13  kernel size is considered but the explanation is not clear yet. 
* ShuffleMixer is an important contribution. Therefore, a deep ablation study is needed, mixing the experiments with the settings of Table 2 and Table 3. 

Minor errors:

* Abstract: what is the Fused-MBConv? Most people just read the abstract.
* As you stated in the first paragraph of your Introduction, it is well known the advantage of SR in Smartphones. However, a couple of citations are needed, or at least one, to show the formal usage in S22 Ultra / P50 Pro.
* 4 Experimental results: in this section experiments and their results are reported, please,  check the past time of some statements. For example, “We train our models on the DF2K dataset”.


questions: * Why did you evaluate Table 1 in Y-channel and Table 2 in RGB channels? 
* What about table 3, is it in Y channel or RGB channel?

limitations: Yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a ShuffleMixer architecture to combine large spatial depthwise conv and efficient channel and spatial mixing to perform efficient and state of the art single image super resolution results.

Edit: I think both the work in the paper and the authors' additional details and results allow me to update my overall paper rating.
strengths_and_weaknesses: Strengths:
The Tiny model has strong results with a very small amount of compute and the regular sized model is competitive or is state of the with less compute.

This work introduces large receptive fields while keeping compute and parameters down while maintaining strong results.

Ablations on different parts of the architectures and kernel sizes show a principled search done in this architecture.


Weaknesses:
Poor PSNR/SSIM results on Urban100 (This is discussed in supplemental and I have follow up questions below).
No mobile runtime. (Importance of SISR in embedded / mobile discussed but no result shared. Additionally, line 78 declares a mobile friendly port).
questions: 1. How do all of these results compare to bilinear/bicubic (Table 1 and Fig 1 results)? Having a non-learned baseline is an important comparison in fast SR results.

2. Reference [7] was an accepted paper to CVPR, please update the reference.

3. Does a different loss function increase the PNSR results? L1 is used both on pixel and fourier features, but would L2 on pixel help or hurt overall performance?

4. The paper boosts the performance of large kernels and reference [7] mentions 31x31 kernels. Why were ablations past 13x13 not performed?

5. Table 3 shows ablations for 3x3 to 13x13 kernels. Was a similar ablation done on the channel features for the model? E.g. is it more efficient to increase a model from 3x3 to 13x13 or to increase C appropriately to the same number of flops?

6. The introduction references mobile/embedded platforms with their high screen resolutions and the importance of super resolution. What is the runtime performance of these methods on such a mobile platform? Are there any practical considerations that large kernels or channel mixing have on devices with smaller caches?

7. For the results on Urban100, there is a note that the PSNR is lower than other methods. Is the PSNR low across the image, or are there patches where the PSNR-Y suffers more than others? I notice the patches chosen in the supplementary materials are very high frequency areas that I agree are perceptually similar to the state of the art method, but I wonder if a more in-depth analysis of the PSNR was done.

8. What is the importance of the SiLU vs other activation functions in this model? What is the performance gain over baseline ReLU? Are there efficiency concerns with implementing SiLU on mobile?
limitations: Authors sufficiently discuss limitations and impact of work.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: This paper proposes an efficient SR network based on depthwise convolutions with large kernel sizes and channel shuffling. The proposed network achieves better performance while with fewer parameters and multi-adds. 
strengths_and_weaknesses: Strengths:
This paper is well-written and easy to follow. The proposed network can achieve better performance while with fewer parameters and multi-adds. 

Weakness:
1. The novelty of this paper may be limited. This paper uses depthwise convolution and channel shuffling to reduce the number of parameters and multi-adds, while these techniques have already been studied by many other methods.

2. The authors argue that a larger receptive field is better for SR. While there are no experiments to analyze the difference between small and large kernels. For example, the authors can perform an ablation study by replacing the large kernels in the proposed network with small kernels and extendinging the depth to keep the model size.
questions: How about the comparisons between speed?  Large kernel size may heavily slow the whole network, which is also unfriendly to terminal devices.
limitations: The novelty of this paper may be limited and the analysis may be inadequate. See the weakness and questions above.
review_title: null

Meta-review: metareview: The paper received divergent reviews (with two reviewers leaning to reject and two leaning to accept). The key strengths of the paper are simple ideas and strong results on efficient and accurate image super-resolution. 

The raised weakness include:
- "might fall out of scope for the conference" [Reviewer gjyf]
The AC disagrees with the statement. As authors' responses pointed out, CNN designs for computer vision applications are welcome at NeurIPS. 

- "comparison with the top winners of the challenges" [Reviewer gjyf]
The authors' rebuttal include direct comparisons with previous winners. 

- "how the performance is if the kernel size" [Reviewer gjyf]
The authors' provide the additional ablation. 

There were no follow-up questions and no rebuttal acknowledgement from Reviewer gjyf. From the responses alone the AC thinks that the concerns have been sufficiently addressed. 

Another major weakness is 
- "The novelty of this paper may be limited. " [Reviewer YLsE]. 

It is true that basic components like depthwise convolution and channel shuffling have been explored in efficient CNN design. But as the reviewers' rebuttal described, the adoptation and modification of DW Conv or channel shuffle are non-trivial for the context of image SR.

In sum, the AC reads the reviews and rebuttal. Weighting both the strength and the weakness of the work, the AC decides to side with reviewer X1ut and xXRb and recommends to accept.

confidence: Certain
Decision: Accept