ShuffleMixer: An Efficient ConvNet for Image Super-Resolution
Authors: Long Sun
Jinshan Pan
Jinhui Tang
Venue: NeurIPS 2022 Accept (2022)

Abstract: Lightweight and efficiency are critical drivers for the practical application of image super-resolution (SR) algorithms. We propose a simple and effective approach, ShuffleMixer, for lightweight image super-resolution that explores large convolution and channel split-shuffle operation. In contrast to previous SR models that simply stack multiple small kernel convolutions or complex operators to learn representations, we explore a large kernel ConvNet for mobile-friendly SR design. Specifically, we develop a large depth-wise convolution and two projection layers based on channel splitting and shuffling as the basic component to mix features efficiently. Since the contexts of natural images are strongly locally correlated, using large depth-wise convolutions only is insufficient to reconstruct fine details. To overcome this problem while maintaining the efficiency of the proposed module, we introduce Fused-MBConvs into the proposed network to model the local connectivity of different features. Experimental results demonstrate that the proposed ShuffleMixer is about $3 \times$ smaller than the state-of-the-art efficient SR methods, e.g. CARN, in terms of model parameters and FLOPs while achieving competitive performance. 

ShuffleMixer: An Efficient ConvNet for Image Super-Resolution
Long Sun, Jinshan Pan", Jinhui Jinhui Tang" Nanjing University of Science and Technology
{es longsun, jspan, jinhuitang) edu. en
Abstract
Lightweight and efficiency are critical drivers for the practical application of image super-resolution (SR) algorithms. We propose simple and effective approach, ShuffleMixer, for lightweight image super-resolution that explores large convolu- tion and channel split-shuffle operation. In contrast to previous SR models that sim- ply stack multiple small kernel convolutions or complex operators to learn represen- tations, we explore large kernel ConvNet for mobile-friendly SR design. Specifi- cally, we develop a large depth-wiss wise convolution and two projection layers based channel splitting and shuffling as the basic component to mix features efficiently. Since the contexts of natural images are strongly locally correlated, using large depth-wise convolutions only is insufficient reconstruct fine details. To overcome this problem while maintaining the efficiency of the proposed module, we intro- duce Fused-MBConys into the proposed network to model the local connectivity of different features. Experimental results demonstrate that the proposed ShuffleMixer is about 3x smaller than the state-of-the-ar efficient SR methods, e.g. CARN, in terms of model parameters and FLOPs while achieving competitive performance. The code is available at wtttttt///// ptttps://githbb com/ hamm//uun211999Shhaa Shuff ShuffleMizer
Introduction
Single image super-resolution (SISR) aims to recover a high-resolution image from a low-resolution one. This is classical problem that has attracted lots of attention recently due the rapid devel- opment of high-definition devices, such as Ultra-High Definition Television, Samsung Galaxy $22 Ultra, iPhone 13 Pro Max, and HUAWEI P50 Pro, and so on. Thus, is of great interest develop an efficient and effective method to estimate high-resolution images to be better displayed on these devices.
have achieved impressive reconstruction performance. However, these networks hierarchically extract local features, which highly rely on stacking deeper or more complex models to enlarge the receptive fields for performance improvements. As result, the required computational budget makes these heavy SR models challenging to deploy on resource-constrainee mobile devices in practical applications [44
Recently, convolutional neural network (CNN) based SR models 9
16 25
To alleviate heavy SR models, various methods have been proposed to reduce model complexity or speed up runtime, including efficient operation design [32
36
16
1
33
43
23
27
neural architecture search 66 (35), knowledge distillation 112
13 and structural ce-parameterizatioon 44 These methods are mainly based on improved small spatial convolutions or rarely explored. Moreover, they mostly focus on one of the efficiency indicators and do not perform well in real resource-conse constrained tasks.
methodology
advanced training strategies, and large kernel convolutions
Thus, the demand for better trade-off between complexity, latency, and SR quality is imperative.
Jinshan Pan and Jinhui Tang are the corresponding authors.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

27.65
ShuffleMixer
LAPAR-A
27.60
CARN
27.55
SMSR
EDSR- baseline
27.30
PSNN (di3)
ECBSR-M16GE M16C64
27.45
ShurfleMixerTiny
CARN-M
27.40
27.35
VDSR
27.30
LapSRN
27.25
10
102
103
FLOPs (G)
Figure 1: Model complexity and performance comparison between our proposed ShuffleMixer family and other lightweight methods on B 1100 [2] for x4 SR. Circle sizes indicate the number of parameters. ShuffleMixer achieves a better trade-off.
large receptive field involves more feature interactions, which helps reconstruct more refined results in tasks such as supeu-resolution that require dense per-pixel predictions. Recent visual transformer employ a multi-head self-attention (MHSA) mechanism to learn long-range feature representations, which leads to their fatte-of-the-ard performance in various vision tasks. However, MHSA is not friendly to enlarging the receptive field of an efficient SR design. Its complexity grows quadratically with the input resolution (the size is usually large and constant during SR training). Regular convolution with large kernels also a simple but heavyweight approach obtaining efficient receptive fields. To make large kernel convolutions practical, using depth-wise convolutions with large kernel sizes (27 39 [ is an effective alternative. Since depth-wise convolutions share connection weights between spatial locations and remain independent between channels, this property makes it challenging to capture sufficient interactions. It essential to improve the learning capability of depth-wise convolutions (DW Convs) in lightweight network design.
(ViT)-based approaches [10 26 30 24
In this paper, we develop a simple and effective network named ShuffleMixer that introduces large kernel convolutions for lightweight SR design. The core idea is fuse non-local and local spatial locations within a feature mixing block with fewer parameters and FLOPs. Specifically, we employ depth-wise convolutions with large kernel sizes to aggregate spatial information from large region. For channel mixing, we introduce channel splitting and shuffling strategies to reduce model parameters and computational costs and improve network capability. We then build effective shuffle mixer layer based on these two operators. To further improve the learning capability, we embed the Fused-MBConv into the mixer layer boost local connectivity. Taken together, we find that the ShuffleMixe network with a simple module can obtain state-of-the-ar performance. Figure I shows that our ShuffleMixer achieves a better trade-off with the least parameters and FLOPs among all existing lightweight SR methods.
The contributions of this paper summarized follows: (1) We develop efficient SR design by exploring a large kernel ConvNet that involves more useful information for image SR. (2) We introduce channel splitting and shuffling operation to perform feature mixing of the channel projection efficiently. (3) To better explore the local connectivity among cross-group features from the shuffle mixer layer, we utilize Fused-MBConys in the proposed SR design. We formulate the aforementioned modules into an end-to-end trainable network, which is named as ShuffleMixer. Experimental results show that ShuffleMixer is about 3x smaller than the state-of-the-art methods in terms of model parameters and FLOPs while achieving competitive performance compared to the state-of-the- the-ttte-rt methods.
Related Work
CNN-based Efficient SR. CNN-based methods adopt various ways reduce model complexity. FSRCNN [9] and ESPCN 33 employ post-upsampling layers to significantly reduce the compu-

---

tational burden from predefined inputs. Namhyuk et al. [ uses group convolution and cascading connection upon a recursive network to save parameters. Hui et al. [66] proposes lightweight information multi-distillation network (IMDN) to aggregate features by applying feature splitting and concatenation operations, and the improved IMDN variants 4 won the AIM2020 and NTIRE2022 Efficient SR challenge. Meanwhile, an increasingly popular approach is to search for a well-constrained architecture as a multi-objective evolution problem [6 35). Another branch compresses and accelerates heavy deep model through knowledge distillation I3 [2]. Note that fewer parameters and FLOPs do not sufficiently mean faster runtime on mobile devices because FLOPs ignore important latency-related related factors such memory access cost (MAC) and degree of parallelism [28] 32). In this paper. we analyze factors affecting the efficiency of SR models and develop a mmobile-friendee SR network.
Transformer- based SR. Transformers were initially proposed for language tasks, which stacked the multi-head self-attention and feed-forward MLP layers to learn long-range relations among its inputs. Dosovitskiy et al. [10 first applied a vision transformer to image recognition. Since then, ViT-based models have become increasingly applicable to both high-level and low-Ievel vision tasks. For image super-resolution, Chen et al. 4 develop a pre-trained image processing transformer (IPT) that directly applies the vanilla VIT to non-overlapped patch embeddings. Liang et al. (24] follow Swin Transformer (26) and propose a window- -ased self self-attention model for image restoration tasks and achieve excellent results. Window-based self celf-attention is much more computationally efficient than global self-attention, but it is still a time-consuming and memory-intensive operation.
Models with Large Kernels. AlexNet 20 is classic large-kernel convolutional neural network model that inspired many subsequent works. Global Convolutional Network [31 uses symmetric, separable large filters to improve semantic segmentation performance. Due to the high computational cost and a large number of parameters, large-size convolutional filters became not popular after VGG-Net [34]. However, large convolution kernels have recently gained attention with the devel- opment efficient convolution techniques and new architectures such as transformers and MLPs. ConvMixer (39) replaces the mixer component of ViT' [26 10] or MLPs (38 with large kernel depth-wiss wise convolutions. ConvNeXt (27) uses 7 x 7 depth-wise wise kernels to redesign a standard ResNet and achieves comparable results to Transformers. RepLKNeet [7] enlarges the convolution kernel to 31 x 31 to build pure CNN model, which obtains better results than Swin Transformer [26] on Ima- geNet. Unlike these methods that focus building big models for high-level vision tasks, explore the possibility of large convolution kernels for lightweight model design in image super-resolution.
Proposed Method
We aim to develop an efficient large-kernel CNN model for the SISR task. To this end, we introduce key designs the feature mixing block employed to encode information efficiently. In this section, we first present the overall pipeline of our proposed ShuffleMixer network. Then, we formulate the feature mixing block, which acts as a basic module for building the ShuffleMixer network. Finally, we provide details on the training loss function.
3.1 ShuffleMixxr architecture
The overall ShuffieMixer architecture. Given a low-resolution image ILR E RCXHXW where C, H x W denote the number of channels and the spatial resolution, respectively. For a color image, the value of C is 3. The proposed ShuffleMixer first extracts feature Zo E RDXH*W by convolution operation with a filter size of 3 x 3 pixels and D channels. Then, we develop feature mixing block (FMB) consisting of a Fused-MBBCnv (36) and two shuffle mixer layers, which takes the feature Zo as input to produce a deeper feature Z1 E Rx HWW Next, we utilize an upsampler module with a scale factor s to upscale the spatial resolution of the features generated by sequence of FMBs. To save parameters of the enlargement module as much as possible, we only use convolutional layer of size 1 x and pixel shuffling layer (33]. For the x4 scale factor, we progressively upsample the resolution by repeating the x2 upsampler two times. Finally, we use convolutional layer to map the upscaled feature to the residual image IR E Cx**H x and add it to the upscaled IIR by bilinear interpolation to get the final high-resolution image IsR: Isn =1 (Tr.)) + T,. where 1 (.) denotes the bilinear interpolation with scale factor s. In the following, we explain the proposed method in detail.
3

---

Bilinear Upsampling
Caear 3:3
Resture
Stuttle
Flnech
MBQuIv
Resure
Upsamples
Cara 3-3
Mixtng
Mixer
Mixing
Feature Mixing Block
Skimp-conettionn
mawwwae MEP
Projection
Chooot Spiiting
Sh Elle
Conk
Com Den I
Depthwise Copvolution
Nom
KLE
SILU
23
Come 1x]
Channel Projection
Idencity
Cone ixl
2b
Point-wris (d)
Fused-Ni
Shuffle Mixer Layer
C Channel Projection
(e)
[b)
An overview (a) of the proposed ShuffleMixer. The input low-resolution image is first converted to feature space by a convolution layer with kernel size 3 x 3 pixels. Sequential Feature Mixing Block (FMB) modules are then applied to extract representative features and upsample them for final feature reconstruction. The FMB module contains one (c) Fuued-MBBom and two (b) shuffle mixer layers. Each shuffle mixer layer has large kemee depth-wise convolution and two (c) channel projection submodules. Other components include: channel splitting and shuffling operations, (d) point-wise MLP layers, skip connections, and LayerNorm on the channels.
Figure 2:
The Feature Mixing Block developed explore local and non-local information for better feature representations. To obtain non-local feature interactions effectively, we apply shuffle mixer layers on Zo. illustrated in Figure A().. For each shuffle mixer layer, employ large kernel DW Conv to mix features at spatial locations. This operation enjoys large receptive fields with fewer parameters, which can encode more spatial information for reconstructing complete and accurate structures. As we investigated in Table 3 depth-wise convolutions with larger sizes consistently improve SR performance while maintaining computational efficiency.
To mix features at channel locations, we we employ point-wise MLPs to perform channel projection. With the help of depth-wise wise convolution, the computational cost of the shuffle mixer layer mainly caused by channel projections. We further introduce a channel splitting and shuffling (CSS) strategy [28] improve the efficiency of this step. Specifically, the input feature Zo is first split into Zj and 23: then, a point-wise MIP performs channel mixing on the split feature Z): finally, a channel shuffling operation is employed to enable the exchange of information on the concatenate feature. Therefore, the parameter complexity of the channel projection layer drops from ?(4C?) to ((C²). This procedure can be formulated as follow:
[Z) Z Spli((LaerNom((2))
2 = Vi(o(Wo(Z)))))
2, = Shuffle([2]] z)l)) + Zu
where SiLU nonlinearity function M. Wo and W1 are the point-wise convolutions; Split(-) and Shuffle(-) represent the splitting and shuffling of features in the channel dimension. This splitting operation limits representational capability since exclude the other half of the input tensors from channel interactions. The channel shuffle operation cannot guarantee that all features are processed. Inspired by the MobileNetv2 block 32 we thus repeat the channel projection layer and arrange them before and after the large depth-wise convolution to learn visual representations. From our study, as listed Table the enhanced mixer layer achieves quite similar results to the Mixer block [39] while using fewer parameters and FLOPs.

---

Since the content of natural images is locally correlated, the stacked Shuffle Mixer Layers do not fully exploit local features. It requires more capacity model feature representations for better SR performance. Therefore, we embed few convolutional blocks into the proposed model to enhance local connectivity. Concretely, we evenly add the Fused-MBCone after every two shuffle mixer layers. The original Fused-MBCony contains an expansion convolution of size 3 x 3, an SE layer [4] (i.e., the commonly used channel attention), and a reduction convolution of size 1 x 1. Using such a Fuscd-MBConv significantly increases parameters and FLOPs, which motivates us to make some changes to match the computational requirements. We remove the SE layer first, as the SiLU function can be somewhat treated gating mechanism. Note that the inference time is much slower as the hidden dimension expands. Instead of expanding the hidden channel rapidly with a large factor (where the default expansion factor is usually set to be 6) of this expansion convolution, the number of output channels and expand it to C C' (C' is experimentally set 16), as shown in Figure d]. We also study several operations for this mixing process, and more details are included in Sec.
then limit
3.2 Learning strategy
To constrain the network training, a straightforward way is to ensure that the content of the network output is close to that of the ground truth image:
Cp |s IGT l1a,
(2)
where Isn and lrr denote the network output and the corresponding ground truth HR image. We note that only using the above pixel-wine wise loss function does not effectively help high-frequency details estimation [3]. We accordingly employ frequency constraint to regularize network training, where the final loss function is defined as:
L = Lp (|TRR)) F(Ter)ll,
where F denotes the Fast Fourier transform, and À is weight parameter that is be 0.1 empirically.
4 Experimental Results
4.1 Datasets and implementation
Datasets. Following existing methods 122 24 23 we train our models on the DF2K dataset, a
merged dataset with DIV2K (37 and Flickr2K 25 which contains 3450 (800 + 2650) high-quality
images. We adopt standard protocols to generate LR images by bicubic downscaling of reference HR images. During the testing stage, we evaluate our models with the peak signal to noise ratio (PSNR) and the structural similarity index (SSIM) on five publicly available benchmark datasets Set5 [3].
Set14 42). B100 22. Urban 100 [15] and Mangal09 29 All PSNR and SSIM results are calculated on the Y channel from the YCbCr color space.
Implementation details. We train our model in RGB channels and augment the input patches with random horizontal flips and rotations. In each training mini-batch, we randomly crop 64 patches of size 64 64 from LR images as the input. The proposed model is trained by minimizing L1 loss and the frequency loss (5) with Adam [19] optimizer for 300,000 total iterations. The learning rate is set to constant 5 10-4 All experiments are conducted with the PyTorch framework on an Nvidia Tesla V100 GPU.
We provide two models according to the number of feature channels and DW Conv kernel size, and the number FMB modules 5. The number of channels and convolution kernel sizes is 64 and 7 x 7 pixels for the ShuffleMixer model and 32 and x 3 pixels for the ShuffleMixer model.
4.2 Comparisons with state-of-the-art methods
To evaluate the performance of our approach, we compare the proposed ShuffleMixer with state- of-the-art lightweight algorithms, including SRCNN (8) FSRCNN [9, ESPCN 33 VDSR [18]
DRCN [7.. LapSRN (11. CARN (1. EDSR-baseline 255 FAI.S-- IMDN 16
LAPAR 22
ECBSR 44 and SMSR 44).
5

---

Table I: Comparisons on multiple benchmark datasets for efficient SR networks. All results are calculated on the Y-channel. The FLOPs are calculated corresponding to an HR image of size 1280 x 720. Best and second-best performance are in remarked red and blue color, respectively. Blanked entries link results not reported in previous works.
Scale
Method
Params
Fl.OPs
Sen5
B100
Urban100
Manga
Bicubic
33.66/0.9299
30.24/0.860
29.5640.8431
26.88/0.8403
8000 9339
SRCNN g
57K
53G
36.66 16.66/0.9542
32.42/0.906.
31.36/0.8879
29.500.8944
35.74/0.9661
FSRCNN 9
12K
6G
37.0000.9558
32.63/0.9088
31.5340.8920
29. 80.8.000000
36.67/0.9694
ESPCN 35
21K
5G
36.8300.9564
32/40/0.9096
31.2990.8117
29. 39.48/0.8975
VDSR
665K
37.53/0.9587
33.03/0.9124
31,9000.9966 8966
30.76/0.9140
37.22/0.9977 9222/99779
DRCN 17
1,774K
17.974G
37,6330,9588
33.04/0.9115 18
31.8500.8942
30.75/0.9133
37 37.6:30.97723
TapSRN 21
813K
30G
37.52/0.9590
33.08/0.91.8 30
31.5000.9990
10.41/009 100
37.27/0. 9744009944
CARN-M 1
412K
91G
37. 37.53/0,9558
33, 20/20/0.000 9141
31 8900
31 11.30999999
x2
CARN 1
1,592K
223G
37.76/0. 37.7690.950 9590
33.52/0.9166
32. 32.099908978
31, $1.92/0,9250
EDSR-baseline
1,370K
3164
37.9900.9604
33.57/0.9175
82.1640.899
31.98/0.9272
38.54/0.9769
FALSR-A 6
1021K
235G
37,82 97.8920,9595
33,55/0.9168
32.12X1.8967
31.93/0.9256
IMDN 16
694K
161G
38, 38.0000 38.000009600
3.3. 61..6.7709977
19.19.8.8999
32. 17.0.999
38, 188.88/0.00 188.88/099774
LAPAR-C C 22
87K
35G
97.6550.9593
33,20/0.9141
31 $1.95/0.8900
31. 178
37.75/0.975
LAPAR-A
548K
171G
9605
33 33.62/0.9188
32, 19/0 8999
32. 10/0,99999 9228
02.667/0.00 078667/09977
ECBSR M16C64 241
596K
L37G
91.900999115
33,34/0.9173 778
32. 1011
SMSR (4
98SK
132G
38. 00/0096011
33 69..6/9/99999
32, 17/0.8900
32, 19A0.99244
38.7/V0.90 9771
ShuffleMixer-Tmg Thny (Ours)
108K
25G
37 $7.85/0.9000 9600
05.33300.153
31. 9990.9977
31.22/0.9188
38. 25.0.996666
ShuftleMixer (Ours)
394K
91G
38.01 96606
33. 12.63/00.18800
17/0.999999
31.89/0.9257
38.83/0.9774
Bicuhic
30.39/0.8682
27. 27.550.7742
27.21
26.95/0.65 8556
SRONN X
S7K
53G
32,75/0.9009 9090
29. 29..28/0.8 8.00
28 28.41/0077 7863
26. 24/009999
30.59/0 9107
FSRCNN 9
12K
5G
33.16/0.9140
29.43/0.82442
28. 28.53/0.7910
26.43/008000
30, 30.98/0.9211
VDSR 18
665K
613G
33. 155.6600.9113
29.77/0.831
28 7976
27. 1-4408227
32.01/0.9310
DRCN 17
1,774K
17,974G
33 9226
29. 29/76/0.8311 8311
28 8000, 7963
27. 827.
CARN-M J
412K
33.9990.9236
30.08/0.88 8367
28. 28.91/0.8000
27.55/0.8385
CARN I
1,592K
1196
29/0,999225
30.29/0.8400 8407
29. (6..0660..00 8034
28. 25..066088444
x3
EDSR baseline
25
1,555K
160G
34.37/0 9270
30.28/0.844 8417
29 29.0991.8052
28. 1570.827
33 133.55/99439
IMDN 16
703K
72G
34. 34.36/0. 9270
30. 20.32/00.441 8417
29. 09..0990..00 8046
28, I 1790
33.6 9445
LAPAR-C
99K
28G
33.9 13.91/0
30.02/0.888
28. 90/0
32.42228335
32.54/0.
LAPAR-A
594K
114G
34.36/0.9267
30.34/0 8421
29. 114.8054
28. 150.8552
33,51/0
SMSR 50
993K
68G
34,4000.9270
20.33/0.8412
10,00000.800
28. 8536
33 13.6680099445
ShuflleMixer- Tiny (Ours)
114K
12G
$4.07/0.9250
30.14/0.8362
28.9400.8009
27. 8373
33.03/0.9400
ShuflleMixer (Ours)
415K
43G
34,4000 9272
30.37/0.8423
29.12/0.8053
28, 8888088/08888
33,69/0. 9448
Bicuhic
26.0000.7022
25.96/0.6675
147.6776
24.89/0.7566
SRONN X
57K
53G
30.48X0.88 8628
077.49/07700
26. 7001
24.52/0.7221
27 77.66/0066 8505
FSRCNN 9
12K
5G
30.7 00.7110.86557 8657
27. 17.59/00.7335
26 980
24, 62/00720000
27. 77.990O... 8517
ESPCN 33
25K
1G
30.52/0.8600 8697
27. 42/0.760666
26.87/0.7216
24. 39/07217
VDSR 18
665K
613G
31.35/0.8838
28.01/0.7674
27.29/0.7251
25, 18/0.7524
28. 23.83/0.8800 8809
DRCN
1,774K
17,974C
31.53/0.8854
255.0200.7666
27.2340.7233
.1440.7500
LapSRN 21
813K
149G
31,5440.8850
28. 19/007200
27.3220.7280
20.2500
29 199.09/0.om 09.0990.8845 8845
CARN-M I
412K
33G
31.92/0.890 2903
28, 7222
27.44/0.75 3004
2.5 65.66609769
x4
CARN 1
1,592K
91G
32. 37
28. 60V0.7806
27. 78/0.
26.07/0.78.7
EDSR-basclioe
1,518K
114G
09/009988838
28. 58/0./07713
27. 27.57/0.7357
26. 04/0.78774
30.3 30.35/0.000 9067
IMDN
715K
32.21/0.89418
28.58/0.7811
27.56/1.7353
06.004007888
30.45
LAPAR-C
M2
115K
25G
31,72/0. 8884
28. 1/0.7740
27.40N0.7292
29. 20..5000..0 8951
LAPAR-A
23
659K
94G
32.15/0.8944
28. 6 1/..610.78888
27.61/0.7366
26. 14A0.7871
40/422/.9074
ECBSR-MI6C61
25
603K
35G
31.92/0.8946
28.34/0.7817
27.48/0.7393
25.81/0.7773
SMSR
1006K
12/12/0.8932
28.55/0.7808
27.55HU.7351
26. 1/0.7868
30.54/0
ShuflleMiser-Tin (Ours)
113K
31.88/0.8912
28.46/0.7779
27.45K0.7313
25.66/0.7699
99996600.80 9006
ShufleMixer (Ours)
4K
28G
32.21X0.8955
28, 66.666007727
27.61/0.7366
26.08/078835
30 00.66570.00 30.65/0.9093 9093
Table shows quantitative comparisons on benchmark datasets for the upscaling factors of x2, x3. and x4. In addition to PSNR/SSIM metrics, we list the number of parameters and FLOPs. The number of FLOPs is tested under setting of super-resolving
image to 1280 x 720 pixels. In Figure D we compare FLOPs and the number of parameters on the x4 B 100 dataset. Here, our ShuffleMixer model obtains competitive results with even fewer parameters and FLOPs. Especially, ShuffleMixer has a similar number of parameters to CARN-M. but our model outperforms it by large margin all benchmark datasets. Even with only 1 113K parameters, ShuffleMixer-T Tiny performs better than many existing methods. With regard to the scale factor x2 and x3, the proposed ShuffleMixer family can achieve similar performance.
Although IMDN 16. LAPAR-A 22 and SMSR 40 obtain comparable PSNR/SSIM performance, ShuffleMixer requires only relatively small amount of model complexity. Mcanwhile, we compare

---

thing thing thing
PowerPoint
How Do Everything
2002
Gratee affuctie
(a) HR patch
(b) Bicubic
(c) VDSR 18
DRCN 17
Medoovlie awa
thing thing thing thing
Icllnn Pinkoabetein
ppt3 from Set14
LapSRN 21
CARN
(g) IMDN 16
(h) ShuffleMixer
(a) HR patch
(b) Bicubic
(c) VDSR 18
(d) DRCN IV
img078 from Urban 100
(c) LapSRN 21
(f) CARN
(g) IMDN 16
(h) ShuffleMixer
(a) HR patch
(b) Bicubic
(c) VDSR 18
(d) DRCN 17
img095 from Urban100
(e) LapSRN 21
(1 CARN MI
(g) IMDN 16
(h) ShuffleMixer
Figure 3: Visual comparisons for x4 SR on Set14 and Urban 100 datasets. The proposed algorithm
recovers the image with clearer structures.
BELL
eELL
(a) LR patch
(b) Bicubic
(c) SelfEx
C BELL
BELL
BELL
img004 from historical dataset
(d) CARN 1
(e) LAPAR-A 22
(f) ShuffleMixe
Compared with the results in (b), the super-resolved image (f) generated by our approach is much clearer with fewer artifacts.
Figure 4: Visual comparisons for SR on historical dataset [2]]

---

Table 2: Ablation studies of the shuffler mixer layer and the feature mixing block on x4 DIV2K validation set 37 The FLOPs are calculated by ffcore on an LR image with a resolution of 256 x 256 pixels (FLOPs in Table 3 and Table 4 are also measured with this setting).
(a) Shuffle Mixer Layer
(b) Feature Mixing Block
Baseline
CSS
CDC
Conv
S-Conv
C-Conv
S-ResBlock
S-FMBConv
Params(K) K)
55.9
24.7
35.5
81.7
81.7
128
128
113
FLOPs(G)
5.2
3.2
3.8
6.9
6.9
9.9
9.9
8.9
PSNR(dB)
29.96
29.83
29.99
30.12
30.16
30.20
30.24
30.21
SSIM
0.8288
0.8231
0,8259
0.8299
0.8305
0,8316
0.8327
0.8321
Table 3: Experimental results for different kernel size settings. We test PSNR/SSIM re- sults on the DIV2K validation set, and the kernel size ranges from 3 x 3 to 21 x 21 pixels.
Table 4: Effect of increasing the depth (#B) or width (#C) of the proposed tiny model. All the methods are evaluated on x4 DIV2K validation set.
Kemeel Size
PSNRIdHASSIM
Params(K)
FLOPP(()
Sizz
PSNRIBB-SUMM
3 X
3
30.2000:3221
113
8.9
5 X.
5
30 8344
30.2410:3266
118
9.2
B
32
12
7
x.3
38220119342
118
125
9.2
9.7
9 X
9
80:29/008339
136
10.4
76
1625818220
11 x 11
30.28/0.8339
148
11.2
36258192222
20.29/08337
12:2
32
3x3
374
124
21 x
30.2910.8339
17.9
the GPU run time with fast and lightweight models on X4 SR: CARN D. CARN-M and LAPAR-A [22] and the proposed method has a faster inference speed. Our ShuffleMixer- Tiny and ShuffleMixer reconstruct an HR image with a resolution of 1280 x 720 pixels with 0.016s and 0.021s, respectively. As comparison, the runtimes are 0.017s, 0.019s, and 0.031s for CARN-M, CARN, and LAPAR-A. Note that Pytorch has poor support for large-kernel depth wise convolution; employing optimized depth- wise convolutions further accelerate the inference time of method, as suggested in 77. All these results demonstrate the effectiveness of our method.
Figure 3 presents visual comparisons on Set14 and Urban datasets for a x4 scale. The qualitative comparison results demonstrate that our proposed methods can produce more visually pleasing results. The structures and details are better recovered.
We further evaluate our approach on real low-quality images. One example from the historical dataset [21] is shown in Figure [4] The results by [15 [2] show visible artifacts. Our method and CARN generate smooth details, but our results have clearer structure.
4.3 Analysis and discussions
The core idea of ShuffleMixer lies in the shuffle mixer layer, feature mixing block, and large kernel convolution. In this subsection, we evaluate them on the proposed tiny model and train them on the x4 DIV2K dataset (37].
Effectiveness of the shuffle mixer layer. To verify the efficiency of the shuffle mixer, we use 10 ConvMixer 39 blocks to build a baseline model. Unlike the original ConvMixer module, we replace BatchNorm with LayerNorm and apply only before the point-wise MLP layer because BatchNorm tends to bring artifacts in the generated results 25 41). The kernel of depth-wise convolution is set to 3. and the number of channels is 32. When applying the channel splitting and shuffling (CSS) strategy, the number of parameters is reduced from 55.9K to 24.7K, and the performance is also 0. lower than the baseline. This result reflects that the split operation limits the representation capability of the channel projection layer. To compensate for the lack of PSNR, we repeat the CSS-based projection layer to enable more cross-group feature mixing (denoted by CDC). Table shows a quantitative comparison where we find that CDC achieves similar performance the baseline model while reducing parameters from 55.9K to 35.5K and FLOPs from 5.2G to 3.8G.
Effectiveness of the feature mixing block. To validate the effectiveness of the proposed feature mixing block, we take the CDC model as baseline and embed a convolution layer with kernel size of 3 3 pixels after each two shuffle mixer layers. This conduct brings gain of 13BB over
8

---

-3
-10
(a) w/o Conv
(b) Conv
-28
(c) S-Conv
(d) C-Conv
Figure 5: Visualization of learned feature maps before the upsampler module. We show the average features over the channel dimension.
the baseline. We then investigate feature element-wise summation or concatenation to analyse the effect of feature fusion manners. The input features are aggregated by the aforementioned ways and refined by a convolutional layer with kernel size of 3 x 3, which we denote as S-Conv and C-Conv, respectively. Table (bb) shows that they all improve over the baseline; C-Conv achieves better PSNR performance while having more computational cost. channel axis before the upsampler module, which illustrates that enhancing local connectivity helps grab more delicate high-frequency contents. Based on the S-Conv, we replace the convolution layer with basic residual blocks (S-ResBlock) and Tused-MBCony (S-FMBConv). Table ([b) shows that S-FMBConv obtains balanced trade-off between model complexity and SR performance. Thus, we choose S-FMBConv to strengthen the local interactions within features in this paper.
exxuiiii the average feature map of the
Effectiveness of large depth-wist wise convolution. To demonstrate the effect of a large kernel, we use different kernel sizes ranging from 3 x 3 to 21 21 pixels and test their performance separately. Table shows that using larger kernel size will improve the performance. In particular, the PSNR value of the method using kernel size of 7 pixels is 0.07dB higher than that of using 3 kernel size while only increasing 12K parameters and 0.8G FLOPs. In addition, we note that the pixels. Thus, the kernel size
performance gains are minor when the kernel size larger than set to be 7 x pixels as trade-off between accuracy and model complexity in this paper.
Moreover, investigate the effect of increasing the depth (#B) width (#C) of the proposed tiny model on the reconstruction performance. Table 3 shows that increasing the depth or width of the model with small kernel size does not yield good performance gain enlarging the kernel size. These results also demonstrate the effectiveness of employing large depth-wise convolution.
Conclusion
In this paper, we propose lightweight deep model to solve the image super-resolution problem. The proposed deep model, i.e., ShuffleMixer, contains shuffler mixer layer with larger effective receptive field to extract non-local feature representations efficiently. We also introduce the Fused- MBCоnv to modulate the local connectivity of features generated by the shuffler mixer layer, which critical for improving SR performance. We both qualitatively and quantitatively evaluate proposed ShuffleMixer on commonly used benchmarks. Experimental results demonstrate that the proposed ShuffleMixer is much more efficient while achieving competitive performance than the state-of-the-- methods.

---

Acknowledgments. This work has been supported in part by the National Key R&D Program of China (No. 2018AAA0102001), the National Natural Science Foundation of China (Nos. 61922043, 61872421, 61925204), and the Fundamental Research Funds for the Central Universities (No. 30920041109).
Broader Impact
This paper is an exploratory work on lightweight and efficient image super-resolution using a large- kernel ConvNet. This approach can be deployed in some resource-constraincd environments to improve image quality, such processing pictures taken by smartphones and reducing bandwidth during video calls or meetings. However, super-resolution technology has also brought some negative effects, such criminals using this technology enhance people's 's facial body features, thereby allowing identity information to leak. It is worth noting that the positive social impact of image nuper-resolutio far outweighs the potential problems. We call on people to use this technology and its derivative applications without harming the personal interests of the public.
References
[1] Namhyuk Ahn, Byungkon Kang, and Kyung-Ah Sohn. Fast, accurate, and lightweight super-resolution with cascading residual network. In ECCV, 2018.
[2] Pablo Arbeláez, Michael Maire, Charless C. Fowlkes, and Jitendra Malik. Contour detection and hierarchi- cal image segmentation. PAMI, 33(5):898 916, 2011.
[3] Marco Bevilacqua, Aline Roumy, Christine Guillemot, and Marie line Alberi Morel. Low-complexity single-image super-rosolution based on nonnegative neighbor embedding. In BMVC, 2012.
[4] Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao. Pre-trained image pracessing transformer. In CVPR, 2021.
[5] Sung-Jin Cho, Seo-Won Ji, Jun-Pyo Hong. Seung-Wor Won Jung, and Sung-Jea Ko. Rethinking coarse-to-fine approach in single image deblurring. In ICCV, 2021.
[6] Xiangxiang Chu, Zhang, Hailong Ma, Ruijun Xu, and Qingyuan Li. Fast, accurate and lightweight super-resolution with neural architecture scarch. In ICPR, 2021.
Xiaohan Ding, Xiangyu Zhang, Yizhuang Zhou, Jungong Han, Guiguang Ding, and Jian Sun. Scaling up your kernels to 31x31: Revisiting large kernel design in cnns. CVPR, 2022.
Chao Dong, Chen Change Loy. Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. PAMI, 38(2):295-307, 2016.
Chao Dong. Chen Change Loy, and Xiaoou Tang. Accelerating the super-resolution convolutional neural network. In ECCV, 2016.
[10] Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakoh Uszkoreit, and Neil Houlsby. An image is worth 16x words: Transformers for image recognition at scale. ICLR, 2021.
[11] Stefan Elfwing. Eiji Uchibe, and Kenji Doya. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Networks, 107:3-1 -1, 2018.
[12] Qinquan Gao, Yan Zhao, Gen Li, and Tong Tong. Image super-resolution using knowledge distillation. In ACCV, 2019.
[13] Zihin He, Tao Dai, Jian Lu, Yong Jiang, and Shu-Tao Xia. Fakd: Feature-affintty hased knowledge distillation for efficient image super-resolution. ICIP, 2020.
[14] Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excittion networks. In CVPR, 2018.
[15] Jia-Bin Huang. Abhishek Singh, and Narendra Ahuja. Single image super-resolution from transformed self-exemplars. In CVPR, 2015.
[16] Zheng Hui. Xinbo Gao, Yunchu Yang, and Xiumei Wang. Lightweight image super-resolution with information multi-distillation network. In ACM MM, 2019.
10

---

[17] Jiwon Kim, Jung Kwon Lee. and Kyoung Mu Lee. Deeply-recursive convolutional network for image super-resolation. In CVPR, 2016.
[18] Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. Accurate image super-resolution using very deep convolutional networks. In CVPR, 2016.
[19] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In JCLR, 2015.
[20] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NewrlPS, 2012.
[21] Wei- Wei-Sheng Lai, Jia- Bin Huang, Narendra Ahuja, and Ming- Hauan Yang. Deep laplacian pyramid networks for fast and accurate super-resolution. In CVPR, 2017.
[22] Wenbo Li, Kun Zhou, Lu Qi, Nianjuan Jiang. Jiangbo Lu, and Jiaya Jia. LAPAR: Linearly-aseembled pixel-adaptive regression network for single image super-resolution and beyond. In NewrlPS, 2020.
[23] Yawei Li, Kai Zhang, Lue Van Gool, Radu Timofte, et al. NTIRE 2022 challenge on efficient super- resolution: Methods and results. In CVPR Workshops, 2022.
[24] Jingyun Liang. Jiezhang Cao, Guolei Sun, Kai Zhang, Lue Van Gool, and Radu Timofte. SwinIR: Image restoration using swin transformer. In ICCV Workshops, 2021.
[25] Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. Enhanced deep residual networks for single image super-resolution. In CVPR Workshops, 2017.
[26] Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang. Stephen Lin, and Baining Guo. Swin Transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021.
[27] Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. ConvNet for the 2020s. In CVPR, 2022.
[28] Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. ShuffleNet V2: Practical guidelines for efficient enn architecture design. In ECCV, 2018.
[29] Yusuke Matsui, Kota Ito, Yuji Aramaki, Toshihiko Yamasaki, and Kiyoharu Aizawa. Sketch-hased manga retrieval using manga 109 dataset. arXiv preprint arXiv:1510.04330 2015.
[30] Sachin Mehta and Mohammad Rastegari. MobileViT: Light-weight, general-purpose, and mobile- friendly vision transformer. In ICLR, 2022.
[31] Chao Peng, Xiangyu Zhang, Gang Yu. Guiming Luo, and Jian Sun. Large kernel matters improve semantic segmentation by global convolutional network. CVPR, 2017.
[32] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chich Chen. MobileNetV2: Inverted residuals and linear bottlenecks. CVPR, 2018
[33] Wenzhe Shi, Jose Caballero, Ferene Huszar, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In CVPR, 2016.
[34] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. arXiv preprint arviv: 1w1.11199.5550 1556, 2014.
[35] Dehua Song, Chang Xu, Xu Jia, Yiyi Chen, Chunjing Xu, and Yunhe Wang. Efficient residual dense block search for image super-resolution. In AAAI, 2020.
[36] Mingxing Tan and Quac Le. EfficientNetV2: Smaller models and faster training. In JCML, 2021.
[37] Radu Timofte, Firikur Agustsson, Luc Van Gool, Ming- Hsuan Yang. and Lei Zhang NTIRE 2017 challenge on single image super-resolution: Methods and results. In CVPR Workshops, 2017.
[38] Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. MLP-Mixer: An MLP architecture vision. In NeurlPS, 2021.
[39] Asher Trockman and Zico Kolter. Patches are all you need? In ICLR, 2022.
[40] Longguang Wang. Xiaoyu Dong, Yingqian Wang. Xinyi Ying, Zaiping Lin, Wei An, and Yulan Guo. Exploring sparsity in image super-resolution for efficient inference. In CVPR, 2021.
11

---

[41] Xintao Wang. Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. ESRGAN: Enhanced super-resolution generative adversarial networks. In ECCV Workshops, 2018.
[42] Roman Zeyde. Michael Elad, and Matan Protter. On single image scale-up using sparse-representationn.. In Curves and Surfaces, 2012.
[43] Kai Zhang, Martin Danelljan, Yawei Li, and et al. AIM 2020 challenge on efficient super-resolution: Methods and results. In ECCV Workshops, 2020.
[44] Xindong Zhang, Hui Zeng, and Lei Zhang. Edge-oriented convolution block for real-time super resolution on mobile devices. In ACM MM, 2021.
[45] Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu. Image super-resolution using very deep residual channel attention networks. In ECCV, 2018.
Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] [Yes], [No] or [N/A]- You are strongly encouraged to include a justification to your answer, either by referencing the appropriate section of your paper or providing brief inline description.
Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [No]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as URL.? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you' 're
using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
12

---

(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "qoZO7SFao1e", "reviewer": "Reviewer_X1ut", "summary": "The paper tackles one of the hot topics of computer vision, image super-resolution, which can be applied in different commercial usages (e.g. improving medical image analysis). The main characteristic of ShuffleMixer is the efficient construction of its architecture that uses depthwise convolution, projection layers, and mixing blocks.", "questions": "* Why did you evaluate Table 1 in Y-channel and Table 2 in RGB channels? \n* What about table 3, is it in Y channel or RGB channel?\n", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n* The manuscript is well structured and easy to follow  with its three proposals formed in a shuffle mixer layer, feature mixing block, and large kernel convolution.\n* The proposed architecture only has 394 parameters (x2 scale) and is capable of running  on 91G FLOPs.\n* The lightweight proposal, in x4 scale, achieves the state-of-the-art result, with less computational costs, in most of the standard datasets used for image super-resolution.\n\nWeaknesses:\n\n* Although the model is well structured in its proposal, some of its acronyms presented in the paper are not defined in the proper location. A deep revision of this lack is needed.\n* Although the model is efficient and lightweight that outperforms the state of the art, since your proposal is the architectural settings, I feel that more experiments with different settings are needed. For example, the large kernel size goes till 13x13, why not evaluate, at least, till 15x15?. I see PSNR dropped when the 13x13  kernel size is considered but the explanation is not clear yet. \n* ShuffleMixer is an important contribution. Therefore, a deep ablation study is needed, mixing the experiments with the settings of Table 2 and Table 3. \n\nMinor errors:\n\n* Abstract: what is the Fused-MBConv? Most people just read the abstract.\n* As you stated in the first paragraph of your Introduction, it is well known the advantage of SR in Smartphones. However, a couple of citations are needed, or at least one, to show the formal usage in S22 Ultra / P50 Pro.\n* 4 Experimental results: in this section experiments and their results are reported, please,  check the past time of some statements. For example, “We train our models on the DF2K dataset”.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "cd1Lu-0V1gE", "reviewer": "Reviewer_gjyf", "summary": "This paper proposes an efficient image super-resolution network, namely ShuffleMixer. The core design of the network is the feature mixing block that learns a deep representation for the input images. The feature mixing block is inspired by recent architecture developments such as the layer normalization and MLP from transformers and the channel split and fusion from IMDN. To balance the parameter efficiency and the network performance, convolution with large kernels is used. It is reported that the proposed network achieves competitive to the state-of-the-art.", "questions": "1. The authors mentioned channel shuffling operation in the paper which is a submodule in the channel projection layer. As far as I understand, the channel shuffling operation is just a $1 \\times 1$ convolution, which is quite different from the ShuffleNet operations. Thus, using the term of channel shuffle might be a little bit confusing.\n2. Is there a discussion about the loss function? For example, how does the frequency constraint influence the performance of the network?\n3. In the abstract, it is written that \"Experimental results demonstrate that the proposed ShuffleMixer is about 6× smaller than the state-of-the-art methods in terms of model parameters and FLOPs while achieving competitive performance.\" Without mentioning the baseline network, this seems to be an overclaim.\n", "limitations": "The authors did not discuss the limitation of their work. Considering the field of research, the discussion of potential negative societal impact is not necessary.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is well-written and easy to follow. The main figure clearly shows the design of the network.\n2. Compared with the previous methods, the proposed method performs quite good under similar setting of number of parameters and computational complexity.\n\nWeakness:\n1. The paper is more about applications. Thus, it might fall out of scope for the conference that is more focused on theories.\n2. Besides the number of parameters and computational complexity, the inference time is also factor that is at least of equal importance. The authors also mentioned the recent developments for efficient SR in AIM and NTIRE challenges. But unfortunately, there is no direct comparison with the top winners of the challenges (except IMDN) in Table 1 and Figure 1. In Figure 1, even IMDN is not listed, which is not preferrable. It is encouraged to evalutate different methods more thoroughly.\n3. The maximum kernel size is $13 \\times 13$. It not clear how the performance is if the kernel size is further enlarged.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "FZ3MmhWByT", "reviewer": "Reviewer_YLsE", "summary": "This paper proposes an efficient SR network based on depthwise convolutions with large kernel sizes and channel shuffling. The proposed network achieves better performance while with fewer parameters and multi-adds. ", "questions": "How about the comparisons between speed?  Large kernel size may heavily slow the whole network, which is also unfriendly to terminal devices.", "limitations": "The novelty of this paper may be limited and the analysis may be inadequate. See the weakness and questions above.", "rating": 3, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThis paper is well-written and easy to follow. The proposed network can achieve better performance while with fewer parameters and multi-adds. \n\nWeakness:\n1. The novelty of this paper may be limited. This paper uses depthwise convolution and channel shuffling to reduce the number of parameters and multi-adds, while these techniques have already been studied by many other methods.\n\n2. The authors argue that a larger receptive field is better for SR. While there are no experiments to analyze the difference between small and large kernels. For example, the authors can perform an ablation study by replacing the large kernels in the proposed network with small kernels and extendinging the depth to keep the model size.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "8IFwCcfJk3r", "reviewer": "Reviewer_xXRb", "summary": "This paper proposes a ShuffleMixer architecture to combine large spatial depthwise conv and efficient channel and spatial mixing to perform efficient and state of the art single image super resolution results.\n\nEdit: I think both the work in the paper and the authors' additional details and results allow me to update my overall paper rating.", "questions": "1. How do all of these results compare to bilinear/bicubic (Table 1 and Fig 1 results)? Having a non-learned baseline is an important comparison in fast SR results.\n\n2. Reference [7] was an accepted paper to CVPR, please update the reference.\n\n3. Does a different loss function increase the PNSR results? L1 is used both on pixel and fourier features, but would L2 on pixel help or hurt overall performance?\n\n4. The paper boosts the performance of large kernels and reference [7] mentions 31x31 kernels. Why were ablations past 13x13 not performed?\n\n5. Table 3 shows ablations for 3x3 to 13x13 kernels. Was a similar ablation done on the channel features for the model? E.g. is it more efficient to increase a model from 3x3 to 13x13 or to increase C appropriately to the same number of flops?\n\n6. The introduction references mobile/embedded platforms with their high screen resolutions and the importance of super resolution. What is the runtime performance of these methods on such a mobile platform? Are there any practical considerations that large kernels or channel mixing have on devices with smaller caches?\n\n7. For the results on Urban100, there is a note that the PSNR is lower than other methods. Is the PSNR low across the image, or are there patches where the PSNR-Y suffers more than others? I notice the patches chosen in the supplementary materials are very high frequency areas that I agree are perceptually similar to the state of the art method, but I wonder if a more in-depth analysis of the PSNR was done.\n\n8. What is the importance of the SiLU vs other activation functions in this model? What is the performance gain over baseline ReLU? Are there efficiency concerns with implementing SiLU on mobile?", "limitations": "Authors sufficiently discuss limitations and impact of work.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe Tiny model has strong results with a very small amount of compute and the regular sized model is competitive or is state of the with less compute.\n\nThis work introduces large receptive fields while keeping compute and parameters down while maintaining strong results.\n\nAblations on different parts of the architectures and kernel sizes show a principled search done in this architecture.\n\n\nWeaknesses:\nPoor PSNR/SSIM results on Urban100 (This is discussed in supplemental and I have follow up questions below).\nNo mobile runtime. (Importance of SISR in embedded / mobile discussed but no result shared. Additionally, line 78 declares a mobile friendly port).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept