Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes Attention Retractable Transformer (ART) for image restoration, which integrates both dense and sparse attention modules in the transformer-based network. The dense attention block and sparse attention blocks emerge alternately to enable interactions among tokens extracted from a sparse area during restoration. The paper claims that the proposed ART enhances representation ability of Transformer while providing retractable attention on the input image. Extensive experiments are conducted on image super-resolution, denoising, and JPEG compression artifact reduction tasks.
strength_and_weaknesses: Strength:
+  This paper introduces sparse attention mechanism to low-level vision task and enable tokens from sparse areas of the image to interact with each other to enlarge the receptive field of transformer.
+ The paper is well organized and clearly presented. In the paper, the differences and relations with other related works are discussed and analyzed.

Weakness:
- The improvement in performance is limited. In the SR results. The ART-s has similar parameter size and FLOPS as SwinIR.  The gain obtained by ART-s is insignificant in terms of PSNR and SSIM on dataset Set5, Set14, and B100. ART and ART+ outperforms the SwarIR by 0.1dB, but they are equipped with more parameters. For the image denosing task, the gap between the SwinIR and restormer is not remarkable. 
- The proposed scheme introduces the features from dilated position to enable distant feature access, which benefits restoration of the images with repeated pattern, such as buildings in Urban100.  Table 2 and figure 4, the Urban100 are chosen to be compared with peer works.  The paper should provide more comprehensive result comparison to verify the advantage of ART.
- The description of the method could be further refined. E.g. The symbol “I” represents original input and intervals in the proposed method. 

clarity,_quality,_novelty_and_reproducibility: The paper is well organized and clearly presented. The non-local attention modules have been proposed and used in several works. This paper leverage sparse attention in the low-level vision task. Therefore, the novelty of paper is not significant. 
summary_of_the_review: The paper proposed sparse attention module and design interlaced attention blocks to leverage near and distant feature attention. The designed ART could achieve remarkable improvements on the images with repeat patterns, but it cannot prove its advantages on other general datasets.  The non-local attention modules have been proposed by other recognition works.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors propose an Attention Retractable Transformer (ART), which incorporates sparse and dense attention to widen the receptive field size. The ART method achieves pretty good results in several image restoration tasks, like super-resolution, denoising, compression artifact reduction.
strength_and_weaknesses: The idea about using sparse and dense self-attention in Transformer is simple but effective. It is good to see such an investigation in image restoration and good performance for different tasks.

Also, the alternating application of sparse and dense attention blocks help capture the local and global information. The corresponding ablation study supports the claim well.

The main comparisons with recent methods for different tasks are extensive. The ART achieves quantitative gains and also show obviously better visual results in some challenging cases.

The supplementary file provides more experiments to show the effectiveness of the ART. They also submit the demo code and pretrained models, which make their work reproducible and more solid.

The paper is prepared pretty well and organized clearly. It is also easy to read and follow with the provided code.

Weaknesses:

The proposed method is claimed for image restoration. The authors show experiments on several image restoration tasks, like image super-resolution, color, and real image denoising, and JPEG compression artifact reduction. How would be the method performed for other restoration tasks? Like image deblurring, deraining, dehazing, and so on. It would be good if the authors can provide some additional results for one/some of those tasks. Or at least, the authors can briefly discuss them in the conclusion part.

In image denoising, the main architecture of ART is different from that of Restormer. Also, Restormer uses a progressive training strategy, which is not used in this paper. Do the authors think that ART would further benefit from the U-Net structure and progressive training used in Restormer? If so, why not use a similar structure and progressive training? 

clarity,_quality,_novelty_and_reproducibility: The paper is clearly organized, like some illustration figures, difference summary table. The motivation and method details are also clear. The method is novel and reproducible with the submitted code, models, and scripts.
summary_of_the_review: The authors propose a new Transformer-based image restoration method with several new components, which are demonstrated by the extensive ablation study and main experiments. Furthermore, the paper is well prepared and reproducible based on the submitted code and models.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper investigates an efficient Transformer based method for high-quality image restoration. Specifically, the proposed Attention Retractable Transformer (ART) incorporates sparse and dense attention to sense larger receptive field. The alternative manner of those two attention blocks help capture global and local information. The ablation study and main comparisons support the claimed components well with SOTA performance.
strength_and_weaknesses: Strengths

1. The paper is well-written and easy to follow. The idea about attention retractable Transformer is simple yet effective. The combination of sparse and dense attention help sense global and local information for high-quality image restoration.

2.The authors provide extensive experiments to show the effects of each proposed component and the whole method. They also discuss the differences between the proposed method and other related works, which highlight the novelty of the paper.

3.Some experimental curves (e.g., Figs. 4, 8, and 9) provide good visualization during the training. Those curve comparisons make the method more convincing.

4.The quantitative results are good. In Tabs. 2, 4, and 5, the proposed ART achieves the best PSNR/SSIM values in most cases. Such improvements over the previous methods are notable, especially for image SR. The authors also provide model size and flops comparisons, trying to keep fair comparisons with similar parameters and flops.

5.The visual comparisons (e.g., Figs. 5 and 7) show impressive results of the proposed method, which obtains obviously better outputs than others. Such comparisons are also provided in the supplementary file.

6.The supplementary file provides many useful experiments, like more ablation study results, analyses and performance comparisons with related work, real image denoising, and other quantitative and visual comparisons. All those supplements further demonstrate the effectiveness of the proposed method.

7.The authors provide demo code on an anonymous website, where the pretrained models and testing scripts are provided to reproduce results in the main paper and supplementary file. The code availability makes the work more solid and convincing.

Weaknesses

1.Some details are not clear enough. For example, in Tab. 7 of the supplementary file, real image denoising part, did the authors use the same pretrained model to test on SIDD and DND data? Or use different pretrained models for each dataset? Also, did the authors still use the architecture shown in Fig. 2(a) for real image denoising? (As I know, Restormer uses a different structure.) There are some training strategies in Restormer, like progressive training. Did the authors also use such a strategy?

2.It would be much better if the authors can provide some discussions or/and experiments of the method applied for other image restoration applications, like deraining and deblurring.

3.It would be much better if the authors could provide more curve and/or visual comparisons. For example, how about the validation curves and visual results between the proposed method and others, like Restormer.
clarity,_quality,_novelty_and_reproducibility: The paper is organized well, stated clearly, and easy to follow. The work about attention retractable Transformer is novel and achieves promising results for several classic image restoration applications. The submitted code further makes it solid and reproducible.
summary_of_the_review: Overall, the paper is high-quality with a novel method (ART). The experiments are very extensive and support the contributions well. The submitted code and models make the method more solid.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors proposed an attention retractable Transformer (ART) for accurate image restoration, which consists of dense and sparse attention modules. The experiments are extensive and include several general image restoration tasks, where the proposed method achieves SOTA performance.
strength_and_weaknesses: Strength:

1. The paper is well-written and easy to follow. It is obvious that the authors pay much attention to most details.

2. The idea about attention retractable Transformer (ART) is motivated by some observations from image restoration tasks and is more specifically designed. The authors provide illustrations of differences between the proposed method and others, like in Figure 1 and Table 1.

3. They propose a sparse attention to compensate the defect of dense attention used in most Transformer-based image restoration methods. In the sparse area of an image, the token interactions help enlarge the receptive field of the module.

4. Alternating sparse and dense self-attention blocks helps provide retractable attention. Then, the ART model can capture both local and global information simultaneously.

5. The ablation study experiments well support the claimed contributions. The main results with promising performance show the superior and effectiveness of the proposed method.

6. In the supplementary file, the authors not only provide more results (e.g., real image denoising), but also provide the source code, pretained models, and detailed readme information (e.g., scripts). Those supplements make the paper stronger and more solid.

Weakness:

1. For image denoising, the authors mainly provide color image denoising results in the main paper and real image denoising in supplementary file. It would be better to include real image denoising in the paper too.

2. In the demo code, the authors mainly provide testing scripts. Will it be possible to give all the training details in the future? That would be good to the image restoration community.

3. Some details are not very clear. For image denoising, did the authors use the framework and/or training strategy (e.g., progressive training) as used in Restormer?
clarity,_quality,_novelty_and_reproducibility: The paper is high-quality, clear, and novel. Furthermore, the authors provide source code and pretrained models, which make it be easily reproducible.
summary_of_the_review: A well-prepared paper with solid and extensive experiments.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents a variant of transformers, an attention retractable Transformer (ART) that allows for accurate image restoration. The proposed model is innovative. It combines dense and sparse attention modules. The proposed sparse attention can allow token interactions in sparse image regions and thus enlarge the receptive field of the module. The combination of sparse and dense modules allows for global and local interactions while being tractable.

The proposed model is validated on substancial experiments for several image restoration settings. Even more results are provided in the supplementary material. Yet, results on other restoration tasks (beyond images) ar not provided. Source code is also given in the supp. mat.. .

All reviewers give positve scores and agree that the paper not only proposes an interesting module for image restoration but is also well-written, motivated and illustrated.
Yet, the reviewers also find that some details should be more clearly described. AC recommends to thoroughly go through the remaining questions and provide clarifications in the supplementary material.

justification_for_why_not_higher_score: While the proposed approach is interesting and contributes to improved image restoration with transformers, the importance for the machine learing community could be stronger if restoration beyond images were addressed. 
justification_for_why_not_lower_score: The paper makes a strong technical contribution and is well evaluated. It deserves to be presented as a spotlight!
Decision: Accept: notable-top-25%