Title: PromptIR: Prompting for All-in-One Image Restoration
Venue: NeurIPS 2023
Accepted: yes
Keywords: Image Restoration
Abstract: Image restoration involves recovering a high-quality clean image from its degraded version. Deep learning-based methods have significantly improved image restoration performance, however, they have limited generalization ability to different degradation types and levels. This restricts their real-world application since it requires training individual models for each specific degradation and knowing the input degradation type to apply the relevant model. We present a prompt-based learning approach, PromptIR, for All-In-One image restoration that can effectively restore images from various types and levels of degradation. In particular, our method uses prompts to encode degradation-specific information, which is then used to dynamically guide the restoration network. This allows our method to generalize to different degradation types and levels, while still achieving state-of-the-art results on image denoising, deraining, and dehazing.  Overall, PromptIR offers a generic and efficient plugin module with few lightweight prompts that can be used to restore images of various types and levels of degradation with no prior information on the corruptions present in the image. Our code and pre-trained models are available here: https://github.com/va1shn9v/PromptIR

PromptIR: Prompting for All-in-One Blind Image Restoration

Abstract

Image restoration involves recovering a high-quality clean image from its degraded version. Deep learning-based methods have significantly improved image restoration performance, however, they have limited generalization ability to different degradation types and levels. This restricts their real-world application since it requires training individual models for each specific degradation and knowing the input degradation type to apply the relevant model. We present a prompt-based learning approach, PromptIR, for All-In-One image restoration that can effectively restore images from various types and levels of degradation. In particular, our method uses prompts to encode degradation-specific information, which is then used to dynamically guide the restoration network. This allows our method to generalize to different degradation types and levels, while still achieving state-ofthe-art results on image denoising, deraining, and dehazing. Overall, PromptIR offers a generic and efficient plugin module with few lightweight prompts that can be used to restore images of various types and levels of degradation with no prior information on the corruptions present in the image. Our code and pre-trained models are available here: https://github.com/va1shn9v/PromptIR .

1 Introduction

During image acquisition, degradations (such as noise, blur, haze, rain, etc.) are often introduced either due to the physical limitations of cameras or unsuitable ambient conditions. Image restoration refers to the process of recovering a high-quality clean image from its degraded version. It is a highly challenging problem due to its ill-posed nature as there exists many feasible solutions, both natural and unnatural. Recently, deep learning based restoration approaches [47, 12, 69, 45, 76, 55, 43, 74] have emerged as more effective choice in comparison to conventional methods [19, 36, 13, 53, 27, 42, 28].

Deep neural network-based methods broadly differ in their approach to addressing the image restoration problem. Some works incorporate explicit task-specific knowledge in the network to deal with the corresponding restoration task, such as denoising [45, 76], deblurring [43, 74], and dehazing [47, 12, 36]. However, these methods lack generalization beyond the specific degradation type and level. On the other hand, some works [56, 68, 59, 71, 70, 8] focus on developing a robust architecture design and learn image priors from data implicitly. These methods train separate copies of the same network for different degradation types, degradation levels, and in more extreme cases on different datasets. However, replicating the same restoration model for different degradation types, levels, and data distributions is a compute-intensive and tedious process, and oftentimes impractical for resource-constrained platforms like mobile and edge devices. Furthermore, to select an appropriate restoration model during testing, these approaches require prior knowledge regarding the degradation present in the input image.

Prompts

Degradation

Plug-ln Prompt

Blocks

Context

All-in-one

RestorationNetwork

(single stage training,no apriori

degradation specification)

Therefore, there is a pressing need to develop an all-in-one method that can effectively restore images from various types and levels of degradation.

One recent method, AirNet [29], addresses the all-in-one restoration task by employing the contrastive learning paradigm. This involves training an extra encoder to differentiate various types of image degradations. Although AirNet [29] yields state-of-the-art results, it struggles to model fully disentangled representations of different corruption types. Furthermore, the usage of an additional encoder for contrastive learning leads to a higher training burden due to the two-stage training approach.

To overcome these challenges, in this paper, we present a prompt-learning-based approach to perform all-in-one image restoration (see Fig. 1). Our method utilizes prompts, which are a set of tunable parameters that encode crucial discriminative information about various types of image degradation (as shown in Fig. 2). By interacting prompts with the feature representations of the main restoration network, we dynamically enhance the representations with degradation-specific knowledge. This adaptation enables the network to effectively restore images by dynamically adjusting its behavior. The main highlights of our work include,

We present a prompting-based all-in-one blind restoration framework PromptIR that relies solely on the input image to recover a clean image, without requiring any prior knowledge of the degradation present in the image.

Our prompt block is a plug-in module that can be easily integrated into any existing restoration network. It consists of a prompt generation module (PGM) and a prompt interaction module (PIM). The goal of the prompt block is to generate input-conditioned prompts (via PGM) that are equipped with useful contextual information to guide the restoration network (with PIM) to effectively remove the corruption from the input image.

Hazy

Rainy

Noisy

Figure 2: The figure shows tSNE plots of the degradation embeddings used in PromptIR (ours) and the state-of-the-art AirNet [29]. Distinct colors denote different degradation types. In our case, the embeddings for each task are better clustered, showing the effectiveness of prompt tokens to learn discriminative degradation context that helps in restoration.

Our comprehensive experiments demonstrate the dynamic adaptation behavior of PromptIR by achieving state-of-the-art performance on various image restoration tasks, including image denoising, deraining, and dehazing using only a unified PromptIR model.

2 Related Works

Multi-degradation Image Restoration: While single degradation image restoration methods [68, 47, 12, 69, 45, 76, 55, 43, 74, 51] have received significant interest, multi-degradation image restoration is relatively under-explored in the literature. A body of work focuses on images corrupted due to multiple weather conditions e.g., snow, fog, and rain [37, 57, 32]. However, they train specific encoder or decoder parallel pathways for each weather degradation which requires knowing specific degradation type and is less scalable. Chen et al. [6] build a unified model for multiple restoration tasks, like super-resolution, denoising, and deraining, however, the model needs prior information about the corruption present in the input image as it uses a multi-head-tail architecture. In blind image restoration, we have no prior information on the degradation present in the image. This kind of problem setting has been tackled in the context of image super-resolution [73, 39, 10]. Li et al. [32] introduce a unified model for denoising, draining, and dehazing, which uses an image encoder trained through contrastive learning to model good representations of the degradation, which are later used to predict the deformable convolution offsets in another network to perform the restoration. This method requires two-stage training and the effectiveness of contrastive learning hinges on accurately choosing the positive-negative pairs and the amount of data available. In comparison, our work is focused on developing a single-stage training pipeline for unified all-in-one image restoration that is conceptually simpler and works as a drop-in module for multiple degradations.

Transformer-based restoration: Transformer [58] architectures have found great success across various computer vision tasks [25] such as image recognition [15, 54, 67], object detection [5, 80, 38] and semantic segmentation [64, 59, 78]. Owing to their strong feature representation capability, they are extended to image restoration tasks [7, 59, 56, 9]. However, naive self-attention has quadratic complexity w.r.t. the image size and this poses a challenge for image restoration tasks where inputs are typically high-resolution. To address this, some works have proposed efficient transformer architectures [33, 68, 35] to reduce the computational costs. Specifically, SwinIR [35] uses windowed self-attention blocks along with convolutional layers to improve the efficiency of the model. Restormer [68] uses multi-depth convolution head attention to reduce the number of operations. In this work, we apply our PromptIR to Restormer owing to its efficient design and high performance, however, our prompt block is generic and can work with other architectures.

Prompt learning: In natural language processing, prompting-based methods are means to provide incontext information to models to finetune them on a target task [3]. However, instead of using specific manual instruction sets as prompts, learnable prompts enable better parameter-efficient adaptation of models [79]. Prompt learning techniques can effectively model task-specific context hence they have been used for finetuning to vision tasks [23, 34, 26] and incremental learning [62, 49, 61]. Prompt learning-based techniques have also been applied in the case of multitask learning [20, 60], where choosing the right prompt for each task remains critical. All these approaches target high-level vision problems, however, our goal here is to develop a generic model for low-level vision that can dynamically restore inputs based on their interaction with the prompts. The prompts act as an adaptive lightweight module to encode degradation context across multiple scales in the restoration network.

3 Method

In "All-in-one" image restoration, we aim to learn a single model M to restore an image I from a degraded image ˜ I , that has been degraded using a degradation D , while having no prior information about D . While the model is initially "blind" to the nature of degradation, its performance in recovering a clean image can be enhanced by providing implicit contextual information about the type of degradation. In this paper, we present prompt learning-based image restoration framework PromptIR , shown in Fig. 3. Prompting is an efficient[23] and suitable[20] method for supplementing the model with relevant knowledge of the degradation type while recovering the clean image. The key element of PromptIR is the prompt blocks that first generate learnable prompt parameters, and then use these prompts to guide the model during the restoration process. Next, we describe the overall pipeline of our PromptIR framework and its components in detail.

xE

Transformer

Transformer

3×3

EXMXH

Fo

BlockL1

Skipconnections

Block(x2) L1

Input (I)

JXMXH

Restored (I)

PIM

PromptInteraction

Module

Transformer

BlockL2

Hx

X2C

2

Transformer

BlockL2

Prompt

Block

Components

Prompt

PGM

PromptGeneration

Module

Xx2C

3x3

3×3convolution

Transformer

1

4

Transformer

Prompt

Prompt

1×1

1 x1 convolution

BlockL3

BlockL3

Block

Components

Global Average

GAP

Pooling

HxW×4C

Softmax operation

HxW×8C

PIM:PromptInteractionModule

Wi

PromptComponent

Element-wise

Weight

Transformer

Block L4

F

PGM

PIM

↑

F

Transformer

Block

→

ExE

→

PromptBlock

P

Channel-wise

concatenation

Combination

Linear

Fi

GAP

ExE

Upsampling

Downsampling

PGM:Prompt

Prompt

GenerationModule

Components

Overall pipeline. From a given degraded input image I ∈ R H × W × 3 , PromptIR first extracts low-level features F 0 ∈ R H × W × C by applying a convolution operation; where H × W is the spatial resolution and C denotes the channels. Next, the feature embeddings F 0 undergo a 4-level hierarchical encoder-decoder, transforming into deep features F r ∈ R H × W × 2 C . Each level of the encoder-decoder employs several Transformer blocks, with the number of blocks gradually increasing from the top level to the bottom level to maintain computational efficiency. Starting from the high-resolution input, the goal of the encoder is to progressively reduce the spatial resolution while increasing channel capacity, thereby yielding low-resolution latent representation F l ∈ R H 8 × W 8 × 8 C . From the low-resolution latent features F l , the aim of the decoder is to gradually recover the highresolution clean output. In order to assist the decoding process, we incorporate prompt blocks in our PromptIR framework. Prompt blocks are adapter modules that sequentially connect every two levels of the decoder. At each decoder level, the prompt block implicitly enriches the input features with information about the degradation type for a guided recovery. Next, we describe the proposed prompt block and its core building modules in detail.

3.1 Prompt Block

In NLP [3, 48, 21, 34] and vision tasks [23, 26, 18, 50], prompting-based techniques have been explored for parameter-efficient finetuning of large frozen models trained on a source task S onto a target task T . The effective performance of prompting-based techniques is attributed to their ability to efficiently encode task-specific contextual information in prompt components. In the proposed PromptIR, prompt components are learnable parameters, that interact with the input features in order to enrich them with degradation type. Given N prompt-components P c ∈ R N × ˆ H × ˆ W × ˆ C and input features F l ∈ R ˆ H × ˆ W × ˆ C , the overall process of prompt block is defined as:

The prompt block consists of two key components: a prompt generation module (PGM) and a prompt-interaction module (PIM), each of which we describe next.

3.1.1 Prompt Generation Module (PGM)

Prompt components P c form a set of learnable parameters that interact with the incoming features to embed degradation information. One straightforward method for features-prompt interaction is to directly use the learned prompts to calibrate the features. However, such a static approach may yield suboptimal results, as it is agnostic to the input content. Therefore, we present PGM that dynamically predicts attention-based weights from the input features and apply them to prompt components to yield input-conditioned prompts P . Furthermore, PGM creates a shared space to facilitate correlated knowledge sharing among prompt components.

To generate prompt-weights from the input features F l , PGM first applies global average pooling (GAP) across spatial dimension to generate feature vector v ∈ R ˆ C . Next, we pass v through a channeldownscaling convolution layer to obtain a compact feature vector, followed by the softmax operation, thus yielding prompt-weights w ∈ R N . Finally, we use these weights to make adjustments in prompt components, followed by a 3 × 3 convolution layer. Overall, the PGM process is summarized as:

Since at inference time, it is necessary for the restoration network to be able to handle images of different resolutions, we cannot use the prompt components P c with a fixed size. Therefore, we apply the bilinear upsampling operation to upscale the prompt components to the same size as the incoming input features.

3.1.2 Prompt Interaction Module (PIM)

The primary goal of PIM is to enable interaction between the input features F l and prompts P for a guided restoration.

In PIM, we concatenate the generated prompts with the input features along the channel dimension. Next, we pass the concatenated representations through a Transformer block that exploits degradation information encoded in the prompts and transforms the input features.

The main contribution of this paper is the prompt block, which is a plug-in module, and architecture agnostic. Therefore, in the proposed PromptIR framework, we use an existing Transformer block [68], instead of developing a new one. The Transformer block is composed of two sequentially connected sub-modules: Multi-Dconv head transposed attention (MDTA), and Gated-Dconv feedforward network (GDFN). MDTA applies self-attention operation across channels rather than the spatial dimension and has linear complexity. The goal of GDFN is to transform features in a controlled manner, i.e., suppressing the less informative features and allowing only useful ones to propagate through the network. The overall process of PIM is:

where [ ; ] is concatenation operation. MDTA is formulated as Y = W p V · Softmax ( K · Q /α ) + X . Where X and Y are the input and output features. Q , K and V respectively represent query, key, and value projections that are obtained by applying 1 × 1 point-wise convolutions followed by 3 × 3 depthwise convolutions on the layer normalized input feature maps. W p is the point-wise convolution, α denotes a learnable scaling parameter, and ( · ) represents dot-product interaction. The process of GDFN is defined as Z = W 0 p ( ϕ ( W 1 d W 1 p ( LN ( Y ))) ⊙ W 2 d W 2 p ( LN ( Y )) ) + Y . Where, W ( · ) d is the 3 × 3 depth-wise convolution, ⊙ denotes element-wise multiplication, ϕ is the GELU non-linearity, and LN is the layer normalization [2]. The block diagram and additional details on the Transformer block are provided in the appendix.

4 Experiments

To demonstrate the effectiveness of the proposed PromptIR, we perform the evaluation on three representative image restoration tasks: image dehazing, image deraining, and image denoising. Following [29], we conduct experiments under two different experimental settings: (a) All-in-One, and (b) Single-task.

Input

AirNet

PromptIR

(Ours)

In the All-in-One setting, we train a unified model that can recover images across all three degradation types. Whereas, for the Single-task setting, we train separate models for different restoration tasks. The image quality scores for the best and second-best methods are highlighted and underlined in the tables.

Implementation Details. Our PromptIR framework is end-to-end trainable and requires no pretraining of any individual component. The architecture of our PromptIR consists of a 4-level encoder-decoder, with varying numbers of Transformer blocks at each level, specifically [4, 6, 6, 8] from level-1 to level-4.

We employ one prompt block between every two consecutive decoder levels, totaling 3 prompt blocks in the overall PromptIR network. The total number of prompt components are 5. The model is trained with a batch size of 32 in the all-in-one setting, and with a batch of 8 in the single-task setting. The network is optimized with an L 1 loss, and we use Adam optimizer ( β 1 = 0 . 9 , β 2 = 0 . 999 ) with learning rate 2 e -4 for 200 epochs. During training, we utilize cropped patches of size 128 x 128 as input, and to augment the training data, random horizontal and vertical flips are applied to the input images.

Datasets. We prepare datasets for different restoration tasks, following closely the prior work [29]. For image denoising in the single-task setting, we use a combined set of BSD400 [1] and WED [40] datasets for training. The BSD400 dataset contains 400 training images and the WED dataset has 4,744 images. From clean images of these datasets, we generate the noisy images by adding Gaussian noise with different noise levels σ ∈ { 15 , 25 , 50 } . Testing is performed on BSD68 [41] and Urban100 [22] datasets. For single-task image deraining, we use the Rain100L [65] dataset, which consists of 200 clean-rainy image pairs for training, and 100 pairs for testing. Finally, for image dehazing in the single-task setting, we utilize SOTS [31] dataset that contains 72,135 training images

Input

AirNet

PromptIR

(Ours)

Input

AirNet

PromptIR

(Ours)

and 500 testing images. Finally, to train a unified model under the all-in-one setting, we combine all 4 aforementioned datasets and train a single model that is later evaluated on multiple tasks.

4.1 Multiple Degradation All-in-One Results

We compare the proposed PromptIR with several general image restoration approaches [52, 17, 14, 71] as well as with specialized all-in-one methods [16, 29]. Results are reported in Table 1. When averaged across different restoration tasks, our algorithm yields 0 . 86 dB performance gain over the previous best method AirNet [29], and 1 . 89 dB over the second best approach DL [71]. Specifically, the proposed PromptIR significantly advances state-of-the-art by providing 2 . 64 dB PSNR improvement on the image dehazing task. The visual examples provided in Fig. 4 show that PromptIR effectively removes haze from the input images, and generates cleaner results than AirNet [29]. Similarly on the image deraining task as shown in Table 1, the proposed PromptIR achieves a substantial gain of 3 . 73 dB compared to DL [16] and 1 . 47 dB over AirNet [29]. Visual comparisons in Fig. 5 show that PromptIR is capable of removing rain streaks of various orientations and generates visually pleasant rain-free images. Finally, on the denoising task, our method provides 1 . 16 db boost over the DL algorithm [16] for a high noise level of σ =50 . Qualitative examples are presented in Fig. 6, where our method reproduces noise-free images with better structural fidelity than the AirNet algorithm [29].

4.2 Single Degradation One-by-One Results

In this section, we evaluate the performance of our PromptIR under the single-task setting, i.e., a separate model is trained for different restoration tasks. This is to show that content-adaptive prompting via prompt block is also useful for single-task networks. Table 2 presents dehazing results. It shows that our PromptIR achieves 8 . 13 dB improvement over AirNet [29], and 0 . 44 dB gain over the baseline method Restormer [68]. Similar trends can be observed for deraining and denoising tasks. For instance, when compared to the AirNet [29], our method yields performance gains of 2 . 13 dB on the deraining task (Table 3) and 0 . 51 dB on denoising task for noise level σ =50 on Urban100 dataset [22] (see Table 4).

4.3 Ablations Studies

Weperform several ablation experiments to demonstrate that our contributions in PromptIR framework provides quality improvements.

Impact of PGM. We carry out this ablation experiment on Rain100L [65] for deraining task. Table 5 shows that the prompt block in our PromptIR network brings performance gains of 0 . 3 dB over the baseline [68]. Further, it demonstrates that generating dynamic prompts conditioned on input content via PGM provides a favorable gain of 0 . 19 dB over the fixed prompt components.

Position of prompt blocks. In the hierarchical architecture of our PromptIR, we analyze where to place prompt blocks on the decoder side. Table 6 shows that using only one prompt block in the latent space degrades the network's performance. Whereas, incorporating prompt blocks between every consecutive level of the decoder performs the best.

Generalization to unseen degradation level. We take the model that is trained only on the noise levels σ ∈ { 15 , 25 , 50 } and test its performance on the unseen noise level of σ = 100 .

Table 7 shows that our PromptIR demonstrates significantly superior generalization capabilities compared to AirNet [29], yielding ∼ 7 dB PSNR difference.

Performance on spatially variant degradation. Here we evaluate PromptIR performance on images that are corrupted with varying degradations. For this, we follow closely the work of AirNet [29], and prepare a new test set from BSD68 [41] by applying Gaussian noise of varying levels σ = [0 , 15 , 25 , 50] at different spatial locations of the images. Results in Table 8 show that our PromptIR framework is more effective in restoring these images than AirNet [29], providing 0 . 23 dB improvement.

Training model with different combinations of degradation. In Table 1, we report the results of training an all-in-one model on combined datasets from all three restoration tasks. Here, we evaluate the impact on PromptIR performance by different combinations of degradation types (tasks). Table 9) shows that with an increasing number of degradation types, it becomes increasingly difficult for the network to restore images, thereby leading to a performance drop.

Specifically, the presence of hazy images in the combined dataset seems to negatively affect the model. Interestingly, a model trained on the combination of rainy and noisy images achieves good performance, indicating a positive correlation between the deraining and denoising tasks. Such phenomenon is also observed in the AirNet work [29].

Impact of Prompt Block. To effectively evaluate the benefit of the proposed framework, we evaluate it against the Restormer model [68]. Restormer is scaled up to match the parameter count of PromptIR. We perform this evaluation under the All-in-one setting. As shown in Table 10, the PromptIR framework provides an average improvement of 0 . 38 dB over the Restormer model [68].

Number of prompt components. We conduct an abalation study to understand the effect of the number of prompt components P c employed on the performance of the framework. As shown in Table 11, we find that utilizing more than 5 components yields marginal improvements while incurring additional computational overhead. As a result, we choose to utilize five components

5 Conclusion

Existing image restoration models based on deep neural networks can work for specific degradation types and do not generalize well to other degradations. However, practical settings demand the ability to handle multiple degradation types with a single unified model without resorting to degradationspecific models that lack generalization and require apriori knowledge of specific degradation in the input. To this end, our work proposed a drop-in prompt block that can interact with the input features to dynamically adjust the representations such that the restoration process is adapted for the relevant degradation. We demonstrated the utility of prompt block for all-in-one image restoration by integrating it within a SoTA restoration model that leads to significant improvements on image denoising, deraining, and dehazing tasks. In the future, we will extend the model for a broader set of corruptions toward the goal of universal models for better generalization in image restoration tasks.

References

Arbelaez, P., M. Maire, C. Fowlkes, and J. Malik (2011). Contour detection and hierarchical image segmentation. TPAMI .

Ba, J. L., J. R. Kiros, and G. E. Hinton (2016). Layer normalization. arXiv:1607.06450 .

Brown, T. B., B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. (2020). Language models are few-shot learners. arXiv:2005.14165 .

Cai, B., X. Xu, K. Jia, C. Qing, and D. Tao (2016). Dehazenet: An end-to-end system for single image haze removal. IEEE Transactions on Image Processing 25 (11), 5187-5198.

Carion, N., F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko (2020). End-to-end object detection with transformers. In ECCV .

Chen, H., Y. Wang, T. Guo, C. Xu, Y. Deng, Z. Liu, S. Ma, C. Xu, C. Xu, and W. Gao (2021a). Pre-trained image processing transformer. In CVPR , pp. 12299-12310.

Chen, H., Y. Wang, T. Guo, C. Xu, Y. Deng, Z. Liu, S. Ma, C. Xu, C. Xu, and W. Gao (2021b). Pre-trained image processing transformer. In CVPR .

Chen, L., X. Chu, X. Zhang, and J. Sun (2022). Simple baselines for image restoration. In ECCV .

Chen, X., X. Wang, J. Zhou, and C. Dong. Activating more pixels in image super-resolution transformer. arxiv 2022. arXiv preprint arXiv:2205.04437 .

Cornillere, V., A. Djelouah, W. Yifan, O. Sorkine-Hornung, and C. Schroers (2019). Blind image superresolution with spatially variant degradations. ACM Transactions on Graphics (TOG) 38 (6), 1-13.

Dabov, K., A. Foi, V. Katkovnik, and K. Egiazarian (2007). Color image denoising via sparse 3d collaborative filtering with grouping constraint in luminance-chrominance space. In 2007 IEEE International Conference on Image Processing , Volume 1, pp. I-313. IEEE.

Dong, H., J. Pan, L. Xiang, Z. Hu, X. Zhang, F. Wang, and M.-H. Yang (2020). Multi-scale boosted dehazing network with dense feature fusion. In CVPR .

Dong, W., L. Zhang, G. Shi, and X. Wu (2011). Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. TIP .

Dong, Y., Y. Liu, H. Zhang, S. Chen, and Y. Qiao (2020). Fd-gan: Generative adversarial networks with fusion-discriminator for single image dehazing. In Proceedings of the AAAI Conference on Artificial Intelligence , Volume 34, pp. 10729-10736.

Dosovitskiy, A., L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 .

Fan, Q., D. Chen, L. Yuan, G. Hua, N. Yu, and B. Chen (2019). A general decoupled learning framework for parameterized image operators. IEEE transactions on pattern analysis and machine intelligence 43 (1), 33-47.

Gao, H., X. Tao, X. Shen, and J. Jia (2019). Dynamic scene deblurring with parameter selective sharing and nested skip connections. In CVPR , pp. 3848-3856.

Gao, Y., X. Shi, Y. Zhu, H. Wang, Z. Tang, X. Zhou, M. Li, and D. N. Metaxas (2022). Visual prompt tuning for test-time domain adaptation. arXiv preprint arXiv:2210.04831 .

He, K., J. Sun, and X. Tang (2010). Single image haze removal using dark channel prior. TPAMI .

He, Y., S. Zheng, Y. Tay, J. Gupta, Y. Du, V. Aribandi, Z. Zhao, Y. Li, Z. Chen, D. Metzler, et al. (2022). Hyperprompt: Prompt-based task-conditioning of transformers. In ICML , pp. 8678-8690. PMLR.

Houlsby, N., A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly (2019). Parameter-efficient transfer learning for nlp. In ICML , pp. 2790-2799. PMLR.

Huang, J.-B., A. Singh, and N. Ahuja (2015). Single image super-resolution from transformed selfexemplars. In Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 51975206.

Jia, M., L. Tang, B.-C. Chen, C. Cardie, S. Belongie, B. Hariharan, and S.-N. Lim (2022). Visual prompt tuning. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXIII , pp. 709-727. Springer.

Jiang, K., Z. Wang, P. Yi, C. Chen, B. Huang, Y. Luo, J. Ma, and J. Jiang (2020). Multi-scale progressive fusion network for single image deraining. In CVPR , pp. 8346-8355.

Khan, S., M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan, and M. Shah (2022). Transformers in vision: A survey. ACM computing surveys (CSUR) 54 (10s), 1-41.

Khattak, M. U., H. Rasheed, M. Maaz, S. Khan, and F. S. Khan (2023). Maple: Multi-modal prompt learning. CVPR .

Kim, K. I. and Y. Kwon (2010). Single-image super-resolution using sparse regression and natural image prior. TPAMI .

Kopf, J., B. Neubert, B. Chen, M. Cohen, D. Cohen-Or, O. Deussen, M. Uyttendaele, and D. Lischinski (2008). Deep photo: Model-based photograph enhancement and viewing. ACM TOG .

Li, B., X. Liu, P. Hu, Z. Wu, J. Lv, and X. Peng (2022, June). All-in-one image restoration for unknown corruption. In CVPR , pp. 17452-17462.

Li, B., X. Peng, Z. Wang, J. Xu, and D. Feng (2017). Aod-net: All-in-one dehazing network. In ICCV , pp. 4770-4778.

Li, B., W. Ren, D. Fu, D. Tao, D. Feng, W. Zeng, and Z. Wang (2018). Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing 28 (1), 492-505.

Li, R., R. T. Tan, and L.-F. Cheong (2020). All in one bad weather removal using architectural search. In CVPR , pp. 3175-3185.

Li, W., X. Lu, J. Lu, X. Zhang, and J. Jia (2021). On efficient transformer and image pre-training for low-level vision. arXiv preprint arXiv:2112.10175 .

Li, X. L. and P. Liang (2021). Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190 .

Liang, J., J. Cao, G. Sun, K. Zhang, L. Van Gool, and R. Timofte (2021). SwinIR: Image restoration using swin transformer. In ICCV Workshops .

Liu, J., H. Wu, Y. Xie, Y. Qu, and L. Ma (2020). Trident dehazing network. In CVPR Workshops .

Liu, L., L. Xie, X. Zhang, S. Yuan, X. Chen, W. Zhou, H. Li, and Q. Tian (2022). Tape: Task-agnostic prior embedding for image restoration. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XVIII , pp. 447-464. Springer.

Liu, Z., Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo (2021). Swin transformer: Hierarchical vision transformer using shifted windows. arXiv:2103.14030 .

Luo, Z., Y. Huang, S. Li, L. Wang, and T. Tan (2022). Learning the degradation distribution for blind image super-resolution. In CVPR , pp. 6063-6072.

Ma, K., Z. Duanmu, Q. Wu, Z. Wang, H. Yong, H. Li, and L. Zhang (2016). Waterloo exploration database: New challenges for image quality assessment models. TIP .

Martin, D., C. Fowlkes, D. Tal, and J. Malik (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV .

Michaeli, T. and M. Irani (2013). Nonparametric blind super-resolution. In ICCV .

Nah, S., S. Son, J. Lee, and K. M. Lee (2022). Clean images are hard to reblur: Exploiting the ill-posed inverse task for dynamic scene deblurring. In ICLR .

Qu, Y., Y. Chen, J. Huang, and Y. Xie (2019). Enhanced pix2pix dehazing network. In CVPR , pp. 8160-8168.

Ren, C., X. He, C. Wang, and Z. Zhao (2021). Adaptive consistency prior based deep network for image denoising. In CVPR .

Ren, W., S. Liu, H. Zhang, J. Pan, X. Cao, and M.-H. Yang (2016). Single image dehazing via multi-scale convolutional neural networks. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14 , pp. 154-169. Springer.

Ren, W., J. Pan, H. Zhang, X. Cao, and M.-H. Yang (2020). Single image dehazing via multi-scale convolutional neural networks with holistic edges. IJCV .

Sanh, V., A. Webson, C. Raffel, S. H. Bach, L. Sutawika, Z. Alyafeai, A. Chaffin, A. Stiegler, T. L. Scao, A. Raja, et al. (2021). Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207 .

Smith, J. S., L. Karlinsky, V. Gutta, P. Cascante-Bonilla, D. Kim, A. Arbelle, R. Panda, R. Feris, and Z. Kira (2022). Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. arXiv preprint arXiv:2211.13218 .

Sohn, K., Y. Hao, J. Lezama, L. Polania, H. Chang, H. Zhang, I. Essa, and L. Jiang (2022). Visual prompt tuning for generative transfer learning. arXiv preprint arXiv:2210.00990 .

Song, Y., Z. He, H. Qian, and X. Du (2023). Vision transformers for single image dehazing. IEEE Transactions on Image Processing 32 , 1927-1941.

Tian, C., Y. Xu, and W. Zuo (2020). Image denoising using deep cnn with batch renormalization. Neural Networks .

Timofte, R., V. De Smet, and L. Van Gool (2013). Anchored neighborhood regression for fast examplebased super-resolution. In ICCV .

Touvron, H., M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou (2021). Training data-efficient image transformers & distillation through attention. In ICML .

Tsai, F.-J., Y .-T. Peng, C.-C. Tsai, Y .-Y . Lin, and C.-W. Lin (2022). BANet: A blur-aware attention network for dynamic scene deblurring. IEEE Transactions on Image Processing .

Tu, Z., H. Talebi, H. Zhang, F. Yang, P. Milanfar, A. Bovik, and Y. Li (2022). MAXIM: Multi-axis MLP for image processing. In CVPR , pp. 5769-5780.

Valanarasu, J. M. J., R. Yasarla, and V. M. Patel (2022). Transweather: Transformer-based restoration of images degraded by adverse weather conditions. In CVPR , pp. 2353-2363.

Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017). Attention is all you need. In NeurIPS .

Wang, Z., X. Cun, J. Bao, and J. Liu (2021). Uformer: A general u-shaped transformer for image restoration. arXiv:2106.03106 .

Wang, Z., R. Panda, L. Karlinsky, R. Feris, H. Sun, and Y. Kim (2023). Multitask prompt tuning enables parameter-efficient transfer learning. arXiv preprint arXiv:2303.02861 .

Wang, Z., Z. Zhang, S. Ebrahimi, R. Sun, H. Zhang, C.-Y. Lee, X. Ren, G. Su, V. Perot, J. Dy, et al. (2022). Dualprompt: Complementary prompting for rehearsal-free continual learning. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXVI , pp. 631-648. Springer.

Wang, Z., Z. Zhang, C.-Y. Lee, H. Zhang, R. Sun, X. Ren, G. Su, V. Perot, J. Dy, and T. Pfister (2022). Learning to prompt for continual learning. In CVPR , pp. 139-149.

Wei, W., D. Meng, Q. Zhao, Z. Xu, and Y. Wu (2019). Semi-supervised transfer learning for image rain removal. In CVPR .

Xie, E., W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo (2021). Segformer: Simple and efficient design for semantic segmentation with transformers. arXiv:2105.15203 .

Yang, F., H. Yang, J. Fu, H. Lu, and B. Guo (2020). Learning texture transformer network for image super-resolution. In CVPR .

Yasarla, R. and V. M. Patel (2019). Uncertainty guided multi-scale residual learning-using a cycle spinning cnn for single image de-raining. In CVPR .

Yuan, L., Y. Chen, T. Wang, W. Yu, Y. Shi, Z. Jiang, F. E. Tay, J. Feng, and S. Yan (2021). Tokens-to-token vit: Training vision transformers from scratch on imagenet. arXiv:2101.11986 .

Zamir, S. W., A. Arora, S. Khan, M. Hayat, F. S. Khan, and M.-H. Yang (2022). Restormer: Efficient transformer for high-resolution image restoration. In CVPR .

Zamir, S. W., A. Arora, S. Khan, M. Hayat, F. S. Khan, M.-H. Yang, and L. Shao (2020a). CycleISP: Real image restoration via improved data synthesis. In CVPR .

Zamir, S. W., A. Arora, S. Khan, M. Hayat, F. S. Khan, M.-H. Yang, and L. Shao (2020b). Learning enriched features for real image restoration and enhancement. In ECCV .

Zamir, S. W., A. Arora, S. Khan, M. Hayat, F. S. Khan, M.-H. Yang, and L. Shao (2021). Multi-stage progressive image restoration. In CVPR .

Zhang, H. and V. M. Patel (2018). Density-aware single image de-raining using a multi-stream dense network. In CVPR .

Zhang, K., J. Liang, L. Van Gool, and R. Timofte (2021). Designing a practical degradation model for deep blind image super-resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pp. 4791-4800.

Zhang, K., W. Luo, Y. Zhong, L. Ma, B. Stenger, W. Liu, and H. Li (2020). Deblurring by realistic blurring. In CVPR , pp. 2737-2746.

Zhang, K., W. Zuo, Y. Chen, D. Meng, and L. Zhang (2017). Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing 26 (7), 3142-3155.

Zhang, K., W. Zuo, S. Gu, and L. Zhang (2017). Learning deep CNN denoiser prior for image restoration. In CVPR .

Zhang, K., W. Zuo, and L. Zhang (2018). Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. IEEE Transactions on Image Processing 27 (9), 4608-4622.

Zheng, S., J. Lu, H. Zhao, X. Zhu, Z. Luo, Y. Wang, Y. Fu, J. Feng, T. Xiang, P. H. Torr, et al. (2021). Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. In CVPR .

Zhou, K., J. Yang, C. C. Loy, and Z. Liu (2022). Learning to prompt for vision-language models. International Journal of Computer Vision (IJCV) .

Zhu, X., W. Su, L. Lu, B. Li, X. Wang, and J. Dai (2020). Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 .

Appendix

A Additional Ablation Studies

We conduct further ablation studies to illustrate the effectiveness of various design choices of the PromptIR framework. We examine various key design choices like the usage of prompt tokens and plugging in prompt blocks only on the decoder branch of the network.

A.1 Contrastive learning-based Degradation Encoder embedding v/s Prompt Tokens

To strengthen the design rationale for incorporating prompts instead of following recent methods [29] that use embeddings learned through contrastive training, we replace the generated prompt from our PGM module with embeddings extracted from the Contrastive- learning based Degradation Encoder of the AirNet [29] model. We observed that the use of contrastive embeddings resulted in significantly weaker performance compared to prompt tokens. Moreover, achieving good performance with contrastive embeddings requires a custom-designed restoration network, whereas our Prompt Blocks can be seamlessly integrated as plug-and-play modules into any restoration network.

A.2 Prompt Blocks on both Encoder branch and Decoder branch

We study the importance of decoder-only prompting by evaluating the usage of prompt blocks on both the encoder and decoder branches. We show that it is important the prompt block is only used on the decoder side.

B Transformer Block in PromptIR Framework

As mentioned in section 3.1.2 of the main manuscript, we present the block diagramB.1 of the transformer block and further, elaborate on the details of the transformer block employed in the PromptIR framework. The transformer block follows the design and hyper-parameters outlined in [68]

To begin, the input features X ∈ R H l × W l × C l are passed through the MDTA module. In this module, the features are initially normalized using Layer normalization. Subsequently, a combination of 1 × 1 convolutions followed by 3 × 3 depth-wise convolutions are applied to project the features into Query ( Q ), Key ( K ), and Value ( V ) tensors. An essential characteristic of the MDTA module is its computation of attention across the channel dimensions, rather than the spatial dimensions. This effectively reduces the computational overhead. To achieve this channel-wise attention calculation, the Q and K projections are reshaped from H l × W l × C l to H l W l × C l and C l × H l W l respectively,

MDTA

Multi-Dconv head

Transposed Attention

GDFN

Gated Dconv Feed-

Forward Network

1×1

X

Dconv

Norm

Layer Normalization

Norm

HxWxyC

Z

Z

Dconv

3x3

3x3depth-wise

convolution(Wa)

HxW×C

→

1×1

x

Dconv

白白

1×1

+

1×1

1 x1 convolution(W,)

GDFN

b

Softmax operation

MDTA

1x1

Dconv

Q

HW×C

Element-wise

R

X

X

Transposed

Element-wise

multiplication

TransformerBlock

Norm

1×1

X

Dconv

K

Attention Map (A)

CxC

MatrixMultiplication

H×W×C

Reshape

1×1

Dconv

V

R

HW×C

(x)

×

+

HxWxC

before calculating dot-product, hence the resulting transposed attention map with the shape of C l × C l . Bias-free convolutions are utilized within this submodule. Furthermore, attention is computed in a multi-head manner in parallel.

After MDTA Module the features are processed through the GDFN module. In the GDFN module, the input features are expanded by a factor γ using 1 × 1 convolution and they are then passed through 3 × 3 convolutions. These operations are performed through two parallel paths and the output of one of the paths is activated using GeLU non-linearity. This activated feature map is then combined with the output of the other path using element-wise product.

C Qualitative results:

We present more qualitative results from single-task models to further elucidate the effectiveness of prompt-block even when under the single-task setting.

C.1 Dehazing

Input

AirNet

PromptIR

(sno)

Ground

Truth

C.2 Deraining

Input

AirNet

PromptIR

(sno)

Ground

Truth

C.3 Denoising

Noisy Image

Ground truth

AirNet

PromptIR(Ours)

Peer reviews:
comments: SUMMARY:
The paper proposes PromptIR, a prompt-based learning approach for all-in-one image restoration. The method utilizes prompts to encode degradation-specific information and dynamically guide the restoration network. It achieves state-of-the-art results on various image restoration tasks.

STRENGTHS:
(1) The proposed method offers a generic and efficient plugin module with a few lightweight prompts that can be used to restore images of various types and levels of degradation with no prior information of corruption.

(2) The method achieves state-of-the-art results in image denoising, deblurring, and dehazing.

(3) The paper is well-written and easy to follow.

WEAKNESSES:
(1) The paper lacks discussions on the limitations of the proposed method, such as the sensitivity of the results to the number of prompts used and how the method performs on combined types of degradation, such as noise + rain.

(2) The paper lacks the visualization of the learned prompt and an explanation of how and why it could work.

(3) The paper lacks comparisons on complexity, including runtime and parameter number.

QUESTIONS:
(1) Could you please provide more details on the sensitivity of the results to the number of prompts used and how to choose the optimal number of prompts?

(2) How does the Restorformer perform in the setting of Table 1?

(3) How about the perceptual metric comparison between this method and the baselines?

LIMITATIONS:
See the weakness and question part.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an all-in-one image restoration method that can restore images from various types and levels of degradation. Specifically, it utilizes prompts to encode degradation-specific information, which is used to dynamically guide the restoration network. Experimental evaluations on image denoising, deraining, and dehazing are conducted to demonstrate the effectiveness of the proposed method.\n\nSTRENGTHS:\nPrompts are used to encode degradation-specific information for all-in-one image restoration.\n\nExperimental evaluations and ablation studies are conducted to demonstrate the superior performance of the proposed method.\n\nWEAKNESSES:\nThere are several unclear statements listed as follows.\n\nQUESTIONS:\n1. What do the degradation embeddings used in Figure 2 refer to?\n2. What does the feature F_p in Figure 3 refer to?\n3. The key component of the proposed approach is the prompt block, which takes the prompt-components P_c and input features F_l as input. However, few information about P_c is provided. Please explain in detail how do the authors obtain the prompt-components P_c.\n4. Do the authors retrain all the competed methods using the same training dataset as the proposed method?\n5. For the single task setting, more recent and representative methods should be compared to demonstrate the effectiveness of the proposed method. For example,\n1) for image dehazing, \n      a) Song et al., Vision Transformers for Single Image Dehazing, IEEE TIP, 2023.\n      b) Tu et al., MAXIM: Multi-Axis MLP for Image Processing, CVPR 2022.\n2) for image deraining,\n      c) Chen et al., Pre-Trained Image Processing Transformer, CVPR 2021.\n6. To show the impact of PGM, it would be better to compare with a baseline that replaces all the prompt blocks in Figure 3 with the transformer blocks in Table 5.\n\nLIMITATIONS:\nThis paper does not claim the limitations and potential negative societal impact of their work. It would be better to discuss about the limitations in real applications and potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, authors study to restore images with unknown degradation types with a unified framework. To achieve it, a prompt-based block is proposed which generates suitable prompt first and then interact them with original features. Experiments show that combining the proposed block with restormer network can achieve new SOTA performance.\n\nSTRENGTHS:\n1. Prompt learning is recent a hot method in both language and vision domains. Different from previous works focus on high-level understanding tasks, this paper tempts to explore its application on low-level image restoration task.\n2. The writing is good and easy to understand.\n\nWEAKNESSES:\n1. Typos or confused explanations: \n     a. In Eq. 2, subscript of w should be the same with P in $\\sum_{c=1}^{N} w_{I}P_{c}$? And the subscript of w in the second formula should be removed?\n     b. Line 208, \"Table 1 Similarly on the image Deraining task, \".\n     c. Line 204, the second best approach is not DL in Table 1, is MRPNet. \n     d. Line 231-232, it mentions that using only one prompt block in the latent space (36.76 dB) degrades the performance. However, the baseline is 36.74 dB, which lower than 36.76 dB.\n2. Lack some comparisons. For example, the performance restormer is needed in Table 1. And other all-in-one weather image removing methods[1][2] are still very related to this work and needed to be compared. In addition, all results are in the synthesized images, how about its general performance on real world images (e.g., real rain images).\n3. Ablation study can be further improved. For example, the number of prompt is not studied. And the additional computational cost and model parameters should be discussed. Is the improvement gains come from the additional parameters?\n[1] Transweather: Transformer-based restoration of images degraded by adverse weather conditions;\n[2] Learning Multiple Adverse Weather Removal via Two-Stage Knowledge Learning and Multi-Contrastive Regularization: Toward a Unified Model;\n\nQUESTIONS:\nI think applying prompt technology into low-level image restoration task is very interesting, but this work seems suffer from many issues. Some of them are mentioned in the weaknesses. Still, there are some other questions as followed:\n\n1. As mentioned in the paper, the proposed prompt block can be used in various existing networks. However, no experiments are conducted. And the ablation studies in table 6 and table 2 (supplementary) show that different networks may require various block configurations.\n2. Why not use cross attention to interact learnable prompt with features? In [1], it seems work well. Thus, I suggest to compare this two ways in detail (the learnable parameters in [1] can be also regarded as prompt).\n[1] Transweather: Transformer-based restoration of images degraded by adverse weather conditions;\n\nLIMITATIONS:\nAuthors have not discussed the limitations. In my opinions, one of the potential limitations is the unproved generalization ability in terms of networks (e.g., utilization on AirNet) and data (e.g., real world images).", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper restores images from various types and levels of degradation. In detail, degradation information is encoded as prompts and used to guidance the restoration. Based on the classical UNet architecture for restoration, there are two added blocks: prompt generation module and prompt interaction module. With a single network, it achieves state-of-the-art performance on image denoising, deraining and dehazing.\n\nSTRENGTHS:\n1, A prompting-based single-model blind restoration framework.\n\n2, A plug-in module with prompt generation and prompt interaction, which guide the backbone model to remove corruptions effectively.\n\n3, State-of-the-art performance on multiple tasks with one single model\n\nWEAKNESSES:\n1, The ablation studies cannot prove this basic concern of the main contribution and intuition in this paper. Is the prompt (5 learned tensors as a dictionary) really working? What have the prompts learned? Do we really need the prompts? Since there are only 6 different types of degradations, I guess $softmax(Conv_{1\\times 1}(GAP(F_1)))$ learns a classification model to decide which degradation the input image has. If we directly inject this information to the UNet, it should work as well. This paper still follows the pipeline of \"degradation estimation + degradation-aware restoration\".\n\n2, Missing comparison on model size, runtime, FLOPs, etc. According to the model file sizes, I suspect there exists unfair comparison. The model size of PromptIR (file size: 388MB) is about 4 times of Restormer (file size: 99.9MB) and 11 times of Airnet (file size: 35.6MB). This means that the model might use 100MB for each sub-task and the rest 88MB for estimating the degradation type.\n\n3, For the single-task setting, for example, image denoising with sigma=50, what's the meaning of using prompt generation and interaction?\n\n4, Missing ablation studies on prompt design, such as number of prompts.\n\n5, Why adding prompt blocks at level 4 is nearly of no use (36.76 v.s. 36.74dB)? Note that it has more parameters in the prompt generation module and prompt interaction module. Besides, how about using level 2 only?\n\n6, In table 7 and 8, why PromptIR is significantly better than Airnet on unseen noise level, but is only slightly better on spatially variant degradation?\n\nQUESTIONS:\nSee weakness\n\nLIMITATIONS:\nSee weakness", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

comments: SUMMARY:
The paper proposes prompt based image restoration method that can handle different degradations. the proposed method uses prompts to encode degradation-specific information, which is then used to dynamically guide the restoration network. Given any instance, prompt generation module (PGM) and a prompt interaction module (PIM) are used generate input-conditioned prompts (via PGM) that are equipped with useful contextual information to guide the estoration network (with PIM) to effectively remove the corruption.

STRENGTHS:
- Paper is well organized 
- PGM, and PIM modules are used to generate the prompts and recover the image. Proposed method is clearly explained

WEAKNESSES:
- Can the proposed method method handle degradations like snow, night, blur, and turbulence conditions, since the proposed method mentions it is generic image restoration.
- experiments are limited, since the experiments showed in the paper covered only haze, rain, and noise, specifically these the datasets used for comparison are synthetic and very small datasets. it is hard understand the limits of the proposed method from these experiments
- Additionally the experiments performed contains only single degradation at any instance, can the proposed method handle multiple degradations in the single input image
- Can authors performs the experiments showing if the wrong prompt was propagated (instead of the prompt from the PGM) to understand the importance of PGM module.

QUESTIONS:
Please refer weaknesses

LIMITATIONS:
Limitations of the proposed method were not discussed
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-09