AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation
Authors: Yuning Cui
Syed Waqas Zamir
Salman Khan
Alois Knoll
Mubarak Shah
Fahad Shahbaz Khan
Venue: ICLR 2025 Poster (2025)
Keywords: All-in-one image restoration
frequency mining
frequency modulation
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: In the image acquisition process, various forms of degradation, including noise, blur, haze, and rain, are frequently introduced. These degradations typically arise from the inherent limitations of cameras or unfavorable ambient conditions. To recover clean images from their degraded versions, numerous specialized restoration methods have been developed, each targeting a specific type of degradation. Recently, all-in-one algorithms have garnered significant attention by addressing different types of degradations within a single model without requiring the prior information of the input degradation type. However, most methods purely operate in the spatial domain and do not delve into the distinct frequency variations inherent to different degradation types. To address this gap, we propose an adaptive all-in-one image restoration network based on frequency mining and modulation. Our approach is motivated by the observation that different degradation types impact the image content on different frequency subbands, thereby requiring different treatments for each restoration task. Specifically, we first mine low- and high-frequency information from the input features, guided by the adaptively decoupled spectra of the degraded image. The extracted features are then modulated by a bidirectional operator to facilitate interactions between different frequency components. Finally, the modulated features are merged into the original input for a progressively guided restoration. With this approach, the model achieves adaptive reconstruction by accentuating the informative frequency subbands according to different input degradations. Extensive experiments demonstrate that the proposed method, AdaIR, achieves state-of-the-art performance on different image restoration tasks, including image denoising, dehazing, deraining, motion deblurring, and low-light image enhancement. The code is available at https://github.com/c-yn/AdaIR.

Published as a conference paper at ICLR 2025
ADAIR: ADAPTIVE ALL-IN-ONE IMAGE RESTORATION VIA FREQUENCY MINING AND MODULATION
Yuning Cui", Syed Waqas Zamir?,
Salman 3han
Alois Knoll', Mubarak Shah?,
Fahad Shahbaz Khan 3.6
Technical University of Munich
2 Inception Institute of Artificial Intelligence
3 "Mohammed Bin Zayed University of AI 5 University Central Florida
Australian National University
6 LLnkööping University
ABSTRACT
the image acquisition process, various forms of degradation, including noise, blur, haze, and rain, are frequently introduced. These degradations typically arise from the inherent limitations of cameras or unfavorable ambient conditions. To recover clean images from their degraded versions, numerous specialized restora- tion methods have been developed, each targeting a specific type of degradation. Recently, all-in-one algorithms have garnered significant attention by addressing different types of degradations within a single model without requiring the prior information of the input degradation type. However, most methods purely operate in the spatial domain and do not delve into the distinct frequency variations inherent different degradation types. To address this gap, we propose an adaptive all-in- one image restoration network based on frequency mining and modulation. Our approach is motivated by the observation that different degradation types impact the image content on different frequency subbands, thereby requiring different treat- ments for each restoration task. Specifically, we first mine low- and high-frequency information from the input features, guided by the adaptively decoupled spectra of the degraded image. The extracted features are then modulated by a bidirectional operator to facilitate interactions between different frequency components. Finally, the modulated features are merged into the original input for a progressively guided restoration. With this approach, the model achieves adaptive reconstruction by accentuating the informative frequency subbands according to different input degra- dations. Extensive experiments demonstrate that the proposed method, AdaIR, achieves state-of-the-art performance on different image restoration tasks, includ- ing image denoising, dehazing, deraining, motion deblurring, and low-light image enhancement, The code is available at wittttt::/// /tl:::/gittin /tttttt...thggggg/gg com/ c-yn Adw.nIII
INTRODUCTION
Image restoration is the task of generating a clean image by removing degradations (e.g., noise, haze, blur, rain) from the original input (Zamir et al,, 2021 Cui et al. 2024c). It serves as a vital component in numerous downstream applications across diverse domains, including image/video content creation, surveillance, medical imaging, and remote sensing, Given its inherently ill-posed nature, effective image restoration demands learning strong image priors from large-scale data, To this end, deep neural network- image restoration approaches (Zamir et al., 2020a Tsai et al.)
2022b Cui al. 2023c c Wang et al.) [2024b] Liu et al. (0225) have emerged as preferable choices over the conventional handcrafted algorithms (He et al. 2010 Kim & Kwon 2010; Michaeli &
Trani 2013). Deep-learingg methods learn image priors either implicitly from data (Ren et al. 2021
Nah et al.] 2022 Dong et al.] (020a), or explicitly by incorporating task specifcc knowledge into the network architectures (Tu et al.) 2022: Wang et al. 2022 Zamir et al. [002aa] 2020b Cui et al.. 2024b). Despite promising results on individual restoration tasks, these approaches are either not generalizable beyond the specific degradation types and levels which hinders their broader application, require training separate copies of the same network on different degradation types, which is computationally expensive and tedious procedure, and maybe infeasible solution for deployment on resource-constraiit edge-devices. Therefore, there need to develop all-in-one image restoration

---

Published as a conference paper at ICLR 2025
Lrw Danarrg
10
Dircie andi
Dancllly
20-
606 1AM 37m
Low-Light
Dehazing
Deraining
Deblurring
Demoising (15) Denoising (25) Denoising
Figure I: Left, from top to bottom: degraded images, ground truth, and Fourier spectra of residual images obtained by subtracting the degraded images from the ground truth. The images obtained from LOL-v1 (Wei et al., 2018), SOTS (Li et al. 2018). Rain 100L (Li et al.) 2018), GoPro (Nah) et al. (017), and BSD68 (Martin et al. 2001) with different noise factors, respectively. Right, the sub-graph illustrates the mean values of Fourier spectra on the square of length shown on the x-axis, across five tasks. The spectra all resized to 320 x 320 for comparisons. As seen, different tasks pay different attention to different frequency subbands. For example, there are larger discrepancies in low frequency between degraded and target pairs of the low-light image enhancement and dehazing datasets. In contrast, the frequency differences are generally evenly distributed for image denoising. method that can handle images with different degradation types, without requiring prior information regarding the corruption present in the input images.
Recently, an increasing number of at- tempts have been made (Ma et al., 2023 Shi et al.] 20244; Gao al. 2023 Cui et al. 2024d) to address multiple degradations with sin- gle model. These include using degradation-aware encoder in the restoration network learned via con- trastive learning paradigm (Li et al., 2022); designing a two-stage frame- work IDR (Zhang et al., 2023). where
Don
Defaze
AirNet
U-WADN
PromptR
Ours
Figure 2: The t-SNE results of intermediate features produced by the three-task all-in-one models. Our model is better at learning discriminative degradation contexts.
the first stage is dedicated to task-oriented knowledge collection based on underlying physics charac- teristics of degradation types, and the second stage responsible for ingredients-oriented knowledge integration that progressively restores the image; or developing prompt-learning strategies (Potlapalli
ctt al. 2023 Ma et al. 2023) inspired from their success in the natural language processing (Brown 2020 Nonetheless, most existing approaches purely operate in the spatial domain and do not consider frequency information. However, as shown in Fig. we observe that different degradations may impact the image content different frequency subbands. For instance, and rainy images are contaminated with high-freque frequency content, while on the other hand, low-light and hazy images dominated by owwfrreueecyy degraded content, thus indicating the need treat each restoration task on its own merits.
al.
the one hand, noisy
In this paper, propose an adaptive all-in-one image restoration framework based frequency mining and modulation. Specifically, the frequency mining module extracts different frequency signals from the input features, guided by an adaptive spectra decomposition of the degraded input image. The extracted features are then refined using a bidirectional module, which facilitates the interactions between different frequency components by exchanging complementary information. Finally, these modulated features are used to transform the original input features via an efficient transposed cross-attention mechanism. With the proposed key design choices, our method can learn discriminative degradation context more effectively than other competing approaches, shown in Fig. Overall, the following are the main contributions of our work.
We propose an adaptive all-in-one image restoration framework that leverages both spatial and frequency domain information to effectively decouple degradations from the desired clean image content.
We introduce the Adaptive Frequency Learning Block (AFLB), which is a plugin block specifically designed for easy integration into existing image restoration architectures. The AFLB performs two sequential tasks: firstly, through its Frequency Mining Module (FMiM), it generates low- and high-frequency feature maps via guidance obtained from the spectra
2

---

Published as a conference paper at ICLR 2025
decomposition of the original degraded image; secondly, the Frequency Modulation Module (FMoM) within the AFLB calibrates these features by enabling the exchange of information across different frequency bands to effectively handle diverse types of image degradations.
Extensive experiments demonstrate that our AdaIR algorithm sets new state-of-the-ar performance on several all-in-one image restoration tasks, including image denoising, dehazing, deraining, motion deblurring, and low-light image enhancement.
2 RELATED WORK
Single-Task Taage Image Restoration. Image restoration aims to reconstruct a clean image from its degraded counterpart. Since it is a highly ill-posed problem, many conventional methods have been proposed that utilize hand-crafted features to reduce the solution space (Berman et al. [0166] He et al. 2010). Such solutions, though perform well on some datasets, may not generalize well to complicated real-world images (Zhang al., 2022). Recently, with the rapid advancements in deep learning, great number of convolutional neural network (CNN) based methods have been proposed and attained superior performance over traditional methods various image restoration tasks, such as image denoising (Zhagg et al.
2017a: 2017a;2018) dehazing (Qin et al. [2200] Cui et al. 2024a), deraining (Jiang 2019). and motion deblurring (Cho et al. 2021 Cui et al., 2023f). For
et al., 2020; Ren et al.
example, the recent UHDVD obtains one of the best reconstruction accuracies when tested on UHD
2024. To model long-range dependencies, Transformer models have been introduced to low-level tasks and significantly advanced state-of-the-ard performance (Guo et al. Tsai et al. 2022ap. Despite the obtained promising performance, these task-specific methods lack generalization beyond certain degradation types and levels. For general image restoration, several network design- based approaches are proposed, which perform favorably on different restoration tasks (Wang et al. [202]] Liang et al. [222]] Zamir et al. (022a)). Although these networks demonstrate robust performance on various restoration tasks, they require training separate copies on different datasets and tasks. Furthermore, applying a separate model for each possible degradation is resource- intensive, and often impractical for deployment, especially on edge devices.
blurry frames (Ren et al.
2022
All-in-One Image Restoration. All-in-one image restoration methods address numerous degrada- tions within a single model (Yang et al. 2024 Jiang et al., 2023 Chen & Peil (023). Early unified models (Chen et al.] 2021b] Li et al. (2020) employ distinct encoder and decoder heads to attend to different restoration tasks. However, these non-blind methods need prior knowledge about the degradation involved in the corrupted image in order channelize it to the relevant restoration head. To achieve blind all-in-one restoration, AirNet (Li et al. 2022) learns the degradation representation from the corrupted images using contrastive learning, and the learned representation is then used to restore the clean image. The subsequent method, IDR (Zhang et al., (023), models different degradations depending on the underlying physics principles and achieves all-in-one image restora- tion in two stages. Zhu et al. (Zhu et al. 2023) formulates an efficient unified model by learning weather-general and weather-specifh features in two stages. Recently, several prompt-learning-- based schemes have been proposed (Ma et al. 2023 Conde et al. 2024 Ai et al. 2024). For instance, PromptIR (Potlapalli et al. 2023) presets a series of tunable prompts to encode discriminative infor- mation about degradation types, which involve a large number of parameters. Different from most methods, which operate only in the spatial domain (Park et al. 2023). ). this paper presents an all-in-one image restoration algorithm that exploits information both in spatial and frequency domains.
Frequency Networks. Frequency processing has become a prevalent technique in the field of image restoration. For example, several works (Zhou et al., 2024 Cui et al. 2023d b) employ adaptive convolutions and softmax mechanisms to decouple features. However, these methods operate exclusively in the spatial domain, limiting their ability to capture a broad spectrum of frequencies and diminishing their effectiveness in frequency learning. Furthermore, their use of concatenation or channel attention for frequency interactions fails to exploit the unique properties of different frequency bands. Other approaches (Kong et al., 2023: Mao et al., 2023; 2024) leverage frequency transformation techniques, such as Fourier or Wavelet transforms, to map spatial features into the frequency domain, followed by convolutions or learnable parameters for spectral refinement. However, these methods lack explicit frequency interactions, and their parameters remain fixed after training. hindering adaptability to diverse degradation types. Zheng et al (Zheng et al. 2021 employ a deep CNN block to learn bandpass filters for image demoireing. In the context of all- in-one image restoration, a few methods (Gao et al. 2023] Shi et al.) 2024) employ manual or non non-adaptive approaches for feature separation and execute frequency interactions without accounting for the distinct characteristics of different frequency components. Unlike the above algorithms, our
3

---

Published as a conference paper at ICLR 2025
Skip Connection
21
Conv 3x33
Ya
T8
FMMM
TB
AFLB
TB
Conv 3x33
Manc
Degraded (1) (irttng
Restored (1)
(a) AdaIR
Globall
Global
Downsemple
Upsample
1B
Transformer
X
Matrix Muttplication
Element-wise Addition
Element-wisse
Mullipication
SAF
Average
Pooling.
GMP
Pooling
Max
Doony Depthwise comy
(d) CA
Cony xx
Doonv
Come 3x3
Mj
Conv 1x
Conv 1x1
Deanw 3x3
Softmax
Mi
L+H
X
x
Mask
X2
Conv 1x
Conv 1x7
X
Doonw 3x3
CA
X
(b) FMIM
(c) FMMM
MGB
( H-L
Conv 1x1
Conv
1x1
Exte
(g) L-H
GAP
Dxter
Cony 1x1
Conv 1x1
Sigmooi
X
CoTV
7K7
Sigmon
matro Ap-1
RoLO
Sigmoid
GELU
GMP
Corv
Conv
Figure 3: (a) The overall pipeline of AdaIR. is Transformer-based encoder-decoder architecture, employing TB (Zamir et al. (222a)) and novel Adaptive Frequency Learning Blocks (AFLB). Each AFLB contains (b) Frequency Mining Module (FMiM) that extracts different frequency components from input features guided by the adaptively decoupled spectra of the degraded input image, and (c) Frequency Modulation Module (FMoM) that exchanges the complementary information between different frequency features. (d) Cross Attention (CA) (Zamir et al.) 2022a). (e) Mask Generation Block (MGB) that yields frequency boundary for spectra decomposition. (f) H-L unit (Woo
et al. 2018) delivers high-frequency attention maps to enrich Low-frequency features. (g) L-H unit enhances frequeryy features by complementing them with low-frequency features. FFT and IFFT denote the Fast Fourier Transform and its inverse operator, respectively.
approach explicitly operates the frequency domain and realizes adaptability to various degradations. Furthermore, employ distinct attention mechanisms to facilitate frequency interactions, leveraging the unique characteristics different frequency bands to enable more effective frequency learning.
3 METHOD
3.1
OVERALL PIPELINE
Fig. 3 presents the pipeline of AdaIR. The overall goal of our AdaIR framework is to learn a unified model M that recover clean image i from degraded image I, without any prior information of degradation type present in the input image Formally, given a degraded image E H x3 AdaIR first extracts shallow features Yo IR. XxXXC using 3 x 3 convolution; where H x W denotes the spatial size and C' represents the number of channels. Next, these features Yo are processed through 4-level encoder-decoder network. Each level of the encoder employs multiple Transformer blocks (TBs) (Zamir al. 2022a), where the number of blocks gradually increases from the top level the bottom level, facilitating a computationally efficient design. The encoder takes high-resolution features Y, as input, and progressively transforms them into lower-resolution latent 4 R xSC On the decoder side, the latent features Y, are processed with
representation Y; E interleaved Adaptive Frequency Learning Block (AFLB) and TBs to progressively reconstruct high- resolution clean output. Particularly, between every two levels of the decoder, adaptively segregates the degradation content from the clean image content in the frequency domain, and subsequently assists in refining features in the spatial domain for effective image restoration.
insert the AFLB that
4

---

Published as a conference paper at ICLR 2025
Since different types of degradations affect image content different frequency bands (as shown in Fig. D). we specifically design the Adaptive Frequency Learning Block (AFLB) that extracts low- and frrqueerreneny components from the input features and then modulate them to accentuate the corresponding informative subbands for each degradation. Next, we describe the two key components of AFLB: (1) Frequency Mining Module (FMiM) and Frequency Modulation Module (FMoM).
3.2 FREQUENCY MINING MODULE (FMIM)
As shown in Fig. B)) given as inputs both the degraded image and the intermediate features X x XC, FMiM mines different frequency representations from X with the guidance of adaptively decoupled spectra of I. Primarily, FMiM consists of three steps, i.e., domain transformation, mask generation, and feature extraction.
For the domain transformation. FMiM applies a 3 x convolution on the degraded image expand the channel capacity to align with that of the input features into spectral domain representation
X. These output features are transformed IR H xxxWxx xC via the Fast Fourier Transform (FFT).
Since we want to adaptively extract different frequency parts from the input features X, we design lightweight Mask Generation Block (MGB) to generate a 2D mask that serves as frequency boundary to separate the spectra of input image I. The cutoff frequency boundary adaptively changes according to the type of degradation present in the image. As illustrated Fig. (Jj). the projected feature map Pi is first mapped into vector using global average pooling operator and then passes through two x 1 convolution layers with the GELU activation function in between to produce two factors ranging from 0 to 1, which define the mask size by multiplying with the width and height of the spectra. The mask generation process can be formally expressed as:
|a, = 5 (W2 (o (W1x1 (GAP, (P))))
(I)
where GAP, denotes spatial global average pooling, o is the GELU activation, and ō indicates the Sigmoid function. The convolution W₁ and W₂, have the reduction ratios of r1 and 3r. respectively,
progressively downsampling the channel dimensions to 2. Subsequently, the binary mask M. E {0, 1] W for extracting low frequency can be obtained by setting M H -a % # +a K
H W 3
W 3 = 1, where k is set to a value of 128, as the curve junction is relatively small in Fig. k
2
Accordingly, the mask for high frequeney M, is obtained by setting the values within the remaining region as 1. Subsequently, we can obtain the adaptively decoupled features by applying the learned masks the spectra via element-wise multiplication and using the inverse Fourier transform.
Next, we adapt the multi-dconv head transposed cross attention (Fig. d)) (Zamir et al. 2022a Chen et al. 2021a) a) mine the different feature parts from the input features with the guidance of F, and Fn. Overall, the feature extraction process is defined as:
X, softmax (QK /a) V,
where,
Q DW, (F.x)((F)).
DW2 (Wix1 (X)) v= DW3 (W!xl(X) where, = (M, ① F),
where (1, hJ is an indicator for low/high frequency, DW represents 3 x 3 depth-wise convolu- tion, 3 is element-wise multiplication, FF-1 indicates the inverse fast Fourier transform, Q. K and V are query, key and value projections, respectively, which are separately generated with sequential application of x 1 convolution and 3 x depth-wise convolution, and a is a learnable scaling factor to control the magnitude of the dot product result of Q and K before using the softmax function.
3.3 FREQUENCY MODULATION MODULE (FMOM)
We devise FMoM to facilitate the cross interaction between low-frequency mined features and high- frequency mined features (see Fig. c)). The goal is to cross complement one type of mined features with the other. For instance, high-frequency features contain edges and fine texture details, and thus we use this information to enrich low-frequency mined features via a super-lightweight spatial attention unit (H-L) (Fig. f)). Similarly, the global information present in freeuency features is passed to the high-frequency branch through the channel attention unit (L-H), illustrated in Fig. g
This unit computes the spatial attention map from high-frequency mined features that used to complement features of the low oowffreuueccy branch. The H-L unit (Woo et al.) 2018) uses two different channel-wis pooling techniques in parallel to produce two single-channel spatial feature
H-L Unit:
5

---

Published as a conference paper at ICLR 2025
maps, each of size H x W x 1. These maps then concatenated along the channel dimension. The concatenated features are further refined with a 7 x 7 convolution, followed by a sigmoid operation to generate the final spatial attention map, which is then used to obtain the modulated low- wwwfrequency features via deieentwise multiplication. Overall, the process of the H-L Unit is given by:
X = Xi 10AHHLL AH- 8 (X.) ((G)AP(X.,,GMMM(XX)) GM,)(XX)))..
where,
(5)
(6)
where We has a channel reduction ratio of 2. 8 is the sigmoid function. GAP, and GMP, are the channel-wise global average pooling and max pooling, respectively. [s]] indicates concatenation.
L-H Unit: It is a dual branch module that processes incoming low-frequency mined features, yielding feature descriptor that is subsequently used to attend to the high- high-freueeccy mined features. Specifically, given the mined low-frequency features X, E "xxxx xC the top branch of the L-H unit applies global average pooling along spatial dimension obtain a feature vector of size 1 x C, followed by two convolutional layers with the ReLU activation function in between. The bottom branch of the L-H unit employs the same structure, with the only difference of Max pooling at the head. The results of the two branches are added together, on which the sigmoid function is applied to produce the final attention descriptor Ar.- E RIXIxC which used to modulate the mined features X,. The process of the L-H Unit is expressed by:
X = X,, C AL-H, where, A - 5 (1 (W2 (GAP, + W (((W3'((GM) (GMP,(X.)))
(8)
where 5 is the sigmoid function, X, is the modulated high-frequency features, GAP, and GMP, represent the global average pooling and max pooling along the spatial dimensions, respectively. ? indicates the ReLU activation function. W7 and Wg have a reduction ratio of r2 for the channel adjustment, while Ws s and W1o have an increasing ratio of r2. The parameters are shared among W; and Wg. Ws and Woo for computational efficiency.
Subsequently, the modulated high-frequency features X,, and low-frequency features X, are aggre- gated and processed via a x 1 convolution to obtain Xm. which is merged into the original input X using the cross-attention unit, where the query Q tensor is produced from X while Xm yields the key K and value V tensors. By using FMiM and FMoM, the high-frequency and low-frequency contents of the input features are separately and adaptively modulated according the degradation type present in the corrupted input image, leading to adaptive all-in-one image restoration.
4 EXPERIMENTS
To validate the efficacy of the proposed AdaIR, we conduct experiments by strictly following previous state-of-the-art works (Potlapalli et al., 2023 Li al. 2022) under two different settings: (1) All- in-One, and (2) Single-task. In the All-in-One setting, a unified model is trained to perform image restoration across multiple degradation types. Whereas, within the Single-task setting, separate models are trained for each specific restoration task. We provide single-task results, additional ablation experiments, visual examples, and more details on the architecture the Appendix. In tables, the best and beest image fidelity scores (PSNR and SSIM (Wang et al. 2004)) are highlighted in red and blue, respectively.
Implementation Details. Our AdaIR presents an end-to-end trainable solution without the necessity for pretraining any individual component. The architecture of AdaIR employs a 4-level encoder- decoder structure, with varying numbers of Transformer blocks (TB) at each level, specifically [4, 6, 6. 8] from level-1 to level-4. We integrate one AFLB block between every two consecutive decoder levels, amounting to a total of three AFLBs in the overall network.
For training, we adopt a batch size of 32 in the all-in-one setting and a batch size of in the single- task setting. The network optimization is achieved through an LI loss function, employing the Adam optimizer (31 = 0.9 and 32 = 0.999), with a learning rate of 2e -4, over the course of 150 epochs. During the training process, cropped patches sized at 128 128 pixels are provided input, with additional augmentation applied via random horizontal and vertical flips. All experiments are conducted on NVIDIA Tesla A100 40G GPUs using Py PyToch
Datasets, In preparing datasets for training and testing, we closely follow prior works (Potlapalii Li et al. 2022). For single-task image dehazing, we use one of the first standard dehazing
al. 2023

---

Published as a conference paper at ICLR 2025
Table 1: Comparisons under the three-degradation all-in-one setting: unified model is trained combined set of images obtained from all degradation types and levels. On Rain (Yang et al. 2019 for image deraining, AdaIR yields 0.7 dB gain over ArtPrompdR (Wu al, 2024).
Dehazing on SOTS
Deraining on Rain
Denoising on BSD68
Method
-5
50
Average
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
Params
BRONer (Tian al
2020
23.23
0.895
27.42
0.895
32.26
0.898
29.76
0.836
26.34
0,693
27.80
0.843
1.11M
LPNer
0,828
26.47
24.77
0.552
23.64
0.738
2.84M
FDGAN
Doon ot 2006
24.71
0.929
29.89
0.933
30.25
0.910
28.81
0.368
26.43
0.776
28.02
0.883
MPRNet
Zamsr ef mi 2021
25.28
0.955
33.57
0.954
33.54
0.927
30.89
0.880
27.56
0.779
50.17
0.899
20
26.92
0.931
32.62
0.931
33.05
0.914
30.41
0.861
26.90
0.740
29.98
0.876
2,05M
27.94
0.962
0.968
31,26
28.00
0.910
Restommr
2022
27.78
0.958
33.78
0.958
33.72
0.930
30.67
0.365
27.63
0.792
30.75
0.901
26.13M
Prompilk
30.58
0.974
36,37
0.972
33.98
0.933
31.31
0.888
28.06
0.799
32.06
0.913
M2.96M
U-WADN
29.21
0.971
35.36
0.968
33.73
0.931
31.14
0.885
27.92
0.793
31,47
0.910
6M
Aflrseeu
Wardtal
0.979
0.982
31.06
0.934
31.42
28.11
0.801
0.917
AdalR (Oues)
31.06
0,980
38.61
0.983
31,12
0.935
31.15
0.892
28.19
0.802
32.69
0
28.77M
7.84 dB
23.09 dB
25.30 dB
30.80 dB
PSNR
10.82 dB Input
27.49 dB AirNet
28.75 dB PrompuR
31.68 dB AdaIR
PSNR Reference
2018 between all-in-one methods. Compared to other algorithms, our method is more effective in haze removal.
Figure Image dehazing comparisons on SOTS (Li al.,
datasets, SOTS et al. 2018), which comprises 135 training images and 500 testing images. For single-task image deraining, we utilize Rain 100L (Yang et al. 2019), which contains 200 clean-rainy image pairs for training and 100 pairs for testing. For single-task image denoising, we combine images of BSD400 (Arbelaez et al., 2010) and WED (a ct al. 2016) datasets for model training; the BSD400 encompasses 400 training images, while the WED dataset consists of 4.744 images. Starting from these clean images of BSD400 (Arbelaez et al. 2010) and WED (Ma et al. 2016). we generate their corresponding noisy versions by adding Gaussian noise with varying levels (a (15, 25, 50)). Denoising task evaluation is performed on BSD68 (Martin et al. 2001) and Urban 100 (Huang et al., 2015). Finally, under the all-in-one setting, we train single model on the combined set of the aforementioned training datasets and directly test it across multiple restoration tasks.
4.1 ALL-IN-ONE RESULTS: THREE DISTINCT DEGRADATIONS
We evaluate the performance of our all-in-one AdaIR on three different restoration tasks, including image dehazing, deraining, and denoising. We compare AdaIR against various general image restoration methods (BRDNet (Tian et al. 2020). LPNet (Gao et al. 2019). FDGAN (Dong et al. 2020b), MPRNet (Zamir al., [2021) ), and Restormer (Zamir et al.) (022aa). ). as well as specialized all- in-one approaches (DL (Fan et al. 2019), AirNet (Li et al. 2022). PromptIR (Potlapalli et al. 2023). U-WADN (Xu al.) 22024), and ArtprempuR (Wu et al.) 2024)). Table T shows that AdaIR provides consistent pertormance gain over the other competing approaches. When averaged across various restoration tasks and settings, our AdaIR obtains 0.2 dB PSNR gain over the recent best method ArtprompdR (Wu et al.] 2024), and 0.63 dB improvement over the recent algorithm PromptlR (Potlapalli) et al., 2023). Specifically, compared to ArtprompdR (Wu et al. 2024), AdaIR yields a substantial boost 0.7 dB on the deraining task, and 0.23 dB on the dehazing task. We provide visual examples in Fig. for dehazing, Fig. 5 for deraining, and Fig. 6 for denoising. These examples show that our AdaIR is effective in removing degradations, and generates images that are visually closer to the ground truth than those of the other approaches (Potlapalli et al. 2023 Li et al., (022). Particularly, in the restored images, our method preserves better structural fidelity and fine textures.
7

---

Published as a conference paper at ICLR 2025
23.16 dB
31.12 dB
33.64 dB
38.66 dB
PSNR
27.49 dB Input
35.53 dB AirNet
39.25 dB PromptR
44.90 dB AdaIR
PSNR Reference
Figure 5: Image deraining results on Rain I00L (Yang et al. 2019) between all-in-one methods. AdaIR yields high-fidelity rain-free images with structural fidelity and without streak artifacts.
14.95 dB
33.11 dB
32.91 dB
34.02 dB
PSNR
15.63 dB Input
26.73 dB AirNet
26.18 8 dB PromptR
27.12 dB AdalR
PSNR Reference
Figure 6: Image denoising comparisons BSD68 (Martin et al. 2001 between all-in-one methods. The image reproduction quality of our AdaIR is more visually faithful to the ground truth.
Table Comparisons for deveddggaaatton all-in-one restoration. Denoising results are reported for the noise level 25. The top super-row methods denote the general image restoration approaches, and the rest are specialized all-in-one approaches. On SOTS (Li et al. 2018) for dehazing, AdaIR
attains remarkable gain of 3.43 dB over InstructIR (Conde et al. 20224).
Dehazing
Deraining
Deaeising
Deblurring
Low-Light LOL
Method
Rain100L.
BSD68
GoPro
PSNR
SSIM
PSNR SSIM
PSNR SSIM
PSNR SSIM
PSNR
SSIM
PSNR
SSIM
Params
NAFNet Chen al
2022
25.23
0.939
35.56
0.967
31.02
26.53
0.808
20.49
0.809
27.76
0.881
17.11M
HINer Chell et II. 20211
24.74
0.937
35.67
0.969
31.00
0.881
26.12
0.788
19.47
0,800
27.40
0.875
DGUNet
Moi et di 2012
24.78
0.940
36.62
0.971
31.10
0.833
27.25
0.837
21.87
0.823
28.32
0.891
17.33M
MIRNetVZ
0.799
27.34
0.875
SwinIR (Liang er ni 2024
21.50
0.891
30.78
0.923
30.59
0.868
24.52
0.773
17.81
0.723
25.04
0.835
0.91M
Restormer
24.09
0.927
34,81
0.962
31.49
0.884
27.22
0.829
20.41
0.506
27.60
0.881
26.
DL. Tun al. 2019
20.54
0.826
21.96
0.762
23.09
0.745
19,86
0.672
19.83
0.712
21.05
0.743
2.09M
Transwesheer (Vllnaagsu
21.21
0.792
TAPE
ku et al 20123
22.16
0.861
29.67
0.904
30.18
0.855
24.47
0.763
18.97
0.621
25.09
0.801
1.77M
IDR 2hhang St al 20133
25.24
0.943
35.63
0.965
31.60
0.887
27.87
0.846
21.34
0.826
28.34
0.893
15.34M
Prompux (Pormpol 2023
26.54
36.37
31.47
0.886
28.71
0.881
0.832
0.904
32.96M
Gridformet Waang et 21174g
26.79
0.951
36.61
0.971
31.45
0.885
29.22
0.884
22.59 59
0.831
29.33
0.904
34.07M
InstrucIR (Conde
27.10
0.956
36.84
0.973
31.40
0.887
29.40
0
23.00
0.836
29.55
0:907
15.80M
AdaIR (Ours)
30.53
0.978
38.02
0.981
31.35
0.889
28.12
0.858
23.00
0.845
30.20
0.910
28.77M
4.2 ADDITIONAL ALL-IN-ONE RESULTS: FIVE DISTINCT DEGRADATIONS
Following the recent work IDR (Zhang et al. 2023 we further verify the effectiveness of AdaIR by performing experiments five restoration tasks: dehazing, deraining, denoising, deblurring, and low-light image enhancement. For this, we train an all-in-one AdaIR model on combined datasets gathered for five different tasks. These include datasets from the aforementioned three-task setting
8

---

Published as a conference paper at ICLR 2025
Table 3: Image denoising results of directly applying the pre-trained model under the five-degradation
setting to the Urban (Huang et al. 2015), Kodak24 (Rich) 1999) and BSD68 (Martin et al. 2001
datasets. The results are PSNR scores. On Urban (Huang et al. 2013) for the noise level 25,
AdaIR produces a significant performance gain of 0.39 dB PSNR over IDR (Zhang et al. 2023).
U00
Kodak24
BSD68
Method
a 15 a 25 o = 50
` = 15 a = 25 a = 50
15 a = 25 o = 50 Average
DL. (Fan et al.
2019)
21.10
21.28
20.42
22.63
22.66
21.95
23.16
23.09
22.09
22.04
TAPE (Lau et al. 2022
32.19
29.65
25.87
33.24
30.70
27.19
32.86
30.18
26.63
29.83
AirNet
(Li ct al. 2022)
33.16
30.83
27.45
34.14
31.74
28.59
33.49
30.91
27.66
30.89
IDR (Zhang
2023)
33.82
31.29
28.07
34.78
32.42
29.13
34.11
31.60
28.14
31.48
AdaIR (Ours)
34.10
31.68
28.29
34.89
32.38
29.21
34.01
31.35
28.06
31.55
Table 4: Ablation studies for the proposed components. Fixed
Table 5: Spectra decomposition. Adaptive uses adaptive methods following (Zhou et al. 2024).
uses fixed square mask with sides of 10. FLOPs are measured
on the patch size of 256 x 256 x 3.
FMiM Net Baseline| Fixed MGBIL L-H H-L PSNR SSIM Params. FLOPs
FMoM
Overhead
Method Pool Gaussian Ours
PSNR 30.59 30.22
30.25 31.24
(a)
28.21 0.966 16.13M 141.24G
Table 6: Degradation sources.
(b)
29.79 0.969 27.73M 145.09G
(c)
30.37 0.975 28.74M 147.44G
Method
Embedding
Ours
(d)
30.52 0,976 28.74M 147.44G
PSNR SSIM
29.29
30.52
(c)
31.24 0.978 28.77M 147.45G
0.969
0.976
well as additional datasets: GoPro (Nah et al. 2017) for motion deblurring, and LOL-v1 (Wei et al. 2018) for low-light image enhancement.
Table [ shows that AdaIR achieves 0.25 dB gain compared to the recent best method Instruc- UR (Conde et al., (024). when averaged across five restoration tasks. Particularly, the performance improvement is over 3 dB on dehazing. Table reports denoising results on three different datasets with various noise levels. can be seen that our method performs favorably well compared to the other competing approaches.
4.3 ABLATION STUDIES
In this section, we conduct ablation studies to test the impact of various individual components to the overall performance of AdaIR. All ablation experiments are performed on the image dehazing task by training models for 20 epochs.
Impact of individual architecture modules. Table summarizes the performance benefits of individual architectural contributions. Table 4bb demonstrates that the proposed frequency mining mechanism (FMiM) brings gains of 1.58 dB PSNR over the baseline model, using only a fixed mask to decompose the spectra of input images. Furthermore, the L--H unit boosts the performance to 30.37 dB PSNR; see Table c). It can be seen in Table 4 d) that we use both L-H and H-L. units, and the performance reaches 30.52 dB PSNR. Finally, Table 4e) shows that the overall AdaIR brings 3.03 dB improvement over the baseline, while incurring a small computational overhead of 2.64M parameters and 6.21 GFlops. These results corroborate the effectiveness of our design.
Strategies for spectral decomposition. We carry out this ablation to test different strategies to segregate low- and high-frequency representations from the degraded input images. We compare the proposed mask mask-guided adaptive frequency decomposition approach with the Average pooling, Gaussian filtering, and Adaptive (Zhou et al. 2024) strategies. Results are provided in Table [ Following (Cui et al.] 2023a), we use average pooling to obtain the low-frequency features which are then subtracted from the input features to obtain the high-frequency features. This strategy provides a PSNR of 30.59 (see column I in Table 5). which is 0.65 dB lower than our method. Similarly, when we switch to the Gaussian filter of size 5 x 5, the model achieves only 30.22 dB PSNR (second column). Moreover, spatial convolutions for spectral decomposition. Our method of applying a flexible mask for Fourier spectra decomposition performs the best, yielding 31.24 dB.
method is superior to the alternative (Zhou et al. (224)) that uses dynamic

---

Published as a conference paper at ICLR 2025
Table 7: Results on the unseen desnowing task with the CSD (Chen et al. (222d)) dataset.
Table 8: Results of performing image denois- ing on the Rain 100L dataset (a = 50).
Method
AirNet
PromptIR
Ours
Method
AirNet
PromptIR
Ours
PSNR
19.32
20.47
20.54
PSNR
27.25
27.34 0.791
27.51
SSIM
0.733
0.7638
0.7643
SSIM
0.790
0.798
Blurry 0.00016 0.00099
0.00257
0.00046
0.00561
0.00801
0.00234
0.00288
0.04374
AH-L
0.72464 0.34558 0.89120 0.81372 0.80288 0.92845 0.59886 0.95265 0.93757
Figure 7: First column shows the blurry image and the spatial attention map in Aи-r. Others are the channel-wise features before H-L and the corresponding attention scores in Ar.
Frequency representation mining at image-level vs. feature-level. Each AFLB block in AdaIR decoder receives the original degraded image as input, on which FMiM applies the procedure of spectra decomposition. To verify the efficacy of this design, switch using the input embedding features X (rather than degraded image) for frequency representation. This ablation result in Table 6 shows performance drop from 30.52 dB to 29. 29 dB, indicating that the raw input image offers better discriminative information about the degradation for effective spectra separation.
Generalization to -ut-of-distribution degradations. To demonstrate the generalization ability of our AdaIR, we take the all-in-one model trained in the three-task setting and directly apply it to out-of-distribution datasets. Table shows that, on the unseen task of image desnowing, AdaiR provides more favorable results than other approaches. We then perform image denoising using the Rain 100L Yang et al. (2019) dataset instead of BSD68 Martin et al. (200T). Table 88 depicts that our method is more robust in the out-of-distribution scenes than PromptIR (Potlapalli et al., 2023) and
AirNet (Li et al.) 2022).
Mechanism of FMoM. In FMiM, we extract different frequency components from input features. These features are then categorized into low- and high-frequency groups using the dynamic, learn- able module MGB, which adaptively adjusts the cutoff frequency boundary based on the specific degradation observed in the image. Once the low- and high-frequency features are segregated, they processed by the FMoM. This module is responsible for either suppressing allowing specific frequency components to pass through, depending on the nature of the degradation, effectively enhancing the restoration process. To better illustrate the interaction between frequency features, we visualize the attention weights generated by the High-to-Lon (H-L) and Low-to-High (L-H) modules in Fig. 7 The high-frequency features, rich in spatial signals, assist the low-frequency branch in focusing on and effectively addressing severely impacted regions, such as the girl in the image. Conversely, the low-frequency features, which provide global view, help the high-frequency features to avoid overemphasizing those challenging regions.
5 CONCLUSION
This paper introduces AdaIR, an all-in-one image restoration model capable of adaptively removing different kinds of image degradations. Motivated by the observation that different degradations affect distinct frequency bands, we have developed two novel components: a frequency mining module and frequency modulation module. These modules are designed to identify and enhance the relevant frequency components based on the degradation patterns present in the input image. Specifically, the frequency mining module extracts specific frequency elements from the image's intermediate features, guided by adaptive decomposition of the input's spectral characteristics that reflect the underlying degradation. Subsequently, the frequency modulation module further refines these elements by facilitating the exchange of complementary information across different frequency features. Incorporating the proposed modules into a U-shaped Transformer backbone, the proposed network achieves sate-of-the-art performance range of image restoration tasks.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Abdullah Abuolaim and Michael S Brown. Defocus deblurring using dual-pixel data. In ECCV, 2020.
Yuang Ai, Huaíbo Huang, Xiaoqiang Zhou, Jiexiang Wang, and Ran He. Multimodal prompt perceiver: Empower adaptiveness generalizability and fidelity for all-in-one image restoration. In CVPR, pp. 25432-25444, 2024.
Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik. Contour detection and hierarchical image segmentation. TPAMI, 2010.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. 167.v46600 2016.
Dana Berman, Shai Avidan, al. Non-local image dehazing. In CVPR, 2016.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. NeurlPS, 2020.
Chun-Fu Richard Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross-attention multi-scale vision transformer for image classification. In ICCV, 2021a.
Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao. Pre-trained image processing transformer. In CVPR, 2021b.
Liangyu Chen, Xin Lu, Jie Zhang, Xiaojie Chu, and Chengpeng Chen. Hinet: Half instance normalization network for image restoration. In CVPR Workshops, 2021c
Liangyu Chen, Xiaojie Chu, Xiangyu Zhang, and Jian Sun. Simple baselines for image restoration. In ECCV, 2022.
Wei-Ting Chen, Hao-Yu Yu Fang, Cheng-Lin Hsieh, Cheng-Che Tsai, Chen, Jian-Jiun Ding, Sy-Yen Kuo, et al. All snow removed: Single image desnowing algorithm using hierarchical dual-tree complex wavelet representation and contradict channel loss. In ICCV, 2021d.
Yu-Wei Chen and Soo-Chang Pei. Always clear days: Degradation type and severity aware all-in-one adverse weather removal. arXiv:2310. 18293, 2023.
Sung-Jin Cho, Seo- Seo-Won Ji, Jun-Pyo Hong. Seung- Won Jung, and Sung-Jea Ko. Rethinking coarse-to- fine approach in single image deblurring. In ICCV, 2021.
Marcos V Conde, Gregor Geigle, and Radu Timofte. High-quality image restoration following human instructions. In ECCV, 2024.
Yuning Cui, Wenqi Ren, Xiaochun Cao, and Alois Knoll. Focal network for image restoration. In ICCV, 2023a.
Yuning Cui, Wenqi Ren, Xiaochun Cao, and Alois Knoll. Image restoration via frequency selection. TPAMI, 2023b.
Yuning Cui, Wenqi Ren, Sining Yang, Xiaochun Cao, and Alois Knoll. Irnext: Rethinking convolu- tional network design for image restoration. In ICML, 2023c.
Yuning Cui, Yi Tao, Zhenshan Bing, Wenqi Ren, Xinwei Gao, Xiaochun Cao, Kai Huang, and Alois Knoll. Selective frequency network for image restoration. In ICLR, 2023d.
Yuning Cui, Yi Tao, Luoxi Jing, and Alois Knoll. Strip attention for image restoration. In IJCAI, 2023e.
Yuning Cui, Yi Tao, Wenqi Ren, and Alois Knoll. Dual-domain attention for image deblurring. In AAAI, 2023f.
Yuning Cui. Mingyu Liu, Wenqi Ren, and Alois Knoll. Hybrid frequency modulation network for image restoration. In IJCAI, 2024a.
11

---

Published as a conference paper at ICLR 2025
Yuning Cui, Wenqi Ren, Xiaochun Cao, and Alois Knoll. Revitalizing convolutional network for image restoration. TPAMI, 2024b.
Yuning Cui, Wenqi Ren, and Alois Knoll. Omni-kernel network for image restoration. In AAAI, 2024c.
Yuning Cui, Wenqi Ren, and Alois Knoll. Omni-kemnel modulation for universal image restoration. TCSVT, 2024d.
Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Color image denoising via sparse 3d collaborative filtering with grouping constraint in luminance-chrominancee space. In ICIP, 2007.
Hang Dong, Jinshan Pan, Lei Xiang, Zhe Hu, Xinyi Zhang, Fei Wang, and Ming-Hsuan Yang. Multi-scale boosted dehazing network with dense feature fusion. In CVPR, 2020a.
Yu Dong, Yihao Liu, He Zhang, Shifeng Chen, and Yu Qiao. Fd-gan: Generative adversarial networks with fusion-discriminntoo for single image dehazing. In AAAI, 2020b.
Dawei Du, Yuankai Qi, Hongyang Yu, Yifan Yang, Kaiwen Duan, Guorong Li, Weigang Zhang, Qingming Huang, and Qi Tian. The unmanned aerial vehicle benchmark: Object detection and tracking. In ECCV, 2018.
Qingnan Fan, Dongdong Chen, Lu Yuan, Gang Hua, Nenghai Yu, and Baoquan Chen. A general decoupled learning framework for parameterized image operators. TPAMI, 2019.
Hongyun Gao, Xin Tao, Xiaoyong Shen, and Jiaya Jia. Dynamic scene deblurring with parameter selective sharing and nested skip connections. In CVPR, 2019.
Tao Gao, Yuanbo Wen, Kaihao Zhang, Jing Zhang, Ting Chen, Lidong Liu, and Wenhan Luo. Frequency-oriented efficient transformer for all-in-one -eegaee-degraded image restoration. TCSVT, 2023.
Chun-Le Guo, Qixin Yan, Saeed Anwar, Rummin Cong, Wenqi Ren, and Chongyi Li. Image dehazing transformer with transmission-wware 3d position embedding. CVPR, 2022.
Kaiming He, Jian Sun, and Xiaoou Tang. Single image haze removal using dark channel prior. TPAMI, 2010.
Jia-Bin Huang, Abhishek Singh, and Narendra Ahuja. Single image super-resolution from transformed self-exemplars. In CVPR, 2015.
Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Baojin Huang, Yimin Luo, Jiayi Ma, and Junjun Jiang. Multi-scale progressive fusion network for single image deraining. In CVPR, 2020.
Yitong Jiang, Zhaoyang Zhang, Tianfan Xue, and Jinwei Gu. Autodir: Automatic all-in-one image restoration with latent diffusion. arXiv:2310.10123, 2023.
Kwang In Kim and Younghee Kwon. Single-image super-resolution using sparse regression and natural image prior. TPAMI, 2010.
Lingshun Kong, Jiangxin Dong, Jianjun Ge, Mingqiang Li, and Jinshan Pan. Efficient frequency domain- transformers for high-quality image deblurring. In CVPR, 2023.
Boyi Li, Wenqi Ren, Dengpan Fu, Dacheng Tao. Dan Feng. Wenjun Zeng, and Zhangyang Wang. Benchmarking imagle-imaee dehazing and beyond. TIP. 2018.
Boyun Li, Xiao Liu, Peng Hu, Zhongqin Wu, Jiancheng Lv, and Peng. All-in-one image restoration for unknown corruption. In CVPR, 2022.
Ruoteng Li, Robby T Tan, and Loh Cheong. All in one bad weather removal using architectural search. In CVPR, 2020.
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Lue Van Gool, and Radu Timofte. SwinIR: Image restoration using swin transformer. In ICCV Workshops, 2021.
12

---

Published as a conference paper at ICLR 2025
Lin Liu, Lingxi Xie, Xiaopeng Zhang, Shanxin Yuan, Xiangyu Chen, Wengang Zhou, Houqiang Li, and Qi Tian. Tape: aask-agn prior embedding for image restoration. In ECCV, 2022.
Mingyu Liu, Yuning Cui, Wenqi Ren, Juxiang Zhou, and Alois C Knoll. Liednet: A lightweight network for low-light enhancement and deblurring. TCSVT, 2025.
Jiaqi Ma, Tianheng Cheng, Guoli Wang, Qian Zhang, Xinggang Wang, and Lefei Zhang. Prores: Exploring degradation-aware visual prompt for universal image restoration. 2rXiv:2306./3653 2023.
Kede Ma, Zhengfang Duanmu, Qingbo Wu, Zhou Wang, Hongwei Yong, Hongliang Li, and Lei Zhang. Waterloo exploration database: New challenges for image quality assessment models. TIP, 2016.
Xintian Mao, Yiming Liu, Fengze Liu, Qingli Li, Wei Shen, and Yan Wang. Intriguing findings of frequency selection for image deblurring. In AAAI, 2023.
Xintian Mao, Jiansheng Wang, Xingran Xie, Qingli Li, and Yan Wang. Loformer: Local frequency transformer for image deblurring. In ACMMM, 2024.
David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. A database of human segmented natural images and its application evaluating segmentation algorithms and measuring ecological statistics. In ICCV, 2001.
Tomer Michaeli and Michal Irani. Nonparametric blind super-resolution. In ICCV, 2013.
Chong Mou, Qian Wang, and Jian Zhang. Deep generalized unfolding networks for image restoration. In CVPR, 2022.
Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In CVPR, 2017.
Seungjun Nah, Sanghyun Son, Jaerin Lee, and Kyoung Mu Lee. Clean images are hard to reblur: Exploiting the ill-posed inverse task for dynamic scene deblurring. In ICLR, 2022.
Dongwon Park, Byung Hyun Lee, and Se Young Chun. All-in-one image restoration for unknown degradations using adaptive discriminative filters for specific degradations. In CVPR, 2023.
Vaishnav Potlapalli, Syed Waqas Zamir, Salman Khan, and Fahad Shahbaz Khan. Promptir: Prompting for all-in-one image restoration. NeurIPS, 2023.
Rui Qian, Robby T Tan, Wenhan Yang. Jiajun Su, and Jiaying Liu, Attentive generative adversarial network for raindrop removal from a single image. In CVPR, 2018.
Xu Qin, Zhilin Wang, Yuanchao Bai, Xiaodong Xie, and Huizhu Jia. Ffa-net: Feature fusion attention network for single image dehazing. In AAAI, 2020.
Chao Ren, Xiaohai He, Chuncheng Wang, and Zhibo Zhao. Adaptive consistency prior based deep network for image denoising. In CVPR, 2021.
Dongwei Ren, Wangmeng Zuo, Qinghua Hu, Pengfei Zhu, and Deyu Meng. Progressive image deraining networks: A better and simpler baseline. In CVPR, 2019.
Wenqi Ren, Senyou Deng, Kaihao Zhang, Fenglong Song, Xiaochun Cao, and Ming- Yang. Fast ultra high-definition video deblurring via multi-scale separable network. 1/CV, 2024.
Franzen Rich. Kodak lossless true color image suite. 1999.
Zenglin Shi, Tong Su, Pei Liu, Yunpeng Wu, Le Zhang, and Meng Wang. Learning frequency-aware dynamic transformers for all-in-one image restoration. arXiv preprint arXiv:2407.0166 2024.
Chunwei Tian, Yong Xu, and Wangmeng Zuo. Image denoising using deep enn with batch renormal- ization. Neural Networks, 2020.
13

---

Published as a conference paper at ICLR 2025
Fu-Jen Tsai, Yan-Tsung Peng, Yen- Yu Lin, Chung-Ch -hi Tsai, and Chia-Wen transformer for fast image deblurring. In ECCV, 2022a.
Wen Lin. Stripformer: Strip
Fu-Jen Tsai, Yan-Tsung Peng, Chung-Chi Tsai, Yen-Y Yu Lin, and Chia-Wen Lin. BANet: A blur-aware attention network for dynamic scene deblurring. TIP, 2022b.
Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. MAXIM: Multi-axis MLP for image processing, In CVPR, 2022.
Jeya Maria Jose Valanarasu, Rajeev Yasarla, and Vishal M Patel. Transweather: Transformer-based restoration of images degraded by adverse weather conditions. In CVPR, 2022.
Jiangang Wang, Yuning Cui, Yawen Li, Wenqi Ren, and Xiaochun Cao. Omnidirectional image super-resolution via bi biojroecttion fusion. In AAAI, 2024a.
Qiang Wang, Yuning Cui, Yawen Li, Yaping Ruan, Ben Zhu, and Wenqi Ren. Rffnet: Towards robust and flexible fusion for low-light image denoising. In ACM MM, 2024b.
Tao Wang, Kaihao Zhang, Ziqian Shao, Wenhan Luo, Bjorn Stenger, Tong Lu, Tae-Kyun Kim, Wei Liu, and Hongdong Li. Gridformer: Residual dense transformer with grid structure for image restoration adverse weather conditions. IJC,, 2024c.
Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li Uformer: A general u-shaped transformer for image restoration. CVPR, 2022.
Zhou Wang, Alan Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. TIP, 2004.
Chen Wei, Wenjing Wang. Wenhan Yang, and Jiaying Liu. Deep retinex decomposition for low-light enhancement. arXiv: 048880000660 2018.
Sanghyun Woo, Jongchan Park, Joon- Young Lee, and In So Kweon. Cbam: Convolutional block attention module. ECCV, 2018
Gang Wu, Junjun Jiang, Kui Jiang, and Xianming Liu. Harmony in diversity: Improving all-in-one image restoration via multi-task collaboration. In ACM MM, 2024.
Yimin Xu, Nanxi Gao, Zhongyun Shan, Fei Chao, and Rongrong Ji. Unified-width adaptive dynamic network for all-in-on one image restoration. arXiv preprint arXivv2401.13222 2024.
Hao Yang, Liyuan Pan, Yan Yang, and Wei Liang. Language-driverr all-in-one adverse weather removal. In CVPR, 2024.
Wenhan Yang, Robby T Tan, Jiashi Feng, Zongming Guo, Shuicheng Yan, and Jiaying Liu. Joint rain detection and removal from a single image with contextualized deep networks. TPAMI, 2019.
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming- Hsuan Yang, and Ling Shao. CycleISP: Real image restoration via improved data synthesis. In CVPR, 2020a.
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming- Hsuan Yang, and Ling Shao. Learning enriched features for real image restoration and enhancement. In ECCV, 2020b.
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming Hsuan Yang, and Ling Shao. Multi-stage progressive image restoration. In CVPR, 2021.
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming- Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. CVPR. 2022a.
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Hssan Yang, and Ling Shao. Learning enriched features for fast image restoration and enhancement. TPAMI, 2022b.
14

---

Published as a conference paper at ICLR 2025
Jinghao Zhang, Jie Huang, Mingde Yao, Zizheng Yang. Hu Yu, Man Zhou, and Feng Zhao. Ingredient- oriented multi multi-degradation learning for image restoration. CVPR, 2023.
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang, Beyond a gaussian denoiser: Residual learning deep enn for image denoising. TIP, 2017a.
Kai Zhang, Wangmeng Zuo, Shuhang Gu, and Lei Zhang. Learning deep CNN denoiser prior for image restoration. In CVPR, 2017b.
Kai Zhang, Wangmeng Zuo. and Lei Zhang. Fidnet: Toward a fast and flexible solution for cnn-based image denoising. TIP, 2018.
Kaihao Zhang, Wenqi Ren, Wenhan Luo, Wei-Sheng Lai, Bjöm Stenger, Ming-Hsuan Yang, and Hongdong Li. Deep image deblurring: A survey. IJCV, 2022.
Bolun Zheng, Shanxin Yuan. Chenggang Yan, Xiang Tian, Jiyong Zhang, Yaoqi Sun, Lin Liu, Aleš Leonardis, and Gregory Slabaugh. Learning frequency domain priors for image demoireing.
TPAMI, 2021.
Shihao Zhou, Jinshan Pan, Jinglei Shi, Duosheng Chen, Lishen Qu, and Jufeng Yang. Seeing the unseen: A frequency prompt guided transformer for image restoration. In ECCV, 2024.
Yurui Zhu, Tianyu Wang, Xueyang Fu, Xuanyu Yang, Xin Guo, Jifeng Dai, Yu Qiao, and Xiaowei Hu. Learning weather-general and weather-specific features for image restoration under multiple adverse weather conditions. In CVPR, 2023.
15

---

Published as a conference paper at ICLR 2025
APPENDIX
This appendix provides generalization evaluation (Sec. A more experimental results under the
single-task setting (Sec. [B. additional ablation studies (Sec. computational comparisons (Sec. D
visualization for FMiM (Sec. architectural details of the transformer block (Sec. F). and additional
visual results (Sec. G
More qualitative comparisons on different datasets are provided the supplementary material.
A GENERALIZATION EVALUATION
We assess the generalization capability of our model on additional out-o--diitribution degradations and compare the results against state-of-the-ar all-in-one algorithms. As presented in Table our method demonstrates superior performance on two previously unseen degradation types: defocus deblurring and raindrops. Additionally, we evaluate our approach the real-world UAVDT (Du ict al. 2018) dataset, which consists of images captured by UAVs at varying altitudes and exhibiting diverse levels of hazy degradation. Figure 8 illlsstrats that our model is more robust in real-world scenarios by restoring shaper images.
Table 9: Generalization evaluation of all-in-one algorithms. The models are trained under the three- task setting and directly applied to the DPDD (Abuolaim & Brown 2020) and AGAN (Qian et al. 2018) datasets for defocus deblurring and raindrop removal, respectively.
DPDD Abuulaam & Brown 2020
AGAN (Qian et al. 2018)
Method
PSNR
SSIM
PSNR
SSIM
AirNet
20.17
0.662
22.09
0,822
PromptR
21.76
0.661
22.98
0.827
Ours
22.93
0.711
23.14
0.826
Hazy Images
AirNet
PromptIR
AdaIR
Figure 8: Visual comparisons on the UAVDT (Du al. 2018) dataset.
B SINGLE DEGRADATION ONE-BY-ONE RESULTS
Consistent with previous works Li et al. 2022 Potlapalli et al. (023). we further evaluate AdaIR under the single-task experimental protocol. To this end, train separate copies of the AdaIR model
16

---

Published as a conference paper at ICLR 2025
for each restoration task. The numerical results SOTS for image dehazing presented in Table [0] Our method significantly outperforms previous state-of-the-art all-in-one approaches, PromptIR (Potlapalli et al.] (023) and AirNet (Li et al. 2022). by 0.49 dB and 8.62 dB, respectively, attributed to the adaptive frequency separation and modulation ability for haze degradations of different densities. Similarly,
the deraining task, Table shows that our AdaIR advances the state-of-the-art (Potlapalli et al. 2023) by 1.86 dB. Compared to our baseline model (Zamir et al. 2022a), the accuracy gain is 2.16 dB PSNR, suggesting the efficacy of our designs. Furthermore, we provide experimental results for image denoising on two datasets with different noise levels. As can be seen in Table [2] our method yields an average performance gain of 0.13 dB PSNR over the strong competitor PromptiR. Compared to other methods, our method has more advantages on the Urban 100 dataset than BSD68. This phenomenon is probably due to the higher resolution of Urban images, enabling more accurate frequency modulation.
Table 10: Dehazing results in the single-task task setting on the SOTS-Outdoor (Li et al. 2018) dataset. Compared to PromptIR (Potlapalli et al. 2023), our method generates 0.49 dB PSNR improvement.
Method DehazeNet
MSCNN
AODNet
EPDN
FDGAN
AirNet
Restormer
PromptI
AdaIR
PSNR
22.46
22.06
20.29
22.57
23.15
23.18
30.87
31,31 0.973
31.80
SSIM
0.851
0.908
0.877
0.863
0.921
0.900
0.969
0.981
Table 11: Deraining results in the single-task setting the Rain100L (Yang al. 2019) dataset. Our
AdaIR obtains a significant performance boost of 1.86 dB PSNR over PromptiR R (Potlapali et al.
2023
Method
DIDMDN
UMR
SIRR
MSPEN
LPNet
AirNet
Restormer
PrompuR
AdaIR
PSNR
23.79
32.39
32.37
33.50
33.61 0.958
34,90 0.977
36.74 0.978
37.04 0.979
38,90
SSIM
0.773
0.921
0.926
0.948
0.985
Table 12: Denoising results in the single-task setting on Urban 100 (Huang et al. 2015) and BSD68 (Martin al., 2001). On Urban 100 (Huang et al. 2015) for the noise level 50, AdaIR
yields a 0.31 dB gain over PromptlR (Potlapalli et al. 2023).
Utbun100
BSD68 25
Method
50
Average
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
CBM3D Dabov
2007)
33.93
0.941
31.36
0.909
27.93
0.840
33.50
0.922
30.69
0.868
27.36
0.763
30.80
0.874
DrCNN Zhang al
701
32.98
0.931
30.81
0.902
27.59
0.835
33.89
0.930
31.23
0,883
27.92
0.789
30.74
0.878
IRCNN
Zhang
27.59
0.909
27.70
33.87
31.18
0.882
27.88
0.790
29.90
0.864
FF-NNet
Zhune
2018
33.83
0.942
31.40
0.912
28.05
0.848
33.87
0.929
31.21
0.882
27.96
0.789
31,05
0.884
BRDNet
Tian el al.
(120
34,42
0.946
31.99
0.919
28.56 0.858
34.10
0.929
31.43
0.885
28.16
0.794
31.44
0.889
AirNet [T.
34,40
0.949
32.10
0.924
28.88
0.871
34,14
0.966
31.48
0.893
28.23
0.806
31.54
0.897
PromptRK (Polappll al
2023
34.77
0.952
32.49
0.929
29.39
0.881
34,34
0.938
31.71
0.897
28.49
0.813
31.87
0.902
AdaIR (Ours)
34.96
0.953
32.74
0.931
29.70
0.885
34.36
0.938
31.72
0.897
28.49
0.813
32.00
0.903
C ADDITIONAL ABLATION STUDIES
AFLBs in encoder and decoder? We run an experiment to assess the feasibility of employing AFLB modules on either the encoder side, decoder side, or both. Table [ shows that utilizing AFLBs in both the encoder and decoder leads to notable performance degradation compared to AFLBs solely integrated into the decoder.
Placement of AFLB in the network. Next, we conduct ablation experiment to study where to place AFLBs in our hierarchical network. Table [4 demonstrates that employing only one AFLB (between level 1 and level 2) leads to deterioration in the network's performance (29.58 dB in the top row). Conversely, integrating AFLBs between every consecutive level of the decoder yields the best performance.
Design choices of FMoM. We investigate different choices for the frequency modulation module (FMoM). As shown in Fig. aa), we leverage the commonly used spatial attention (Woo al. 2018)
17

---

Published as a conference paper at ICLR 2025
Table 13:
Comparisons of image dehazing under the single-task setting: between the of AFLBS on either the encoder-side, decoder-side, or both.
Dehazing SOTS (Li al. 2018)
Method
PSNR
SSIM
Encoder+ AFL.
29.70
0.973
AdaIR (Ours)
30.52
0.976
Table 14: AFLB position. Results are reported on the SOTS
(Li et 2018 dataset.
Method
PSNR
SSIM
Level 2
28.58
0.973
Level 2+3
29.83
0.975
Level 2+3+4
30.52
0.976
Xx_
X
GAP iord
A miatad
H-L
Conv
Sigmoid
7x7
GMP
A Howwi
N
(a) Spatial attention, 29.67 dB/0.973
(b) Ours, 30.52 dB/0.976
Figure 9: Different choices for FMoM. (a) Using widely adopted spatial attention (Woo et al. 2018) modulate different frequency features, where the attention map is generated without discriminating different frequency inputs. (b) Using specially designed attention units to exchange complementary information across different frequency features. GAP and GMP denote the global average pooling and global max pooling, respectively. The experiments are conducted on image dehazing under the single-task task setting.
modulate different frequency features without discriminating different inputs. Overall, the process is formally given by:
X AG + X O Ar.
where,
(9)
A A = Split (0(A)):
where,
(10)
Á A 7x77 ([GAP(XX,, Xi)), GMP([N.. X.))))
(11)
where @ represents element-wise multiplication, Split indicates splitting the features among the channel dimension, is the Sigmoid function, W is 7 convolution, and [] is concatenation operator. GAP and GMP are global average pooling and global max pooling among the channel dimensions, respectively. The experiments are performed on the image dehazing task under the single-task setting. This variant achieves only 29.67 dB PSNR, which is 0.85 dB lower than our FMoM, shown in Fig. b), indicating the effectiveness of our design.
Furthermore, we conducted experiments to evaluate the impact of using different attention strategies in the two branches. As shown Table [5] employing the same attention mechanism in both branches results in lower performance compared to our approach. This highlights the effective- ness of performing frequency interactions tailored to the distinct properties of different frequency components.
Combinations of different degradations. We investigate the influence of various combinations of degradation types on model performance, as presented in Table [6] As expected, including more degradation types make it increasingly difficult for the model to perform restoration. Notable, hazy images in a combined dataset lead to a larger performance drop than rainy or noisy images. One reason could be that the aim of the restoration model in deraining and denoising tasks is to focus
18

---

Published as a conference paper at ICLR 2025
Table 15: Comparisons between different attention types.
Unit
Attention Type
PSNR
(a) H-L/L-H
Channel/Channel
30.10
(b) H-L/L-H
Spatial/Spatial
30.36
(c) H-L/L-H (Ours)
Spatial/Channel
30.52
more restoring high-frequency content (noise, rain), whereas, in the dehazing task the goal is focus on removing now-frequency (hazy) content.
Table 16: Ablation studies on the combinations of degradations for the three-task setting. Results are presented in the form of PSNR (dB //S/...M.
Degradation
Denoising on BSD68
Deraining on on Rain 100L
Dehazing on SOTS
Noise
Rain
Haze
15
25
σ 50
34.36/0.938
31.72/0.897
28.49/0.813
38.90/0.985
3 31.00/0.981
34.31/0.938
31.67/0.896
28.42/0.81
38.22/0.983
34.11/0.935 31.48/0.892
28. 19/0..0
30.89/0.980
38. 18..4/00.98 .983
30.54/0.978
34.12/0.935
31.45/0.892
28. 19/
38.64 18.64/0.983
31 31.06/0.980
D COMPUTATIONAL COMPARISONS
Table 17 shows that the proposed AdaIR strikes better tradeoff between accuracy and complexity than other all-in-one competing methods.
Table 17: Computational comparisons of all-in-one methods under the three-degradation setting. The average PSNR across three tasks is reported here (see Table of the main paper for more detailed
results). FLOPs are measured on the patch size of 256 x 256 x 3.
Params.
FLOPs
PSNR
Method
(M)
(G)
AirNet (Li et al. 2022
8.93
311
31.20
PromptÍK (Potlaaali et al. 2023
35.59
158.4
32.06
AdaIR
28.77
147.45
32.69
E VISUALIZATION FOR FMIM
Figure [0 visualizes the FMiM process, illustrating how various frequency components are separated from the mput image and extracted from the features. Specifically, MGB produces a mask to decouple the input image into different frequencies (2, 3) Next, the obtained spectra are used to extract corresponding features from the input features (O), shown in ( and ④, which then interact in FMoM. The visualizations demonstrate the efficacy of our design. Additional examples of frequency decomposition for low-light image enhancement and dehazing tasks provided in Figure []] As shown, our model adaptively decouples images into different frequency bands. Furthermore, Figure 12 Fillussrates comparisons of features obtained before and after our AFLB module. The results demonstrate that our module effectively generates sharper features, contributing to high-fidelity reconstruction.
19

---

Published as a conference paper at ICLR 2025
Ma
Set
Comv 1x
Mask
CA
(c]
Figure 10: Visualizations for intermediate features and spectra. Our modules can decouple the image/features into different frequencies as expected. Attention weights in 7 and & are shown in Fig. 7 of the main paper.
Degraded Image
Reference
FFT
FFT
Figure 11: Visualizations of frequency decoupling. The two images are obtained from the LOL- vI (Wei et al., 2018) and SOTS (Li al., 2018) datasets for low-light image enhancement and dehazing, respectively.
Before AFLB
After AFLB
Before AFLB
After AFLB
Figure 12: Feature comparisons based on the two images in Figure features.
Our module generates sharper
TRANSFORMER BLOCK IN THE ADAIR FRAMEWORK
In the AdaIR framework, ct al. 2022a).. Fig. 13 presents its architectural details. It consists two successive components, multi-dcone head transposed attention (MDTA) and gated-dcony feed-forward network (GDFN).
Transformer Blocks (TB) based on the design proposed in Zaain
MDTA first normalizes the input X E I XxCCC using layer normalization operator (Ba et 2016). and then generates the query (QE I xC)) key (Ké X'XWxC)) and value (Ve R" "
projections using combinations of 1 x convolution and x 3 depth-wiss wise convolution layers. The transposed-attentionn map of size C x C yielded by applying the Softmax function to the dot-product
20

---

Published as a conference paper at ICLR 2025
Conv 1xx
Doamy
Nom
Inakett
Come
9x
Dicon
3663
GDFN MDTA
Cony
Dcaov
Transformer Block (TB)
CSC
Conv
Door
3
Conv
Dion
Norm
Leyer
Normalization
Addition
Element-wise
Mutiplication
Matrix
Elemnett-wise Muttiplication
Reshape
Figure 13: Architectural details of the Transformer Block (TB) (Zamir et al. 2022a) used in the AdaIR framework. TB involves two elements: multi-dconv head transposed attention (MDTA) and gated-dcony feed-forward network (GDFN).
results of the reshaped query and key projections. Overall, the process of MDTA is given by:
w Attention Attention (Q', V V' Softmax (K'- QQ/aa)
+X, where,
(12)
(13)
where X is the output MDTA. wlIx denotes a x convolution. a is a learnable factor to control the magnitude of the dot product result of K and Q. Q' K' and V obtained by reshaping tensors from the original size R #
Similarly, GDFN first applies a layer normalization operator normalize the input X R Hx The result then passes through two branches, each including a x 1 convolution with a factor 7 to expand channels, followed by 3 x depth-wise convolution layer. Two branches converge using element-wist multiplication after activating one branch via GELU function. Overall, the GDFN process is formally expressed as:
x W 1xXL Gating(X) + X,
where,
(14)
Gating X) o (DW3xx 3x3 (Wa x (LN(X)))) O DW 3x3 (VN()(L)(X)))
(15)
where LN is the layer normalization, denotes element-wise multiplication, DW/3x3 represents 3 x 3 depth-wise convolution, and indicates the GELU non-linearity.
G ADDITIONAL VISUAL RESULTS
In this section, we provide the t-SNE result of our method under the five-degradation setting in Fig. [4]] It can be seen that our method is capable of discriminating degradation contexts for five different degradation types. It is worth noting that the cluster for low-light image enhancement is closer to the dehazing cluster than others, suggesting the effectiveness of our model, since these two degradation types mainly impact the image content on low- oowfrreuency components.
21

---

Published as a conference paper at ICLR 2025
Denoise
Derain
Dehaze
Deblur
Low-light
Figure 14: The t-SNE result of our model under the five-degradation setting.
22


Reviews:
[{"review_id": "QsqGWjL0lH", "reviewer": "Reviewer_Jdae", "summary": "The paper presents AdaIR, which aims to address multiple forms of degradation, such as noise, blur, haze, and rain, within a single model. AdaIR is motivated by the observation that different types of degradation impact image content on different frequency subbands.  AdaIR leverages both spatial and frequency domain information to effectively separate and remove these degradations using customised modules.", "questions": "Address weaknesses above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper highlights a key insight for any image restoration: Noisy and rainy images are contaminated with high-frequency content while the low-frequency content degrades more in Low-light and hazy images.\n\nExploiting frequency domain is a key differentiation, as previous methods like AirNet, IDR, and PromptIR operate solely on spatial domain information\n\nFrequency Mining Module (FMiM) extracts relevant frequency signals from degraded images through adaptive spectral decomposition, targeting degradation-specific frequency content. The use of a dynamic, learnable Mask Generation Block (MGB) to separate low- and high-frequency representations based on the input image's degradation type allows AdaIR to adapt to different degradations effectively\n\nFrequency Modulation Module (FMoM) refines these frequency features by enabling interaction between different frequency components, enhancing the model’s effectiveness on various degradation types.\n\nExperiments on a wide set of benchmarks demonstrates consistent performance gains over competing approaches in both three-degradation and five-degradation settings\n\nThe improved quality of discriminative feature learning (shown in Fig. 2) is impressive.", "weaknesses": "While the paper demonstrates the generalisation ability of AdaIR by testing it on an unseen desnowing task and synthetic images with mixed degradations (rain and noise) in Table 7 and 8, there are no experiments on real out-of-distribution images. Extensive qualitative comparisons on real images should be included. Evaluation on a wider range of out-of-distribution degradations would strengthen the claims of the model's generalisation capabilities. \n\nThe quantitative tables primarily compare AdaIR with other all-in-one methods but not the scores of single-task trained state-of-the-art models. Including the highest scores achieved by SotA single task models on each individual task would provide a more comprehensive assessment of AdaIR's performance relative to specialised approaches.\n\nWhile Table 4 shows the overhead of AFLB in terms of parameters and FLOPs, a more in-depth analysis of its computational cost and potential impact on real-time applications would be beneficial.\nFLOPs are only reported for ablation studies, but there are no FLOPs comparisons with competing methods.\nIn addition to FLOPs, there should also be comparison between inference time of different models.\nusing frequency domain processing inherently introduces additional processing for transformations eg IFFT. \nAlthough they are beneficial for accuracy, a potential drawbacks of such approaches is the computation cost and incompatibility with edge platforms. \n\nThe paper doesn't cite or compare with any existing models that use frequency domain processing. There have been many such studies [refs 1-7 below]. Some of these papers are quite related as they propose transformer modules that focus on high/low frequencies.\n\nFurthermore, comparing with alternative transform-domain methods, such as wavelet transforms or other adaptive filtering techniques, commonly used in related papers, could enhance the performance and adaptability of AdaIR.\n\nThe MGB dynamically generates a mask to separate low and high frequencies based on the input image. It would be interesting to see the difference in generated masks for different tasks. This vanilla approach could be prone to overfitting, especially since the training dataset does not adequately cover the diversity of real-world degradations. A more robust approach might involve specifically incorporating prior knowledge about frequency characteristics of different degradations or exploring regularization techniques to prevent overfitting\n\nPaper lacks deeper analysis on how the interaction between different frequency bands contributes to the restoration process. Such analysis could lead to more informed design choices and potentially improve the interpretability of the model.\n\nThe graph in Fig. 1 right doesn’t clearly demonstrate the stark frequency-domain differences between different tasks.\n\n\n[1] Seeing the Unseen: A Frequency Prompt Guided Transformer for Image Restoration (ECCV 2024)\n\n[2] Selective Frequency Network for Image Restoration (ICLR 2023)\n\n[3] Image Restoration via Frequency Selection, TPAMI 2024\n\n[4] Intriguing Findings of Frequency Selection for Image Deblurring, AAAI 2023\n\n[5] Efficient Frequency Domain-based Transformer for High-Quality Image Deblurring, CVPR 2023 \n\n[6] LoFormer: Local Frequency Transformer for Image Deblurring [ACM MM 2024]\n\n[7] Learning Frequency Domain Priors for Image Demoireing, TPAMI 2022", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ULZF3Nyzoe", "reviewer": "Reviewer_K6x2", "summary": "This paper presents an all-in-one image restoration framework that can generate clean images from multiple degradation patterns, such as noise, blur, haze, and rain. Different from existing all-in-one algorithms that perform only in the spatial domain, the proposed network deals with all-in-one tasks from the perspectives of frequency mining and modulation. The network extracts different frequencies from the degradation images and modulates features for high-quality reconstruction. The adaptive strategy enhances the learning of informative frequency signals based on input degradation. Comprehensive experiments on two kinds of all-in-one settings demonstrate the model's effectiveness.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1) The paper presents a novel adaptive all-in-one image restoration architecture that can effectively decouple the input into different frequencies and modulate them for high-quality reconstruction. The paper is well-motivated, and Figure 1 makes sense.\n2) The proposed blocks are reasonable and can be easily injected into other algorithms.\n3) The visualizations well demonstrate the efficacy of the proposed model and motivation.\n4) The model is computationally efficient. It achieves sota, and the experiments on two kinds of all-in-one settings are thorough, while other algorithms only experiment on a single setting.", "weaknesses": "1) The comparisons using only PSNR/SSIM seem insufficient to measure the quality of the resulting images. LPIPs can be used to further verify the proposed model's superiority.\n2) The architecture is mainly based on Restormer. However, Tab. 1 does not include Restormer's performance.\n3) The paper lacks comparisons with recent methods, such as Harmony in Diversity: Improving All-in-One Image Restoration via Multi-Task Collaboration.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7I8zh4Cwx5", "reviewer": "Reviewer_FFqs", "summary": "This paper proposes an adaptive all-in-one image restoration model called AdaIR using frequency information. A key motivation is that different degradation types impact the image’s different frequency sub-bands. AdaIR mines low- and high-frequency information from input features (FMiM) and modulates the extracted features by facilitating the cross-interaction between low-frequency mined features and high-frequency mined features (FMoM).", "questions": "- Could the authors provide both the original and enlarged images in the figures? If this is not possible, assessing the performance of the proposed methods becomes challenging. For instance, while AdaIR appears to excel in rain removal, other models might perform better in different regions.\n- The total number of parameters appears large (28.77M). Can authors add a column to their comparison tables (e.g. Tables 1 and 2) showing the number of parameters for each model? This would allow for a more comprehensive comparison of model efficiency alongside performance metrics.\n- Can the authors clarify the term \"Fixed\" in Table 4? It is unclear whether the mask value is also fixed or not. The current caption in Table 4 only mentions that the mask shape is fixed at $10 \\times 10$.\n- Could the authors demonstrate that AdaIR, as an all-in-one model, performs well on unseen and mixed degradation using the UDC-SIT dataset [8], which features complex degradations?\n- It would be helpful if the authors could explain more clearly the significance of the channel-wise features and the corresponding attention scores depicted in Figure 7.\n- Can the authors elaborate on why AdaIR outperforms the Encoder + Decoder + AFLB configuration presented in Table 12? The explanation should include the advantages of AdaIR's design compared to the alternative.\n- The additional ablation study regarding the design choices of FMoM in Appendix B may not be entirely fair. For example, the proposed design in Figure 8(b) does not initially concatenate $X_h$​ and $X_l$. Instead, each feature passes through the H-L or L-H unit before being cross-multiplied. The reviewer believes that the spatial attention comparison in Figure 8(a) should also avoid concatenation initially; instead, $X_h$​ and $X_l$​ should go through channel attention separately before their respective spatial attention maps are cross-multiplied. Could the authors provide a comparison based on this suggestion?\n\n[8] Ahn, K., Ko, B., Lee, H., Park, C., & Lee, J. (2024). UDC-SIT: a real-world dataset for under-display cameras. Advances in Neural Information Processing Systems, 36.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The motivation for incorporating frequency information in an all-in-one image restoration model is well-founded. The authors substantiate their claim that various degradation types influence image content across frequency subbands. For instance, the t-SNE visualizations in Figures 2 and 11 illustrate how degradations can be clustered based on their types.\n- The authors successfully demonstrate the function of each module within the model through extensive ablation studies. For example, in Figure 9, the visualizations of intermediate features and spectra effectively highlight the roles of the AFLB module, which is composed of the FiMM and FMoM components.", "weaknesses": "- The authors claim that all previous approaches operate solely in the spatial domain and do not consider frequency information (L086-L087). However, several studies [1-2] have already explored the frequency domain. The authors should clarify how their approach differs from or advances these existing frequency-based methods. Given the existence of prior frequency domain exploration, the primary motivation for this work may need to be revised.\n- The proposed AdaIR appears to be a straightforward combination of existing techniques, such as Restormer [3] and CBAM [4]. Specifically, all encoders and decoders utilize multiple Transformer blocks from Restormer, and the cross attention (CA) mechanisms in both FMiM and FMoM leverage the MDTA approach from Restormer. Additionally, the H-L unit in FMoM seems to replicate the channel attention mechanism found in CBAM. \n- The authors should appropriately cite the sources in both the figures and the main text. For example, in Figure 3, Restormer should be referenced for the TB in (a) and the CA in (d). Furthermore, CBAM should be cited since its channel attention mechanism is employed in the H-L unit of FMoM, both in Figure 3 and the main text. The authors should add citations to Restormer and CBAM in the caption of Figure 3, as well as in the relevant sections of the main text where these components are described.\n- The roles of the H-L and L-H units appear to be similar in facilitating cross-interaction between low and high-frequency mined features. However, the authors have implemented different designs for these two units. It would be helpful for the authors to clarify the rationale behind using distinct methods for each.\n- The metrics in the tables require clarification. For instance, Table 1 should include column headings that explicitly indicate the inclusion of PSNR and SSIM. The authors should add column headers in Table 1 (and any other relevant tables) to clearly indicate which metrics are being reported (e.g. \"PSNR / SSIM\").\n- The benchmark models referenced are somewhat outdated. The authors should consider updating the comparison models to more recent ones [5-7]. Additionally, they should include comparisons with all-in-one restoration models that account for frequency information [1-2].\n\n\n[1] Wen Y, Zhang K, Zhang J, Chen T, Liu L, Luo W. Frequency-oriented efficient transformer for all-in-one weather-degraded image restoration. IEEE Transactions on Circuits and Systems for Video Technology. 2023 Jul 27.\n\n[2] Shi Z, Su T, Liu P, Wu Y, Zhang L, Wang M. Learning Frequency-Aware Dynamic Transformers for All-In-One Image Restoration. arXiv preprint arXiv:2407.01636. 2024 Jun 30.\n\n[3] Zamir SW, Arora A, Khan S, Hayat M, Khan FS, Yang MH. Restormer: Efficient transformer for high-resolution image restoration. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition 2022 (pp. 5728-5739).\n\n[4] Woo S, Park J, Lee JY, Kweon IS. Cbam: Convolutional block attention module. InProceedings of the European conference on computer vision (ECCV) 2018 (pp. 3-19).\n\n[5] Ai Y, Huang H, Zhou X, Wang J, He R. Multimodal Prompt Perceiver: Empower Adaptiveness Generalizability and Fidelity for All-in-One Image Restoration. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024 (pp. 25432-25444).\n\n[6] Park D, Lee BH, Chun SY. All-in-one image restoration for unknown degradations using adaptive discriminative filters for specific degradations. In2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023 Jun 17 (pp. 5815-5824). IEEE.\n\n[7] Yang H, Pan L, Yang Y, Liang W. Language-driven All-in-one Adverse Weather Removal. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024 (pp. 24902-24912).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2XfkodHuP0", "reviewer": "Reviewer_AFeH", "summary": "This work solves the all-in-one problem in image restoration from a frequency perspective and demonstrates its effectiveness. But my main problem is that I think the insights and designed modules are fallen behind and there are already many homogenized work. Therefore, the novelty of this paper does not appeal to me.", "questions": "Some works as following also have discussed using the frequency domain perspective to solve various degradation problems. [1] and [2] observe a specific degradation from the perspective of frequency domain.  [3] and [4] also adopt frequency mining and modulation to solve various degradations condition. It is suggested  to provide a point-by-point comparison of the proposed method with these works, highlighting key similarities and differences in approach and performance.\n\n[1] Efficient Frequency Domain-Based Transformers for High-Quality Image Deblurring, CVPR, 2023\n[2] Wavelet-based Fourier Information Interaction with Frequency Diffusion Adjustment for Underwater Image Restoration, CVPR, 2024\n[3] Hybrid Frequency Modulation Network for Image Restoration, IJCAI, 2024\n[4] Selective Frequency Network for Image Restoration, ICLR, 2023", "limitations": "", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The AdaIR model offers significant advantages by adaptively addressing various types of image degradations through innovative frequency analysis. Its frequency mining module precisely extracts relevant frequency components that reflect underlying degradations, while the frequency modulation module optimizes these components by allowing the exchange of complementary information across different features. Integrated into a U-shaped Transformer backbone, these components enable AdaIR to achieve state-of-the-art performance across diverse image restoration tasks, highlighting its robust adaptability and effectiveness.", "weaknesses": "1. In the t-sne clustering in Figure 2, the model is better at learning discriminative degradation contexts. First, the comparison method is relatively old, so some more recent comparison methods is suggested in the t-SNE analysis.\n\n2. Moreover, I do not think that this completely separated state can help solve the all-in-one problem more effectively. In fact, the correlation between various degradations may need to be learned to solve it better. For example, an image of rainy scene should also have heavy haze, so these two degradations should have something in common. And the rain lines can be regarded as a kind of noise, so they may also be correlated in AirNet and PromptIR of Fig 2, rather than completely decoupled.  In other word, I doubt that the reason for the decoupling of the t-sne analysis may be the overfitting of the model on each dataset.\n\n3. As the core motivation of this work, the idea in Figure 1 has been discussed extensively, and I think it is not innovative. Please see the question section for detailed discussion.\n\n4. The method design is also based on the combination of the very traditional U-Net network and the attention mechanism. This design has obviously fallen behind. Diffusion model may be a more proper framework to solve this problem.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)