UniRestorer: Universal Image Restoration via Adaptively Estimating Image Degradation at Proper Granularity

## Abstract

Abstract Recently, considerable progress has been made in all-in-one image restoration.
Generally, existing methods can be degradation-agnostic or degradation-aware.
However, the former are limited in leveraging degradation-specific restoration, and the latter suffer from the inevitable error in degradation estimation.
Consequently, the performance of existing methods has a large gap compared to specific single-task models.
In this work, we make a step forward in this topic, and present our UniRestorer with improved restoration performance.
Specifically, we perform hierarchical clustering on degradation space, and train a multi-granularity mixture-of-experts (MoE) restoration model.
Then, UniRestorer adopts both degradation and granularity estimation to adaptively select an appropriate expert for image restoration.
In contrast to existing degradation-agnostic and -aware methods, UniRestorer can leverage degradation estimation to benefit degradation-specific restoration, and use granularity estimation to make the model robust to degradation estimation error.
Experimental results show that our UniRestorer outperforms state-of-the-art all-in-one methods by a large margin, and is promising in closing the performance gap to specific single-task models.
The code and pre-trained models will be publicly released.

## 1 Introduction

Figure: Figure 1: Illustration of representative all-in-one image restoration frameworks. (a) Degradation-agnostic methods [85, 105] train a shared backbone using data from all tasks and are limited in leveraging degradation-specific restoration. (b) PEFT-based methods [40, 102, 67, 82, 24, 91, 4] apply learnable prompts or adapters to the backbone to adapt various tasks. (c) Expert-based methods [95, 14, 12, 87, 47, 66, 11, 19, 48, 94, 117] train specific MoE, LoRA, or filters for specific tasks. However, both (b) and (c) suffer from the inevitable error in degradation estimation. (d) We construct a multi-granularity degradation set and a multi-granularity MoE restoration model. By adaptively estimating image degradation at proper granularity, our UniRestorer can be effective in leveraging degradation-specific restoration while being robust to degradation estimation error.
Refer to caption: x1.png

Image restoration is a fundamental task in computer vision, it aims to restore high-quality (HQ) images from corresponding low-quality (LQ) counterparts, including denoising [111, 1], deblurring [112, 64, 73], de-weathering [53, 57, 101, 23], low-light enhancement[113, 86, 93], etc.
In the past decade, advanced deep neural architectures [27, 83, 58, 110, 70] have driven image restoration methods to evolve from task-specific [103, 20, 34, 72, 79, desnownet] to task-agnostic [99, 49, 15, 100, 51] backbones.
Recently, influenced by the success of foundation models in natural language processing and high-level vision, there has been increasing interest in addressing multiple image restoration tasks within a single framework, known as all-in-one image restoration [32].

A straightforward approach is to utilize data from all tasks to train a degradation-agnostic model (see Fig. [1](https://arxiv.org/html/2412.20157v3#S1.F1) (a)), but it easily yields unsatisfactory results due to the conflicts between certain tasks (*e.g*., denoising and deblurring).
Inspired by parameter-efficient fine-tuning (PEFT) methods [9, 28],
learnable prompts [40, 102, 67, 82, 24, 91] and adapters [4] are integrated into the restoration model to modulate intermediate features (see Fig. [1](https://arxiv.org/html/2412.20157v3#S1.F1) (b)), allowing the model to process different tasks more specifically.
Furthermore, some methods introduce mixture-of-experts (MoE) [95, 14, 12, 87, 47] (MoE-like using filters [66, 11, 19, 48, 94, 117] and low-rank adaptation [81, 3]) modules, routing the current feature to specific experts based on deep features and estimated degradation (see Fig. [1](https://arxiv.org/html/2412.20157v3#S1.F1) (c)).

Despite rapid progress, these methods show limited performance and fall significantly behind specific single-task models.
Degradation-agnostic models cannot leverage the benefits from degradation estimation-based priors, limiting their effectiveness in all-in-one image restoration.
PEFT-based and MoE-based methods provide degradation-aware ways for processing different corruptions by conditioning on degradation estimation.
However, their effectiveness heavily relies on accurate degradation estimation, which is inherently challenging due to the vast and diverse degradation space of all-in-one image restoration.
Consequently, degradation-aware models are inevitably vulnerable to estimation errors, which can lead to inappropriate prompt or expert routing, ultimately degrading restoration quality.

From the above analysis, one plausible solution to improve all-in-one restoration is to estimate and leverage image degradation while being robust to degradation estimation error.
To this end, we propose to represent image degradation at multiple levels of granularities rather than a single-level, and construct a multi-granularity degradation representation (DR) set by hierarchizing the degradation space.
We then build a corresponding multi-granularity MoE restoration model, where fine-grained experts specialize in specific degradations to enhance restoration accuracy, while coarse-grained experts generalize across broader degradation spaces.
To enable more robust expert routing for a degraded image, beyond vanilla degradation estimation, we introduce granularity estimation to indicate the degree of degradation estimation error.
For example, in Fig. [1](https://arxiv.org/html/2412.20157v3#S1.F1) (d), given an image suffering from large snow degradation, if the degradation estimation is ambiguous, the granularity estimation will assign a coarser-grained expert, *e.g*., a model for general snow, to alleviate the effect of estimation error. Conversely, when the degradation estimation is confident, granularity estimation will assign a finer-grained expert to maximize the restoration quality, *e.g*., a model for large snow.
By jointly leveraging both degradation and granularity estimation, our approach can mitigate the adverse effects of inaccurate degradation estimation, achieving high restoration and generalization performance simultaneously.

Specifically, we take the following steps to implement our UniRestorer.
First, we optimize a degradation extractor which is aware of finer-grained image degradation.
Second, all low-quality images in the training set are fed into the degradation extractor, and their features are then hierarchically clustered into multiple DR groups at different granularities, respectively, which forms the multi-granularity degradation set.
Third, we train a multi-granularity MoE restoration model based on the constructed multi-granularity degradation set.
Fourth, given an image with unknown degradation, we learn routers based on both degradation and granularity estimation results. The router identifies which DR group the degradation and granularity best match in the degradation set, and then allocates the corresponding expert for inference.

Figure: Figure 2: Comparisons with task-agnostic methods, all-in-one methods, and single-task models. Ours and Ours†denote auto and instruction modes, which are respectively used for a fair comparison with all-in-one and specific single-task models. (a) Comparisons on single-degradation all-in-one setting. (b) Comparisons with specific single-task models. (c) Comparisons on mixed-degradation all-in-one (in-of-distribution) setting. (d) Comparisons on mixed-degradation all-in-one (out-of-distribution) setting.
Refer to caption: x2.png

To demonstrate the effectiveness of the proposed method, we conduct experiments on both single-degradation and mixed-degradation scenarios.
For the single-degradation scenario, we unify seven widely studied image restoration tasks. For the mixed-degradation scenario, we introduce six types of degradations.
In addition, we conduct a comprehensive comparative evaluation.
The results indicate that our method not only significantly outperforms existing all-in-one methods but also achieves performance that is competitive with, or even superior to, specific single-task models (see Fig. [2](https://arxiv.org/html/2412.20157v3#S1.F2)).

The main contributions can be summarized as follows:

- •
In this paper, we propose UniRestorer, a novel all-in-one image restoration framework that can estimate and leverage robust degradation representation to adaptively handle diverse image corruptions.
- •
We propose to train a multi-granularity MoE restoration model, where finer-experts specialize in specific degradations to enhance restoration accuracy, while coarser-experts generalize on broader degradation spaces to ensure generalization ability.
- •
To alleviate the impact of degradation estimation error on expert routing, we propose to estimate a proper granularity for image degradation. By jointly leveraging degradation and granularity estimation, our method adaptively allocates the most appropriate expert for different corruptions.
- •
Extensive experiments are conducted on single-degradation and mixed-degradation scenarios, showing that our method significantly outperforms existing all-in-one methods and effectively closes the performance gap with specific single-task models.

## 2 Related Work

Image Restoration for Specific Tasks
Pioneer works propose tailored frameworks for specific degradations, *e.g*., DnCNN [103] for denoising, SRCNN [20] for superresolution, DeblurGANv2 [34] for deblurring, PReNet [72] for draining, Dehazeformer [79] for dehazing, and DesnowNet [desnownet] for desnowing.
Benefits from advanced deep neural networks, some works tend to develop task-agnostic models, such as MPRNet [99], SwinIR [49], NAFNet [15], and Restormer [100], which become strong baselines in various image restoration tasks.

PEFT-based All-in-One Image Restoration
To adapt to various corruptions, pioneer all-in-one image restoration methods [67, 102, 82, 40, 91, 4] integrate PEFT-based techniques [9, 28] and feature modulation [104] into the restoration backbone.
For example,
TransWeather [82] and AirNet [40] learn degradation-related embeddings as prompts to modulate deep features based on different corrupted inputs.
Similarly, PromptIR [67] and IDR [102] employ prompt learning in a more powerful backbone [100] and achieve state-of-the-art performance.
MiOIR [91] further exploits the benefits of incorporating explicit prompt and adaptive prompt learning.

Expert-based All-in-One Image Restoration
Mixture-of-Experts (MoE) has proven to be a solution to multi-task learning due to its conditional processing capability [17, 16, 116].
Although prompt learning-based techniques work similarly to MoE in conditional processing, they are still limited in performance and task range.
Recent MoE-based all-in-one image restoration methods can be categorized into implicit MoE [95, 19, 94, 11] and explicit MoE [117, 66, 97, 12, 81, 14, 3].
The experts in the former methods are implicitly learned in training and do not correspond to a specific task.
Methods belonging to explicit MoE learn experts for specific tasks, unleashing the power of additionally increased parameters.
Based on a shared backbone, WGWSNet [117] and ADMS [66] learn specific filters for specific tasks.
To be more efficient, DaAIR [97], InstructIPT [81], and LoRA-IR [3] learn individual LoRA weights or adapters for each task. By combining knowledge of the shared backbone and task-specific modules, these methods can adapt to different tasks effectively.
GRIDS [12] and RestoreAgent [14] divide one complete degradation space into subspaces and learn specific expert full models to alleviate potential conflicts between different degradations.

Despite progress achieved, most existing methods learn DRs at a single, coarse granularity and rely solely on degradation estimation as the routing condition, making them vulnerable to estimation errors. In contrast, our approach explores DRs across multiple granularities, jointly utilizing coarse levels (the same as task-agnostic settings) and finer levels beyond those used in existing methods.
In addition, we incorporate degradation estimation and its error into route process, enabling more robust decision-making.

## 3 Proposed Method

In this section, we introduce our all-in-one image restoration framework, UniRestorer.
We first optimize one degradation representation (DR) extractor that learns DRs at a fine-grained level.
Based on DRs extracted from all low-quality training data, we conduct hierarchical data clustering that separates DRs into multiple DR groups at different granularities (see Fig. [3](https://arxiv.org/html/2412.20157v3#S3.F3)), leading to a multi-granularity degradation set and MoE restoration model.
To alleviate the inevitable error caused by degradation estimation in routing, we adopt both degradation and granularity estimation to allocate the most appropriate expert to given unknown corrupted inputs.

### 3.1 Preliminaries

Mixture-of-Experts [75, 22, 37, 118, 68, 71] is a promising way for scaling up and deploying large or gigantic models due to its competitive computational bounds and latency compared to the vanilla single model.
Expert network set $\{\mathcal{F}\}$ and routing network $\mathcal{G}(\cdot)$ are two basic components of MoE architecture. The expert network can be a sub-layer or a sub-network (*e.g*., convolution layer or feed-forward layer), and it also can be an independently trained network.
Given input $\mathbf{X}$, the routing network $\mathcal{G}(\cdot)$ is used to calculate the contribution of each expert to the final output,

$$ $y=\sum_{i=0}^{N}\mathcal{G}(h)\cdot\mathcal{F}_{i}(\mathbf{X}),$ (1) $$

where $h$ is commonly a type of representation of input $\mathbf{X}$ which the routing network can assign expert conditioning on.
The routing network is typically a noisy top-k routing network parameterized with $w_{g}$ and $w_{n}$, which models $P(\mathcal{F}_{k}|h)$ as the probability of using the $k$-th expert $\mathcal{F}_{k}$ and selects experts with top-k probability to contribute to the final output. The noisy routing process can be described as follows,

$$ $\small\mathcal{G}(h)=\operatorname{Softmax}(h\cdot w_{g})+\mathcal{N}(0,1) \operatorname{Softplus}(h\cdot w_{n}),$ (2) $$

where $\operatorname{Softplus}$ is the smooth approximation to the function $\operatorname{ReLU}$. Then, the $\operatorname{TopK}(\cdot,k)$ function is used to select the experts with the largest $k$ values according to $\mathcal{G}(h)$.

Figure: Figure 3: Illustration of our proposed UniRestorer. We develop a multi-granularity degradation set by hierarchical clustering on extracted DRs at different granularities. Based on the multi-granularity degradation set, we train a multi-granularity MoE restoration model. Besides vanilla degradation estimation, we introduce granularity estimation to indicate the degree of degradation estimation error. Adopting both degradation and granularity estimation, we train routers to adaptively allocate an expert to unknown corrupted input.
Refer to caption: x3.png

### 3.2 Multi-Granularity Degradation and Expert

Fine-Grained DR Extractor
Considering the convenience of using text to represent the type of degradation on corrupted images,
we adopt a CLIP-based method, DA-CLIP [59], as our basic backbone.
Using contrastive learning, DA-CLIP trains an image controller to separate different corrupted inputs.
However, aiming to exploit finer-grained DRs, it is essential to enable the DR extractor to be aware of finer-grained representation of degradation.
Motivated by the above, we train a fine-grained DR extractor to fit our goal.
We construct a dataset with fine-grained level degradations.
In our experiments, we found that the estimation of degradation degree is sensitive to distortions, *e.g*., noise can be heavily alleviated by transformations in the pre-processing of CLIP.
To alleviate the effect of distortion on the original degradation,
In training, we first crop high-resolution clean images into $224\times 224$ patches and then add synthetic degradations with different types and degrees to obtain low-quality inputs.
During inference, we center-crop the main body of given low-quality inputs to alleviate the influence of the resolution gap between training and inference.
Following [59], we adopt the same training setups to train our DR extractor.
The fine-grained DR $e$ can be learned by the DR extractor $\mathcal{D}$, *i.e*.,

$$ $e=\mathcal{D}(\mathbf{X}).$ (3) $$

When training is finished, we employ $\mathcal{D}$ to obtain the DR set $\{e\}$ from all training samples.

Hierarchical Degradation Clustering
To construct the proposed multi-granularity degradation set, we utilize an unsupervised data clustering approach [62], which is simple yet effective.
The constructed degradation set is hierarchical.
The $i$-th level of granularity consists of non-overlapped DR groups, each of which is related to a data distribution, and samples drawn from one distribution are with similar DRs.
In our implementation, we construct a hierarchical degradation set with $n$ levels of granularities ($n$$=$$3$).
The coarsest level is 0 0-th level and the finest level is $($$n$$-$$1$$)$-th level.
We hierarchically cluster training samples from top to down.
For example, we first conduct data clustering at the coarsest-grained level by,

$$ $\{u\}\leftarrow\operatorname{K-means}(\{e\}),$ (4) $$

where $\{u\}$ is the set of cluster centers of finer-grained level. Based on $\{u\}$, we use function $\operatorname{argmin}_{u_{j}\in\{u\}}(||e-u_{j}||_{2}^{2})$ to decide which cluster the samples belong to, accordingly we separate data and $\{e\}$ into the same number of groups as cluster centers.
We hierarchically conduct the same process as Eq. ([4](https://arxiv.org/html/2412.20157v3#S3.E4)) based on newly obtained DR groups until the finest-grained level.
As a result, we construct a multi-granularity degradation set, and we can separate training samples into non-overlapped groups from the coarsest-grained level to the finest-grained level. Referring to the total number of degradation types and degrees in training DR extractor, we empirically set $1$, $7$, and $19$ clusters for the 0 0-th level to the $($$n$$-$$1$$)$-th level, respectively.

Multi-Granularity MoE Restoration Model
We train a multi-granularity MoE restoration model based on the obtained multi-granularity degradation set.
Expert networks are trained with $\ell_{1}$ loss as the reconstruction loss between predicted results $\hat{\mathbf{Y}}$ and target $\mathbf{Y}$, *i.e*.,

$$ $\ell_{1}=||\hat{\mathbf{Y}}-\mathbf{Y}||_{1}.$ (5) $$

Experts trained at coarser-grained levels can generalize to a large range of degradations, and experts trained in finer-grained levels can highly improve the restoration performance on degradations that they are good at.
Therefore, given corrupted inputs, we can estimate their DRs and assign them to the most appropriate expert.

Note that our main contribution lies in the design of estimating and leveraging robust DRs to improved all-in-one image restoration performance. To achieve this, we adopt MoE framework, where we follow prior works such as GRIDS [12] and RestoreAgent [14], employing SOTA image restoration networks as experts. Though straightforward, this strategy is simple and effective in practice. Moreover, our approach is orthogonal to the specific design of the MoE structure and can be readily combined with lightweight expert architectures or efficient fine-tuning techniques, such as LoRA [29, 28].
More discussion can be found in Sec. C of Suppl.

### 3.3 Routing via Degradation and Granularity Estimation

It makes sense to use estimated DRs as the routing condition $h$ in Eq. ([1](https://arxiv.org/html/2412.20157v3#S3.E1)).
However, degradation estimation is error-prone, which may lead to inaccurate routing and thus impede restoration performance.
To this end, based on the constructed multi-granularity degradation set, we estimate image degradation at the proper granularity level, and perform routing based on both degradation and granularity estimation, which makes our method robust to degradation estimation error.
We train two separate branches ($\mathcal{H}_{d}$ and $\mathcal{H}_{g}$) upon $\mathcal{D}$ for degradation estimation vector $e_{deg}$ at the finest-grained level and granularity estimation vector $e_{gran}$.
Then, we further train two routers $\mathcal{G}_{d}(\cdot)$ and $\mathcal{G}_{g}(\cdot)$ as Eq. ([2](https://arxiv.org/html/2412.20157v3#S3.E2)) based on degradation and granularity estimation, respectively.
The routing process can be described as follows,

$$ $\displaystyle\mathcal{F}_{n-1,k}$ $\displaystyle=\operatorname{TopK}(\mathcal{G}_{d}(e_{deg})|\{\mathcal{F}\}_{n- 1})$ (6) $\displaystyle\mathcal{F}_{i,j}$ $\displaystyle=\operatorname{TopK}(\mathcal{G}_{g}(e_{gran})|\{\mathcal{F}_{0,0 },\cdots,\mathcal{F}_{n-1,k}\}).$ $$

$\mathcal{G}_{d}$ is conducted in the finest-grained level. Based on its routing result, it further selects a set of experts that can solve degradations belonging to DR group $\{e\}_{n-1,k}$ and are trained in different granularity levels from 0 0 to $n$$-$$1$.
Furthermore, $\mathcal{G}_{g}$ is responsible for selecting an expert training in the proper granularity according to granularity estimation.

Inspired by data uncertainty learning [13], we learn $e_{deg}$ and $e_{gran}$ by introducing loss as,

$$ $\mathcal{L}_{dg}=\frac{1}{2e_{gran}}(u_{y_{i}}-e_{deg})^{2}+\frac{1}{2} \operatorname{ln}e_{gran},$ (7) $$

where $u_{y_{i}}$ is the weight center in the finest-grained level obtained by Eq. ([4](https://arxiv.org/html/2412.20157v3#S3.E4)).
We use the first term to build the relationships of $e_{deg}$ and $e_{gran}$, when the distance between current degradation estimation and corresponding weight center is high, a larger $e_{gran}$ is used to ensure that $\mathcal{L}_{dg}$ is not too high. Therefore, the granularity can be seen as the degree of the current degradation estimation error.
If estimated with a large error, granularity estimation will allocate a coarser-grained expert; Conversely, if degradation estimation is precise, granularity estimation will allocate a finer-grained expert.
Thus, conditioning on both $e_{deg}$ and $e_{gran}$, we can dynamically allocate given corrupted inputs with the most appropriate expert.
For stable training, and to avoid the result of $\mathcal{G}_{g}$ collapse to the finest-grained level, we further adopt load-balance loss as used in [75],

$$ $\mathcal{L}_{load}={\sigma}/{\mu},$ (8) $$

where $\mu$ and $\sigma$ are mean and standard deviation of experts load in granularity level, respectively.
At last, we train our routers by jointly adopting loss as,

$$ $\mathcal{L}_{total}=\ell_{1}+\alpha\mathcal{L}_{dg}+\beta\mathcal{L}_{load},$ (9) $$

where $\alpha$ and $\beta$ are two hyper-parameters to trade-off the effect of $\mathcal{L}_{dg}$ and $\mathcal{L}_{load}$.

Instruction Mode
Typically, our framework automatically conducts the processing pipeline of extraction-recognition-routing-restoration (auto mode). When users are convinced about the type of degradation, we provide options that users can provide the type of degradation to instruct the routing process (instruction mode), as shown in Fig. [3](https://arxiv.org/html/2412.20157v3#S3.F3). The instruction plays the role of pruning, it utilizes a set of masks to prune the corresponding DR groups from the multi-granularity degradation set, leading to a more efficient and accurate routing.

**Table 1: All-in-One image restoration (single-degradation) results. ‘$n$T’ means the average performance of $n$ tasks (3T: Derain, Dehaze, Denoise; 5T: Derain, Dehaze, Denoise, Deblur, Lowlight; 7T: all).**
|  | Derain | Dehaze | Denoise | Deblur | Lowlight | Desnow | CAR | Average |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Rain100L[96] | SOTS[39] | BSD68[61] | GoPro[64] | LOLv1[86] | Snow100K[57] | LIVE1[76] | 3T | 5T | 7T |  |  |  |  |  |  |  |
| Methods | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR |  |  |
| MPRNet [99] | 25.56 | 0.903 | 16.37 | 0.806 | 23.92 | 0.702 | 25.40 | 0.858 | 7.75 | 0.124 | 19.33 | 0.740 | 27.77 | 0.889 | 18.51 | 22.68 | 19.66 |
| SwinIR [49] | 21.20 | 0.837 | 15.94 | 0.775 | 24.12 | 0.780 | 16.86 | 0.735 | 13.80 | 0.409 | 18.60 | 0.718 | 15.61 | 0.746 | 17.55 | 17.09 | 18.45 |
| NAFNet [15] | 31.32 | 0.967 | 22.43 | 0.907 | 30.27 | 0.918 | 26.53 | 0.877 | 23.09 | 0.796 | 26.33 | 0.889 | 31.26 | 0.946 | 24.55 | 25.76 | 26.28 |
| Restormer [100] | 34.25 | 0.981 | 29.64 | 0.971 | 31.00 | 0.929 | 27.50 | 0.896 | 22.80 | 0.821 | 28.24 | 0.914 | 31.21 | 0.946 | 30.46 | 28.56 | 28.27 |
| DL-3T [21] | 32.62 | 0.931 | 26.92 | 0.391 | 30.12 | 0.838 | - | - | - | - | - | - | - | - | 28.09 | - | - |
| AirNet-3T [40] | 34.90 | 0.967 | 27.94 | 0.962 | 31.06 | 0.872 | - | - | - | - | - | - | - | - | 29.29 | - | - |
| PromptIR-3T [67] | 36.37 | 0.972 | 30.58 | 0.974 | 31.12 | 0.873 | - | - | - | - | - | - | - | - | 31.50 | - | - |
| InstructIR-3T [19] | 37.98 | 0.978 | 30.22 | 0.959 | 31.32 | 0.875 | - | - | - | - | - | - | - | - | 31.49 | - | - |
| DaAIR-3T [97] | 37.10 | 0.978 | 32.30 | 0.981 | 31.06 | 0.868 | - | - | - | - | - | - | - | - | 32.51 | - | - |
| PromptIR-TUR-3T [88] | 38.57 | 0.984 | 31.17 | 0.978 | 31.19 | 0.872 | - | - | - | - | - | - | - | - | 32.67 | - | - |
| MoCEIR-3T [98] | 38.57 | 0.984 | 31.34 | 0.979 | 31.24 | 0.873 | - | - | - | - | - | - | - | - | 32.73 | - | - |
| DL-5T [21] | 21.96 | 0.762 | 20.54 | 0.826 | 23.09 | 0.745 | 19.86 | 0.672 | 19.83 | 0.712 | - | - | - | - | 21.02 | 20.28 | - |
| Transweather-5T [82] | 29.43 | 0.905 | 21.32 | 0.885 | 29.00 | 0.841 | 25.12 | 0.757 | 21.21 | 0.792 | - | - | - | - | 23.31 | 24.41 | - |
| TAPE-5T [52] | 29.67 | 0.904 | 22.16 | 0.861 | 30.18 | 0.855 | 24.47 | 0.763 | 18.97 | 0.621 | - | - | - | - | 24.10 | 24.28 | - |
| AirNet-5T [40] | 32.98 | 0.951 | 21.04 | 0.884 | 30.91 | 0.882 | 24.35 | 0.781 | 18.18 | 0.735 | - | - | - | - | 23.82 | 24.10 | - |
| IDR-5T [102] | 35.65 | 0.965 | 25.20 | 0.938 | 31.09 | 0.883 | 26.65 | 0.810 | 20.70 | 0.820 | - | - | - | - | 27.36 | 26.86 | - |
| InstructIR-5T [19] | 36.84 | 0.973 | 27.10 | 0.956 | 31.40 | 0.887 | 29.40 | 0.886 | 23.00 | 0.836 | - | - | - | - | 28.99 | 29.19 | - |
| DaAIR-5T [97] | 36.28 | 0.975 | 31.97 | 0.980 | 31.07 | 0.878 | 29.51 | 0.890 | 22.38 | 0.825 | - | - | - | - | 32.20 | 30.24 | - |
| Transweather-TUR-5T [88] | 33.09 | 0.952 | 29.68 | 0.966 | 30.40 | 0.869 | 26.63 | 0.815 | 23.02 | 0.838 | - | - | - | - | 29.96 | 28.56 | - |
| DCPT-PromptIR-5T [31] | 37.32 | 0.978 | 30.72 | 0.977 | 31.32 | 0.885 | 28.84 | 0.877 | 23.35 | 0.840 | - | - | - | - | 31.46 | 30.31 | - |
| MoCEIR-5T [98] | 38.04 | 0.982 | 30.48 | 0.974 | 31.34 | 0.887 | 30.05 | 0.899 | 23.00 | 0.852 | - | - | - | - | 31.39 | 30.58 | - |
| Ours | 41.68 | 0.996 | 36.44 | 0.984 | 31.43 | 0.936 | 31.48 | 0.948 | 26.67 | 0.863 | 31.12 | 0.940 | 30.57 | 0.926 | 36.71 | 33.38 | 31.34 |

**Table 2: All-in-One image restoration (mixed-degradation) results. The performance is evaluated on DIV2K-valid with in-dist. and out-dist. degradation parameters. ‘$n$T’ means the average performance of $n$ tasks (3T: H-R, H-N, H-N-R; 5T: H-R, H-N, H-N-R, LL-H-N-R, LL-H-N-B-R; 7T: all).**
|  | H-R | H-N | H-N-R | LL-H-N-R | LL-H-N-S | LL-H-N-B-R | LL-H-N-B-S | Average |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 3T | 5T | 7T |
| Methods | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR |  |  |
| In-distribution |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MPRNet [99] | 17.22 | 0.691 | 19.02 | 0.717 | 17.18 | 0.639 | 17.25 | 0.549 | 17.18 | 0.583 | 16.99 | 0.466 | 16.80 | 0.506 | 17.80 | 17.57 | 17.37 |
| SwinIR [49] | 16.35 | 0.620 | 18.74 | 0.693 | 16.48 | 0.585 | 16.78 | 0.513 | 16.73 | 0.555 | 16.54 | 0.434 | 16.10 | 0.482 | 17.19 | 17.01 | 16.81 |
| NAFNet [15] | 19.14 | 0.775 | 22.71 | 0.843 | 18.56 | 0.723 | 17.41 | 0.603 | 17.09 | 0.631 | 16.37 | 0.505 | 15.83 | 0.530 | 20.13 | 18.98 | 18.15 |
| Restormer [100] | 19.85 | 0.795 | 20.44 | 0.820 | 17.64 | 0.719 | 16.86 | 0.588 | 16.56 | 0.623 | 15.92 | 0.483 | 15.47 | 0.520 | 19.31 | 18.27 | 17.53 |
| AirNet [40] | 18.67 | 0.752 | 23.34 | 0.855 | 19.03 | 0.726 | - | - | - | - | - | - | - | - | 20.34 | - | - |
| PromptIR [82] | 22.20 | 0.825 | 24.23 | 0.886 | 20.86 | 0.772 | - | - | - | - | - | - | - | - | 22.43 | - | - |
| MiOIR [91] | 24.58 | 0.892 | 24.46 | 0.879 | 22.42 | 0.818 | 19.46 | 0.661 | - | - | 18.12 | 0.561 | - | - | 23.82 | 21.80 | - |
| InstructIR [19] | 21.57 | 0.827 | 23.80 | 0.838 | 21.14 | 0.775 | 19.21 | 0.666 | - | - | 18.06 | 0.564 | - | - | 22.17 | 20.75 | - |
| OneRestore [24] | 17.35 | 0.711 | 18.70 | 0.669 | 16.41 | 0.617 | 16.66 | 0.621 | 20.24 | 0.769 | - | - | - | - | 17.48 | - | - |
| Ours | 29.63 | 0.950 | 26.93 | 0.880 | 26.33 | 0.875 | 24.29 | 0.830 | 22.71 | 0.790 | 21.87 | 0.735 | 20.10 | 0.729 | 27.63 | 25.81 | 24.55 |
| Out-of-distribution |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| MPRNet [99] | 15.71 | 0.636 | 16.38 | 0.598 | 15.57 | 0.560 | 14.84 | 0.345 | 14.59 | 0.355 | 14.58 | 0.286 | 14.33 | 0.299 | 15.88 | 15.41 | 15.14 |
| SwinIR [49] | 15.10 | 0.556 | 15.79 | 0.553 | 14.50 | 0.486 | 14.72 | 0.341 | 14.32 | 0.353 | 14.29 | 0.282 | 13.95 | 0.298 | 15.13 | 14.88 | 14.66 |
| NAFNet [15] | 16.77 | 0.713 | 19.68 | 0.751 | 15.11 | 0.595 | 14.40 | 0.385 | 14.05 | 0.390 | 13.89 | 0.317 | 13.27 | 0.324 | 17.18 | 16.00 | 15.31 |
| Restormer [100] | 17.44 | 0.723 | 17.31 | 0.738 | 17.44 | 0.723 | 14.11 | 0.378 | 13.59 | 0.388 | 13.60 | 0.314 | 12.83 | 0.318 | 17.39 | 15.97 | 15.18 |
| AirNet [40] | 17.50 | 0.732 | 20.16 | 0.777 | 15.07 | 0.380 | - | - | - | - | - | - | - | - | 17.57 | - | - |
| PromptIR [82] | 19.84 | 0.797 | 20.57 | 0.795 | 18.22 | 0.672 | - | - | - | - | - | - | - | - | 19.54 | - | - |
| MiOIR [91] | 20.85 | 0.850 | 22.39 | 0.821 | 18.87 | 0.700 | 15.27 | 0.406 | - | - | 14.70 | 0.332 | - | - | 20.70 | 18.41 | - |
| InstructIR [19] | 20.33 | 0.825 | 19.02 | 0.636 | 17.74 | 0.597 | 15.34 | 0.386 | - | - | 14.52 | 0.296 | - | - | 19.03 | 17.39 | - |
| OneRestore [24] | 17.81 | 0.732 | 16.77 | 0.580 | 15.10 | 0.595 | 17.49 | 0.566 | 21.04 | 0.682 | - | - | - | - | 16.56 | - | - |
| Ours | 27.32 | 0.937 | 23.82 | 0.811 | 23.42 | 0.801 | 20.21 | 0.661 | 20.15 | 0.631 | 17.49 | 0.529 | 18.03 | 0.543 | 24.85 | 22.45 | 21.49 |

## 4 Experiments

### 4.1 Experimental Setup

Task and Dataset
We train and evaluate our framework on single-degradation and mixed-degradation scenarios.
In single-degradation scenario, we train a unified model to handle seven distinct image restoration tasks. We collect and integrate data from seven widely studied tasks. We adopt Rain100L [96] for deraining (‘R’), RESIDE [39] for dehazing (‘H’), DFBW [2, 50, 61, 60] for Gaussian color denoising (‘N’) and compression artifacts removal (‘CAR’), GoPro [64] for single-image motion deblurring (‘B’), LOLv1 [86] for low-light enhancement (‘LL’), and Snow100K [57] for desnowing (‘S’).
In mixed-degradation scenario, we develop a more complex degradation pipeline based on [105], which includes six types of degradation. We design seven challenging mixed degradation scenarios for evaluation. The training data is synthesized from clean images in DF2K [2, 50].
To demonstrate the generalization ability, we conduct comparisons on real-world and unseen degradations, including LHP [25] for real-world deraining, LOLv2 [93] for real-world low-light enhancement, RealSnow [57] for real-world desnowing, RainDrop [69] for raindrop removal, TOLED [115] for under-display camera image restoration, UIEB [41] for underwater image restoration, and out-of-distribution (out-dist.) degradation parameters for mixed-degradation.

Figure: Figure 4: Visual comparison of results on All-in-One image restoration (single-degradation). Our method can effectively remove the degradation pattern (*i.e*., rainstreak, haze, and noise), restore clearer texture (*i.e*., ‘branches’ and ‘eyes’ pointed by red arrows) and closer color (*i.e*., ‘bridge’). More results can be seen in Suppl.
Refer to caption: x4.png

**Table 3: Compare to single-task image restoration models. The task-agnostic methods are trained from scratch on each task. † denotes instruction mode. PSNR scores are reported.**
| Methods | Derain | Denoise | Desnow | CAR. | Lowlight | Deblur | Dehaze |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | 200H | 200L | DID | DDN | BSD68[61] | Snow100K[57] | LIVE1[76] | LOLv1 | GoPro | SOTS |  |  |  |  |  |  |  |
|  | [96] | [96] | [101] | [23] | $\sigma$=$15$ | $\sigma$=$25$ | $\sigma$=$50$ | S | M | L | $q$=$10$ | $q$=$20$ | $q$=$30$ | $q$=$40$ | [86] | [64] | [39] |
| MPRNet [99] | 31.53 | 35.92 | 35.34 | 34.19 | 34.91 | 32.67 | 29.62 | 36.22 | 34.49 | 30.63 | 27.87 | 30.23 | 31.57 | 32.50 | 23.51 | 32.66 | 31.07 |
| SwinIR [49] | 29.69 | 38.35 | 34.51 | 33.26 | 34.81 | 32.51 | 29.29 | 33.45 | 32.00 | 28.40 | 26.16 | 28.62 | 29.93 | 30.81 | 19.87 | 29.09 | 28.35 |
| Restormer [100] | 30.98 | 40.33 | 35.26 | 34.16 | 34.92 | 32.70 | 29.70 | 36.50 | 34.77 | 30.86 | 27.89 | 30.21 | 31.56 | 32.48 | 23.66 | 32.92 | 33.18 |
| NAFNet [15] | 29.08 | 38.93 | 35.21 | 32.84 | 34.80 | 32.58 | 29.50 | 35.94 | 33.95 | 29.50 | 27.95 | 30.24 | 31.56 | 32.47 | 23.75 | 33.69 | 22.70 |
| Ours† | 32.01 | 41.61 | 35.50 | 34.40 | 35.00 | 32.75 | 29.75 | 36.94 | 35.18 | 31.14 | 27.94 | 30.31 | 31.66 | 32.58 | 27.25 | 33.71 | 37.45 |

**Table 4: Generalization performance on real-world datasets. All methods are evaluated without additional training. PSNR scores are reported.**
|  | Deraining | Low-light | Desnowing | Average |
| --- | --- | --- | --- | --- |
| Methods | LHP [25] | LOLv2 [93] | RealSnow [117] |  |
| MPRNet [99] | 29.16 | 9.67 | 25.82 | 27.10 |
| SwinIR [49] | 21.74 | 18.67 | 18.45 | 20.91 |
| NAFNet [15] | 26.04 | 26.46 | 27.51 | 26.33 |
| Restormer [100] | 27.23 | 28.92 | 26.73 | 27.26 |
| PromptIR [67] | 25.68 | - | - | - |
| MiOIR [91] | 28.69 | 10.62 | - | - |
| InstructIR [19] | 28.93 | 30.31 | - | - |
| OneRestore [24] | 24.55 | 17.76 | 20.12 | 23.24 |
| Ours | 30.04 | 30.40 | 28.04 | 29.70 |

Baseline and Metric
We adopt task-agnostic methods [99, 49, 100, 15] and recent all-in-one methods [21, 82, 52, 40, 67, 102, 19, 24, 98, 31, 88, 97] as baselines.
In comparison, the performance of task-agnostic methods and ours is reported using models trained on seven tasks and the developed mixed degradation space in single-degradation and mixed-degradation setups, respectively.
For other methods, we follow the evaluation setting of [14] and only evaluate on tasks they support.
We adopt PSNR and SSIM as metrics. In comparison of single-task models, ablation study, and Suppl., we only report PSNR.
Following [67], metrics are calculated in RGB-channels, except that metrics are calculated on Y channel of YCbCr color space in single deraining task comparison [100].

**Table 6: Effect of DR extractors.**
|  | Rain | Rain | Test | Test |
| --- | --- | --- | --- | --- |
| Methods | 200H | 200L | 1200 | 2800 |
| VGG [78] | 31.47 | 40.59 | 35.26 | 34.18 |
| DDR [54] | 30.97 | 40.37 | 35.25 | 34.17 |
| DA-CLIP [59] | 31.65 | 40.67 | 35.24 | 34.12 |
| Manual | 31.88 | 41.43 | 35.32 | 34.23 |
| Ours | 32.01 | 41.61 | 35.50 | 34.40 |

Training Details
For the training of experts, we generally use the AdamW optimizer with batch-size of 8 and patch-size of 128, we take the CosineAnnealing learning rate scheduler with the initial learning rate of $3e^{-4}$. After the training, the weights of experts are frozen, and we train degradation and granularity estimation based routers. For the training of routers, we adopt Adam optimizer with batch-size of 8 and patch-size of 256, we adopt a fixed learning rate of $1e^{-3}$, and we set $\alpha$ to $0.1$ and $\beta$ to $0.01$ in Eq. ([9](https://arxiv.org/html/2412.20157v3#S3.E9)).
All experiments are trained on 8 NVIDIA A6000 GPUs.
More details of the experimental setups and training details can be seen in Suppl.

### 4.2 Comparison with State-of-The-Art Methods

Comparison with All-in-One Models
As shown in Tab. [1](https://arxiv.org/html/2412.20157v3#S3.T1) and Fig. [4](https://arxiv.org/html/2412.20157v3#S4.F4), our method can significantly outperform task-agnostic and all-in-one methods by a large margin in single-degradation.
In mixed-degradation, our method continuously achieves higher performance than task-agnostic and all-in-one methods from simple to complex mixed degradation (Tab. [2](https://arxiv.org/html/2412.20157v3#S3.T2)).
In addition, comparisons on out-dist., real-world (Tab. [5](https://arxiv.org/html/2412.20157v3#S4.T5)), and unseen (Tab. [5](https://arxiv.org/html/2412.20157v3#S4.T5)) image restoration datasets reveal that our method also guarantees better generalization performance than others.
We found that it becomes challenging for all-in-one models to achieve significant performance gains over SOTA task-agnostic methods as the number of restoration tasks increases, suggesting that the use of tailored prompts or expert modules may have limited impact during training. This can be attributed to the used trivial degradation representations, which may fail to effectively guide methods to learn different restoration process.

Comparison with Single-Task Models
For a fair comparison, we evaluate the performance of our method in instruction mode (Ours†).
The results in Tab. [3](https://arxiv.org/html/2412.20157v3#S4.T3) show that our method achieved competitive or better performance compared to single-task models across various image restoration tasks.
The main reason lies in the fact that when single-task models are required to handle a broader range of degradation patterns (*e.g*., all levels of snow), their performance on specific degradation (*e.g*., large snow) tends to be not guaranteed. Our method can learn image degradations in finer-grained levels, allowing each expert to focus on specific degradation (*e.g*., different levels of snow). As a result, our approach can achieve competitive or better performance compared to single-task models.

### 4.3 Ablation Study

Effect of DR Extractors
In Tab. [8](https://arxiv.org/html/2412.20157v3#S4.T8), we compare the performance of experts trained on data clustered by different DR extractors on the deraining task, ‘Manual’ refers to data split by the original dataset source. Since our degradation extractor is aware of finer-grained DRs, the clustered training data can be more consistent in DR, thus our method can achieve better performance.

Effect of DR Fineness
In Tab. [8](https://arxiv.org/html/2412.20157v3#S4.T8), we investigate the impact of #DRs by varying the granularity of clustering. As grouped into finer-grained DR groups, the performance on in-dist. consistently improves. However, due to the increased difficulty of routing, the performance gain plateaus when #DRs exceeds 8.
For generalization performance, clustering DR groups in proper coarse-grained level performs better than finer-grained levels, suggesting that it is essential to combine multiple granularities to keep high performance on both in-dist. and out-dist.

Effect of Multiple Granularities
As shown in Tab. [8](https://arxiv.org/html/2412.20157v3#S4.T8), incorporating both fine-grained levels and coarse-grained levels can maintain much better restoration performance and generalization ability than employing single granularity. Taking performance, training cost, and learning difficulty into account, we finally adopt $\{$1, 4, 8$\}$ in our experiments.

Others
More discussions are in Suppl. Analysis of routing method is in Sec. D, including expert usage statistics (Tab. C), loss functions (Tab. D) and routers (Tab. E). Overhead comparison is in Tab. F and Tab. A. More visual results are in Sec. H.

## 5 Conclusion

In this paper, we introduced our UniRestorer, a novel universal image restoration framework designed to leverage degradation priors for improved restoration performance while alleviating the inevitable error in degradation estimation.
To this end, we learn all-in-one image restoration in multiple granularities and propose to estimate image degradation at proper granularity.
Specifically, we perform hierarchical clustering on degradation space and develop a multi-granularity degradation set as well as a MoE restoration model.
Besides vanilla degradation estimation, granularity estimation is further introduced to indicate the degree of degradation estimation error.
By jointly utilizing degradation and granularity estimation, we train routers to adaptively allocate unknown corrupted inputs to the most appropriate expert.
Experimental results demonstrate our superior performance and promising potential in closing the performance gap to specific single-task models.

The content of the supplementary material involves:

- •
Datasets, degradation pipeline, and degradation parameters are in Sec. [A](https://arxiv.org/html/2412.20157v3#S1a).
- •
Implementation and training details are in Sec. [B](https://arxiv.org/html/2412.20157v3#S2a).
- •
Discussion of the effect of expert design is in Sec. [C](https://arxiv.org/html/2412.20157v3#S3a).
- •
Discussion of the effect of the proposed routing method is in Sec. [D](https://arxiv.org/html/2412.20157v3#S4a).
- •
Overhead comparison is in Sec. [E](https://arxiv.org/html/2412.20157v3#S5a).
- •
Limitation and future works are in Sec. [F](https://arxiv.org/html/2412.20157v3#S6).
- •
Broader impacts are in Sec. [G](https://arxiv.org/html/2412.20157v3#S7).
- •
More visual comparisons are in Sec. [H](https://arxiv.org/html/2412.20157v3#S8).

## A Data

### A.1 Dataset

We detail the usage of datasets in training our models on single-degradation and mixed-degradation:
(1) Single-degradation:
For training, we adopt Rain100L [96] dataset for deraining, RESIDE [39] dataset for dehazing, DFBW (DIV2K [2], Flickr2K [50], BSD [61], and WED [60]) datasets for Gaussian color image denoising and compression artifacts removal, GoPro [64] dataset for motion deblurring, LOLv1 [86] dataset for low-light enhancement, Snow100K [57] dataset for desnowing.
Following [67, 102], we evaluate our method on Rain100L [96] for deraining, SOTS [39] for dehazing, BSD68 [61] for Gaussian color image denoising, GoPro [64] for motion deblurring, LOLv1 [86] for low-light enhancement, Snow100K-L [57] for desnowing, and LIVE1 [76] for compression artifacts removal.
(2) Mixed-degradation:
In mixed-degradation, we synthesize mixed degradation datasets from clean images in DF2K (DIV2K [2] and Flickr2K [50]) datasets for training and evaluation, respectively. The degradation pipeline is illustrated in Fig. [A](https://arxiv.org/html/2412.20157v3#S1.F1a).
(3) Single-task comparison:
Due to the scale limitation of Rain100L, we collect Rain200H [96], Rain200L [96], DID [101], and DDN [23] as training data to enlarge the degradation space and the number of training data for deraining task. Based on the clustered results, we retrain experts for degradation set related to rain degradation and replace original deraining experts used in single-degradation with newly trained ones. For other tasks, we use the same datasets as those used in the single-degradation.
(4) Ablation study and experiments in supplementary material:
Since the effectiveness of our method can be revealed in both performance on in-distribution and out-of-distribution samples, we conduct ablation studies on mixed-degradation, in which we can adjust degradation parameters to control data distribution. To simplify the evaluation process, we adopt Urban100 [65] as high-quality clean data and the same degradation pipeline as mixed-degradation.

### A.2 Degradation pipeline and Parameters

We follow the degradation pipeline of [108] to construct synthetic low-quality datasets for the training of the degradation extractor and all-in-one image restoration (mixed-degradation setup). As shown in Fig. [A](https://arxiv.org/html/2412.20157v3#S1.F1a), given high-quality clean inputs $y$, we orderly add synthetic lowlight, blur, noise, rain, snow, and haze to synthesize corresponding low-quality outputs $x$. We adopt gates between different degradations to gap one of them probably, the probability of all gates is generally set to $0.5$.
The way of each degradation is introduced as follows:

Blur
Following [104], we synthesize blurring degradation as,

$$ $x=y\otimes k,$ (A) $$

where $k$ is the blur kernel. In our implementation, the type of blur kernel includes isotropic and anisotropic, each of which is employed with the probability of $0.5$. The kernel size is sampled in the range from $7$ to $23$. The standard deviation of the Gaussian kernel is set from $1$ to $3$ for in-dist. and $3$ to $4$ for out-dist.

Noise
Gaussian noise is synthesized as,

$$ $x=y+n,$ (B) $$

where $n$ is Gaussian noise with variance $\sigma^{2}$. $\sigma$ is set from $15$ to $35$ for in-dist. and $35$ to $50$ for out-dist.

Rain
Raining degradation is synthesized as,

$$ $x=y+r,$ (C) $$

where $r$ is rain streaks. Following [91], $r$ is synthesized with random noise and Gaussian blur, where noise is related to the strength of the rain, and blur kernel is related to the rain direction, length, and width.
We adopt the number of rain streaks ranges from $50$ to $100$ in in-dist. and $100$ to $150$ for out-dist. Rain direction is generally adopted from $-45^{\circ}$ to $45^{\circ}$, rain length is set from $20$ to $40$, and rain width is uniformly sampled from $\{3,5,7,9,11\}$.

Figure: Figure A: Degradation Pipeline.
Refer to caption: x5.png

Haze
Following [39], hazing degradation is synthesized as,

$$ $\displaystyle x=T(y)y+a(1-T(y)),$ (D) $\displaystyle T(y)=e^{-\beta D(y)},$ $$

where $a$ is the global atmospheric light, $T(y)$ is the transition matrix, $D(y)$ is the depth map, and $\beta$ is the scattering coefficient of the atmosphere. In haze degradation synthesizing, the value of $\beta$ largely decides the thickness of haze. We adopt one publicly released project $\operatorname{MegaDepth}$(^1^11[https://github.com/zhengqili/MegaDepth](https://github.com/zhengqili/MegaDepth)) to estimate the depth image. In our experiments, the global atmospheric light is set from $0.8$ to $1$, the $\beta$ is set from $0.5$ to $1.5$ for in-dist. and $1.5$ to $2$ for out-dist.

Snow
Snowing degradation is synthesized as,

$$ $x=y\times(1-m)+m,$ (E) $$

where $m$ is the snow mask randomly sampled from Snow100K [57] dataset.

Lowlight
Following [24], lowlight degradation is synthesized as,

$$ $x=\frac{y}{I(y)}I(y)^{\gamma}$ (F) $$

where $I(y)$ is the illusion map obtained from RetinexFormer [10] and $\gamma$ is the darkening coefficient. Since we explicitly include noise degradation in our degradation pipeline, we do not additionally consider noise in this step. We set $\gamma$ from $1$ to $2$ for in-dist. and $2$ to $2.5$ for out-dist.

Besides, JPEG compression is synthesized by employing the publicly released project $\operatorname{DiffJPEG}$(^2^22[https://github.com/mlomnitz/DiffJPEG](https://github.com/mlomnitz/DiffJPEG)).

## B Implementation and Training Details

Degradation Extractor
Our degradation extractor is based on previous work, DA-CLIP [59].
DA-CLIP uses a lightweight image controller, a copy of the CLIP image encoder augmented with zero-initialized residual connections, to guide the image encoder output toward producing both content-related features and degradation-related embeddings. Using contrastive learning, the content embedding is aligned with clean text descriptions, and degradation embedding indicates the type of degradation, aligned with degradation-related text prompts.
For more details, please refer to the original paper.
To demonstrate the effectiveness of our degradation extractor, we visualize the T-SNE results in Fig. [B](https://arxiv.org/html/2412.20157v3#S2.F2) (a).
For example, in the single dehazing task, both Restormer and PromptIR fail to capture fine-grained DRs, as shown in Fig. [B](https://arxiv.org/html/2412.20157v3#S2.F2) (a) and Fig. [B](https://arxiv.org/html/2412.20157v3#S2.F2) (b). In contrast, our method learns DRs at finer levels of granularity and allows restoration on more specific degradations, thus achieving competitive or even better performance to single-task models.
While PromptIR can separate different types of degradations, it still exhibits misidentification across various cases.
Such cases may render the dedicated prompts less effective, which are originally as guidance for specific tasks.
As a result, PromptIR may offer limited performance improvements over task-agnostic methods in some of the restoration tasks.

Figure: Figure B: T-SNE visualization of deep features (Restormer [100], PromptIR [67]) and degradation representations (Ours). (a) and (b): Restormer and PromptIR are not aware of fine-grained degradation representations. (c) While PromptIR demonstrates a certain capability to distinguish between different types of degradations, it still exhibits misclassification across various cases. (d) Our method distinguish degradation representations well in both coarse-grained and fine-grained levels.
Refer to caption: x6.png

Multi-Granularity MoE Restoration Model
After training the degradation extractor, we construct the DR set for all training samples.
Then, we hierarchically cluster the obtained DR set into multiple groups in different granularities, constructing multi-granularity degradation sets.
Considering the task-agnostic capability, we generally adopt Restormer [100] as the expert model across most tasks.
However, in single-degradation, we found Restormer cannot perform well in low-light enhancement.
We thus observe clusters predominantly composed of low-light degradations and select RetinexFormer [10] as the expert model for those clusters.
As a result, our MoE restoration model has three granularities. In the coarsest-grained level, we use single expert model to solve all observed degradations; in the second level, we use $7$ experts to solve degradation sets with coarser DRs; and in the finest-grained level, we have $19$ experts, and each expert is responsible for degradation sets with more fine-grained DRs.
In mixed-degradation, we have $1$, $4$, and $8$ experts for the first, the second, and the third granularity level, respectively.

Structure of $\mathcal{H}$ and $\mathcal{G}$
Without dedicated design, we adopt 5-layer MLP networks for both $\mathcal{H}_{d}$ and $\mathcal{H}_{g}$, in which the number of input and output channels is $512$ (the same as the dimension of DR $e$). The structures of $\mathcal{G}_{d}$ and $\mathcal{G}_{g}$ are one-layer MLP networks with input channels of $512$ and the number of output channels equal to the number of experts.

Instruction Mode, Ours†
The instruction mode of our method allows inference based on given task instructions, *i.e*., the task name. In our implementation, the task instructions can convert to specific binary masks, which represent pruning or not pruning on the corresponding expert model in our MoE restoration model. According to the masks, we first prune out the corresponding expert models, then we conduct degradation- and granularity-based routing to find the most appropriate expert model for corrupted inputs. In this way, the error in degradation estimation is alleviated, the routing accuracy is highly improved, and thus the restoration performance is further improved.

Ablation Study on Degradation Extractor
We adopt pre-trained vgg16, pre-trained ESRGAN on bicubic 4$\times$ super-resolution, and pre-trained DACLIP as DR extractors.
We construct multi-granularity degradation sets by leveraging DRs extracted from different DR extractors.
Then we re-train expert networks based on each obtained degradation set.
The experiments are conducted on deraining task.
Since our degradation extractor is aware of finer-grained DRs, the clustered training data can be more consistent in DR, thus our method can achieve better performance.

Training of Degradation Extractor
To train our DR extractor, we adopt the ways in Sec. [A](https://arxiv.org/html/2412.20157v3#S1a) to synthesize various types of low-quality data.
According to the parameter that decides the degradation degree, we separate synthetic data into multiple groups.
For deraining degradation, strength from 0 0 to $50$ is small rain, $50$ to $100$ is medium rain, and $100$ to $150$ is large rain;
In denoising, $\sigma$ of Gaussian noise from 0 0 to $15$ is small noise, $15$ to $35$ is medium noise, and $35$ to $50$ is large noise;
In dehazing, scattering coefficient $\beta$ larger than $0.1$ is thick haze, $\beta$ smaller than $0.1$ is normal haze;
In deblurring, kernel size of motion blur kernel larger than $15$ is large blur, kernel size smaller than $15$ is small blur;
In desnowing, we use snow masks from Snow100K dataset and adopt the official separation.
In compression artifacts removal, image quality from 0 0 to $15$ is large compression, $15$ to $35$ is medium compression, $35$ to $50$ is small compression;
In lowlight enhancement, the darkening coefficient from $1$ to $1.5$ is light corruption and $1.5$ to $3$ is heavy corruption.

Training of Multi-Granularity MoE Restoration Model
In the training of experts, we first conduct hierarchical clustering based on collected datasets, after that, all degraded training images are divided into multiple groups in different granularities. In each granularity, we record for each group the data samples that is allocated to it. Then, we use the recorded data to train expert restoration network for each group.
After training of experts, the weights of experts are frozen, and we train routers based on degradation- and granularity- estimation.
Most works in high-level vision and NLP tasks train experts and routers jointly. To well deal with the conflicts between restoring different degradations, MoE system in image restoration works or all-in-one image restoration works prefer separate training [117, 66, 12, 81, 14, 3], *i.e*., sequentially training experts and routers.
Similarly to the mentioned works, our experts are explicitly determined to solve degradations in each degradation set, experts should be learned in advance and are not affected by the routers to guarantee the stability of the whole model.
Thus, we learn experts and routers sequentially rather than simultaneously.

## C Effect of Expert Design

Motivation of Expert Design
MoE is an effective solution for training large models in high-level vision and NLP tasks. One common approach is to deploy sub-networks as experts based on one shared backbone weight. For example, in all-in-one image restoration, some works [117, 66] adopt this strategy to develop their MoE system for alleviating conflict among different restoration tasks.
Another way is to use a full-network as an expert directly. For examples, early work [87] in real-world super-resolution adopted this way, and recent all-in-one image restoration works [12, 14] also followed this paradigm.
In our work, using full-network and sub-network as expert have similar computational cost.
However, using full-network has several advantages. First, it is easier to build and expand. Second, it can assign its full parameters for specific degradation, which has a more powerful fitting ability. Third, we can benefit much from the previous advanced architecture design towards specific tasks, unleashing their full power in solving all-in-one image restoration topic.

Lightweight Experts
We have designed more lightweight experts (by scaling down Restormer [100] and NAFNet [15]) and provided complexity analysis in Tab. [A](https://arxiv.org/html/2412.20157v3#S3.T1a). It shows that it achieves obvious performance gains with a much smaller inference cost than PromptIR [67].

**Table A: All-in-One image restoration results (three tasks single-degradation). We replace original experts with lightweight NAFNet (Lite-NAFNet) and Restormer (Lite-Restormer) as experts. ‘Inf.’ means during inference. The reported Params. mainly includes restoration networks.**
|  | Derain | Dehaze | Denoise | #FLOPs | Latency | Params.(M) | Mem. | Time(h) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | Rain100L | SOTS | BSD68 | (G) | (s) | Inf./Train | (M) Inf. | Train |
| PromptIR [67] | 36.37 | 30.58 | 31.12 | 1266.2 | 0.355 | 39.6/39.6 | 3552 | 201 |
| Ours | 41.68 | 36.44 | 31.43 | 1155.8 | 0.484 | 26.1/339.3 | 4258 | 578 |
| Ours (Lite-NAFNet) | 38.98 | 31.69 | 31.28 | 154.7 | 0.156 | 17.1/222.4 | 1818 | 169 |
| Ours (Lite-Restormer) | 41.41 | 35.38 | 31.35 | 672.7 | 0.395 | 14.8/192.5 | 3526 | 442 |

**Table B: All-in-One image restoration result (mixed-degradation). ‘Inf.’ means during inference. ‘-Large’ indicates the version of large scale. The reported Params. mainly includes restoration networks. PromptIR-L is trained in FP16.**
|  | MiO (Avg.) | #FLOPs | Latency | Params.(M) | Mem. | Time (h) |
| --- | --- | --- | --- | --- | --- | --- |
| Methods | In/out-dist. | (G) | (s) | Inf./Train | (M) Inf. | Train |
| Restormer | 22.06 /17.23 | 1128.9 | 0.368 | 26.1/26.1 | 3244 | 48 |
| Restormer-Large | 23.98 /18.65 | 17360.1 | 1.375 | 441.8/441.8 | 13408 | 3792 |
| PromptIR-Large | 24.13 /18.76 | 18139.6 | 1.283 | 439.3/439.3 | 12962 | - |
| Ours | 24.46/19.45 | 1155.8 | 0.484 | 26.1/339.3 | 4258 | 578 |

Compare to Vanilla Scaling Up
According to the degradation- and granularity- estimation, the routers select the most appropriate expert for each input low-quality image, leading to a sparse inference strategy (see Tab. [A](https://arxiv.org/html/2412.20157v3#S3.T1a) and Tab. [F](https://arxiv.org/html/2412.20157v3#S5.T6)).
Though our method has comparable inference #parameters, FLOPs, and latency to methods with similar backbones, one may know the effect of training parameters.
To mitigate the gap of training #parameters, we scale up the training #parameters of Restormer and PromptIR to match those of ours, and we conduct experiments on mixed-degradation.
As shown in Tab. [B](https://arxiv.org/html/2412.20157v3#S3.T2a), even after scaling up training parameters, Restormer-Large and PromptIR-Large cannot have marginal performance gains, it reveals that simply increasing the model size is not the essence of improving the performance, which is different from the scaling law in high-level vision and NLP tasks.
From this perspective, our method provides an effective paradigm for scaling up low-level vision image restoration models, enabling both improved performance and efficient computation.

**Table C: Expert usage statistics.**
|  | in-dist. | out-dist. |
| --- | --- | --- |
| Level | 0 / 1 / 2 | 0 / 1 / 2 |
| Expected | 7 / 13 / 80 | 29 / 46 / 25 |
| Practical | 5 / 17 / 78 | 34 / 50 / 16 |

## D Effect of the Proposed Routing Method

Routing Usage Statistics
If degradation estimation (DE) on the finest granularity level is accurate, using corresponding fine-grained experts is optimal. However, fine-grained DE is error-prone. The proposed granularity estimation measures the uncertainty of DE. When the uncertainty is large, it means that the finest-grained expert is probably not suitable for this degradation. Instead, a coarser-grained expert is more robust to estimation error, thus being selected.
In Tab. [E](https://arxiv.org/html/2412.20157v3#S3.T5), we report the practical $\#$loads and expected $\#$loads (by maximizing PSNR) of different granularity levels during inference on Urban100 dataset with mixed-degradation, and our routing accuracies are respectively 92$\%$ and 82$\%$ on in-dist. and out-dist.

Effect of $\mathcal{G}_{d}$ and $\mathcal{G}_{g}$
As shown in Tab. [E](https://arxiv.org/html/2412.20157v3#S3.T5), using both $\mathcal{G}_{d}$ and $\mathcal{G}_{g}$ leads to more accurate routing.
Without using any routers, ‘Clustering’ cannot distinguish DRs in different granularities and cannot guarantee the right routing results, leading to lower performance.
Using only $\mathcal{G}_{d}$ improves the performance on in-dist., while without the awareness of granularity, it does not generalize well on out-dist.

Effect of Loss Functions
As shown in Tab. [E](https://arxiv.org/html/2412.20157v3#S3.T5), both $\mathcal{L}_{dg}$ and $\mathcal{L}_{load}$ can benefit more accurate routing.
In particular, $\mathcal{L}_{dg}$ and $\mathcal{L}_{load}$ can avoid the routing to collapse to the finest-grained level, improving performance on out-dist.

## E Overhead Comparisons

We calculate FLOPs and latency with $512\times 512\times 3$ input, on a single NVIDIA A6000 GPU. As shown in Tab. [F](https://arxiv.org/html/2412.20157v3#S5.T6), thanks to mixture-of-experts, the overhead of our method is slightly higher than the used image restoration backbone, Restormer [100]. In addition, the overhead of our method can be largely reduced by replacing the image restoration backbone with more lightweight ones, *e.g*., lightweight NAFNet (Lite-NAFNet) and lightweight Restormer (Lite-Restormer).

**Table F: Overhead comparisons.**
| Methods | #FLOPs (G) | Latency (s) |
| --- | --- | --- |
| Restormer [100] | 1128.9 | 0.368 |
| NAFNet [15] | 505.5 | 0.186 |
| PromptIR [67] | 1266.2 | 0.355 |
| OneRestore [24] | 94.5 | 0.238 |
| Ours (Lite-NAFNet) | 154.7 | 0.156 |
| Ours (Lite-Restormer) | 672.7 | 0.395 |
| Ours | 1155.8 | 0.484 |

## F Limitations and Future Works

On the one hand, the training data mainly comes from publicly released datasets, as required for fair comparison with baseline methods. However, the scale of training data is essential, particularly in developing large-scale models. In image restoration works, the scale of training data is often limited, *e.g*. Rain100L [96] contains only 200 pairs of training data. Previous works [85] has exploited the benefits brought by scaling up the training data, we thus plan to collect more high-quality training data to bring further improvement on our method.
On the other hand, we plan to expand and align our degradation space with real-world scenarios, aiming to well restore real-world image degradations.

## G Broader Impacts

In this paper, we propose UniRestorer, a novel all-in-one image restoration framework that can estimate and leverage robust degradation representation to adaptively handle diverse image corruptions. Our research aims to contribute to the advancement of the relevant community without any negative social impact.

## H More Results and Visual Comparisons

More Visual Comparisons.
We show more visual comparison results in this section. Compared to task-agnostic methods, all-in-one methods, and specific single-task models, our method can restore more clear results with detailed textures.
The single-degradation results of all-in-one image restoration comparisons can be seen in Fig. [C](https://arxiv.org/html/2412.20157v3#S8.F3) to Fig. [I](https://arxiv.org/html/2412.20157v3#S8.F9). The comparison results with specific single-task models can be seen in Fig. [J](https://arxiv.org/html/2412.20157v3#S8.F10) to Fig. [O](https://arxiv.org/html/2412.20157v3#S8.F15).
In the evaluation of mixed degradation, since the resolution of images from DIV2K-valid is too high, limiting inference on the whole image for all methods. We first crop high-resolution images into overlapped $512\times 512$ crops, after inference, we inverse the cropping process and recompose crops back to one whole image, though averaging the overlapped area, the border artifacts cannot be avoided. Comparison result of in-dist. and out-dist. can be seen in Fig. [P](https://arxiv.org/html/2412.20157v3#S8.F16) and Fig. [Q](https://arxiv.org/html/2412.20157v3#S8.F17), respectively.

Figure: Figure C: Visual comparison of results on All-in-One image restoration (deraining).
Refer to caption: x7.png

Figure: Figure D: Visual comparison of results on All-in-One image restoration (Gaussian-denoising).
Refer to caption: x8.png

Figure: Figure E: Visual comparison of results on All-in-One image restoration (dehazing).
Refer to caption: x9.png

Figure: Figure F: Visual comparison of results on All-in-One image restoration (motion-deblurring).
Refer to caption: x10.png

Figure: Figure G: Visual comparison of results on All-in-One image restoration (desnowing).
Refer to caption: x11.png

Figure: Figure H: Visual comparison of results on All-in-One image restoration (compression artifacts removal).
Refer to caption: x12.png

Figure: Figure I: Visual comparison of results on All-in-One image restoration (lowlight enhancement).
Refer to caption: x13.png

Figure: Figure J: Visual comparison of results with specific models (deraining).
Refer to caption: x14.png

Figure: Figure K: Visual comparison of results with specific models (Gaussian-denoising).
Refer to caption: x15.png

Figure: Figure L: Visual comparison of results with specific models (dehaze).
Refer to caption: x16.png

Figure: Figure M: Visual comparison of results with specific models (motion-deblurring).
Refer to caption: x17.png

Figure: Figure N: Visual comparison of results with specific models (desnowing).
Refer to caption: x18.png

Figure: Figure O: Visual comparison of results with specific models (lowlight enhancement).
Refer to caption: x19.png

Figure: Figure P: Visual comparison of results on All-in-One image restoration (mixed-degradation, in-dist). Zoom in for more details.
Refer to caption: x20.png

Figure: Figure Q: Visual comparison of results on All-in-One image restoration (mixed-degradation, out-dist). Zoom in for more details.
Refer to caption: x21.png

## References

- Abdelhamed et al. [2018]
Abdelrahman Abdelhamed, Stephen Lin, and Michael S. Brown.
A high-quality denoising dataset for smartphone cameras.
In *CVPR*, 2018.
- Agustsson and Timofte [2017]
Eirikur Agustsson and Radu Timofte.
Ntire 2017 challenge on single image super-resolution: Dataset and study.
In *CVPRW*, July 2017.
- Ai et al. [2024a]
Yuang Ai, Huaibo Huang, and Ran He.
Lora-ir: Taming low-rank experts for efficient all-in-one image restoration.
*arXiv preprint arXiv:2410.15385*, 2024a.
- Ai et al. [2024b]
Yuang Ai, Huaibo Huang, Xiaoqiang Zhou, Jiexiang Wang, and Ran He.
Multimodal prompt perceiver: Empower adaptiveness, generalizability and fidelity for all-in-one image restoration.
In *CVPR*, 2024b.
- Alpher [2002]
FirstName Alpher.
Frobnication.
*IEEE TPAMI*, 12(1):234–778, 2002.
- Alpher and Fotheringham-Smythe [2003]
FirstName Alpher and FirstName Fotheringham-Smythe.
Frobnication revisited.
*Journal of Foo*, 13(1):234–778, 2003.
- Alpher and Gamow [2005]
FirstName Alpher and FirstName Gamow.
Can a computer frobnicate?
In *CVPR*, pages 234–778, 2005.
- Alpher et al. [2004]
FirstName Alpher, FirstName Fotheringham-Smythe, and FirstName Gamow.
Can a machine frobnicate?
*Journal of Foo*, 14(1):234–778, 2004.
- Brian et al. [2021]
Lester Brian, Al-Rfou Rami, and Constant Noah.
The power of scale for parameter-efficient prompt tuning.
In *EMNLP*, November 2021.
- Cai et al. [2023]
Yuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang, Radu Timofte, and Yulun Zhang.
Retinexformer: One-stage retinex-based transformer for low-light image enhancement.
In *ICCV*, 2023.
- Cao et al. [2024a]
Jin Cao, Yi Cao, Li Pang, Deyu Meng, and Xiaoyong Cao.
Hair: Hypernetworks-based all-in-one image restoration.
*arXiv preprint arXiv:2408.08091*, 2024a.
- Cao et al. [2024b]
Shuo Cao, Yihao Liu, Wenlong Zhang, Yu Qiao, and Chao Dong.
Grids: Grouped multiple-degradation restoration with image degradation similarity.
In *ECCV*, 2024b.
- Chang et al. [2020]
Jie Chang, Zhonghao Lan, Changmao Cheng, and Yichen Wei.
Data uncertainty learning in face recognition.
In *CVPR*, 2020.
- Chen et al. [2024]
Haoyu Chen, Wenbo Li, Jinjin Gu, Jingjing Ren, Sixiang Chen, Tian Ye, Renjing Pei, Kaiwen Zhou, Fenglong Song, and Lei Zhu.
Restoreagent: Autonomous image restoration agent via multimodal large language models.
In *NeurIPS*, 2024.
- Chen et al. [2022]
Liangyu Chen, Xiaojie Chu, Xiangyu Zhang, and Jian Sun.
Simple baselines for image restoration.
In *ECCV*, 2022.
- Chen et al. [2023a]
Zitian Chen, Mingyu Ding, Yikang Shen, Wei Zhan, Masayoshi Tomizuka, Erik Learned-Miller, and Chuang Gan.
An efficient general-purpose modular vision model via multi-task heterogeneous training.
*arXiv preprint arXiv:2306.17165*, 2023a.
- Chen et al. [2023b]
Zitian Chen, Yikang Shen, Mingyu Ding, Zhenfang Chen, Hengshuang Zhao, Erik Learned-Miller, and Chuang Gan.
Mod-squad: Designing mixtures of experts as modular multi-task learners.
*CVPR*, 2023b.
- Clark et al. [2022]
Aidan Clark, Diego de las Casas, Aurelia Guy, Arthur Mensch, Michela Paganini, Jordan Hoffmann, Bogdan Damoc, Blake Hechtman, Trevor Cai, Sebastian Borgeaud, George van den Driessche, Eliza Rutherford, Tom Hennigan, Matthew Johnson, Katie Millican, Albin Cassirer, Chris Jones, Elena Buchatskaya, David Budden, Laurent Sifre, Simon Osindero, Oriol Vinyals, Jack Rae, Erich Elsen, Koray Kavukcuoglu, and Karen Simonyan.
Unified scaling laws for routed language models.
In *NeurIPS*, 2022.
- Conde et al. [2024]
Marcos V. Conde, Gregor Geigle, and Radu Timofte.
Instructir: High-quality image restoration following human instructions.
In *ECCV*, 2024.
- Dong et al. [2014]
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang.
Learning a deep convolutional network for image super-resolution.
In *ECCV*, 2014.
- Fan et al. [2019]
Qingnan Fan, Dongdong Chen, Lu Yuan, Gang Hua, Nenghai Yu, and Baoquan Chen.
A general decoupled learning framework for parameterized image operators.
*IEEE TPAMI*, 2019.
- Fedus et al. [2022]
William Fedus, Barret Zoph, and Noam Shazeer.
Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.
*Journal of Machine Learning Research*, 2022.
- Fu et al. [2017]
Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley.
Removing rain from single images via a deep detail network.
In *CVPR*, 2017.
- Guo et al. [2024]
Yu Guo, Yuan Gao, Yuxu Lu, Ryan Wen Liu, and Shengfeng He.
Onerestore: A universal restoration framework for composite degradation.
In *ECCV*, 2024.
- Guo et al. [2023]
Yun Guo, Xueyao Xiao, Yi Chang, Shumin Deng, and Luxin Yan.
From sky to the ground: A large-scale benchmark and simple baseline towards real rain removal.
In *ICCV*, 2023.
- He et al. [2019]
Jingwen He, Chao Dong, and Yu Qiao.
Modulating image restoration with continual levels via adaptive feature modification layers.
In *The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, June 2019.
- He et al. [2016]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.
Deep residual learning for image recognition.
In *CVPR*, 2016.
- Houlsby et al. [2019]
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly.
Parameter-efficient transfer learning for nlp.
In *ICML*, 2019.
- Hu et al. [2022]
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.
LoRA: Low-rank adaptation of large language models.
In *ICLR*, 2022.
- Jacobs et al. [1991]
Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton.
Adaptive mixtures of local experts.
*Neural Computation*, 1991.
- JiaKui et al. [2025]
Hu JiaKui, Zhengjian Yao, Jin Lujia, and Lu Yanye.
Universal image restoration pre-training via degradation classification.
In *ICLR*, 2025.
- Jiang et al. [2024]
Junjun Jiang, Zengyuan Zuo, Gang Wu, Kui Jiang, and Xianming Liu.
A survey on all-in-one image restoration: Taxonomy, evaluation and future trends.
*arXiv preprint arXiv:2410.15067*, 2024.
- Jordan and Jacobs [1993]
M.I. Jordan and R.A. Jacobs.
Hierarchical mixtures of experts and the em algorithm.
In *Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan)*, 1993.
- Kupyn et al. [2019]
Orest Kupyn, Tetiana Martyniuk, Junru Wu, and Zhangyang Wang.
Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better.
In *ICCV*, Oct 2019.
- LastName [2014a]
FirstName LastName.
The frobnicatable foo filter, 2014a.
Face and Gesture submission ID 324. Supplied as supplemental material fg324.pdf.
- LastName [2014b]
FirstName LastName.
Frobnication tutorial, 2014b.
Supplied as supplemental material tr.pdf.
- Lepikhin et al. [2021]
Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen.
{GS}hard: Scaling giant models with conditional computation and automatic sharding.
In *ICLR*, 2021.
- Lewis et al. [2021]
Mike Lewis, Shruti Bhosale, Tim Dettmers, Naman Goyal, and Luke Zettlemoyer.
Base layers: Simplifying training of large, sparse models.
In *NeurIPS*, 2021.
- Li et al. [2019]
Boyi Li, Wenqi Ren, Dengpan Fu, Dacheng Tao, Dan Feng, Wenjun Zeng, and Zhangyang Wang.
Benchmarking single-image dehazing and beyond.
*IEEE TIP*, 28(1):492–505, 2019.
- Li et al. [2022a]
Boyun Li, Xiao Liu, Peng Hu, Zhongqin Wu, Jiancheng Lv, and Xi Peng.
All-In-One Image Restoration for Unknown Corruption.
In *CVPR*, 2022a.
- Li et al. [2020]
Chongyi Li, Chunle Guo, Wenqi Ren, Runmin Cong, Junhui Hou, Sam Kwong, and Dacheng Tao.
An underwater image enhancement benchmark dataset and beyond.
*IEEE TIP*, 29:4376–4389, 2020.
- Li et al. [2024a]
Hao Li, Xiang Chen, Jiangxin Dong, Jinhui Tang, and Jinshan Pan.
Foundir: Unleashing million-scale training data to advance foundation models for image restoration.
*arXiv preprint arXiv:2412.01427*, 2024a.
- Li et al. [2022b]
Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi.
BLIP: bootstrapping language-image pre-training for unified vision-language understanding and generation.
In *ICML*, 2022b.
- Li et al. [2023a]
Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi.
BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models.
In *ICML*, 2023a.
- Li et al. [2023b]
Junyi Li, Zhilu Zhang, Xiaoyu Liu, Chaoyu Feng, Xiaotao Wang, Lei Lei, and Wangmeng Zuo.
Spatially adaptive self-supervised learning for real-world image denoising.
In *CVPR*, 2023b.
- Li et al. [2024b]
Junyi Li, Zhilu Zhang, and Wangmeng Zuo.
Tbsn: Transformer-based blind-spot network for self-supervised image denoising.
*arXiv preprint arXiv:2404.07846*, 2024b.
- Li et al. [2022c]
Youwei Li, Haibin Huang, Lanpeng Jia, Haoqiang Fan, and Shuaicheng Liu.
D2c-sr: A divergence to convergence approach for real-world image super-resolution.
In *ECCV*, 2022c.
- Liang et al. [2022]
Jie Liang, Hui Zeng, and Lei Zhang.
Efficient and degradation-adaptive network for real-world image super-resolution.
In *ECCV*, 2022.
- Liang et al. [2021]
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte.
Swinir: Image restoration using swin transformer.
In *ICCVW*, 2021.
- Lim et al. [2017]
Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee.
Enhanced deep residual networks for single image super-resolution.
In *CVPRW*, July 2017.
- Lin et al. [2024]
Jingbo Lin, Zhilu Zhang, Yuxiang Wei, Dongwei Ren, Dongsheng Jiang, Qi Tian, and Wangmeng Zuo.
Improving image restoration through removing degradations in textual representations.
In *CVPR*, 2024.
- Liu et al. [2022a]
Lin Liu, Lingxi Xie, Xiaopeng Zhang, Shanxin Yuan, Xiangyu Chen, Wengang Zhou, Houqiang Li, and Qi Tian.
Tape: Task-agnostic prior embedding for image restoration.
In *ECCV*, 2022a.
- Liu et al. [2024]
Xiaohui Liu, Zhilu Zhang, Xiaohe Wu, Chaoyu Feng, Xiaotao Wang, Lei Lei, and Wangmeng Zuo.
Learning real-world image de-weathering with imperfect supervision.
In *AAAI*, 2024.
- Liu et al. [2022b]
Yihao Liu, Anran Liu, Jinjin Gu, Zhipeng Zhang, Wenhao Wu, Yu Qiao, and Chao Dong.
Discovering distinctive "semantics" in super-resolution networks.
*arXiv preprint arXiv:2108.00406*, 2022b.
- Liu et al. [2023]
Yihao Liu, Jingwen He, Jinjin Gu, Xiangtao Kong, Yu Qiao, and Chao Dong.
Degae: A new pretraining paradigm for low-level vision.
In *CVPR*, June 2023.
- Liu et al. [2018a]
Yun-Fu Liu, Da-Wei Jaw, Shih-Chia Huang, and Jenq-Neng Hwang.
Desnownet: Context-aware deep network for snow removal.
*IEEE TIP*, 27(6):3064–3073, 2018a.
- Liu et al. [2018b]
Yun-Fu Liu, Da-Wei Jaw, Shih-Chia Huang, and Jenq-Neng Hwang.
Desnownet: Context-aware deep network for snow removal.
*IEEE TIP*, 27(6):3064–3073, 2018b.
- Liu et al. [2021]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.
Swin transformer: Hierarchical vision transformer using shifted windows.
In *ICCV*, 2021.
- Luo et al. [2024]
Ziwei Luo, Fredrik K Gustafsson, Zheng Zhao, Jens Sjölund, and Thomas B Schön.
Controlling vision-language models for universal image restoration.
In *ICLR*, 2024.
- Ma et al. [2017]
Kede Ma, Zhengfang Duanmu, Qingbo Wu, Zhou Wang, Hongwei Yong, Hongliang Li, and Lei Zhang.
Waterloo exploration database: New challenges for image quality assessment models.
*IEEE Transactions on Image Processing*, 2017.
- Martin et al. [2001]
David R. Martin, Charless C. Fowlkes, Doron Tal, and Jitendra Malik.
A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics.
In *ICCV*, 2001.
- Mitchell [1997]
Tom M Mitchell.
*Machine learning*, volume 1.
1997.
- Mustafa et al. [2022]
Basil Mustafa, Carlos Riquelme Ruiz, Joan Puigcerver, Rodolphe Jenatton, and Neil Houlsby.
Multimodal contrastive learning with LIMoe: the language-image mixture of experts.
In *NeurIPS*, 2022.
- Nah et al. [2017a]
Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee.
Deep multi-scale convolutional neural network for dynamic scene deblurring.
In *CVPR*, July 2017a.
- Nah et al. [2017b]
Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee.
Deep multi-scale convolutional neural network for dynamic scene deblurring.
In *CVPR*, July 2017b.
- Park et al. [2023]
Dongwon Park, Byung Hyun Lee, and Se Young Chun.
All-in-one image restoration for unknown degradations using adaptive discriminative filters for specific degradations.
In *CVPR*, 2023.
- Potlapalli et al. [2023]
Vaishnav Potlapalli, Syed Waqas Zamir, Salman Khan, and Fahad Khan.
Promptir: Prompting for all-in-one image restoration.
In *NeurIPS*, 2023.
- Puigcerver et al. [2024]
Joan Puigcerver, Carlos Riquelme Ruiz, Basil Mustafa, and Neil Houlsby.
From sparse to soft mixtures of experts.
In *ICLR*, 2024.
- Qian et al. [2018]
Rui Qian, Robby T. Tan, Wenhan Yang, Jiajun Su, and Jiaying Liu.
Attentive generative adversarial network for raindrop removal from a single image.
In *CVPR*, June 2018.
- Radford et al. [2021]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.
Learning transferable visual models from natural language supervision.
In *ICML*, 2021.
- Rajbhandari et al. [2022]
Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, and Yuxiong He.
Deepspeed-moe: Advancing mixture-of-experts inference and training to power next-generation ai scale.
In *ICML*, 2022.
- Ren et al. [2019]
Dongwei Ren, Wangmeng Zuo, Qinghua Hu, Pengfei Zhu, and Deyu Meng.
Progressive image deraining networks: A better and simpler baseline.
In *CVPR*, 2019.
- Rim et al. [2020]
Jaesung Rim, Haeyun Lee, Jucheol Won, and Sunghyun Cho.
Real-world blur dataset for learning and benchmarking deblurring algorithms.
In *ECCV*, 2020.
- Ruiz et al. [2021]
Carlos Riquelme Ruiz, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, and Neil Houlsby.
Scaling vision with sparse mixture of experts.
In *Advances in Neural Information Processing Systems*, 2021.
- Shazeer et al. [2017]
Noam Shazeer, *Azalia Mirhoseini, *Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean.
Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.
In *ICLR*, 2017.
- [76]
H.R. Sheikh, Z.Wang, L. Cormack, and A.C. Bovik.
Live image quality assessment database release 2.
[http://live.ece.utexas.edu/research/quality](http://live.ece.utexas.edu/research/quality).
- Shen et al. [2023]
Sheng Shen, Zhewei Yao, Chunyuan Li, Trevor Darrell, Kurt Keutzer, and Yuxiong He.
Scaling vision-language models with sparse mixture of experts.
In *EMNLP*, 2023.
- Simonyan and Zisserman [2015]
Karen Simonyan and Andrew Zisserman.
Very deep convolutional networks for large-scale image recognition.
In *ICLR*, 2015.
- Song et al. [2023]
Yuda Song, Zhuqing He, Hui Qian, and Xin Du.
Vision transformers for single image dehazing.
*IEEE TIP*, 32:1927–1941, 2023.
- Sun et al. [2018]
Yujing Sun, Yizhou Yu, and Wenping Wang.
Moiré photo restoration using multiresolution convolutional neural networks.
*IEEE TIP*, 27(8):4160–4172, 2018.
- Tian et al. [2024]
Yuchuan Tian, Jianhong Han, Hanting Chen, Yuanyuan Xi, Guoyang Zhang, Jie Hu, Chao Xu, and Yunhe Wang.
Instruct-ipt: All-in-one image processing transformer via weight modulation.
*arXiv preprint arXiv:2407.00676*, 2024.
- Valanarasu et al. [2022]
Jeya Maria Jose Valanarasu, Rajeev Yasarla, and Vishal M. Patel.
Transweather: Transformer-based restoration of images degraded by adverse weather conditions.
In *CVPR*, 2022.
- Vaswani et al. [2017]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin.
Attention is all you need.
In *NeurIPS*, 2017.
- Wang et al. [2018]
Feng Wang, Jian Cheng, Weiyang Liu, and Haijun Liu.
Additive margin softmax for face verification.
*IEEE Signal Processing Letters*, 2018.
- Wang et al. [2021]
Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan.
Real-esrgan: Training real-world blind super-resolution with pure synthetic data.
In *ICCVW*, 2021.
- Wei et al. [2018]
Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu.
Deep retinex decomposition for low-light enhancement.
In *BMVC*, 2018.
- Wenlong et al. [2023]
Zhang Wenlong, Li Xiaohui, Guangyuan SHI, Chen Xiangyu, Qiao Yu, Zhang Xiaoyun, Wu Xiao-Ming, and Dong Chao.
Real-world image super-resolution as multi-task learning.
In *NeurIPS*, 2023.
- Wu et al. [2025]
Gang Wu, Junjun Jiang, Yijun Wang, Kui Jiang, and Xianming Liu.
Debiased all-in-one image restoration with task uncertainty regularization.
In *AAAI*, 2025.
- Wu et al. [2022]
Lemeng Wu, Mengchen Liu, Yinpeng Chen, Dongdong Chen, Xiyang Dai, and Lu Yuan.
Residual mixture of experts.
*arXiv preprint arXiv:2204.09636*, 2022.
- Wu et al. [2024]
Xun Wu, Shaohan Huang, and Furu Wei.
Mixture of loRA experts.
In *ICLR*, 2024.
- Xiangtao et al. [2024]
Kong Xiangtao, Dong Chao, and Zhang Lei.
Towards effective multiple-in-one image restoration: A sequential and prompt learning strategy.
*arXiv preprint arXiv:2401.03379*, 2024.
- Xie et al. [2021]
Liangbin Xie, Xintao Wang, Chao Dong, Zhongang Qi, and Ying Shan.
Finding discriminative filters for specific degradations in blind super-resolution.
In *NeurIPS*, 2021.
- Xu et al. [2022]
Xiaogang Xu, Ruixing Wang, Chi-Wing Fu, , and Jiaya Jia.
Snr-aware low-light image enhancement.
In *CVPR*, 2022.
- Xu et al. [2024]
Yimin Xu, Nanxi Gao, Yunshan Zhong, Fei Chao, and Rongrong Ji.
Unified-width adaptive dynamic network for all-in-one image restoration.
*arXiv preprint arXiv:2405.15475*, 2024.
- Yang et al. [2023]
Qinhong Yang, Dongdong Chen, Zhentao Tan, Qiankun Liu, Qi Chu, Jianmin Bao, Lu Yuan, Gang Hua, and Nenghai Yu.
Hq-50k: A large-scale, high-quality dataset for image restoration.
*arXiv preprint arXiv:2306.05390*, 2023.
- Yang et al. [2017]
Wenhan Yang, Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan.
Deep joint rain detection and removal from a single image.
In *CVPR*, 2017.
- Zamfir et al. [2024]
Eduard Zamfir, Zongwei Wu, Nancy Mehta, Danda Dani Paudel, Yulun Zhang, and Radu Timofte.
Efficient degradation-aware any image restoration.
*arXiv preprint arXiv:2405.15475*, 2024.
- Zamfir et al. [2025]
Eduard Zamfir, Zongwei Wu, Nancy Mehta, Yuedong Tan, Danda Pani Paudel, Yulun Zhang, and Radu Timofte.
Complexity experts are task-discriminative learners for any image restoration.
In *CVPR*, 2025.
- Zamir et al. [2021]
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao.
Multi-stage progressive image restoration.
In *CVPR*, 2021.
- Zamir et al. [2022]
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang.
Restormer: Efficient transformer for high-resolution image restoration.
In *CVPR*, 2022.
- Zhang and Patel [2018]
He Zhang and Vishal M. Patel.
Density-aware single image de-raining using a multi-stream dense network.
In *CVPR*, 2018.
- Zhang et al. [2023a]
Jinghao Zhang, Jie Huang, Mingde Yao, Zizheng Yang, Hu Yu, Man Zhou, and Feng Zhao.
Ingredient-oriented multi-degradation learning for image restoration.
In *CVPR*, 2023a.
- Zhang et al. [2017]
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang.
Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising.
*IEEE TIP*, 26(7):3142–3155, 2017.
- Zhang et al. [2018]
Kai Zhang, Wangmeng Zuo, and Lei Zhang.
Learning a single convolutional super-resolution network for multiple degradations.
In *CVPR*, 2018.
- Zhang et al. [2021a]
Kai Zhang, Jingyun Liang, Luc Van Gool, and Radu Timofte.
Designing a practical degradation model for deep blind image super-resolution.
In *ICCV*, 2021a.
- Zhang et al. [2011]
Lei Zhang, Xiaolin Wu, Antoni Buades, and Xin Li.
Color demosaicking by local directional interpolation and nonlocal adaptive thresholding.
*J. Electronic Imaging*, 20(2):023016, 2011.
- Zhang et al. [2023b]
Ruofan Zhang, Jinjin Gu, Haoyu Chen, Chao Dong, Yulun Zhang, and Wenming Yang.
Crafting training degradation distribution for the accuracy-generalization trade-off in real-world super-resolution.
In *ICML*, 2023b.
- Zhang et al. [2022a]
Wenlong Zhang, Guangyuan Shi, Yihao Liu, Chao Dong, and Xiao-Ming Wu.
A closer look at blind super-resolution: Degradation models, baselines, and performance upper bounds.
In *CVPRW*, June 2022a.
- Zhang et al. [2021b]
Xinyi Zhang, Hang Dong, Jinshan Pan, Chao Zhu, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, and Fei Wang.
Learning to restore hazy video: A new real-world dataset and a new method.
In *CVPR*, pages 9239–9248, 2021b.
- Zhang et al. [2023c]
Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian.
Controlvideo: Training-free controllable text-to-video generation.
*arXiv preprint arXiv:2305.13077*, 2023c.
- Zhang et al. [2021c]
Zhilu Zhang, Haolin Wang, Ming Liu, Ruohao Wang, Jiawei Zhang, and Wangmeng Zuo.
Learning raw-to-srgb mappings with inaccurately aligned supervision.
In *ICCV*, 2021c.
- Zhang et al. [2022b]
Zhilu Zhang, RongJian Xu, Ming Liu, Zifei Yan, and Wangmeng Zuo.
Self-supervised image restoration with blurry and noisy pairs.
In *NeurIPS*, 2022b.
- Zhang et al. [2024]
Zhilu Zhang, Shuohao Zhang, Renlong Wu, Zifei Yan, and Wangmeng Zuo.
Bracketing is all you need: Unifying image restoration and enhancement tasks with multi-exposure images.
*arXiv preprint arXiv:2401.00766*, 2024.
- Zhong et al. [2024]
Zihan Zhong, Zhiqiang Tang, Tong He, Haoyang Fang, and Chun Yuan.
Convolution meets loRA: Parameter efficient finetuning for segment anything model.
In *ICLR*, 2024.
- Zhou et al. [2021]
Yuqian Zhou, David Ren, Neil Emerton, Sehoon Lim, and Timothy Large.
Image restoration for under-display camera.
In *CVPR*, 2021.
- Zhu et al. [2022]
Jinguo Zhu, Xizhou Zhu, Wenhai Wang, Xiaohua Wang, Hongsheng Li, Xiaogang Wang, and Jifeng Dai.
Uni-perceiver-moe: Learning sparse generalist models with conditional moes.
In *NeurIPS*, 2022.
- Zhu et al. [2023]
Yurui Zhu, Tianyu Wang, Xueyang Fu, Xuanyu Yang, Xin Guo, Jifeng Dai, Yu Qiao, and Xiaowei Hu.
Learning weather-general and weather-specific features for image restoration under multiple ad verse weather conditions.
In *CVPR*, 2023.
- Zoph et al. [2022]
Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeff Dean, Noam Shazeer, and William Fedus.
St-moe: Design stable and transferable sparse expert models.
*arXiv preprint arXiv:2202.08906*, 2022.

## Peer Reviews

confidence: 5
contribution: 2
initial_rating: 6
presentation: 2
questions: Please check the weakness
soundness: 3
strengths: 1. The paper proposes UniRestorer, the first framework that simultaneously exploits degradation and granularity estimation to overcome the inherent limitations of both degradation-agnostic and degradation-aware restoration methods.
2. Extensive quantitative and qualitative experiments convincingly demonstrate the superiority of UniRestorer over existing all-in-one baselines and its competitive performance against task-specific models.
3. The idea of granularity-aware expert selection is clearly articulated and technically grounded; it offers a fresh insight that could inspire future work on robust universal image restoration.
summary: Existing all-in-one restoration schemes are either degradation-agnostic or degradation-aware, leaving a clear performance gap to single-task experts. This paper proposes UniRestorer, which first hierarchically clusters the degradation space and trains a multi-granularity mixture-of-experts (MoE) network. At inference it jointly estimates degradation type and granularity to activate the most suitable expert. Extensive experiments show that UniRestorer significantly surpasses state-of-the-art all-in-one competitors and narrows the gap to dedicated single-task models.
weaknesses: 1. The paper lacks a quantitative analysis of the proposed hierarchical degradation-clustering step.
2. No comparison or discussion is provided against alternative clustering strategies (e.g., the spectral clustering adopted in SEAL).
3. It is unclear whether the Restormer baseline in Table 1 was re-trained under exactly the same degradation protocol and parameter budget; an ablation that removes both degradation and granularity estimation while keeping the backbone capacity fixed would better isolate the gain of the proposed method.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: Report inference FPS on RTX 3090 for 512×512 input — how much slower than PromptIR?
Test on real-world degradations (e.g., DND, SIDD, GoPro real blur).
Ablate dynamic clustering — can granularity be learned end-to-end without offline K-means?
soundness: 3
strengths: Granularity estimation elegantly handles degradation estimation noise — robust and practical.
Hierarchical clustering + MoE scales well across 15+ degradation types.
Large, consistent gains (e.g., +2.1 dB PSNR on mixed test sets).
Comprehensive ablations (granularity levels, routing loss, expert count).
Clean figures: t-SNE of degradation space, expert activation heatmaps.
summary: UniRestorer proposes a multi-granularity MoE framework for all-in-one image restoration. It (1) hierarchically clusters degradation space, (2) trains multi-level MoE experts, and (3) uses joint degradation + granularity estimation to route inputs to the most suitable expert. Claims large gains over SOTA all-in-one models and narrows gap to single-task models. Code and models will be released.
weaknesses: Clustering is offline and static — no online adaptation to new/unseen degradations.
Granularity estimator adds overhead — no inference latency reported (vs. PromptIR, AirNet).
No theoretical justification for hierarchical clustering choice (e.g., why 3 levels?).
Evaluation limited to synthetic degradations — no real-world camera pipeline (e.g., RAW → ISP).
MoE training unstable? No mention of load balancing loss or expert collapse.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: 1. How sensitive is performance to the number of levels (l = 3) or cluster count per level? Could adaptive clustering improve robustness?
2. Can you visualize which experts are selected for different degradations and how granularity affects routing under ambiguous inputs?
3. Could the same hierarchical expert tree generalize to unseen degradation combinations(e.g., low-light + blur + noise) without retraining?
4. In hybrid usage, can user-provided degradation cues override granularity routing? Would it further close the gap with single-task models?
soundness: 3
strengths: 1. This paper introduces a multi-granularity degradation representation that unifies coarse- and fine-grained experts. The idea of granularity estimation to quantify degradation uncertainty and guide routing is novel and intuitive.
2. Comprehensive experiments: covers 7 single-degradation and 11 mixed-degradation settings, plus real-world and unseen tasks.
3. Paper is well-organized and technically detailed with intuitive figures (especially Fig. 3 illustrating routing).
4. The hierarchical MoE design could inspire cross-domain generalization and adaptive restoration architectures.
summary: The paper proposes UniRestorer, a universal all-in-one image restoration framework that bridges degradation-agnostic and degradation-aware paradigms. It performs hierarchical clustering on degradation representations to build a multi-granularity mixture-of-experts (MoE) model. Through degradation and granularity estimation, the router adaptively selects fine- or coarse-grained experts, achieving robustness against estimation errors. Experiments on single- and mixed-degradation benchmarks show that UniRestorer surpasses prior all-in-one methods and nearly matches single-task performance
weaknesses: 1. While granularity estimation is conceptually convincing, there’s no formal uncertainty-theoretic or probabilistic analysis of its behavior.
2. Each granularity level adds parameters and routing complexity; actual FLOPs and latency comparisons are limited. The LoRA variant helps, but trade-offs between full and LoRA experts could be better quantified.
3. The K-means–based clustering assumes a consistent degradation embedding space; sensitivity to clustering hyperparameters (number of clusters, feature normalization) is not reported.
4. Evaluation primarily uses synthetic degradations; more real-world degradation diversity (e.g., motion blur, ISP artifacts) would strengthen the claim of universality.
5. The effect of routing noise or errors in granularity estimation itself is underexplored; visualization of routing confidence would add interpretability.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: The second weakness is the main issue that confuses me the most. I would appreciate a clear explanation, and I may consider raising my rating if it is addressed convincingly.
soundness: 3
strengths: 1. The paper’s fine-grained degradation representation learning is a clear strength. By retraining a DA-CLIP–based extractor with fine-grained textual labels (e.g., light/medium/heavy noise or haze) and contrastive supervision, the authors enable the model to capture not only degradation types but also intensity levels.  Supplementary t-SNE visualizations results show that these representations are separable at both coarse and fine granularity, providing a solid foundation for the subsequent hierarchical clustering and multi-granularity expert design.
2. Demonstrates strong cross-distribution generalization, maintaining stable PSNR/SSIM across single, mixed, OOD, and real-world degradations, with zero-shot gains on unseen types.
3. Uncertainty-aware hierarchical routing adaptively selects coarse or fine experts for robustness or precision, reducing routing ambiguity and representation conflict while matching or surpassing single-task performance.
4. Comprehensive evidence beyond parameter scaling: broad baselines and ablations show gains arise from division of labor and routing, with the LoRA variant retaining near full-model performance.
summary: UniRestorer presents a strong universal image restoration framework that achieves impressive performance across single, mixed, OOD, and even real-world degradations. Its combination of multi-granularity degradation representations, hierarchical expert specialization, and uncertainty-aware routing enables both robustness and precision, outperforming prior all-in-one baselines and approaching single-task SOTA results.
However, the core mechanism raises conceptual concerns: the training pipeline is overly complex, and the single-expert activation strategy appears suboptimal and parameter-inefficient. Clarification on these design choices—especially how the MoE routing behaves under mixed degradations—would be crucial to fully assess the method’s contribution.
weaknesses: 1. The inference scheme activates only a single expert at a time, which limits the expressive power of the MoE and introduces substantial parameter redundancy, as many experts remain unused for each input.
2. When the MoE system encounters mixed degradations, how frequently does the router fall back to the 0-th level (coarse) expert?
If this fallback occurs in most cases, it is unclear why the model significantly outperforms a single Restormer trained directly on mixed degradations. Conversely, if the router instead activates a fine-level expert (e.g., a “rain” or “haze” expert) for a mixed input such as rain-haze, it would contradict the intended degradation clustering principle and could potentially degrade performance. Clarification on the router’s behavior and expert selection under mixed-degradation inputs is needed.
3. The overall training pipeline is overly complex and resource-intensive. It requires first training a degradation extractor, then performing hierarchical clustering, followed by separate training for multiple experts and an additional router stage.
4. The paper does not clarify how data sufficiency is ensured for the fine-granularity experts. Since the hierarchical clustering process recursively divides the training set into smaller subsets, some fine-level clusters may contain only a limited number of samples. It is unclear whether the authors applied any method to avoid underfitting or data imbalance across experts.


Decision: Accept (Poster)