Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model
Authors: Chunming He
Chengyu Fang
Yulun Zhang
Longxiang Tang
Jinfa Huang
Kai Li
Zhenhua Guo
Xiu Li
Sina Farsiu
Venue: ICLR 2025 Spotlight (2025)
Keywords: Illumination degradation image restoration
Latent diffusion model
Retinex theory
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: The first latent diffusion model-based methods with strong generalizability in illumination degradation image restoration problems and promising performance in downstream tasks

Abstract: Illumination degradation image restoration (IDIR) techniques aim to improve the visibility of degraded images and mitigate the adverse effects of deteriorated illumination. Among these algorithms, diffusion-based models (DM) have shown promising performance but are often burdened by heavy computational demands and pixel misalignment issues when predicting the image-level distribution. To tackle these problems, we propose to leverage DM within a compact latent space to generate concise guidance priors and introduce a novel solution called Reti-Diff for the IDIR task. Specifically, Reti-Diff comprises two significant components: the Retinex-based latent DM (RLDM) and the Retinex-guided transformer (RGformer). RLDM is designed to acquire Retinex knowledge, extracting reflectance and illumination priors to facilitate detailed reconstruction and illumination correction. RGformer subsequently utilizes these compact priors to guide the decomposition of image features into their respective reflectance and illumination components. Following this, RGformer further enhances and consolidates these decomposed features, resulting in the production of refined images with consistent content and robustness to handle complex degradation scenarios. Extensive experiments demonstrate that Reti-Diff outperforms existing methods on three IDIR tasks, as well as downstream applications.

Published as a conference paper at ICLR 2025
RETI-DIFF: ILLUMINATION DEGRADATION IMAGE RESTORATION WITH RETINEX-BASER
LATENT DIFFUSION MODEL
Chunming Hel Chengyu Fang2.*.1 Yulun Zhang Longxiang Tang", Jinfa Huang" Kai Li Zhenhua Guo Xiu Li and Sina Farsiu i-f Duke University, Tsinghua University. Shanghai Jiao Tong University, PPking University, 6 Meta, Tianyi Traffic Technology.
BB
Ground Truth Figure 1: Results of Retinex-based cutting-edge image restoration methods, where Reti-Diff can better highlight details and correct color distortions. The dashed boxes indicate failure cases or artifacts produced by existing methods, which can be properly addressed by our approach.
Input
Retformer
Diff-Retinex
Ours
ABSTRACT
Illumination degradation image restoration (IDIR) techniques aim to improve the visibility of degraded images and mitigate the adverse effects of deteriorated il- lumination. Among these algorithms, diffusion-based models (DM) have shown promising performance but are often burdened by heavy computational demands and pixel misalignment issues when predicting the image-level distribution. To tackle these problems, we propose to leverage DM within a compact latent space to generate concise guidance priors and introduce a novel solution called Reti- Diff for the IDIR task. Specifically, Reti-Diff comprises two significant compo- nents: the Retinex-based latent DM (RLDM) and the Retinex-guided transformer (RGformer). RLDM is designed to acquire Retinex knowledge, extracting re- flectance and illumination priors to facilitate detailed reconstruction and illumi- nation correction. RGformer subsequently utilizes these compact priors to guide the decomposition of image features into their respective reflectance and illumi- nation components. Following this. RGformer further enhances and consolidates these decomposed features, resulting the production of refined images with consistent content and robustness to handle complex degradation scenarios. Ex- tensive experiments demonstrate that Reti-Diff outperforms existing methods on three IDIR tasks, as well as downstream applications. The source code is available at https: / / hubb com/ Chunninggte -Reti-Diff.
INTRODUCTION
Illumination degradation image restoration (IDIR) seeks to enhance the visibility and contrast of degraded images while mitigating the adverse effects of deteriorated illumination, e.g., indefinite noise and variable color deviation. IDIR has been investigated in various domains, including low- light image enhancement (Cai et al., 2023), underwater image enhancement (Guo al., 2023), and backlit image enhancement (Liang et al., 2023). By addressing illumination degradation, the
"Equal Contribution, t Corresponding Author, Contact: chunming he @dukc.cdu.

---

Published as a conference paper at ICLR 2025
Low-light enhancement
Underwater image enhancement
Backlit image enhancement
light object detection
20
76
LyL
LSUI
23
BAID
f
1.12
3.3
PSNR
27-
PSNR
PSNR
1.86 dB
mAP
26-
25-
SNR Retbomer Our Figure 2: Our Reti-Diff achieves a leading place in three IDIR tasks and the low-light object detec- tion task, and outperforms the corresponding cutting-edge techniques on these tasks, where CLIP and Diff-Re are short for CLIP-LIT (Liang et al., 2023), Diff- biff-Retinex (Yi et al., 2023).
MRQ IAGG Retorner Ours
ADP PUGAN NUZNet Ours CUP Retfomer Ours RUAS
enhanced images are expected to exhibit improved visual quality, making them more suitable for decision-making or subsequent tasks like nighttime object detection and segmentation.
Traditional IDIR approaches (Fu et al., 2016a; Ueng & Scharf, 1995) primarily rely on manually crafted enhancement techniques with limited generalization capabilities. Leveraging the robust feature extraction capabilities of convolutional neural networks and transformers, series of deep learning-base methods (Cai al., 2023; Jiang et al., 2021) have been proposed and have achieved remarkable success in the IDIR domain. However, as depicted in Figs. and 2, they still face chal- lenges in complex illumination degradation scenarios due to their constrained restoration capacity.
To overcome this, deep generative models, like generative adversarial networks (He et al., 2023a), have gained popularity for their generative abilities. Recently, the diffusion model (DM) (Yi al., 2023) has been introduced to the IDIR field for high-quality image restoration. However, existing DM-based methods, e.g., Diff-Retinex (Yi et al., 2023) and GSAD (Jinhui et al., 2023), apply DM directly image-level generation, leading to two main challenges: (1) These methods incur high computational costs, as predicting the image-level distribution requires large number of inference steps. (2) The enhanced results may exhibit pixel misalignment with the original clean image in terms of restored details and local consistency. For example, as shown in Fig. 1, feff-Retinex fails to recover the car's details in the top and introduces severe artifacts in the bottom row.
To address the above challenges, we introduce a latent diffusion model (LDM) to solve the IDIR problem. The computational burden is reduced by applying DM in the low-dimensional, compact latent space. In addition, by integrating LDM with transformers, we prevent pixel misalignment in generated images (see Fig. 1). common issue in deep generative models. Unlike existing LDM- based methods that rely solely on priors extracted from the RGB domain, our method, tailored to the specific characteristics of IDIR tasks, empowers LDMs to extract Retinex information from both the reflectance and illumination domains. This adaptation allows our method to generate high- fidelity Retinex priors directly from low-quality input images. The compact priors preserve high- quality information while minimizing the impact of degradation. Thus, our method simultaneously enhances image details using the reflectance prior and corrects color distortions with the illumination prior, resulting in visually appealing images with favorable downstream tasks.
With this inspiration, we present Reti-Diff, the first LDM-based solution to tackle the IDIR problem. Reti-Diff, depicted in Fig. 3, consists of two primary components: the Retinex-based LDM (RLDM) and the Retinex-guided transformer (RGformer). Initially, RLDM is employed to generate Retinex priors, which are then integrated into RGformer to produce visually appealing results. To ensure the generation of high-quality priors, we propose two-phase training approach, wherein Reti- Diff undergoes initial pretraining followed by subsequent RLDM optimization. In phase I, introduce a Retinex prior extraction (RPE) module to compress the ground-truth image into the highly compact Retinex priors, namely the reflectance prior and the illumination prior. These priors are then sent to RGformer to guide feature decomposition and the generation of reflectance and illumination features. Afterward, RGformer employs the Retinex Retinex-guided multi-head cross attention (RG-MCA) and dynamic feature aggregation (DFA) module to refine and aggregate the decomposed features, ultimately producing enhanced images with coherent content and ensuring robustness and generalization in extreme degradation scenarios. In phase II, we train RLDM in reflectance and illumination domains to estimate Retinex priors from the low-quality image, with the constraint of consistency with those extracted by RPE from the ground-truth image. Therefore, the extracted Retinex priors can guide the RGformer in detail enhancement and illumination correction, resulting in visually appealing results with favorable downstream performance.
2

---

Published as a conference paper at ICLR 2025
Our contributions are summarized follows:
We propose novel DM-based framework, Reti-Difff for the IDIR task. To the best of our knowledge, this is the first practice of the latent diffusion model to tackle the IDIR problem.
We propose to let RLDM learn Retinex knowledge and generate high-quality reflectance and illumination priors from the low-quality imput, which serve as critical guidance in detail enhancement and illumination correction and can be integrated with various methods.
We propose RGformer, which integrates extracted Retinex priors to decompose features into reflectance and illumination components. Subsequently, RG-MCA and DFA are em- ployed to refine and aggregate these decomposed features, ensuring robustness and gener- alization in complex illumination degradation scenarios.
Extensive experiments on four IDIR tasks verify our superiority, efficiency, and gener- alizability to existing methods in terms of image quality and favorability in downstream applications, including low-light object detection and image segmentation.
RELATED WORK
Illumination Degradation Image Restoration. Early IDIR methods mainly include three ap- histogram equalization (HE) (Cheng & Shi, 2004), gamma correction (GC) (Huang et al., 2012), and Retinex theory (Land, 1977). HE-based and GC-based methods focused on directly am- plifying the low contrast regions but ignore illumination factors. Retinex-based variants (Fu al., 2016b; Li et al., 2018) proposed priors to constrain the solution space for reflectance and illumina- tion maps. However, these methods still rely hand-crafted priors, limiting their generalization ability. With the development of deep learning, methods based on CNNs and transformers (Peng et al, 2025; He et al., 2023a; Jin al., 2022; 2023; Pu et al., 2024; Li al., 2020) have succeeded in IDIR. For instance, DIE (Wang et al., 2019) integrated Retinex cues into a learning- -based structure, presenting one-stage Retinex-bass based solution for color correction. To enhance generative capacity, Diff-Retinex (Yi et al., 2023) and GSAD (Jinhui et al., 2023) introduced DM to the IDIR field by directly applying it to image-level generation. However, they entail significant computational costs and may lead to pixel misalignment with the original input, particularly concerning restored image details and local consistency.
proaches:
Diffusion Models. Diffusson models (DMs) have verified great success in density estima- tion (Kingma et al., 2021; Lin et al., 2024) and data generation (He ct al., 2024a; Zhu et al., 2024b;a). Such probabilistic generative model adopts parameterized Markov chain to optimize the lower variational bound on the likelihood function, enabling them to generate target distributions with greater accuracy. Recently, DMs have been introduced to solve the IDIR problem (Yi et al., 2023; Jinhui et al., 2023; Jin et al., 2024a;b; He et al., 2023b). However, when directly applied to image- level generation, these methods bring computational burdens and pixel misalignment. To overcome this, we employ LDM to estimate priors within a oww-dieesiional latent space and then integrate these priors into the transformer-
framework, addressing the above problems. Besides, unlike existing LDM methods (Xia et al., 2023; Chen et al., 2023) that solely rely on priors extracted from the RGB domain, our method, tailored to the IDIR task, empowers L.DMs to extract Retinex information from both the reflectance and illumination domains. This adaptation allows our method to generate high-fidelity compact Retinex priors directly from low-quality input images but avoid the impact of degradation. By doing so, this novel approach enables us to simultaneously enhance image details using the reflectance prior and correct color distortions with the illumination prior, resulting in visually appealing results with favorable downstream tasks.
3 METHODOLOGGY
In this paper, we propose Reti-Diff, the pioneering method based on Latent Diffusion Models (LDM) for IDIR tasks. Reti-Diff is specifically tailored to address the challenges inherent in IDIR tasks by leveraging high-quality Retinex priors extracted from both the illumination and reflectance domains to guide the restoration process. This innovative approach utilizes the extracted Retinex prior repre- sentation dynamic modulation parameters, facilitating simultancous enhancement of restoration details through the reflectance prior and correction of color distortion via the illumination prior. This ensures the generation of visually compelling results while positively impacting downstream tasks.
3

---

Published as a conference paper at ICLR 2025
Aunddaa Slock
Dri DecipooBte
Teaty RoAi
Txt Depthrattee Conwirtiion
Laper Norm
Concenrator
Remnape
Hademand product
Deansing Nehwork
Rororeer Block
DEUL
Retinx Decomposition
RLON
Block WRL
B occ (oMi))
RG-MCA
74
Block
200 (400)
1
RGformer
[e] RG-CCA
DFA
Figure 3: Framework of Reti-Diff. In Phase I, we pretrain Reti-Diff with RGformer and RPE to ensure the robust learning of RLDM and then optimize RLDM to generate high-quality Retinex priors in Phase II, which guide RGformer in detail enhancement and illumination correction. In (a), we omit the auxiliary decoder D+ for simplicity. panel (c), we illustrate the use of RLDM to extract the reflectance prior; the illumination prior can be extracted similarly. Zoom in for clarity.
As shown in Fig. 3, our Reti-Diff comprises two parts: the Retinex-guided transformer (RGformer) and the Retinex-based latent diffusion model (RLDM). To ensure the generation of high-quality priors, Reti-Diff undergoes two-phase training strategy, involving the initial pretraining of Reti- Diff and the subsequent optimization of RLDM. In this section, we provide in-depth explanation of the two- -ho-phase training approach and elucidate the entire restoration process.
3.1 PRETRAIN RITI-DIFF
We first pretrain Reti-Diff to encode the ground truth image into compact priors with Retinex prior extraction (RPE) module and use the extracted Retinex priors to guide RGformer for restoration.
Retinex prior extraction module. Given the low-quality (LQ) image ILQ E IR H x and its corresponding ground truth IGr E IR x3 we initially decompose them into the reflectance image E R #xxx3 x3 and the illumination map E R HxW according to Retinex theory:
LL = RIQ @ LLQ, ler = Rcr @ LGT,
(I)
where @ is Hadamard product. Following URetinex (Wu et al., 2022), we use a pretrained decom- posing network (-) to decompose ILQ and ICT, comprising three Conv+LeakyReLU layers and a Conv+ReLU layer. Then we concatenate the corresponding components of ground truth and LQ image and use the RPE module RPE(.) to encode them into Retinex priors ZR IR 30" Zz E RC; ZR RPE(down(conic) Coc.oaaa.rr.....o..oo RGTT, RıQ))), ZL (PP((dow((concu(Ter) LIQ))), where conca(-) denotes concatenation and down(+) represents downsampling that operated by PixelUnshuffle. The Retinex priors, ZR and Zz are then fed into RGformer to serve as dynamic modulation parameters for detail restoration and color correction.
(2)
Retinex-guicee transformer. RGformer mainly consists of two parts in each block, i.e., Retinex- guided multi-head cross attention (RG-MCA) and dynamic feature aggregation (DFA) module. In RG-MCA, we first split the input feature F E IR #'x' into two parts Fi E and F2 ((C//) along the channel dimension. Afterwards, we integrated ZR and Z,, as the corresponding dynamic modulation parameters to generate reflectance- guided feature FR R xW x(3C/4) and illumination-gudder feature Fr, E & x(²/4)
(3) where Norm(.) ) is layer normalization. Li(-) means linear layer. Afterward, aggregate global spa- tial information by projecting FR into query Q = FOFR and key K = W x Fr, and transforming Fr. into value V F.. where W is the combination of point-wise convolution and
FR Li, (Zz) @ Norm(F1) +
F1 = (ZI) O Norm(F2) + Liz (Z.),
4

---

Published as a conference paper at ICLR 2025
3 x 3 depth-wise convolution. We then perform tross-attention and get the output feature F:
F =
SoftMax (QK"/VC).r V.
(4)
By doing so, RG-MCA introduces explicit guidance to fully exploit Retinex knowledge at the fea- ture level and use cross attention mechanism to implicitly model the Retinex theory and refine the decomposed features, which helps to restore missing details and correct color distortion.
Then we employ DFA for local feature aggregation. Apart from the 1 Conv and depth- wise Conv for information fusion, DFA adopts GELU, termed GELU(+), to ensure the flexibility of aggregation (He et al., 2023c). Thus, given F and Z, where Z = conca ZR. Zr.), the output F is
F F + (EL C W2F', F' Li (Z) C (orm(F) + LZ))
(5)
Optimization. Having gotten the enhanced result L we propose a reconstruction loss with I, norm ||- to jointly train RPE and RGformer, which can facilitate the extraction of Retinex priors:
Lece ||T - -Iell.-
(6)
To ensure that the separated features within RG-MCA capture reflectance and illumination knowl- edge, we use an auxiliary decoder (+(+) with the same structure as that in (Locatello et al., 2020).
Da(+) takes F as input and outputs the reconstructed reflectance image Re and illumination map only apply (+)+) for the first transformer block in encoder to get R/ and
LRe- For efficiency, Lhe and for the last block in decoder to get fie and fit- D.(+) is supervised by Retinex loss:
LR R1QQ - R + I L 1 + ||ooT R ll + | Le
(7)
By constraining the input and output ports, Eq. (7) ensures the preservation of essential Retinex in- formation throughout the network. This integration only facilitates the incorporation of Retinex theory into the split features but also enhances the overall restoration capability.
In Phase I, the final loss LP1 formulated with the assistance of hyperparameter A (A1 1): (8)
LPi = LRee + LR
3.2 BETINEX-BASER LATENT DIFFUSION MODEL
In Phase II, we train the RLDM to predict Retinex priors from the low-quality input, which are expected to be consistent with that extracted by RPE from the ground-truth image. Unlike conven- tional LDMs trained on the RGB domain, we introduce two RLDMs with a Siamese structure and train them on distinct domains: the reflectance domain and the illumination domain. This approach, grounded in Retinex theory, equips our RLDM generate more generative reflectance prior ZR to enhance image details, and a more harmonized illumination prior 2r, for color correction. The compact priors retain high-quality information while effectively mitigating the effects of degrada- tion. Note that RLDM is constructed upon the conditional denoising diffusion probabilistic models, with both forward diffusion process and reverse denoising process. To simplify, provide detailed derivation for ZR herein, while that of ŻL can be found in the appendix.
Diffusion process. In the diffusion process, we first use the pretrained RPE to extract the reflectance prior ZR. which is treated the starting point of the forward Markov process, i.e., ZR Z We then gradually add Gaussian noise to ZR by T iterations and each iteration can be defined as:
a ZZ 1) = N (Zhi - .31))
(9)
where t = T....,T. Z denotes the noisy prior at time step t, 8 is the predefined factor that controls the noise variance, and A is the Gaussian distribution. Following (Kingma & Welling, 2013), define a² Hr and ot II a², allowing simplify Eq. (9) as follows:
Z O = ZR:: Va'za (1 (1-a')I)
(10)
Reverse process. In the reverse process, RLDM aims to extract the reflectance prior from pure Gaussian noise. Thus, RLDM samples a Gaussian random noise map Z and then gradually denoise it to run backward from Z to z with the corresponding mean j¹ and variance o²:
p (Zk
1- Z z =N (Z/. z (a (
(11)
5

---

Published as a conference paper at ICLR 2025
Medhook
Sounes
PSNR
SSIM
LOL.Vi
F1D
FID
BIQE
PSNK
LOI comiheeti
SSIM
PSNK
SSIM
STD
BIQU
MiRNet at.mretaa, 2000)
BCE
34.14
0.835
71.16
47.75
30:02
0.8:0
82.25
41.18
31.94
0.876
40.18
96.39
20.84
0,805
81.57
40.6
EaGAN (lung 201))
TIP21
17.48
0.656
153.98
55,83
18:25
0.617
171.28
51.06
16.57
0.714
96.66
45.59
17.25
0.549
77.51
13.47
(Inett,, 2021)
CVPR21
18:33
0.723
127.60
45.17
18:27
0.723
151.62
34.73
16.55
0.652
91.40
46.38
18.44
0.581
72.18
45.02
IPT ctl, 3021)
2112))
CVPR21
16.27
0.504
158.83
20.35
19:50
0.813
07.24
31.37
18.30
0.811
76.79
42.35
20.53
0,618
70.55
36.71
UReiina al 2022)
CVPR22
21.33
03535
35.59
50.37
20.44
0.806
76.74
28.55
24.73
0.897
33.25
33.46
22,09
0.633
71.55
38.44
UFumerr (Wang st,l, 2022)
CVPR22
16.36
0.771
166.69
41.06
18:82
0.771
164.43
40.36
19.65
0.871
58.69
39.75
18.54
0.577
100.14
42.13
Reskomer (Zamir 2002)
CVPR22
22.43
0.823
78.75
53.18
19.94
0.827
114.35
37.27
21.41
0.830
46.89
15.06
22.27
0.649
75.47
32.49
SNR-Nat (Noctal,
CVPR22
34.61
0.43
66.47
28.71
31 46
0.849
68.55
18.53
24.14
0.928
90.52
13.47
22.87
0.625
74.78
30.08
SMG (Xu 2023)
CVPR22
24.82
0.335
69.47
50.15
12.62
0.857
71.76
30.32
25.62
0,905
23.36
20.35
25.18
0.644
77.55
31.50
PyDifl ctae 2003a)
JICA123
21.15
0.857
49.47
21.13
Retfonmer ata. 2023)
ICCV23
25.16
0.343
72.38
26,68
12,80
0.840
79.55
34,39
25.67
0,330
22.78
30.26
24.44
0.680
82.64
35.04
Dur Retines (Y) 2023)
ICCV23
21.58
0.852
51.33
19.62
20.17
0.826
46.67
24.18
24.30
0.921
26.74
26.35
23.62
0,665
58,91
31.17
MRQ (Lib ol. 20233
ICCV23
25:24
0.855
53.32
22.73
22.37
0.854
68.89
13.61
25.54
0,940
20,26
25.09
24.62
0,683
61.09
27.81
(AGC 2022)
ICCV23
24.53
0.442
59.75
25.50
22.20
0.863
70.34
11.30
25.55
0.941
21.28
30.32
24.00
0.688
63.72
29.55
DiflR DXa al, 20233
1CCV23
23.15
0.22
70.13
26.38
21.15
0.816
72.33
28.15
26.76
0.921
28.87
27:74
2517
0.640
78,90
30.56
cueing atal,, 2023)
1CCV23
21.36
0.41
69.5
27.15
21.19
0.829
07.05
28.83
26.41
0.917
31:35
15.83
25225
0.652
77.38
28.85
chshi atal, 20231
NIP525
23.23
0883
51.64
19.96
20.19
0.847
18.85
24.22
0.927
19:24
25.76
AST (Zha at al, 2014g
CVPR24
21.09
0888
8767
21.23
21 68
0.856
91.81
15.17
2225
0.937
17.10
78.78
MambalR (Chanee al, 2024
BCTV24
22:23
0.663
63.99
20.17
11
0.857
56.09
24-66
25.75
0.988
19.75
20.37
21.14
0.856
154.76
76
12.72
Rab-Dol
Ous
25.35
0.866
49.14
17.75
22.97
0.853
43.18
23.66
27.53
0,951
13.26
15.77
25.53
0.692
51.66
25.58
Table I: Results on the LLIE task.
The best two results are in red and blue fonts, respectively.
3r é is the noise in Z/r, and we employ denoising network eg(+) to estimate 0. To operate in the latent space, we further introduce another RPE module RPE(-) to extract the conditional reflectance vector VR E R3C" from the reflectance image RLQ of the LQ image, i.e., VR = RPE(down(RLQ)) Therefore, the denoising network can be represented by co (Z VR.t) By setting the variance to a', we get
where (Z = Vae (ZR. I-a' 6) and (o²)² = I-a'
- I-
(ZR 1- a' (o VR. - VV. a'e',
Z
(12)
VI-a"
where& E¹ ~ ~N(O,)), I). By using Eq. (12) for T iterations, we can get the predicted prior 2R and use it guide RGformer for image restoration. Because the size of the predicted prior Zn E R330 is much smaller than the original reflectance image R.Q.Q E TR HxW xC RLDM needs much less iterations than those image-level diffusion models (Yi et al., 2023). Thus, we can run the complete I' iterations for the prior generation rather than randomly selecting one time step.
Optimization. We propose the diffusion loss to restrict the predicted priors 2R and Zz, generated by two RLDMs with specific weights, be consistent with those extracted from the ground truth:
LD.f ZR
ZR 11 + Iiz Z1
(13)
For restoration quality, we propose joint training RPE, RGformer, and RLDM with the Phase II loss:
Lp2 = LD. + A2 Ree + AgLRR
(14)
and are set as in this paper. The constraints imposed by La and LDif, combined with our approach to extracting Retinex priors in compact space, ensure the generation of high-quality priors that significantly reduce interference from degraded inputs.
where X2 and Ag are two
3.3 INFERENCE
In the inference phase, given the LQ input L.Q. Reti-Diff first uses RPE to extract the conditional vectors VR and VL. and then generates predicted Retinex priors ÊR and 2, with two RLDMs. Un- der the guidance of the Retinex priors, RGformer generates the restored HQ image I IQQ Benefiting from our detinex-based diffusion framework, Iæq enjoys richer texture details and more harmo- nized illumination, presenting visual-appealing results and further enhancing downstream tasks.
4 EXPERIMENT
4.1
EXPERIMENTAL SETUP
Our Reti-Diff's is implemented in PyTocch on four RTX4090 GPUs and is optimized by Adam with momentum terms (0.9, 0.999). In phases I and II, we train the network for 300K iterations and the learning rate is initialized as 2 x 10-4 and gradually reduced to x 10-6 with the cosine annealing (Loshchilov, 2016). Random rotation and flips are used for augmentation. Reti-Diff comprises RLDM and RGformer. For RLDM, the channel number C" and the total time step T are set as 64 and 4. g linearly increase from 3 0.1 to 8T 0.99. RGformer adopts a 4-level cascade structure. We set the number of transformer blocks, the attention heads, the channel number as [3. 3, 3,3), 3), [1, 2, 4, 8). (64, 128, 256, 512) from level to 4. We abandon GT-mean for faimess.
6

---

Published as a conference paper at ICLR 2025
Input
Retromeer Figure 4: Visual results the low-light image enhancement (LLIE) task.
Ground Truth
Input
S-uwnet Figure 5: Visual results on the underwater image enhancement (UIE) task.
FUGAN
U-shape
NU2Net
MambalR
Our
Ground Truth
EdS
Input
Unetinex
SNR-Net
Refformer
CLIP-IT
M MambalR
Qurs
Ground Truth
Figure 6: Visual results on the backlit image enhancement (BIE) task.
4.2
COMPARATIVE EVALUATION
Low-light Image Enhancement. We conduct experiments on four datasets: LOL-vl (Wei et al., 2018), LOL-v2-real (Yang et al., 202 1), LOL-v2-syn (Yang et al., 2021), and SID (Chen et al., 2019), and involves four metrics: PSNR, SSIM, FID (Heusel et al., 2017), and BIQE (Moorthy & Bovik, 2010). Larger PSNR and SSIM, as well as smaller FID and BIQE, denote superior results. Ad- method against 17 cutting-edge
hering to the training manner in (Cai et al., 2023), we compare techniques and report the results in Table 1. As depicted in Table I, our method emerges as the top performer across all datasets, surpassing the second-best method (Diff-Retinex) by 13.2%. un- derscoring our superiority. Fig. 4 presents qualitative results, showcasing our capacity to generate restored images with corrected illumination and enhanced texture, even in extremely challenging conditions. In contrast, existing methods struggle to address these challenges, such the boundaries of power lines, color distribution of lakes, and textures of wooded areas. Besides, we also compare the ef- ficiency of the diffusion model-based methods with the size of 256 256. As shown in Table 2, our Reti-
Metries
DDiff-Retines PyDiff GSAD
Ours
Parameter (M)
56.88
97.89
17.17
26.11
MAC' (G) FPS
396.32
459.69
1340.63
156.55
4.25
3.63
2.33
12.27
Table 2: Efficiency analysis in diffusion model-based methods.
Diff has the lowest MACs, highest FPS, and the second smallest parameters. This efficiency can be attributed to our utilization of the diffusion model within a oow-dimensional compact latent space. For fairness, results from the compared methods generated by their provided models.

---

Published as a conference paper at ICLR 2025
(a) Orgin
(b) CUE
(c) Retformer
(d) Ren- Difr
Figure 7: Visual results on real-world fundus images, where we employ the tre-trained model from the LOL-vl dataset for inference. Our Reti-Diff presents enhanced details with less color distortion.
Malbods
Soarcss
PSNR
TSSIM
UUIQEE
PSNRR
Mathada
PSNR
RUGAN Odda et ol. 20100
BRAI.20
17:41
03:442
0.537
2.534
32.16
0.857
11.576
2.967
EAGGAN (lanngllnnge al 20013
TIP3
17.96
[1519
0.192
43.55
EaGAN Dlass 2021)
TIP21
17:23
0.233
0.520
2.465
19:30
0.551
0.587
2,817
RLAA (Lis elal. 2021)
CVPR21
18.92
0.813
0.262
40.00
Leoler (Li 2023)
TIP21
0.337
2.735
URtetnex 2002)
19.08
0.545
0.206
CNP927
2022)
BCCV22
3.601
0.502
2.914
Restones
2002)
CVPR22
21.07
0.52
0,192
Ushupe (Peng dl, 2027)
TIP25
2291
0.65
0,592
2.596
24.16
0.917
0.003
3.022
Redioner (Cal et oll, 2023)
10CV23
22.00
0.62
0,173
45.23
PUGAN (Congas di al. 2003)
TIP33
23,05
0.887
0.666
1.502
25.06
0.916
0.82
3.106
CLISSITT (1 2tal,, 2015)
100923
21.13
018)
0.159
ADP (2bo 21033)
DCV33
23.00
0.822
0.211
3.005
24.28
0.913
D.506
3.074
Dift Recinex eViet 2003)
I00V25
22.07
0.66
0.160
38.0
NEE2Na ct 2023)
AAA12
0.387
3112
100N/1
0.175
22.0
2024)
0.651
Manballt 20200
30CV24
2.591
0.916
3.11%
ManbolR (Gan 2034)
BCCV24
33.07
0.874
0.151
29.13
Dill
Ouzs
24.02
0,910
0.631
3.668
28.10
0,929
0.645
3.205
Ren--NH
Oues
33.19
0.876
0.147
37.47
Table 3: Results on the UIE task.
Table 4: Results on the BIE task.
Methods
Sources
P1_ NIOE
DNCH
[11
ZIME
NIQE
P1L
MEF
NB0E
NPE
N10E
Dotsets
Metsics
PSNR 1 SSIM
L.22
PSNR 1 SSIM
L-12ss
EaGAN Dhasg stal, 2021)
KuDDe (Zhugg
UCV31
TIP2]
4.173
3.635
4.064
3.569
3.785
4.908
4.593
[4.155
4.016
4.557
4.705
13,226
3.955
3,905
1.386
3.773
3.822
4.047
R11 RIDD
wioo RLDM
web
21.25
21.72
0,522
0.530
25:38
25.83
0.915
0.927
Net Uiiifor (1 2025)
20223
CVPR22
3,630
1.709
3312
4,455
3.424
4.598
3.706
3.615
3.286
DEF
22.25
0,540
26.49
0.925
5.64
4.525
4.124
4,400
1.813
4.331
1.335
3,867
1.319
4.310
RC36ommee
weo RG-MCA
31.21
0.340
15.92
0.913
PuRLLE (Pu 2022)
CVPR23
3,605
4.054
3,397
4.507
4.33
4.065
1.736
4.157
3,334
3.5%
wio 17,(1
22.58
0.547
26.80
0,944
CSAD (Jtiba 2023)
CDP et 2023)
CVPR23
NIPS15
3.553
4.350
3,465
4.115
4.591
4.517
3.0A4
3.815
4.609
3,097
4,032
3.80m
3.451
4.83
3.355
Triin
wio jokns
22.83
0.553
27.18
0,947
Reti-Dif
Oues
3,355
2.877
3460
3,792
3.59
3
3,000
Reti Duf (Oum))
22.07
37.53
Table 5: Results on the real-world IDIR task.
Table 6: Break down ablation.
Underwater Image Enhancement. We select two widely-used underwater image enhancement datasets: UIEB (Li et al., 2019) and ISSU (Peng al., 2023). Following (Guo et al., 2023), employ two metrics tailored for underwater images, namely UCIQE (Yang & Sowmya, 2015) and UIQM (Panetta et al., 2015). In all cases, higher values indicate better performance. The results are presented in Table 3. As shown in Table 3, our method achieves the highest performance and outper- forms the second-ber best method (MambalR) by 2.30%. qualitative analysis is presented in Fig. 5, illustrating our capacity to correct underwater color aberrations and highlight texture details.
Backlit Image Enhancement. Following CLIP-LIT (Liang et al., 2023), we select the BAID (Lv et al., 2022) dataset for network training. Apart from PSNR and SSIM, our evaluation also selects two perception metrics: LPIPS (Zhang et al., 2018) and FID (Heusel et al., 2017), where lower values denote better performance. We report our results in Table 4. As demonstrated in Table 4, our method outperforms all other methods across all metrics. Besides, a visual comparison in Fig. provides additional evidence of our superiority in detail reconstruction and color correction.
Real-world Illuiinaiion Degradation Image Restoration. We also explore our applicability in real-world IDIR tasks. Following CIDNet (Feng et al., 2024), we selected five commonly-use used real- world datasets, i.e., DICM (Lee et al., 2013), LIME (Guo et al., 2016), MEF (Wang et al., 2013), NPE (Ma et al., 2015), and VV (He al., 2024b), with only low-quality images available. Therefore, akin to (Feng et al., 2024), we leverage the model pretrained on LOL-v2-syn for inference and select PI (Blau et al., 2018) and NIQE (Mittal al., 2012) as evaluation metrics, where lower scores indicate better results. As presented in Table 5, our method achieves optimal results and surpasses the second-baad based method (DCC-Net (Zhang et al., 2022)) by 13.39%.
8

---

Published as a conference paper at ICLR 2025
Imput
wio Retinex Z
wi Reflechucce Zn
Ground Truth (a) Break down ablation Retinex priors. Figure 8:
w/ Illumiaaioon Z
w/ Retines (Ouns)
Input
Snap 2 Swap our Retinex priors with that extracted from GT.
Ours
Soap Zu
Ground Truth
(b)
Visual validation of the effectiveness of Retinex priors.
Dotoset
Metrics
Zu.
Zz
All daza
(Ous)
Swap Zn
Swap Zu.
Extreme condition (similar Zn)
Za
Zz w/ (Ouns)
Extreme condition tsimilar Zz,)
Zz, (Our))
L-12-r
SSIM
PSNR
21.63
0.830
22.13
0.842
22.35
0.839
0.888
22.97
0.862
23.31
0.863
23.17
0.233
21.37
21,66
0.834
0.43
0.82
23,57
0.80
21.19
23.06
0.845
0.811
21,85
24.06
0,868
L-
SSIM
PSNR
26.25
0.939
26.82
0.945
27.02
0.911
27.53
0,951
0,957
27.92
0.956
27.75
25.68
0.923
26.17
0,900
27.05
0.511
0.966
28.57
0,920
25.42
0.958
27.34
26.24
0.936
28,80
0.965
Table 7: Effect of Retinex priors in all data and two extreme conditions (each with 10 images).
Datasets
Metrics
Ufo
Ufo+RLDM
Res
Res+RDDM
Ret
ReteRLDM
0.31
PSNR
18.82
21.37
19.94
21.56
22.80
23.16
0:4
L-v2-r
SSIM
0.771
0.794
0.827
0.877
0.840
0.849
Gain
8,27%
4.67%
1.33%
032
PSNR
19.66
22.08
21
24.15
25.67
26.81
L.22ar
SSIM
0.871
0.889
0.830
0,862
0.930
0,942
Gain
7.19%
8.33%
2.87%
Nurrdte of Baesiorn
Kurtber Renaten
Generalization of Retinex priors. "Ufo", and "Ret" are Uformer, Restormer, and Retformer.
Table 8:
"Res", Figure 9: Ablation study of the number
of iterations in RLDM on LOL-v2-syn.
4.3 ABLATION STUDY
Effect of RLDM. As shown in Tables 6 and 7, we ablate RLDM by directly removing RLDM, replacing the diffusion model with a linear model that shares the same structure with the denois- ing network (w/o DM), or retraining RLDM in the RGB domain. i.e. w/o Z, rather than in the reflectance and illumination domain (RGformer is guided by one RGB prior instead). The three changes bring significant performance drops, underscoring the critical role of RLDM in enhancing the restoration process and the importance of using the diffusion model to extract compact priors.
Effeet of RGformer We analyze the impact of our RGformer by removing key modules, such as DFA, RG-MCA, and the auxiliary decoder (-)-). As shown in Table 6, the outcomes indicate performance decreases when these modules are removed, highlighting their essential roles. Addi- tionally, also conduct an evaluation to affirm the significance of joint training in method.
Effect of Retinex priors. We explore the effect of our Retinex prior from three aspects: (1) We conduct the break down ablation for the Retinex priors and report the results in Fig. 8 and Table 7. These findings demonstrate the effect of our Reflectance prior ZR in detail enhancement and our Illumination prior Zr in illumination correction. (2) We then swap our Retinex priors with those extracted from ground truth. As shown in Fig. 8 and Table 7, the results guided by the swapped ground-truth priors exhibit limited performance gains. This indicates our RLDM can already gen- erate high-quality priors. which is attributed to the constraints in Eqs. 7 and 13 and our approach to extracting Retinex priors in compact space, significantly reducing interference from degraded inputs. (3) We further explore the potential of Retinex priors under extreme conditions where the re- flectance or illumination priors exhibit high similarity between low-quality and ground-truth images. To validate this, five human subjects rated the similarity of the Retinex priors between low-quality and ground-truth images. Two sets of images, each with 10 images, were selected based on the highest similarity in reflectance and illumination priors. The results in Table 7 verify the effect of our Retinex priors even in this condition. This is attributed to the generative capacity of our RLDM and the information aggregation capacity of our RGformer.
Generalization of Retinex priors. To assess our generalizability, we incorporate our RLDM into existing cutting-edge methods, namely Ufo (Uformer (Wang et al., 2022)), Res (Restormer (Zamir et al., 2022)) and Ret (Retformer (Cai et al., 2023)), and use the extracted Retinex priors to guide these methods for image enhancement, where the training settings are kept consistent with Reti-Diff. The results are shown in Table 8. Table 8 reveals that RLDM significantly improves the performance of all frameworks, indicating the strong generalization capabilities of our Retinex priors.

---

Published as a conference paper at ICLR 2025
Methods
L-vl
L-12-r
L-v2-s
SID
Mcan
Methods (AP) Bicycle Boat
Car
Cup
Dog
Moter
People
Table
Mean
KinD
2.31
2.25
2.46
2.33
2.34
Baseline
74.7
64.9
70.7
84.2
79.7
47.3
58.6
67.1
64.1
66.2
73.9
45.7
66.4
FaGAN
2.63
1.69
2.23
1.24
1.95
RetinetNet
72.8
66.4
80.6
68.5
71.3
67.5
RUAS
3.57
3.06
3.01
2.23
2.97
KinD
73.2
67.1
64.6
86.8
79.5
58.7
63.4
67.5
67.4
62.3
75.5
51.4
68.1
Restormer
3.26
3.32
3.41
2.53
3.13
MiRNet
74.9
69.7
68.3
89.7
77.6 57.8
56.9
56.4
69.7
64.6
74.6
53.4
68.6
Uretinex
3.82
3.98
3,70
3.28
3.70
RUAS
Restormer
77.0
75.7
71,0
71.2
73.5
68.8
91.6
80.1
77.1 62.5
57,3
68.0
69.6
68.3
69.2
74,6
506
19.7
70.2
69.7
SNR Net
3.76
4.12
3.58
3.42
3,72
SCI
73.4
68.0
69.5
86.2
74.5 63.1
59.5
61.0
67.3
63.9
73.2
47.3
67.2
CUE
3.62
3.81
3.28
3.09
3:45
SNR-Net
78.3
74.2
74.5
82.7
74.7
73.3
57.2
71.9
Retformer
3.35
4.02
3,71
3.35
3.61
Retformer
78.1
74.5
74,2
91.2
82.2
65.0
63.3
67.0
75.4
68.6
75.3
55.6
72.5
Ours
4.05
4.33
3,92
3.75
4.01
Ours
82.0
77.9
76.4
92.2
83.3
69.6
67.4
74.4
75.5
74.3
78.3
57,9
75,8
Table 9: User study.
Table 10:
Low- image detection on ExDark.
Mothods (IoU) Bicycle Beat Bottle Bus
Car
Cat
Chair
Dog
Horse
People Mcan Mean
Methoos
Fa
CODIOK
Eg
S.
NCVK
T E, 5.
Baseline
43.5
36.3
48.6
70.5
67.3 46.6
11.2
424
56.7
57.8
48.1
RetinexNet
48.6
41.7
51.7
77.6
68.3
52.
15.8
46.3
60.2
62.3
52.5
Buseline
0.050 0.625
0,812
0.756
0.733 0.816 0.763
KinD
51.3
40.2
53.2
76.8
69.4 50.8
14.6
47.3
60.3
60.9
52.5
RetineeNet
0.041 0.667 0.845 0.789
0.055 0.750 0.842 0.819
MIRNet
RUAS
Restormer
53.0
50.3
53.8
37.3
42.9
43.8
51.4
47,4
68.7
73.6
72.3 47.6
66.8
62.7 50.4
526
21.6
15.9
15.8
50.8
54.8
46.3
61.0
63.6
59.8
63.3
63.3
$1.4
$3.7
Restormer
MIRNet
RUAS
Kinl)
0.037
04039) 0.673 0.849 0.792 0.052 0.762 0.875 01822
0.036 0.200 0.859 2.80000.000
0,857 0.799
0:803 0.051
0.049
0.802 0.888 01833
0.792 0.880 0.830
0.827
SCI
54.5
46.3
57.2
78.4
73.3
49.
22.8
49.0
62.1
66.9
56.0
SCI
0.037 0.710 0,563 0.505 0.051
0.782 0.880 0.836
SNR
57.7
48.6
81.3
74.
50.2
24,4
50.7
64.3
68.7
SNR-Net
0.036 0.703 0.865 0,803
0.049
0.801 0.892 0,838
Retformer
50.9
47.7
58.6
77.2
68.1 53.2
17.
52.0
61.3
71.5
55.8
Reformer
00.337
0.8016
0.052
0,766
Ours
59.8
51.5
85.5
76.6 57.7 28.9
56.3
66.2
73.4
Ours
0,590
0.813
0,841
Table 11: Low-light semantic segmentation, where images Table 12: Low-light concealed object
are darkened by (Zhang et al., 2021a).
segmentation.
4.4
USER STUDY AND DOWNSTREAM TASKS
User Study. We conduct user study assess the subjective visual perception of low-light image enhancement. In this study, 29 human subjects are invited to assign scores to the enhanced results based on four criteria: (1) The presence of underexposed or overexposed regions. (2) The existence of color distortion. (3) The occurrence of undesired noise or artifacts.
(4) The inclusion of essential
structural details. Participants rate the results on a scale from (worst) to (best). Each low- light image is presented alongside its enhanced results, with the names of the enhancement methods concealed. The scores are reported Table 9, where our method receives the highest scores across all four datasets. This highlights our effectiveness in generating visually appealing results.
Low-light Object Detection. The enhanced images are expected to have better downstream perfor- mance. We first verify this on low-light object detection. Following (Cai et al., 2023), all compared methods are performed on ExDark (Loh & Chan, 2019) with YOLO, which is retrained from scratch with their own enhanced results. The "Baseline" represents the performance on low- -ww-qulltty images without enhancement. As shown in Table 10, our Reti-Diff exhibits a substantial advantage over ex- isting methods and our performance surpasses that of the second-best method, Retformer, by 4.72%, verifying our efficacy in facilitating high-level vision understanding.
Low-light Image Segmentation. We also conducted segmentation tasks and retrained the segmen- tor for each method following that in detection. (1) For semantic segmentation, following (Ju et al., 2022), we apply image darkening to samples from the VOC (Everingham et al., 2010) dataset ac- cording to (Zhang et al., 2021a). We then employ Mask2Former (Cheng et al., 2022) to segment the enhanced results of these darkened images and select Intersection over Union (IoU) for evaluation. As shown in Table 1, we achieve the highest performance across all classes, surpassing the second- best method by 6.55%. (2) We further venture into concealed object segmentation (COS) on two datasets, CODIOK (Fan et al., 2021) and NC4K (Lv et al., 2021), which is a challenging task aimed at delineating objects with inherent background similarity. We also apply image darkening and en- list FEDER (He et al., 2023c) to segment the enhanced results. We evaluate the results using four metrics: mean absolute error (M), adaptive F-measure (Fa), mean E-measure (Es), and structure measure (Sa). As depicted in Table 12, our method exhibits superior performance compared to the second- best method, SNR-Net, with a margin of 2. 16% on average.
5 CONCLUSIONS
To balance generation capability and computational efficiency, our approach adopts DM within compact latent space to generate guidance priors. Specifically, we introduce RLDM to extract Retinex priors, which are subsequently supplied to RGformer for feature decomposition, ensur- ing precise detailed reconstruction and effective illumination correction. RGformer then refines and aggregates the decomposed features, enhancing the robustness in handling complex degradation scenes. Our approach is validated through extensive experiments, establishing clear superiority.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Yochai Blau, Roey Mechrez, Radu Timofte, Tomer Michaeli, and Lihi Zelnik-Manor The 2018 pirm challenge on perceptual image super-resolution. In ECCV, pp. 0-0, 2018. 8
Yuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang. Radu Timofte, and Yulun Zhang. Retinexformer: One-stage retinex-based transformer for low-light image enhancement. 12513, 2023. 1, 2, 6, 7, 8, 9, 10
ICCV, pp. 12504-
Chen Chen, Qifeng Chen, Minh N Do. and Vladlen Koltun. Seeing motion in the dark. In ICCV, pp- 3185-3194, 2019. 7
Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng. Zhenhua Liu, Siwei Ma, Chun- jing Xu, Chao Xu, and Wen Gao. Pre-trained image processing transformer. CVPR, pp- 12299-12310, 12310, 2021. 6
Zheng Chen, Yulun Zhang, Ding Liu, Bin Xia, Jinjin Gu, Linghe Kong, and Xin Yuan. Hierarchical integration diffusion model for realistic image deblurring. In NewrIPS, 2023. 3
Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked- attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF
conference on computer vision and pattern recognition, pp. 1290-1299, 2022. 10
Heng-Da Cheng and XJ Shi. simple and effective histogram equalization approach to image enhancement. Digital signal processing, 14(2):158-175) 2004.
Runmin Cong, Wenyu Yang, Wei Zhang, Chongyi Li, Chun-Le Guo, Qingming Huang, and Sam Kwong. Pugan: Physical model-guided underwater image enhancement using gan with dual- discriminators. IEEE Transactions on Image Processing, 2023. 8
Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88: 303-338, 2010. 10
Deng-Ping Fan, Ge-Peng Ji, Ming-Ming Cheng, and Ling Shao. Concealed object detection. IEEE transactions on pattern analysis and machine intelligence, (4110))602-6044, 2021. 10
Ben Fei, Zhaoyang Lyu, Liang Pan, Junzhe Zhang, Weidong Yang. Tianyue Luo, Bo Zhang, and Bo Dai. Generative diffusion prior for unified image restoration and enhancement. In CVPR, pp- 9935-9946, 2023. 8
Yixu Feng, Cheng Zhang, Pei Wang, Peng Wu, Qingsen Yan, and Yanning Zhang. You only need color space: An efficient network for low-light image enhancement. arXiv preprint arXiv:2402.05809 2024. 8
Xueyang Fu, Delu Zeng. Yue Huang, Yinghao Liao, Xinghao Ding, and John Paisley. A fusion- based enhancing method for weakly illuminated images. Signal Processing, 129:82-96. 96, 2016a.
Xueyang Fu, Delu Zeng, Yue Huang, Xiao-Ping Zhang, and Xinghao Ding. A weighted variational model for simultaneous reflectance and illumination estimation. In CVPR, pp. 2782-2790, 201 6b.
Zhenqi Fu, Wu Wang, Yue Huang, Xinghao Ding, and Kai-Kuang Ma. Uncertainty inspired under- water image enhancement. ECCV. pp. 465-482. Springer, 2022. 8
Zhenqi Fu, Yan Yang, Xiaotong Tu, Yue Huang, Xinghao Ding, and Kai-Kuang Ma. Learning a simple low-light image enhancer from paired low-light instances. In CVPR, pp. 22252-22261, 2023. 8
Chunle Guo, Ruiqi Wu, Xin Jin, Linghao Han, Weidong Zhang. Zhi Chai, and Chongyi Li. Under- water ranker: Learn which is better and how to better. In AAAI, volume 37, pp. 702-709, 2023. 1, 8
11

---

Published as a conference paper at ICLR 2025
Hang Guo, Jinmin Li, Tao Dai, Zhihao Ouyang, Xudong Ren, and Sho Xia. Mambair: A simple baseline for image restoration with state-space model. In ECCV, 2024. 6, 8
Xiaojie Guo, Yu Li, and Haibin Ling. Lime: Low- light image enhancement via illumination map estimation. IEEE Trans. Image Process., 26(2):982-99) -93888999990009 2016. 8
Chumming He, Kai Li, Guoxia Xu, Jiangpeng Yan, Longxiang Tang, Yulun Zhang, Yaowei Wang, and Xiu Li. Heg-net: Unpaired medical image enhancement with high sigh-quality guidance. IEEE Transactions Neural Networks and Learning Systems, 2023a. 3,3
Chunming He, Kai Li, Guoxia Xu, Yulun Zhang, Runze Hu, Zhenhua Guo, and Xiu Li. Degradation- resistant unfolding network for heterogeneous image fusion. In ICCV, pp. 12611 1-1222,,, 2023b.
Chunming He, Kai Li, Yachao Zhang, Longxiang Tang, Yulun Zhang, Zhenhua Guo, and Xiu Li. Camouflaged object detection with feature decomposition and edge reconstruction. In CVPR, pp- 22046-22055 2023c. 5, 10
Chunming He, Kai Li, Yachao Zhang, Yulun Zhang, Zhenhua Guo, Xiu Li, Martin Danelljan, and Fisher Yu. Strategic preys make acute predators: Enhancing camouflaged object detectors by generating camouflaged objects. 2024a. 3
Chunming He. Yuqi Shen, Chengyu Fang, Fengyang Xiao, Longxiang Tang, Yulun Zhang, Wang- meng Zuo, Zhenhua Guo, and Xiu Li. Diffusion models in low-level vision: A survey, arXiv preprint arXiv:2406. 11138, 2024b. 8
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to a local nash equilibrium. NeurIPS, 30, 2017. 7,
Shih-Chia Huang, Fan-Chieh Cheng, and Yi-Sheng Chiu. Efficient contrast enhancement using adaptive gamma correction with weighting distribution. IEEE transactions on image processing, 123) 2012,
Md Jahidul Islam, Youya Xia, and Junaed Sattar. Fast underwater image enhancement for improved visual perception. IÉEE Robotics and Automation Letters, 5(2):3227-3234, 2020. 8
Yifan Jiang, Xinyu Gong, Ding Liu, Yu Cheng, Chen Fang, Xiaohui Shen, Jianchao Yang, Pan Zhou, and Zhangyang Wang. Enlightengan: Deep light enhancement without paired supervision. IEEE transactions on image processing, 30:2340-2349, 2021. 2, 6,
Yeying Jin, Wenhan Yang, and Robby Tan. Unsupervised night image enhancement: When layer decomposition meets light-effects suppression. In ECCV, pp. 404-421, 2022. 3
Yeying Jin, Beibei Lin, Wending Yan, Yuan Yuan, Wei Ye, and Robby Tan. Enhancing visibility in nighttime haze images using guided apsf and gradient adaptive convolution. In ACM MM, pp. 2446,2457, 2023. 3
Yeying Jin, Xin Li, Jiadong Wang, Yan Zhang, and Malu Zhang, Raindrop clarity: A dual-focused dataset for day and night raindrop removal. In European Conference on Computer Vision, pp- 1-17. Springer, 2024a.
Yeying Jin, Wei Ye, Wenhan Yang, Yuan Yuan, and Robby T Tan. Des3: Adaptive attention-driven self and soft shadow removal using vit similarity. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 2634-2642 2024b.
HOU Jinhui, Zhiyu Zhu, Junhui Hou, LIU Hui, Huanqiang Zeng, and Hui Yuan. Global structure- aware diffusion process for low-light image enhancement. In NeurIPS, 2023. 2, 3, 6, 8
Mingye Ju, Charles A Guo, Chuheng Chen, Jinshan Pan, Jinhui Tang, and Dacheng Tao. Sllen: Semantic-aware low-light image enhancement network. arXiv preprint 1111:2221.115571, 2022. 10
12

---

Published as a conference paper at ICLR 2025
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. NeurIPS, 14:21696-21777, 2021. 3
Variational diffusion models.
Diederik Kingma and Max Welling, Auto-encoding variational bayes.
arXiv preprint
arXiv:1312.61 14, 2013. 5
Edwin H Land. The retinex theory of color vision. Scientific american, 237(())108- 12t-1900--1,,,,, 1977.
Chulwoo Lee, Chul Lee, and Chang-Su Kim. Contrast enhancement based on layered difference representation of 2d histograms. IEEE Trans. Image Process. 22(12):5372-5384 2013. 8
Chongyi Li, Chunle Guo, Wenqi Ren, Runmin Cong, Junhui Hou, Sam Kwong, and Dacheng Tao. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 29:4376-4389 2019. 8
Chongyi Li, Saeed Anwar, Junhui Hou, Runmin Cong, Chunle Guo, and Wenqi Ren. Underwater image enhancement via medium caasmission-guidee multi-color space embedding. IEEE Trans- actions on Image Processing, 30:4985-5000, 2021.
Chongyi Li, Chun-Le Guo, Man Zhou Zhexin Liang Shangehen Zhou, Ruicheng Feng, and Chen Change Loy. Embeddingfourier for dltra-high-effinition low-light image enhancement. In ICLR, 2023. 8
Kai Li, Yulun Zhang, Kunpeng Li, and Yun Fu. Adversarial feature hallucination networks for few-shot learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13470-13479, 2020.
Mading Li, Jiaying Liu, Wenhan Yang, Xiaoyan Sun, and Zongming Guo. Structure-revcaling low- light image enhancement via robust retinex model. IEEE Transactions Image Processing, 27 (6):2828 2841, 2018.
Zhexin Liang Chongyi Li, Shangchen Zhou, Ruicheng Feng, and Chen Change Loy. Iterative prompt learning for unsupervised backlit image enhancement. In ICCV, pp. 8094-8103, 2023. 1, 2,8
Beibei Lin, Yeying Jin, Wending Yan, Wei Ye, Yuan Yuan, and Robby Tan. Nighthaze: Nighttime image dehazing via self-prion learning. arXiv preprint 2XXx:2403.07408 2024. 3
Risheng Liu, Long Ma, Jiaao Zhang, Xin Fan, and Zhongxuan Luo. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In CVPR, pp. 10561- 10570, 2021. 6,
Yunlong Liu, Tao Huang, Weisheng Dong, Fangfang Wu, Xin Li, and Guangming Shi. Low-light im- age enhancement with multi-stage residue quantization and brightness-awe aware attention. In ICCV, pp. 12140-12149, 2023. 6
Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot atten- tion. NeurlPS, 33: 33:11525-11536 2020. 5
Yuen Peng Loh and Chee Seng Chan. Getting to know low-light images with the exclusively dark dataset. Computer Vision and Image Understanding, 178:30-42, 2019. 10
Loshchilov. Stochastic gradient descent with warm restarts. ICLR, pp. 1-16, 2016.
Xiaoqian Lv, Shengping Zhang, Qinglin Liu, Haozhe Xie, Bineng Zhong, and Huiyu Zhou. Backl- itnet: A dataset and network for backlit image enhancement. Computer Vision and Image Under- standing, 218: 103403, 2022.
Yunqiu Lv, Jing Zhang, Yuchao Dai, Aixuan Li, Bowen Liu, Nick Barnes, and Deng-Ping Fan. Simultaneously localize, segment and rank the camouflaged objects. In CVPR, pp. 11591-11601, 2021. 10
Kede Ma, Kai Zeng, and Zhou Wang. Perceptual quality assessment for multi multi-exposure image fusion. IEFE Trans. Image Process., 24(11):3345-3356 2015. 8
13

---

Published as a conference paper at ICLR 2025
Anish Mittal, Rajiv Soundararajan, and Alan C Bovik. Making "completely blind" image quality analyzer. IEEE Signal Processing Lett., 20(3):209-212) 2012.
Anush Krishna Moorthy and Alan Conrad Bovik. A two-step framework for constructing blind image quality indices. IEEE Signal processing letters, 17(5):513-516, 2010. 7
Ankita Naik, Apurva Swarnakar, and Kartik Mittal. Shallow-uwnet: Compressed model for un-
derwater image enhancement (student abstract). In AAAI, volume 35, pp. 15853-15854, 2021.
Karen Panetta, Chen Gao, and Sos Agaian. measures. IEEE Journal of Oceanic Engineering, 41(3):41--55,, 2015.
underwater image quality
Lintao Peng, Chunli Zhu, and Liheng Bian. U-shape transformer for underwater image enhance- ment. IEEE Transactions on Image Processing, 2023. 8
Long Peng, Wenbo Li, Renjing Pei, Jingjing Ren, Jiaqi Xu, Yang Wang, Yang Cao, and Zheng-Jun Zha. Towards realistic data generation for real-world super-rcsolution. In ICLR, 2025.
Yifan Pu, Yizeng Han, Yulin Wang, Junlan Feng, Chao Deng, and Gao Huang. Fine- Fine-grained recog- nition with learnable semantic data augmentation. TIP, 2024.
Neng-Tue
Neng-Tsann Tsann Ueng and Louis L Scharf. The gamma transform: A local fieeuuryy analysis method. In ACSSC, volume 2, pp. 920-924. IEEE, 1995. 2
Ruixing Wang, Qing Zhang, Chi-Wing Fu, Xiaoyong Shen, Wei-Shi Zheng, and Jiaya Jia. Under- exposed photo enhancement using deep illumination estimation. In CVPR, pp. 6849-6857, 2019.
Shuhang Wang, Jin Zheng, Hai-Miao Hu, and Bo Li. Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans. Image Process., 22(())3538 -3448, 2013. 8
Yinglong Wang, Zhen Liu, Jianzhuang Liu, Songeen Xu, and Shuaicheng Liu. Low-light image enhancement with illumination-avee gamma correction and complete image modelling network. In ICCV, pp. 131 13128-13137, 2023.
Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. Uformer: A general u-shaped transformer for image restoration. In CVPR, pp. 17683-17693,
2022. 6, 9
Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu. Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv: 04560, 2018. 7
Wenhui Wu, Jian Weng, Pingping Zhang, Xu Wang, Wenhan Yang, and Jianmin Jiang. Uretinex- net: Retinex-based deep unfolding network for low-light image enhancement. In CVPR, pp. 5901-5910, 2022. 4, 8
Bin Xia, Yulun Zhang, Shiyin Wang, Yitong Wang, Xinglong Wu, Yapeng Tian, Wenming Yang, and Lue Van Gool. Diffir: Efficient diffusion model for image restoration. In ICCV. 2023. 6,
Xiaogang Xu, Ruixing Wang, Chi- hing Fu, and Jiaya Jia. Snr-aware low-light image enhancement. In CVPR, pp. 17714-17724, 2022. 6,
Xiaogang Xu, Ruixing Wang, and Jiangbo Lu. Low-light image enhancement via structure modeling and guidance. In CVPR, pp. 9893-9903, 2023. 6
Miao Yang and Arcot Sowmya. An underwater color image quality evaluation metric. IEEE Trans- actions on Image Processing. 24( 24(12):6062-6071 2015. 8
Wenhan Yang, Wenjing Wang, Haofeng Huang, Shiqi Wang, and Jiaying Liu. Sparse gradient regularized deep retinex network for robust low-light image enhancement. IEEE Transactions on Image Processing, 30:2072-2086, 2021.
14

---

Published as a conference paper at ICLR 2025
Xunpeng Yi, Han Xu, Hao Zhang, Linfeng Tang, and Jiayi Ma. Diff-retinex: Rethinking low-light image enhancement with a generative diffusion model. In ICCV, pp. 12302-12311 2023. 2, 3, 6,
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming- Hsuan Yang, and Ling Shao. Learning enriched features for real image restoration and enhance- ment. In ECCV, pp. 492-511. Springer, 2020. 6
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming- Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, pp. 5728-5739, 2022. 6, 8,
Fan Zhang, Yu Li, Shaodi You, and Ying Fu. Learning temporal consistency for low light video enhancement from single images. In CVPR, pp. 4967-4976, 2021a. 10
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In CVPR, pp. 586-595, 2018. 8
Yonghua Zhang, Xiaojie Guo, Jiayi Ma, Wei Liu, and Jiawan Zhang. Beyond brightening low-light images. Int. J. Comput. Vision 1 129:1013-1037, 2021b. 8
Zhao Zhang, Huan Zheng, Richang Hong, Mingliang Xu, Shuicheng Yan, and Meng Wang. Deep color consistent network for low-light image enhancement. In CVPR, pp. 1899-1908, 2022. 8
Naishan Zheng, Man Zhou, Yanmeng Dong, Xiangyu Rui, Jie Huang, Chongyi Li, and Feng Zhao. Empowering low-light image enhancer through customized learnable priors. In ICCV, pp. 12559- 12569, 2023.
Dewei Zhou, Zongxin Yang. and Yi Yang. Pyramid diffusion models for low-light image enhance- ment. arXiv preprint arXiv:2305. 10028, 2023a. 6
Jingehun Zhou, Qian Liu, Qiuping Jiang, Wenqi Ren, Kin-Man Lam, and Weishi Zhang. Under- water camera: Improving visual perception via adaptive dark pixel prior and color correction. International Journal of Computer Vision, pp. 1-19, 20236.
Shihao Zhou, Duosheng Chen, Jinshan Pan, Jinglei Shi, and Jufeng Yang. Adapt or perish: Adaptive sparse transformer with attentive feature refinement for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2952-2963,
June 2024. 6, 8
Chenyang Zhu, Kai Li, Yue Ma, Chumming He, and Li Xiu. Multibooth: Towards generating all your concepts in an image from text. arXiv preprint arXiv:2404.142333 2024a. 3
Chenyang Zhu, Kai Li, Yue Ma, Longxiang Tang, Chengyu Fang, Chubin Chen, Qifeng Chen, and Xiu Li. Instantswap: Fast customized concept swapping across sharp shape differences. arXiv preprint arXiv:2412.01197 2024b. 3
15

---

Published as a conference paper at ICLR 2025
SUPPLEMENTARY MATERIALS FOR RETI-DIFF: ILLUMINATION DEGRADATION IMAGE RESTORATION WITH RETINEX-BASED LATENT DIFFUSION MODEL
CONTENTS
Methodology
A.I Retinex-based Latent Diffusion Model
B Experiment
B.1 Ablation Study
B.2 Comparative Evaluation
C Discussions
D Limitations and Future Work
A METHODOLOGY
A.1 RETINEX-BASED LATENT DIFFUSION MODEL
In this section, we provide a detailed derivation for ŽL
Diffusion process. In the diffusion process, we first use the pretrained RPE to extract the reflectance prior ZL which treated as the starting point of the forward Markov process, i.e., Zz Zp. We then gradually add Gaussian noise to ZL by T iterations and each iteration can be defined as:
9 [ 1) N (z vi 3 .8'1)
(1)
where /. = 1, T. Z denotes the noisy prior at time step t, 3e is the predefined factor that controls the noise variance, and A is the Gaussian distribution. Following (Kingma & Welling, 2013), Eq. (1) can be simplified follows:
q [z Z N Va'z
where a²
and a² = - o².
Reverse process. In the reverse process, RLDM aims to extract the reflectance prior from pure Gaussian noise. Thus, RLDM samples Gaussian random noise map Z and then gradually denoise it run backward from zi, to Zi:
-- 1 zD)
(Z),
where mean (Z ) / (Zi,
vI-ar* E) and variance (a²)² - Br. denotes
I-a'
the noise in Zj, and is the only uncertain variable. Following previous practice (Xia et al., 2023), we employ a denoising network ca(+) to estimate 6. To operate in the latent space, we further
introduce another RPE module RPE(+) to extract the conditional reflectance vector V1 E BBC from the reflectance image Lı. of the IQQ image, i.c. VL RPE(down(LI.)) Therefore, the denoising

---

Published as a conference paper at ICLR 2025
Retnex (Oug) Ground Trahh Fig. S1: Effect of Retinex priors in extreme conditions, where the two rows share similarity in reflectance and illumination components, respectively.
Inpur
Ill..
Mlu GT
Ref
Ref.
wi peice
Ref
Ill.
Datasets Metrics
f2-norm & norm (Ours)
PSNR
27.26
27.53
L-v2-s
SSIM
0.949
0.951
PSNR
22.62
22.97
L-v2-r
SSIM
0.853
0.858
Fig. SI: Effect of é, -norm in Loss Functions.
Datriets[Metrics/ Å 0.1
(Ours)
10
0.1 X2 (Ours)
10
0.1 A
1 (Ours)
10
PSNR
27.15
27.53
27.33
27.08
27.53
27.33
27.26
27.53
27.35
L-22-s
SSIM
0.949
0.951
0.948
0.946
0.951
0.947
0.952
0.951
0.946
Lv2--
PSNR
22.86
22.97
22.82
22.36
22.97
22.76
22.33
22.97
22.16
SSIM
0.857
0.858
0.855
0.851
0.858
0.856
0.853
0.858
0.850
Fig. S2: Effect of e norm in Loss Functions.
network can be represented by 6g (Z) VL, f). By setting the variance to - a", we get
a v1 a
zi
Va (Zi
eo (Zi, VL, +v. a'e
(4)
where e' ~N.0
B EXPERIMENT
B. ABLATION STUDY
Following the practice in the manuscript, we select LOL-v2-real and LOL-v2-syn conduct ablation studies, where the two datasets are abbreviated as L-v2-r and L-v2-s.
Effeet of Retinex priors in extreme conditions. We investigate the potential of Retinex priors, i.e., ZR. and ZL, under extreme conditions where the reflectance or illumination components exhibit high similarity between low-quality and ground-truth images. As shown in Fig. S1, the extracted priors have diminished effect when the corresponding component shows the similarity between low-quality and ground-truth images. This because corresponding component undergoes min- imal degradation.
Effeet of 6, norm Loss Functions. We explore the effect fn-norm in loss functions. As shown in Table S1, Reti-Diff achieves better performance when using -nomm. Therefore, our loss functions select -nnor..
Parameter Analysis. Our Reti-Diff is optimized with multiple losses, which are balanced three hyperparameters, i.e., Å1. Ag. and Ag. analyze their impact, we vary one the parameters and fix others, and report the results in Table $2. Overall, find that the different coefficients in the tested range only slightly influence the final performance and d1, X2, and X3 obtain better results when they set to 1. So set those parameters each.
B.2 COMPARATIVE EVALUATION
Low-light Image Enhancement. As shown in Fig. S2, we provide visualization results. Our method can generate enhanced images with corrected illumination and enhanced texture, even in extremely challenging conditions.
2

---

Published as a conference paper at ICLR 2025
H!
Unitner
Raffome
CLIPAIIT
Grund
Fig. S2: Visual results on the low-light image enhancement task.
U-shaee Fig. S3: Visual results on the underwater image enhancement task.
Inpet
EsSAN
Net--Net
QUE
Retfomer
Ours
Groand Trch
Fig. S4: Visual results the backlit image enhancement task.
Underwater Image Enhancement. More qualitative analyses are presented in Fig. S3, illustrating our superiority in underwater color correction and fine texture details reconstruction.
Backlit Image Enhancement. Furthermore, visual comparison in Fig. S4 provides additional evidence of our superiority in detail reconstruction and color correction. All methods are trained by cropping the training data as 256 256 for fairness.
C DISCUSSIONS
Our Reti-Diff is the first LDM-based solution specifically tailored for the IDIR task, setting it apart from existing LDM-based methods applied in other tasks. To illustrate the distinctions, we compare with general enhancement method, DiffIR (Xia et al., 2023): (1) Motivation. Reti-Diff targets enhancing details and correcting degraded illumination. Thus, we enable RLDM to learn Retinex knowledge and generate Retinex priors from the low-quality input. We contend that relying solely on priors extracted from the RGB domain struggles to fully represent valuable texture details and cor- rect illumination cues, leading to suboptimal restoration performance. To verify this, we substitute RLDM for the LDM structure used in DiffIR. In LOL-v2-syne syn, observe that the PSNR rises from 24.76 to 26. and the SSIM increases from 0.921 to 0.933. (2) Implementation. Apart from proposing RI RLDM extract Retinex priors, we further modify the structure of RGformer to implic-
3

---

Published as a conference paper at ICLR 2025
itly model the Retinex theory at the feature level and introduce auxiliary decoder to reconstruct the decomposed Retinex components to the RGB domain.
D LIMITATIONS AND FUTURE WORK
Our Reti-Diff faces challenges in simultancously recovering illumination and restoring texture de- tails when the low-quality inputs suffer from severe illumination degradation. This issue, which per- sists across existing methods, remains unresolved. We attribute it to the loss of texture information during the illumination recovery process. To address this limitation in future research, extracting texture priors from other domains, such as the frequency domain (Xu et al., 2022; He al., 2019). These priors could complement the reflectance priors extracted from the RGB domain, thereby improving the preservation of critical texture features.
propose
Additionally, we aim combine our method with more cutting-edge deep priors, such as that from segment anything model (He et al., 2024), depth anything model (Chen et al., 2024a), or other large foundation model (Tang al., 2024), and extend our method wider applications, such as image super-resolution (Chen et al., 2023d; 2024b), deblurr (Chen et al., 2023e; Zhao al., 2023a), desnow (Chen et al., 2023c;b), dehaze (Fang et al., 2024a; Zhao et al., 2024), derain (Chen al., 2023a; Zhao et al., 2023b), conditional fusion (He et al., 2023; Xu et al., 2023; Ju et al., 2022). Combining our strategies with cutting-edge algorithms, such as spiking neural network (Wang et al., 2023; Fang al., 2024b) and mamba (Xiao et al., 2025; 2024b), is also expected to bring better performance.
Besides, enhanced methods are also expected to facilitate downstream tasks, such image seg- mentation (Xiao et al., 2024a; 2023) and detection (He et al., 2025; Pu et al., 2024; 2023). To achieve this, several strategies expected to be integrated into method, including generating Nownstream-ffrenddy data (Yuan et al., 2024a;b), designing specific augmentation strategies (Ma al., 2024a;b;c).
4

---

Published as a conference paper at ICLR 2025
REFERENCES
Sixiang Chen, Tian Ye, Jinbin Bai, Erkang Chen, Jun Shi, and Lei Zhu. Sparse sampling transformer with uncertainty-driver ranking for unified removal of raindrops and rain streaks. In ICCV, pp- 13106-131 13117, 2023a. 4
Sixiang Chen, Tian Ye, Yun Liu, Taodong Liao, Jingxia Jiang, Erkang Chen, and Peng Chen. Msp- former: Multi-scale projection transformer for single image desnowing. In ICASSP, pp. 1-5. IEEE, 2023b. 4
Sixiang Chen, Tian Ye, Chenghao Xue, Haoyu Chen, Yun Liu, Erkang Chen, and Lei Zhu. Uncertainty-driven dynamic degradation perceiving and background modeling for efficient sin- gle image desnowing. In ACM MM, pp. 4269-4280, 2023c.
Sixiang Chen, Tian Ye, Kai Zhang. Zhaohu Xing. Yunlong Lin, and Lei Zhu. Teaching tailored to talent: Adverse weather restoration via prompt pool and depth-anything constraint. ECCV, pp- 95-115. Springer, 2024a. 4
Zheng Chen, Yulun Zhang, Jinjin Gu, Xin Yuan, Linghe Kong, Guihai Chen, and Xiaokang Yang. Image super-resolution with text prompt diffusion. arXiv preprint arXiv:2311.14222. 2023d. 4
Zheng Chen, Yulun Zhang, Ding Liu, Bin Xia, Jinjin Gu, Linghe Kong, and Xin Yuan. Hicrarchical integration diffusion model for realistic image deblurring. In NeurIPS, 2023e. 4
Zheng Chen, Haotong Qin, Yong Guo, Xiongfei Su, Xin Yuan, Linghe Kong, and Yulun Zhang. Binarized diffusion model for image super-resolution. In NewrlPS, 2024b. 4
Chengyu Fang, Chumming He, Fengyang Xiao, Yulun Zhang, Longxiang Tang, Yuelin Zhang, Kai Li, and Xiu Li. Real-world image dehazing with coherence-basee label generator and cooperative unfolding network. NeurlPS, 2024a. 4
Yuetong Fang, Ziqing Wang, Lingfeng Zhang, Jiahang Cao, Honglei Chen, and Renjing Xu. Spiking wavelet transformer. In ECCV, pp. 19-37. Springer, 2024b. 4
Chunming He. Xiaobo Wang, Lizhen Deng. and Guoxia Xu. Image threshold segmentation based on glle histogram. In CPSCom, pp. 410-415. IEEE, 2019. 4
Chunming He, Kai Li, Guoxia Xu, Yulun Zhang, Runze Hu, Zhenhua Guo, and Xiu Li. Degradation- resistant unfolding network for heterogeneous image fusion. In ICCV, pp. 1261 1-1221,,, 2023.
Chunming He, Kai Li, Yachao Zhang, Guoxia Xu, Longxiang Tang, Yulun Zhang, Zhenhua Guo, and Xiu Li. Weakly-supervised concealed object segmentation with sam-based pseudo labeling and multi-scale feature grouping. NIPS, 36, 2024. 4
Chunming He, Rihan Zhang, Fengyang Xiao, Chenyu Fang, Longxiang Tang, Yulun Zhang, Linghe Kong, Deng-Ping Fan, Kai Li, and Sina Farsiu. Run: Reversible unfolding network for concealed object segmentation. arXiv preprint 28111:2501.1888,, 2025. 4
Mingye Ju, Chunming He, and Juping Liu. Ivf-net: An infrared and visible data fusion deep network for traffic object enhancement in intelligent transportation systems. IEEE Trans. Intell. Transp. Syst., 2022.
Diederik P Kingma and Max Welling. arXiv:1312.61 2013.
auto-encoding variational bayes.
arXiv preprint
Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose-guided text-to-video generation using pose-free videos. In AAAI, volume 38, pp. 4117-4125, 2024a. 4
Yue Ma, Yingqing He, Hongfa Wang. Andong Wang, Chenyang Qi, Chengfei Cai, Xiu Li, Zhifeng Li, Heung Yeung Shum, Wei Liu, et al. Follow-your Open-domain regional image anima- tion via short prompts. arXiv preprint arXiv:2403.nm 08268, 2024b. 4
5

---

Published as a conference paper at ICLR 2025
Yue Ma, Hongyu Liu, Hongfa Wang. Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei controlllll and expressive freestyle portrait animation. In SIGGRAPH Asia, pp. 1-12, 2024c. 4
Cai, Heung- Yeung Shum, Wei Liu, et al. follow-your-ens
Yifan Pu, Yiru Wang, Zhuofan Xia, Yizeng Han, Yulin Wang, Weihao Gan, Zidong Wang, Shiji Song, and Gao Huang. Adaptive rotated convolution for rotated object detection. In ICCV, 2023.
Yifan Pu, Yizeng Han, Yulin Wang, Junlan Feng, Chao Deng, and Gao Huang. Fine- grained recog- nition with learnable semantic data augmentation. TIP. 2024.
Longxiang Tang, Zhuotao Tian, Kai Li, Chunming He, Hantao Zhou, Hengshuang Zhao, Xiu Li, and Jiaya Jia. Mind the interference: Retaining pre-trained knowledge in parameter efficient continual learning of vision-language models. In European Conference Computer Vision, pp. 346-365. Springer, 2024. 4
Ziqing Wang. Yuetong Fang. Jiahang Cao, Qiang Zhang, Zhongrui Wang, and Renjing Xu. Masked spiking transformer. In ICCV, pp. 1761-1771, 2023. 4
Bin Xia, Yulun Zhang, Shiyin Wang, Yitong Wang, Xinglong Wu, Yapeng Tian, Wenming Yang, and Luc Van Gool. Diffir: Efficient diffusion model for image restoration. In ICCV, 2023. 1,3
Fengyang Xiao, Pan Zhang, Chumming He, Runze Hu, and Yutao Liu. Concealed object segmenta- tion with hierarchical coherence modeling. In CAAI, pp. 16-27. Springer, 2023,
Fengyang Xiao, Sujie Hu, Yuqi Shen, Chengyu Fang, Jinfa Huang, Longxiang Tang, Ziyun Yang, Xiu Li, and Chumming He. A survey of camouflaged object detection and beyond. CAAI AIR, 3, 2024a. 4
Yicheng Xiao, Zhuoyan Luo, Yong Liu, Yue Ma, Hengwei Bian, Yatai Ji, Yujiu Yang, and Xiu Li. Bridging the gap: A unified video comprehension framework for moment retrieval and high- light detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18709-18719, 2024b. 4
Yicheng Xiao, Lin Song, Jiangshan Wang, Siyu Song, Yixiao Ge, Xiu Li, Ying Shan, et al. Mam- batree: Tree topology is all you need in state space model. Advances Neural Information Processing Systems, 37:75329-75354 2025. 4
Guoxia Xu, Chunming He, Hao Wang, Hu Zhu, and Weiping Ding. Dm-fusion: Deep model-driven network for heterogeneous image fusion. IEEE Transactions on Neural Networks and Learning Systems, 2023. 4
Lei Xu, Hui Wu, Chunming He, Jun Wang. Changqing Zhang, Feiping Nie, and Lei Chen. Multi- modal sequence learning for alzheimer's disease progression prediction with incomplete variable- length longitudinal data. MIA, 82: 102643, 2022. 4
Shenghai Yuan, Jinfa Huang, Yujun Shi, Yongqi Xu, Ruijic Zhu, Bin Lin, Xinhua Cheng, Li Yuan, and Jiebo Luo. Magictime: Time-lapse video generation models metamorphic simulators. arXiv preprint arXiv:2404.05014 2024a.
Shenghai Yuan, Jinfa Huang, Yongqi Xu, Yaoyang Liu, Shaofeng Zhang, Yujun Shi, Ruijie Zhu, Xinhua Cheng, Jiebo Luo, and Li Yuan. Chronomagie-bench: A benchmark for metamorphic evaluation of -tpt---- video generation. NeurlPS, 2024b. 4
Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Shuang Xu, Zudi Lin, Radu Timofte, and Lue Van Gool. Cddfuse: Correlation-driven dual-branch feature decomposition for multi- modality image fusion. In CVPR, pp. 5906-5916 June 2023a. 4
Zixiang Zhao, Haowen Bai, Yuanzhi Zhu, Jiangshe Zhang, Shuang Xu, Yulun Zhang, Kai Zhang, Deyu Meng, Radu Timofte, and Luc Van Gool. Ddfi: Denoising diffiusonn model for multi- modality image fusion. In ICCV, pp. 8082-8093, October 2023b. 4
Zixiang Zhao, Lilun Deng, Haowen Bai, Yukun Cui, Zhipeng Zhang, Yulun Zhang, Haotong Qin, Dongdong Chen, Jiangshe Zhang, Peng Wang, and Luc Van Gool. Image fusion via vision- language model. In ICMI., 2024. 4


Reviews:
[{"review_id": "CAdAWaToQ6", "reviewer": "Reviewer_p8Wx", "summary": "The paper proposes to use diffusion models in conjunction with a transformer model to perform image restoration of poorly lit images. The proposed method is evaluated in 4 different settings and shown the merits.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ The paper looks at an important problem of illumination restoration in images which is quite relevant for downstream tasks.", "weaknesses": "- The use of intrinsic image decomposition through Retinex is a dated idea. There are several better methods available that have neither been explored nor used. https://www.elenagarces.es/projects/SurveyIntrinsicImages/ \n\n- The work lacks novelty as it uses diffusion to decompose the image and a transformer to reconstruct it. The method has hardly any contribution. \n\n- Though the evaluation of the method is done for several conditions, the process is not built correctly with any conviction, which is a downer. \n\n- The method uses a couple of downstream applications to demonstrate the merits: detection and segmentation. The tasks are very easy to generalize, given the power of deep networks without illumination enhancement. The work could not convince the reader about the merit of this approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jYqzIvYlkY", "reviewer": "Reviewer_uVsX", "summary": "The paper introduces Reti-Diff, a novel latent diffusion model-based framework tailored for Illumination Degradation Image Restoration (IDIR). It leverages Retinex theory to improve image quality by using two primary components: the Retinex-based Latent Diffusion Model (RLDM) and the Retinex-guided Transformer (RGformer).", "questions": "1. How does the proposed method handle edge cases such as extreme noise or heavy color distortions, which may not follow typical low-light degradation patterns?\n\n2. Is there a recommended strategy for tuning the noise variance parameters in the latent diffusion process for optimal performance across varied image qualities?\n\n3. How does Reti-Diff perform in scenarios with extreme lighting conditions, such as overexposed or underexposed images, which may challenge the reliability of Retinex-based priors?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper combined the Retinex model and data-driven methods since the first effort of RetinexNet. \n2. Experiments are sufficient.\n3. Code is a plus.", "weaknesses": "1. For the low-light enhancement task, I want to see more results on LIME, NPE, MEF, DICM and VV. Since these datasets do not have ground truth, then is more fair to justify the effectiveness. Any visual results?\n\n2. Missing citations of real-world low-light enhancement methods,\n[1] Unsupervised Night Image Enhancement: When Layer Decomposition Meets Light-Effects Suppression\n[2] Enhancing Visibility in Nighttime Haze Images Using Guided APSF and Gradient Adaptive Convolution\n \n3. LDM is slow in inference. \nThe paper lacks a discussion on the computational complexity and runtime efficiency of the proposed model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C7GRwLHHew", "reviewer": "Reviewer_NsRN", "summary": "This paper introduces the Retinex-based LDM solution, Reti-Diff, for illumination degradation image restoration (IDIR) tasks, aiming to generate visual fidelity results while decreasing computational burdens. In specific, Reti-Diff proposes RLDM to acquire Retinex priors and then leverage RGformer to guide the image restoration process with the extracted priors, resulting in the production of refined images with consistent content and robustness to handle complex degradation scenarios. Extensive experiments demonstrate that Reti-Diff outperforms existing methods on three IDIR tasks, as well as downstream applications.", "questions": "Can the authors analyze the reason for the failure cases? Will this be because of the failure of one of the Retinex priors or because of the failure of RGformer to integrate the priors?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper proposes RLDM to extract Retinex priors, and RGformer to integrate the priors, ensuring robustness and generalization in complex illumination degradation scenarios.\n2. This paper is well organized.\n3. Experiments on four IDIR tasks verify the superiority, efficiency, and generalizability of the method and demonstrate that Retinex priors can serve as a plug-and-play strategy to improve the quality of existing methods.", "weaknesses": "1. More related works are expected to be discussed, for example, low-light image enhancement via clip-fourier guided wavelet diffusion[1]. \n2. Why do the authors choose to use a cross-attention mechanism to model the Retinex theory?\n3. The authors are encouraged to highlight the motivation for extracting Retinex priors and explain why using a diffusion model to extract the priors. \n4. Additionally, in the experiment, the authors are expected to verify if the Retinex priors perform better than the sole RGB prior and if the diffusion model can serve as a better predictor than the common network structure.\n5. In Phase 2, the authors first train the RLDM before conducting joint training with RGFormer. In the ablation study, the authors provide results without the joint training stage, but they do not compare the results of removing the independent RLDM training stage. Would it yield better results if RLDM, RPE, and RGFormer were trained directly together?\n6. Some typo errors are expected to be fixed, for example, ZR and ZL in Line 90 in the Supp.\n\n[1] Xue M, He J, He Y, et al. Low-light image enhancement via clip-fourier guided wavelet diffusion[J]. arXiv preprint arXiv:2401.03788, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xmxuZEIi49", "reviewer": "Reviewer_72tr", "summary": "This paper focuses on designing a latent diffusion model that utilizes the Retinex priors to address the issue of illumination degradation. To be detailed, the authors developed a module for estimating the Retinex priors from the low-quality and high-quality images and a Transformer backbone for the image enhancement. These components work in synergy to restore images. The authors provide extensive experiments to demonstrate the effectiveness and advancement of their method.", "questions": "1. What’s the motivation of using the latent diffusion model to generate the Retinex priors? What are its advantages? Would it be more efficient and reliable to use a single Transformer or CNN network for it?\n2. DiffIR [1] achieves image restoration using a similar prior. What would be the difference if the entire image enhancement process were simply treated as the prior, compared to using the Retinex-based model as the priors? Demonstrating the necessity of the latent Retinex priors would be more valuable.\n3. About the Retinex priors, how does it perform in terms of prior feature errors when exchanging features between the ground truth and low-light images? In addition to the qualitative and quantitative evaluations of the enhancement results, it would be beneficial to conduct an assessment at the prior level as well.\n4. It is recommended to perform a convergence analysis of using the latent Retinex priors. For example, showcasing the changes in loss or evaluation metrics on the training and validation sets would be beneficial.\n5. Given the introduction of the Retinex priors, does the proposed method heavily depend on the reliability of Retinex decomposition? Since this is an ill-posed problem, it would be helpful to provide an explanation or at least a few qualitative results of the Retinex decomposition to support this claim.\n6. How does the method perform in extreme conditions, specifically when the Retinex decomposition fails? (Extremely bright light and strong shadows) \n\nAlthough there are some concerns, they do not detract from the fact that this is a relatively well-researched paper. I am inclined to provide a positive opinion in this stage.\n\n[1] Xia, Bin, et al. \"Diffir: Efficient diffusion model for image restoration.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The coupling of latent diffusion model with the Retinex priors and Transformer is intriguing. And exploring the robustness of network with the Retinex theory in illumination degradation scenarios is meaningful.\n2. The proposed method, compared to existing diffusion models in image enhancement, demonstrates relatively high computational efficiency and lower resource consumption while maintaining strong performance.\n3. The experimental validation in the paper is thorough, particularly regarding downstream tasks and evaluations across multiple datasets.\n4. The paper is well-organized, clearly written, and easy to understand.", "weaknesses": "1. The paper utilizes diffusion models to get the latent Retinex priors, but further analysis is needed to explore the relationship between diffusion models and the designed latent Retinex priors. (insights)\n2. The mechanisms by which the Retinex priors operate need to be clarified. Ensuring the consistency of the Retinex theory in the latent space is a question worthy of investigation. Further deepening the results-oriented experimental conclusions in the mechanistic level would improve the quality of study.\n3. The design of the method results in a significant reliance on the modeling of Retinex theory. However, there seems to be little discussion on the reliability of Retinex decomposition.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)