Interpretable Unsupervised Joint Denoising and Enhancement for Real-World low-light Scenarios

Authors: Li Huaqiu, HuXiaowan, Haoqian Wang
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: unsupervised learning, low-light image enhancement, image denoising

Abstract:
Real-world low-light images often suffer from complex degradations such as local overexposure, low brightness, noise, and uneven illumination. Supervised methods tend to overfit to specific scenarios, while unsupervised methods, though better at generalization, struggle to model these degradations due to the lack of reference images. To address this issue, we propose an interpretable, zero-reference joint denoising and low-light enhancement framework tailored for real-world scenarios. Our method derives a training strategy based on paired sub-images with varying illumination and noise levels, grounded in physical imaging principles and retinex theory. Additionally, we leverage the Discrete Cosine Transform (DCT) to perform frequency domain decomposition in the sRGB space, and introduce an implicit-guided hybrid representation strategy that effectively separates intricate compounded degradations. In the backbone network design, we develop retinal decomposition network guided by implicit degradation representation mechanisms. Extensive experiments demonstrate the superiority of our method. Code will be available at https://github.com/huaqlili/unsupervised-light-enhance-ICLR2025.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 8, 6

Reviews and discussion:
<thread>
creation date:
1730514857284

thread_type:
official_review

<thread_object>
creator: reviewer_h1de

<content>
summary:
This paper develops a zero-reference joint denoising and low-light enhancement framework designed for real-world applications. 
The framework incorporates a self-supervised optimization strategy and physical priors to effectively capture complex degradations. 
Grounded in Retinex theory, the optimization process generates paired low-light and enhanced images by dynamically adjusting brightness, thereby establishing mutual self-supervision. 
A Discrete Cosine Transform (DCT) is employed to extract degradation representations across varying levels, facilitating degradation decomposition and removal within the frequency domain.

strengths:
1. This paper presents a zero-reference joint denoising and low-light enhancement framework that operates without the need for paired or unpaired images, a crucial feature for real-world applications.

2. The framework introduces a novel neighboring pixel masking strategy and creates paired images with varying brightness levels to enable self-supervised image denoising and decomposition.

3. Experiments on real-world datasets show that the proposed framework outperforms existing zero-reference low-light image enhancement methods.

weaknesses:
1. The reviewer notes that the proposed neighboring pixel masking strategy appears to draw from methods like Neighbor2Neighbor [1] and MAE [2], which are known to sometimes produce over-smoothed results. The authors are encouraged to clarify the differences between their approach and these methods, highlighting any improvements that address over-smoothing issues.

2. Conducting ablation studies on the size of the neighboring mask would be beneficial to assess its impact on preserving texture details. Such analysis could provide a clearer understanding of the mask’s effectiveness in enhancing visual quality.

3. The exposure loss function relies on a pre-defined exposure level, similar to the approach in Zero-DCE. However, in scenarios with non-uniform or overexposure, the results may tend toward overexposure and color distortion. The authors are encouraged to consider integrating an adaptive exposure adjustment strategy in future work to address these limitations.

4. While the authors claim that the proposed framework can both denoise and enhance images, the LOL dataset—characterized by slight noise degradation—may not fully validate its denoising capabilities. It would be beneficial to conduct experiments on low-light image enhancement datasets with significant noise levels, such as LSRW or other image denoising datasets, to substantiate the framework's performance in more challenging conditions.

[1] Huang T, Li S, Jia X, et al. Neighbor2neighbor: Self-supervised denoising from single noisy images[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 14781-14790.

[2] He K, Chen X, Xie S, et al. Masked autoencoders are scalable vision learners[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022: 16000-16009.

questions:
Please refer to Weaknesses
</content>
</thread_object>

</thread>
<thread>
creation date:
1730599218583

thread_type:
official_review

<thread_object>
creator: reviewer_wcsy

<content>
summary:
This manuscript introduces an interpretable, unsupervised framework designed to enhance and denoise real-world low-light images without the need for paired training data. The proposed method leverages physical imaging principles and retinex theory to derive a training strategy based on paired sub-images with varying illumination and noise levels. The framework employs the Discrete Cosine Transform (DCT) to perform frequency domain decomposition in the sRGB space.

strengths:
1. It is technically sound to incorporate the neighbor2neighbor denoiser with the task of unsupervised LLIE 
2. The authors provided code in the supplementary materials.

weaknesses:
1. In Equation (8), the term P1 is introduced without prior explanation, and it appears that the FIcoder is omitted in the process. Additionally, while the authors describe P1 in Equation (15), the description is still quite vague. Could the authors please clarify the shape of P1, how it is learned, what constraints are used, and whether it can be visualized?

2. If Equations (9) and (13) are identical to the standard DCT/IDCT transformations, there is no need to present them in the paper.

3. In Equation (14), the L_{reg} term is not explained. Could the authors please provide an explanation for this term?

4. In Equation (15), what is the difference between the first and third terms? Additionally, the authors are requested to show several decomposed illumination and reflection maps to demonstrate that the constraints in this equation are sufficient to learn reasonable Retinex decomposition results.

5. On Line 150, the reference year for noise2noise is incorrect; the paper was published in 2018, not 1803.

6. Figures 1, 2, 3, and 5 are not explained in the main text (at least I could not find them), while Figure 4 is explained but not correctly referenced.

7. In Figure 6, I do not see the advantage of the author's method over the SCI method, and the authors avoid explaining this point in the paper.

8. In Figure 7, the author's method still leaves noticeable noise, which I did not see in the Neighbor2Neighbor comparative experiment. Is there an issue with the author's training process?

9. The authors should introduce qualitative comparisons on an independent test set to demonstrate the superiority of their generalization performance. I personally believe this is a key experiment to distinguish whether a low-light enhancement model is valuable, and it should also be an advantage of unsupervised methods. Recommended datasets include LIME, NPE, MEF, DICM, and VV, as used in the Retinexformer.

questions:
See the weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730623406020

thread_type:
official_review

<thread_object>
creator: reviewer_v7se

<content>
summary:
In this article, to address the complex degradation issues of images in low-light scenarios, the author proposes a training strategy based on physical imaging principles and retinal theory. Specifically, first, the author derives a novel training strategy from paired images under varying lighting conditions and noise levels; second, the author introduces an implicit guidance hybrid representation strategy through DCT transformation, effectively separating complex composite degradation; finally, the author develops a retinal decomposition network based on the implicit degradation representation mechanism. The article demonstrates the effectiveness and reliability of the proposed method through extensive experiments.

strengths:
1. The author proposes a novel retinal decomposition network based on an implicit degradation representation mechanism in the article, which shows certain effectiveness in image enhancement for low-light scenarios;
2. The language of the article adheres to English writing standards and is fluent;
3. The article provides mathematical proofs for each module of the network, making the logic of the article strong;

weaknesses:
1. On page 4, lines 175-180, the author proposes the use of Discrete Cosine Transform (DCT) to separate different frequencies of images (including chromatic, semantic information, edge contours, and noise intensity). What is the theoretical basis for this approach? Or is it more of an empirical practice? If so, would the performance of the model be affected if different methods were used to decompose the feature maps after Discrete Cosine Transform?
2. On page 6, lines 322-333, the author presents the loss function of the model. However, during the experiments, the author seems to have not conducted an in-depth exploration between the loss function and the performance of the model. If such research were included, it would make the argument of the paper more complete.

questions:
1. On page 4, lines 175-180, the author proposes the use of Discrete Cosine Transform (DCT) to separate different frequencies of images (including chromatic, semantic information, edge contours, and noise intensity). What is the theoretical basis for this approach? Or is it more of an empirical practice? If so, would the performance of the model be affected if different methods were used to decompose the feature maps after Discrete Cosine Transform?
2. On page 6, lines 322-333, the author presents the loss function of the model. However, during the experiments, the author seems to have not conducted an in-depth exploration between the loss function and the performance of the model. If such research were included, it would make the argument of the paper more complete.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730645972263

thread_type:
official_review

<thread_object>
creator: reviewer_sctr

<content>
summary:
This paper proposes an interpretable unsupervised joint denoising and enhancement method suitable for real-world low-light scenes. Based on the physical imaging principles and Retinex theory, discrete cosine transform (DCT) is used for frequency domain decomposition, and an implicitly guided hybrid representation strategy is introduced to effectively separate complex compound degradations, thereby realizing the detection and optimization of complex degradation problems caused by low-light conditions. The authors conduct experiments to verify the effectiveness of the proposed method.

strengths:
1. The authors explore the difficulties traditional methods face in dealing with complex degradation issues such as noise, brightness, and contrast, and propose solutions that significantly outperform existing technologies.
2. Technically, the authors introduce a spatial frequency domain filtering module that uses discrete cosine transformfor explicit multi-band separation, which facilitates the decomposition of enhanced images into illumination and reflectance maps.
3. Experiments depict the effectiveness of the proposed method on multiple datasets.

weaknesses:
1. Some of the latest methods are not compared, such as RetinexFormer， and Rerinex-Diffusion. Besides, the experiments do not provide statistical indicators such as standard deviation or confidence interval in Table 1 and Table 2, which makes the reliability and stability of the results unclear. Moreover, the generalization ability of the proposed method is not explored, for example, when training the method in low-light environment and testing it one other exposure scenes.

2.  Although authors claim the proposed method is new, quantities of methods have explored introducing frequency-based techniques into image enhancement including some components-decomposition-based methods (such as [1])， what are the special characteristics of the proposed method that introducing the techniques that used above into retinex-based mechanism?

[1] Unveiling Advanced Frequency Disentanglement Paradigm for Low-Light Image Enhancement, ECCV 2024.

3.  There are too few visual results, for example, the visualization results of more real-world scenes (other low-light datasets or other exposure scenes) can be provided. Moreover, the detailed mechanism of why the proposed method works is not presented. How it has better performance than previous methods?

questions:
Please see the Weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730745742036

thread_type:
official_review

<thread_object>
creator: reviewer_dq7d

<content>
summary:
This paper introduces an interpretable, unsupervised framework for joint denoising and low-light enhancement of images, focusing on real-world scenarios. 

The method employs physical imaging principles and Retinex theory for decomposing images into illumination and reflection components.

strengths:
1. The paper combined the Retinex model and data-driven methods since the first effort of RetinexNet. 
2. Experiments are sufficient.
3. Frequency domain decomposition is well-used. 
4. The unsupervised low-light enhancement method is a promising direction. 
5.  The self-supervised denoising method based on neighboring pixel masking is well-aligned with the challenges of handling zero-reference images.

weaknesses:
1. For low-light enhancement task, I want to see more results on LIME, NPE, MEF, DICM and VV. Since these datasets do not have ground truth, then is more fair to justify the effectiveness.

2. Missing citations of real-world low-light enhancement methods,
[1] Enhancing Visibility in Nighttime Haze Images Using Guided APSF and Gradient Adaptive Convolution

3. The paper lacks a discussion on the computational complexity and runtime efficiency of the proposed model.

questions:
1. How does the proposed method handle edge cases such as extreme noise or heavy color distortions, which may not follow typical low-light degradation patterns?

2. Given the success of the DCT-based frequency decomposition, have the authors considered combining this with other frequency-domain transforms (e.g., wavelets) to enhance robustness across diverse types of degradation?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732169251892

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Reply

comment:
**Dear Committee Members and Reviewers,**  

We are pleased to inform you that we have replied to all the reviewers' comments. and incorporated the following additional experiments into our revised submission:  

1. **Generalization tests** on unsupervised datasets **LIME, NPE, MEF, DICM, and VV** (Supplementary Materials: **Figure 2, Tables 2 and 3**).  
2. **Comparative experiments** on the **LSRW** dataset (Supplementary Materials: **Figure 5 and Table 8**).  
3. **Generalization tests** on overexposed datasets (Supplementary Materials: **Figure 3**).  
4. **Comparison of computational complexity and runtime efficiency** (Supplementary Materials: **Table 6**).  
5. **Ablation study on the loss function** (Supplementary Materials: **Table 4**).  
6. **Ablation study on different masking strategies and their ratios** (Supplementary Materials: **Table 7**).  

The supplementary materials, detailed in the **sup.pdf** file, contain these additional results. Due to space constraints in the main manuscript and the associated effort required for reformatting, some of the critical findings have been included in the supplementary materials for your reference. We will incorporate these results into the main text in subsequent stages as needed.  

We hope these additions address your concerns and provide further clarity regarding our work. Thank you for your time and constructive feedback. We look forward to your response and remain open to discussing any additional questions or suggestions.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes an interpretable unsupervised joint denoising and enhancement for real-world low-light scenarios. Experimental results show the effectiveness of the proposed method. All reviewers appreciate the contributions of the paper. However, they also raise lots of concerns including more evaluations results, discussions on the computational complexity and runtime efficiency of the proposed model, comparisons with state-of-the-art methods, e.g., RetinexFormer, and differences from eighbor2Neighbor and MAE. 

The authors solve the concerns of reviewers. Based on the recommendations of reviewers, the paper is accepted.