Continuous Exposure Learning for Low-light Image Enhancement using Neural ODEs

Authors: Donggoo Jung, Daehyun Kim, Tae Hyun Kim
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: NeuralODE, Low-light Enhancement

Abstract:
Low-light image enhancement poses a significant challenge due to the limited information captured by image sensors in low-light environments. 
  Despite recent improvements in deep learning models, the lack of paired training datasets remains a significant obstacle. 
  Therefore, unsupervised methods have emerged as a promising solution. 
  In this work, we focus on the strength of curve-adjustment-based approaches to tackle unsupervised methods. 
  The majority of existing unsupervised curve-adjustment approaches iteratively estimate higher order curve parameters to enhance the exposure of images while efficiently preserving the details of the images. 
  However, the convergence of the enhancement procedure cannot be guaranteed, leading to sensitivity to the number of iterations and limited performance.
  To address this problem, we consider the iterative curve-adjustment update process as a dynamic system and formulate it as a Neural Ordinary Differential Equations (NODE) for the first time, and this allows us to learn a continuous dynamics of the latent image. 
  The strategy of utilizing NODE to leverage continuous dynamics in iterative methods enhances unsupervised learning and aids in achieving better convergence compared to discrete-space approaches. Consequently, we achieve state-of-the-art performance in unsupervised low-light image enhancement across various benchmark datasets.

Decision: accept (spotlight)
Reviewer scores: 6, 8, 8

Reviews and discussion:
<thread>
creation date:
1730515131742

thread_type:
official_review

<thread_object>
creator: reviewer_nmzg

<content>
summary:
This manuscript introduces a CLODE method that enhances low-light images by framing exposure adjustment as a Neural Ordinary Differential Equation (NODE) problem.  It improves on traditional curve-adjustment techniques by automatically adjusting exposure without manual input and converges to an optimal exposure using adaptive ODE solvers. CLODE also includes noise removal and curve parameter estimation, and users can control exposure levels by adjusting integration intervals.
===================================================================

The authors have raised many of my concerns during the rebuttal phase. I am now ready to increase the rating.

strengths:
Strengths of the CLODE method:

1. CLODE’s use of Neural ODEs allows it to dynamically adjust exposure levels based on image-specific needs.
2. The approach is really interesting. Mainly the noise removal block is a very good idea for this type of tasks.
3. Ablation study is really exhaustive.
4. Results are evaluated well.
5. Manuscript is well written and organized. Very easy to follow.
6. Technical content is really well written with proper citations.

Good Paper!!!

weaknesses:
Weaknesses:

1. Novelty is somewhat limited.
2. Comparisons with recent-most SOTA methods should be included.
3. Flowcharts should be improved like Figure 2.
4. Conclusion should be improved although I can understand there is a space limitation. Please make other parts smaller and add some key findings in conclusions.
5. Some more recent references from 2024 is required.

By framing the enhancement process as a Neural ODE, CLODE relies on numerical approximation methods (e.g., dopri5) that inherently carry approximation errors. This can provide solvers with adaptive step sizes but they cannot guarantee analytical precision or stability across all image conditions, particularly in complex cases with high dynamic ranges. This approximation could lead to unpredictable or suboptimal results, especially when the integration steps or the neural network's learned parameters are insufficiently fine-tuned for certain low-light conditions. Can the authors provide some low-light images from real-world dark conditions and demonstrate the method.

questions:
1. Adaptive ODE solvers increase computation time and memory usage. Any idea regarding the number of pixel operations taking place here?

2. The curve parameter estimation module might not generalize well to diverse lighting conditions. Can you provide some generalization test regarding this.

3. Manual adjustment of exposure intervals introduces subjectivity. An ablation can give us more insights.

4. NODE-based adjustments may struggle with high-dynamic-range images, risking instability and suboptimal exposure adjustments. Can you provide the separate loss curves for better understanding of the training dynamics.
==========================================================================
</content>
</thread_object>

</thread>
<thread>
creation date:
1730623614387

thread_type:
official_review

<thread_object>
creator: reviewer_asgg

<content>
summary:
In this article, the author discovers that the traditional curve-based iterative methods cannot guarantee the convergence of the enhancement process, leading to sensitivity to the number of iterations and limited performance. The author regards the iterative curve adjustment process as a dynamic system and, for the first time, formulates it as a neural ordinary differential equation (NODE) to learn the continuous dynamics of underlying images. Specifically, this method leverages NODE to harness the continuous dynamics within iterative methods, achieving better convergence than discrete space methods. Finally, the article demonstrates superior performance over current unsupervised low-light image enhancement methods on various benchmark datasets.

strengths:
1. In this article, for the first time, the iterative curve adjustment update process is regarded as a dynamic system and formulated as a neural ordinary differential equation (NODE) to learn the continuous dynamics of underlying images. This approach offers an alternative perspective for addressing low-light issues；
2. The article employs a substantial amount of mathematical language to demonstrate how to transform the iterative curve adjustment update into a continuous process, which enhances the logical strength of the paper；
3. The language of the article is quite fluent and adheres to English writing standards；

weaknesses:
1. Since this article addresses issues in low-light scenarios, presenting more objective results in complex scenarios would enhance the persuasiveness of the article.

questions:
1. Since this article addresses issues in low-light scenarios, presenting more objective results in complex scenarios would enhance the persuasiveness of the article.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730640147702

thread_type:
official_review

<thread_object>
creator: reviewer_b2uw

<content>
summary:
This paper introduces a novel approach for enhancing low-light images by formulating the problem as a Neural Ordinary Differential Equations (NODE) problem. The authors propose CLODE, a dynamic system that leverages continuous dynamics of latent images to improve exposure levels in images. The method is unsupervised, which is particularly useful given the difficulty of obtaining paired low-light and well-exposed images for supervised learning.

strengths:
1. The paper presents a creative application of NODEs to the problem of low-light image enhancement, which is a contribution to the field.

2. Addressing the challenge of lacking paired datasets, the unsupervised nature of CLODE makes it highly applicable to real-world scenarios where ground truth data is scarce.

3. The paper provides a thorough explanation of the CLODE model, including its theoretical foundations and implementation details.

4. The method achieves competitive results across multiple benchmarks, which is a strong point of its potential impact.

weaknesses:
1. The paper does not explicitly address the computational cost of solving NODEs compared to other methods, which is an important consideration for practical applications. Authors should report this results compared with other similar methods.

2. The paper does not discuss the potential for overfitting, especially since the model is tailored to low-light images, which may have unique characteristics.

3. In my opinion, the author's comparative experiment is still not sufficient. It should be compared on more real benchmarks to highlight the superiority of the method. The current comparison seems to be insufficient, including the following:

[1] Contrast enhancement based on layered difference representation.

[2] Perceptual quality assessment for multi-exposure image fusion.

[3] Structure-revealing low-light image enhancement via robust retinex model.

[4] Naturalness preserved enhancement algorithm for non-uniform illumination images.

[5] On the evaluation of illumination compensation algorithms.

4. There are many different paradigms for unsupervised methods. Compared with this type of method, is the author's method more scalable? If it can be improved on other methods, I think this article will have higher value.

In general, I will make further adjustments based on the author's rebuttal.

questions:
Please refer to weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731968418176

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
**To the reviewers,**

Thank you for taking the time to review our paper and provide valuable feedback. Your insightful comments and suggestions have been instrumental in improving our proposed method.

In response to your feedback, we have updated our paper and reuploaded the revised version. The key changes are as follows:

1. **Figure 2 (Flowchart):** We have enhanced the overall flowchart by incorporating detailed network information previously provided in the Appendix. **(NMzG W3)**
2. **Sec. 6 (Conclusion):** We refined the conclusion to articulate our contributions more clearly and precisely. **(NMzG W4)**
3. **Sec. A.7 (Training Dynamics):** We added a new section on training dynamics, including details on loss function behavior (**Figure 19**) and performance metrics (**Figure 20 and 21**). This addition aims to provide a better understanding of the training process. **(NMzG Q4)**

We hope these updates address your concerns and further enhance the quality of our paper. Thank you once again for your constructive feedback and for considering our revisions.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732559851495

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
Dear Reviewers,

Thank you for your efforts and valuable feedback so far. We are especially grateful to **NMzG** for your active engagement during this process.

We kindly request reviewers **b2uw** and **Asgg** to share your thoughts on our responses as well.
As the discussion period is approaching its end, with a deadline set for **Nov. 26th**, we are still awaiting responses from reviewers who have not yet provided feedback. We have carefully prepared specific responses to address each concern raised.

We would greatly appreciate it if you could review our responses and let us know if there is anything unclear or that could be further detailed to enhance the quality of our paper. Your input would be invaluable in helping us improve.

Best regards,

Authors of #4698
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper utilize the neural ODEs and develops a continuous exposure learning approach for low-light image enhancement. Experimental results show the effect of the proposed method. 

The major concerns of reviewers include the computational cost, generalization ability, and limited evaluations. In the rebuttal, the author solve the most concerns of reviewers. However, there are still major issues as detailed below. 

The paper claim that the convergence of the existing method are not guaranteed and the proposed method has better convergence  property. However, this is not explained or demonstrated in the paper. In addition, no theoretical analysis is provided. That is, the contribution of the paper has been exaggerated.

Based on the comments of reviewers and the above-mentioned issues, the recommendation of this paper is borderline.