Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: To tackle the problem of joint luminance enhancement and denoising in low-light imaging, the author proposed 1) a new network architecture, UHDFour, that utilizes authors' observations on noise and signal luminance's relation in the amplitude and phase images in the frequency domain, and 2) a new dataset, UHD-LL, that contains high-resolution 4K low-light / normal light pairs.
strength_and_weaknesses: ### Strength 
1. A reasonably sized, high-resolution new dataset for joint luminance enhancement and denoising is provided.
1. The proposed network architecture cleverly mixes and matches the amplitude and phase of the low light input and normal light GT to constrain the denoising network.
### Weakness
1. The importance of the Fourier branch (FB) compared to a more conventional spatial branch (SB) in FourSpa is less than expected and should be further validated. Swapping FB with SB brings about a $0.6~dB$ difference in PSNR, as ablation studies #1 and #2 show, which made me wonder if 2xSB (i.e. increase the size of the SB) would have similar performance as FB+SB in FourSpa. Moreover, for the adj. block, it's necessary to show the performance of SB only and SB increase to the comparable size of SB+AM+PG. Lastly, there should be a Fourier-free architecture where there are only SBs (of the equivalent size of SB + FB + AM + PG + SB, e.g. #9) in both FourSpa and Adj. Block. The author should consider adding those in the final version.
1. What are the computational and time costs of FFT/IFFT-related operations? It will be useful if the author can add this to the ablation study table.
1. Even for consumer audio/video market, "HDR" and "4K" are oftentimes mentioned together. Moreover, HDR is common for low-light scenes as light sources can often be seen in night-time photography. However, HDR scenes are not considered in this paper. I suggest the author mention such drawbacks of the proposed dataset in discussion and/or indication of the dynamic range of the proposed datasets (i.e. the specs of the cameras used for the data capture).
1. Also, since an important aspect of this paper is luminance enhancement, it's natural to discuss HDR tone-mapping. A simple search on [joint HDR and deoising](https://scholar.google.com/scholar?hl=en&q=joint+hdr+denoising) leads to many recent papers on the topics. I suggest the authors consider including such papers in the related work section.
1. Denoising research is at a stage where one has to "pixel peep" to tell the difference. Please provide zoom-in insets of the denoising performance in the result figures.
1. The proposed UHDFour method is only tested on one external dataset, LOL. It's reasonable to include at least another state-of-the-art dataset
1. For the dataset comparison table, the authors should add SID datasets. The argument that SID only works for particular Bayer pattern is not valid. SID data's Bayer or Fuji pattern raw can be demosaiced to RGB raw, meanwhile All the authors' data also undergoes some demosaicing too.
clarity,_quality,_novelty_and_reproducibility: The paper is well-written and easy to follow. The quality is up to ICLR's bar. The method, to my knowledge, is novel, and the datasets would be a nice addition to the research community in low-light imaging. Detailed network layer description or table, or source code instead, should be provided, otherwise, reproducing the paper's results can be difficult given the complexity of the proposed network architecture and various comparison and ablation studies performed.
summary_of_the_review: Overall, this is a solid paper with interesting insights and new datasets. If the authors and properly address the concerns in the weakness section, I'm happy to upgrade the rating from marginally accept to accept.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper propsoe the UHDFour solution for UHD enhancement. This paper introduce a new dataset for 4K UHD, called UHD LLIE. Also this paper designed a module utilize the frequency domain.
strength_and_weaknesses: Strength:
1. This paper is well written and easy to understand.
2. The performance is great.
3. The novelty is enough for ICLR. The feature modeling in frequency domain and the dataset.

Weakness.
1. I'm little confused about table 2. Why the performance of proposed method is not shown in Table 2?
2. I suggest authors also show the Amplitude and Phase in the network. People can see how the network improve the Amplitude and Phase.
clarity,_quality,_novelty_and_reproducibility: I think the novelty of this paper is enough and the paper is complete and easy to understand.
summary_of_the_review: I think this paper is complete and has enough novelty.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper focuses on the Ultra-High-Definition (UHD) low-light image enhancement, which is a new task. Different from previous low-light image enhancement, the task presented in the paper faces more challenging and practical issues such as dealing with the intricate issue of joint luminance enhancement and noise removal while remaining efficient. The paper first shows the limitations of previous methods and datasets for processing the UHD low-light images.  Then, the paper proposes to embed Fourier transform into a cascaded network to address these issues. The designs are interesting and insightful, which are based on the unique characteristics in the Fourier domain. In comparison to spatial domain, these characteristics of Fourier domain make the task easy to deal with. Besides these designs, the paper also presents the first real UHD low-light image enhancement with paired data. The paper also provides some techniques about how to capture the dataset of this kind. The proposed solution achieves the best performance on UHD low-light image enhancement and fast inference speed. Also, the solution can be extended and used for processing existing low-light image dataset.
strength_and_weaknesses: Strengths:

+ New task. UHD low-light image enhancement task is becoming more and more important, especially on mobile devices. The reviewer is happy to see there is a paper focusing on this issue and proposing a new solution and the first dataset.
+ Novel solution. Facing challenging issues such as joint luminance enhancement and noise removal under the ultra-high-resolution constraint, this paper proposes an interesting and insightful solution. Different from previous spatial methods, this method tasks full use of the unique characteristics of Fourier domain for this task. In the Fourier domain, the amplitude and phase of a low-light image can be separately processed to avoid amplifying noise when enhancing luminance. Moreover, the proposed method can be scalable to UHD images by implementing amplitude and phase enhancement under the low-resolution regime and then adjusting the high-resolution scale with few computations. The solution is novel and interesting.
+ New dataset. This paper contributes the first UHD low-light image enhancement dataset with paired data. The dataset not only provides sufficient and diverse data but also presents some techniques which show how to minimize geometric and photometric misalignment when capturing the paired low-light/normal-light data. The data collection details provide some insights into the pair data collection.
+ Systematical analysis. This paper systematically analyzes the performance of existing low-light image enhancement methods for processing UHD images. The baselines cover almost all state-of-the-art methods, and the experiments include quantitative and quantitative results. From the results, it is clear to see the advantages and disadvantages of previous methods. 
+ Outstanding performance. The proposed method in the paper achieves state-of-the-art performance for UHD low-light image enhancement with the fastest inference speed. Besides, the proposed method can be slightly modified and then achieves state-of-the-art performance on existing low-light image enhancement dataset despite it is not designed for data of this sort.
+ Sufficient experiment and analysis. The paper provides sufficient comparison experiments to show the advantages of the proposed method in the main paper and appendix. In addition, the ablation studies and motivation analysis are convincing.

Weaknesses:

+ The paper analyzes the performance of existing low-light image enhancement for processing UHD low-light images. The baselines are all deep learning-based methods. The reviewer wonders whether traditional methods still cannot solve this challenging issue either. It would be good if some traditional methods could be included for comparison in the final version.
+ For the Sec. 4.1 Benchmarking existing models, the quantitative scores in terms of non-reference metrics could be removed when the paired data is available. It is almost common that non-reference metrics are not much reliable, especially for low-light images. 
+ In Figures 6-8 and Tables 2-4, for each method, the corresponding reference should be provided to improve the readability of the paper. It is the same for the figures in the appendix. 
+ The reviewer wonders whether the main idea in Fourier domain can be extended to other UHD image related low-level tasks.
clarity,_quality,_novelty_and_reproducibility: The paper is of high quality and provides interesting and important insights into UHD low-light image enhancement. The paper is well-written and easy to follow. The proposed method in the paper is original and novel. Besides, the paper presents the first real UHD low-light image enhancement dataset.
summary_of_the_review: The paper presents a novel solution and a new dataset. Moreover, the analysis and motivations of this work provide new insights into this challenging task. The experiments are sufficient, and the results are convincing. These are the important factors in my rating.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a new method (UHDFour) for enhancing ultra high-resolution low-light noisy images. The proposed method is based on two observations. The first observation is that most luminance information exists on amplitudes while noise exists on phases of the Fourier domain. This observation drives the proposed UHDFour method to process amplitudes and phases separately for luminance enhancement and noise removal. The second observation is that a high-resolution image and its corresponding low-resolution version have similar amplitude patterns, which results in a cascaded network design that first processes a low-resolution image and then processes a high-resolution one. The authors construct a new ultra high-resolution image dataset (UHD-LL). The authors conduct experiments to justify the motivations and the effectiveness of proposed method.
strength_and_weaknesses: Strong points.
1. A new method is proposed for ultra high-resolution low-light noisy image enhancement with notable performance advantages against existing methods.
2. The network design has a clear motivation with some interesting experiments to justify the observations.
3. A new ultra high-resolution dataset of 2150 low-noise/normal-clear paired 4K images is constructed.
4. The paper is written well and easy to follow.
5. Code and dataset will be released.

Weak points.
1. The first observation is justified via visual illustrations and quantitative experiments while the second one is illustrated by visual comparisons. It would be better if the authors can provide some interpretations. Can authors explain from the FFT why these phenomena happen? The second observation only mentions that HR images and their corresponding LR images share similar amplitude patterns, but how about the phases? Do they have similar patterns? If not, would the noise removal in LR version affect that in HR images?

2. In the 3rd paragraph, Introduction, there are some vague descriptions when talking about existing methods, e.g., "existing methods are mainly..." and "some studies...". I suggest adding references here so that readers can have direct references.

3. The LRNet/FouSpa block designs seem not to be aligned with the observations. For the FouSpa block, it is mentioned that "we observe that luminance and noise can be decomposed in the Fourier domain", but the design (Fig. 4) shows that the Fourier branch only separately processes the A and P, then combines them together using IFFT. There is still a Spatial branch that works with the Fourier branch. It is not clear why there should still have a spatial branch and how the Fourier branch enhances luminance in A and removes noise in P. Regarding the LRNet design (Fig. 3), it is not clear why FFT and Conv layers are paralleled. Further, do A_r and P_r have any supervision signals? Would the authors show some intermediate visualizations of A_r and P_r?

4. Regarding the dataset acquisition, it seems that images are taken in normal-light scenes, and low-light images are captured with under-exposures. In this case, it seems the noise would not be very severe (e.g., in Figure 5 it is hard to see noise). This makes this paper more related to under-exposure image enhancement but not low-light image enhancement. It would be better if the authors could provide more justifications for this issue. The dataset split is 2000 for training and 115 pairs for test (The train/test ratio is 20:1), which I think is extreme. It would be better if the authors could justify this.

5. A minor issue is that "Table 5" on page 8 should be "Table 4".

6. In Section A of the Appendix, I think it would be better if the authors can discuss how FFT is used in other low-level vision tasks, e.g., image deraining, denoising, and deblurring, and highlight the differences to the proposed method.
clarity,_quality,_novelty_and_reproducibility: The paper is generally written well and easy to read. The paper shows some interesting observations, based on which it presents a new method for ultra-high-resolution image enhancement. A new dataset is constructed and will be released to the public together with the code, as the authors promise.
summary_of_the_review: I am on the positive side of this paper but with some questions listed in the above section. I am willing to raise my rating if the authors can address them.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper introduced a new task of UHD low-light image enhancement, and proposed to embed Fourier transform into a cascade network to address the limitations of previous methods. The authors also created a new UHD LLIE dataset for systematic evaluation. Sufficient experiments (compared to many existing low-light enhancement algorithms) and ablation studies demonstrated superior results.

Out of 4 reviewers, 1 (R #ohoL) is positive, and the other 3 are very positive. The very positive reviewers liked the new task, novel solution, new dataset, systematic analysis, outstanding performance and sufficient experiment & analysis (R #Fqm7, #yq2M), and the paper is written well and easy to follow, and the code & dataset will be released (R #aXaB). The AC agrees with these positive points: the problem is new and difficult, yet the solution is novel, effective and efficient.

R #ohoL raised the issue of whether 2xSB could be similar to FB+SB (and hence to discount FB). The authors responded with an updated Table 5 on page 9 showing the advantage of FB+SB. As to the raised issue of HDR with UHD, the AC does not agree with R #ohoL that they always co-occur (many 4K displays are not HDR); besides, we can be less picky for the first attempt to a new problem. The other 3 very positive reviewers raised some minor issues and were addressed in the rebuttal.

In summary, this is a well-written paper that could have real impact on the imaging and computational photography community.

justification_for_why_not_lower_score: In many ways this paper sets a gold standard of what a good paper should be: new & challenging problem with real impact, novel solutions, new dataset, thorough evaluation, extensive comparisons, and good writing. It's going to have a very positive impact to the community.
Decision: Accept: notable-top-5%