LuSh-NeRF: Lighting up and Sharpening NeRFs for Low-light Scenes

Authors: zefan  qu
ke  xu
gerhard  petrus
hancke  rynson
w h lau
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 15:19:31

Abstract: Neural Radiance Fields (NeRFs) have shown remarkable performances in producing novel-view images from high-quality scene images. However, hand-held low-light photography challenges NeRFs as the captured images may simultaneously suffer from low visibility, noise, and camera shakes.
While existing NeRF methods may handle either low light or motion, directly combining them or incorporating additional image-based enhancement methods does not work as these degradation factors are highly coupled.
We observe that noise in low-light images is always sharp regardless of camera shakes, which implies an implicit order of these degradation factors within the image formation process.
This inspires us to explore such an order to decouple and remove these degradation factors while training the NeRF.
To this end, we propose in this paper a novel model, named LuSh-NeRF, which can reconstruct a clean and sharp NeRF from a group of hand-held low-light images.
The key idea of LuSh-NeRF is to sequentially model noise and blur in the images via multi-view feature consistency and frequency information of NeRF, respectively.
Specifically, LuSh-NeRF includes a novel Scene-Noise Decomposition (SND) module for decoupling the noise from the scene representation and a novel Camera Trajectory Prediction (CTP) module for the estimation of camera motions based on low-frequency scene information.
To facilitate training and evaluations, we construct a new dataset containing both synthetic and real images.
Experiments show that LuSh-NeRF outperforms existing approaches. Our code and dataset can be found here: https://github.com/quzefan/LuSh-NeRF.

Main paper: 
 introduction: 
 Neural Radiance Fields (NeRFs) [31, 2, 3, 5, 24] have achieved notable success in modeling 3D scene information via implicit functions learned from a set of 2D images with known camera poses. Since optimizing NeRFs by measuring the colorimetric errors of training views essentially requires bright and sharp training images, hand-held low-light photography, which is prevalent in our daily life (e.g., in nighttime scenes), cannot be used directly for training NeRFs to produce visually pleasing novel view images (Fig. 1(b) ), due to the co-existence of low visibility, noise, and camera motion blur in the captured images.A straightforward solution is to incorporate existing low-light enhancement/deblurring methods (e.g., [57, 12, 67, 59, 13, 45] ) to preprocess the captured images before using them for training the NeRFs. However, it raises two problems. First, as these methods are typically image-based, they do not consider the multi-view consistency. Second, These enhancement methods may introduce (a) A Low-light Scene (b) NeRF [31] (c) LEDNet [67] +NeRF [31] (d) Restormer [59] +LLNeRF [47] (e) PairLIE [13] +DP-NeRF [19] (f) Ours additional artifacts (e.g., overexposure and unnatural color contrast). We note that there are some NeRF methods proposed for handling low-light scenes [47, 6] and scene motions [26, 48, 19, 7] . However, while the former assumes no camera motions occur during the capture, the latter cannot handle low-light scenes. Directly applying them to our problem does not work. A visual example is shown in Fig. 1((c ) to (e)), where existing methods struggle to render the desired results. 
related works: 
 Neural Radiance Field (NeRF). NeRF [31] has fundamentally changed the way of modeling 3D scenes by learning coordinate-based implicit neural representations from a set of 2D observations. has gained widespread popularity in computer vision and graphics tasks pertaining to neural rendering, leveraging coordinate-based implicit neural representations (INR). It has gained significant popularity in computer vision and graphics tasks, with many NeRF variants proposed for, e.g., accelerating the training and rendering of NeRF [14, 18, 40, 41] , handling dynamic scenes [21, 34, 39, 44, 61] and digital humans body [1, 34, 36, 37, 10] or human head [53, 68, 16] modeling, and the manipulation [4, 38, 28, 23] or generation [8, 15, 25, 33] of scene contents. Many variants of NeRF are also gradually expanding into multiple research areas. For example, [14, 18, 40, 41] are proposed to accelerate the NeRF training procedure, [21, 34, 39, 44, 61] are applied to render dynamic scenarios, [4, 38, 28, 23] are focused on the NeRF relighting methods, [1, 34, 36, 37, 53] are expanded to the non-rigid object rendering, [8, 15, 25, 33] are used for the generation models. These methods typically require high-quality (i.e., bright and sharp) 2D images for optimizing NeRFs.NeRF from Low-light Images. Recently, there are some methods [47, 30, 6] proposed to relax such constraints by learning to reconstruct NeRFs from low-light images. Wang et al. [47] reconstructs a normally illuminated scene with multiple low-light images in an unsupervised manner. Cui et al. [6] can change the luminance of the scene by extending the transmittance function in NeRF. However, these methods do not consider the camera shakes that often occur in hand-held low-light imaging.NeRF from Blurry Images. Rendering scenes with multiple blurry images is frequent and challenging in the real world. Ma et al. [26] firstly propose the deblurring problem in NeRF and simulate the blurring process with a deformable kernel. Most works [19, 48, 20, 17] model the camera motion trajectory in 3D space to handle the blur in normal-light scenes. Peng et al. [35] accelerate the deblurring process with an efficient rendering scheme. Huang et al. [17] utilize a blur generator to model the camera imaging process for the all-in-focus photos. However, these methods cannot handle hand-held low-light photographs as they do not jointly model noise and camera motions under low-light conditions.Low-light Image Enhancement. Deep learning-based methods have been shown to be more effective in this task. Some Retinex-based methods [52, 65, 64, 54] and end-to-end methods [9, 11, 27, 32, 50, 56, 66, 58, 46, 22] are capable of achieving good enhanced results. Dong et al. [9] abandon the bayer-filter to recover the low light color from raw camera data. Dudhane et al. [11] merge sequential images taken in the same scene to enhance the image quality. Xu et al. [56] take an SNR prior to guide the feature learning and formulate the network with a new self-attention model. However, there are few methods that can address the blur present in practical low-light images owing to the long exposure time. Recently, Zhou et al. [67] propose a large dataset for the low-light deblurring task, and some works [67, 62] propose solutions to enhance low-light blurred images.Low-light Deblurring. There are few methods that can address the blur present in practical low-light images owing to the long exposure time. Recently, Zhou et al. [67] propose a large dataset for the low-light deblurring task, and some works [67, 62] propose solutions to enhance low-light blurred images. Zhou et al. [67] propose a highly coupled encoder-decoder architecture to solve the joint artifact. Zhang et al. [62] delves into the multiple degradations and proposes an all-in-one fashion restoration network. While our work addresses a similar problem, we aim for NeRF reconstruction, which is more challenging. 
proposed method: 
 In this section, we will introduce the details of the proposed method LuSh-NeRF, which can reconstruct a normally illuminated, sharp, and clean scenario from a set of hand-held captured low-light 2D images. Specifically, we first propose the SND module to separate noise from the scene information in the pre-processed image (Sec. 3.2), and then use the CTP module to make an accurate prediction of the kernel from the low-frequency information in the image (Sec. 3.3). Our network is supervised by multiple loss functions (Sec. 3.4). The overall structure of the network is shown in Fig. After obtaining the information of each sample point on a ray, the pixel value Ĉ(r) of ray r can be computed as:• • • Rigid Warping Noisei ⊕ Weighted Frequency Mask L Construction • • • • • • • • • • • • • I hl,EQUATION)where δ is the distance between two subsequent sample points on a ray. T i is the accumulated transmittance value that denotes the radiance decay rate of sampled points. In this work, we attempt to optimize the NeRF by decoupling and removing the above defects in the training stage. Modeling and analyzing noise and blur are difficult at low pixel intensity values. Thus, the low visibility of the image should first be raised. Considering that noise in a low-light photo is mainly generated by the imaging process and would not be affected by the camera motions, the implicit order of the defect decomposition should be low visibility, noise, and blur.Specifically, given multiple training images I ll,noisy,blurry from different views, NeRF aims to render a scenario with I hl,clear,sharp . We first scale up the brightness of I ll,noisy,blurry and get I hl,noisy,blurry for more contextual information, but this process simultaneously amplifies the noise in the image. LuSh-NeRF then decomposes the image into scene and noise information with 3D multi-view consistency, removing the noise from the image to obtain I hl,clear,blurry . Finally, the image without noise interference is utilized to model the blur phenomenon and restore I hl,clear,sharp .The enhancement process for LuSh-NeRF is as follows:I hl,clear,sharp = R(LuSh-NeRF) = Sharpen(ScaleUp(I ll,noisy,blurry ) -Noise)EQUATIONwhere Noise is the noise of the different views predicted by the SND module. Sharpen(•) denotes the CTP module to ease the camera motion blur phenomenon, and R(•) is the NeRF rendering function.During training, due to the lack of ground truth, I hl,noisy,blurry (i.e., ScaleUp(I ll,noisy,blurry )) is used as the supervised signal. The inverse of Eq. 2 should be taken to simulate the low-quality image as:EQUATIONAfter training, the correct-light (denoted as hl), sharp and clear NeRF model can be obtained directly by discarding the Sharpen -1 function and Noise. In the following subsections, we introduce how we estimate the Noise under each view with the SND module and how we accurately predict the Sharpen -1 function with the CTP module.Directly training NeRF with I hl,noisy,blurry would still produce defective rendering results, as scaling up the low-light images can amplify noise and distort colors significantly, causing more severe multi-view inconsistency.We note that while the noise is generally sharp and stochastic and has different values in different views, scenario information across views tends to be consistent. This makes it possible to decouple noise and scenario information. To co-optimize with the base network Scenario-NeRF (S-NeRF), we propose a new network, named Noise-Estimator (N-Estimator), which has the same network structure as S-NeRF, to compute the noise of each rendering coordinate. Since the noise is not consistent, N-Estimator discards the volume rendering calculation and directly takes the values of the intermediate sampling points of the input rays as the output. The formula for obtaining the noisy pixel C noisy (r) is:C noisy (r) = CTP(C S-N eRF (r)) + C N -Estimator (r) = CTP( N i=1 w i c i ) + n N 2 ,where nEQUATIONwhere P mid is the intermediate sampling points coordinate, and d is the view direction. The CTP(•) function refers to the Camera Trajectory Prediction module, N is the number of sampled points on a ray, and n N 2 is the noise value rendered by N-Estimator. The volume rendering computation of S-NeRF gradually forces S-NeRF to learn the consistency scenario information, and the noise that does not share this feature is decomposed from the scene representation by the N-Estimator during the optimization process to fit the final noisy training images.To more accurately decouple the scene information and noise, LuSh-NeRF utilizes the Rays Alignment supervision in the SND module. Specifically, An image matching method [43] is applied to the images rendered by S-NeRF after sharpening in each viewpoint to obtain a dense pixel matching matrix M and a certainty matrix C, which represent the coordinate pairs of matching pixels in different views and the confidence level of each matching. The current input ray is then taken as anchor_ray, and the coordinates of its rendered pixels are used to find the matching pixels under other viewpoints in M . The confidence of the matching pairs in C should be higher than the preset threshold θ to ensure that accurate matches can be obtained.With these matching pixel coordinates, we can get the corresponding aligned rays align_ray i under different viewpoints view i . Ideally, the RGB colors computed by anchor_ray and align_ray i should be close to identical owing to the scenario consistency. So a consistency loss L consistency is proposed to reduce the distance within these rendering pixels in S-NeRF to force the decoupling of scene information and noise, as:EQUATIONwhere K is the number of training views. Constrained by L consistency , the information extracted by S-NeRF has a higher viewpoint consistency, leading to a better decomposition of the scene information and noise.Motivated by [19] , the CTP module employs the same thoughts to predict the rigid camera motions with all rays in each view. However, unlike the properly lighted blurry data used in [19] , the practical low light conditions present substantial interferences, which could cause notable errors in the CTP network predictions, as demonstrated in the results of Subsec 4.1.When the SND module is not fully converged, noise still exists in the SND-processed scenario. The kernel prediction using rays with strong noise interference may interrupt the CTP network, and directly lead to the failure of the whole NeRF training. Hence, it is necessary to filter the rays for CTP module optimization. Since the high-frequency and low-intensity regions in the low-light images are more disturbed by noise, corrupting the information in the original raw image [55] , the CTP module utilizes Discrete Fourier Transform (DFT) to obtain the image frequency map as:EQUATION)The noisy high-frequency regions of the image are filtered by the low-pass filter H of radius r:EQUATIONThe retained low-frequency image regions are then converted back to the image via the DFT -1 function to obtain the image informative region mask M ask lowpass (m, n) as:EQUATIONwhere Bi(•, T ) is a binarization function with threshold T . M ask lowpass (m, n) classifies the rays into ray clear and ray noisy , which are used to render the clear and noise-dominated pixels in the training images separately. The gradient of ray noisy to the rigid motion prediction network is detached during NeRF optimization to reduce the impact of noise yet to be removed by the SND module on blur kernel prediction as:EQUATIONwhere the Detach(•) function is used to detach the gradient from the variables. The role of the low-pass frequency filter is shown in Fig. 7 , where the CTP Mask reduces the interference of noisy regions on the blur kernel prediction compared to the mask obtained by directly using the RGB intensity as the threshold.After employing the SND and CTP modules as in Eq. 3, we have the simulated RGB values Ĉhl,noisy,blurry in the image Îhl,noisy,blurry . We can then optimize the network with the MSE loss L consturction as: Low-light Scenes LEDNet [67] +NeRF [31] PairLIE [13] +DP-NeRF [19] Restormer [59] +LLNeRF [47] LuSh-NeRF(Ours) All the modules in the network are trained in an end-to-end manner with the following training losses:EQUATION)EQUATIONwhere α and β are the hyper-parameters to balance the impact of two losses. When rendering a scenario, only the S-NeRF network in LuSh-NeRF is utilized with the volume rendering method to produce images of different views. 
experiments: 
 Our Dataset. Since we are the first to reconstruct NeRF with hand-held low-light photographs, we build a new dataset based on the low-light image deblur dataset [67] for training and evaluation. Specifically, our dataset consists of 5 synthetic and 5 real scenes, for the quantitative and generalization evaluations. We use the COLMAP [42] method to estimate the camera pose of each image in the scenarios. Each scenario contains 20-25 images, at 1120× 640 resolution. The brightness of images in each scenario is extremely low, where the intensities of most pixels are below 50. 80% of the images contain camera shake problems, which pose a significant challenge for NeRF reconstruction. Refer to Appendix A.3 for more details of our dataset.Implementation Details. We have implemented our LuSh-NeRF based on the official code of Deblur-NeRF [26] , and with the same Rigid Blurring Kernel network in [19] . The S-NeRF shares the same structure as NeRF [31] , while the the depth and width of N-Estimator are set to half of those of the S-NeRF. The number of camera motions k and the frequency filter radius in the CTP module are set to 4 and 30. The number of aligned rays K and certainty threshold θ in the SND module are set to 20 and 0.8. Before training, the input images are up-scaled by gamma adjustment and histogram equalization. The batch size is set to 1,024 rays, with 64 fine and coarse sampled points. α and β are set to 1 and 0 during the first 60K iterations for better rendering results, to avoid the inaccuracy matching matrix M interfering with the SND module. The two hyper-parameters are then changed to Input LEDNet [67] +NeRF [31] Restormer [59] +LLNeRF [47] MPRNet [60] +LLNeRF [47] PairLIE [13] +DP-NeRF [19] URetinex [54] +DP-NeRF [19] LuSh-NeRF (Ours) Ground TruthInput LEDNet [67] +NeRF [31] Restormer [59] +LLNeRF [47] MPRNet [60] +LLNeRF [47] PairLIE [13] +DP-NeRF [19] URetinex [54] +DP-NeRF [19] LuSh-NeRF (Ours) Ground TruthInput LEDNet [67] +NeRF Restormer [59] +LLNeRF [47] MPRNet [60] +LLNeRF [47] PairLIE [13] +DP-NeRF [19] URetinex [54] +DP-NeRF [19] LuSh-NeRF (Ours) Ground TruthFigure 5 : Qualitative results of different methods on our synthetic scenes. Our method yields the most natural restoration results while sharpening the image.1 and 1 × 10 -2 in the last 40K iterations. All the experiments in this paper are performed on a PC with an i9-13900K CPU and a single NVIDIA RTX3090 GPU.Evaluation Methods and Metrics. Since we are the first to tackle the NeRF reconstruction from lowlight images captured with camera motions, we design several baseline methods by combining existing state-of-the-art image-based and NeRF-based methods: (1) Low-light image enhancement [54, 49] + Image deblurring method [60, 59] + NeRF [31] ; (2) Joint low-light image enhancement and deblurring method [67] + NeRF [31] ; (3) Low-light image enhancement [13, 54] + deblurring NeRF [19] ; and (4) Image deblurring [59, 60] + low-light NeRF enhancement [47] . All the selected models have been shown to achieve SOTA performances on their individual tasks. PSNR, SSIM, and LPIPS [63] metrics are used to evaluate the performance difference of the rendered images between LuSh-NeRF and the combination of other state-of-the-art methods. Due to the significant variations in color contrast ratio and image brightness of the restored low light images by different methods, we consider LPIPS (Learned Perceptual Image Patch Similarity), which can reasonably assess perceptual image quality and visual differences rather than the pixel-level difference, as the key metric.Visual Comparisons. Fig. 4 presents qualitative comparison of the rendering results from the above three combined methods with those of our LuSh-NeRF in our real low-light scenarios. We can see that the results from LEDNet [67] are still dark, which significantly limits the learning capability of the NeRF. While PairLIE [13] combined with DP-NeRF [19] can reconstruct clear rendering results, they are rather unnatural. The color contrast is severely shifted, resulting in unrealistic images with some lost details and incomplete blur removal. In addition, we can also observe that first deblurring on low-light images and then applying the LLE method, as in Restormer [59] + LLNeRF [47] , is less effective, and there is still substantial blurring in the results. In contrast, our approach can recover detailed scenes with natural colors, resulting in a sharp and clean novel view visualization.Fig. 5 presents qualitative comparison between existing methods and our LuSh-NeRF on our synthetic scenes. All of the above methods suffer serious performance degradation when the image luminance is low and the camera shake is severe (e.g., the bottom of the first example in Fig. 5 ), where LuSh-NeRF can still reconstruct a satisfactory scene by decoupling the defects of the scene. As we can see from the given visualizations on realistic and synthesized data, higher PSNR and SSIM metrics of the methods (e.g., LEDNet + NeRF) do not entirely represent better performances on the NeRF results in our proposed task. This demonstrates that the perceptual metric, LPIPS, is better at assessing the merits of the reconstructed scenario. Refer to Appendix A.5 for more visual results.Quantitative Comparison. We conduct a quantitative comparison of our method against various combinations of SOTA approaches on our synthesized data in Tab. 1. Note that our synthesized data originates from the training and testing sets of LOL-Blur dataset [67] , while the weight of LEDNet [67] is also specifically trained on the synthesized dataset of LOL-Blur. This condition does not constitute a fair comparison. Nonetheless, we report the results of using "LEDNet [67] + NeRF [31] " as a reference.Our method achieves better performances than multiple combinations of existing methods on all 5 synthesized data. In terms of the LPIPS metric on all the scenarios, our method outperforms the rendering results of LEDNet [67] + NeRF [31] in an unsupervised optimization manner, which proves that the SND and CTP modules in LuSh-NeRF fully utilize the 3D information of the scene to aid in the recovery of the irreversible quality degradation in the single image.Ablation Study. Fig. 6 demonstrates the effect of the various components of LuSh-NeRF on a realistic scenario. After the images are preprocessed to enhance the luminance, the noise in the images is significantly amplified (as shown in the white box). Directly using blur kernel prediction [19] on these images is heavily interfered by noise, which prevents recovery from camera shakes (Fig. 6(b) ).In addition, incorrect kernel predictions have a negative impact as they make it difficult to align the multi-view images rendered by S-NeRF and the SND module cannot obtain accurate viewpoint consistency supervision (Fig. 6(c )). When the CTP module is utilized, the blur of the image is alleviated, but the noise problem is still serious (Fig. 6(d) ). First sharpening the image with CTP We show in the Fig. 7 the difference between the two masks obtained by directly performing an RGB intensity threshold and performing a frequency filtering before taking a threshold. (1) The RGB intensities in many noise-dominant regions are also high (e.g., the sky and the grass), which are harmful to the blur estimation but not excluded in the mask produced by thresholding. (2) The CTP module uses a frequency filter to identify the low-frequency-dominated regions of the image, and then obtains the desired mask based on the RGB intensity values. The gradients of rays in high-frequency and dark regions (regions more severely affected by noise) are detached during the Blur Kernel optimization process, which ensures better blur modeling. 
conclusion and limitation: 
 In this paper, we have proposed the NeRF reconstruction task from low-light images with camera motions, and a novel unsupervised reconstruction network LuSh-NeRF. LuSh-NeRF models the NeRF restoration process based on the characteristics of three kinds of defects in the scene, i.e., low-light, noise, and camera blur, and successfully reconstructs a normal-light, clean, and sharp NeRF network from poor-quality low-light images, by exploiting multi-view consistency and frequency-domain information. To facilitate the training and evaluation of the new proposed task, we construct a new dataset which contains 5 synthetic and 5 real scenes low light images with hand-held camera motions. Extensive experiments on our proposed dataset demonstrate that our method is capable of achieving satisfactory NeRF reconstruction results, despite the defects in the input images.The LuSh-NeRF network also has some limitations. One limitation is that it needs to optimize two NeRF networks at the same time and render multiple rays for the blur problem when sharpening images during training, which results in a slower optimization process, even though this would not interfere with the rendering speed. Besides, noise that is relatively similar across views may be difficult to remove due to the reliance on viewpoint consistency. (Refer to Appendix A.8 for details.) As a future work, we would like to address these issues. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: See Subsection 4.1 and 4.2.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: The paper provides extensive information about the statistical significance of our experiments.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: We have followed the Ethics Guidelines to conduct our research and prepare our submission.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: See Appendix A.1.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This work does not pose such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We have cited the corresponding papers when using their codes, data, and models.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: We will provide the documentation along with the new assets in our project page.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: A.1 Broader Impacts While we do not foresee our method causing any direct negative societal impact, it may be leveraged to help create fake images using image generation techniques. The human information in the captured images may have the risk of a leak that raises privacy concerns. We urge the readers to limit the usage of this work to legal use cases.The Rigid Blurring Kernel (RBK) module proposed in the DP-NeRF [19] is utilized in our CTP module. It mimics the scene blur kernel by simulating the 3D camera motions, which contains the following two main parts:Ray Rigid Transformation: The blurring process of an image is modeled by the Ray Rigid Transformation (RRT). The RRT is formulated as ray transformation derived from the deformation of rigid camera motion, defined as the dense SE(3) field for scene s, approximated by the MLPs as follows:EQUATIONwhere l s is the latent code for each view through the embedding layer in [1] , R, L, E are three MLP networks, view img is the training image set. The S s = (r s ; v s ) ∈ R 6 is the matrix which will be used for the RRT modeling as follows:EQUATIONwhere ray s and ray RRT s;q are the orgin ray and the transformed rays in scene s, q ∈ {1, ..., k}, k s a hyper-parameter that controls the number of camera motions contributing to the blur in each scene s. The blurry RGB value at ray s can be acquired by weighting sums of the NeRF volume rendering values C s;0 and C s;q from ray s and ray RRT s;q . Coarse Composition Weights: This part is responsible for computing the weights of each ray obtained by the RRT part, which is given by the following equation:EQUATIONwhere m s is the final weight for each ray in RRT. Finally, blurry color C s for scene s can be computed by the weighted sum operation as shown below:EQUATION)In this work, we collect low-light blurry images from five synthetic and real videos in the LOL-Blur dataset [67] and calculate the camera pose for each image with COLMAP [42] toolbox. For synthetic data, we use ground truth images to estimate camera pose, while for realistic data, the images recovered from LEDNet pre-trained model are utilized to estimate pose since COLMAP is hard to model the scenario with the origin low light images. Images with large shifts in each scenario are removed. The number of images for each scene and the training and evaluation viewpoint allocation are shown in Tab. 2.Realistic Dataset "Dorm" "Poster" "Plane" "Sakura" "Hall" "Campus" "Highway" "Signboard" "Car" "Neighborhood" All images have a resolution of 1120 x 640. The mean value of all pixels in the images is low (less than 50) and camera motion blur is present in 80% of the images in each scene. As illustrated in Fig. 1 , Fig. 4 , Fig. 5 and Fig. 9 , poor image quality in each scene leads to a challenging NeRF reconstruction task. (1) Lines 1 and 2 show that the ScaleUp preprocessing enhances the NeRF's reconstruction capabilities, resulting in an improved image.(2) Lines 3 and 4 show that CTP leverages frequency domain information to refine Blur Kernel predictions, boosting the perceptual quality of the reconstructed images. However, this process may decrease the PSNR and SSIM scores as it neglects noise interference.(3) From Lines 2 and 5, the SND module can disentangle noise and scene information from the input noisy-blurry images, leading to substantial improvements on the PSNR and SSIM metrics. However, the SND module is not capable of resolving the blur problem, which leads to a minor improvement in the image's perceptual quality (more important for the rendered images).(4) The comparisons between Lines 4, 5, and 8 show that combining SND and CTP modules enhances image perceptual quality and outperforms using only CTP in terms of PSNR and SSIM scores.(5) Lines 7 and 8 show that decoupling the noise in the scene first, and then modeling the scene blur is a more robust restoration order, as it can effectively reduce the interference of noise in the deblurring process for obtaining better performances.We have conducted the ablation experiments of the CTP module on two synthetic scenes in the Tab. 4. The experimental results show that the frequency filter provides a more desirable mask, which is superior to the directly obtained mask with RGB intensity thresholds.We provide more visual comparisons between our proposed LuSh-NeRF network and the sota methods combinations, i.e., LEDNet [67] + NeRF, Restormer [59] + LLNeRF [47] , MPRNet [60] + LLNeRF [47] , PairLIE [13] + DP-NeRF [19] and URetinexNet [54] + DP-NeRF [19] in Fig. 9 .COLMAP-free NeRF methods robust to low-light and blur phenomena would be much more helpful for the problem that we propose. However, existing colmap-free NeRF methods may not handle our task easily. The table below compares to a COLMAP-Free NeRF [51] . The results demonstrate that the existing COLMAP-free NeRF method cannot effectively handle low-light scenes with motion blur, due to the inaccurate poses optimized from the image directly.The image-matching method used in LuSh-NeRF is GIM [43] , which is a state-of-the-art generalizable image-matching model pre-trained with abundant Internet videos. As mentioned in Section 3.2, we also utilize a pre-defined threshold to select those high-confidence matches in the results produced by the GIM, avoiding low-quality matches that may affect the network's training negatively.We show some of the matches obtained by GIM on S-NeRF rendered results in Fig. 8 .Figure 8 : The visualization of matching results of GIM [43] on the S-NeRF rendering images.LuSh-NeRF may have some limitations on certain data, as shown in Fig. 10 . Although the noise in low light images is random and fluctuates greatly, if the noise in certain areas maintains similar characteristics in multiple viewpoints, LuSh-NeRF may model it as scene information, resulting in errors in rendering results. Increasing the training image numbers and spanning the range of different views may alleviate this problem to some extent.Input LEDNet [67] +NeRF Restormer [59] +LLNeRF [47] MPRNet [60] +LLNeRF [47] PairLIE [13] +DP-NeRF [19] URetinex [54] +DP-NeRF [19] LuSh-NeRF (Ours) Ground TruthInput LEDNet [67] +NeRF Restormer [59] +LLNeRF [47] MPRNet [60] +LLNeRF [47] PairLIE [13] +DP-NeRF [19] URetinex [54] +DP-NeRF [19] LuSh-NeRF (Ours) Ground Truth Justification: We have provided the project link in the adstract.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.† Joint corresponding authors. 


Decision: Three reviewers recommended a weak accept for this paper, while only one reviewer rated it as borderline reject. The primary concern of the latter reviewer was the clarity of the paper, though they acknowledged its originality, aligning with the views of the other reviewers. I encourage the authors to incorporate the reviewers' feedback in the final version.

Reviews: review: paper_summary: paper_summary: This method proposes a solution for NeRF optimization under low light settings by resolving 3 different forms of degradation: low intensity, camera noise, and motion blur. Low intensity is effectively resolved by scaling up the image, camera noise is resolved by proposing a consistency loss between different views to locate the noise, and motion blur is handled by using a camera trajectory prediction module to predict the camera motion and thus the corresponding sharpening function. The experiments demonstrate state-of-the-art performance on novel view synthesis given low-light conditions. Qualitative ablations are provided to support the contribution of each component of the method. The authors will also make their code readily available as well as a dataset with 5 synthetic and 5 real scenes for evaluating low-light novel view synthesis.
strength_weakness: strength_weakness: The ablations would be more convincing with some quantitative results to back up the qualitative results. 

There appear to be some mistakes in Table 1: in some columns two numbers are underlined, in others the best and second best seem to be incorrectly reported. Please be careful to fix these mistakes. 

There are some important citations missing for some parts of the related work, such as in line 71 when mentioning NeRF methods that model digital humans: 

1. HeadNeRF (CVPR 2022)
2. MoFaNeRF (ECCV 2022)
3. INFAMOUS-NeRF (arxiv, 2023)
questions: questions: As mentioned above, please provide quantitative results to support the ablation studies and be sure to fix the errors in Table 1. I will be considering the rebuttal carefully as well as the thoughts from other reviewers in deciding my final rating.
limitations: limitations: Limitations and broader impact section are both available and appropriate. This work naturally does not have many immediately obvious negative societal impacts as it is simply trying to faithfully render novel-view images in low-light conditions.
paper_summary: paper_summary: The authors propose LuSh-NeRF, a model that reconstructs a clean and sharp NeRF from handheld low-light images by sequentially modeling noise and blur. LuSh-NeRF includes a Scene-Noise Decomposition (SND) module for noise removal and a Camera Trajectory Prediction (CTP) module for estimating camera motions based on low-frequency scene information. Experiments demonstrate that LuSh-NeRF outperforms existing methods in rendering bright and sharp novel-view images from low-light scenes.
strength_weakness: strength_weakness: 1. For me, the ideas and motivation behind this paper are quite good. What I am concern is whether the authors could consider adding some ablation studies, such as the role of various modules in LuSh-NeRF, this could let reader know which part is more effective.

2. In the scenario described by the author, I think the colmap-free NeRF method of this task may be more effective. Low-light blurred images have a certain impact on colmap estimation.
questions: questions: 1. I have another small suggestion. Could the authors consider compressing the images, or perhaps saving them in PDF format? This might help reduce the memory size of the paper's PDF file.

2. Could the author show some comparison with Aleth-NeRF [7] ? which is also a low-light NeRF method.
limitations: limitations: Please refer to the weakness part.
paper_summary: paper_summary: The paper proposed a method to train a NeRF with blurry (due to camera motion), low-light scene images. After training the method allows the recovery of enhanced, sharp images. To solve the problem two modules are proposed:

1) A SND module for noise modeling, which includes both a noise-prediction NeRF (N-NeRF) and a consistency loss, which uses image matching to constrain the radiance to be multiview consistent

2) A CTP module to model the blur induced by camera motion

The paper evaluates the method and baselines on a self-collected dataset.
strength_weakness: strength_weakness: Quality: Even though I believe the dataset to be useful, I find it hard to attribute the dataset to the authors of the paper. If I understand correctly (please correct me if I am wrong), the original images were actually captured by prior work (Lednet), and the authors of LuSh-NeRF only ran an off-the-shelf SFM method to recover the camera parameters? Happy to be shown otherwise, but I do not think in this case the authors can claim the dataset as a contribution of the paper. 
Other than that although the results seem convincing, the contribution of the paper isn’t significant. 

Clarity: I had serious issues understanding the paper, specifically the methods section. Authors introduce a lot of names, which I think would be better replaced by just mathematical notation. Some of the notation is also not properly explained, for example, $n_{N/2}$ in Eq. 5. (see also questions). 

It is also unclear to me how exactly Equations 4 and 5 relate. If I understand correctly Eq 4. suggests that to render the low-light, blurry images the network prediction is first unsharpened and then noise is added, but in Eq 5. noise is directly added to the radiance predictions from the network?

I think part of the CTP module also comes from the paper DP-NeRF, but I think the paper does not exactly mention what this module does. In the interest of making the paper self-sufficient, I think the authors should elaborate on what the module does/what it is exactly. I think what they do elaborate on (the detaching of certain rays) is just their contribution on top of the method from DP-NeRF, if I understand correctly. 


A few typos, there’s probably more I missed:

L145: bracket not closed 

L183: bracket not opened
questions: questions: 1) I think the $n_{N/2}$ refers to the N-NeRF output at the 3D coordinate which is the midpoint of the ray samples, is that right? If we don’t care about the 3D consistency of N-NeRF (since noise is not 3D consistent) why is the noise even modeled with a NeRF? Why not just optimize a tensor per view, or have the noise be a ray-conditioned network (instead of 3D coordinate conditioned)? 

2) I wonder how important it is to low-pass filter the images before thresholding for the CTP module, have the authors tried to just threshold the image without the low-pass filter?
limitations: limitations: Yes.
paper_summary: paper_summary: This work proposes a model to reconstruct a clean and sharp NeRF from a set of hand-held low-light images. The authors recognize the implicit order of the degradations (blur, noise, and low visibility), and sequentially decouple and remove each degradation in the network training. An SND module is proposed for noise removal and a CTP module is introduced for deblurring. They have conducted experiments on a dataset constructed from the LOL-Blur dataset proposed by [61].
strength_weakness: strength_weakness: 1. The presentation of the problem through mathematical equations is not clear. In Eq. 5, $C_{noisy}$ is the color value of a bright light blurry noisy (given in Eq. 4) image. But in Eq. 5, the first term $C_{S-NeRF}$ output is the sharp image and the second term is the noise term. The blurring operation is missing in this equation. Please correct me if I misinterpret anything. Also, N-NeRF does not use any volume rendering calculation to find the noise pixel value. Then, why this module is called a NeRF? It simply finds a pixel value for a given input pose. How the second term in Eq. 5 ($n\frac{N}{2}$) is derived? In Eq. 6, what are $i$ and $j$? In Fig. 2, the align ray is given subscripts $j$ and $k$. Please be consistent with equations and figure. 

2. In the paper, it is given that the camera trajectory prediction (CTP) module is novel (Line 16, 47). But the same idea of [19] (Line 180) is used for camera trajectory prediction with the additional masking of high-frequency pixel regions in the image. The main function of deblur-NeRF in the CTP module is not novel. 

3. Edge portions in the image will have high frequencies. When only rays containing low frequencies are used for gradient computation, typically edges (or high-frequency regions) will not come into the picture. But for deblur-NeRF, excluding the edge areas (high-frequency) is not a good idea since the blurring is mostly visible in the edges other than homogeneous (low-frequency) areas. How your frequency filter radius is selected? Did you consider any such factors for its selection?

4. In Line 224, it is given that for the first 60K iterations, $\beta$ is 0. This means SND does not come into the picture. In that case, the blur kernel estimated (by deblur-NeRF) will be wrong right (according to lines 185-186)? Is it correct to start from these trained weights after 60K iterations? What happens if $\beta$ is not 0 in the initial iterations?

5. SND relies on the image-matching method [40]. How generalizable is the method to your dataset? Have you taken the pre-trained weights? 

6. The contribution of the dataset (given in Line 64) is not that significant since the poses of the available dataset from [61] are computed for the dataset generation using COLMAP. The method is tested on this single dataset (containing synthetic and real videos). Since the work is meant for handheld lowlight photography, a video of a low-light scene with handheld cameras (which typically will have a camera shake) could have been captured and the proposed method could have been tested on the dataset. 

7. You have missed several references for Deblur NeRF papers. "ExBluRF: Efficient Radiance Fields for Extreme Motion Blurred Images", ICCV 2023; "PDRF: progressively deblurring radiance field for fast scene reconstruction from blurry images", AAAI 2023; "Inverting the Imaging Process by Learning an Implicit Camera Model", CVPR 2023. The related works (Sec. 2) section is not proper. There should be subsections for deblur NeRF and lowlight NeRF and briefly explain each work in both, other than explaining NeRF. Also, only low-light image enhancement methods are included. Why are deblurring methods not included? Or, it is better to have a subsection with explanations of lowlight deblurring works [61, 56]. Instead of just giving reference numbers, the essence of each very relevant work should be explained in a sentence in 'Related works'.

8. Some typos are there. In Line 153 and Fig. 6 (a) caption. In Line 35, [29] is not for low-light scenes. Line 41, the abbreviation of ISP is missing.
questions: questions: Please see weaknesses.
limitations: limitations: Yes. Limitations are included in Section 5 and negative societal impacts are included in Section A.1.

Comments: title: Clarification on W1Part1
Clarification on W1Part1 response
comment: Thanks for your suggestions. We promise to **revise our Method section** in the revision to ensure better understanding.
Thank you very much! I appreciate that the discussion is coming to an end, I don't think I need any more input from the authors. I'm not sure I will increase my score but I will relay my thoughts to the AC.

**Method section.** Ultimately, I would just like to implore the authors to rewrite the methods section a bit more clearly. My suggestion would be to limit the use of these short forms such as RRT, CTP etc. I think it's difficult to keep track of all these. 

Maybe use just two short forms for the two main parts of the method (SND, CTP). And make separate paragraphs within sections to describe each component of these separate paths. 

**Dataset.** I would also de-emphasize the dataset claim a bit, as the authors suggested. 

Thanks a lot!
Dear Reviewer 4eod,

Thank you again for your review and reply. We hope that our rebuttal and the following comments could address your questions and concerns.  As the discussion phase is nearing its end, we would be grateful to hear your feedback and wondered if you might still have any concerns we could address.

It would be appreciated if you could raise your score on our paper if we address your concerns. We thank you again for your effort in reviewing our paper.

Best regards,

Authors of Paper #10302
Thanks for your reply and we are glad to see that our response can address most of your raised concerns. We would like to futher clarify the below issues.

### R3W1: Insufficient Dataset Contribution.
**R:** (1) The dataset contribution is one **part of** our **3rd contribution** and we still have **other technical contributions**.

(2) Note that our dataset is necessary as we are handling a **new task** (handling NeRF in low-light scenes with camera motions). To construct this dataset, it took us more than 3 full weeks to select images and tune the camera pose parameters. Note that the colmap-free method [A] does not handle our task well as it is **very difficult** to optimize the camera pose directly in **low-light blurry** scenes.

(3) We can **revise** our third contribution to **emphasize more** on the experimental evaluations and state-of-the-art results of our model. However, we **do need this dataset** for evaluation and we did put efforts into constructing this dataset.

[A] NeRF--: Neural radiance fields without known camera parameters, arXiv:2102.07064, 2021.



### R3W4: Relation between CTP module and DP-NeRF.
**R:** Thansk for your suggestion. The Rigid Blurring Kernel (RBK) module in DP-NeRF[19] models the scene blur kernel by simulating the 3D camera motions via the following two main parts:

**Ray Rigid Transformation (RRT):** The RRT models the blurring process of an image. It is formulated as ray transformation derived from the deformation of rigid camera motion, which is defined as the dense SE(3) field for scene $s$ and modeled by the MLPs as:

$S_s = (r_s;v_s) = (\mathcal{R}(\mathcal{E}(l_s);\mathcal{L}(\mathcal{E}(l_s))), where \  s \in view_{img},$

where $l_s$ is the latent code for each view through the embedding layer in [B], $\mathcal{R}, \mathcal{L}, \mathcal{E}$ are three MLP networks, $view_{img}$ is the training view set. The $S_s = (r_s;v_s) \in \mathbb{R}^6$ is the matrix which will be used for the RRT modeling as follows:

$ray_{s;q}^{RRT} = Rigid-Transform(ray_s, (r_s;v_s)),$

where $Rigid-Transform()$ function is the standard 3D rigid transformation operation, $ray_s$ and $ray_{s;q}^{RRT}$ are the orgin ray and the transformed rays in scene $s$, $q \in \\{1,...,k \\}$, $k$ is a hyper-parameter that controls the number of camera motions contributing to the blur in each scene $s$. The blurry RGB value at $ray_s$ can be acquired by weighting sums of the NeRF volume rendering values $C_{s;0}$ and $C_{s;q}$ from $ray_s$ and $ray_{s;q}^{RRT}$.

**Coarse Composition Weights:** The Coarse Composition Weights are computed for each ray obtained by the RRT:

$m_{s;0,1,...,k} = \sigma(\mathcal{W}(\mathcal{E}(l_s))), where \  \sum_{i=0}^{k}m_{s;i} = 1,$

where $m_s$ is the final weight for each ray in RRT. Finally, blurry color $\mathnormal{C}_s$ for scene $s$ can be computed by the weighted sum operation as shown below:

$C_s = m_{s;0}C_{s;0} + \sum_{q=1}^{k}m_{s;q}C_{s;q}.$

**We will add these information to the revision.**

[B] Optimizing the latent space of generative networks, arXiv:1707.05776, 2017.

### R3Q1: Why is the noise modeled with a NeRF? Why not optimize ...
**R:** We agree with the reviewer that the $(x,y,N)$-based formulation is another possbile implementation of modeling noise comparing to our current one (the rendering results of both **do not differ much**). Note that this **does not affect our goal** of decomposing the scene and noise information.

Regarding the ray midpoint, we **uniformly sample (instead of random sampling in S-NeRF)** between the near and far fields (calculated by COLMAP) of the ray, and select the coordinates $P_{mid}$ of the intermediate sampling point, along with the view direction $d$, as the input of N-NeRF. Since the **bounds of the ray do not change** during the training phase, the midpoints for each ray (one-to-one map with pixels) is same.

We will incorporate all these information into our revision.
Thank you for the response to my comments. 


**W1: Insufficient Dataset Contribution.**

For the discussion on the dataset, I am still not satisfied, I would not attribute the paper any significant contribution for selecting views for 5 synthetic and 5 captured scenes from an existing dataset and estimating the camera parameters, even though I appreciate COLMAP is not always straightforward to use.


**W4: Relation between CTP module and DP-NeRF.**

Thank you very much for this explanation, that helps me better understand the difference between the two modules. But frankly, I think even this explanation is not enough for what the RBK module exactly does, I think its common practice to explain properly the methods used if they are non-standard even if they come from other papers. I would implore the authors to add a more thorough description, even if just in the supplement. 


**Q1. Why is the noise modeled with a NeRF?**

This begs the question, why the authors don't use the (x, y, N) based formulation. Isn't this formulation more efficient and understandable? Presumably, this MLP is smaller, and since there is no spatial consistency required, it makes more sense to use it than a 3D-conditioned MLP? 

EDIT: Maybe I am misunderstanding something, what is the ray midpoint the author's query? This changes as per the random sampling, is that right? So for the same pixel, I could be querying a different midpoint in different iterations? 

Otherwise, I thank the authors for their responses, I am satisfied with the answers provided to all other questions. I think the discussion on the low-pass filter is especially useful, since the Freq. Domain Thresholding is one of the main contributions to the CTP module. I would implore the authors to add these results to the paper.
Thanks very much for your scores and review comments, your detailed comments effectively help us to improve the quality of manuscript.

We promise to incorporate all these missing details in the revision.
We're very glad that our rebuttal has addressed your concerns, and thank you for recognizing the paper, we'll add the missing details in the revision.
Thanks a lot for your acknowledgement and valuable comments, we promise to incorporate all these missing details in the revision.
Thanks for your thorough rebuttal! Please include these missing details in the final version (especially the ablation results) to help deliver a more convincing set of experiments. I will maintain my Weak Accept rating as I see most other reviewers are in agreement.
Thank you for the clarification. I am increasing the rating to 'weak accept'.
Thanks for the reply. We are glad that our rebuttal could address your concerns.

 $n_{\frac{N}{2}} = MLP_{N-NeRF}(P_{mid}, d)$ is correct. We will correct $n$ into $n_{\frac{N}{2}}$ in Line 159 and check elsewhere to maintain the consistency of symbols in our revision.
Thank you for your response. Most of my concerns are addressed by the authors. But I have one small doubt in the response.

In the updated Eq. 5 given in W1Part1 response, $MLP_{N-NeRF}(P_{mid}, d) = n N/2$. But Line 159 of the paper says that '$n$ is the noise value rendered by N-NeRF'. Which one is correct? $MLP_{N-NeRF}(P_{mid}, d) = n N/2$ or $MLP_{N-NeRF}(P_{mid}, d) = n$
Thank you for your response. My concerns have been fully resolved, and this is an excellent work. I improve my rank to weak accept.