Self-Supervised Image Restoration with Blurry and Noisy Pairs
Abstract
When taking photos under an environment with insufficient light, the exposure time and the sensor gain usually require to be carefully chosen to obtain images with satisfying visual quality.
For example, the images with high ISO usually have inescapable noise, while the long-exposure ones may be blurry due to camera shake or object motion.
Existing solutions generally suggest to seek a balance between noise and blur, and learn denoising or deblurring models under either fullor self-supervision.
However, the real-world training pairs are difficult to collect, and the self-supervised methods merely rely on blurry or noisy images are limited in performance.
In this work, we tackle this problem by jointly leveraging the short-exposure noisy image and the long-exposure blurry image for better image restoration.
Such setting is practically feasible due to that short-exposure and longexposure images can be either acquired by two individual cameras or synthesized by a long burst of images.
Moreover, the short-exposure images are hardly blurry, and the long-exposure ones have negligible noise.
Their complementarity makes it feasible to learn restoration model in a self-supervised manner.
Specifically, the noisy images can be used as the supervision information for deblurring, while the sharp areas in the blurry images can be utilized as the auxiliary supervision information for self-supervised denoising.
By learning in a collaborative manner, the deblurring and denoising tasks in our method can benefit each other.
Experiments on synthetic and real-world images show the effectiveness and practicality of the proposed method.
Codes are available at https://github.com/cszhilu1998/SelfIR .
Introduction
It is a common yet challenging task to acquire visually appealing photos with appropriate brightness under a low light environment.
Traditional ways to increase the image brightness include enlarging the aperture, adopting a higher ISO, and reducing the shutter speed ( i.e ., lengthening exposure time).
As for smartphone cameras with fixed aperture, brightness can only be adjusted by setting the sensor gain ( i.e ., ISO) and exposure time.
Nonetheless, they are negatively correlated to maintain the appropriate brightness level of the image, i.e ., the shorter-exposure image generally adopts a higher
ISO, while the longer-exposure image usually has a lower ISO.
Moreover, high ISO configuration introduces inevitable and complex noise due to the limited photon amount and the process of camera image signal processing (ISP) pipeline, while long-exposure is prone to produce blurry images due to the camera shake and scene variations.
Consequently, photographers have to make a compromise between noise and blur.
Recent advances in image restoration make it possible to further improve the visual quality of the acquired low light images by leveraging deep image denoising or deblurring networks.
Taking supervised denoising as an example, synthetic or real-world noisy-clean image pairs are required to train the deep networks [9,18,47–49,52,53].
However, the models trained with synthetic training pairs are hard to generalize to real noisy images, and the real-world clean reference images are usually obtained by averaging hundreds of noisy ones [1] or with complicated capturing and processing procedure [29], making the collection of large scale real-world training datasets laborious, expensive, and time-consuming.
Such problems greatly limit the deployment of models on more devices with different noise distributions.
Alternatively, a surge of self-supervised image denoising methods [11, 13,15–17,26,27,37,39,41,43,55,57] have been developed to avoid the collection of ground-truth (GT) training images, yet are limited in handling complex real-world image noise.
Another possible solution is to perform motion deblurring on long-exposure images.
Early explorations [8,20,31,34,35,45] are mainly given to spatially uniform deblurring caused by camera motion.
Recently, the proposal of non-uniform datasets ( e.g ., GoPro [25], REDS [24], HIDE [36], RealBlur [32], etc .) has greatly boosted the research of deblurring in more practical scenes involving both camera shake and object motion [6,25,38,50,51].
In this paper, we suggest to improve low light imaging by jointly leveraging the short-exposure noisy and long-exposure blurry images.
First, such setting is practically feasible.
For example, multiple cameras have been equipped in modern smartphones, which can be designed to acquire short-exposure and long-exposure images, respectively.
Moreover, one can also synthesize a pair of blurry and noisy images from a long burst of images captured by a camera.
Second, the noisy and blurry images convey complementary information, which is beneficial to improve restoration performance and makes self-supervised image restoration (SelfIR) possible.
We note that several methods [5,14,23,46] have been suggested to combine the blurry image with their noisy counterpart for better image restoration, yet it remains uninvestigated under the self-supervised regime.
We further present a SelfIR model with blurry and noisy pairs.
Even though the blurry and noisy images are both disturbed , the short-exposure images taken with high ISO are hardly blurry , while the long-exposure images taken with low ISO are generally near noise-free .
Thus, the long-exposure and short-exposure images can be used to provide some supervision information for each other.
On the one hand, the noisy image can serve as an alternative of sharp image to supervise deblurring with negligible performance degradation.
On the other hand, the static regions in long-exposure images are noise-free and sharp, which in turn can provide auxiliary supervision information for image denoising.
Taking these two aspects into account, we present a collaborative learning (co-learning) method termed SelfIR for deblurring and denoising, which is effective in leveraging the complementary information of long- and short-exposure images and can be learned in a self-supervised manner.
Extensive experiments on synthetic data are conducted to evaluate our SelfIR.
Both quantitative and qualitative results show that SelfIR outperforms the state-of-the-art self-supervised denoising methods, as well as the supervised denoising and deblurring counterparts.
To further verify the practicality of our SelfIR model, we have also collected a set of 61 real-world blurry and noisy pairs using smartphones.
Since there are no corresponding ground-truth images for calculating full-reference image quality assessment (IQA) metrics, we evaluate the restoration results using no-reference IQA metrics.
The results show that our method also performs favorably against the competing methods.
To sum up, the main contributions of this work include:
• We take a step forward in leveraging blurry and noisy image pairs for image restoration.
Going beyond leveraging their complementarity in improving restoration performance, we show that it can also be utilized for self-supervised learning of the restoration model.
• A self-supervised image restoration model (SelfIR) is proposed, where short-exposure images serve as supervision for the corresponding deblurring task, while the sharp regions in long-exposure images provide auxiliary supervision for self-supervised denoising.
• Extensive experiments on both synthetic and real-world image pairs show that our SelfIR performs favorably against the state-of-the-art self-supervised denoising methods, as well as the baseline supervised deblurring and denoising methods.
Related Work
In this section, we briefly review burst image denoising and deblurring, as well as self-supervised image denoising and deblurring methods.
In addition, we recommend [7] for a comprehensive introduction to the relevant mobile computational photography.
Burst Image Denoising and Deblurring.
In comparison with a single image, burst images can provide more information that is beneficial for image restoration.
Hasinoff et al .
[10] utilize an
FFT-based alignment algorithm and a hybrid 2D/3D Wiener filter to denoise and merge a burst of underexposed frames for low-light photography.
KPN [21] predicts spatially variant kernels for every burst noisy image to merge them.
BPN [44] extends the KPN method with a basis prediction network and achieves larger denoising kernels under certain computing resource constraints.
Aittala et al .
[2] take both noise and blur into account, and restore sharp and noise-free images from burst images in an order-independent manner.
Self-Supervised Image Denoising and Deblurring.
Recently, self-supervised learning has drawn upsurging attention in low-level vision.
DIP [39] utilizes the image prior implicitly captured by the network structure to repair corrupted images.
SelfDeblur [30] respectively models the deep priors of clear image and blur kernel for self-supervised deblurring.
For these methods, the networks are required to re-train from scratch for each test image, which is less efficient, especially for mobile or edge devices.
Noise2Noise [17] demonstrates that noisy pairs with mutually independent noise can be used to train a denoising network, opening the door to self-supervised denoising.
Neighbor2Neighbor [11] utilizes a random neighbor sub-sampler to generate the training pairs from noisy images themselves.
In addition, some works [13, 15, 16, 41, 43] elaborately design blindspot networks to avoid learning the identity mapping for self-supervised denoising.
However, the self-supervised denoising methods are limited in handling complex image noise.
In this work, we utilize the complementarity of long-exposure blurry and short-exposure noisy images for better self-supervised image restoration.
Proposed Method
In this section, we first show the feasibility of taking noisy images as the supervision of deblurring.
Then, we introduce the sharp area detection method in long-exposure images and auxiliary loss for self-supervised denoising.
Finally, we present the proposed co-learning framework SelfIR.
3.1 Deblurring with Noisy Image
When taking long-exposure photos, the shake of the camera and the motion of objects usually lead to a blurry image I B , which can be formulated by,
[Equation]
where I is the latent clear image, K denotes the blur process with non-uniform kernels, N B represents the low-intensity noise.
Existing supervised deblurring methods generally utilize a deep neural network (denoted by D B ) to estimate I from I B .
For training the parameters of D B , which is denoted by Θ B , the optimization objective can be defined by,
[Equation]
where L denotes the loss functions for supervised learning.
However, collecting clear images is troublesome in real-world scenes.
Inspired by Noise2Noise [17], we show that the noisy shortexposure image can be a substitution of the latent clear image to supervise the task of deblurring.
When taking short-exposure photos under low light environment, the limited photon amount and inherent defects of camera ISP make the images noisy (denoted by I N ), which can be formulated by,
[Equation]
where N N represents the noise (with much higher intensity than N B ).
When using the noisy image
I N as the supervision of deblurring, the optimization of Θ B can be expressed as,
[Equation]
Suppose that the loss function L in Eqn.
(4) is the ℓ 2 loss, we have,
[Equation]
where E N N | I B (cid:2)
∥ N N ∥ 22 (cid:3) can be regarded as a constant and be safely discarded from Eqn.
Further, assume that N N is zero-mean, N N and I are independent, we can get,
[Equation]
In this case, the optimal solution Θ ∗B in Eqn.
(4) and that in Eqn.
(2) are the same.
Thus, it is feasible to utilize noisy short-exposure images instead of clear ones as the supervision of deblurring.
3.2 Denoising with Long-Exposure Image
[Figure]
Self-supervised denoising makes it possible to remove the noise without clean image supervision, but may give rise to obvious performance degradation, especially when handling complex real-world noises.
Here we propose to alleviate this problem by introducing some extra supervision from the long-exposure counterpart.
Obviously, taking the whole long-exposure image as supervision will bring adverse effects, making the results to be blurry.
Nonetheless, it is worth noting that, the blur process K in Eqn.
(1) is generally non-uniform, and sometimes not all areas are blurry.
As shown in Fig. 1(d), there exist some approximately sharp regions in the long-exposure image, which can provide partial supervision information that benefits self-supervised denoising.
Therefore, it is crucial to pinpoint the sharp areas in the long-exposure image.
Otherwise, we would prefer to go without sharp areas than accept a shoddy option, as misjudgments of sharp areas will lead to worse denoising results.
However, without any discriminative clues, it is very likely to misjudge the sharp areas.
Considering that I N is nearly non-blurry due to the short exposure time, it may be a reference for sharp area detection.
To avoid noise interference, we pre-process I N with a selfsupervised denoising model, and take the result ˜
I N to help detect sharp regions in the corresponding blurry image I B .
Specifically, we first divide I B and ˜I N into N non-overlapping patches.
For each patch pair I n B and ˜I n N ( 1 ≤ n ≤ N ), our goal is to obtain a mask m n ∈ { 0 , 1 } that indicates whether I n B is a sharp patch.
Since ˜I N is nearly non-blurry, when some severe motion blurs exist in I n B , the difference between I n B and ˜I n N in textures and edges should be evident.
Taking the above into account, we adopt a similarity metric s to detect the areas with severe motion blurs, i.e .,
[Equation]
where structural similarity (SSIM)
[40] is utilized for the similarity metric s , ϵ s denotes the threshold, while max ( a, b ) and sgn ( ·
However, the initial denoising result ˜I N may be over-smooth, in other words, Eqn.
(7) may fail when facing some mildly blurred regions in I n B .
Therefore, we further measure the difference in variance between I n B and ˜I n N .
When the variance of I n B is greater than that of ˜I n N , we consider that I n B is potential to be a sharp patch.
It should be noted that the difference in variance is not suitable for
Restoration network 𝑎𝑏 3 𝑎𝑏 1 𝑏 2 𝑎𝑎 4 𝑏 4 𝑎𝑏𝑏𝑎𝑎 8 𝑏𝑏𝑏 7 𝑎𝑎𝑎 1 𝑎 2 𝑎𝑎 4 𝑎 5 𝑎𝑎 7 𝑎 8 𝑎𝑏 1 𝑏 2 𝑏𝑏 4 𝑏 5 𝑏𝑏 7 𝑏 8 𝑏
Sub-samplingoperation Sub-samplingoperation
Neighboring Sub-Images
Restorationnetwork
Blurry image
Noisy image
Blurry image
Noisy image
Output (a) Training (
[Figure]
Output g 1 ( · ) g 1 ( · ) g 2 ( · ) g 1 ( · ) g 2 ( · ) ( I B ) ( I N ) g 1 ( I B ) g 1 ( I N ) g 2 ( I N )
L aux Eqn.
L rec & L reg Eqns.
(10) and (11)
D
[Figure]
detecting some severely blurry areas with ringing artifacts (see Fig. 1(c)).
In such blurry regions, the variance of I n B may also be greater than that of ˜I n N .
When synthesizing blurry images, some works [24,42,56] remove the artifacts by interpolating the frames before averaging the sharp images.
However, the artifacts also exist in real-world blurry images, especially in areas with flickering lights.
Therefore, we still take the ringing artifacts into consideration in this work.
As a result, we jointly use the SSIM and variance measure for judging sharp regions, m n can be formulated as,
[Equation]
where var ( · ) and ϵ v denote the variance function and the threshold, respectively.
ϵ s and ϵ v are set to 0.99 and 1e-5 when color intensity values of I n B and ˜I n N are normalized to [0 , 1] .
Defining the final output image as ˆ I , the auxiliary loss function for denoising can be denoted as,
[Equation]
3.3 Co-Learning of Deblurring and Denoising
As illustrated in Secs.
3.1 and 3.2, the noisy images can provide effective supervision information for deblurring, while the blurry images can also provide auxiliary supervision for self-supervised denoising.
In other words, we can learn to deblur or denoise without additional clear and clean ground-truths.
However, learning them independently will lead to limited performance.
As some works [5,14,23,46] have suggested, better restoration performance can be obtained by incorporating blurry with noisy images.
Thus, it should be further considered to achieve self-supervised image restoration that takes two disturbed images as input.
In this section, we delicately design the selfsupervised model SelfIR that learns from both deblurring and denoising tasks in a collaborative learning manner.
For training SelfIR, it is required to avoid the trivial solution when taking both blurry and noisy images as the input.
For example, when taking noisy images for supervising deblurring, the model may simply output the noisy images instead of learning the deburring results.
Fortunately, Neighbor2Neighbor [11] shows the noises in two sub-sampled images ( i.e ., g 1 ( I N ) and g 2 ( I N ) , see Fig. 2(b)) from noisy image
I N are almost independent.
Therefore, they can be respectively taken as the input and target for training the denoising subtask in a self-supervised manner.
Simultaneously, combining the derivation in Eqn.
(5), g 1 ( I B ) and g 2 ( I N ) can be respectively taken as the input and target to train the deblurring subtask.
Therefore, our SelfIR can take a step forward and deliver both g 1 ( I B ) and g 1 ( I N ) ( i.e. , { g 1 ( I B , g 1 ( I N ) } ) into the restoration network D , as shown in Fig. 2(a).
The sub-sampled image g 2 ( I N ) is taken for calculating reconstruction loss L rec , which can be written as,
[Equation]
We also calculate regularization loss L reg similar to [11],
[Equation]
where ˆ D has same parameters with D but has no gradient for back-propagation.
Moreover, the sharp areas in g 1 ( I B ) can provide auxiliary supervision for the model learning and the performance can be further improved.
Specifically, we calculate auxiliary loss L aux ( D ( g 1 ( I B ) , g 1 ( I N ))) , g 1 ( I B )) in Eqn.
For obtaining the mask m n , we replace I B and ˜I N in Eqn.
(8) with g 1 ( I B ) and ˆ D ( g 1 ( I B ) , g 1 ( I N )) , respectively.
Finally, the learning objective of parameters Θ D with the colearning manner can be formulated as,
[Equation]
where λ reg and λ aux are hyper-parameters for balancing the loss terms.
Experiments
In this section, we first describe the dataset configuration and the training / evaluation protocols in detail.
Then, both the quantitative and qualitative results on synthetic and real-world images are given for comprehensive evaluation.
4.1 Implementation Details
Synthetic Datasets.
For synthesizing blurry images, early methods [23,46] convolve sharp images with simulated blur kernels, which is quite different from the real-world blur model.
Recently, the
GoPro dataset 2 [25] offers a more realistic way to synthesize blurry images, which has been widely adopted for motion deblurring tasks [6,25,38,50,51].
In the dataset, although slight inherent noise exists in the sharp video frames that are captured by a high-speed camera ( i.e ., GoPro Hero4 Black), it has little effect on model training and evaluation.
We directly regard the sharp images as clean images.
The blurry image is generated by averaging consecutive sharp frames, which can better simulate both camera shake and object motion.
Thus, we can get the blurry-clear pairs as I B and I , respectively.
For synthesizing the noisy image I N , we consider three noise distributions: 1) Gaussian noise with σ ∈
Poisson noise with λ ∈ [5 , 50] , and 3) sensor noise [4].
Please refer to the supplementary material for the noise formulations.
Experiments are conducted in both sRGB and raw-RGB spaces.
For the sRGB space, we add Gaussian or Poisson noise to the sRGB sharp images.
While for the raw-RGB space, the more complex sensor noise is added to the sharp raw-RGB images, and the blurry images are also converted back to the raw-RGB space through the unprocessing [4] pipeline.
Finally, there are 2,103 image pairs for training, and we use the remaining 1,111 pairs for testing.
Real-world Datasets.
Furthermore, we have also collected 61 real-world blurry and noisy raw-RGB pairs with Huawei P40 smartphones in the professional camera mode.
In particular, the blurry images are captured with a low ISO and long exposure time, while the corresponding noisy images are captured with a high ISO and short exposure time in the same scene.
Please refer to the supplementary material for detailed collection process.
Due to the limited amount of real-world images, we usepairs to fine-tune the model pre-trained on raw-RGB images with synthetic sensor noise, and take the remaining 31 pairs for evaluation.
[Table]
Training Details.
Following [11, 15, 17], we adopt a U-Net [33] architecture as our restoration network, where two encoders are respectively deployed to the blurry and noisy input images for better domain-specific feature extraction, and then blurry and noisy features are fused in the decoder part.
Detailed architecture is given in the supplementary material.
During training, the batch size is set to 16 and the patch size is 128 × 128 .
Adam optimizer [12] with β 1 = 0 .
9 and β 2 = 0 .
999 is used to train the network for 200 epochs.
The learning rate is initially set to 3 × 10 − 4 for synthetic experiments and 1 × 10 − 4 for real-world experiments.
And it reduces by half every 50 epochs.
For the hyper-parameters in Eqn.
(12), λ aux is set to 2, λ reg is set to 2 and 4 for experiments in sRGB space and raw-RGB space, respectively.
All experiments are conducted with PyTorch [28] on an
Nvidia GeForce RTX 2080Ti GPU.
Evaluation Configurations.
We convert all raw-RGB results to sRGB space through post-processing pipeline, and the quantitative metrics are computed in the sRGB space.
For the results of synthetic experiments, we take peak signal to noise ratio (PSNR), structural similarity (SSIM) [40] and learned perceptual image patch similarity (LPIPS) [54] as evaluation metrics.
For the results of real-world experiments, due to the lack of ground-truth images, we utilize no-reference IQA metrics ( i.e .,
NIQE [22], NRQM [19], and PI [3]) to evaluate the generated images.
4.2 Experimental Results in sRGB Space
To assess our proposed SelfIR, we build several baseline methods for comparison, including 1) a supervised deblurring baseline (denoted by Baseline B ), 2) a supervised denoising baseline (denoted by Baseline N ), 3) a supervised image restoration baseline which takes both blurry and noisy images as input (denoted by Baseline R ), and 4) Neighbor2Neighbor
[11] since the denoising part of our co-learning framework is based on it.
Note that the network structure is not the focus of this paper, so we deploy a simple U-Net [33] architecture for our restoration network D and the first three baseline methods.
Besides, we choose two classical supervised methods ( i.e ., DeepDeblur [25] and DnCNN [52]) for comparison.
Self-supervised denoising methods ( e.g ., N2N [17], N2V [13],
Laine19 [15], DBSN [43], R2R [27], and Blind2Unblind [41]) are also compared.
For DeepDeblur [25], we use the officially released model for testing.
For other methods, the models are retrained with our synthetic images for a fair comparison.
The quantitative results of synthetic experiments on Gaussian and Poisson noise are given in
Tab. 1.
One can see that our SelfIR outperforms all other methods except the supervised IR model (Baseline R ), which can be seen as the upper limit of the restoration performance with the utilized
U-Net architecture and the blurry-noisy pair input.
In comparison to the self-supervised denoising baseline Neighbor2Neighbor [11], 1.45 dB and 1.59 dB PSNR gains are obtained by our SelfIR on Gaussian and Poisson noise, respectively.
Even compared to the supervised denoising baseline
[Figure] [Figure]
model (Baseline N ), our method also improves the PSNR index by 0.83 dB and 1.12 dB.
Due to the severely ill-posed nature of deblurring, SelfIR brings a PSNR gain by more than 6 dB comparing to the supervised deblurring baseline model (Baseline B ).
The results clearly show that the blurry images can indeed provide beneficial features for denoising, and so do the noisy images for deblurring.
The qualitative results are shown in Figs.
3 and 4.
One can see that our results are sharper and clearer than other denoising or deblurring methods.
In terms of visual quality, the proposed SelfIR is almost comparable to the supervised image restoration model (Baseline R ), and combines the advantages of supervised denoising Baseline N and deblurring Baseline B .
More results can be found in the supplemental material.
4.3 Experimental Results in Raw-RGB Space
Tab. 2 shows the quantitative results of synthetic and real-world experiments on raw-RGB images.
For synthetic experiments with more complex and realistic sensor noise [4], all models are retrained with our synthetic images.
Our SelfIR achieves a 1.69 dB PSNR gain in comparison with Neighbor2Neighbor [11].
For evaluation on real-world images, we directly use the models of N2N [17],
Laine19-pme [15], R2R [27] and all supervised methods, which are pre-trained for sensor noise.
For SelfIR and other self-supervised methods, we fine-tune the pre-trained model on our real-world training set.
The PI [3] metric of SelfIR is improved by 0.55 on real-world testing images through fine-tuning.
And the results on no-reference IQA show that our method is very competitive in comparison with the competing methods on real-world images.
The visual results are given in the supplemental material.
[Table] [Table]
[Table]
Ablation Study 5.1 Feasibility of Deblurring with Noisy Images
In order to verify the feasibility of taking noisy images as the supervision of deblurring in Sec. 3.
[Table]
we replace the clear supervision of Baseline B with noisy images of different noise distributions
[Table]
and the results are shown in Tab. 3.
It can be seen that taking noisy or clear images as the supervision leads to similar performance on the deblurring task
[Table]
which confirms that the noisy images can be an alternative of clear images to supervise the deblurring task
[Table]
5.2 Effect of Auxiliary Loss
In order to evaluate the effect of auxiliary loss L aux in Eqn.
[Table]
we add L aux to the loss terms of selfsupervised denoising method Neighbor2Neighbor [11].
As shown in Tab. 4, we obtain 0.16 dB PSNR gain against the baseline Neighbor2Neighbor [11].
The result indicates that blurry images can provide some auxiliary supervision information and improve performance for self-supervised denoising.
When removing L aux from our SelfIR, the PSNR dropped by 0.09 dB. Since the long-exposure image is taken directly as input, the loss term may be less effective, but is still beneficial.
[Table]
[Table]
5.3 Effect of Different Loss Weights
We conduct ablation studies on different weighting hyper-parameters ( i
[Table]
e ., λ reg and λ reg ) for balancing regularization and auxiliary loss terms.
The experiments are conducted on Gaussian noise in sRGB space.
When varying one hyper-parameter, the other one is set to 2 by default.
As shown in
Tabs.
5 and 6, it can be seen that the sensitivity to λ reg and λ reg of SelfIR is acceptable.
Conclusion
The complementarity between long-exposure blurry and short-exposure noisy images not only improves the performance of image restoration, but also makes it possible to learn a restoration model in a self-supervised manner.
Jointly leveraging the long- and short-exposure images, we present a self-supervised image restoration method named SelfIR.
On the one hand, we take the short-exposure images as the supervision information for deblurring.
On the other hand, we utilize the sharp areas in the long-exposure images as auxiliary supervision information in aid of self-supervised denoising.
SelfIR combines these two aspects by a collaborative learning scheme, and makes the deblurring and denoising tasks benefit from each other.
Experiments on synthetic and real-world images show the effectiveness and practicality of our proposed method.
Limitation, Impact, Etc
This work is still limited in assessing the results on real-world image pairs.
Although no-reference
IQA metrics are adopted, these may be too unstable to report the performance consistent with humans accurately.
We will establish a real-world blurry-noisy pair dataset including high-quality GTs to solve this problem in the future work.
As for societal influence, this work is promising to be applied to terminal devices ( e.g ., smartphones) for obtaining better images under low-light environments.
It has no foreseeable negative impact.
Besides, the images used in this work are from natural or human scenes.
The GoPro dataset used for synthetic experiments is public under CC BY 4.0 license.
There is no personally identifiable information or offensive content in the experimental data.
Acknowledgement
This work was supported by the Major Key Project of Peng Cheng Laboratory (PCL2021A12) and
the
National Natural Science Foundation of China (NSFC) under Grants No.s U19A2073 and 61871381.