Exposure Bracketing Is All You Need For A High-Quality Image
Authors: Zhilu Zhang
Shuohao Zhang
Renlong Wu
Zifei Yan
Wangmeng Zuo
Venue: ICLR 2025 Poster (2025)
Keywords: Exposure Bracketing
Image Restoration and Enhancement
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose to utilize exposure bracketing photography to get a high-quality image by combining denoising, deblurring, high dynamic range imaging, and super-resolution tasks.

Abstract: It is highly desired but challenging to acquire high-quality photos with clear content in low-light environments. Although multi-image processing methods (using burst, dual-exposure, or multi-exposure images) have made significant progress in addressing this issue, they typically focus on specific restoration or enhancement problems, and do not fully explore the potential of utilizing multiple images. Motivated by the fact that multi-exposure images are complementary in denoising, deblurring, high dynamic range imaging, and super-resolution, we propose to utilize exposure bracketing photography to get a high-quality image by combining these tasks in this work. Due to the difficulty in collecting real-world pairs, we suggest a solution that first pre-trains the model with synthetic paired data and then adapts it to real-world unlabeled images. In particular, a temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed. Moreover, we construct a data simulation pipeline to synthesize pairs and collect real-world images from 200 nighttime scenarios. Experiments on both datasets show that our method performs favorably against the state-of-the-art multi-image processing ones. Code and datasets are available at https://github.com/cszhilu1998/BracketIRE.

Published as a conference paper at ICLR 2025
EXPOSURE BRACKETING Is ALL YOU NEED FOR A HIGH-QUALITY IMAGE
Zhilu Zhang, Shuohao Zhang, Renlong Wu, Zifei Yan; Wangmeng Zuo
Harbin Institute of Technology, Harbin, China
csz zhang@out look. com, yhyzshrby@ com, hi hirrenongwwu@gmal com, (yanzifei, wmzuo) @hit edu. en
ABSTRACT
It is highly desired but challenging to acquire high-quality photos with clear content in low-light environments. Although multi-image processing methods (using burst, dual-exposure, or multi-exposure images) have made significant progress in addressing this issue, they typically focus specific restoration enhancement problems, and do not fully explore the potential of utilizing multiple complementary
images. Motivated by the fact that multi-exposure images in denoising, deblurring, high dynamic range imaging, and super-resolution, we propose to utilize exposure bracketing photography to get a high-quality image by combining these tasks in this work. Due to the difficulty in collecting real-world pairs, we suggest solution that first pre-trains the model with synthetic paired data and then adapts it to real-world unlabeled images. In particular, temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed. Moreover, we construct a data simulation pipeline to synthesize pairs and collect real-world images from 200 nighttime scenarios. Experiments on both datasets show that our method performs favorably against the state-of- the-art multi-image processing ones. Code and datasets are available at https: / //gi thub. - com/ cwwhiiihhllluu 998 /Brra//raeeetttee
INTRODUCTION
In low-light environments, capturing visually appealing photos with clear content presents a highly desirable yet challenging goal. When adopting a low exposure time, the camera only captures a small amount of photons, introducing inevitable noise and rendering dark areas invisible. When taking high exposure time, camera shake and object movement result in blurry images, in which bright areas may be overexposed. Although single-image restoration (e.g., denoising (Zhang et al., 2017; 201 8b; Guo et al., 2019; Brooks et al., 2019; Zamir et al., 2020; Abdelhamed et al., 2020; Li et al., 2023), deblurring (Nah et al., 2017; Zhang et al., 2018a; Tao et al., 2018; Cho et al., 2021; Zamir et al., 2021; Mao et al., 2023), and super-resolution (SR) (Dong et al., 2015; Lim et al., 2017; Zhang et al., 2018e;c; Liu et al., 2020a; Liang et al., 2021; Ledig et al., 2017)) and enhancement (e.g., high dynamic range (HDR) reconstruction (Filertsen et al., 2017; Liu et al., 2020b; Zou et al., 2023; Pérez-Pellitero et al., 2021; Lee et al., 2018; Chen et al., 2023)) methods have been extensively investigated, their performance is constrained by the severely ill-posed problems.
Recently, leveraging multiple images for image restoration and enhancement has demonstrated potential in addressing this issue, thereby attracting increasing attention. We provide a summary of several related settings and methods in Tab. 1. For example, some burst image restoration methods (Bhat et al., 2021a;b; Dudhane et al., 2022; Luo et al., 2022; Lecouat et al., 2021; Mehta et al., 2023; Dudhane et al., 2023; Bhat et al., 2023; Wu et al., 2023; Bhat et al., 2022) utilize multiple consecutive frames with the same exposure time as inputs, being able to perform SR and denoising. The works based dual-exposure images (Yuan et al., 2007: Chang et al., 2021; Mustaniemi et al., 2020; Zhao et al., 2022; Zhang et al., 2022b; Shekarforoush et al., 2023; Lai et al., 2022) combine the short-exposure noisy and long-exposure blurry pairs for better restoration. Multi Multi-exposure images are commonly employed for HDR imaging (Kalantari et al., 2017; Yan et al., 2019; Prabhakar et al.,
"Corresponding Author.

---

Published as a conference paper at ICLR 2025
Table 1: Comparison between various multi-image processing manners
Supported Tasks
Setting
Methods
Input Images
Denoising Deblurring IIDR SR
(Godard al., 2018, Xia ct.l., 2020, Rong al, 2020; Gio et 2022)
Burst Demoising
Burst
(Wieschotlek et al, 2017; Polla et 2019;
Burst Dehluming
Alttala & Durand, 2018)
Burst
(Deudon et all, 2020; Wronski et al,, 2019;
Burst SR
Burst
Wei et 2023y
(Bhat et al, lagh Dudhane
2022;
Burst Denoising and SR
Luo ot al, 2022, Lecouat ot ol., 2021; Mehia
2023; Dodhane al, 2023; Bhat et
Burst
2023; Wa et al, 2023; Bhat ot al., 2022)
Burst Denoising and HDR
(Hasinofl et al, 2016; Emst & Wronski, 2021)
Burst
(Chang et 2021; Mustaniemi al., 2020;
Dual- Dual- Exposure Image Restoration
Zhap et al, 2022; Zhang et al, 2022b;
Dual- Expouuee
Slickrforoush al, 2023, Lai al, 2022)
(Kalantari et at, 2017; Yan 2019; Niu et al., 2021; Liu et al., 2022; Yan et al. 2023s;
Basic HDR Imaging
Multi- Maloo-Expouure
Tel 2023; Zhang 2024b) (Hasinoff et al., 2010; Liu et all, 2023; Chi
HDR Imaging with Denoising
et al,, 2023; Pérez. Pellitero et al,, 2021)
Multi-Exposure
HDR Imaging with SK
(Tan al, 2021)
Multi-Exposure Malti- Exposure
HDR Imaging with Denoising and SR
(Lecouat et al. 2022)
Our BracketRE
Malti Expooure
Our BracketIKE+
Multi- Expoi-Exposure
2019; Wu et al., 2018; Niu et al., 2021; Liu et al., 2022; Yan et al., 2023a; Tel et al., 2023; Zhang et al., 2024b; Song et al., 2022).
Nevertheless, in night scenarios, remains unfeasible to obtain noise-free, blur-free, and HDR images when employing these multi-image processing methods. On the one hand, burst and dual-exposure images both possess restricted dynamic ranges, constraining the potential expansion of the two manners into HDR reconstruction. On the other hand, most HDR reconstruction approaches based on multi-exposure images are constructed with the ideal assumption that image noise and blur are not taken into account, which results in their inability to restore degraded images. Although recent works (Liu et al., 2023; Chi et al., 2023; Lecouat et al., 2022) have combined with denoising task, blur in long-exposure images has not been incorporated into them, which is still inconsistent with real-world exposure images.
In fact, considering all multi- nulti-exposure factors (including noise, blur, underexposure. overexposure. and misalignment) is not only beneficial to practical applications, but also offers us an opportunity to combine image restoration and enhancement tasks to get a high-quality image. First, the indepen- dence and randomness of noise (Wei et al., 2020) between images allow them to assist each other in denoising, and its motivation is similar that of burst denoising (Mildenhall al., 2018; Godard et al., 2018; Xia et al., 2020; Rong et al., 2020; Guo et al., 2022). In particular, demonstrated in dual-exposure restoration works (Yuan et al., 2007; Chang et al., 2021; Mustaniemi et al., 2020; Zhao et al., 2022; Zhang al., 2022b; Shekarforoush et al., 2023; Lai et al., 2022), long-exposure images with a higher hignal-to-niise ratio can play a significantly positive role in removing noise from the short-exposure images. Second, the shortest-exposure image be considered blur-free. It can offer sharp guidance for deblurring longer-exposure images. Third, underexposed areas in the short-exposure image may be well-exposed in the long-exposure one, while overexposed regions in the long-exposure image may be clear in the -hoot-exposure one. Combining multi-exposure images makes HDR imaging easier than single-image enhancement. Fourth, the sub-pixel shift between multiple images caused by camera shake or motion is conducive to fellt--raameeee SR (Wronski et al., 2019). In summary, leveraging the complementarity of multi-exposure images offers the potential to integrate the four problems (i.e., denoising, deblurring, HDR reconstruction, and SR) into a unified framework that can generate noise-free, blur-free, high dynamic range, and high-resolution image.
Specifically, in terms of tasks, we first utilize bracketing photography to combine basic restoration (i.e., denoising and deblurring) and enhancement (i.e., HDR reconstruction), named BracketRE.. Then we append the SR task, dubbed BracketRE++ as shown in Tab. 1. In terms of methods, due to the difficulty of collecting real-world paired data, we achieve that through supervised pre- tre-training on synthetic pairs and self- supervisdd adaptation real- weal-world images. On the one hand, we adopt the recurrent network manner as the basic framework, which is inspired by its successful applications in processing sequence images, e.g., burst (Guo al., 2022; Rong al., 2020; Wu et al., 2023)
2

---

Published as a conference paper at ICLR 2025
and video (Wang et al., 2023b; Chan et al., 2021; 2022) restoration. Nevertheless, sharing the same restoration parameters for each frame may result in limited performance, as degradations (e.g.. blur, noise, and color) vary between different multi-exposuue images. To alleviate this problem, we propose a temporally modulated recurrent network (TMRNet), where each frame not only shares some parameters with others, but also has its own specific ones. On the other hand, pre-trained TMRNet on synthetic data has limited generalization ability and sometimes produces unpleasant artifacts in the real world, due to the inevitable gap between simulated and real images. For that, we propose a self-supervised adaptation method. In particular, we utilize the temporal characteristics of eulti-exposure image processing to design learning objectives to fine-tune TMRNet.
For training and evaluation, we construct pipeline for synthesizing data pairs, and collect real-world images from 200 nighttime scenarios with a smartphone. The two datasets also provide benchmarks for future studies. We conduct extensive experiments, which show that the proposed method achieves state-of-the- ahr performance in comparison with other multi-image processing ones.
The contributions can be summarized as follows:
We propose to utilize exposure bracketing photography to get a high-quality (i.e., noise-free, blur-free, high dynamic range, and high-resolution) image by combining image denoising. deblurring. high dynamic range reconstruction, and super-resolution tasks.
We suggest solution that first pre-trains the model with synthetic pairs and then adapts it to unlabeled real-worle images, where a temporally modulated recurrent network and a self-supervised adaptation method are proposed.
Experiments both synthetic and captured real-world datasets show the proposed method outperforms the state-of-the-art multi-image processing ones.
2
RELATED WORK
2.1
SUPERVISED MULTI-IMAGE PROCESSING.
Burst Image Restoration and Enhancement. Burst-based manners generally leverage multiple consecutive frames with the same exposure for image processing. Most methods focus on image restoration, such as denoising, deblurring, and SR tasks, as shown in Tab. 1. And they mainly explore inter-frame aligmment and feature fusion manners. The former can be implemented by utilizing various techniques, e.g., homography transformation (Wei et al., 2023), optical flow (Ranjan & Black, 2017; Bhat et al., 2021a;b), deformable convolution (Dai et al., 2017; Luo et al.,, 2022; Dudhane et al., 2023; Guo et al., 2022), and cross-attention (Mehta et al., 2023). The latter are also developed with multiple routes, e.g., weighted-baset mechanism (Bhat et al., 202 lab), kernel predition (Xia et al., 2020; Mildenhall et al., 2018; Dahary et al., 2021), attention-based merging (Dudhane et al., 2023; Mehta et al., 2023), and recursive fusion (Deudon et al., 2020; Guo et al., 2022; Rong et al., 2020; Wu et al, 2023). Moreover, HDR+ (Hasinoff et al., 2016) joins HDR imaging and denoising by capturing underexposure raw bursts. Recent updates (Ersst & Wronski, 2021) of HDR+ introduce additional well-exposed frames for improving performance. Although such manners may be suitable for scenes with moderate dynamic range, they have limited ability for scenes with high dynamic range.
Enal-Exposure Image Restoration. Several methods (Yuan et al., 2007; Chang et al., 2021; Mustaniemi et al., 2020; Zhao et al., 2022; Zhang et al., 2022b; Shekarforoush et al., 2023; Lai et al., 2022) exploit the complementarity of short-exposure noisy and long-exposure blurry images for better restoration. For example, Yuan et al. (Yuan et al., 2007) estimates blur kernels by exploring the texture of short-exposure images and then employ the kernels deblur long-exposure ones. Mustaniemi el al. (Mustaniemi et al., 2020) and Chang et al. (Chang et al., 2021) deploy convolutional neural networks (CNN) to aggregate dual-exposure images, achieving superior results compared with single-image methods on synthetic data. D2HNet (Zhao et al., 2022) proposes two-phase architecture for real-world image restoration. However, few works join it with HDR imaging, mainly due to the restricted dynamic range of dual-exposure images.
SeblurNet-Enhancee
multi-exposuree images are widely used for HDR image reconstruction. Most methods (Kalantari et al., 2017; Yan et al., 2019; Prabhakar et al., 2019; Wu et al., 2018; Niu et al., 2021; Liu et al., 2022; Yan et al., 2023a; Tel et al., 2023; Zhang et al., 2024b; Song et al., 2022) only focus on removing ghosting caused by image misalignment. For
Multi- Eaptti-Expouueeeee HDR Image Reconstruction.
3

---

Published as a conference paper at ICLR 2025
instance, Kalantari (Kalantari et al., 2017) align multi-exposure images and then propose data- driven approach to merge them. AHDRNet (Yan et al., 2019) utilizes spatial attention and dilated convolution to achieve deghosting. HDR- Transformer (Liu et al., 2022) and SCTNet (Tel et al., 2023) introduce self-attention and cross-attention to enhance feature interaction, respectively. Besides, a few methods (Hasinoff et al., 2010; Liu et al., 2023; Chi et al., 2023; Pérez-Pellitero al., 2021) take noise into account. Kim et al. (Kim & Kim, 2023) further introduce motion blur in the long-exposure image. However, the unrealistic blur simulation approach and the requirements of time-varying exposure sensors limit its practical applications. In this work, we consider more realistic situations in low-light environments, and incorporate both severe noise and blur. More importantly, we propose to utilize the complementary potential of multi-exposuree images to combine image restoration and enhancement tasks, including image denoising, deblurring, HDR reconstruction, and SR.
2.2 SELF-SUPERVISED MULTI-IMAGE PROCESSING
The complementarity of multiple images enables the achievement of certain image processing tasks in self-supervised manner. For self-supervised image restoration, some works (Dewil et al., 2021; Ehret et al., 2019; Sheth et al., 2021; Wang al., 2023c) accomplish multi-frame denoising with the assistance of Noise2Noise (Lehtinen et al., 2018) or blind-spot networks (Laine et al., 2019; Wu et al., 2020; Krull al., 2019). SelfRR (Zhang et al., 2022b) employs collaborative learning framework for restoring noisy and blurry images. Bhat et al. (Bhat et al., 2023) propose self- superviede Burst SR by establishing reconstruction objective that models the relationship between the noisy burst and the clean image. supervuuee real-world SR can also be addressed by combining short-focus and telephoto images (Zhang et al., 2022a; Wang et al., 2021; Xu et al., 2023). For self-supervised HDR reconstruction, several works (Prabhakar et al., 2021; Yan et al., 2023b; Nazarczuk et al., 2022) generate or search pseudo-pain -airss for training the model, while SelfHDR (Zhang et al., 2024b) decomposes the potential GT into constructable color and structure supervision. However, these methods can only handle specific degradations, making them less practical for our task with multiple ones. In this work, instead of creating self-supervised algorithms trained from scratch, we suggest adapting the model trained on synthetic pairs to real images, and utilize the temporal characteristics of multi-exposure image processing design self-supervised learning objectives.
METHOD
3.1 PROBLEM DEFINITION AND FORMULATION
Denote the scene irradiance at time by X(t). When capturing a raw image Y at ty time, we can simplify the camera's image formation model as,
Y -M to
((W)( + N).
In this equation, (1) D is a spatial sampling function, which is mainly related to sensor size. This function limits the image resolution. (2) Al denotes exposure time and W, represents the warp operation that accounts for camera shake. Combined with potential object movements in X(t), the integral formula f can result in a blurry image, especially when At is long (Nah et al., 2017). (3) N represents the inevitable noise, e.g., read and shot noise (Brooks et al., 2019). (4) M maps the signal to integer values ranging from to 2 1, where denotes the bit depth of the sensor. This mapping may reduce the dynamic range of the scene (Lecouat et al., 2022). In summary, the imaging process introduces multiple degradations, including blur, noise, as well as a decrease in dynamic range and resolution. Notably, in low-light conditions, some degradations (e.g., noise) may be more severe.
In pursuit of higher-quality images, substantial efforts have been made to deal with the inverse problem through single-image multi-image restoration (e.g., denoising, deblurring, and SR) and enhancement (e.g., HDR imaging). However, most efforts tend to focus on addressing partial degradations, and few works encompass all these aspects, shown in Tab. 1. In this work, inspired by the complementary potential of multi-exposure images, we propose to exploit bracketing photography to integrate these tasks for noise-free, blur-free, high dynamic range, and high-resolution images.
Specifically, the proposed BracketIRE involves denoising, deblurring, and HDR reconstruction, while Bracket adds support for SR task. Here, provide formalization for them. Firstly, We define
4

---

Published as a conference paper at ICLR 2025
the number of imput multi-exposure images T, and define the raw image taken with exposure time At, as y, where E (1.2..., and Åt; < Åt, Then, we follows the recommendations from multi-exposure HDR reconstruction methods (Yan et al., 2019; Niu et al., 2021; Liu et al., 2022; Yan et al., 2023a; Tel et al., 2023), normalizing Y, to A1/Ar, gamma-transformed image, i.e.
Y, and concatenating it with its
Y, At:/At )7,
(2)
At:/At
where y represents the gamma correction parameter and is generally set to 1/2.2. Finally, we feed these concatenated images into BracketIRE: BracketIRE+ model B with parameters θg, i.e.,
X ( -((Y))
where X is the generated image. Furthermore, the optimized network parameters can be written as,
0 arg min LB 0g where Lg represents the loss functiin, and can adopt & loss. X is the ground-truth (GT) image. T(-) denotes the y--law based tone mopp-in operator (Kalantari al., 2017), i.e.,
(4)
log(1 + xx))
T(X)
log(1 14) where 500.
(5)
Besides, we consider the shortest-exposure image (i) blur-free and take as a spatial alignment reference for other frames. In other words, the output X should be aligned strictly with Y1.
Towards real-world dynamic scenarios, it is nearly impossible to capture GT X. and it is hard to develop self-supervised algorithms trained on real-world images from scratch. To address the issue, suggest pre-training the model on synthetic pairs first and then adapting it real-world scenarios in self-supervised manner. In particular, we propose temporally modulated recurrent network for BracketIRE and BracketIRE+ tasks in Sec. 3.2, and self-supervised adaptation method in Sec. 3.3.
3.2 TEMPORALLY MODULATED RECURRENT NETWORK
Recurrent networks have been successfully applied to burst (Wu et al., 2023) and video (Wang et al., 2023b; Chan et al., 2021; 2022) restoration methods, which generally in- volve four modules, i.e., feature extraction, aligmment, aggre- gation, and reconstruction module. Here we adopt unidirec- tional recurrent network as our baseline, and briefly describe its pipeline. Firstly, the multi-exposure images {Y:]T are fed into encoder for extracting features {F; [F,), Then, the aligmment module is deployed to align F; with reference fea- ture F1. getting the aligned feature F;. Next, the aggregation module .A takes F, and the previous temporal feature H,-1 as inputs, generating the current fused feature H,, ie..
F
F
A
A
A
H
H2
H
(a) Baseline Recurrent Network
F
F
A
A
..
A
G, 4
G2
Gr
A
A
AS
H; A(F,, -,
(6)
H
H
Hr
(b) Our TMRNet
where θ denotes the parameters of A. Finally, Hy is fed into the reconstruction module to output the result.
Figure I: Illustration of bascline re- current network (c.g., RBSR (Wu al., 2023)) and our TMRNet. In- stead of sharing parameters of aggre- gation module A for all frames, divide it into a common one . for all frames and specific one Ai only for i-th frame. Modules with differ- ent colors have different parameters.
The aggregation module plays a crucial role in the recurrent framework and usually takes up most of the parameters. In burst and video restoration tasks, the degradation types of mul- tiple input frames are generally the same, so is appropriate for frames to share the same aggregation network parameters A. A In BracketRRE and BrackeuRE++ tasks, the noise models of multi-exposure images may be similar, as they can be taken by the same device. However, other degradations are varying. For example, the longer the exposure time, the more serious the image blur, the fewer underexposed areas, and the more overexposed ones. Thus, sharing θ may limit performance.
5

---

Published as a conference paper at ICLR 2025
[X
B
X
Xy
7
Trainable
Frozen
Copy
Leats
EMA
Lama
Data Flow
Loss Flow
MD
(X
Parameters Flow
(a) Temporally Self- Superviid Loss
(b) EMA Regularization Loss
Figure 2: Self-supervised loss terms for real-image adaptation. denotes TMRNet BracketRRE or BracketRRE+ task. In sub-figure (a), integer from to R (R T) is randomly chosen In sub-figure (b), EMA denotes exponential moving average.
To alleviate this problem, we suggest assigning specific parameters for each frame while sharing some ones, thus proposing temporally modulated recurrent network (TMRNet). As shown Fig. 1, we divide the aggregation module A into common one A for all frames and specific one A only for i-th frame. Features are first processed via A and then further modulated via A². Eq. (6) can be modified as,
G; = A (F, O.
H; A, Ox;):
where G, represents intermediate features, A and A denote the parameters of A and Aç. respectively. We do not design complex architectures for A and A and each one only consists of 3x3 convolution layer followed by some residual blocks (He al., 2016). More details of TMRNet can be seen in Sec. 5.1.
3.3 SELF-SUPERVISED REAL-IMAGE ADAPTATION
It is hard to simulate multi-exposure images with diverse variables (e.g., noise, blur, brightness, and movement) that are completely consistent with real-world ones. Due to the inevitable gap. models trained on synthetic pairs have limited generalization capabilities real scenarios. Undesirable artifacts are sometimes produced and some details are missed. To address the issue, we propose to perform sele-super adaptation for real-worke unlabeled images.
Specifically, we explore the temporal characteristics of multi multi-exposure image processing to design self-supervised loss terms elaborately, as shown in Fig. 2. Denote the model output of inputting the previous r frames {YT)== by X,. Generally, X performs better than X, (r T), as shown in Sec. 6.1. For supervising X,, although no ground-truth is provided, Xy can be taken as the targe.-target. Thus, the temporally self self-supervised loss be written as,
Cuelf = [|T(8,) - -T(sg(Xr))
(8)
where is randomly selected from to R (R < T), sg(-) denotes the stop-gradient operator.
Nevertheless, only deploying Lseif can easily lead to trivial solutions, as the final output X is not subject to any constraints. To stabilize training process, we suggest an exponential moving average (EMA) regularization loss, which constrains the output X of the current iteration to be not too far away from that of previous ones. can be written as,
Lema [T(), ((sg(Xemm)))l
where Xema = (( 1= Ogme) and Ogma denotes EMA parameters the current iteration. Denote model parameters in the k-th iteration by θB,- the EMA parameters in the k-th iteration can be written as,
ema = aQama Bx-1 + (1 a)) +
(10)
B₁
where Ogee OB, and a = 0.999.
The total adaptation loss is the combination of Lema and Liel iee.
Lada Lema + deet Czelf;
(1I)
where Asetf is the weight of L.dff-

---

Published as a conference paper at ICLR 2025
4 DATASETS
4.1 SYNTHETIC PAIRED DATASET
Although it is unrealistic to synthesize perfect multi-exposure images, we should still shorten the gap with the real images as much as possible. In the camera's s imaging model in Eq. (1), noise, blur, motion, and dynamic range of multi-exposure images should be carefully designed.
Video provides a better basis than a single image in simulating motion and blur of multi-exposure images. We start with HDR videos from Frochlich al. (Frochlich et al., 2014)' to construct the simulation pipeline. First, we follow the suggestion from Nah et al. (Nah et al., 2019) to perform frame interpolation, as these low frame rate (~25 fps) videos are unsuitable for synthesizing blur. RIFE (Huang et al., 2022) is adopted for increasing the frame rate by 32 times. Then, we convert these RGB videos to raw space with Bayer pattern according to UPI (Brooks et al., 2019), getting HDR raw sequences {V,, m Jmz The first frame V1 is taken as a GT.
Next, utilize (V m ) m=1 and introduce degradations to construct multi-exposure images. The process mainly includes the following 5 steps. (1) Bicubic 4x down-sampling is applied to obtain low-resolution images, which is optional and serves for BrackeuRE+ task. (2) The video is split into I' non-overlapped groups, where i-th group should be used to synthesize Y,. Such grouping utilizes the motion in the video itself simulate motion between T multi-ex multi-exposuree images. (3) Denote the exposure time ratio between Y, and Y by S. We sequentially move S (i2 1-th power of S) consecutive images into the above i-th group, and sum them up to simulate blurry images. (4) We transform the HDR blurry images into low dynamic range (LDR) ones by cropping values outside the specified range and mapping the cropped values to 10-bit unsigned integers. (5) We add the heteroscedastic Gaussian noise (Brooks et al., 2019; Wang et al., 2020; Hasinoff et al., 2010) to LDR images to generate the final multi-cxposurre images (i.e., {Y.)-=)) The noise variance is a function of pixel intensity, whose parameters are estimated from the captured real-world images in Sec. 4.2. More details of
conversion, frame interpolation, and noise can be seen in Appendix A. Appendix A.2, and Appendix A.3, respectively.
Besides, we set the exposure time ratio S to 4 and the frame number T to 5, as it can cover most of the dynamic range with fewer images. The GT has resolution of 1,920 1,080 pixels. Finally, obtain 1,335 data pairs from 35 scenes. 1,045 pairs from 31 scenes are used for training, and the remaining 290 pairs from the other 4 scenes are used for testing.
4.2 REAL-WORLD DATASET
Real-world multi-exposure images are collected with the main camera of Xiaomi 1OS smartphone at night. Specifically, we utilize the bracketing photography function in ProShot (Games, 2023) application (APP) to capture raw images with a resolution of 6,016x4,512 pixels. The exposure time ratio S is set to 4, the frame number is set to 5, ISO set to 1,600; these values are also the maximum available settings in APP. The exposure time of the medium-exposee
image (i.e., Y3) is automatically adjusted by APP. Thus, other exposures can be obtained based on S. It is worth noting that we hold the smartphone for shooting, without any stabilizing device, which aims to bring in the realistic hand-held shake. Besides, both static and dynamic scenes are collected, with a total of 200. 100 scenes are used for training and the other 100 are used for evaluation.
5 EXPERIMENTS
5.1 IMPLEMENTATION DETAILS
Network Details. The input multi-exposure images and ground-truth HDR image are both 4-channel data packed from raw images with the Bayer pattern. Following settings in RBSR (Wu et al., 2023), the encoder and reconstruction module consist of 5 residual blocks (He et al., 2016), the alignment module adopts flo-guided deformable approach (Chan et al., 2022). Besides, the total number of residual blocks in aggregation module remains the same as that of RBSR (Wu et al., 2023), i.e., 40,
'The dataset licensed under CC BY and is publicly available at the site.
7

---

Published as a conference paper at ICLR 2025
Inpot Frames
Burstormer
RBSR
SCTNet
al.
Ours
Figure 3: Visual comparison on the synthetic dataset of BracketIRE task. Our method restores sharper
edges and clearer details.
where the common module has 16 and the specific one has 24. For BracketRRE+ task, we additionally deploy PixelShuffle (Shi et al., 2016) at the end of networks for up-sampling features.
Training Details. We randomly crop patches and augment them with flips and rotations. The batch size is set The input patch size is 128 128 and 64 x 64 for BracketRRE and BracketREE+ tasks, respectively. We adopt Adam W (Loshchilov & Hutter, 2017) optimizer with 31 0.9 and B2 0.999. Models are trained for 400 epochs (~~ 60 hours) on synthetic images and fine-tuned for 10 epochs (~ 2.6 hours) on real-world ones, with the initial learning rate of 10-4 and 7.5 x 10-5, respectively. Cosine annealing strategy (Loshchilov & Hutter, 2016) is employed to decrease the learning rates 10-6 is randomly selected from {1, 2, 3). Aself set to 1. Moreover, BracketIRE+ models are initialized with pre-trained BracketRR models on synthetic experiments. All experiments are conducted using Py Torch (Paszke et al., 2019) on single Nvidia RTX A6000 (48GB) GPU.
5.2 EVALUATION AND COMPARISON CONFIGURATIONS
Evaluation Configurations. For quantitative evaluations and visualizations, first convert raw results to linear RGB space through a post-processing pipeline and then tone-map them with Eq. (5), getting 16-bit RGB images. All metrics are computed the RGB images. For synthetic experiments, we adopt PSNR, SSIM (Wang et al., 2004), and LPIPS (Zhang et al., 2018d) metrics. 10 and 4 invalid pixels around the original input image are excluded for BracketRRE and BracketIRE+ tasks, respectively. Kindly refer to Appendix A.4 for the reason. For real- real-world ones, we employ no-reference metrics i.e., CLIPIQA (Wang al., 2023a) and MANIQA (Yang et al., 2022).
Comparison Configurations. We compare the proposed method with 10 related state-of-the-art networks. including burst processing ones (i.e., DBSR (Bhat et al., 2021a), MFIR (Bhat al., 2021b), BIPNet (Dudhane et al., 2022), Burstormer (Dudhane et al., 2023) and RBSR (Wu et al., 2023)) and HDR reconstruction ones (i.e., AHDRNet (Yan et al., 2019), HDRGAN (Niu et al., 2021), HDR-Tran. (Liu et al., 2022), SCTNet (Tel et al., 2023) and Kim ef al. (Kim & Kim, 2023)). For a fair comparison, we modify their models to adapt inputs with 5 frames, and retrain them on our synthetic pairs following the formulation in Sec. 3.1. When testing real-world images, their trained models are deployed directly, while our models are fine-tuned on real-world training images with the proposed self- superviedd adaptation method.
5.3 EXPERIMENTAL RESULTS
Results on Synthetic Dataset. We summarize the quantitative results in Tab. 2. On BracketRRE task, we achieve 0.25dB and 0.26dB PSNR gains than RBSR (Wu al., 2023) and Kim al. (Kim & Kim, 2023), respectively, which are the latest ofatc-of-the-art methods. On BracketREE+ task, the improvements are 0. and 0.37dB, respectively. It demonstrates the effectiveness of our TMRNet, which handles the varying degradations of multi-exposure images by deploying frame- specific parameters. Moreover, the qualitative results Fig. show that TMRNet recovers more realistic details than others.
Results on World Dataset. We achieve the best no-reference scores on BracketIRE task and the highest CLIPIQA (Wang et al., 2023a) BracketRRE+ task. But note that the no-reference
8

---

Published as a conference paper at ICLR 2025
Input Frames
Burstormer
RBSR
HDR-Tran.
SCTNet
Kim
Ours
Figure 4: Visual comparison on the real-world dataset of BracketIRE task. Note that there is no ground-truth. Our results have fewer ghosting artifacts.
Table 2: Quantitative comparison with state-of-the-arr methods on the synthetic and real-world datasets BracketIRE and Bracket[RE+ tasks, respectively. 'Ada.' means the self-super real-image adaptation. The top two results marked in bold and underlined, respectively.
BrackedRE
BracketRRE+
Method
Synthetic
Real-World
Synthetic PSNRY/SSIM::LPPSLL
Real-World
PSNRY/SSIX::LPUPS
CLIPAQATIANNNQA
CLIPQAAT/MANIAAT
DBSR
33. 11.000992/0.11111 188
0.1359/0.1653
29. 79/0 799/0.854600... 8546/0.335
03334/0.2911
Burst
MFIR
35.64/0.91611 916101111111777
0.2192/0.2310
30.06/0.8591/00
0.3402/0.2908
Processing
BIPNet
/0 148
0.2234/0.2348
30.02/0 8582/0 324
0357770.2979
Networks
Burstormer
37.06/0.93448 11000.9344/0.15..
0.2399/0.258 2390
99/0.86611111 861786677
0,3549/0.
RBSR
99.10/0.94980.111
0.2174/0.2341
30.49/0 3/0/
0.3425/0.2895
AHDRNet
36.68/0. (..68/0.9279/..1. 9279/0.159
0.201 2259
86/0 01800.8880...80 308
0.3382/0.2900
HDR
HDRGAN
114/0.917770.188
0.1995/0.211 71
30,00/0 8590/0.337
0,3555/0.3109
Reconstruction
HDR-Tran.
97.62/0.9356/0 129
0.2043/0.214 144
38. 8662/0.279
.3245/002933
Networks
SCTNet
37.47/0.9444 201009443/.1111
0,2348/0.226/
30. 13/0 8644/0.29
0,3415/0.2936
Kim al.
99.09/0.9494/0.11
0,2467/0 2388
30.28/0.8658M0 8658/0.26
0.3302/0.299
Ada.
39.35/0.91100.11
0.2003/0.2181
30.65/0.87250.200
0.3422/0.286 22922/0.2898
Our TMRNet
w/ Ada.
0.2537/0.2422
0.3676/0.3020
metrics are not completely stable and are only used for auxiliary evaluation. The actual visual results can better demonstrate the effect different methods. As shown in Fig. 4, applying other models trained on synthetic data to the real world easily produces undesirable artifacts. Benefiting from the proposed self-supervised image adaptation, our results have fewer artifacts and more satisfactory content. More visual comparisons can be seen in Appendix J.
Inference Time. Our method has a similar inference time with RBSR (Wu al., 2023), and a shorter time than recent state-of-ther -fetth-art ones, ie.., BIPNet (Dudhane et al., 2022), Burstormer (Dudhane et al., 2023), HDR-Tran. (Liu et al., 2022), SCTNet (Tel et al., 2023) and Kim et al. (Kim & Kim, 2023). Overall, our method maintains good efficiency while improving performance compared to recent state-of-the-art methods. Detailed comparisons can be seen in Appendix B.
6
ABLATION STUDY
6.1 EFFECT OF NUMBER OF INPUT FRAMES
To validate the effect of the number of input frames, we conduct experiments by removing relatively higher exposure frames one by one, as shown in Tab. 3. Naturally, more frames result in better performance. In addition, adding images with longer exposure will lead to exponential increases of shooting time. The higher the exposure time, the less valuable content in the image. Considering these two aspects, we only adopt frames. Furthermore, we conduct experiments with more combinations of multi-exposure images in Appendix G.
6.2 EFFECT OF TMRNET
We change the depths of common and specific modules to explore the effect of temporal modulation in TMRNet. For fair comparison, we keep the total depth the same. From Tab. completely
9

---

Published as a conference paper at ICLR 2025
Input Frames
w/o Adoptation
w/ Adapcation
Input Frames
wlo Adaptation
w/ Adaptation
Figure 5: Effect of self supervise real-image adaptation. Our results have fewer ghosting artifacts and more details in the areas indicated by the red arrow. Please zoom in for better observation.
Table 3: Effect of number of input multi- exposure frames.
Table 4: Effect of number (i.e., ae and ag) of common and specific blocks.
BrackelRRE PSNRYYSSIMMLLPIPS]
BrackelRE++
BracketRE PSNR1/SSIM//L
BrackedRE+ PSNR1/SSIM
Input
/SNR1//SSIM/LLIISS.
29, 111.000235/0..550 340
25. 1330022890..6
40
38.96/) 9/09 120
30.41.0.870000.270
33.93/0. 33.933008992330.22 8923/0
27. 399/0.0003/,.30
39/26/0 9512/0. 115
30.70/0.8721 /0.270
24
39,35/0, 0..3550.95166... 9516/0.112
30 20.66...77255.22200
36 11..0.9994/01166
29.70/0 8446/0.324
(190944770. 117
38. 20.0000004400 11/099460D. 127
30.41/0 8645866455222
32
39. 16/0 16/0,55000, 117
30,590 01000.82222 8722/0,275
39.35.0.951600.112
30.65/0.87250.2770
39. 19.100094990.111
30.4990.8713X0.275
taking common modules or specific ones does not achieve satisfactory results, as the former ignores the degradation difference of multi-exposure images while the latter may difficult optimize. Allocating appropriate depths to both modules can perform better. In addition, we also conduct experiments by changing the depths of the two modules independently in Appendix
6.3 EFFECT OF SELF-SUPERVISED ADAPTATION
We regard T'RNNet trained synthetic pairs as baseline validate the effectiveness of the proposed adaptation method on BracketIRE task. From the visual comparisons Fig. 5, the adaptation method reduces artifacts significantly and enhances some details. From the quantitative metrics, it improves CLIPIQA (Wang et al., 2023a) and MANIQA (Yang et al., 2022) from 0.2003 and 0.2181 to 0.2537 and 0. 0.2422, respectively. Please kindly refer Appendix I for more results.
CONCLUSION
Existing multi-image processing methods typically focus exclusively either restoration enhance- ment, which are insufficient for obtaining visually appealing images with clear content low-light conditions. Motivated by the complementary potential of multi-exposure images in denoising, deblur- ring. HDR reconstruction, and SR, we utilize exposure bracketing photography to combine these tasks to get high-quality image. Specifically, we suggested a solution that initially pre-trains the model with synthetic pairs and subsequently adapts it to unlabeled real-world images, where a temporally modulated recurrent network and a self-supervised adaptation method are presented. Moreover, we constructed a data simulation pipeline for synthesizing pairs and collected real-world images from 200 nighttime scenarios. Experiments on both datasets show our method achieves better results than state-of-the-arts.
8 APPLICATIONS AND LIMITATIONS
Applications. A significant application of this work is HDR imaging night, especially in dynamic environments, aiming to obtain noise-free, blur-free, and HDR images. Such images can clearly show both bright and dark details in nighttime scenes. The application is not only challenging but also practically valuable. We also experiment with it on a smartphone (i.e., Xiaomi 1OS), as shown in Figs. G and H.
Limitations. Given the diverse imaging characteristics (especially noise model parameters) of various sensors, method necessitates tailored training for each sensor. In other words, our model trained on images from one sensor may exhibit limited generalization ability when applied to other sensors. We leave the investigation of a more general model for future work.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by the National Key R&D Program China (2022YFA 1004100) and the National Natural Science Foundation of China (NSFC) under Grant U22B2035.
REFERENCES
AbdeIrahman Abdelhamed, Mahmoud Afifi, Radu Timofie, and Michael S Brown. Ntire 2020 challenge on real image denoising: Dataset, methods and results. In CVPR Workshops, 2020.
Miika Aittala and Frédo Durand. Burst image deblurring using permutation invariant convolutional neural networks. ECCV, 2018.
Goutam Bhat, Martin Danelljan, Lue Van Gool, and Radu Timofte. Deep burst super-resolution. In CVPR, 2021a.
Goutam Bhat, Martin Danelljan, Fisher Yu, Luc Van Gool, and Radu Timofte. Deep reparametrization of multi-frame super-resolution and denoising. In CVPR, 2021b.
Goutam Bhat, Martin Danelljan, Radu Timofte, Yizhen Cao, Yuntian Cao, Meiya Chen, Xihao Chen, Shen Cheng, Akshay Dudhane, Haoqiang Fan, et al. Ntire 2022 burst super-resolution challenge. In CVPR Workshops, 2022.
Goutam Bhat, Michaël Gharbi, Jiawen Chen, Lue Van Gool, and Zhihao Xia. Self-supervised burst super-resolution. In ICCV, 2023.
Tim Brooks, Ben Mildenhall, Tianfan Xue, Jiawen Chen, Dillon Sharlet, and Jonathan T Barron. Unprocessing images for learned raw denoising. In CVPR, 2019.
Kelvin CK Chan, Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. Basicvsr: The search for essential components in video super-resolution and beyond. In CVPR, 2021.
Kelvin CK Chan, Shangchen Zhou, Xiangyu Xu, and Chen Change Loy. Basicvsr++: Improving video super-resolution with enhanced propagation and alignment. In CVPR, 2022.
Meng Chang, Huajun Feng, Zhihai Xu, and Qi Li. Low-light image restoration with short-and long-exposure raw pairs. IEEE TMM, 2021.
Su-Kai Chen, Hung-Lin Yen, Yu-Lun Liu, Min-Hung Chen, Hou-Ning Hu, Wen-Hsiao Peng, and Yen- Yu Lin. Learning continuous exposure value representations for single-image hdr reconstruction. In ICCV, 2023.
Yiheng Chi, Xingguang Zhang, and Stanley H Chan. Hdr imaging with spatially varying signal-to- noise ratios. CVPR, 2023.
Sung-Jin Cho, Seo- Gon Ji, Jun-Pyo Hong, Seung- Won Jung, and Sung-Jea Ko. Rethinking coarse-to- fine approach in single image deblurring. In ICCV, 2021.
Yuning Cui, Syed Waqas Zamir, Salman Khan, Alois Knoll, Mubarak Shah, and Fahad Shahbaz Khan. Adair: Adaptive all-in-one image restoration via frequency mining and modulation. arXiv preprint arXiv:2403. 110224403..1144 14614, 2024.
Omer Dahary, Matan Jacoby, and Alex M Bronstein. Digital gimbal: End-to-end deep image stabilization with learnable exposure times. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11936-11945, 2021.
Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. Deformable convolutional networks. In ICCV, 2017.
Michel Deudon, Alfredo Kalaitzis, Israel Goytom, Md Rifat Arefin, Zhichao Lin, Kris Sankaran, Vincent Michalski, Samira E Kahou, Julien Comebise, and Yoshua Bengio. Highres-net: Recursive fusion for multi-frame super-resolution of satellite imagery. arXiv preprint arXiv:2002.0644 2020.
11

---

Published as a conference paper at ICLR 2025
Valéry Dewil, Jérémy Anger, Axel Davy, Thibaud Ehret, Gabriele Facciolo, and Pablo Arias. Self- supervised training for blind multi- frame video denoising. In WACV, 2021.
Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. TPAMI, 2015.
Akshay Dudhane, Syed Waqas Zamir, Salman Khan, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Burst image restoration and enhancement. In CVPR, 2022.
Akshay Dudhane, Syed Waqas Zamir, Salman Khan, Fahad Shahbaz Khan, and Ming-Hsuan Hsuan Yang. Burstormer: Burst image restoration and enhancement transformer. CVPR, 2023.
Thibaud Ehret, Axel Davy, Jean-Michel Morel, Gabriele Facciolo, and Pablo Arias. Model-blind video denoising via frame-to-frame training. In CVPR, 2019.
Gabriel Eilertsen, Joel Kronander, Gyorgy Denes, Rafal K Mantiuk, and Jonas Unger. Hdr image reconstruction from a single exposure using deep cnns. ACM TOG, 2017.
Manfred Ernst and Bartlomiej
Wronski.
Hdr+
with bracketing on google/2021/04/
pixel phones, 2021. hdr hhd-wittee wwwwwwiittttteetteeeeeeeet wtt.--/wwaaeeiiingo.oo..aaaepppe html.
Jan Froehlich, Stefan Grandinetti, Bernd Eberhardt, Simon Walter, Andreas Schilling, and Harald Brendel. Creating cinematic wide gamut hdr-video for the evaluation of tone mapping operators and hdr-displays. In Digital photography X, 2014.
Rise Up Games. Proshot, 2023. https: / www. riseupgames wtt.cco///p......
Clément Godard, Kevin Matzen, and Matt Uyttendacle. Deep burst denoising. ECCV, 2018.
Shi Guo, Zifei Yan, Kai Zhang, Wangmeng Zuo, and Lei Zhang. Toward convolutional blind denoising of real photographs. In CVPR, 2019.
Shi Guo, Xi Yang, Jianqi Ma, Gaofeng Ren, and Lei Zhang. A differentiable two-stage alignment scheme for burst image reconstruction with large shift. In CVPR, 2022.
Samuel W Hasinoff, Frédo Durand, and William T Freeman. Noise-optimal capture for high dynamic range photography. In CVPR, 2010.
Samuel W Hasinoff, Dillon Sharlet, Ryan Geiss, Andrew Adams, Jonathan T Barron, Florian Kainz, Jiawen Chen, and Marc Levoy. Burst photography for high dynamic range and low-light imaging mobile cameras. ACM TOG, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
Zhewei Huang, Tianyuan Zhang, Wen Heng, Boxin Shi, and Shuchang Zhou. Real-time intermediate flow estimation for video frame interpolation. In ECCV, 2022.
Siddhant Jain, Daniel Watson, Eric Tabellion, Ben Poole, Janne Kontkanen, et al. Video interpolation with diffusion models. In CVPR, 2024.
Nima Khademi Kalantari, Ravi Ramamoorthi, et al. Deep high dynamic range imaging of dynamic scenes. ACM TOG, 2017.
Jungwoo Kim and Min H Kim. Joint demosaicing and deghosting of time-varying exposures for single-shot hdr imaging. In ICCV, 2023.
Alexander Krull, Tim-Oliver Buchholz, and Florian Jug. Noise2void-leaming denoising from single noisy images. In CVPR, 2019.
Wei-Sheng Lai, Yichang Shih, Lun-Cheng Chu, Xiaotong Wu, Sung-Fang Tsai, Michael Krainin, Deqing Sun, and Chia-Kai Liang. Face deblurring using dual camera fusion on mobile phones. ACM TOG, 2022.
12

---

Published as a conference paper at ICLR 2025
Samuli Laine, Tero Karras, Jaakko Lehtinen, and Timo Aila. High-quality self-supervised deep image denoising. NeurlPS, 2019.
Bruno Lecouat, Jean Ponce, and Julien Mairal. Lucas-kanade reloaded: End-to-end super-resolution from raw image bursts. In ICCV, 2021.
Bruno Lecouat, Thomas Eboli, Jean Ponce, and Julien Mairal. High dynamic range and super- resolution from raw image bursts. ACM TOG, 2022.
Christian Ledig, Lucas Theis, Ferenc Huszár. Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, et al. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR, 2017.
Siyeong Lee, Gwon Hwan An, and Suk-Ju Kang. Deep recursive hdri: Inverse tone mapping using generative adversarial networks. In ECCV, 2018.
Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. Noise2noise: Learning image restoration without clean data. In ICML, 2018.
Yawei Li, Yulun Zhang, Radu Timofte, Luc Van Gool, Zhijun Tu, Kunpeng Du, Hailing Wang, Hanting Chen, Wei Li, Xiaofei Wang, et al. Ntire 2023 challenge on image denoising: Methods and results. In CVPR Workshops, 2023.
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. In ICCV, 2021.
Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. Enhanced deep residual networks for single image super-resolution. In CVPR Workshops, 2017.
Ming Liu, Zhilu Zhang, Liya Hou, Wangmeng Zuo, and Lei Zhang. Deep adaptive inference networks for single image super- ruseutresolution. In ECCV Workshops, 2020a.
Shuaizheng Liu, Xindong Zhang, Lingchen Sun, Zhetong Liang, Hui Zeng, and Lei Zhang. Joint hdr denoising and fusion: real-world mobile hdr image dataset. In CVPR, 2023.
Yu-Luu Liu, Wei-Sheng Lai, Yu-Sheng Chen, Yi-Lung Kao, Ming-Hsuan Yang, Yung-Yo Yu Chuang, and Jia-Bin Huang. Single-image hdr reconstruction by learning to reverse the camera pipeline. In CVPR, 2020b.
Zhen Liu, Yinglong Wang, Bing Zeng, and Shuaicheng Liu. Ghost-free high dynamic range imaging with context-aware transformer. In ECCV, 2022.
Nya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts.
arXiv:1608.088 1608..09008,,.. 2016.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv:1111.0510/1 2017.
Ziwei Luo, Youwei Li, Shen Cheng, Lei Yu, Qi Wu, Zhihong Wen, Haoqiang Fan, Jian Sun, and Shuaicheng Liu. Bsrt: Improving burst super-resolution with swin transformer and www-guided deformable alignment. In CVPR, 2022.
Xintian Mao, Yiming Liu, Fengze Liu, Qingli Li, Wei Shen, and Yan Wang. Intriguing findings of frequency selection for image deblurring. In AAAI, 2023.
Nancy Mehta, Akshay Dudhane, Subrahmanyam Murala, Syed Waqas Zamir, and Khan. Gated multi-resolution transfer network for burst restoration and enhancement. CVPR, 2023.
Ben Mildenhall, Jonathan T Barron, Jiawen Chen, Dillon Sharlet, Ren Ng. and Robert Carroll. Burst denoising with kernel prediction networks. In CVPR, 2018.
Janne Mustaniemi. Juho Kannala, Jiri Matas, Simo Särkkä, and Janne Heikkilä. Lsd joint denoising and deblurring of short and long exposure images with cnns. In BMVC, 2020.
Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In CVPR, 2017.
13

---

Published as a conference paper at ICLR 2025
Seungjun Nah, Sungyong Baik, Seokil Hong, Gyeongsik Moon, Sanghyun Son, Radu Timofte, and Kyoung Mu Lee. Nure 2019 challenge on video deblurring and super-resolution: Dataset and study. In CVPR Workshops, 2019.
Michal Nazarezuk, Sibi Catley-Chandar, Ales Leonardis, and Eduardo Pérez Pellitero. Self-supervised hdr imaging from motion and exposure cues, arXiv preprint arXiv:2203.1233: 2022.
Yuzhen Niu, Jianbin Wu, Wenxi Liu, Wenzhong Guo, and Rynson WH Lau. Hdr-gan: Hdr image reconstruction from multi-exposed ldr images with large motions. IEEE TIP, 2021.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, et al. Pytorch: An imperative style, high-performance deep learning library. NeurIPS, 2019.
Fidel Alejandro Guerrero Peña, Pedro Diamel Marrero Fernández, Tsang Ing Ren, Jorge de Je- sus Gomes Leandro, and Ricardo Massahiro Nishihara. Burst ranking for blind multi-image deblurring. IEEE TIP, 2019.
Eduardo Pérez-Pelliters Sibi Catley-Chandar, Ales Leonardis, and Radu Timofte. Ntire 2021 challenge on high dynamic range imaging: Dataset, methods and results. In CVPR Workshops, 2021.
K Ram Prabhakar, Rajat Arora, Adhitya Swaminathan, Kunal Pratap Singh, and Venkatesh Babu. A fast, scalable, and reliable deghosting method for extreme exposure fusion. ICCP, 2019.
Ram Prabhakar, Gowtham Senthil, Susmit Agrawal, R Venkatesh Babu, and Rama Krishna Sai Gorthi. Labeled from unlabeled: Exploiting unlabeled data for few-shot deep hdr deghosting. In CVPR, 2021.
Anurag Ranjan and Michael J Black. Optical flow estimation using a spatial pyramid network. In CVPR, 2017.
Xuejian Rong, Denis Demandolx, Kevin Matzen, Priyam Chatterjee, and Yingli Tian. Burst denoising via temporally shifted wavelet transforms. In ECCV, 2020.
Shayan Shekarforoush, Amanpreet Walia, Marcus A Brubaker, Konstantinos G Derpanis, and Alex Levinshtein. Dual-camera joint deblurring-denoisiee arXiv preprint arXiv:2309.088. 2023.
Dev Yashpal Sheth, Sreyas Mohan, Joshua I. Vincent, Ramon Manzorro, Peter A Crozier, Mitesh M Khapra, Eero P Simoncelli, and Carlos Fernandez-Granda. Unsupervised deep video denoising. In ICCV, 2021.
Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In CVPR, 2016.
Jou Won Song, Ye-In Park, Kyeongbo Kong, Jaeho Kwak, and Suk-Ju Kang. Selective transhdr: Transformer-based selective hdr imaging using ghost region mask. In ECCV, 2022.
Xiao Tan, Huaian Chen, Kai Xu, Yi Jin, and Changan Zhu. Deep sr-hdr: Joint learning of super- resolution and high dynamic range imaging for dynamic scenes. JEEE TMM, 2021.
Xin Tao, Hongyun Gao, Xiaoyong Shen, Jue Wang, and Jiaya Jia. Scale-recurrent network for deep
image deblurring, In CVPR, 2018
Steven Tel, Zongwei Wu, Yulun Zhang, Barthélémy Heyrman, Cédric Demonceaux, Radu Timofte, and Dominique Ginhac. Alignment-free hdr deghosting with semantics consistent transformer. In ICCV, 2023.
Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Exploring clip for assessing the look and feel of images. In AAAI, 2023a.
Ruohao Wang, Xiaohui Liu, Zhilu Zhang, Xiaohe Wu, Chun-Mei Feng, Lei Zhang, and Wangmeng Zuo. Benchmark dataset and effective inter- inter-frame alignment for rorl-world video super-resolution. In CVPRW, 2023b.
14

---

Published as a conference paper at ICLR 2025
Tengfei Wang. Jiaxin Xie, Wenxiu Sun, Qiong Yan, and Qifeng Chen. Dual-camera super-resolution with aligned attention modules. In ICCV, 2021.
Yuzhi Wang, Haibin Huang, Qin Xu, Jiaming Liu, Yiqun Liu, and Jue Wang. Practical deep raw image denoising on mobile devices. In ECCV, 2020.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from eπror visibility to structural similarity. TIP, 2004.
Zichun Wang, Yulun Zhang, Debing Zhang, and Ying Fu. Recurrent seff-supervise video denoising with denser receptive field. In ACM MM, 2023c.
Kaixuan Wei, Ying Fu, Jiaolong Yang, and Hua Huang. A physics-based noise formation model for extreme low-light raw denoising. In CVPR, 2020.
Pengxu Wei, Yujing Sun, Xingbei Guo, Chang Liu, Guanbin Li, Jie Chen, Xiangyang Ji. and Liang Lin. Towards real-world burst image super-resolution: Benchmark and method. In ICCV, 2023.
Patrick Wieschollek, Bernhard Schölkopf, Hendrik PA Lensch, and Michael Hirsch. End-to-end learning for image burst deblurring. In ACCV, 2017.
Bartlomicj Wronski, Ignacio Garcia-Doraa Manfred Ernst, Damien Kelly, Michael Krainin, Chia- Kai Liang, Mare Levoy, and Peyman Milanfar. Handheld multi-frame super-resolution. ACM TOG, 2019.
Renlong Wu, Zhilu Zhang, Shuohao Zhang, Hongzhi Zhang, and Wangmeng Zuo. Rbsr: Efficient and flexible recurrent network for burst super-resolution. In PRCV, 2023.
Shangzhe Wu, Jiarui Xu, Yu-Wing Tai, and Chi-Keung Tang. Deep high dynamic range imaging with large foreground motions. In ECCV, 2018.
Xiaohe Wu, Ming Liu, Yue Cao, Dongwei Ren, and Wangmeng Zuo. Unpaired learning of deep image denoising. In ECCV, 2020.
Zhihao Xia, Federico Perazzi, Michaël Gharbi, Kalyan Sunkavalli, and Ayan Chakrabarti. Basis prediction networks for effective burst denoising with large kernels. In CVPR, 2020.
Ruikang Xu, Mingde Yao, and Zhiwei Xiong. Zero-shot dual-lens super-resolution. In CVPR, 2023.
Qingsen Yan, Dong Gong, Qinfeng Shi, Anton van den Hengel, Chunhua Shen, Ian Reid, and Yanning Zhang. Attention-guided network for ghost-free high dynamic range imaging. In CVPR, 2019.
Qingsen Yan, Weiye Chen, Song Zhang, Yu Zhu, Jinqiu Sun, and Yanning Zhang. A unified hdr imaging method with pixel and patch level. In CVPR, 2023a.
Qingsen Yan, Song Zhang, Weiye Chen, Hao Tang, Yu Zhu, Jinqiu Sun, Luc Van Gool, and Yanning Zhang. Smae: Few-shot learning for hdr deghosting with saturation-awan masked autoencoders. In CVPR, 2023b.
Sidi Yang. Tianhe Wu, Shuwei Shi, Shanshan Lao, Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujiu Yang. Maniqa: Multi-dimennion attention network for no-reference image quality assessment. In CVPR, 2022.
Lu Yuan, Jian Sun, Long Quan, and Heung- Yeung Shum. Image deblurring with blurred/noisy image pairs. In SIGGRAPH, 2007.
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. Cycleisp: Real image restoration via improved data synthesis. CVPR, 2020.
Syed Waqus Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming- -suan Yang, and Ling Shao. Multi-stage progressive image restoration. In CVPR, 2021.
15

---

Published as a conference paper at ICLR 2025
Jiawei Zhang, Jinshan Pan, Jimmy Ren, Yibing Song, Linchao Bao, Rynson WH Lau, and Ming- Hsuan Yang. Dynamic scene deblurring using spatially variant recurrent neural networks. In CVPR, 2018a.
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond gaussian denoiser: Residual learning of deep cnn for image denoising, JEEE TIP, 2017.
Kai Zhang, Wangmeng Zuo, and Lei Zhang. Ffdnet: Toward a fast and flexible solution for enn-based image denoising. IEEE TIP, 2018b.
Kai Zhang, Wangmeng Zuo, and Lei Zhang. Learning a single convolutional super-resolution network for multiple degradations. In CVPR, 2018c.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In CVPR, 2018d.
Xu Zhang, Jiaqi Ma, Guoli Wang, Qian Zhang, Huan Zhang, and Lefei Zhang. Perceive-ir: Learning to perceive degradation better for all-in-one image restoration. arXiv preprint arXiv:2408, 15994, 2024a.
Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang. Bineng Zhong, and Yun Fu. Image super-resolution using very deep residual channel attention networks. In ECCV, 2018e.
Zhilu Zhang, Ruohao Wang, Hongzhi Zhang, Yunjin Chen, and Wangmeng Zuo. Self-supervised learning for real-world super-resolution from dual zoomed observations. In ECCV, 2022a.
Zhilu Zhang, Rongjian Xu, Ming Liu, Zifei Yan, and Wangmeng Zuo. Self-supervised image restoration with blurry and noisy pairs. NeurlPS, 2022b.
Zhilu Zhang, Haoyu Wang. Shuai Liu, Xiaotao Wang, Lei Lei, and Wangmeng Zuo. Self-supe high dynamic range imaging with multi-exposure images in dynamic scenes. In ICLR, 2024b.
Yuzhi Zhao, Yongzhe Xu, Qiong Yan, Dingdong Yang, Xuehui Wang, and Lai-Man Po. D2hnet: Joint denoising and deblurring with hierarchical network for robust night image restoration. In ECCV, 2022.
Zhihang Zhong, Gurunandan Krishnan, Xiao Sun, Yu Qiao, Sizhuo Ma, and Jian Wang. Clearer frames, anytime: Resolving velocity ambiguity video frame interpolation. ECCV, 2024.
Yunhao Zou, Chenggang Yan, and Ying Fu. Rawhdr: High dynamic range image reconstruction from single raw image. In /CCV, 2023.
16

---

Published as a conference paper at ICLR 2025
APPENDIX
The content of the appendix involves:
More implementation details in Appendix
Comparison of computational costs in Appendix B.
Results on other datasets in Appendix C.
Comparison with all-in-one methods in Appendix D.
Comparison with step-by-step processing in Appendix E.
Comparison with burst imaging in Appendix
Effect of multi-exposure combinations in Appendix G.
Effect of TMRNet in Appendix H.
Effect of self-supervised adaptation in Appendix I.
More visual comparisons Appendix J.
A IMPLEMENTATION DETAILS
A.1 RGB-TO-REE RAW CONVERSION IN DATA SIMULATION PIPELINE
We provide an illustration to visually demonstrate the pipeline of synthesizing data in Fig.
The original HDR videos (Frochlich et al., 2014) in the synthetic dataset are shot by the Alexa camera, CMOS sensor based motion picture camera made by Arri UPI (Brooks et al., 2019) is used to convert these RGB videos to raw space. Note that we do not use the same camera parameters as UPI (Brooks et al., 2019) setting, but use the parameters from Alexa camera during RGB-to-RAW conversion.
A.2 FRAME INTERPOLATION IN DATA SIMULATION PIPELINE
RIFE (Huang et al., 2022) is used to interpolate between two frames, and it does not affect the data distribution. From visual observation of interpolation results, it also supports this point. Nonetheless, limited the ability of RIFE, the synthetic blur still has gap with the real-world one. In this work, the proposed supe-vueer real-image adaptation method interpolation models are more capable of dealing with large motion (Jain et al., 2024) and complex textures (Zhong al., 2024). We believe this problem can also be alleviated with the advancement of interpolation models.
alleviate this gap. Besides, recent
A.3 NOISE IN DATA SIMULATION PIPELINE
The noise in raw images is mainly composed of shot and read noise (Brooks et al., 2019). Shot noise can be modeled Poisson random variable whose mean is the true light intensity measured in photoelcctrons. Read noise can be approximated as Gaussian random variable with a zero mean and fixed variance. The combination of shot and read noise can be approximated as a single heteroscedastic Gaussian random variable N, which can be written as,
N ~ NO Aread X)hot
(A)
where X is the clean signal value. Xread and Ashot are determined by sensor's analog and digital gains.
In order to make our synthetic noise as close as possible to the collected real-world image noise, we adopt noise parameters of the main camera sensor in Xiaomi IOS smartphone, and they (i.e., Xshot
17

---

Published as a conference paper at ICLR 2025
5'
4x Down- Sampling (Optional)
S: Exposure Time Ratio Let Qi
5-1
Split (Simulate Motion)
RGB Video Clip
Frame Interpolation
(V mamm=
Q (Vm -m-m--Queth
(Vm) (Vm
RGB
Sum (Simulate Blur)
HDR RAW Video Clip (Vechm-
HDR to LDR
Select First Frame
Add Noise
Yy
Ground-Tren
Multi-Exposure Images
Figure A: Overview of data simulation pipeline. We utilize HDR video to synthesize multi-exposure
images { (Y; === and the corresponding GT image X. S denotes the exposure time ratio between Y; and Y 1- Y, is obtained by summing and processing S ({i-1)-th power of S) images from HDR raw video V. Q; denotes the total number of images from V that participate in constructing {
and Xread) can be found in the metadata of image file. Specifically, the ISO of all captured real-world images set to 1,600. At this ISO, Ashot R 2.42 x 10-3 and Aread R 1.79 x 10-5. Moreover, in order synthesize noise with various levels, we uniformly sample the parameters from ISO = 800 to ISO = 3,200. Finally, Aread and Aahot can be expressed as,
(og(A)hot) ~ (((lg(0000022) lo.(00044)))
(B)
log( Aread) log(Ashoi ~ (1.8669 (gg(Atot) + 0.3276, 0.3?),
where U(a, b) represents uniform distribution within the interval [a, bj.
A.4 EVALUATION SETTINGS
For synthetic experiment evaluation, 10 and 4 invalid pixels around the original input image are excluded for Bracket|RE and BracketRE++ tasks, respectively. Here is the reason. The surrounding 10 pixels of the original HDR videos² (Froehlich et al., 2014) are all 0 values. Thus, the surrounding 10 and pixels of the synthetic input image are all 0 values for BracketIRE and BracketIRE+ task, respectively. In practice, this situation hardly occurs, so we exclude them for evaluation. Nonetheless, the model can deal with marginal areas. Actually, in the early version of our paper, we used pixels for evaluation. After the suggestions from peers, we changed the evaluation way, as the current way more in line with the actual situation.
B COMPARISON OF COMPUTATIONAL COSTS
We provide comparisons the inference time, as well as the number of FLOPs and model parameters in Tab. A. We suggest inference time as the main reference for computational cost comparison, as the testing time is more significant than the number of FLOPs and model parameters in practical applications. It can be seen that method has a similar time with RBSR (Wu et al., 2023), and shorter time than recent state-of-the-art ones, i.e., BIPNet (Dudhane et al., 2022), Burstormer (Dud- hane et al., 2023), HDR-Tran. (Liu al., 2022), SCTNet (Tel al., 2023) and Kim et al. (Kim & Kim, 2023). Overall, our method maintains good efficiency while improving performance compared recent state-of-the-ar ohttr--rtttttttet methods.
Further, in the recent state-of-the-art methood BIPNet (Dudhane et al., 2022), RBSR (Wu et al., 2023), and our TMRNet are based on the convolutional neural network (CNN), while Burstormer (Dudhane et al., 2023), HDR-Tran. (Liu et al., 2022), SCTNet (Tel et al., 2023). and Kim et al. (Kim & Kim, 2023) are based on Transformer. TMRNet has similar #FL.OPs RBSR and lower #FL.Ps than BIPNet, but higher #FLOPs than these Transformer-basee methods. We think this is acceptable and
Phtpps: h//////wwww /ww.. hdm- stuttgarr rt..ttrrr....... de/vm1 hhh
18

---

Published as a conference paper at ICLR 2025
Table A: Comparison of #parameters and computational costs with state-of-the-ar methods when generating a 1920 x 1080 raw image on BracketIRE task. Note that the inference time can better illustrate the method's efficiency than #parameters and #FLOPs for practical applicability.
Method
#Params (M)
#FL.Pss (G)
Time (ms)
DBSR (Bhat et al., 2021a)
12.90
16,120
850
MFIR (Bhat al., 2021b)
12.03
18,927
974
BIPNet (Dudhane ct al., 2022)
6.28
135,641
6,166
Burstormer (Dudhane ct al., 2023)
3.11
9,200
2,357
RBSR (Wu et al., 2023)
5.64
19,440
1,467
AHDDReet (Yan et al., 2019)
2.04
2,053
208
HDRGAN (Niu al., 2021)
9.77
2,410
158
HDR-Tran. (Liu al., 2022)
1.69
1,710
1,897
SCTNet (Tel et al., 2023)
5.02
5,145
3,894
Kim et al. (Kim Kim, 2023)
22.74
5,068
1,672
TMRNet
13.29
20,040
1,425
Table B: Results on Kalantari al. (Kalantari al., 2017) dataset for HDR image reconstruction.
Method
PSNR"/SSIME
AHDRNett (Yan et al., 2019)
1L.14/0.970)
HDRGAN (Niu al., 2021)
41.57 0.9865
HDR-Tran. (Liu et al., 2022)
42.18 0.9884
SCTNet (Tel et al., 2023)
42.29 0.9887
Kim et al. (Kim & Kim, 2023)
41.99 0.9890
TMRNet
42.43 0.9893
understandable for two reasons. First, although Transformer based methods have an advantage in #FL.OPs, they bring higher inference time and it is difficult for them deploy into embedded chips for practical application. Second, the main idea of TMRNet is to assign specific parameters for each frame while sharing some parameters. We implement this idea based on a more recent CNN-based method (i.e., RBSR), and the basic modules only adopt simple residual blocks. For TMRNet, the basic modules can be easily replaced with #FLOPs-friende modules, which has great potential for #FLOPs reduction. We plan to experiment with it and provide a lightweight TMRNet in the next version.
C RESULTS ON OTHER DATASETS
C.I EFFECTIVENESS TMRNET ON OTHER DATASETS
To evaluate the effectiveness of TMRNet other datasets, we conduct experiments on Kalantari al. (Kalantari et al., 2017) dataset for HDR image reconstruction. We compare our TMRNet with recent HDR reconstruction methods. The results in Tab. B show that TMRNet achieves the best results. We also conduct experiments on BurstSR (Bhat et al., 2021a) dataset for burst image super-resolution. We compare our TMRNet with recent burst super-resolution methods. The results in Tab. C show that TMRNet still achieves the best results.
C.2 EFFECTIVENESS OF SELF-SUPERVISED REAL-IMAGE ADAPTATION ON OTHER DATASETS
To evaluate the effectiveness of self-supervised real-image adaptation on other datasets, we conduct an experiment on burst image super-resolution dataset (Bhat et al., 2021a), which is the only commonly used muult--image processing dataset with both synthetic and real-world data. We first pre-train our TMRNet on synthetic data, and then dataset. The results in Tab. show that our self-supervised loss brings 0.87 dB PSNR gain on real-world testing dataset, demonstrating its effectiveness.
self-supervised loss to fine-tune it real-world training
19

---

Published as a conference paper at ICLR 2025
Table C: Results on BurstSR (Bhat al., 2021a) dataset for burst image super-resolution.
Method
PSNRT / SSIMT
DRSR (Bhat et al., 2021a)
48.05 0.994
MFIR
(Bhat et al., 2021b)
48.33 / 0.985
BIPNet (Dudhane et al., 2022)
48.49 0,985
Burstormer (Dudhane el al., 2023)
48.82/ 0.986
RBSR (Wu et al., 2023)
48.80 / 0.987
TMRNet
48.92 / 0.987
Table D: Effectiveness of self-supervised real-image adaptation on BurstSR (Bhat et al., 2021a) dataset for burst image super-resolution.
Method
PSNR / SSIM
w/o Self-Supervised Adaptation
44.70 0,9690 45.57 0.97734
w/ Self-Superviee Adaptation
D COMPARISON WITH ALL-IN-ONE METHODS
All-in-one models (Zhang et al., 2024a; Cui al., 2024) mean that the models process images with different degradations. There are three main differences between them and our work. First, all-in-one models generally input single image, and output single image. Our model inputs multi-exposure images, and outputs a single image. Second, in all-in-one models, the degradation type across input samples
be different. In our model, the degradation across input samples is basically consistent, and the degradation is different between multiple images within an input. Third, all-in-one models utilize the capabilities of the model itself achieve multiple tasks. Our model can additionally exploit the complementarity of the input images to achieve multiple tasks.
We have conducted an experiment using AdaIR (Cui al., 2024). The original AdaIR model can only input one image. For a fair comparison, we concatenate multi-exposure images aligned by an optical flow network (Ranjan & Black, 2017) together as the imput of AdaIR. Its PSNR result 38 06 dB, which is lower than our 39.35 dB. We argue that the main reason for AdaIR's 's poor performance that it is not specifically designed for multi-image processing. In contrast, the methods compared in Tab. 2 are all specific multi-image processing methods.
E COMPARISON WITH STEP-BY-STEP PROCESSING
Here we demonstrate advantages by conducting ablation study that compares the joint processing and progressive processing manners on BracketIRE task on synthetic dataset. We mark multi-task joint processing way IDenoising&&Debburrnngg... decompose the whole task with three steps: (1) first denoising, (2) then deblurring, and (3) finally HDR reconstruction. We mark the way as Drenoisigg-Debluurngg--HD data pairs and modify our TMRNet as the specialized network for each step. The inputs of each step all multi-exposure images concatenated together, which aim to exploit the complementarity of multi-exposure images in denoising, deblurring, and HDR reconstruction task, respectively. During inference, we sequentially cascade the networks all steps test. The results are shown in the Tab. F. It can be seen that step-by-step processing is inferior to joint processing.
In the ablation study,
During training, we construct
Actually, during step-by-step processing. the denoising model performs well. The main reason for the unsatisfactory performance is that the deblurring model has limited effect when dealing with the severe blur in the long-exposure image. It prevents the HDR reconstruction model from working well. Specifically, in the training phase, the input multi-exposure images of 'HDR' model are blur-free and noise-free. free. However, in the testing phase, there may still be some blur remaining in the input of 'HDR' model, due the limited capabilities of 'Deblurring' models. Thus, data gap between training and testing appears in 'HDR' model, and it hurts the model performance. contrast, joint processing avoid this problem.
20

---

Published as a conference paper at ICLR 2025
Table E: Comparison with step processing.
Manner
PSNRT SSIM LPIPS|
Step-by-Step Processing (3 Steps, Demoosing--Dbblurinn--HHK
37.93 0.9367 0.1220
Our Joint Processing (1 Step, Demoossingg&&DebuurnngggEEE
39.35 0.9516 0.112
Table F; Comparison with burst processing manner. {Y)) denote the burst images with exposure
time At.
BracketIRE PSNR SSIM1 LPIPS}
Bracket[RE+ PSNRt SSIM LPIPS
Input
{Y] 15
32.22 0.8606 0.271
26.89 0.7663 0.416
(Y2
35.05 / 0,9237 / 0.171
28,93 0.8289 / 0.345
[Y3]
31.75 0.9284 0.144
28.24 0.8581/ / 0.302
(Y
26.30/ / 0.8853 0.215
24.46 / 0,8225 / 0.381
[Y))
20.04 0.8247 0.364
20.59 0.8062 0.450
(Y:)
39.35 0.9516 00.112
30.65 0.8725 0.270
Table G: Effect of multi--xposure image combinations on BracketIRE task.
Input
PSNR / /LPIPS.
Y1, Y2, Ys)
36.98 0,9294 0..165
Y1, Y3, Ys
37.54 0,9388 / 146
Y2, Ya, Y
36.48 0.9463 0. 0.127
(Ys. Y.,Y
31.31 / 0.9291 00164
YY, Y., Y.)
38.70/ / 0.9460/ 0.7
Y2, Y, Y.
36.54 / 0.9483/ 0.122
YY, Y2, Ys, Ys,
39.35 / 0.9516 0.112
Besides, joint processing only produces model for deploying. It can simplify the complexity of the entire imaging system and make it casier to deploy in actual scenarios. Benefiting from this, joint processing way is also being pursued by some mobile phone manufacturers, far as we know.
F COMPARISON WITH BURST IMAGING
To validate the effectiveness of leveraging multi-exposure frames, we compare our method with burst imaging manner that employs multiple images with the same exposure. For each exposure time At, use our data simulation pipeline construct 5 burst images { [Y as inputs. The quantitative results are shown in Tab. F. can be seen that the models using moderate exposure bursts (e.g., Y, and Y3) achieve better results, as these bursts take good trade-offs between noise and blur, well as posure and underexposure. Nevertheless, their results are still weaker than ours by wide margin, mainly due the limited dynamic range the input bursts.
G EFFECT OF MULTI-EXPOSURE COMBINATIONS
We conduct experiments with different combinations of multi-exposure images in the Tab. G. From Tab. G, the more frames, the better the results. More generally, we argue that as the number of frames increases, the worst case is that the model does not extract useful information from the increased frame. In other words, adding frames does not lead to the worse results, but leads to the similar or better ones. In this work, we adopt the frame number T and exposure time ratio = 4, as it can cover most of the dynamic range using fewer frames. Additionally, without considering shooting and computational costs, it is foresecable that a larger or smaller would perform better when keeping the overall dynamic range the same.
21

---

Published as a conference paper at ICLR 2025
Table H: Effect depth of specific blocks while keeping common blocks the same on BracketIRE task. 4g and ag denote the number of common and specific blocks, respectively.
Q
Time (ms)
PSNR* / SSIM//SSM///// LPIPSS LPIPS.
16
808
38.66 0,9462 0.125
16
1,016
38.87 /0.9480 00.122
16
16
1,224
39.12 / 0,9496 / /0.116
16
24
1,425
39.35 0,9516 0..122
16
1,633
39.36 0.9518 00.14
Table I: Effect of depth of common blocks while keeping specific blocks the same BracketIRE task. On and ag denote the number of common and specific blocks, respectively.
Cke
at2
Time (ms)
PSNR SSIMT LPIPS.
0
1,015
38.91 0.9484 00.121
8
1,219
39.15 0.9502 00.117
16
24
1,425
39.35 0.9516 0.112
24
24
1,637
39.31 / 0.9512 00.115
Table J: Effect of loss terms for self- self-super real-image adaptation. '-' denotes TMRNet trained on synthetic pairs. 'NaN' implies the training collapse. Note that the no-reference metrics are not completely stable and are provided only for auxiliary evaluation.
BrackeURE
BracketRRE+
Lema
Lseif
CLIPIQAI / MANIQAT
CLIPIQAt MANIQAT
0,2003 / 0.2181
0.3422 / 0.2898
x
0.2003 / 0.2181
0.3422 / 0.2898
X
V
NaN / NaN
NaN / NaN
V
Aself 0.5
0.2295 / 0.2360
0.3591 / 0.2978
Aself
0.2537 0.2422
0.3676 0,3020
V
Aad!/
0,2270 0,2391
0,3815 0.3172
/ Aself 4
0.1974 0,2525
0.3460 / 0.3189
H EFFECT OF TMRNET
For TMRNet, conduct experiments by changing the depths of the common and specific blocks independently, whose results are shown in Tab. H and Tab. I, respectively. Denote the number of common and specific blocks a, and a, respectively. On the basis of a. 16 and a, 24, adding their depths did not bring significant improvement while increasing the inference time. We speculate that it could be attributed to the difficulty of optimization for deeper recurrent networks.
EFFECT OF SELF-SUPERVISED ADAPTATION
In the main text, we regard TMRNet trained synthetic pairs baseline to validate the effec- tiveness of the proposed adaptation method. Here we provide more visual comparisons Fig. B, where results have fewer speckling and ghosting artifacts, well more details both in static and dynamic scenes.
We also provide the quantitative comparisons in Tab. J. It can be seen that the proposed adaptation method can bring both CLIPIQA (Wang et al., 2023a) and MANIQA (Yang et al., 2022) improvements. In addition, only deploying Leme would make the network parameters be not updated. Without Lamas the self-supervised fine- tine-tuning would lead to a trivial solution, thus collapsing. This because the result of inputting all frames is not subject any constraints at this time.
22

---

Published as a conference paper at ICLR 2025
UE 188
Input Frames
Adaptation
Adaptation
Imput
Aduptation
Adaptation
Figure B: Effect of self-supervised real- real-image adaptation. Our results have fewer speckling and ghosting artifacts, as well as more details. The top four show the visual effects of static scenes (but with camera motion), and the bottom four show the visual effects of moving objects. Please zoom in for better observation.
Moreover, empirically adjust the weight A of Leet and conduct experiments with different Agely- From Tab. J, the effect of Xeely on the results is acceptable. It worth noting that although higher Aeels (e.g., Aself
and Xeelf 4) sometimes achieves higher quantitative metries, the image contrast decreases and the visual effect is unsatisfactory at this time. This also demonstrates the no-reference metrics are not completely stable, thus we only take them for auxiliary evaluation. Focusing on the visual effects, we set Aself
J MORE VISUAL COMPARISONS
We first provide more visual comparisons on BracketREE+ task. Figs. Cad D show the qualitative comparisons on the synthetic images. Figs. E and show the qualitative comparisons on the real- world images. It can be seen that our method generates more photo-realistic images with fewer artifacts than others.
Moreover, in order to observe the effect of dynamic range enhancement, we provide some full-image results from real-world dataset. Note that the size of the original full images is very large, and here we downsample them for display. Fig. G shows the full-image visualization results BracketRRE task. Fig. H shows the full-image visualization results on BracketIRE+ task. Our results preserve both bright and dark details, showing higher dynamic range.
23

---

Published as a conference paper at ICLR 2025
Iaput Frames
DBSR
MFRI
BIPNet
Burstommr
RBSK
AHDRNet
HDRGAN
SCTNeE
Kim
Figure C: Visual comparison on the synthetic dataset of BracketRRE+ task. Our result restores clearer details. Please zoom in for better observation.
Inpat Frames
DBSR
MFIR
BIPNet
Burstormer
RBSR
AHDRNet
HDRGAN
HDR-Tran.
SCTNet
Kim
GT
Figure D: Visual comparison on the synthetic dataset BracketRE+ task. Our result restores more fidelity content. Please zoom in for better observation.
24

---

Published as a conference paper at ICLR 2025
Input
AHDRNet
HDRGAN
HDR-TTaa.
SCTNet
Kim at.
Ours
Figure E: Visual comparison on the real-world dataset of Bracket|RE+ task. Our result restores clearer textures. Please in for better observation.
Ingut
AHDRNet
HDRGAN
HDR-Tran.
SCTNet
Kim et al.
Ours
Figure F: Visual comparison on the real-world dataset of BracketIRE+ task. Our result has fewer artifacts. Please zoom in for better observation.
25

---

Published as a conference paper at ICLR 2025
MH
Input Input
Input
Input
5-th Input Frame
Output
Inpul Input Frame
Input
4-th Inpul
5-th Input
Output
Input Frame
2-nd Input Frame
3-rd Input Frame
4-th Input Frame
5-th Input Frame
Output
Figure G: Full-image results on the real-world dataset of Bracket|RE task. Our results preserve both the bright areas in short-exposure images and the dark areas in long-exposure images. Note that the size of the original full images is very large, and here we downsample them for display. Please zoom for better observation. Moreover, provide higher resolution version in README. of the supplementary material.
1-st Input Frame
2-nd Frame
3-rd Iaput Frame
4-th Input Frame
5-th Input Frame
Output
1-st Inpat Frame
2-nd Input Frame
3-nd Input Frume
4-th Input Frume
S-th Inpat Frame
Output
1sst Input Frame
2-nd Input Frame
3-nd Input Frame
4-th Input Frame
5-th Input Frame
Output
Figure H: Full-image results the real-world dataset of BracketRRE+ task. Our results preserve both the bright areas in short- exposure images and the dark areas in long exposure images. Note that the size of the original full images is very large, and here downsample them for display. Please zoom in for better observation. Moreover, we provide higher resolution version in README of the supplementary material.
26


Reviews:
[{"review_id": "Prq1NAjvG7", "reviewer": "Reviewer_jyRM", "summary": "The authors propose a framework for a new setting of image restoration. It uses raw images as input and process several kinds degradation in one network. The architecture can perform well.", "questions": "1. The way to convert RGB into raw is not suitable. For the method used in UPI, it need parameters from cameras. What is the type of CMOS for those videos? Is it the same with the setting of UPI? Would the RIFE affect data distribution of the original videos?\n2. As the input is raw images, do the authors perform packing operation for those data? Many methods would do packing to split color channels.\n3. Why do the authors  exclude 10 and 4 invalid pixels around the original input image? Do the proposed method deal with marginal areas?\n4. The FLOPs of TMRNet is huge compared with others.\n5. It seems to be the first work using this kind of unified image restoration setting. For the BracketIRE, can it compare with some all-in-one methods, such as AdaIR and Perceive-IR?\n6. Why do the authors add SR for BracketIRE+? To match the title as restoration tasks or to show the ability on low resolution images?\n\nScores can be raised upon replies.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. A novel setting combing several image restoration tasks, utilizing raw space images.\n2. The performance is good enough.\n3. Extensive experiments are done to prove characteristics of the proposed method.", "weaknesses": "1. The BracketIRE and BracketIRE+ are similar with all-in-one. The authors should discuss on this issue.\n2. The computational cost seems to be high. The fairness of comparison is doubtful.\n3. The datasets must be publicly, otherwise the work is not so essential.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PiMPHfHaBM", "reviewer": "Reviewer_GQoP", "summary": "This paper proposes a temporally modulated recurrent network based on exposure bracketing photography for image restoration and enhancement in low-light environments. This paper simulates the paired data and pre-trains the temporally modulated recurrent network with synthetic images. Additionally, a self-supervised adaptation method is utilized to improve the model's robustness in real-world unlabeled images. Experiments presented in the paper validate its effectiveness for multi-image processing.", "questions": "Refer to Weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This work is well-complete overall, including data simulation, real-world data collection, and method design and experiments. \n2. The experimental results show improvements in both visual quality and quantitative metrics, demonstrating its effectiveness.", "weaknesses": "1. The comparison methods in Table 2 are all trained on synthetic data, while TMRNet is fine-tuned on real-world data. This comparison is somewhat unfair. It is recommended that the authors include the results of TMRNet without fine-tuning on real-world data and provide a note.\n2. This paper claims that more frames result in better performance, would it introduce frames with over-exposure or large blur, making the restoration process more challenging?\n3. In summarizing the contributions, the authors claim to unify image restoration and enhancement tasks, which seems somewhat exaggerated. First, this is not a new task; in fact, methods of HDR have already been addressing issues such as noise, blur, or ghosting. Additionally, even if the input includes images with various degradations, this has also appeared in previous papers, such as R1 and R2. Therefore, it would be better if the authors could change this contribution point or use a more appropriate wording.\n\nR1. Low-light image restoration with short-and long-exposure raw pairs;\nR2. Digital gimbal: End-to-end deep image stabilization with learnable exposure times;", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VxwfXxqDDA", "reviewer": "Reviewer_QkhS", "summary": "The authors proposed a neural network capable of performing denoising, deblurring, HDR, and SR altogether using exposure bracketing. The proposed network structure has a novel feature, called temporally modulated recurrent network, which is claimed to be effective in handling multiple images with different degradation. To train the network, the authors also constructed a dataset by applying synthetic degradation sequentially. The two loss functions are also developed to train the network in a self-supervised manner.", "questions": "Since the paper is especially well-written, I do not have specific questions for clarification. See my concerns in the Weakness section.", "limitations": "", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The proposed method is the first method that performs the four restoration and enhancement tasks altogether.\n2. The necessity of the temporal modulated recurrent network is clearly explained, and its effectiveness is supported by experimental results. \n3. The necessity of the introduced loss functions is clearly explained, and their effectiveness is supported by experimental results. \n4. The constructed synthetic and real datasets will be helpful for researchers in the related field. \n5. The paper is well-written and well-organized.", "weaknesses": "1. I understand that ICLR accepts application-oriented papers. However, from the theoretical point-of-view, the manuscript does not contain sufficient technical novelties. The network can perform the four different tasks altogether since it is simply trained using the synthetic (and real) dataset containing four degradation. In other words, except for the dataset, the network does not have significant novelties compared to existing video restoration networks.  \n\n2. The proposed temporal modulation network seems not very original. This reviewer cannot see a significant difference between the proposed network and a commonly used conv-LSTM (with different conv layers for each layer). Several other temporal modulation networks have also been explored, e.g., Temporal Modulation Network for Controllable Space-Time Video Super-Resolution, CVPR 2021. \n\n3. The proposed self-supervised loss functions are also not very different from existing loss terms, e.g., Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning, CVPR 2021. \n\n4. The effectiveness of the proposed network module and loss functions needs to be justified by other public datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GomRQzciOp", "reviewer": "Reviewer_MkdG", "summary": "This paper addresses the challenge of acquiring high-quality photos in low-light environments by unifying image restoration and enhancement tasks using exposure bracketing photography. This paper proposes a solution that includes a temporally modulated recurrent network (TMRNet) and a self-supervised adaptation method, pre-training the model on synthetic paired data and then adapting it to real-world unlabeled images. They also construct a data simulation pipeline and collect real-world images from diverse nighttime scenarios. Experiments show that their method outperforms state-of-the-art multi-image processing techniques.", "questions": "I think the flowchart of the synthetic paired dataset is critical, would it be better to move it from the supplementary material to the manuscript?\n\n[After Rebuttal]: The authors have addressed my concerns, thus I am inclined to improve my rating.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This paper presents a paradigm for addressing low-light imaging through exposure bracketing photography with a simple yet effective network structure. This practical technical approach is orthogonal to mainstream research, which can more easily inspire subsequent studies.\nThe core contribution of this paper lies in the construction of the dataset and the corresponding training methods, particularly the self-supervised adaptation method. The dataset creation method is solid, and the design of the self-supervised adaptation method is clever.\nThe paper is well-organized, with clearly articulated problems, solutions, and experimental results that are easy to understand.\nThe impressive experiments show clear advantages over the compared methods.", "weaknesses": "1. **The use of RIFE**\n\n   In previous work simulating realistic blur on the UPI-based REDS video deblurring dataset, I tried using RIFE to interpolate frames to 1920fps. While RIFE is an advanced frame interpolation algorithm, I observed that it still introduces artifacts in areas with significant motion and complex textures. These limitations may reduce the generalization ability of models trained on such synthetic data when applied to real-world blurred scenes.\n\n2. **Comparison with AI-ISP**\n\n   While the proposed self-supervised adaptation method shows promise, it still faces challenges when paired data is available compared to supervised methods. It would be helpful if the authors could clarify how this approach compares to AI-ISP, which addresses RAW denoising, super-resolution, deblurring, and HDR processing as independent subtasks. By decomposing these tasks, it is possible (though challenging) to obtain paired real data to replace synthetic data, potentially enhancing performance in practice. Alternatively, the authors could demonstrate that exposure bracketing provides a clear advantage over handling each subtask independently on synthetic data as an indirect validation of its benefits.\n\nAddressing these concerns would clarify the practical advantages of the proposed approach, and I would be inclined to improve my rating accordingly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)