HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting
Authors: Yuanhao Cai
Zihao Xiao
Yixun Liang
Minghan Qin
Yulun Zhang
Xiaokang Yang
Yaoyao Liu
Alan Yuille
Venue: NeurIPS 2024 poster (2024)
Keywords: Applications
Computer Vision
Low-level Vision
Computational Photography
High Dynamic Range Imaging
Novel View Synthesis
3D Gaussian Splatting
Primary area: machine_vision
TL;DR: This work is the first attempt to explore the potential of Gaussian splatting in 3D high dynamic range imaging

Abstract: High dynamic range (HDR) novel view synthesis (NVS) aims to create photorealistic images from novel viewpoints using HDR imaging techniques. The rendered HDR images capture a wider range of brightness levels containing more details of the scene than normal low dynamic range (LDR) images. Existing HDR NVS methods are mainly based on NeRF. They suffer from long training time and slow inference speed. In this paper, we propose a new framework, High Dynamic Range Gaussian Splatting (HDR-GS), which can efficiently render novel HDR views and reconstruct LDR images with a user input exposure time. Specifically, we design a Dual Dynamic Range (DDR) Gaussian point cloud model that uses spherical harmonics to fit HDR color and employs an MLP-based tone-mapper to render LDR color. The HDR and LDR colors are then fed into two Parallel Differentiable Rasterization (PDR) processes to reconstruct HDR and LDR views. To establish the data foundation for the research of 3D Gaussian splatting-based methods in HDR NVS, we recalibrate the camera parameters and compute the initial positions for Gaussian point clouds. Comprehensive experiments show that HDR-GS surpasses the state-of-the-art NeRF-based method by 3.84 and 1.91 dB on LDR and HDR NVS while enjoying 1000$\times$ inference speed and only costing 6.3\% training time. Code and data are released at https://github.com/caiyuanhao1998/HDR-GS

HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting
Yuanhao Cai 1 Zihao Xiao , Yixun Liang Minghan Qin Yulun Zhang 4.* Xiaokang Yang Yaoyao Liu S, Alan Yuille" 1 Johns Hopkins University. 2 HKUST, Tsinghua University, Shanghai Jiao Tong University, University of Illinois Urbana-Champaign
Abstract
High dynamic range (HDR) novel view synthesis (NVS) aims to create photoreal- istic images from novel viewpoints using HDR imaging techniques. The rendered HDR images capture wider range of brightness levels containing more details of the scene than normal low dynamic range (LDR) images. Existing HDR NVS methods mainly based on NeRF. They suffer from long training time and slow inference speed. In this paper, we propose a new framework, High Dynamic Range Gaussian Splatting (HDR-GS), which can efficiently render novel HDR views and reconstruct LDR images with a user input exposure time. Specifically, we design Dual Dynamic Range (DDR) Gaussian point cloud model that uses spherical harmonics to fit HDR color and employs an MLP-bass tone-mapper to render LDR color. The HDR and LDR colors are then fed into Parallel Differentiable Rasterization (PDR) processes to reconstruct HDR and LDR views. To establish the data foundation for the research of 3D Gaussian splatting- based methods in HDR NVS, we recalibrate the camera parameters and compute the initial positions for Gaussian point clouds. Comprehensive experiments show that HDR-GS surpasses the state-of-the-art NeRF-based method by 3.84 and 1.91 dB on LDR and HDR NVS while enjoying 1000 inference speed and only costing 6.3% training time. Code and data are released at https: // //g thub. com/ caw.uanhaaoooooooooo 19w.//999///// HHDR-GDRS
Introduction
Speed
Compared to normal low dynamic range (LDR) images, high dy- namic range (HDR) images capture broader range of luminance levels to retain the details dark and bright regions, allowing for more accurate representation of real-worke scenes. Novel view synthesis (NVS) aims to produce photo-realisee images of scene unobserved viewpoints, given set of posed images of the same scene. NVS has been widely applied in autonomous driving [1-4), image editing [5-8], digital human [9-12], elc. NVS is challenging topic in computer vision because the lim- ited capacity of camera sensor usually leads to a low dynamic range (from to 255) of luminance in rendered images. This re- sults in the loss of image details in very bright or dark areas, color distortions, and limited capacity to display subtle gradations in light and shadow that the human eye can normally perceive. Hence, there is growing demand to render HDR (from 0 to +o0) views for better image quality and visualization performance.
WHELL SRE. COOAA
HDR-G5 (Ours)
6.122
HDBR-
0:28
D072
0013
Figure 1: HDR-GS vs. HDR-NeRF Our HDR-GS achieves better PSNR in dB, SSIM, and IPIPS performance with shorter training time minutes and faster inference speed in fps.
"Corresponding Author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

ar = 055
dc 32
3DGS
3DGS
Ours
Ours HDR
HDR- (Ours)
Viewpoint
Viewpoint
Viewpoint
Viewpoint
Figure 2: Comparisons of point clouds (left) and rendered views (right) between the original 3DGS [15] (top) and our HDR-GS GS (bottom). (i) 3DGS [15] renders blurry LDR views when training with images under different exposures. Its point clouds suffer from severe color distortion and not accurately represent the scene. In addition. 3DGS cannot control the exposure of the rendered images. Our HDR-GS GS can not only reconstruct clear HDR images with 3D consistency but render LDR views with controllable exposure time At.
Existing HDR NVS methods are mainly based on neural radiance fields (NeRF) [13]. However, the ray tracing scheme in NeRF is very time-consuming because it needs sample many 3D points and then compute their densities and colors for every single ray, severely slowing down the training and inference processes. For instance, the state-of-the-an NeRF-based method HDR-NeRF [14] takes hours to train and 8.2 to infer an image at the spatial size of 400x400. This limitation impedes the application of NeRF-based algorithms rendering real-time dynamic scenes.
Recently, 3D Gaussian Splatting (3DGS) [15] has achieved impressive inference speed than NcRF- based methods while yielding comparable results LDR NVS, which inspires another technical route for HDR NVS. However, directly applying the original 3DGS to HDR imaging may encounter three issues. Firstly, the dynamic range of the rendered image is still limited to [0, 255], which severely degrades the visual quality. Secondly, training 3DGS with images under different exposures may lead to a non-convergence problem because the spherical harmonics (SH) of 3D Gaussians can not adaptively model the change of exposures. This results in artifacts, blur, and color distortion in the rendered images, as shown the upper part of Fig. 2. Thirdly. 3DGS cannot adapt the exposure level of the synthesized views, which limits its applications. especially in AR/VR, film, and gaming where specific moods and atmospheres are usually evoked by controlling the lighting condition.
To cope with these problems, we propose a novel 3DGS-based method, namely High Dynamic Range Gaussian Splatting (HDR-GS). for 3D HDR imaging. More advanced than the original 3DGS, our HDR-GS can not only render HDR views but also reconstruct LDR images with a controllable exposure time, depicted in the lower part of Fig. 2. Specifically, we present Dual Dynamic Range (DDR) Gaussian point cloud model that can jointly model the HDR and LDR colors. We achieve this by using the SH of 3D point clouds model the HDR color. Then three independent MLPs are employed to model the classical nonparametric camera response function (CRF) calibration process [16] in RGB channels, respectively. By this means, the HDR color of the 3D point is tone-mapped to the corresponding LDR color with the user input exposure time, Subsequently, the HDR and LDR colors are fed into two Parallel Differentiable Rasterization (PDR) processes render the HDR and LDR images. In addition, we also notice that existing datasets only provide the camera poses in the normalized device coordinate (NDC) system, which not suitable for 3DGS based methods. To establish the data foundation for the research of 3DGS-basel methods in HDR imaging, we recalibrate the camera parameters and compute SfM [17] points to initialize 3D Gaussians. With the proposed techniques, HDR-GS outperforms tate-of-the-art (SOTA) NeRF-based methods by 1.91 dB on the HDR novel view synthesis task while enjoying 1000 inference speed and only requiring 6.3% training time, shown in Fig. 1. In nutshell, our contributions can be summarized as:

---

(i) We propose a novel framework, HDR-GS, for 3D HDR imaging. To the best of our knowledge, this is the first attempt to explore the potential of Gaussian splatting 3D HDR reconstruction.
(ii) We present a Dual Dynamic Range Gaussian point cloud model with two Parallel Differentiable Rasterization processes that render HDR images and LDR views with controllable exposure time.
(iii) We establish a data foundation by recalibrating camera parameters and computing initial points for 3DGS -based methods on the multi-view HDR datasets [14]. Experiments show that our HDR-GS dramatically outperforms SOTA methods while enjoying much faster training and inference speed.
2 Related Work
High Dynamic Range Imaging. Conventional HDR imaging [18] techniques recover HDR images by directly fusing a series of LDR images under different exposure levels at a fixed pose [19] or calibrating the camera response function (CRF) from the LDR images [16, 20]. These traditional methods yield compelling results in static scenes but produce unpleasant ghost artifacts in dynamic scenes. To address this issue, later works [21-25] adopt optical estimator to detect motion regions in the LDR images and then remove or align these regions in further fusion. With the development of deep learning, convolutional neural networks (CNNs) [26-29] and Transformers [30-33] have been used to learn an implicit mapping function from an LDR image to its HDR counterpart. Yet, these 2D HDR imaging methods lack 3D perception capabilities and are unable render novel HDR views.
Neural Radiance Field. NeRF [13] learns an implicit mapping function from the position of a 3D point and view direction to the point color and volume density. NeRF achieves impressive performance on the NVS task, inspiring many follow-up works to improve its reconstruction quality [34-38] and inference speed [39-45] or expand its application area [14, 46-446...... For example, Huang et al. present HDR-NeRF [14] that employs an MLP following the vanilla NeRF to learn an implicit mapping from physical radiance to digital color. Although good results are achieved, these NeRF-basek methods suffer from slow training and inference speed due to their conecconunnngg ray-tracing scheme.
Gaussian Splatting, 3DGS [15] explicitly represents a scene by millions of Gaussian point clouds. Its parallelized rasterization in view rendering allows it to enjoy much faster inference speed than NeRF-based methods that suffer from the time-consuming ray-tracing scheme. Thus, 3DGS has been rapidly and widely applied many areas such as dynamic scene rendering [50-52], SLAM [53-56]. inverse rendering [57-59], digital human [60-62], 3D generation [63-65], medical imaging [66]. etc. However, the illuminance modeled by 3DGS is still limited to low dynamic range. The potential of 3DGS in HDR imaging still remains under-explored. This work aims to fill this research gap.
Method
Figure
depicts the overall framework of our HDR-GS. To begin with, we use the structure- -rom--
motion (SfM) [17] algorithm to recalibrate the camera parameters of the scene and initialize the
Gaussian point clouds, as shown in Fig. (a). Then we propose a Dual Dynamic Range (DDR) Gaussian point cloud model jointly fit the HDR and LDR colors, as illustrated in Fig. 3 (b). The 3D Gaussians directly use the spherical harmonics (SH) to model the HDR color. Then three independent MLPs are employed to learn the tone- mapping operation in RGB channels. This tone coneme renders the LDR color from the corresponding HDR color and a controllable exposure time At. Subsequently, the LDR and HDR colors are fed into two Parallel Differentiable Rasterization (PDR) processes to render the HDR and LDR images, as depicted in Fig. (c). In this section, we first introduce the DDR point cloud model, then PDR processes, and finally the initialization and optimization of HDR-GS.
3.1 Dual Dynamic Range Gaussian Point Cloud Model
scene can be represented by a set of our Dual Dynamic Range (DDR) Gaussian point clouds & as
G (Gi Di,os ci, ci At, 0) 1, N,):
(1)
where Np is the number of 3D Gaussians and G; represents the i-th Gaussian. Its center position, covariance, opacity, LDR color, and HDR color are denoted μa R3, Σ, e 33x3 aq E R.
3

---

Trining Earation
Splatting Camera Pose Must
HDR
Rasterication
HDR Image 1
High Dynamic Range
Deua
Controllable Exposure
A
Fsn
Cameraa Poses
Tonc
Sineed
LDR
Splatting Camera Pose Mast
Racorizztion
Mou
Sparse LDR Views
Low Dynamic Range
Image 11 (AM)
[a] Recalibration and Initialization
(b) Dual Dynamic Range Gaussian Paint Cloud Model
(c) Parallel Differentiable Rasterization
Figure 3: Pipeline of our method. (a) SfM [17] algorithm is used to recalibrate camera parameters and initialize 3D Gaussians. (b) Dual Dynamic Range Gaussian point clouds spherical harmonies to model the HDR color. Three MLPs are employed tone-map the LDR color from HDR color and user input exposure time. (c) The HDR and I.DR colors are into two Parallel Differentiable Rasterization to render the HDR and LDR views.
cj E R and ce E R3. Besides these attributes, each G; also contains an exposure time At that controls the light intensity of the LDR view and three global-shared MLPs with parameters 6.
Σ; is represented by rotation matrix R. RR and scaling matrix S E R& as
Σ; = R.S.S.
Hr. R4, S₁, Os and 0 are learnable parameters. The tone-mapping operation frw(()) models the camera response function (CRF) that non-linearly maps the HDR color of into the L.DR color e as
cf = f (c) At),
where the exposure time At can be read from the metadata of photos. We propose to employ MLPs to learn the mapping process. There are two options. The first option is to directly model frM (-), which may result in the vanishing gradient problem because the multiplication operations may cause numerical overflow or underflow. Besides, the multiplication also leads to the nonlinearity and discontinuity of the input signal of MIPs, which also exacerbates the training instability. The second option is following the traditional non-parametric CRF calibration method of Debevec and Malik [16] that transforms frw(() (-) from lincar domain to logarithmic domain to enhance the stability of MLP training. We adopt the second option. Specifically, (-) in Eq. (3) is inversed and transformed as
log fTM (c)) = loog ci + log At,
where log(.) refers to the natural logarithm function and its base is = 2. .... Subsequently, take the inverse function of Eq. (4) on both sides and reformulate
....
ci = (log
c + log At).
(5)
Then we use three MLPs 0 model the function (log fT in RGB channels independently because the RGB colors are tone-mapped by different CRFss For simplicity, we define the mapping function of our ameemapper (x) A (log ( )
(Ir) Then Eq. (5) is reformulated
g ce + log At).
(6)
here log ci is modeled by the spherical harmonics (SH) with of coefficients k [k]** < <
L, -1 < m & 1} E R(++1)xx3 Each kt E 3R3 is a set three coefficients corresponding to the RGB components. is the degree of SH. Then of at the view direction d = (0, Φ) is derived by
c (d. exp()
Σ
Σ KTT Y (0.
1=0

---

where Yi" : - IR is the SH function that maps 3D points on the sphere to real numbers and exp(-)
represents the the exponential function. By substituting Eq. (7) into Eq. (6), we obtain ca as
(d k. At) go Σ 1=0
Σ
kr (0, + log At b),
(8)
here we add constant bias b E that helps adjust the SH function better fit the data. The detailed architecture of our MLP-based tone-mapper is shown in Fig. 3 (b). ga(-) equals to the process that the RGB channels of log cj. respectively undergo an independent MLP containing a fully connected (fc) layer, a ReLU activation, an fc layer, and sigmoid activation to produce the LDR color c².
3.2 Parallel Differentiable Rasterization
The computed HDR color c in Eq. (7) and LDR color ct in Eq. (8) of Gaussian point clouds fed into two parallel differentiable rasterization processes to jointly render the HDR and LDR views, as shown in Fig. (c). The HDR rasterization FHDR and LDR rasterization FLDR represented as
Ist FIDR (Mints Ments (Mr Ei, a, of Ii=1);
(9)
No I' (At) FİLDR (Minty Meats (us E,, og ((
where 1s and (2()) xx x3 denote the rendered HDR image and LDR image with the exposure time At, H and W refers to the height and width of the images, Mezt E I represents the extrinsic matrix, and Mint E 23 refers to the intrinsic matrix. Please note that we omit d and in cf and cf for simplicity. Then we introduce the details the parallel rasterization processes. First of all, we derive the possibility value of the i-th 3D Gaussian distribution at the point position x E RR as
P(x E,) exp( j(x T p4))
(10)
Subsequently, the splatting operation projects the 3D Gaussians to the 2D imaging plane. In this projection process, the center position μ4 is firstly transferred from the world coordinate system to the camera coordinate system and then projected the image coordinate system as
Vi = [v., 1] Mest p. Ment [nc 1]T,
u; = [u. 1] Mint V; = Mint [v., I]T, (11)
where u, R and v. E R refer to the image coordinate and camera coordinate of μ4; ü; V., and 14, are the homogeneous versions of u, vi, and μ4; respectively. The 3D covariance Σ; is also transferred from the world coordinate system to Σ E 33x3 in the camera coordinate system as
J.
(12)
where J; E R3x3 represents the Jacobian matrix of the affine approximation of the projective transformation Min: W; E R3x3 is the viewing transformation obtained by taking the first three rows and columns of Mest- Similar to previous works [15, 66-66,, the 2D covariance matrix x E 12x²2 is derived by directly skipping the third row and column of Σ. Subsequently, the 2D projection divided into non-overlapping tiles. The 3D Gaussians (,,,E,) are assigned to the tiles where their 2D projections E") cover. For each tile, the assigned 3D) Gaussians are sorted according to the view space depth. Then the RGB value (p)p) and (t) E R& at pixel is obtained by blending N ordered points overlapping pixel p the corresponding tile as
j j-1
('(
Σ JEN
5 II
(w)
P' (f(tt) o, I oR).
(13)
JEA
where oj ax P(x; Mý. Ej) and x, is the j-th intersection 3D point between the ray, which starts from the optical center the camera and lands at pixel p. and the Gaussian point clouds 3D space. di and c/ (At) are the HDR color and LDR color with exposure time At of x,, respectively.
5

---

3.3 Initialization and Optimization
An obstacle to the research of 3DGS-based based methods in 3D HDR imaging that the original multi- view HDR datasets [14] only provide the camera poses in the normalized device coordinate (NDC) system. This NDC system is not suitable for EDGS-based methods for two main reasons. Firstly, NDC focuses on describing the positions on the 2D screen after perspective projection. However, 3D Gaussian is an explicit 3D representation. 3DGS requires transforming and projecting Gaussian point clouds in 3D) space. Secondly, NDC rescales the coordinates into the range [-1, 1] or [0, 1]. The voxel resolution is limited, making it challenging to capture fine details in the scene. Besides, the original datasets [14] do not provide SIM [17] point clouds for the initialization of 3DGS.
To address these issues and establish data foundation for the research of 3DGS-based algorithms in 3D HDR imaging, we use the SfM algorithm [17] to recalibrate the camera parameters and compute the initial positions for 3D Gaussians. The mapping function of SfM FSIM summarized as
Mint; (Mezt) Np. ( FSIM ( (t.)..
(14)
where No represents the number of viewpoints and (t.) E R x3 refers to the LDR image at
the j-th viewpoint with the exposure time i, in the multi-view HDR datasets [14]. The intrinsic matrix Mint does not change with the viewpoint. Please note that we take the HDR views under the same exposure time as the inputs of SfM algorithms because SfM based on multi-view vieti feature detection and matching. Changes in exposure conditions may degrade the accuracy of SiM. Then we use the computed Np and {m4);-) in Eq. (14) to initialize G in Eq. (1). Other learnable parameters N. of G are randomly initialized. The recalibrated camera pose-image data pairs (M² Eaet 1 { in Eq. (14) are used train our HDR-GS with the weighted sum of L1 loss and D-SSIM loss as
B 4p Σ ((T) + X SSIM (A)))) (At,)),
(15)
where B is the training batch size and À is a hyperparameter. Similar to HDR-NeRF [14] that uses the ground truth CRF correction coefficient Co restrict the HDR color on the synthetic scenes, we also enforce a constraint to the rendered HDR image in the μ-law [14,23, 69,7,,,,77 70] LDR domain as
B
lo + p. norm(T)))
log(1 1 + norm (1)
(16)
log(1 + 14)
log(1 + 14)
where y is the amount of compression. norm(-) is the min-mas normalization. I and i #xxxx denote the rendered and ground-truth HDR image at the j-th viewpoint. The overall training loss is
C + 7...y.4. Le.
(17)
where y is hyperparameter that controls the relative importance between Cp and Le. We do not use C. in the real scenes since the ground truth HDR images are not provided in the real datasets. Therefore, we set y 0.6 and 0 in the experiments on the synthetic and real datasets, respectively.
4 Experiments
4.1 Experimental Settings
Dataset. We adopt the multi-view image datasets collected by [14), including 4 real scenes captured by a camera and 8 synthetic scenes created by the software Blender [71]. Images with different exposure time {t1, f2, t3, t,,ts) are captured at 35 different viewpoints. Following HDR-NeRF [14], images at 18 views with exposure time randomly selected from {t,, t3, tz are used for training while other 17 views at exposure time {t1, (fi,t,:C (s} and {12, tu) and HDR images are used for testing.
Implementation Details. We implement HDR-GS by PyTorch [72]. The models are trained with the Adam optimizer [73] (3, = 0.9, 82 = 0.999, and e = 1 1x10- 15) for 3x10 iterations. The learning rate for point cloud position is initially set to 1.6x10- and exponentially decays to 1.6x 10 The learning rates for point feature, opacity, scaling, and rotation are set to 2.5 10-3, 5x100 10-2 5x 10-3, and xx 10-3 The learning rate for the tone mapper network is initially set as 5x 10-1 and exponentially decays to 5x 10-5 All experiments conducted on single RTX A5000 GPU.

---

Training
L.DR-OR (f1: fai
LDR-NE (2214)
Metbod
Time (min)
Speed (fps)
PSNR1
SSIM
[.PPS.]
PSNR1
SSIM1
I.PPSL
PSNR
SSIM1
I.IPSS
NeRF [13]
405
0.190
13.97
0.555
0.376
14.51
0.522
0.428
3DGS [15]
38
121
19.46
0.690
0.276
18.97
0.778
0.309
NeRF-W [77]
0.178
0.936
0.047
29:22
0.927
0.050
HDR-NeRF
542
0.122
39.07
0.973
0.026
37.53
0.966
0.024
36.40
0.936
0.018
HDR-GS (Ours)
34
126
41.10
0,982
0.011
36.33
0.977
0.016
38.31
0.972
0.013
Table Quantitative results on the synthetic datasets. Metrics are averaged over scenes. LDR-OE denotes the LDR results with exposure time L1, la, and Ls. LDR-NE denotes the LDR results with exposure time l2 and 44. HDR denotes the HDR results. HDR-GS -DR-GS yields the best results on all tracks.
WRIGHTT
GAUD
WRIGHT
WRIGHT
05
NeRF
SDGS
NeRF-W
HDR-NeRF
HDR-GS (Ours)
Ground Truth
Figure 4: LDR visual comparisons on the synthetic scenes. Previous methods introduce unplcasant black spots render blurry images. Our method controls the exposure better while reconstructing more detailed structures.
Evaluation Metrics. We adopt the peak signal-to-noise ratio, PSNR (higher is better), and structural similarity index measure, SSIM [74] (higher is better), to quantitatively evaluate the objective performance. Learned perceptual image patch similarity, LPIPS [75] (lower is better), is adopted as the perceptual metric. Similar to [14], we also quantitatively evaluate the rendered HDR images in the tone-man mapped domain and qualitatively show HDR results tone-mane cone-mapped by Photomatix pro [76]. In addition, frames per second, fps (higher is faster), is used to measure the model inference speed.
4.2 Quantitative Results
Comparisons the Synthetic Datasets. The quantitative results of LDR and HDR NVS the synthetic datasets are reported Table 1. We compare our HDR-GS with three NeRF-based methods (NeRF [13], NeRF-W [77], and HDR-NeRF [14]) and the original 3DGS [15]. Table lists the training time, inference speed, PSNR. SSIM, and LPIPS results on LDR-OE, LDR-NE, and HDR. LDR-OE represents the LDR NVS results with exposure time t, t3. and ts. LDR-NE denotes the LDR NVS results with exposure time t2 and f4. HDR refers to the HDR NVS results. Please note that only HDR-NeRF and HDR-GS can output both LDR and HDR views. Other methods can only render LDR images. Our method outperforms SOTA methods on all tracks except for the PSNR on LDR-NE. (i) When compared
recent best method HDR-NeRF, our HDR-GS GS outperforms by 2.03 and 1.91 dB on LDR-OE and HDR tracks while enjoying 1000x faster inference speed and only (ii) When compared to the original 3DGS, our HDR-GS 21.64 and 17.36 dB higher on LDR-OE and LDR-NE, respectively. Interestingly, HDR-GS is slightly faster
costing 6.3% training time.
7

---

LDR-OE (f1, 13.
LDR-NE (t2, 14)
Method
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
NeRF [13]
14.95
0.661
0.308
14.44
0.731
0.255
3DGS [15]
17.19
0.806
0.103
19.50
0.727
0.152
NeRF-W [77]
28.55
0.927
0.094
28.64
0.923
0.089
HDR-NeRF [14]
31.63
0.948
0.069
31.43
0.943
0.069
HDR-GS (Ours)
35.47
0.970
0.022
31.66
0.965
0.030
Table Quantitative results on the real datasets. Metries are averaged across all scenes. LDR-OE represents the LDR results with exposure time /1, 13. and lg. LDR-NE denotes the LDR results with exposure time 12 and tq. HDR refers to the HDR results. HDR-GS yields the best results on all tracks.
mac
NeRF
3DGS
NeRF-W
HDR-GS (Ours)
Figure 5: LDR visual comparisons on real scenes. Previous methods introduce unpleasant black spots or render blurry images. Our method controls the exposure better while reconstructing more detailed structures.
than 3DGS. This is because 3DGS cannot adapt to different exposure levels. It is fragile and hard to converge when training with LDR images under different lighting intensities. The color change of the scene misleads the adaptive density control in 3DGS to split more Gaussian point clouds with distorted color to represent the complex variances in exposure levels, prolonging the training process.
To intuitively show the superiority of our method, Fig. plots a radar chart that features concentric polygons representing the HDR NVS performance across 5 metrics of the SOTA method HDR-NeRF and our HDR-GS. can observed that our HDR-GS forms much larger outermost polygon fully enclosing that of HDR-Net -NRR, indicating superior performance across all evaluated aspects. These results strongly demonstrate the advantages of our method effectiveness and model efficiency.
Comparisons on the Real Datasets. Table 2 shows the quantitative comparisons on the real datasets. Please note that the real datasets do not provide HDR ground truth for quantitative evaluation. Hence, we only report the LDR NVS results in Table 2. When compared to the recent best method HDR- NeRF, our HDR-GS is 3.84 and 0.23 dB higher in PSNR on LDR-OE and LDR-NE. When compared to the original 3DGS, HDR- GS significantly surpasses it by 18.28 and 12. 16 dB on LDR-OE -EE and LDR-NE. These results suggest the outstanding generalization ability and effectiveness of our method.
4.3 Qualitative Results
LDR Novel View Rendering, The comparisons of LDR novel view rendering with different exposure times on the synthetic (dog and sofa) and real (box and luckycat) scenes are shown Fig. 4 and 5. can be observed that NeRF, 3DGS, and NeRF-W fail to control the exposure. They either introduce black stripes and spots, or over-smooth the image, or over-enhance the image while distorting the color. HDR-NeRF can adapt the light intensity but it also produces blurry images. In contrast,

---

Method
Buseline
Cameru Recalibration
SIM Points
DDR Model
Domain
Linear
Logarithmic
HDR
38. 38.31
HDR
26.18
38.31
12.35
41.10
LDR-OE LDR-NE
41.10
LDR-NE
11.83
14.41
18.97
36.33
27.44
36.33
(a) Break-down ablation study towards better performance
(b) Study on the CRF domain
Exposure
[ts]
fa. (s)
0.125
fg
4-
HDR
22.86
32,06
38.31
38,50
HDR
36,88
57.90
38.16
38.31
38.05
LDR-OE
23.11
34.73
41.10
41.32
LDR-OE
39.71
41.07
40.98
41.10
41.21
LDR-NE
22.37
32,90
36.33
36.48
LDR-NE
35,28
35.92
36.25
36.33
36.20
(c) Study the exposure times used in training
(d) Study on the exposure time ts in recalibration
Table 3: Ablations on the synthetic datasets. The PSNR results on HDR, LDR-OE, and LDR-NE are reported.
0og
GOg
dog
5OGO
5003
2013
HDR-NeRF
HDR-GS (Ours)
Ground Truth
HDR-NeRF
HDR-GS (Ours)
Ground Truth
Ladeyaw
HDR-NeRF
HDR-GS (Ours)
HDR-NeRF
HDR-GS (Ours)
HDR-NeRF
HDR-GS (Ours)
Figure 6: HDR visual comparisons on synthetic (upper) and real (lower) scenes. method can recover the details in both dark and bright regions while suppressing color distortion. Please zoom in for better view.
HDR-GS can only control the exposure level of LDR views according the user input time but also reconstruct clearer images with high-frequency textures and structural contents.
HDR Novel View Rendering. The comparisons of HDR novel view synthesis on the synthetic (upper) and real (lower) datasets are depicted in Fig. 6. Please note that only HDR-NeRF and our HDR-GS GS reconstruct HDR images. As can be seen, HDR-NeRF yields low-contrast and over-smooth images while sacrificing fine-gra seee-griiedd details and introducing undesirable chromatic artifacts and black spots. On the contrary, HDR-GS render perceptually pleasing HDR images with sharper textures and preserve the color and spatial smoothness of homogeneous regions.
4.4 Ablation Study
In this section, adopt the synthetic datasets to conduct ablation study. Table 3 lists the PSNR results averaged across all scenes on the LDR-OE, LDR-NE, and HDR tracks, respectively.
Break-down Ablation. We adopt 3DGS [15] trained with the original coordinates (NDC) as the baseline to conduct break-down ablation on the synthetic datasets. Our goal study the effect of each component towards higher performance. The results are reported in Table 3a. (i) The baseline model can only render LDR views. achieves 12.35 and 11.83 dB on LDR-OE and LDR-NE. (ii) When using the recalibrated camera poses, the model yields an improvement of 2.27 and 2.58 dB on LDDROE and LDR-NE because liberated from the constraint of the NDC system. (iii) When we apply the SIM points for the initialization of 3D Gaussians, the model gains by 4.84 dB and 4.56 dB

---

because the SfM points provide a general shape of Gaussian point clouds to alleviate the overfitting issues of 3DGS, However. the model still cannot render HDR views nor change the exposure level of the LDR views until now, leading to limited LDR NVS performance. (iv) Then we apply our DDR point clouds, the model is enabled to render HDRR views with 38.31 dB in PSNR performance. Besides, the model yields 21.6 21.64 and 17.36 dB improvements on LDR-OE and LDR-NE because our DDR point clouds allow the model to adapt the lighting intensity with controllable exposure time,
CRF Domain. We conduct experiments to compare the effects of modeling CRF in linear domain and logarithmic domain. As shown in Table 3b, when the MLPs 6 directly models fras (-) in Eq. (3), our method yields poor results of only 26.18. 18, 29.53, and 27, dB on HDR, LDR-OE, and LDR- NE. In contrast, when the MLPs models ge(-) in Eq. (6), the performance is 12.13, 11.57, and 8.89 dB higher on HDR, LDR-OE, and LDR-NE. This is because the multiplication in fru(() transferred into the addition in go))), which enhances the training stability by suppressing the numerical nonlinearity and discontinuity problems. This evidence verifies our analysis in Sec. 3.1.
Exposure Time Used for Training. We conduct experiments in Table 3c to study the effect of the number of exposure times used in training. (i) According to the research of Debevec and Malik [16], modeling CRF requires at least
exposures. Thus, when we only use single exposure (ta), HDR-GS fails to reconstruct HDR views and LDR images with novel exposure time. (ii) When two exposures { } are used for training, HDR-GS gains by 9.20, 11.62, and 10.53 dB on HDR, LDR-OE, and LDR-NE, (iii) The performance of using three exposures {f1, 13, ts J is close to that of using five exposures {t1, 12, f3, 44. f6). Hence, it a reasonable choice to use three exposure times.
Recalibration of Camera Parameters. In Eq. (14), we use the SIM algorithm to recalibrate the camera parameters and compute the initial positions of 3D Gaussians at the same exposure time t.. Here, we conduct experiments to study the effect of t, in Table 3d. The performance achieves its maximum value when t, f 8 seconds. Therefore, the optimal choice of t. is t4 8
5 Limitation and Broader Impact
The main limitation of this work is that the memory usage of 3DGS-based methods is non-trivial and maybe unaffordable some low-RAM mobile devices. HDR imaging is a very important topic in computational photography. Nowadays, billions of LDR images are captured by mobile phones and cameras. Therefore, how to enhance the quality of these images, adapt the exposure level, and render HDR views is worth studying. Our HDR-G EDR-GS is capable of reconstructing better HDR and LDR views with controllable exposure time at 1000 speed than SOTA methods, showing great value in practical applications. Until now, 3D HDR imaging techniques have no negative social impact yet. Our proposed HDR-GS does not present any negative foreseeable societal consequences, either.
Conclusion
This paper focuses studying the efficiency problem of 3D HDR imaging. We propose the first Gaussian Splatting-based framework, HDR-GS, for HDR novel view synthesis. Our HDR-GS is based the Dual Dynamic Range Gaussian point clouds that can jointly model HDR color and LDR color with user input exposure time. Then, the HDR and LDR colors are fed into two Parallel Differentiable Rasterization processes render the HDR and LDR views. To avoid the limitations of NDC system and establish data foundation for the research of 3DGS-based methods, we recalibrate the camera parameters and compute the initial positions for Gaussian point clouds. Experiments show that our HDR-Gi GS outperforms the SOTA NeRF-based method by 1.91 and 3.84 dB on HDR and LDR novel view rendering, while enjoying x000x inference speed and requiring only 6.3% training time.
Acknowledgement
This work was supported by the office of Naval Research with award N000142412696.
10

---

References
[1] Z. Yang, Y. Chai, D. Anguelov, Y. Zhou, P. Sun, D. Erhan, S. Rafferty, and H. Kretzschmar, "Surfelgan: Synthesizing realistic sensor data for autonomous driving," in CVPR, 2020.
[2] S. Huang. Z. Gojcie, Wang, Williams. Y. Kasten, Fidler, K. Schindler, and Litany, "Neural lidar fields for novel view synthesis," in ICCV, 2023.
[3] G. Wang, Z. Chen, C. C. Loy, and Z. Liu, "Sparsenerf: Distilling depth ranking for few-shot novel view synthesis," in ICCV, 2023.
[4] M. Tancik, V. Casser, X. Yan, S. Pradhan, B. Mildenhall, P. P. Srinivasan, J. T. Barron, and H. Kretzschmar, "Block-nerf: Scalable large scene neural view synthesis," in CVPR, 2022.
[5] S. Liu, X. Zhang, Z. Zhang, R. Zhang. J.-Y. Zhu, and Russell, "Editing conditional radiance fields," in ICCV, 2021.
[6] J. Sun, X. Wang. Y. Shi, L. Wang, Wang, and Y. Liu, "Ide-3d: Interactive disentangled editing for high-resolution 3d-aware portrait synthesis," ACM ToG, 2022.
[7] Y.-J. Yuan, Y.-T. Sun. Y.-K. Lai, Y. Ma, R. Jia, and L. Gao, "Nerf-editing; geometry editing of neural radiance fields," " in CVPR, 2022.
[8] S. Kobayashi, E. Matsumoto, and V. Sitzmann, "Decomposing nerf for editing via feature field distillation," in NeurlPS, 2022.
[9] L. Liu, Habermann, Rudnev, Sarkar, J. Gu, and Theobalt, "Neural actor: Neural free-view synthesis of human actors with pose control," ACM TOG, 2021.
[10] T. Hu, K. Sarkar, L. Liu, M. Zwicker, and C. Theobalt, "Egorenderer: Rendering human avatars from egocentric camera images," " ICCV, 2021.
[11] J. Zheng, Y. Jang, A. Papaioannou, C. Kampouris, R. A. Potamias, F. P. Papantoniou, E. Galanakis, A. Leonardis, and S. Zafeiriou, "Iish: The imperial light-stage head dataset for human head view synthesis," in ICCV, 2023.
[12] Z. Zheng, H. Huang, T. Yu, H. Zhang, Y. Guo, and Y. Liu, "Structured local radiance fields for human avatar modeling." in CVPR, 2022.
[13] B. Mildenhall, P. Srinivasan, M. Tancik, Barron, R. Ramamoorthi, and Ng. "Nerf: Representing scenes as neural radiance fields for view synthesis," in ECCV, 2020.
[14] X. Huang. Q. Zhang. Y. Feng. H. Li, X. Wang, and Q Wang. "Hdr-nerf: High dynamic range neural radiance fields," CVPR, 2022.
[15] B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis, "3d gaussian splatting for real-time radiance field rendering," ACM Transactions Graphics, 2023.
[16] P. E. Debevec and J. Malik, "Recovering high dynamic range radiance maps from photographs," SIGGRAPH, 1997.
[17] J. L.. Schönberger and J.MM. Frahm, "Smmuctureffrommmattee revisited," in CVPR, 2016.
[18] R. Szeliski, Computer vision: algorithus and applications. Springer Nature, 2022.
[19] T. Mertens, J. Kautz, and F. Van Reeth, "Exposure fusion," in Pacific Conference on Computer Graphics and Applications, 2007.
[20] G. Ward, E. Reinhard, and P. Debevec, "High dynamic range imaging & image-based lighting." in SIGGRAPH, 2008.
[21] T. Grosch et al., "Fast and robust high dynamic range image generation with camera and object movement," Vision, Modeling and Visualization, RWTH Aachen, 2006.
[22] K. Jacobs, Loscos, and G. Ward, "Automatic high-dynamic range image generation for dynamic scenes." IEFE Computer Graphics and Applications, 2008.
[23] N. K. Kalantari, R. Ramamoorthi, al., "Deep high dynamic range imaging of dynamic scenes.," ACM ToG, 2017.
[24] O. T. Tursun, A. O. Akyüz, A. Erdem, and E. Endem, "The state of the art in hdr deghosting: A survey and evaluation," in Computer Graphics Forum, 2015.
[25] Q Yan, Y. Zhu, and Y. Zhang, "Robust artifact-free high dynamic range imaging of dynamic scenes." Multimedia Tools and Applications, 2019.
[26] G. Eilertsen, Kronander, G. Denes, R. K. Mantiuk, and J. Unger, "Hdr image reconstruction from single exposure using deep enns," ACM TOG, 2017.
[27] Z. Khan, M. Khanna, and S. Raman. "Fhdr: Hdr image reconstruction from a single Idr image using feedback network." in IEEE Global Conference on Signal and Information Processing, 2019.
11

---

[28] J. Kim, S. Lee, and S.-J. Kang. "End-to-end differentiable learning to hdr image synthesis for multi- exposure images," " in AAAI, 2021.
[29] Z. Liu, W. Lin. X. Li, Q Rao, T. Jiang, M. Han, H. Fan, J. Sun, and S. Liu, "Adnet: Attention- guideed deformable convolutional network for high dynamic range imaging." in CVPRW, 2021.
[30] R. Chen, B. Zheng, H. Zhang, Q. Chen, C. Yan, G. Slabaugh, and S. Yuan, "Improving dynamic hdr imaging with fusion transformer," in AAAI, 2023.
[31] Z. Liu, Wang, B. Zeng, and Liu, "Ghost-free high dynamic range imaging with context-aware transformer," in ECCV, 2022.
[32] W. Song, Y.-I. Park, K. Kong, J. Kwak, and S.-J. Kang. "Selective transhdr: Transformer-based selective hdr imaging using ghost region mask," in ECCV, 2022.
[33] Y. Yu, Wang. T. Luo, H. Fan, and Zhang, "Magic: Multi- modality guided image completion." in ICLR, 2024.
[34] J. T. Barron, B. Mildenhall, M. Tancik, P. Hedman, R. Martin-Brualla, and P. P. Srinivasan, "Mip-nerf: multiscale representation for anti-aliasing neural radiance fields." in ICCV, 2021.
[35] J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman, "Mip-nerf 360: Unbounded anti-aliased neural radiance fields," in CVPR, 2022.
[36] D. Verbin, P. Hedman, B. Mildenhall, T. Zickler, J. T. Barron, and P. P. Srinivasan, "Ref-nerf: Structured view-depene dependent appearance for neural radiance fields." in CVPR, 2022.
[37] W. Hu, Y. Wang, I.. Ma, B. Yang, L. Gao, X. Liu, and Y. Ma, "Tri-miprf: Tri-mip representation for efficient anti-aliasing neural radiance fields," in ICCV, 2023.
[38] J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman, "Zip-nerf: Anti-aliased grid-based neural radiance fields," in /CCV, 2023.
[39] T. Müller, A. Evans, C. Schied, and A. Keller, "Instant neural graphics primitives with a multircsoluuton hash encoding," ACM ToG, 2022.
[40] C. Reiser, R. Szeliski, D. Verbin, P. Srinivasan, B. Mildenhall, Geiger, Barron, and Hedman, "Merf: Memory-efficient radiance fields for real-time view synthesis in unbounded scenes," TOG, 2023.
[41] A. Chen, 7. Xu, Geiger, J. Yu, and Su, "Tensorf: Tensorial radiance fields," in ECCV, 2022.
[42] R. Li, Н. Gao, M. Tancik, and A. Kanazawa, "Nerface: Efficient sampling accelerates nerfs." in ICCV, 2023.
[43] L. Yariv, P. Hedman, C. Reiser, D. Verbin, P. P. Srinivasan, R. Szeliski, J. T. Barron, and B. Mildenhall, "Bakedsdf:: Meshing neural sdfs for real-time view synthesis," in SIGGRAPH, 2023.
[44] Z. Chen, T. Funkhouser, P. Hedman, and A. Tagliasacchi, "Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures." in CVPR, 2023.
[45] T. Hu, S. Liu, Chen, T. Shen, and J. Jia, "Efficientnerf efficient neural radiance fields," CVPR, 2023.
[46] Z. Cui, L. Gu, X. Sun, X. Ma, Y. Qiao, and T. Harada, "Aleth-nerf: Illumination adaptive nerf with concealing field assumption,"" " in AAAI, 2024.
[47] Y. Cai, J. Wang, A. Yuille, Z. Zhou, and A. Wang, "Structure-aware sparse-view x-ray 3d reconstruction." in CVPR, 2024.
[48] I. Ma, X. from blurry images," in CVPR, 2022.
Li, J. Liao, Q. Zhang, X. Wang, J. Wang, and P. V. Sander, "Deblur-nerf: Neural radiance fields
[49] N. Pearl, T. Treibitz, and S. Korman, "Nan: Noise-aware nerfs for burst-denoising." in CVPR, 2022.
[50] Z. Yang, H. Yang. Z. Pan, X. Zhu, and L.. Zhang, "Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting." arXiv preprint arXiv:2310. 10642, 2023.
[51] G. Wu, T. Yi, Fang, Xie, Zhang, W. Wei, W. Liu, Q. Tian, and X. Wang. "4d gaussian splatting for real-time dynamic seene rendering." arXiv preprint arXiv:2310.08525 2023.
[52] Luiten, G. Kopanas, B. Leibe, and D. Ramanan, "Dynamic 3d gaussians: Tracking hy persistent dynamic view synthesis." arXiv preprint arXiv:2308. 09713, 2023.
[53] N. Keetha, J. Karhade, K. M. Jatavallabhula, G. Yang, S. Scherer, D. Ramanan, and J. Luiten, "Splatam: Splat, track & map 3d gaussians for dense rgb-d slam." arXiv preprint arXiv:23/2.02/226, 2023.
[54] V. Yugay, Y. Li, T. Gevers, and M. R. Oswald, "Gaussian-llam: Photo-realistic dense slam with gaussian splatting," " arXiv preprint 2rXw::2312.10070, 2023.
[55] H. Matsuki, R. Murai, P. H. Kelly, and A. J. Davison, "Gaussian splatting slam," arXiv preprint mmXiv:2312.06741,22 2023.
12

---

[56] C. Yan, D. Qu, D. Wang, D. Xu, Z. Wang. B. Zhao, and X. Li, "Gs-slam: Dense visual slam with 3d gaussian splatting," arXiv preprint arXiv:231 1.17000, 2023.
[57] Z. Liang, Q. Zhang. Y. Feng. Y. Shan, and K. Jia, "Gs-ir: 3d gaussian splatting for inverse rendering," " arXiv preprint arXiv:2311. 100.23311.667,, 2023.
[58] T. Xie, Z. Zong. Qiu, X. Li. Y. Feng. Y. Yang, and Jiang. "Physgaussian: Physics-integrane 3d gaussians for generative dynamics." arXiv preprint arXiv:2311.22990 2023.
[59] Y. Jiang, J. Tu, Y. Liu, X. Gao, X. Long, W. Wang, and Y. Ma, "Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces," arXiv preprint arXiv:2311.17977, 2023.
[60] X. Liu, X. Zhan, J. Tang, Y. Shan, G. Zeng. D. Lin, X. Liu, and Z. Liu, "Humangaussian: Text-driven 3d human generation with gaussian splatting," arXiv preprint arXiv:2311.10611, 2023.
[61] Kocabas, J.-H. R. Chang, J. Gabriel, O. Tuzel, and A. Ranjan, "Hugs: Human gaussian splats," arXiv preprint ar Xiv: 1. 179910, 2023.
[62] S. Hu and Z. Liu, "Gauhuman: Articulated gaussian splatting from monocular human videos," arXiv preprint arxiv: 2023.
[63] J. Tang, J. Ren, H. Zhou, 7. Liu, and G. Zeng. "Dreamgaussian: Generative gaussian splatting for efficient 3d content creation," " arXiv preprint arXiv:2309. 16653, 2023.
[64] T. Yi, J. Fang, G. Wu, L. Xie, X. Zhang, W. Liu, Q. Tian, and X. Wang. "Gaussiandreamerr: Fast generation from text to 3d gaussian splatting with point cloud priors," arXiv preprint arXiv:2370.08529, 2023.
[65] Y. Liang, X. Yang. Lin, Li, X. Xu, and Y. Chen, "Luciddreamer:: Towards high-fidelity text-to-3d generation via interval score matching." arXiv preprint 1Xix:2311.11284 2023.
[66] Y. Cai, Y. Liang. J. Wang, A. Wang, Y. Zhang. X. Yang, Z. Zhou, and A. Yuille, "Radiative gaussian splatting for efficient x-ray novel view synthesis," " ECCV. 2024.
[67] M. Zwicker, H. Pfister, J. Van Baar, and M. Gross, "Ewa volume splatting," in Proceedings Visualization, 2001. VIS'O1., IEEE, 2001.
[68] G. Kopanas, J. Philip, T. Leimkühler, and G. Drettakis, "Point-basee neural rendering with per-view optimization."" in Computer Graphics Forum, 2021.
[69] R. Prabhakar, S. Agrawal, D. Singh. Ashwath, and V. Babu, "Towards practical and efficient high-resolution hdr deghosting with cnn," in ECCV, 2020.
[70] Q Yan, D. Gong, Q Shi, A. d. Hengel, C. Shen, Reid, and Y. Zhang, "Attention-guide network for ghost-free free high dynamic range imaging." in CVPR, 2019.
[71] "Blender." httpa: //www. blender org//
[72] A. Paszke, S. Gross, F. Massa, Lerer, J. Bradbury, G. Chanan, Killeen, Z. Lin, N. Gimelshein,
L. Antiga, et al., "Pytorch: An imperative style, high-performance deep learning library," in NeurlPS, 2019.
[73] D. Kingma and J. L. Ba, "Adam: method for stochastic optimization," in ICLR, 2015.
[74] 7. Wang, A. C. Bovik, H. R. Sheikh, and F. P. Simoncell, "Image quality assessment: from error visibility to structural similarity," 7IP, 2004.
[75] R. Zhang, Isola, A. A. Efros, Shechtman, and O. Wang, "The unreasonable effectiveness of deep features as perceptual metric," in CVPR, 2018.
[76]
"Photomatix Pro 6." https:: //www. hdrsoft.com.
[77] R. Martin-Brualla N. Radwan, M. S. Sajjadi, J. T. Barron, A. Dosovitskiy, and D. Duckworth, "Nerf in the wild: Neural radiance fields for unconstrained photo collections." in CVPR, 2021.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please refer to the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please refer to Sec.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach addressing problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
This paper does not include any theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in the appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Please refer to Sec. and Sec. 4.1 for details, Code and data are publicly available https: /// htt//gittttt github. com/ caiyuanhao1998/HD
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification:
Code and data are available at https:///ithubb wwt. /998/
HDR-GS
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.. cc/
oolicy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: haaliii/ggdes//eeee
/ /nips...
sssisssnn wwsssssssiiiyyyyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer Sec. 1 for the implementation details of training and testing.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Please refer to the experiment part, i.e., Sec. 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer to Sec. 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https https://teet httppp::///uripp/// cc/public/tthiccciiiiee
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer to Sec.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have carefully credited all previous works we used in the paper. The license and terms are properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide source code with instructions.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
19


Reviews:
[{"review_id": "wrlvvdpmx2", "reviewer": "Reviewer_E3vH", "summary": "This paper introduces HDR-GS, a framework designed for efficient rendering of high dynamic range (HDR) novel views. HDR-GS leverages a Dual Dynamic Range (DDR) Gaussian point cloud model that utilizes spherical harmonics for HDR color fitting and an MLP-based tone-mapper for low dynamic range (LDR) color rendering. Given an exposure time, HDR-GS can reconstruct the corresponding LDR image, achieving a form of controllable tone-mapping. The method demonstrates significant improvements over state-of-the-art NeRF-based methods in terms of both speed and image quality.", "questions": "My main concern is about the novelty issue mentioned in the Weaknesses section.\n\nIs it possible to apply HDR-GS to data where each viewpoint has a different exposure time?", "limitations": "The authors may discuss the difficulty of data acquisition in real-world settings.", "rating": 5, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "HDR-GS achieves 1000x faster inference speed compared to HDR-NeRF. Its training is also efficient. The results shows significant improvement in image quality.\n\nThe framework  is sound. The authors provide a detailed derivation process, demonstrating the motivation and rationale behind the framework design. The tone mapper design enables controllable exposure time for reconstrcting LDR image.", "weaknesses": "This method requires taking photos with different exposure settings at each camera position and additional HDR image data in $L_c$ to calculate the loss function. These photos and data are relatively difficult to obtain in practice.\n\nThe entire pipeline is quite similar to HDR-NeRF, including the tone mapper MLP. Essentially, the authors replace the NeRF MLP with Gaussian splatting. To adapt HDR-NeRF to Gaussian splatting, the authors propose several key modifications: 1) camera recalibration and point cloud generation; 2) a constant bias $b$ in Equation 8; and 3) using $L_c$ instead of a unit exposure loss $L_u$. However, these modifications are minor and contribute only slightly. Moreover, the poor performance of the baseline model in Table 3a indicates that LDR supervision alone, without GT HDR images, is insufficient for reconstructing the HDR point cloud. This somewhat weakens the novelty. There is not ablation study for the constant bias $b$ in Equation 8.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "&nbsp;\n## Response to Reviewer E3vH\n&nbsp;\n\n`Q-1:` Questions about the training data\n\n`A-1:` (i) Actually, our method only requires an LDR image with a single exposure time $\\in$ {$t_1, t_3, t_5$} at each view to train in Eq.(15).\n\n(ii) **As claimed in Lines 198 - 200, we do not use HDR images for supervision $\\mathcal{L}_c$ in real scenes, $\\gamma = 0$ in Eq.(17).** We only use HDR supervision for synthetic scenes to align the HDR output with the Blender rendering style. This is because the transformation from LDR to HDR is not unique. It is affected by camera parameters, rendering software settings, etc. In our work, synthetic HDR images are synthesized by the software Blender with specific settings and the evaluation metrics for HDR images are hard pixel alignment. Thus, it is reasonable to enforce a constraint on the rendered HDR images. To this end, HDR-NeRF uses the GT camera response function (CRF) correction coefficient $C_{0}$ to rectify the HDR output. But $C_{0}$ is a strong scene-specific prior and not available in real practice because CRF is the target to model, not a given condition. Hence, we use tone-mapped HDR images for supervision instead. For fair comparison, we conduct experiments without using HDR restrictions in the following table. HDR-GS is 14.91 dB higher than HDR-NeRF on HDR.\n\n|Method|HDR-NeRF|HDR-GS|\n|:-|:-:|:-:|\n|HDR|13.35|28.26|\n|LDR-OE|36.48|40.82|\n|LDR-NE|34.47|36.01|\n\nIn Figure 6, our method renders more visually pleasant HDR images without HDR supervision than HDR-NeRF in real scenes.\n\n&nbsp;\n\n`Q-2:` Comparison of HDR-GS and HDR-NeRF\n\n`A-2:` Our HDR-GS is different from HDR-NeRF in\n\n(i) Motivation. HDR-NeRF aims to learn the neural HDR radiance fields. It is an implicit 3D representation that lacks geometric information.  In contrast, HDR-GS aims to reconstruct 3D HDR Gaussian point clouds that capture the scene geometry. It is an explicit 3D representation with better controllability and interactivity.\n\n(ii) Technique. (1) In Lines 40 - 45, HDR-NeRF suffers from a time-consuming rendering scheme. It samples many points to compute densities and colors for each ray. In contrast, HDR-GS adopts rasterization to render different tiles in high parallelism on GPU, enjoying much faster speed. (2) The tone-mapping operation of HDR-GS is also fundamentally different from that of HDR-NeRF. HDR-NeRF conducts tone-mapping in a ray-tracing manner. It converts HDR to LDR at all sampled points for every single ray. This ray-tracing tone-mapping only extracts the color information along a single ray and further slows down the inference speed. In contrast, HDR-GS performs a global tone-mapping operation that converts HDR colors of all 3D Gaussians to LDR colors at one time. This operation captures more contextual color information for HDR-to-LDR transformation and accelerates the inference speed. The following table shows the results of using the two tone-mapping methods on HDR-GS. Our global tone-mapping is better and faster.\n\n|Tone-mapping|Train Time (min)|Infer Speed (fps)|LDR-NE|LDR-OE|HDR|\n|:-|:-:|:-:|:-:|:-:|:-:|\n|Ray Tracing (HDR-NeRF)|58|78|39.51|34.68|36.12|\n|Global Infer (Ours)|34|126|41.10|36.33|38.31|\n\n(iii) Data. HDR-NeRF uses the normalized device coordinate (NDC) system. But, as analyzed in Lines 174 - 181, the NDC system restricts the representing ability and transformations of 3D Gaussians. Plus, the datasets collected by HDR-NeRF do not provide point clouds for initialization. To address these problems, we recalibrate the camera parameters and compute the initial point clouds as Eq.(14). See Table 3 (a), our data recalibration and point cloud initialization lead to 2.27/2.58 and 4.84/4.56 dB improvements on LDR-OE/LDR-NE, while alleviating the over-fitting issue.\n\n(iv) Performance. HDR-GS shows significant advantages over HDR-NeRF. HDR-GS surpasses HDR-NeRF by 1.91 dB on HDR in synthetic scenes (Table 1) and 3.84 dB on LDR in real scenes (Table 2), while enjoying 1000x inference speed. In Figure 6, HDR-GS reconstructs clearer HDR details and brightness.\n\n&nbsp;\n\n`Q-3:` Performance of 3DGS trained with GT HDR images\n\n`A-3:` We conduct experiments to compare HDR-GS with 3DGS directly trained with GT HDR images in the following table. HDR-GS outperforms 3DGS by 25.98, 22.86, and 8.16 dB on LDR-NE, LDR-OE, and HDR.\n\n|Method|LDR-NE|LDR-OE|HDR|\n|:-|:-:|:-:|:-:|\n|3DGS|15.12|13.47|30.15|\n|HDR-GS|41.10|36.33|38.31|\n\nWe analyze these results:\n\n(i) The suboptimal HDR result of 3DGS stems from that the high-frequency details and broader range of brightness in HDR images are hard to capture by the spherical harmonics (SH) with limited order in original Gaussian point clouds. \n\n(ii) The LDR results of 3DGS are poor because the original 3DGS cannot control the light intensity according to the exposure time.\n\n(iii) Besides, training 3DGS with HDR images is hard in real scenes where HDR images are difficult to obtain. In contrast, HDR-GS only requires LDR images for supervision in practice.\n\n&nbsp;\n\n`Q-4:` Ablation of $b$ in Eq.(8)\n\n`A-4:` We follow your advice to do an ablation of $b$ in the following table.\n\n|Method|LDR-NE|LDR-OE|HDR|\n|:-|:-|:-|:-|\n|w/o $b$|40.72|36.08|38.05|\n|with $b$|41.10|36.33|38.31|\n\n&nbsp;\n\n`Q-5:` Can HDR-GS be applied to data where each view has a different exposure time?\n\n`A-5:` Yes. HDR-GS only requires an LDR image with a single exposure time at each view to train. We do experiments with training views of different exposure times in the following table. HDR-GS performs better and faster.\n\n|Method|Train Time (min)|Infer Speed (fps)|LDR-NE|LDR-OE|HDR|\n|:-|:-:|:-:|:-:|:-:|:-:|\n|HDR-NeRF|551|0.12|37.94|36.21|35.83|\n|HDR-GS|35|123|39.76|37.40|38.05|\n\n&nbsp;\n\n`Q-6:` The difficulty of data acquisition in real world\n\n`A-6:` The real-world data is easy to obtain. In real scenes, HDR-GS only requires a single LDR image at each view to train and no HDR images are required. The exposure time can be easily set in the camera and read from the EXIF files.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QOmNn37Spk", "reviewer": "Reviewer_tLXM", "summary": "This paper proposes a 3D Gaussian Splatting-based method, HDR-GS, for the high dynamic range novel view synthesis. To efficiently perform this task, a new Dual Dynamic Range Gaussian point cloud model is presented (in Section 3.1). This point cloud model has more attributes including the HDR color, exposure time, and a global-shared neural network functioning as a tone-mapper. Then in Section 3.2, two Parallel Differentiable Rasterization processes are designed to render the low and high dynamic range colors. Besides, in Section 3.3, the authors recalibrate the camera parameters for the real and synthetic multi-view HDR datasets to make the scene-level data suitable for 3D Gaussian Splatting-based algorithms.", "questions": "I have two questions:\n\na) In the paper of HDR-NeRF, the training set and testing sets of real scenes are completely separate and have no intersection according to the implementation details. However, in the official github repository of HDR-NeRF, the training and testing sets for real experiments have intersection, which makes me very confusing. So I want to figure out, in your real experiments, did you separate the training and testing sets or just follow the official code of the HDR-NeRF? I think this is important.\n\nb) I want to know the training stability of the proposed method since I found HDR-NeRF easily collapse and need to train multiple times to make it work on some scenes.", "limitations": "Yes, the authors have analyzed the limitations and broader impact of the method in section 4 and 5 of the supplementary pdf.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "+ It is a good attempt to design the first 3DGS-based framework for the task of high dynamic range novel view synthesis. The core idea of assigning more attributes to the Gaussian point cloud model and rendering the high dynamic range and low dynamic range views in the parallel rasterization is novel and cool, which makes the proposed 3D Gaussian model multifunctional and have great practical values in photography, film making, etc.\n\n+ The performance is superior. The running speed of the proposed HDR-GS is more than a thousand time that of the state-of-the-art NeRF-based method, HDR-NeRF. As compared in Table 1 of the main paper, previous NeRF-based methods suffer from the slow inference speed (< 0.2 fps). The proposed HDR-GS can not only infer at a much faster speed of 126 fps (>> 30 fps) but also surpass the SOTA method by large margins. These advantages enable the HDR-GS to capture and measure dynamic scenes, e.g., the camera on the robot, in real time. \n\n+ The writing style is clear and easy to follow. I notice that the authors did not follow the mainstream introduction of the 3D Gaussian Splatting part, which I think is very confusing. Instead, the authors adopted the strategy of summarizing first and then dividing. They first introduced what attributes the Gaussian point cloud model contains, and then gradually introduced its working pipeline. This introduction order can give the readers an overall knowledge of what the Gaussian model is (its attributes) and thus helps the readers better understand the method.\n\n+ The data re-calibration is a bonus. In my opinion, the biggest obstacle to researching the topic of high dynamic range novel view synthesis is the data issue. Although the multi-view synthetic and real datasets are collected, the normalized device coordinates and lack of SfM points for initialization cannot make the 3DGS work because of the severe blur and overfitting problems, especially for the unbounded scene-level reconstruction. How to re-calibrate the data is critical and executing the SfM algorithm is time-consuming.", "weaknesses": "There are some minor issues:\n\n- In Line 124 – 131, the authors analyzed the advantages of using log tone-mapper than linear one from the point of view of training stability. Yet, my understanding is that the option of taking the logarithm can shorten the gaps between the training data samples, which makes the originally discrete data samples become more continuous. The processed data samples are easier to fit by neural networks. The exposure time in table 3d is an example. So, adding this analysis can help better explain the motivation of taking the logarithm instead of directly using the linear form.\n\n- In Eq (14), the authors just used part of images of a scene to re-calibrate the camera poses without explanation. My concern is why not use all of the images instead of just using the views under the same exposure time? Did you try that? It is interesting to know and analyze this result.\n\n- It would be better to add some legend or annotation to the teaser figure like the unit of the numerical results, higher is better or lower is better. Because some results are higher is better, e.g., PSNR while some are lower is better such as training time.\n\n- More details of the experimental setup in section 4.1 could be provided to make the implementation clearer. For instance, the authors used the LPIPS as one of the metrics but they did not specify which perceptual network is adopted since this choice may drastically affect the LPIPS score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "&nbsp;\n\n## Response to Reviewer tLXM\n\n&nbsp;\n\n`Q-1:` Analysis of logarithmic tone-mapper vs. linear tone-mapper\n\n`A-1:` Thanks for providing a mathematical explanation of the logarithmic tone-mapping operation. We will add this analysis in the revision with acknowledgment.\n\n&nbsp;\n\n`Q-2:` Why using the images under the same exposure of different views to recalibrate the camera parameters instead of using all images?\n\n`A-2:` The recalibration is done by the Structure-from-Motion (SfM) algorithm. SfM needs to detect and then match feature keypoints between different views. Thus, if using all images for calibration, the feature detection and matching might be less accurate due to the change of light intensity at the same position. As a result, the calibrated camera poses may also be inaccurate. We conduct the experiment of using the camera parameters calibrated from all images on the synthetic datasets to train HDR-GS, the model achieves results of only 34.21, 36.19, and 33.75 dB on HDR, LDR-OE, and LDR-NE. Compared to Table 3 (d), these results are 2.67, 3.52, and 1.53 dB lower than the lowest results of single-exposure recalibration with $t_1$ = 0.125 $s$ on HDR, LDR-OE, and LDR-NE.\n\n&nbsp;\n\n`Q-3:` Add some legend or annotation to the teaser figure\n\n`A-3:` Thanks for reminding us. In Figure 1, we report five metrics of HDR novel view synthesis including PSNR in dB, inference speed in fps (frames per second), training time in minutes, SSIM, and LPIPS score. In particular, lower values are better for LPIPS and training time, while higher values are better for the other metrics. We will add a legend to explain the metrics.\n\n&nbsp;\n\n`Q-4:` Which perceptual network is used for the LPIPS score?\n\n`A-4:` Following the same test settings as HDR-NeRF for fair comparison, we use AlexNet [77] as the perceptual network to compute the LPIPS score. We will add more experimental details in Section 4.1.\n\n[77] ImageNet Classification with Deep Convolutional Neural Networks. In NIPS 2012.\n\n&nbsp;\n\n`Q-5:` Are the training sets and testing sets of real scenes completely separate?\n\n`A-5:` Yes, of course. You have a good observation. We also found this mistake in the official repo of HDR-NeRF. We rewrite the data splitting code to make sure there is no overlap between training and testing sets of real scenes. Please check the submitted code, which is consistent with the description of the implementation details in our main paper.\n\n&nbsp;\n\n`Q-6:` The training stability of our HDR-GS\n\n`A-6:` We did not experience model collapse during the training process. Our models were trained once and succeeded. We conduct five repeated experiments on the synthetic datasets. The PSNR results are shown in the following table. The performance fluctuation is within 0.21, 0.16, and 0.09 dB on LDR-OE, LDR-NE, and HDR. These results suggest the robustness and training stability of our HDR-GS.\n\n| Experiment | 1 | 2 | 3 | 4 | 5 | Avg |\n|:-------------|:--:|:--:|:--:|:--:|:--:|:--:|\n| HDR    | 38.31 | 38.23 | 38.34 | 38.22 | 38.29 | 38.28 |\n| LDR-OE | 41.10 | 40.95 | 41.13 | 40.89  | 41.06 | 41.03 |\n| LDR-NE | 36.33 | 36.17 | 36.39 | 36.15 | 36.31 | 36.27 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3BQjb7jqsi", "reviewer": "Reviewer_VwS8", "summary": "The manuscript presents a novel approach to High Dynamic Range (HDR) novel view synthesis (NVS) by proposing a new framework called High Dynamic Range Gaussian Splatting (HDR-GS). The proposed HDR-GS framework addresses the limitations of existing HDR NVS methods, which are primarily based on Neural Radiance Fields (NeRF) and suffer from long training times and slow inference speeds.", "questions": "1)\tThe authors' method is simple and effective, and the significant speed improvement due to 3D Gaussian Splatting (3D-GS) is not surprising. However, it would be beneficial if the authors could provide an analysis of what specifically leads to the substantial performance difference between HDR-GS and HDR-NeRF.\n2)\tThe authors chose to model HDR color first using the initialized DDR Gaussian point cloud rather than LDR color. It would be helpful if the authors could elaborate on the rationale behind this choice and how it positively impacts the generation of HDR and LDR results.", "limitations": "Section 3.2 could be omitted or condensed. The mathematical explanation of 3D-GS rendering and the snowballing process does not seem to be a primary innovation of this paper. Additionally, the emphasis on parallel HDR and LDR rendering and rasterization is somewhat unclear. The authors should clarify why parallel rendering is necessary for their approach, as it seems that similar results could be achieved without parallel rendering. Specifically, the necessity of parallel rendering for the subsequent loss calculation involving 𝐼𝑙 and 𝐼ℎ should be justified.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Comprehensive experiments demonstrate that HDR-GS outperforms state-of-the-art NeRF-based methods while achieving 1000× faster inference speed and requiring only 6.3% of the training time. The methodology is straightforward and well-organized, and the experimental results are impressive. The paper is logically structured and easy to follow.", "weaknesses": "While the methodology is straightforward and the experimental results are strong, the authors could further highlight their innovations and provide detailed explanations of their model design and theoretical underpinnings. This would enhance the manuscript's persuasiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "&nbsp;\n## Response to Reviewer VwS8\n&nbsp;\n\n`Q-1:` Highlight of innovations and explanations of model design and theoretical underpinnings\n\n`A-1:` We propose the first 3DGS-based framework with 1.91 dB improvements and 1000x inference speed for HDR novel view synthesis (NVS). Firstly, we find that directly applying the original 3DGS to HDR NVS does not work well. In Table 3 (a), 3DGS only yields 12.35/11.83 dB on LDR-OE/LDR-NE and cannot render HDR images. To improve the performance, we make innovations in algorithm and data.\n\n(i) Algorithm. The original 3DGS is designed for LDR imaging. If directly enforcing an HDR constraint to the output of 3DGS like Eq.(16), 3DGS only yields a suboptimal result of 30.15 dB on HDR, as shown in the following table. This is because the broader range of brightness and high-frequency details in HDR images are difficult to model by the spherical harmonics with limited order in the original Gaussian point clouds. Plus, 3DGS cannot change the light intensity of the rendered LDR images, thus yielding poor results on LDR.\n\n|Method|LDR-NE|LDR-OE|HDR|\n|:-|:-:|:-:|:-:|\n|3DGS + HDR supervision|15.12|13.47|30.15|\n|HDR-GS|41.10|36.33|38.31|\n\nTo address these problems, we design the DDR Gaussian point cloud model based on the tone-mapping theory [16]. Our DDR Gaussian model in Eq.(1) contains more attributes required for HDR NVS, such as HDR color, exposure time, etc. We perform a global tone-mapping operation on all Gaussian point clouds at one time to extract more contextual color information and accelerate the inference speed. Then, we develop the parallel differentiable rasterization to render the LDR and HDR images. As shown in Table 3 (a) of the paper and the above table, our innovation in Gaussian point cloud model leads to significant improvements of 21.64, 17.36, and 8.16 dB on LDR-OE, LDR-NE, and HDR.\n\n(ii) Data. As analyzed in Lines 174 - 181, the normalized device coordinate (NDC) system restricts the representing ability and spatial transformations of 3D Gaussians. Besides, the datasets collected by HDR-NeRF do not provide point clouds for initialization. To address these problems, we recalibrate the camera parameters and compute the initial point clouds as Eq. (14). In Table 3 (a), the data recalibration and point cloud initialization lead to 2.27/2.58 and 4.84/4.56 dB improvements on LDR-OE/LDR-NE, while alleviating the over-fitting issue of 3DGS.\n\n&nbsp;\n\n`Q-2:` Analysis of the performance difference between HDR-GS and HDR-NeRF\n\n`A-2:` We are not sure if the \"performance difference\" you mention refers to speed or image quality. Thus, we analyze both aspects. \n\nFirstly, two techniques lead to the speed advantage of HDR-GS.\n\n(i) Rendering Scheme. As analyzed in Lines 40 - 45, HDR-NeRF suffers from a time-consuming rendering scheme for LDR and HDR images. It needs to sample many 3D points and then compute their densities and colors for every single ray, severely slowing down the training and inference processes. In contrast, HDR-GS adopts the parallel differentiable rasterization for LDR and HDR rendering. As described in section 3.2, the rasterization computes different tiles divided from the 2D projection in high parallelism on GPU, thus enjoying much faster inference speed.\n\n(ii) Tone-mapping Method. HDR-NeRF conducts tone-mapping in a time-consuming ray-tracing manner. It needs to convert HDR to LDR for all sampled points along every single ray. It can only exploit the limited color information along a single ray for the tone-mapping transformation and requires repeated computation many times. In contrast, HDR-GS performs a global tone-mapping operation that converts the HDR colors of all Gaussian point clouds at one time. This operation can extract more contextual color information for HDR-to-LDR transformation and accelerate the inference speed. The following table compares the effects of the two tone-mapping methods on HDR-GS. Our global tone-mapping method performs better and faster than the ray-tracing tone-mapping of HDR-NeRF.\n\n|Tone-mapping|Train Time (min)|Infer Speed (fps)|LDR-NE|LDR-OE|HDR|\n|:-|:-:|:-:|:-:|:-:|:-:|\n|Ray Tracing (HDR-NeRF)|58|78|39.51|34.68|36.12|\n|Global Infer (Ours)|34|126|41.10|36.33|38.31|\n\nSecondly, although our work is based on 3DGS, directly using 3DGS for HDR NVS yields poor image quality. To improve the quality to surpass HDR-NeRF, we make innovations. Please refer to `A-1` for details.\n\n&nbsp;\n\n`Q-3:` Why modeling the HDR color first?\n\n`A-3:` This is because HDR images capture a broader range of illuminance levels retaining the details in dark and bright regions. HDR images contain information stored in LDR images. Thus, HDR-to-LDR transformation is an easy information compression process. In contrast, the transformation from LDR to HDR is an ill-posed problem because it needs to reconstruct the missing brightness and details that are not captured in LDR images. This process usually involves more complex computations. We do experiments to compare HDR-to-LDR and LDR-to-HDR transformations in the following table. HDR-to-LDR performs better.\n\n|Transformation|LDR-NE|LDR-OE|HDR|\n|:-|:-:|:-:|:-:|\n|LDR-to-HDR|37.65|34.79|29.54|\n|HDR-to-LDR|41.10|36.33|38.31|\n\n&nbsp;\n\n`Q-4:` Condensing section 3.2\n\n`A-4:` We will follow your advice to condense this section\n\n&nbsp;\n\n`Q-5:` Questions about the parallel differentiable rasterization\n\n`A-5:` In fact, parallel differentiable rasterization does not mean parallel computing for HDR and LDR  in one rasterization. We perform HDR rasterization and LDR rasterization, separately. The word \"parallel\" here has two meanings:\n\n(i) The HDR and LDR rasterization processes query the same attributes and spatial transformations of the same 3D Gaussians in Eq. (10) - (13), except for the color.\n\n(ii) The tiles divided from the 2D projection are computed in parallel on GPU during the rasterization to accelerate the speed, see Line 165 - 169 and Eq. (13).\n\nWe will add an explanation in the revision", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
&nbsp;

## General Response to All Reviewers

&nbsp;

Thanks for your time and valuable comments. We really appreciate you for recognizing our framework soundness (`E3vH`,`tLXM`, and `VwS8`), method novelty (`tLXM` and `VwS8`), outstanding performance (`E3vH`,`tLXM`, and `VwS8`), and good writing and presentation (`E3vH`,`tLXM`, and `VwS8`). We have written a separate detailed response to each of you, respectively. We address all the issues raised in detail and clarify a few miscommunications

Our code, models, and recalibrated data will be released to the public.

Feel free to ask us if you have any other questions. Looking forward to discussing with you.

Meta-review: This paper received three positive leaning reviews -- one 5 (borderline accept), one 6 (weak accept), and one 7 (accept). 

There was general appreciation for the problem addressed in the paper --- HDR novel view synthesis using 3D GS that allows for considerable speedups both during training and inference. Reviewers also consistently liked the presentation of the paper, quality of the results and were convinced about the effectiveness of the proposed method. There were some concerns about the relatively low technical novelty of the method --- claiming that the paper is a relatively straightforward extension of HDR NeRFs. But on the other hand, there was a consensus that the paper takes a principled approach towards integrating HDR captures into 3DGS framework that provides substantial practical benefits. Given this, an acceptance decision was reached.
Decision: Accept (poster)