IBGS: Image-Based Gaussian Splatting
Authors: Hoang Chuong Nguyen
Wei Mao
Jose M. Alvarez
Miaomiao Liu
Venue: NeurIPS 2025 poster ()
Keywords: 3D vision
novel-view synthesis
Primary area: applications
TL;DR: Image-based Gaussian Splatting

Abstract: 3D Gaussian Splatting (3DGS) has recently emerged as a fast, high-quality method for novel view synthesis (NVS). However, its use of low-degree spherical harmonics limits its ability to capture spatially varying color and view-dependent effects such as specular highlights. Existing works augment Gaussians with either a global texture map, which struggles with complex scenes, or per-Gaussian texture maps, which introduces high storage overhead. We propose Image-Based Gaussian Splatting, an efficient alternative that leverages high-resolution source images for fine details and view-specific color modeling. Specifically, we model each pixel color as a combination of a base color from standard 3DGS rendering and a learned residual inferred from neighboring training images. This promotes accurate surface alignment and enables rendering images of high-frequency details and accurate view-dependent effects. Experiments on standard NVS benchmarks show that our method significantly outperforms prior Gaussian Splatting approaches in rendering quality, without increasing the storage footprint. Our project page is available at https://hoangchuongnguyen.github.io/ibgs.

IBGS: Image-Based Gaussian Splatting
Hoang Chuong Nguyen'
Wei Mao
Jose M. Alvarez?
Miaomiao Liu
Australian National University
2NVIDIA miaomiao liu@anu edu.au au
boangehuong.gguuendauuuccm edu. edu.
wei.mao www..reeearchemaii.ccom com josea@nvidia.com com
Abstract
3D Gaussian Splatting (3DGS) has recently emerged as fast, high-quality method for novel view synthesis (NVS). However, its use of low-degree spherical harmonics limits its ability to capture spatially varying color and view-dependent effects such specular highlights. Existing works augment Gaussians with either a global texture map, which struggles with complex scenes, or per-Gaussian texture maps, which introduces high storage overhead. We propose Image-Basee Based Gaussian Splatting, an efficient alternative that leverages high-resolution source images for fine details and view-specific color modeling. Specifically, we model each pixel base color from standard 3DGS rendering and a Iearned residual inferred from neighboring training images. This promotes accurate surface alignment and enables rendering images high- frequecyy details and accurate fiew-dependent effects. Experiments standard NVS benchmarks show that our method significantly outperforms prior Gaussian Splatting approaches in rendering quality, without increasing the storage footprint. Our project page is available at httpp::/hhaagccuuongggyyya github. io/ibga
color as a combination
Introduction
Recently, Neural Radiance Fields (NeRF) 23 and 3D Gaussian Splatting (3DGS) [66 have emerged as advanced techniques for Novel View Synthesis (NVS) thanks to their high-quality image rendering. 3DGS, in particular, outperforms NeRF in terms rendering speed and faster optimization. However, since each Gaussian primitive in 3DGS can only represent single color at given camera viewpoint, 3DGS struggles to recover high-frequency details of the scene appearance without large number of Gaussians 4.. Furthermore, due to the smooth characteristic of the color representation i.e., spherical harmonic (SH) functions, it is hard for 3DGS capture complex view-dependent effects such as reflections and specular highlights [4]
To address this issue, recent works have attempted to model Gaussian's 's spatially varying colors by either mapping each Gaussian-ray intersection to a global texture map [34), or a per-Gaussian texture map 28 33 While global texture maps perform well for single-object scenes 34 they struggle with complex multi-object scenes due to the difficulty of learning global mapping. Per-Gaussian can handle real- world scenes with multiple objects, but incurs significant storage overhead because the number of parameters per Gaussian grows quadratically as the texture- map resolution increases. This storage overhead constrains the resolution of per-Gaussian's 's texture map, leading to inferior modeling of high-frequency details in the rendered images. Additionally, such texture map still cannot handle view-dependent effects.
texture map 28 33
In this work, we propose drastically different approach to render high-frequency details and handle view-dependent effects while avoid significantly increasing the storage memory. Specifically, inspired by image- based rendering techniques 8 22), we introduce an Image Based Gaussian Splatting (IBGS) approach that utilizes the fighffreueeccy details and view-dependent effects captured in training images. During rendering, the color of a pixel consists of two components i.e., the base color
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

from the SH functions following the standard rasterization process of 3DGS, and the residual color learnt from the corresponding pixel intensities of neighboring training images. The base color is used to handle most surface appearance while the color residual augments the base color with fine-grained details and view-dependent effects which are preserved in the training images. To model the color residuals, we propose a novel color residual prediction module. Specifically, for each ray/pixel, we first project the intersection points between the ray and Gaussians onto neighboring training images to obtain pixel colors, which are then aggregated to get warped colors. Then, the warped colors together with the base colors are processed by a lightweight neural network to predict the color residual for each pixel. We further introduce an image synthesis loss that leverages those warped colors, enforcing both geometric accuracy and image quality. This leads to more precise Gaussian parameters with high opacity centered around the true surface, allowing to prune more Gaussians of low opacity while maintaining the rendering quality.
Furthermore, leveraging neighboring views enables our method to address inconsistent exposure across training views caused by the auto-exposure behavior of modern cameras. Different from prior work 66 17 that jointly optimize affine transformation matrix for each training view, we assume that images taken at nearby locations share similar global lighting conditions, and thus correct the camera exposure at novel views by mimicking the exposure setting of the closest source view. Unlike existing works 6 17 that only correct the exposure at training views, our strategy can generalize to images rendered at any viewpoint.
In summary, our contributions are: We propose an image-based Gaussian Splatting pipeline that captures both high-frequency details and riew-dependent effects that are challenging for prior methods to address. ii) We introduce a color residual module that leverages the training images to obtain better rendering quality with less number of Gaussians. iii) We introduce an exposure correction strategy, helping to improve the brightness of images rendered at any viewpoint by mimicking the exposure settings of their nearest neighbouring view. Our method sets new state-of-the-art performance on three benchmark datasets: Tanks and Temples, Deep Blending, and Mip-NeRF360.
2 Related Works
Image-based rendering aims to generate novel views by "borrowing" pixels from set of source images. The target pixel is a weighted blending of corresponding pixels obtained from those images. In early works 8 11 22), such blending weights are computed based on ray distance
22 or scene geometry [ Other works either tried to improve the scene geometry [5] 13 or use optical flow for
better correspondence [I 3 9. More recently, with the advance of neural rendering techniques (2), researchers have explored integrating it with image-based rendering 22
31 In particular, to render target pixel/ray, Suhail al. [29] first finds the corresponding epipolar lines in source views and sample points along such lines to obtain color features. These features are further fed into two feature aggregation modules subsequently for the final color. IBRNet (31) follows the volume rendering process as in NeRF [23] The color and densities of the sampled points on target ray are computed by a transformer [30) with the features from source views as input. Despite their impressive results, their rendering is time-consuming due to the use of large feature aggregation networks (i.c., the transformers) and uniform sampling along the ray. By contrast, to the best of our knowledge, we propose the first image-based Gaussian splatting method that not only obtains fine- mme-grained details from source images but also maintain fast rendering. Thanks to 3DGS, we only require projecting intersection points of ray with Gaussians, which are fairly sparse, to source views for aggregating image features. Moreover, instead of directly learning the final color from a large network, we propose to learn residual to the base color, which only requires lightweight network i.e., nine-layer convolutional network with 3 x 3 with kernels.
Gaussian Splatting, 3DGS [16] renders images at novel-views by performing alpha blending of the Gaussians color splatted onto the image plane. Although each splatted Gaussian can have large extent, its color is shared across all pixels, making it challenging for 3DGS to reconstruct fine-grained textures without using many Gaussians. To address this problem, prior works 4 28 B 34
attempt to model Gaussian spatially varying color by learning mapping from each Gaussian-ra ray intersection to a texture map. However, while learning a global texture map is challenging in scenes having multiple objects, learning per- Gau-saani texture maps 4 28 33 leads to higher storage requirement since it requires to store texture maps of all Gaussians. Additionally, these methods struggle in recovering complex view-dependent color as they utilize low-degree SH functions which have limited capacity
2

---

Weighted everage warped color
Camera position. orlentation aitference
Target view
Ad
warp
Ad
Source image
Medien tey-Gaussian Intersection
Gaussian rasterization
aD-1a-20 mapping
Color
Residual
Ac
Predicted color residual
Prediction
30
Network
mapping
A2
colar
Adg
C2
warD
Ad
C
position. orientation cifference
Weighted
Geussian- rasterized color
worped color
Figure I: Our pipeline. The color of each pixel (clinal) consists of two components: base color (in pink boxes) which follows the standard 3DGS rendering process and a color residual Ac predicted from the warped color of different source views cm While the figure shows example of using only two source views, in practice, our method can process an arbitrary number of source views.
to handle complex view-dependent colors 4 Unlike these methods, we leverage information from training views to render photorealistic images by predicting high residuals for pixels whose color cannot fully recovered via Gaussian rasterization (i.e., the base color), especially regions with fine-grained details or view-dependent colors.
Apart from color modeling, existing works also target improving other aspects of Gaussian Splatting. In particular, 7
12 [5] pursue more accurate 3D reconstruction by enforcing flat Gaussians via propose compression and quantization methods to reduce the memory requirements for storing optimized Gaussians while preserving rendering quality. (35] improves the adaptive density control strategy of 3DGS by exploiting the per-pixel gradient directions, whereas [18] views the Gaussian densification process as state transition of Markov Chain Monte Carlo samples. Our method orthogonal to these works and can be integrated with them to further boost the performance or reduce memory usage.
hard 7 15 or soft constraint [2]. the other hand, [10 25 26
3 Method
3.1 Preliminary: 3D Gaussian Splatting
represents the scene as set of 3D Gaussian primitives. Each 3D Gaussian G is pa- rameterized by a 3D position μ R3 and a covariance matrix R3x3 where the covariance is decomposed into rotation matrix R E S3(3) and a diagonal scale matrix E 33x3 such that RSS R , To render an image viewpoint, each Gaussian splatted on the image plane obtain 2D Gaussian G2D. The color of pixel p can be then computed using the volume rendering equation,
3DGS [16]
c(p)
wi
Σ w. Ur (h v:), where w; a:Ti and T. (1- (I)
with V; denoting the vector from the camera center the center of the 7th Gaussian, (, v) mapping the SH coefficients h of the Gaussian to a color conditioned on the direction v. The scalar E indicates the SH degree, which determines the expressivity of (h, v). The weight o; = (,G)D(()) (p) is defined as the product of the Gaussian opacity and 2D Gaussian value evaluated pixel p.
During training, the attributes of each Gaussian, including (1) its colors represented using spherical harmonic coefficients h E 3=((())) (2) the opacity E R, (3) the center μ E R3, (4) the rotation matrix parameterized by quaternion q E IR1, (5) scaling factors s E TR3 are optimized using the color rendering loss.

---

3.2 Modeling Spatially Varying and View-Dependent Color
In the color modeling of 3DGS [6], at given viewpoint, although Gaussian G; can cover multiple pixels in the image, it represents only a single color, as the SH coefficients h and view direction v are shared across all pixels. This limits the model's capacity in modeling color of regions with high-frequency details. Moreover, due to the low-degree SH function (1 S 3) utilized by 3DGS [16] to model view-dependent color, this method struggles to capture complex view-dependent effects, such as reflections or specular highlights. A naive solution is to increase the SH degree 1, which quadratically increases the number of SH coefficients, thereby leading to high storage requirement.
In this work, we present a solution to model high-frequency details and view-dependent color of the image without increasing the storage footprint. Specifically, we propose IBGS, an image- image-based Gaussian splatting method that models (1) the base color from SH functions, (2) color residual term capturing view-specific and high-frequency information from neighboring source images. We formulate it as follows:
N
C Ginal (p)
wi Wr(h. v.)
+ F(c(D). d(p), fAcm (p)) fddm (p)
(2)
Base color c(p)
Color residual Ac(p)
where d(p) E R is the direction of the camera ray passing through pixel Acm. E R and Adm E R$ denotes appearance and camera features extracted from the m nearby source view, respectively. F(-) lightweight network that takes the extracted features as input predict residual term Ac(p) supplementing the base color, c(p), produced by Gaussian rasterization.
Given the extracted multi-view features Acm. that capture high-frequency details and color variation across different viewpoints, we use this features to predict the pixel color in the current view. By utilizing the multi-vien color observations, our model learns how lighting effect changes across viewpoint, thus being able to produce accurate view-dependent color for the target view.
In the following sections, we first describe the two main components of our method: feature extraction from source views (Sec. (.3) and color residual prediction (Sec. (344). Then, we present an exposure correction approach (Section. 3.5 to correct the exposure of the rendered images caused by inconsistent exposure camera setting. Fig. depicts the overall pipeline of our method.
3.3 Feature Extraction from Source Views
To predict color residual, Ac(p), we first extract features from multiple source images. For target pixel p. we obtain the color information Acm (p) from each source image by first finding the intersection between (1) the camera ray originating at the camera center with direction d(p) and (2) the plane parameterized by the Gaussian center μ4; and normal vector n.
nf (H, o) n d() d(p). d())
(p)
Here incorporate normal vector n; E 3 as additional learnable attribute of each Gaussian G.. Given the intersection point x; (p). we project it onto the image plane of nearby source views from which the color information is extracted. For the th source view, this is achieved as,
(p) B(TTm (x, (p)), Creal) C,
warp
with Tm. (x) denoting a function that projects the intersection point to the image plane of the source view. The function B(-) takes an image coordinate input produce bilinear-inten interpolaad color obtained from the input source image Creal "/x'33
warp (p) is only accurate if the intersection point (())) is close to Based on Eq. 4 the warped color c
the actual surface. This implies that it is not necessary to project all the Gaussian-ray intersections to the source views, as floating Gaussians that are far from true surface introduce noise into the extracted appearance features. Following 2DGS [15), we consider the actual surface lies near the median intersections such that the accumulated transmittance T; (see Eq, I is close to 0.5, and thus only project the K median intersection points set of K warped colors {c werrp K Jk=1 for each source view
(x )= to the source views. As result, obtain
k==

---

Subsequently, we compute the weighted average color for each source view and measure its deviation from the Gaussian rasterized craterien color c(p) (computed via Eq. as below,
Wk worP (P)
Acm (p)
Charr (p) c(p),
warp (p)
(5)
k=
where Wy the same blending weight of the Gaussian colors computed from Eq. T Intuitively, Sm cwaP(()) is used to approximate the true pixel color by leveraging the information from neighboring views. Thus, it constrains the weight Wy the Gaussians near the true surface to be larger than those of others. Apart from the appearance features, we also compute Adm. the difference in the camera position and orientation between the target and each source view,
Ad
Om -0 dm (p)
dm (p)
x(p) Om (p)
x(P)
Wk (p)
(6)
Om
km
Wk
with Om being the camera center of the mhth source view. We repeat this feature extraction process for the M nearby source views, yielding a set of color features [Acm)m= and camera features {Adm JM m=1 which are used as input the color residual prediction network.
3.4 Color Residual Prediction
We employ lightweight network to predict the color residuals, consisting of two main components: per-pixel feature extractor E(.) and a CNN decoder D(-). The extractor has PointNet-style structure 27 to handle an arbitrary number of source views M. For each view m, processes the color feature Acm (p) and camera features Adm (p) through two linear layers of 32 output dimension followed by ReLU activation function,
fm (p) = E(Acm (p), Adm
We then apply max- max-pooling to the set of vectors {fm (p) to obtain the aggregated feature f(p) E 3 Assembling these features across all pixels yields a feature map F E RR "x x32 Similarly, by stacking c(p) and (p) over all pixels, we form the Gaussian-rasterizz image C E #x and the ray-direction map D R x W respectively. This information are then fed into nine-layer convolutional decoder (with kernel size of 3) to predict the color residual map AC E TR HRHx
AC = D(C, D. F).
(8)
Finally, obtain the final image by adding the predicted residuals to the Cautsian-rasterized image,
Clinal = + AC
(9)
3.5 Exposure Correction
Due to varying lighting conditions at different locations, cameras with an auto-exposure setting can capture images with inconsistent brightness, which introduces noise into the optimization of the Gaussians. To stabilize the training, prior works 66 [7 optimize color affine transformation matrix for each training view. This approach, however, can not generalize to correct the exposure of images rendered novel views. To address this issue, assume that images taken at nearby locations share similar global lighting conditions and thus propose to correct the exposure of the Gaussian- rasterized image by mimicking the exposure setting of the closest source view. In particular, we first obtain an affine transformation matrix A representing the exposure at the target view by solving the following least-square problem,
((()) 1
2
A arg min
Σ PEX
C
warD (p)
(10)
A
ACRSX4
where x is set of pixels with valid coordinate when mapping to the source view, and c warp (p) is color warped from the nearest source view. After that, we use A to correct the exposure of the
rendered image as, "xxp (p) = c(()] Note that exposure correction is applied, the color
with corrected exposure exxpo (p) used in place of the originally rendered color c(p) for computing appearance features (Eq. 5). residual prediction (Eq, g and obtaining the final image (Eq. 9)
5

---

3.6 Optimization
The overall loss function used to train our method is,
L Ligh + X1 Lpooto + xermall
(I)
where A1, >g are loss weights.
Color Rendering Loss Ligb. We compute the loss for both the final image and the Gaussian- rasterized image (i.c., the base image). The two terms are balanced by the weight as follows, Ergb = (C,C Creal) + (1 (- Creal).
(12)
In particular, L is defined as
L(C, C BCI C + (1- CSSM (C, Cat). where 3 is set to 0.8 and C, C E R. 1 XxxXX x3 are the rendered and ground-trum -ruth image, respectively.
(13)
Multi-view Color Consistency Loss L-phot- We also enforce photometric consistency across neighboring views to encourage accurate pixel matching,
M Σ
Photo
L(C C
(14)
M
where Cwarr is the warped image obtained by stacking all (p) computed in Eq. 5
Normal Consistency Loss Lnormal- Following 2DGS
[5] we apply the normal consistency loss to
improve the overall geometry,
norral
Σ - (1-) (depth(p))
(15)
PER
with Ω being a set of all pixel coordinates and N denoting the rasterized normal map. Neepth is the normal map derived via finite difference of the point map X constructed from x(p) (see Eq. 6
Visibility-based Source Views Selection. To find the nearby source views, we first compute the distance between the target and each source view, then use the closest S views as candidates to search for M visible source views (i.e., M I S). Specifically, for each pixel p. we only use the t source view for feature extraction if it satisfies the following the condition,
[x(x(p)) 2xx"arP (p))
(x())) + =(x; warp (p)) S. with X (p)
(x())) X.)
(16)
where T is depth error threshold, z(x) denotes the depth value of the 3D point The point map X, be obtained by transforming the point map X, of source view the coordinate system of the target view. Intuitively, this approach performs depth consistency check to exclude the source views in which the point x(p) is not visible.
4 Experiments
4.1
Experimental Setup
Dataset. Following 3DGS [16,, we evaluate the NVS performance of our method using 2 scenes in the Tanks and Temples (TNT) 20 dataset, 2 scenes in the Deep Blending [4] dataset, and scenes in the Mip-NeRF360 2 dataset. We also show the results on 3 scenes in the Shiny dataset [32] which pose challenging view-dependent effects including specular highlight, reflection and disc diffraction. For all scenes, we use every 8th image for evaluation, and the rest for training.
Implementation Details. Similar 3DGS [16,, we train our method for 30, 000 iterations. During the first 7, 000 iterations, we set λ A = 0 and only activate the photometric and normal consistency loss thereafter with \ 0.3 and A 0.03. The weight ? initially set to and then decreased to 0.5 during the last 20, 000 iterations. Regarding syper-parameters, we set SH degree 2, number of median intersection points
4, number of candidate source views S S= 4, number of visible source views M = 3 and depth error threshold Gaussians with opacity lower than 0.05. Following (6), we apply the exposure compensation from (6 and our proposed exposure correction method only to the TNT dataset. We use Adam optimizer [9 to train the residual prediction network. The initial learning rate is 0. 0.001, which halves iterations 18, 000 and 25,000. All experiments are conducted using single RTX 4090 GPU.
0.001. We also prune

---

Table 1: Comparison between our method and previous works in three datasets. We measure the storage memory (Mem) in MB and the number of Gaussians (#Gauss) in millions. We report two released result from TexturedGausian
[4, with and without total memory usage.
Dataset
Mip-NeRI 360
Tanka& Temples (TNT)
Deep blending
Method MMerrc
PSNRT
SSIM
LPIPS]
#Gaaus
Mem
PSNR
SSIM
LPIPS #Gaus
Mem
PSNK
SSIM
LPIPS
#Gauss
Mem
Mip-NeRF 360 [2
27.69
0.792
0.237
x
22.22
0.759
0.257
29.40
0.901
0.245
x
8.6
Dnsi 24
25,30
0.671
0.371
x
13
21.72
0.723
0.330
X
13
23.62
0.797
0.423
x
13
3DGS 16
27.69
0.825
0.203
3.22
764
23.11
0.840
0.184
1.75
415
29.53
0.904
0.242
3.14
745
SuperGauss
35
27.31
0,815
0.209
3,04
1021
23.72
0.847
0.179
1.50
502
28.85
0.901
0.250
2.27
762
TexturedGauss
27.35
0.827
0.186
24.26
0.854
0.168
28.33
0.891
0.270
TextureGauus
27.26
3.50
1047
24.28
1.30
691
28.52
1,00
668
Ours
28.33
0.837
0.186
1.59
291
24.84
0.869 0.148
0.75
143
30.12
0.912 0.237
1.11
197
Table 2: Comparison on three scenes in the Shiny dataset with challenging view- dependend colors.
Scene (effect)
Guitars (specular highlight)
Lab (reflection)
(diffiaacion)
MetrodiMetric
PSNRT
SSIM?
LPIPS].
#Gauss
Mem
PSNR
SSIM
LPIPS #Gauss
Mcm
PSNR
SSIM
LPIPS
#Gauss
Mcm
3DGS [6]
29.37
0.947
0.131
0.41
97
29.17
0.927
0.123
0.63
150
29.10
0.935
0.110
0.51
121
SuperGauss
33
30.43
0.952
0.121
0.39
131
29.38
0.932
0.107
0.61
204
29.49
0.944
0.091
0.70
234
Ours
35.65
0.953
0.105
0.18
35.06
0.966
0.056
0.27
66
35.23
0.955
0.060
0.27
4.2
Results
In Tab. [ we show the comparison of our method with prior methods in terms of NVS performance, number of Gaussians and storage usage. The results reveal that our method consistently achieve the best image quality across all datasets. For the PSNR metrics, our method gains an improvement of at least 0.64, 0.56 and 0.59 dB in the Mip-NeRF 360, TNT and Deep blending datasets, respectively. Notably, on the Mip-NeRF 360 and TNT datasets, our method reduces the number of Gaussians and the storage by at least 62% and 42% respectively, compared to existing Gaussian Splatting methods 4 16 33 and still outperforms them. For the Deep blending dataset, although we use slightly more Gaussians than TexturedGauss (4, our method consumes 70% less storage. This is because TexturedGauss needs to store the texture maps of all Gaussians, which requires significantly more memory compared to storing the source images as our method.
Tab. 2 presents the comparisons on three scenes in the Shiny 132 dataset with challenging view- dependent effects. For this dataset, train 3DGS [16) and SuperGaussian 33 using their official implementations. Despite using fewer Gaussians, our method achieves significantly better NVS performance, with at least 5.22 dB gain in PSNR. This demonstrates the superior capability of our method in modeling view-dependent color compared to previous works.
Qualitative Results. Fig. 2 shows the qualitative comparison between method, 3DGS [66] and SuperGauss 333. The results in the first two scenes reveal that 3DGS and SuperGauss struggle in reconstructing frequecyy details, while our method delivers more photorealistic results. We also show the two color components of our method, including a base image and a predicted residual map. While the base image alone exhibits the same detail deficiencies as 3DGS, adding our predicted color residuals helps to restore realistic textures. For scenes with complex view-dependent color, our method also demonstrates more compelling visual results, as shown in the last two scenes of Fig. 2 In these cases, 3DGS [16] and SuperGaussian [4] fail to capture the specular highlights and reflection effects, while our method successfully recovers the complex view-dependent colors in the final rendered images by predicting high residuals for these challenging regions. Interestingly, in the zoomed-in in region of the Guitars scene, our method can decompose the color into diffuse and specular component modeled via the base color and predicted residuals, respectively. Comparisons with more baseline methods and additional qualitative results can be found in the supplementary material.
4.3 Ablation Studies
Tab. presents the ablation study results. Base color only: Discarding the predicted color residual leads to a significant drop in the image quality. This highlights the importance of the color residual module. W/o color consistency loss Ephoro: Training without the loss Cphoto results in less accurate projections onto the source views, thereby reducing the quality of the rendered images. Use source colors Cm as network's 'ss input: Here, we use the full colors warp obtained from the source images
7

---

Ground-truth
3DGS 116
SuperGaussian 38
Our final image Canal
Our basc image C
Our predicted residuals AC
713
Figure 2: Qualitative results. Our method can render images with both high-frequency details (first two scenes) and view-dependent effect (last two scenes). However this cannot be achieved by 3DGS (16) and SuperGaussian 33
8

---

Table 3: Ablation study on the exposure correction, color consistency loss and the network's ' input.
Tanks& (TNT)
MipNeRF-360
Method
PSNRT
SSIM1
LPIPS]
PSNR
SSIM
LPIPS
Full
24.84
0.869
0.148
28.33
0.837
0.186
Bave color unly
23.06
0.836
0.202
27.08
0.814
0.227
W/o color consistency loss photo sraup network's input source color em
24.70
0,866
0.152
28.31
0.833
0.192
24.61
0.867
0.150
28.21
0.837
0.187
exposure correction
24.28
0.152
Table 4: Ablation study on opacity threshold.
Dataset
Mip Mip-NeRF 360
Tanks&Temples
Deep blending
Method (threshold)
PSNRT
SSIM
LPIPS]
#Gauss
Mem
PSNR
SSIM
LPPPS
#Gauss
Mem
PSNR
SSIM
LPIPS
#Gauss
Mem
3DGS (0.005)
0.825
0.203
23.11
1.75
415
29.53
0.242
3.14
745
Our (0.005)
28.42
0.836
0.183
2.61
456
24.76
0.869
0.146
1.23
220
29.91
0.907
0.234
2.41
405
3DGS (0.05)
27.51
0,818
0.221
1.46
346
23.52
0.837
0.202
0.74
175
29.16
0.902
0.256
1.03
243
Ours (0.05)
28.33
0.837
0.186
1,59
291
24.84
0.869
0.148
0.75
143
30.12
0.912
0.237
1.11
197
as input to the residual prediction network, instead of their difference Acm from the base color. As result, this approach performs consistently worse than our full model. W/o exposure correction: Removing the exposure correction also results in performance drop in the TNT dataset which exhibits inconsistent camera exposure across viewpoints
Fig. illustrates that our method can improve the exposure of the rendered image in both underexposure and overexposure cases, leading to higher image quality. More ablation studies can be found in the supplementary materials.
Additionally, we compare the sensitivity of our method and 3DGS [17] to the total number of Gaussians. For this experiment, we train our method and 3DGS with different opacity thresholds (0.005 and 0.05) used for pruning the Gaussians. Tab. reveals that 3DGS requires a large number of low-opacity Gaussians achieve good rendering quality, its performance degrades when higher opacity threshold is used. In contrast, with the same large threshold, our method can reduce the number of Gaussians while retaining most of the image quality.
Conclusion
In this paper, we present IBGS, an image-bused Gaussian Splatting pipeline that is capable of rendering photorealistic images with both high-frequency details and view-dependent effects. Our key contribution is the color residual module, which leverages fine-grained textures and vienndependent information in nearby source images predict residual term added to the base Gaussian-rasteriza color. Moreover, we introduce the exposure correction module to improve the brightness of the rendered image by mimicking the exposure of the closest source view. Extensive experimental results show that our method consistently outperforms previous works across different datasets.
WESTER
WESTER
WESTER PAC
PAC
PAC
713
713
713
Ground truth
With exposure correction
W/o exposure correction
Figure 3: Comparison of rendered images with and without exposure correction. Our method can correct the exposure in both underexposure (top) and overexposure (bottom) cases.

---

Limitations. Our method may struggle in a sparse-view setting, in which obtaining dense pixel correspondences used for residual prediction is challenging. Additionally, due to the additional computations in the rendering process, our method achieves lower rendering speed and requires higher runtime memory compared to 3DGS [6.. We discuss this in more detail in the supplementary material.
Broader Impacts. Our method has no immediate societal impacts. However, its downstream applications, such as 3D reconstruction [S or controllable human modeling [2I], can potentially be abused for malicious purposes such as unauthorized reconstructions, identity fraud.
Acknowledgement. This research was supported in part by the Australia Research Council ARC Discovery Grant (DP200102274).
References
Floating textures. In Computer graphics forum, pages 409-418. Wiley Online Library, 2008
[2] Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2022.
[3] Dan Casas, Christian Richardt, John Collomosse, Christian Theobalt, and Adrian Hilton. 4d model flow: Precomputed appearance aligmment for real-time 4d vidco interpolation. In Computer Graphics Forum, pages 173-182, Wiley Online Library, 2015.
[4] Brian Chao, Hung-Yi Yu Tseng, Lorenzo Porzi, Chen Gao, Tuotuo Li, Qinbo Li, Ayush Saraf, Jia-Bin Huang, Johannes Kopf. Gordon Wetzstein, et al. Textured gaussians for enhanced 3d scene appearance modeling. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2025.
[5] Gaurav Chaurasia, Sylvain Duchene, Olga Horking. and George Drettakis. Depth synthesis and local warps for plausible image-based navigation. ACM transactions on graphics (TOG), 32(3):1-12, 2013.
[6] Danpeng Chen, Hai Li, Weicai Ye, Yifan Wang. Weijian Xie, Shangjin Zhai, Nan Wang, Haomin Liu, Hujun Bao, and Guofeng Zhang. Pgsr: based gaussian splatting for efficient and high-fidelity surface reconstruction. IEEE Transactions on Visualization and Computer Graphics, 2024.
[7] Pinxuan Dai, Jiamin Xu, Wenxiang Xie, Xinguo Liu, Huamin Wang. and Weiwei Xu. High-quality surface reconstruction using gaussian surfels. In ACM SIGGRAPI PH 2024 Conference Papers, pages 1-11, 2024.
[8] Paul E Debevec, Camillo Taylor, and Jitendra Malik. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 465-474. 2023.
[9] Ruofei Du, Ming Chuang. Wayne Chang, Hugues Hoppe, and Amitabh Varshney. Montage4d: interactive seamless fusion of multiview video textures. In 13D. pages 5-1, 2018.
[10] Sharath Girish, Kamal Gupta, and Abhinav Shrivastava. Eagles: Efficient accelerated 3d gaussians with lightweight encodings. In European Conference on Computer Vision, pages 54-71. Springer, 2024.
[11] Steven J Gortler, Radek Grzeszzzuk, Richard Szeliski, and Michael F Cohen. The lumigraph. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2. pages 453 1644.46477 2023.
[12] Antoine Guédon and Vincent Lepetit. Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5354-5363, 2024.
[13] Peter Hedman, Tobias Ritschel, George Drettakis, and Gabriel Brostow. Scalable inside-out image-hased rendering. ACM Transactions Graphics (TOG), 35-6):1-11, 2016.
[14] Peter Hedman, Julien Philip, True Price, Jan-Michael Frahm, George Drettakis, and Gabriel Brostow. Deep blending for free-viewpoint image-based rendering. ACM Transactions on Graphics (ToG), 37(6):1-15. 2018.
[15] Binbin Huang, Zehao Yu. Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. In ACM SIGGRAPH 2024 conference pupers, pages 1-11, 2024.
[16] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139-1 2023.
10

---

[17] Bernhard Kerbl, Andreas Meuleman, Georgios Kopanas, Michael Wimmer, Alexandre Lanvin. and George Drettakis. A hierarchical 3d gaussian representation for real-time rendering of very large datasets. ACM Transactions on Graphics (TOG), 43(4):1-15, 2024.
[18] Shakiba Kheradmand, Daniel Rehain, Gopal Sharma, Weiwei Sun, Yang-Che Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. 3d gaussian splatting as markov chain monte carlo. Advances Neural Information Processing Systems, 37:80965-80900 2024.
[19] Diederik P Kingma and Jimmy Ba. Adam: arXiv:1412.6980, 2014.
method for stochastic optimization. arXiv preprint
[20] Amn Knapitsch, Jaesik Park, Qian- Yi Zhou, and Vladien Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics (ToG), 36(4): 13, 2017.
[21] Muhammed Kocahas, Jen-Hao Rick Chang, James Gabriel, Oncel Tuzel, and Anurag Ranjan. Hugs: Human gaussian splats. In Proceedings of the IEEF/CVF conference on computer vision and pattern recogmition, pages 505-515, 2024.
[22] Marc Levoy and Pat Hanrahan. Light field rendering. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2. pages 441 141-452.27 2023.
[23] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65 (1):99-106, 2021.
[24] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM Irunsactions on graphics (TOG), 41(4):1-15, 2022.
[25] KL Navaneet, Kossar Pourahmadi Meibodi, Soroush Abbasi Koohpayegani. and Hamed Pirsiavash. Compgs: Smaller and faster gaussian splatting with vector quantization. In European Conference on Computer Vision, pages 330-349. Springer, 2024.
[26] Simon Niedermayr, Josef Stumpfegger, and Rüdiger Westermann. Compressed 3d gaussian splatting for accelerated novel view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2024.
[27] Charles Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652-660, 2017.
[28] Victor Rong. Jingxiang Chen, Sherwin Bahmani, Kiriakos Kutulakos, and David Lindell. Gstex: Per-primitive texturing of 2d gaussian splatting for decoupled appearance and geometry modeling. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). pages 3508-3518. IEBE, 2025.
[29] Mohammed Suhail, Carlos Esteves, Leonid Sigal, and Ameesh Makadia. Light field neural rendering. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, pages 8269-8279, 2022.
[30] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances neural information processing systems, 30, 2017.
[31] Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul P Srinivasan, Howard Zhou, Jonathan T Barron, Ricardo Martin-Brualla, Noah Snavely, and Thomas Funkhouser. Ibrnet: Learning multi-view image-based rendering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690-4699,
[32] Suttisak Wizadwongsa, Pakkapon Phongthawee, Jiraphon Yemphraphai, and Supasor Suwajanakorn. Nex: Real-time view synthesis with neural basis expansion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3534-8543, 2021.
[33] Rui Xu, Wenyue Chen, Jiepeng Wang, Yuan Liu, Peng Wang, Lin Gao, Shiqing Xin, Taku Komura, Xin Li, and Wenping Wang. Supergaussians: Enhancing gaussian splatting using primitives with spatially varying colors. arXiv preprint 241. 18966, 2024.
[34] Tian-Xing Xu, Wenbo Hu, Kun Lai, Ying Shan, and Song- Hai Zhang. Texture-gs: Disentangling the geometry and texture for 3d gaussian splatting editing. In European Conference on Computer Vision, pages 37-53. Springer, 2024.

---

[35] Zongxin Ye. Wenyu Li, Sidun Liu, Peng Qiao, and Yong Dou. Absgs: Recovering fine details in 3d gaussian splatting. In Proceedings of the 32nd ACM International Conference Multimedia, pages 1053-1061, 2024.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our claims made in the abstract and introduction accurately reflect the paper's contributions and scope. We further demonstrate it in our methodology and experiment sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Yes, we discuss the limitation of our method in the main paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
12

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented
by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide all the information needed to reproduce the main experimental results in the main paper, supplementary materials
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
13

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [No]
Justification:
Our code will released upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://mips.co
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
7 //nips. htcp//blcc//uddeeeeeeeeeeeeeeeee
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide experimental details in the main paper and supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
This paper does not report error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
14

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: provide experimental details, including the computer resources needed to reproduce the experimental results in the main paper and supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: This work conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We provide discussion on the potential societal impacts of our method in the main paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
15

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have cited the original owners of all assets used in this paper properly.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
16

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
That the paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
17

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /ttpp/////euips..... wwwparripssccooooeeeeeee cc:Conferencess///////eeoecc/eee for what should or should be described.
18


Reviews:
[{"review_id": "KW9mjM18VS", "reviewer": "Reviewer_kSME", "summary": "This paper focus on improving the representation efficiency and rendering quality of 3D Gaussian Splatting when dealing with spatially varying colors and view-dependent effects. The key idea is to introduce image-based rendering techniques into the existing 3DGS rendering pipeline. The proposed image-based rendering approach leverages highly-detailed texture information from nearby source views to supplement the area that cannot be expressed with high quality using Gaussian Splats alone in the target view. Experiments demonstrate that the proposed approach surpass existing Gaussian-based methods across several common used datasets, especially outperforming on datasets containing challenging view-dependent colors. In addition, an interesting finding is that the proposed image-based rendering approach can achieve such high-quality rendering while using much fewer Gaussian Splats than the baseline.", "questions": "1. **Concerns on rendering speed**: I hope that authors can showcase the rendering speed of this solution, as one of the major advantages of Gaussian Splats (GS) lies in its rendering efficiency. If the IBR-based approach results in a significant loss of rendering speed, the practical value of this work may be greatly impacted.", "limitations": "As discussed in the weakness section, image-based rendering not only requires a large collection of images as input but also potentially slow down the rendering of Gaussian Splats. This could hinder its efficiency in transmission and distribution, making it potentially suitable only for devices with high-end GPUs to perform novel view synthesis.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n1. **Strong performance**: The proposed image-based rendering solution appears to be very effective. It achieves very high visual quality metric scores, ranking top on the highly competitive novel view synthesis benchmark leaderboard. Additionally, the subjective quality of the rendered images shown in Figure 2 highlights its exceptional rendering quality and demonstrates the contribution of the image-based rendering module.\n2. **Novelty of the key idea**: To the best of the reviewer's knowledge, no prior work is to combine Gaussian Splatting with IBR-based techniques and achieve such significant results. For the reviewer, this work is sufficiently novel.\n## Weakness\n1. **Potential decrease in rendering speed**: From the point of the reviewer, the trade-off for introducing image-based rendering is a decrease in rendering speed. In the proposed pipeline, each ray in the rendered view must query multiple points to obtain the corresponding reference color from nearby views, followed by processing through a per-pixel feature extractor and a CNN decoder to produce the final color. While this pipeline is highly effective, it also appears to introduce additional complexity. This could significantly impact rendering efficiency, potentially compromising the method's ability to achieve real-time rendering speeds (>30 FPS) for online applications.\n2. **Reliance on a large collection of source views**: The proposed solution, when performing novel view synthesis, not only requires Gaussian Splats as the scene representation but also relies on a large number of source views and a lightweight neural network to obtain residual color. However, the need for a large number of source views may introduce a heavier storage burden, which could also impact its application.", "was_revised": true, "final_justification": "I maintain my \"borderline accept\" rating because this work is sufficiently novel and achieves excellent rendering quality, making it worthy of acceptance. What prevents me from giving a higher score are the significant impact on rendering speed and the potential burden of requiring a large collection of source views when the work might be applied to transmission scenarios in the future. Overall, this is a solid piece of work where the merits outweigh the limitations, and it deserves to be accepted.", "rebuttal": "We sincerely appreciate the reviewer for spending time reviewing our paper. The provided comments are very constructive and valuable to improve our paper. We address the reviewer's concerns as below.\n\n---\n\n### 1. Potential decrease in rendering speed\n\nWhile IBGS introduces additional computations for predicting color residuals, resulting in slower rendering speed compared to 3DGS (as shown in Tab. 3 and Tab. 6, supplementary material), it consistently maintains rendering speeds **above 30 FPS** in most cases. This demonstrates that our method can still achieve real-time rendering, making it suitable for online applications. Furthermore, as demonstrated in Tab. 6 (supp. mat.), predicting the color residual at half resolution allows IBGS to nearly double its rendering speed while still achieving superior image quality compared to 3DGS and SuperGaussians.\n\n---\n\n### 2. Reliance on a large collection of source views\n\nWe would like to clarify that the \"memory values\" reported for our method in Tab. 1 and Tab. 2 (main paper) are the total storage required for both the Gaussians and the source images. The results indicate that our method uses **at least 42% less storage** compared to 3DGS, SuperGaussians, and TexturedGaussian. We further show the breakdown of the storage requirement of our method in the table below. The reported storage is in Megabytes (MB), averaged across all scenes within each dataset.\n\n|Dataset|Source images and cameras|Color residual network|Gaussians|Total|\n|-------|:-----------------------:|:---------------------:|:---------:|:-------:|\n|MipNeRF-360|35.99|0.27|255.62|291.89|\n|Tanks & Temples|22.06|0.27|120.35|142.68|\n|Deep blending|18.96|0.27|178.28|197.51|\n|Shiny|21.34|0.27|38.71|60.32|"}, {"review_id": "aF5bWUzjdg", "reviewer": "Reviewer_3NJ8", "summary": "This paper proposes Image-Based Gaussian Splatting (IBGS), a hybrid approach for novel view synthesis that augments 3D Gaussian Splatting (3DGS) with image-based rendering. IBGS predicts residual colors from nearby source images and includes a learned exposure correction, leading to improved photorealism, particularly for high-frequency and view-dependent effects. Results on multiple benchmarks indicate better visual quality compared to existing methods.", "questions": "Please address the questions in the weakness listed above.", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Pros:\n- The paper is well-written and clearly identifies the limited capacity of 3DGS to model high-frequency and view-dependent effects, addressing this through a hybrid strategy.\n- The method combines residual learning from source images with 3DGS, improving appearance while managing memory usage. The exposure correction is a practical addition for inconsistent lighting across views.\n- IBGS achieves higher quantitative metrics and better visual results than prior methods on standard benchmarks.\n\nCons:\n1. Reduced rendering speed: One main advantage of 3DGS is its rendering speed, but IBGS sacrifices this by requiring per-pixel, per-view feature warping and neural prediction at inference time. I did not see results regarding the inference speed.\n\n2. Robustness: How reliable are the rendered depths and normals, especially during the early stages of training when these quantities may be inaccurate or unstable? Since the intersection points—determined by depths and normals—are essential for your method’s success, can you provide quantitative comparisons and visualizations of rendered depths, normals, and reconstructed meshes, particularly for reflective or specular regions? Additionally, illustrating how these evolve throughout training would help assess and demonstrate the robustness of the proposed approach.\n\n3. Unclear total memory/storage requirements: IBGS requires access to all source images and depth maps (for accelerating rendering) at inference, which may become a bottleneck for large scenes and dense views. The experimental comparisons with standard 3DGS methods should also explicitly include the memory/storage for images and (precomputed) depth maps. Please clarify if the memory reports reflect these requirements, as in Supplementary Table 3.\n\nPlease provide a clarification and report on both training time and total memory usage, including storage for source images and precomputed depth maps\n\n4. Assumptions on Capture Trajectories. Does the proposed method assume “closed-loop” capture trajectories to ensure access to “nearby” views? For scenes with a forward-moving camera, there may be many out-of-frustum points when unprojecting to adjacent views.\n\n\n5. Exposure correction only for TNT dataset: Since exposure correction is presented as a key contribution, why is it applied only to the TNT dataset (as stated in line 237)? Please clarify the rationale and discuss whether the approach can generalize to other datasets and scenarios.\n\n6. Quantitative gain: The quantitative gain is impressive. Is the gain coming more from specular cases and fine-grained details? If it is more from fine-grained details, can it be resolved simply by adding more Gaussians?", "was_revised": true, "final_justification": "The authors did a good job in the rebuttal. Most of my concerns have been addressed. Although the proposed method does slow down the rendering and increase the storage space, the cost is acceptable, considering its noticeable performance gain. One thing missing is a detailed analysis of performance gains in specular regions, which, however, is not critical. Thus, I upgrade my rating to accept.", "rebuttal": "We sincerely appreciate the reviewer for spending time reviewing our paper. The provided comments are very constructive and valuable to improve our paper. We address the reviewer's concerns as below.\n\n---\n\n### 1. Reduced rendering speed\nWhile IBGS introduces additional computations for predicting color residuals, resulting in slower rendering speed compared to 3DGS (as shown in Tab. 3 and Tab. 6, supp. mat.), it consistently maintains rendering speeds **above 30 FPS** in most cases. This demonstrates that our method can still achieve real-time rendering, making it suitable for online applications. Furthermore, as demonstrated in Tab. 6 (supp. mat.), predicting the color residual at half resolution allows IBGS to nearly double its rendering speed while still achieving superior image quality compared to 3DGS and SuperGaussians.  To the best of our knowledge, we are the first image-based Gaussian splatting method capable of producing photorealistic images while preserving the fast rendering speed advantage of the Gaussian-splatting approach. \n\n---\n\n### 2. Training stability at early stage\nWe agree with the reviewer that the rendered depths and normal could be inaccurate especially in the early training stage and could lead to an unstable training if we apply the losses involving the rendered depth and normal. To cope with this, we deactivate the color residual module during early training stage, and only activate it when the 3D Gaussian primitives can reliably represent the scene geometry. Specifically, we train our method for a total of 30,000 iterations following the training strategy outlined below:\n\n- **First 7,000 iterations:**  We only optimize the 3D Gaussians using the color rendering loss $L_{rgb}$. At this stage, the geometric losses and the color residual prediction modules are deactivated by setting $\\lambda_1 = \\lambda_2 = 0$ (Eq. 10, main paper) and $\\gamma = 1.0$ (Eq. 11, main paper).\n- **Starting from the 7,000th iteration:** We activate the geometric losses, including the multi-view color consistency loss $L_{photo}$ and the normal consistency loss ${L}_{normal}$, to accurately model the scene geometry. This is done by setting $\\lambda_1 = 0.3$ and $\\lambda_2 = 0.03$.\n- **After the 10,000th iteration:** We activate the color residual prediction modules by linearly reducing $\\gamma$. In particular, $\\gamma$ reaches 0.5 at the 20,000th iteration and remains constant thereafter.\n\nIn summary, the color residual prediction module is only utilized when the geometry represented by the Gaussians are sufficiently reliable, and our strategy of decaying $\\gamma$ helps to avoid noisy gradient back-propagated to Gaussians during the initial training phase of the color residual prediction module.\n\n---\n\n### 3. Quantitative comparison of the reconstructed scene geometry\nIn Tab. 1 (supp. mat.), we provided the comparison of our method with others in reconstructing the scene geometry and novel-view synthesis. For all methods, given the rendered depth maps, TSDF fusion is applied to obtain reconstructed meshes, which are then evaluated by computing the F1 score against ground truth. The result reveals that our method achieves the second best performance in 3D scene reconstruction. Additionally, as requested by the reviewer, we provide the comparison for the rendered depth maps in the table below. Overall, our method produces the second best results in most scenes. \n\n(We report the absolute relative error for the rendered depth maps in the table below)\n\n|Scene/Method|3DGS|2DGS|GOF|RaDe-GS|PGSR|Ours|\n|:-----------:|:--:|:--:|:--:|:-----:|:--:|:--:|\n|Barn|0.063|0.010|_0.004_|_0.004_|**0.002**|_0.004_|\n|Caterpillar|0.104|0.014|0.011|0.010|**0.002**|_0.005_|\n|Courthouse|0.163|0.062|0.177|0.242|**0.005**|_0.051_|\n|Ignatius|0.058|0.008|0.004|0.004|**0.001**|_0.003_|\n|Meetingroom|0.247|0.046|0.027|_0.026_|**0.009**|0.031|\n|Truck|0.041|0.014|0.005|0.005|**0.002**|_0.004_|\n|Average|0.113|0.026|0.038|0.048|**0.004**|_0.016_|\n\nWhile PGSR [1] achieves the lowest depth error, our method significantly outperforms PGSR in novel-view synthesis with **2.82 dB** improvement in the PSNR metric (see Tab. 1, supp. mat). This shows IBGS not only obtains good geometric accuracy, but also illustrates a good balance between reconstructing the scene geometry and appearance. \n\n**Visualization of the rendered depths and normals and reconstructed meshes**: In accordance with NeurIPS 2025 policies, we are unable to provide figures for the rendered depths, normals and images in the rebuttal. We thanks the reviewer for understanding and will provide the visualizations in the revised version.\n\n---\n\n### 4. Clarification on training time and memory/storage requirements\nThe table below shows the average training time (in minutes) for IBGS, 3DGS, and SuperGaussians across three datasets. All methods are trained on a single RTX 4090 GPU. Overall, IBGS requires more training time than 3DGS due to the additional computations involved in predicting color residuals. In contrast, compared to SuperGaussians which aim to learn per Gaussian spatially varying textures, our method requires less training time. \n\n(We report the training time minutes, average across all scene in a dataset)\n\n|Method / Dataset|MipNeRF-360|Tanks&Temples|Deep blending|\n|---------------|:---------:|:-----------:|:-----------:|\n|3DGS|19|11|19|\n|SuperGaussian|55|30|60|\n|Ours|44|21|39|\n\nRegarding the storage requirement, we would like to clarify that IBGS requires to store all source images but will compute depth maps for neighboring source views during inference. In other words, it does not require storing depth maps for inference. In Tab. 3 and Tab. 6 (sup. mat.), we have reported (1) the total storage for storing the source images, source camera intrinsics/extrinsics, the color residual network's weights, the Gaussians and (2) rendering speed during inference. In the table below, we further show the detailed storage requirement for each dataset (in MB, average acrross all scenes within a dataset). \n\n|Dataset|Source images and cameras|Color residual network|Gaussians|Total|\n|-------|:-----------------------:|:---------------------:|:---------:|:-------:|\n|MipNeRF-360|35.99|0.27|255.62|291.89|\n|Tanks & Temples|22.06|0.27|120.35|142.68|\n|Deep blending|18.96|0.27|178.28|197.51|\n|Shiny|21.34|0.27|38.71|60.32|\n\nAdditionally, we also provide a comparison with 3DGS and SuperGaussian in terms of peak GPU VRAM consumption during inference in the table below. Our approach requires pre-loading all source images into VRAM, resulting in higher peak memory usage compared to 3DGS and SuperGaussians. This design choice eliminates the overhead of on-the-fly GPU memory allocation for images, thereby accelerating rendering. However, we agree with the reviewer's observation that this strategy can become bottleneck especially in large and dense scenes. One solution to mitigate this issue is to load each image into VRAM only when it is selected as a neighboring view required for rendering. Nevertheless, this approach introduces a trade-off, as it can slow down the rendering speed of IBGS. We thank the reviewer for this insightful comment and will discuss this limitation of IBGS in the revised version.\n\n(The VRAM memory is reported in GB)\n\n|Method|MipNeRF-360|Tanks and Temples|Deep blending|\n|------|:--------:|:---------------:|:-----------:|\n|3DGS|2.03|1.25|2.13|\n|SuperGaussians|2.76|1.45|2.44|\n|Ours|6.12|2.97|5.36|\n\n---\n\n### 5. Assumptions on Closed-loop Capture Trajectories:\nIBGS does not require such assumption for the camera trajectories as it has demonstrated good performance on three datasets that do not exhibit closed-loop camera trajectories, including the Deep blending dataset (Tab. 1, main paper), Shiny dataset (Tab. 2, main paper) and DTU dataset (Tab. 2, supp. mat.). Regarding the out-of-frustum points, the predicted color residual for those points is **0** as there is no source views pass the visibility check (Eq. 15, main paper). In other words, we only use the Gaussian-rasterized color for out-of-frustum points. \n\n---\n\n### 6. Why exposure correction is only applied for TNT dataset\nFollowing [1], we only apply the exposure correction to the TNT dataset since the camera auto-exposure setting is enabled while capturing images in this dataset, as discussed in [2]. In practice, the exposure correction module can be activated by default without concern for performance degradation, as it has minimal impact when there is no exposure variation across the training views. To validate this, we applied the exposure correction module to the MipNeRF-360 dataset and present the results below. It can be observed that the image quality remains largely unchanged when the exposure correction is applied. \n\n|                      |PSNR|SSIM|LPIPS|\n|----------------------|:---:|:---:|:----:|\n|W/o exposure correction|28.41|0.837|0.185|\n|With exposure correction|28.39|0.836|0.189|\n\n---\n\n### 7. Can adding more Gaussians result in more fine-grained details in the rendered images? \nWe agree with the reviewer that adding more Gaussians increases the capacity in modeling fine-grained details. However, it also leads to a heavier storage burden and still struggles to capture challenging view-dependent effects. In contrast, IBGS does not require a large number of Gaussians and is capable of modeling both fine-grained details and view-dependent color by leveraging information from the source images.\n\n---\n\n[1] Chen, Danpeng, et al. \"Pgsr: Planar-based gaussian splatting for efficient and high-fidelity surface reconstruction.\" IEEE Transactions on Visualization and Computer Graphics (2024).\n\n[2] Barron, Jonathan T., et al. \"Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields.\" Proceedings of the IEEE/CVF international conference on computer vision. 2021."}, {"review_id": "05qYRCNmuB", "reviewer": "Reviewer_efex", "summary": "This paper proposes IBGS, which improves the 3DGS's rendering by considering the color information from training-view images. The motivation comes from that the low-degree SH coefficient of 3dgs meets challenges to model the view-dependent rendering in some situations. By introducing the nearby-view training images, 3DGS can reconstruct the coarse and base color by SH coefficient and use colors from training views for fine-grained and view-dependent rendering. The results demonstrate the effectiveness of IBGS on several popular benchmarks.", "questions": "In the rebuttal, I would like to see further comparisons about other fine-grained detail recovery methods, and analysis about training costs.", "limitations": "yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The idea of introducing color information from nearby training-view images to improve 3DGS rendering is novel. The method implementation is intuitively reasonable, and the effect has been verified experimentally.\n2. From the Supplementary Material, the comparison of surface reconstruction is interesting. A framework that links rendering and accurate geometry is worth exploring. In this framework, geometry can be better constrained when optimizing color.\n\nWeaknesses:\n1. From the qualitative results and analysis, in addition to perspective-specific rendering, the improvement brought by the method mainly comes from the recovery of high-frequency details. An analysis and comparison with other methods designed for reconstructing details is important, such as AbsGS [1] and Octree-GS.\n2. The proposed methods introduce a group of training views in one iteration, which may cause an efficiency problem. The authors provide an analysis of inference fps in Material table 6. However, there is a lack of analysis about training costs.\n\n[1] AbsGS: Recovering Fine Details for 3D Gaussian Splatting (ACM MM 2024)\n\n[2] Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians (TPAMI 2025)", "was_revised": true, "final_justification": "I find that the authors have satisfactorily addressed the issues I raised in the initial review. I encourage the authors include the analyasis and results in the revision. Therefore, I will keep my original rate 'Accept' unchanged.", "rebuttal": "We thank the reviewer for spending time reviewing our paper, as well as acknowledging the strength and novelty of our method. We address the reviewer's concern as below. \n\n---\n\n### 1. Comparison to AbsGS\n\nWe include a comparison with AbsGS in Tab. 4 (supp. mat.).  Results indicate that our method consistently achieves better performance over AbsGS across all datasets. Note that AbsGS focuses on recovering high-frequency details through a novel homodirectional gradient-based densification strategy. Therefore, the contribution of AbsGS can be considered orthogonal and complementary to that of our method, and its densification approach can be integrated into ours to yield further performance improvements. This is confirmed by the results in Tab. 4 (see _\"Ours + [20]\"_) where we incorporate the densification strategy proposed in AbsGS into our training pipeline and further enhance the performance of our method.\n\n\n---\n\n### 2. Comparison to Octree-GS\nAs suggested, we present the comparison with Octree-GS in the table below. The results indicate that IBGS significantly outperforms Octree-GS on the Mip-NeRF 360 and TNT datasets, while showing comparable performance on the Deep Blending dataset. We thank the reviewer for the suggestion and will include the comparison with Octree-GS in the revised version.\n\n|Method/Dataset|Mip-NeRF 360|Tanks & Temples (TNT)|Deep blending|\n|--------------|:---------:|:-------------------:|:-----------:|\n|Octree-GS|28.05 / 0.819 / 0.214|24.68 / 0.866 / 0.153|**30.49** / **0.912** / 0.241|\n|**Ours**|**28.41** / **0.837** / **0.185**|**24.82** / **0.869** / **0.149**|30.02 / **0.912** / **0.235**|\n\n(We report the image metrics as PNSR/SSIM/LPIPS in the table above)\n\n---\n\n### 3. Training costs of IBGS\nWe shows the average training time (in minutes) of our method, 3DGS and SuperGaussian\nacross three datasets in the table below. All methods are trained using a single RTX 4090 GPU. On average, the training time of IBGS is faster than that of SuperGauss which learns per-Gaussians spatially-varying texture. However, our method requires longer training time compare to 3DGS due to additional computations for color residual predictions. We will include the discussion for the training time in the revised version. \n\n|Method / Dataset|MipNeRF-360|Tanks&Temples|Deep blending|\n|---------------|:---------:|:-----------:|:-----------:|\n|3DGS|19|11|19|\n|SuperGaussian|55|30|60|\n|Ours|44|21|39|\n\n(Training time is reported in minutes, average across all scenes within a dataset.)\n\n---"}, {"review_id": "E68i2B3SVt", "reviewer": "Reviewer_HuVm", "summary": "This paper introduces IBGS, a novel approach that enhances 3D Gaussian Splatting by incorporating a residual color prediction network guided by neighboring source views. The method aims to better preserve high-frequency details and model complex view-dependent effects, such as specular highlights, while maintaining a low memory footprint and a compact set of Gaussians. Additionally, the authors propose a lightweight exposure correction module to improve brightness consistency under varying auto-exposure conditions. The approach is validated through extensive experiments across multiple datasets, demonstrating performance improvements over existing methods.", "questions": "1. Could the authors elaborate on how IBGS leads to more precise Gaussian parameters with high-opacity centers, particularly in relation to the limitations of low-degree spherical harmonics?\n\n2. Regarding the exposure correction module, could the authors clarify whether its primary function is to brighten images or to improve consistency and fidelity across views?\n\n3. Why was exposure correction applied only to the TNT dataset? Does this reflect dataset-specific characteristics, or are there broader implications for generalization?\n\n4. Could the authors explain the rationale behind selecting Texture-Gaussians and SuperGaussians as baselines, rather than more directly comparable methods like Spec-Gaussians [1] or SpecTRe-GS [2]? Also, why are only two baselines included in Table 2?\n\n5. Could the authors provide training time and rendering FPS comparison to better understand the proposed method from computational point of view?", "limitations": "While the manuscript includes a discussion of limitations, this section could be expanded to provide a more comprehensive perspective. In particular, it would be valuable to discuss potential limitations such as the additional computational cost and the scalability challenges when processing a large number of input images.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. IBGS effectively balances fidelity and efficiency by introducing a residual color prediction module instead of relying on a large model to learn full color, thereby enhancing view-dependent rendering while keeping the model lightweight.\n\n2. The global affine brightness correction, derived from the nearest neighboring view, enables generalization to novel views without requiring per-image parameters.\n\n3. The method consistently outperforms prior works across several datasets, including challenging scenes from the Shiny dataset, while using fewer Gaussians and less memory.\n\n**Weaknesses**\n\n1. Sections 3.2 and 3.3 suggest that the image synthesis loss using multi-view warped colors encourages accurate geometry by concentrating high-opacity Gaussians around true surfaces. This is an interesting claim that would benefit from additional visualizations or statistical evidence to reinforce its validity.\n\n2. The description of the exposure correction module could be clarified. The current phrasing might be interpreted as simply brightening images. It would be helpful to emphasize that the goal is to normalize brightness variations across views to improve consistency and fidelity. Including examples in Figure 3 where the module reduces overexposure would further support this point.\n\n3. The exposure correction module is only applied to the TNT dataset. It would be valuable to explain why it was not used for Mip-NeRF360 or Deep Blending. Does this imply those datasets do not exhibit exposure variation? Clarifying this would help assess the generalizability of the module.\n\n4. Table 1 compares IBGS with five methods, including recent approaches focused on high-frequency detail. However, given IBGS’s emphasis on both detail and view-dependent effects, it would be more appropriate to include comparisons with methods like Spec-Gaussians [1] and SpecTRe-GS [2], which are more aligned with IBGS’s objectives. Additionally, the strong performance of vanilla 3DGS suggests that some of the chosen baselines may not be the most suitable for highlighting IBGS’s contributions.\n\n5. Table 2 reports results on the Shiny dataset for only 3DGS and SuperGaussians. Including additional baselines would provide a more comprehensive evaluation, especially since this dataset is designed to benchmark view-dependent effects.\n\n6. While the proposed method demonstrates clear advantages in terms of reduced Gaussian count and memory usage, it would further strengthen the evaluation to include comparisons of training time and rendering FPS. These metrics would help provide a more comprehensive understanding of the computational trade-offs introduced by the image-based 3D Gaussian Splatting approach.\n\n7. While the use of warped captured images contributes to the method’s effectiveness, it may introduce additional memory overhead, particularly when handling reconstructions involving a large number of input images. It would be helpful to discuss how the approach scales in such scenarios and whether any strategies are employed to manage memory usage efficiently.\n\n[1] Yang Z, Gao X, Sun Y T, et al. Spec-Gaussian: Anisotropic View-dependent Appearance for 3D Gaussian Aplatting. Advances in Neural Information Processing Systems, 2024, 37: 61192-61216.\n\n[2] Tang J, Fei F, Li Z, et al. SpecTRe-GS: Modeling Highly Specular Surfaces with Reflected Nearby Objects by Tracing Rays in 3D Gaussian Splatting. Proceedings of the Computer Vision and Pattern Recognition Conference. 2025: 16133-16142.", "was_revised": true, "final_justification": "I would like to thank the authors for providing a thorough and thoughtful rebuttal, which has effectively addressed my concerns. I encourage the authors to incorporate the additional information presented in the response into the manuscript to further strengthen its clarity and impact. \n\nBased on these improvements, I am pleased to raise my rating to accept.", "rebuttal": "We appreciate the reviewer for the constructive comments that are valuable to improve our method. We address the reviewer's concerns below. \n\n---\n\n### 1. How the multi-view color consistency loss of IBGS leads to more precise Gaussian parameters with high-opacity concentrating around the true surfaces\nTo verify IBGS leading to higher opacity for Gaussianss that are close to the true surface, we conduct the following experiment. First, for each pixel in the image, we find $K=8$ Gaussian-ray intersections that are closest to the ground truth surface points. This process is done for both IBGS and 3DGS, resulting in 8 opacity maps for each method. For each opacity map, we compute the average opacity across all pixels and compare the average values produced by IBGS and that produced 3DGS. As a result, on the Ignatius scene in the Tanks and Temples dataset, the average opacity values of IBGS are approximately 26% higher than that of 3DGS. This result demonstrates that IBGS have Gaussians with higher opacity locating near the ground-truth surface compared to 3DGS. \n\nFurthermore, we evaluate the rendered depth map of IBGS and 3DGS by using the $K$ median Gaussians (Eq. 5, main paper). Intuitively, it can be seen from the Eq. 5 that, in order to obtain accurate depth value, either of the two following conditions must be satisfied: (1) All the intersection points $x_k$ concentrate around the ground-truth surfaces and higher weights $w_i$ which are related to the Gaussian opacity or (2) there may exist intersection points $x_k$ that are far from the surfaces; however, their weights $w_i$ are relatively smaller than those of the points closer to the surfaces. Therefore, methods that yield higher depth accuracy better satisfy the two conditions, and thus having more high-opacity and surface-aligned Gaussians. The depth error in the table below shows that IBGS significantly outperforms 3DGS, achieving an average error reduction of **63%**. This result is a strong indication that IBGS produces more high-opacity Gaussians concentrated near the true surfaces compared to 3DGS. In our training pipeline, the weights $w_i$ are optimized via the multi-view color consistency loss $L_{photo}$. Therefore, this loss is the primary component driving the formation of high-opacity, surface-aligned Gaussians in IBGS.\n\n(We report the absolute relative depth error in the table below)\n\n|Method|Barn|Caterpillar|Courthouse|Ignatius|Meetingroom|Truck|Average|\n|:----:|:---:|:----------:|:--------:|:-------:|:----------:|:-----:|:------:|\n|3DGS*|0.029|0.025|0.099|0.014|0.066|0.030|0.044|\n|**Ours**|**0.004**|**0.005**|**0.051**|**0.003**|**0.031**|**0.004**|**0.016**|\n\n**Relation to the limitations of low-degree spherical harmonics**:  Due to the low-degree spherical harmonics, 3DGS struggles to handle complex view-dependent color. In contrast, IBGS relies on color information from multiple source views to infer the view-dependent color at a target viewpoint. Given the high-opacity Gaussians concentrating the true surfaces, IBGS can accurately represent the scene geometry, yielding more precise pixel correspondences between views which are essential for obtaining information regarding the view-dependent effect in the neighboring images. \n\n---\n\n### 2. Clarification for the exposure correction module\nWe agree with the reviewer that the ultimate goal of the exposure correction module is to encourage exposure consistency across views and improve fidelity. We would like to further clarify that the proposed module helps correct the exposure of a rendered image by mimicking the exposure of the nearest neighboring views. This exposure correction can lead to either an increase or a decrease in the brightness of a rendered image. We thank the reviewer for the valuable suggestion. In the revised version, we will provide more clarification for this module, as well as include a visualization of a case where the exposure correction module helps reduce overexposure. \n\n---\n\n### 3. Why exposure correction is only applied for TNT dataset\n\nFollowing [1], we only apply the exposure correction to the TNT dataset since the camera auto-exposure setting is enabled while capturing images in this dataset, as discussed in [2]. In practice, the exposure correction module can be activated by default without concern for performance degradation, as it has minimal impact when there is no exposure variation across the training views. To validate this, we applied the exposure correction module to the MipNeRF-360 dataset and present the results below. It can be observed that the method’s performance remains largely unchanged when exposure correction is incorporated. \n\n|                      |PSNR|SSIM|LPIPS|\n|----------------------|:---:|:---:|:----:|\n|W/o exposure correction|28.41|0.837|0.185|\n|With exposure correction|28.39|0.836|0.189|\n\n---\n\n### 4. Comparison to more baselines, including Spec-Gaussians\nIn Tab. 4 (supp. mat.), we present a comparison with additional baseline methods, including Spec-Gaussians. The results indicate that our method consistently achieves the best SSIM across all datasets. Compared to Spec-Gaussians, IBGS demonstrates superior image quality in all evaluated metrics. Unlike Spec-Gaussians, which focuses solely on modeling the specular component of scene appearance, IBGS is not limited to modeling specular color and can also capture various types of view-dependent effects, such as reflection and diffraction. To further demonstrate the superiority of IBGS in modeling view-dependent color, we compare IBGS and Spec-Gaussians on three scenes with challenging view-dependent color from the Shiny dataset. The results in the table below show that IBGS achieves significantly better performance, outperforming Spec-Gaussians by at least **4.52 dB** in PSNR.\n\n| Method/Dataset | Guitars (specular highlight) | Lab (reflection) | CD (diffraction) |\n|---------------|:----------------------------:|:---------------:|:---------------:|\n| Spec-Gauss    | 30.62 / **0.955** / 0.120    | 30.53 / 0.946 / 0.103 | 30.69 / 0.954 / 0.081 |\n| **Ours**      | **35.78** / 0.954 / **0.104** | **35.07** / **0.967** / **0.056** | **35.21** / **0.955** / **0.062** |\n\n(We report the image metrics as PNSR/SSIM/LPIPS in the table above)\n\n---\n\n### 5. Comparison to more baselines, including SpecTRe-GS\nUnfortunately, we were unable to compare with SpecTRe-GS, as it was published at CVPR 2025, which occurred after the NeurIPS 2025 submission deadline. Moreover, the official implementation of this method has not yet been released.\n\n---\n\n### 6. Comparison in terms of training time and rendering FPS\nWe provide a rendering speed comparison between IBGS and prior methods in Tab. 3 and Tab. 6 (supplementary material). While IBGS introduces additional computations compared to 3DGS, resulting in lower rendering FPS, it still achieves real-time performance of **over 30 FPS** in most cases, as shown in both tables. Furthermore, Tab. 6 presents a study of the trade-off between rendering speed and image quality by varying the resolution of the predicted color residual map. By reducing the resolution by half, IBGS achieves a rendering speed of at least **58 FPS** across all datasets while still delivering the better image quality compared to 3DGS and SuperGaussian.\n\nRegarding the training time, the table below shows the comparison between our method, 3DGS and SuperGaussian. Due to the additional computation, the training time of our method is longer than 3DGS's. On the other hand, compared to SuperGaussian which learn Gaussian spatially-varying texture, the training time of our method is faster in all datasets. We will include the training time comparision in the revised version. \n\n(We report the training in minutes, average across all scene in a dataset)\n\n|Method / Dataset|MipNeRF-360|Tanks&Temples|Deep blending|\n|---------------|:---------:|:-----------:|:-----------:|\n|3DGS|19|11|19|\n|SuperGaussian|55|30|60|\n|Ours|44|21|39|\n\n---\n\n### 7. Scalability of IBGS in case of large scenes\nWe agree with the reviewer that IBGS could encounter memory bottlenecks when handling large scenes or a substantial number of images. Specifically, IBGS requires pre-loading the source images into GPU VRAM to avoid the overhead associated with on-the-fly memory allocation for these images. However, this approach leads to higher VRAM usage of IBGS compared to that of 3DGS and SuperGaussian, as illustrated in the table below. A feasible solution to mitigate this limitation is to load each image into VRAM only when it is selected as a neighboring view needed for rendering. Nevertheless, this approach introduces a trade-off between speed and memory usage, as dynamic VRAM allocation can adversely affect the rendering speed of IBGS. We thank the reviewer for this valuable insight and we will include a discussion of this speed-and-memory trade-offs in the revised version.\n\n(The VRAM memory is reported in GB)\n\n|Method|MipNeRF-360|Tanks and Temples|Deep blending|\n|------|:--------:|:---------------:|:-----------:|\n|3DGS|2.03|1.25|2.13|\n|SuperGaussians|2.76|1.45|2.44|\n|Ours|6.12|2.97|5.36|\n\n---\n\n### 8. More discussion for the method limitations\nWe thank the reviewer for the suggestion and will expand the discussion of our method’s limitations, including (1) requiring additional computations, leading to slower training times/rendering speed compared to 3DGS, and (2) the trade-off between speed and memory usage when dealing with large scenes.\n\n---\n\n[1] Chen, Danpeng, et al. \"Pgsr: Planar-based gaussian splatting for efficient and high-fidelity surface reconstruction.\" IEEE Transactions on Visualization and Computer Graphics (2024).\n\n[2] Barron, Jonathan T., et al. \"Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields.\" Proceedings of the IEEE/CVF international conference on computer vision. 2021."}]

Author rebuttal:

Meta-review: After careful consideration of the reviewers’ assessments and the authors’ rebuttal, this submission demonstrates significant technical merit and novelty in integrating image-based rendering with 3D Gaussian Splatting. The reviewers consistently highlight the improvements in visual quality, geometry accuracy, and high-frequency detail recovery, with all major concerns satisfactorily addressed in the rebuttal. Minor suggestions, such as additional clarifications on high-opacity Gaussians, exposure correction, and memory/speed trade-offs, have been incorporated or clarified. AC recommends acceptance of this paper. AC also suggest that the authors include the content of their rebuttal addressing critical reviewer concerns in the final camera-ready version. This decision has been reviewed and approved by the SAC.
Decision: Accept (poster)