MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relighting
Authors: Yumeng He
Yunbo Wang
Venue: NeurIPS 2025 spotlight ()
Keywords: Neural rendering
3D relighting
meta-learning
Primary area: applications

Abstract: Out-of-distribution (OOD) 3D relighting requires novel view synthesis under unseen lighting conditions that differ significantly from the observed images. Existing relighting methods, which assume consistent light source distributions between training and testing, often degrade in OOD scenarios. We introduce **MetaGS** to tackle this challenge from two perspectives. First, we propose a meta-learning approach to train 3D Gaussian splatting, which explicitly promotes learning generalizable Gaussian geometries and appearance attributes across diverse lighting conditions, even with biased training data. Second, we embed fundamental physical priors from the *Blinn-Phong* reflection model into Gaussian splatting, which enhances the decoupling of shading components and leads to more accurate 3D scene reconstruction. Results on both synthetic and real-world datasets demonstrate the effectiveness of MetaGS in challenging OOD relighting tasks, supporting efficient point-light relighting and generalizing well to unseen environment lighting maps.

MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relighting
Yumeng He
Yunbo Wang"
MoE Key Lab of Artificial Intelligence, Al Institute, School of Computer Science Shanghai Jiao Tong University {ymhe, yunbow)osjtuu edu..edueco
Abstract
iit-of--dittrbution (OOD) 3D relighting requires novel view synthesis under un- seen lighting conditions that differ significantly from the observed images. Existing relighting methods, which assume consistent light source distributions between training and testing, often degrade in OOD scenarios. We introduce MetaGS tackle this challenge from two perspectives. First, we propose a meta-leaminn approach to train 3D Gaussian splatting, which explicitly promotes learning gen- eralizable Gaussian geometries and appearance attributes across diverse lighting conditions, even with biased training data. Second, we embed fundamental physical priors from the Blinn-Phone reflection model into Gaussian splatting, which en- hances the decoupling of shading components and leads more accurate 3D scene reconstruction. Results on both synthetic and real-world datasets demonstrate the effectiveness of MetaGS challenging OOD relighting tasks, supporting efficient point-light relighting and generalizing well to unseen environment lighting maps.
Introduction
3D scene relighting generates novel lighting effects that interact with the observed 3D environment, with recent advances in saaning-based volume rendering offering effective solutions 21 36 23 40
124 [22 3 [00. A typical approach captures substantial number of multi-view images of scene under individual lighting conditions, then trains NeRF 3D Gaussian splatting (3DGS) model to generalize to new point light positions [28] 13
41 g Notably, such idealized data requirements are often impractical in real-world scenarios. To simulate a more realistic capturing process, we build on prior work using One Light At Time (OLAT) setup [34]]]]]]]] 14 [. where training data collected with moving point light and moving camera. Recent studies have further explored low-cost OLAT capture using a smartphone flashlight as the moving light source [5].
9
While most OLAT methods assume coherent lighting distribution between training and testing, real-world scenarios often involve light sources that are randomly distributed around the scene, which may result in biased training data. This leads an out-of-distributio (OOD) issue where test-time light sources deviate from the training distribution. This OOD relighting presents a more natural and challenging task, serving as a robust test for evaluating model's performance under "truly" novel lighting conditions. Compared with the standard OLAT setup, it complicates the learning of true object illumination properties and geometries due limited training corpus with entangled, time- varying lighting and viewing directions and, more critically, spatially biased lighting. As shown in Figure existing OLAT methods struggle with the OOD relighting task, often producing unrealistic lighting effects, such as chaotic specular highlights and shadows, due to overfitting to the training samples. Also, because of the lack of generalizability, when new lighting distributions are introduced, they typically require full retraining or finetuning to adapt.
Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Train
Tost
Ground Truth
GS
NRHints
Ours
Preliminary results. NRHints [34] and GS [ struggle with out-of-distribution light positions (as specified in Section 2 the blurry novel view synthesis results and errоneous normal predictions indicate. GS3 does not require well-defined surface normals and does not directly output the normal image.
Figure 1:
In this paper, we present two novel techniques to enable better OOD relighting. First, we provide pilot study on meta-learning-basee 3DGS and introduce MetaGS to mitigate overfitting and improve the generalizability of the OLAT models. This approach leverages bilevel optimization techniques effectively address the challenges associated with limited training data and complex lighting variations. Specifically, we frame OLAT as multi-task learning problem, treating rendering under specific lighting positions as distinet tasks. The core idea is to validate the learned Gaussian illumination properties under given lighting condition using data from other lighting conditions. The resulting second-order gradients 3DGS parameters can explicitly encourage the model to generalize to unseen illumination rather than overfitting to specific training samples, as is common with standard 3DGS loss functions.
Furthermore, MetaGS integrates simple yet fundamental physical priors into Gaussian splatting by incorporating a learnable Blinn-Phong reflection model 22. This approach effectively decou- ples different shading componentsddiffuse specular, and ambient-leading to a more physically grounded understanding the interactions between objects and lighting the training data. This design draws inspiration from prior research [39] 28 13] indicating that modular and compositional representations enhance generalization capabilities. The decoupled rendering improves the model's ability to generalize to novel lighting conditions, allowing it to independently adjust each shading component according to the specific lighting scenario.
We evaluate MetaGS on both synthetic and real-world datasets. supports efficient point-light relighting under highly constrained training illuminations and significantly outperforms existing OLAT methods in OOD relighting tasks. We further demonstrate that both meta-learning and the differentiable Phong model indivicually contribute improved generalization. Notably, MetaGS generalizes well to unseen environment maps, despite being trained exclusively in OLAT scenarios.
2 Preliminaries
Definition and challenges of OLAT. OLAT refers to a specialized setting for 3D reconstruction and relighting. This involves illuminating the subject with a single point light in sequential exposures, capturing one image at each light position. Each capture represented as tuple (O,, Vz, Pt), where O, denotes the observed image, Vú represents the camera configuration, and P. contains lighting information. In our context, P: specifies the position of the point light. Unlike the "multiple lights, multiple cameras" setup, which requires multi-view images captured by multiple timh-yynchoonized cameras for each lighting condition, OLAT relies on a single camera. This significantly simplifies data collection, particularly in scenarios with dynamic lighting. The rapidly changing light positions in OLAT introduce new challenge for 3D reconstruction. The color of a 3D point can vary significantly, increasing ambiguity for estimating true object geometries and illumination properties.
Preliminary findings. We summarize existing OLAT relighting methods in Table In the prelim- inary experiments, we evaluate NRHints (34), the prior art NeRF-based relighting models. We observe that it underperforms in testing scenarios with oft-of- a feature that we believe should crucial for relighting methods. Specifically, in Figure [ the lighting in the training set is arranged on one side of the hemisphere, while the lighting in the test set is arranged on the opposite side (cameras are located on both sides). such cases, NRHints fails to generate reasonable rendering results, likely due to its implicit modeling of shadows and
(OOD) lighting positions,

---

Table 1: Comparison of the OLAT relighting methods.
35
Ours
Point light
Shadow computation
XX
V
V
OOD light
V
specular reflections. We additionally evaluate concurrent EDGS-based approach [ and observe similar degraded results under the OOD relighting setup These findings highlight that understanding intrinsic illumination properties under arbitrary lighting variations remains challenging task, the model tends to overfit to perspective-constee constraind observations and fail to leverage enough physical principles when dealing with unseen lighting distribution.
3
Method
In this section, we present the details MetaGS for addressing the OOD OLAT learning challenge:
Section [.1] we incorporate physical shading priors within Gaussian splatting via a differentiable Phong reflection model to decouple mixed illumination components.
Section 3.2 we introduce a bilevel optimization scheme to estimate light-independent scene geometries and intrinsic illumination properties, marking an early effort in volume rendering.
In Section we discuss the entire training pipeline and the implementation details.
3.1
Differentiable Phong Model
Our MetaGS method leverages simple yet generalizable physical priors from the Blinn- Phong model [9]. which captures three fundamental components of light transport: ambient, diffuse, and specular reflections. The core idea is disentangle these illumination components by learning the interactions between (i) the normal vectors of the Gaussian points, (ii) the viewing directions, and (iii) the ray direction from the point light.
Specifically, the ambient component represents constant environmental illumination, simulating indirect scattering from surrounding surfaces to establish baseline brightness level. The diffuse component, based on Lambertian law, describes light scattering in multiple directions from rough surfaces. The specular reflection is computed based on the angle between the light direction and the bisector (h) between the viewing direction (v) and the light direction (I), with a shininess exponent that represents different degrees of glossiness.
In MetaGS, we extend the original 3DGS by introducing the decoupled computation for different shading components. As shown in Figure 2 in addition to the basic Gaussian attributes (e.g., position x, rotation R, scale S, opacity a, and spherical harmonic coefficients f), each Gaussian point is further associated with normal vector coefficient ki.. The newly added attributes enhance the model's 's understanding of lighting effects, facilitating the learning process of light-independent object geometries. The overall color of Gaussian point determined by:
3-channel diffuse color ka. and a 1-channel specular
Lp = La La Lg
La +
Σ (kaTa + KgTs);
(1)
[lihht
where kja.s) and [(d.s) are the colors and intensities of the diffuse and specular components. |lights|= in OLAT. graphics, zero-order spherical harmonies (SH) coefficients typically represent the basic, uniform component of function defined over the sphere, essentially the average or constant part of a lighting environment. Therefore, we restrict Gaussians' SH coefficients to zero order, denoted by fo. corresponding to the ambient color. We compute the diffuse and specular light transports by multiplying the color and intensity each component, where the intensities la and I, are defined as:
La max(0, -1). I, , max(0, n h)?,
where I denotes the light emitted intensity, which is global learnable parameter, I denotes the point- vil denotes the bisector of the point-to-camera normalized to-light normalized vector, and h [[+I]]

---

Geometry R, Opacity
Computation step
Visibility
Zero-level SH fo
Ambient
BP
Normal
Diffuse
Phong coef ky.
max(0.
Specular
Prediction
Ground Truth
Learnable Parameters
Scene
Composition
Figure 2: The model design of MetaGS. Our model decomposes the illumination effects by interacting the learned Gaussian points with rays originating from both the viewer and the light source.
vector v and We model the coefficients ka and kg implicitly, where kú multiplied by the RGB color of the point light.
Shadow computation. We calculate shadow effects using a B BVH-based ray tracing method [8] Similar to the camera-to-ponee torrntt accumulated transmittance, the received light intensity Thight represents the total transmittance light along the light-to-point ray. It is therefore affected by the opacity of the surfaces encountered along this path. For each point, we determine light visibility by tracing ray from the Gaussian's center to light source. Since only the diffuse and specular colors are influenced by incident light intensity, update the color of each point by incorporating the light visibility factor into the diffuse and specular terms from Eq. (I):
Lp La +
Tilghtt
(kaTa + IT).
(3)
[lihhs
This explicit formulation accounts for physics-based shadowing effects, provides strong interpretabil- ity. and improves generalizability compared to the implicit, high-dimensional shadow modeling methods. However, integrating this module directly into MetaGS's pipeline presents challenges. Under OLAT conditions, learning coherent geometry is challenging, leading to difficulties in accu- rately separating shadows from the scene. This, in turn, results in erroneous color predictions. The following meta-learning scheme promotes the mutual learning of scene geometry and appearance, alleviating the difficulties in learning coherent attributes under varying illuminations.
3.2 Meta-Learned Gaussian Relighting
Existing 3D relighting methods exhibit performance degradation when handling cut-of-distribution relighting, primarily due to overfitting lighting patterns to perspective constrrieee observations, resulting in producing unreasonable lighting components (such as wrong specular and shadows).
To address this, we introduce a meta-learning framework based on bilevel optimization, which has been shown effectively bridge the distribution shift between the training and testing domains, facilitating the generalization of optimized variables unseen scenarios [4]. This training strategy can also promote coherent geometry learning in complex OLAT tasks, as shown in the ablation study, In MetaGS, the intuition of incorporating bilevel gradient update is to mitigate overfitting specific light conditions by explicitly simulating test samples with OOD light sources during each gradient update, thereby improving the model's ability generalize varied lighting scenarios.
As illustrated in Alg. [ we organize the training processes with different (O,, V,, Pr) as multiple learning tasks. We alternate model training between these tasks, validating the optimized variables under one lighting condition using data sampled from other conditions. This learning procedure encourages both the lighting attributes (f, ha, k;)) and the geometric attributes (x, R. S, a) of the Gaussian points to converge cohesively.
To simulate test-time conditions, the training data is divided into support (training) and query (validation) sets. At each iteration, 2m samples are drawn from the training data to form the support set Duup and query set Diuery (Line 5). Each support sample pairs with a query sample, denoted by subscript During each training step, the model alternates between these tasks in the inner optimization loop (Lines 6-9) and the outer loop (Line 10).

---

Algorithm Meta-Training for OOD Relighting
1: Input: Training set (Drh.r. where D,
{light position, camera parameters, RGB image)
2: Hyperparameters: Learning rates a, 8
3: Initialized parameters: Pretrained Gaussian attributes {0+}, where 0x = (x, n. R, ka,
4: while not converge do
5: Sample disjoint data Di.m. and Dism from {De)i:r
6: for task in {1 : m) do
Inner optimization loop
7:
0g f- a aV (0; (. 8: end for
9: [0} <- {0} B E
VeLnua D!uary
P Outer optimization step
10: end while
Inner optimization loop. In the inner loop, the model is trained on each support set to independently generate m sub-models, representing the hypothetical estimates 8/ under each lighting condition. All sub-models start with the same parameters of B. The light intensity is a global parameter that is also learnable and optimized jointly with the Gaussian attributes. It is omitted here for clarity, We specify the loss function Linat in subsequent Section 3.3
In the outer loop, global gradient update is performed across all sampled tasks, updating the Gaussian attributes. We first compute the loss function for each query sample, (i D, using the corresponding inner-loop model hypotheses 8'. We then aggregate the losses to update the initial B. This approach, involving task-specific adaptation followed by a global update, enables the model to learn generalizable representations across varying lighting conditions.
Outer optimization loop.
3.3 Entire Training Pipeline
We employ a three-stage training scheme. The first stage, similar to 3DGS, focuses on training the core Gaussian attributes, including position x, rotation R, scale S, opacity a, and spherical harmonic coefficients f. After this stage, the model tends to capture a rough geometry and an average color across different illuminations, serving as an effective initialization for further refinement. In the second stage, we incorporate the normal attributes into the optimization process, acknowledging that learning Phong model heavily relies on accurate normal estimations. the final stage, we integrate the diffuse and specular components, training all parameters concurrently through meta-learning.
Objective functions. We follow 3DGS 10 to compute the deviation of the predicted image from the ground truth, using an LI regularization and a D-SSIM term as the RGB loss. We also employ a sparse loss [9] to encourage the opacity values of the Gaussian spheres to approach either or 1, thereby facilitating learning with opaque objects. For normal estimation, we follow GaussianShader [9] to compute the normal vectors using Gaussian's shortest axis direction, and progressively align them with the depth-inferred nneudos.ommas...
Implementation details. We implement MetaGS using PyTorch 18]. For optimization, we use the Adam optimizer [I]] with the same parameters those specified by [10]. Our training procedure contains three stages: the first stage last for 10k iterations, the second stage last for 5k iterations, followed by the final meta-learning stage of 10k iterations, All experiments conducted single
NVIDIA RTX 3090 GPU.
4 Experiments
4.1 Experimental Settings
We perform three types of OLAT relighting experiments:
OOD relighting: We introduce a novel view synthesis setup to assess the generalizability of relighting models under OOD lighting conditions. The point lights in the training set positioned on one side of the upper hemisphere, while the lighting in the test set is placed on the opposite side.
Camera-light colocated higlllocaig relighting: This setup mirrors the one used IRON (35,, where the camera and point light are co-located for each image. Similar to the OOD setting, the colocated
5

---

Layerd Woven Ball
map
high error
Plastic Cup
error map
low crror
Rubber Cup
error
NRIlints
Wildlight
GS
Ours
GT
Figure 3: OOD relighting results on synthetic data. We present rendered novel views and error maps. While baselines often misrepresent shadows light-dependent effects (e.g., incorrect shadows in Plastic Cup), our model better infers surface appearance.
Table 2: PSNR results for 0OD relighting. See Appendix for full comparisons.
Synthetic PlasCup
Real-world
Method
RubCup
Cat
Catsmall
CupFabric
Fish
FurScene
Pikachu
NRHints 3
17.25
23.92
27.44
18.04
24.63
24.65
22.57
21.55
24.00
23.03
WildLight
21.73
20.95
24.02
18.65
22.53
24.03
21.47
20.33
19.09
20.22
GS 1
18.84
20.30
24.37
17.66
23.34
25.04
21.12
17.34
24.11
19.63
Ours
26.76
27.54
27.95
26.45
26.44
27.29
24.68
24.82
25.54
25.65
configuration also represents constrained lighting regime, the light source is fixed to the camera pose. For fair comparison, we generate colocated versions of the synthetic scenes accordingly.
Environment map relighting: We evaluate the generalization of the models exclusively trained in the OLAT setup to novel view synthesis with unseen environment lighting maps.
We evaluate MetaGS on synthetic scenes and real captured scenes from NRHints featuring diverse range of materials, complex object shapes, significant self-occlusion, and intricate shadow effects. Each image rendered (or captured) with unique camera pose and point-light position. Full details of the datasets are provided in the Appendix.

---

NRHHOS
Wildighh
Flonter
Wrong shadow
GS
Ours
Figure 4: OOD relighting results on real-world datu. As highlighted with the red boxes, baseline models struggle with some level global shading consistency, including color shifts, incorrect shadows, and floating artifacts. Our approach presents physically plausible specular highlights and geometrically consistent shadows that closely match ground truths.
w/o Meta learning
Full model
Normal
Ambient
Diftuse
Specular
NVS(w/o shadow)
NVS
GT
Figure 5: Ablation studies with qualitative results. As shown in the decoupling components, the meta-learning scheme is essential for learning object illumination properties and geometries. Without it, the model generates plain lighting effects, particularly for the diffuse and specular components.
For each synthetic scene, generate total of 600 OLAT images using Blender, with 500 for training and 100 for testing, following out-of-distribute or colocated data patterns. For the real-world scenes, split the data according their point light positions construct out-of-distribution datasets, and use 600 training images per scene-2 significantly smaller subset than the full NRHints dataset.
We compare MetaGS against both NeRF-based and Gaussian-based models, with particular focus on the OOD relighting setup. The baseline models include state-of-the-an approaches from the past two years: G [1) NRHints (34), WildLight (5), Relightable 3DGS (8), GaussianShader [9, and
IRON
The evaluation metrics include PSNR, SSIM (25]. and LPIPS (8).
4.2 Results for Out-of-distribution OLAT Relighting
We evaluate MetaGS in scenarios with OOD point light positions to explore its generalizability to unseen illuminations. The qualitative and quantitative results are respectively presented in Figure 3 Figure 4. and Table Our model remarkably outperforms the baselines in handling complex light interactions. For example, in Figure the lighting and shading effects on the cat's fur are rendered with high fidelity, while baseline models tend produce incorrect color tones. Furthermore, as indicated by the error maps in Figure all of three OLAT baselines tend to provide unrealistic results,

---

Ours
R3G
Figure 6: Relighting with environment maps: Our method, trained under OLAT settings, successfully generalizes to unseen environmental lighting; while the compared method is trained in an all-light-on setup that provides multiple lighting conditions per training view, it still fails to generalize to perform relighting with novel environment maps and exhibits visible artifacts.
especially for regions with significant specular reflections or shadows. Please video visualizations on OOD relighting, where baselines' performance drops as the light enters the OOD) region, while model provides better relighting results.
Appendix for
The generalizability of MetaGS to OOD illumination stems from two key factors. First, due to the Phong model's ability to capture the underlying principles of light interaction. For example, instead of simply viewing specular highlights as the view-dependent color a point (which not generalizable), method explicitly calculates the interactions of rays and surface normals, enabling the model to generate accurate highlights even in areas not directly illuminated during training. Second, meta-learning further enhances generalization by simulating test-time conditions during training, effectively reducing overfitting to specific illuminations. In the Appendix, we empirically show that the benefit of meta-learning does not come from larger batch size.
Ablation studies. We conduct a series ab- lation studies to validate the effectiveness of the meta-learning scheme as well as the shadow computing. From Table Table and Figure 5 ob- serve that the proposed meta- leatallenning training scheme strongly impacts performance. Without it, the model struggles to converge smoothly and make unrealistic component estimations, result- ing in plain rendering quality.
Table 3: Ablation studies of each model component for OOD relighting. We show the average results of all three synthetic scenes.
Method
PSNR
SSIM
LPIPS
Full model
27.42
0.9546
0.0505
w/o Mcta-lcaming
19.14
0,8781
0.0892
w/o Shadow
21.53
0.9105
0.0735
4.3 Generalization to Other Relighting Setups with Constrained OLAT Training
relighting. We evaluate MetaGS in the camera--lght-colocatee setup, as intro- duced by IRON [35]. Like the OOD scenario, the colo- cated setup also imposes limited lighting diversity, as the point light is tied to the camera's 's position through- out training. Quantitative results in Table that MetaGS outperforms IRON, achieving more accurate illumination inference under new point positions, and generating fewer rendering artifacts. Visualizations are presented in the Appendix.
Camera--ight-colocee
Table 4: Novel view synthesis results in PSNR under cumera-tight-colocaae ing setup. Full results in all evaluation met- rics are included in the Appendix.
relight-
Method
Ball
PlaCup
RubCup
IRON (35] Ours
26.99
34.43
36.22
38.72
36.90
38.89
Few-light and Fixed-elevation-ligh Relighting, To evaluate MetaGS under constrained illumina- tion conditions, we additionally design the following settings that vary in light distribution, coverage, and sparsity, simulating more realistic capture scenarios. In the few-light setting, only three training lights are sampled, each paired with 200 camera views in a one-light-mlli-camert configuration. Two variants are considered: (1) interpolation, where training lights uniformly distributed over

---

Table 5: Novel view synthesis results in the few-light and fixed-elevation relighting setup.
Fixed-elevation
Few lights (Interp)
Few lights (Extrap)
Method
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
NRHints 34
25.76
0.9633
0.0402
18.05
0.8823
0.0911
18.62
0.8960
0.0681
GS [
22.93
0.9393
0.0580
19.92
0.9165
/0.0693
18,06
0.8781
0.0920
Ours
27.48
0.9590
0.0371
24.90
0.9476
0.0462
23.94
0.9369
0.0533
Table 6: Results of the proposed meta-learning scheme integrated with various relighting models. PSNR values demonstrate the generalization performance, with A Mase) indicating the improvement from applying meta-learning.
Method
Ball
PlasCup
RubCup
Cal
Catsmall
CupFubric
Fish
FurSoene
Pikachu
Pixiu
A
Ours
26.76
27.54
27.95
26.45
26.44
27.29
24.68
24.82
25.54
25.65
26.31
NRHint
17.25
23.92
27.44
18.04
24.63
24.65
22.57
21.55
24.00
23.03
22.71
17.70
28.18
25.83
24.51
24.18
RNG
20.22
22.72
24.94
NaN
24.20
25.09
NaN
20,81
23.55
NaN
23,08
+meta
22.65
23.93
25.88
21.94
25.07
26.52
23.66
21.75
24.90
22.75
23.91
1,31
GS
18.84
20.30
24.37
17.66
23.34
25.04
21.12
17.34
24.11
19.63
21.18
21.54
23.18
26.15
26.96
24.38
21.22
25.02
22.98
23.81
2.63
the hemisphere and test lights are randomly sampled from the full hemisphere, and (2) extrapolation, where training lights are restricted to one side forming a triangular configuration, and test lights are sampled from the opposite side. In the fixed-elevation-lighe setting, all training lights are placed at fixed elevation of 45", uniformly distributed along a horizontal ring, while test lights are drawn from elevation bands outside this range (i.e., I 30 or 2 55%), simulating fixed-height capture trajectories. We report average results on three synthetic scenes in Table 5 Our method consistently achieves strong performance across various lighting settings. In contrast, both baselines degrade significantly under sparse or biased lighting. In particular, NRHints fails to reconstruct the Ball scene under the Three Lights setting, producing collapsed geometry. which similar to the failure case described in the paper's 's OOD setting. These results demonstrate that MetaGS not only generalizes well to unseen lighting conditions, but also remains robust under extremely limited clustered illumination. This highlights its practical applicability in real-world relighting scenarios where lighting is sparse, biased, or expensive to capture.
Environment map generalization. Our model generalizes effectively to environment map relight- ing. We approximate global illumination by importance sampling point lights from the environment map based on their pixel intensities. Notably, for our model in particular (unlike the compared methods), the sampled light directions at test time may follow a distribution different from those encountered during training, presenting significant challenge for OOD generalization. Instead of assuming infinite light source distance, we simulate distant lighting by placing point lights along envi- ronment directions at fixed distance (twice the scene radius). We compare and GaussianShader (9]. Both methods fail to converge when trained directly on distr-ibbttiitiionn (OOD) data, we construct training data that covers the full hemispherical illumination space. Even with this setup, GaussianShader still fails to converge, while Relightable 3DGS struggles with the OLAT learning task and exhibits noticeable artifacts. As shown in Figure 6 MetaGS produces diverse and visually plausible relighting results under complex environment maps, demonstrating its robustness and generalization capability.
method with R3G [8]
To further validate MetaGS's capability in handling complex lighting conditions, we conduct exper- iments with test sets containing 2-3 light sources. We also present the results for free-viewpoint relighting with in-distribution point light positions. Please refer to the Appendix for details.
4.4 Meta-Learning with Different Relighting Models
The proposed meta-learning scheme is inherently applicable to other rendering frameworks. To evaluate the generality of the proposed meta-learning scheme, we integrate it into several alternative relighting models across diverse families, including both NeRF-based and 3DGS-base methods,

---

and spanning from BRDF- based to fully implicit formulations, including NRHints RNG 77.. The PSNR results are summarized in T
34
GS
and
While meta-learning consistently enhances OOD generalization across various rendering models, its benefits most pronounced when paired with explicit geometric modeling and shading methods. Notably, the meta-lcarning approach yields the largest improvement on our Gaussian Phong rendering model. This is because the Phong model's simple yet physically grounded design introduces strong inductive biases that enable meta-learning to capture generalized scene structural information even under limited lighting conditions.
Related Work
Recent differentiable volume rendering techniques, including NeRF-based [77. SDF-based K22 K3
and 3DGS- [TO] methods, have significantly improved the quality and efficiency of novel view synthesis for 3D scenes. NeRF-based methods utilize deep neural networks to model volumetric scene functions, encoding color and density to synthesize high-quality images from sparse viewpoints [[7
15 37.. However, these approaches typically require substantial training time. 3DGS significantly reduces both training and inference time. It converts point cloud data into a continuous volumetric representation by applying Gaussian kernels to point cloud data, facilitating rendering and further processing. This technique is now widely adopted for efficient 3D and 4D reconstructions across
varied data types [28 66 26 31).
The task of 3D relighting involves altering the illumination in a 3D scene while maintaining its geometry. It requires decomposing materials and lighting of a scene from multiple images, which is challenging due its high-dimennioonl nature. Recent advances in volume rendering have introduced various solutions for this task, including those based on neural fields [39] (20 21 (23) (22) 30]
and the methods based on Gaussian point clouds 28 9 g 36). A limitation of these methods that
they require a substantial volume of multi-view images captured under individual lighting conditions, making them impractical real-world scenarios. Several studies have proposed NeRF-based 3D
relighting techniques within the OLAT framework [29] 39 34,, which greatly reduce the demands on training data. Nonetheless, these techniques tend to be computationally intensive because of NeRF's inherent complexity in the volume rendering process. In parallel, concurrent JDGS-based approaches 66 I have also explored the OLAT relighting challenge. Despite their efforts, these approaches still face difficulties when dealing with lighting conditions not encountered during training, similar to the limitations observed in NeRF- methods.
In contrast, our method addresses this issue by incorporating specific physical priors, represented by the Phong reflection model, into the Gaussian splatting framework. We have also designed a meta-learning approach to enhance the method's generalizability to OOD relighting scenarios.
6 Conclusions and Limitations
In this paper, we explored a novel and challenging problem: out-of-distribution (OOD) 3D relighting. To address this, we proposed MetaGS, which builds on Gaussian splatting and presented novel bilevel optimization- based meta-learning framework that explicitly promotes generalizable Gaussian geometry and appearance learning. This meta-learning formulation allows the model to adapt to varying light sources and viewpoints, even when the training data is biased or sparsely sampled in the lighting space. Furthermore, MetaGS incorporates a differentiable Blinn-Phong reflection model within Gaussian splatting, effectively disentangling lighting effects into ambient, diffuse, and specular shading components, thereby improving the physical realism and reconstruction fidelity under diverse lighting. Extensive experiments on both synthetic and real-world datasets demonstrate that MetaGS significantly outperforms existing OLAT relighting approaches under OOD relighting conditions. It also achieves strong generalization to complex environment maps.
Despite these advances, several limitations remain. An open challenge is ensuring the robustness of our approach under more complex lighting conditions. Moreover, our current framework only accounts for direct illumination; incorporating indirect lighting could further enhance the quality of relighting. Additionally, the current model assumes simple Phong reflection, which may limit fidelity when modeling materials with strong subsurface scattering anisotropic reflectance.
10

---

Acknowledgements
This work was supported by the National Natural Science Foundation of China (Grant 62250062), the Shanghai Municipal Science and Technology Major Project (Grant 2021SHZDZZX0022) and the Fundamental Research Funds for the Central Universities.
References
[1] Zoubin Bi, Yixin Zeng. Chong Zeng, Fan Pei, Xiang Feng, Kun Zhou, and Hongzhi Wu. GS Efficient relighting with triple gaussian splatting. In ACM SIGGRAPH Asia, 2024.
[2] James F Blinn. Models of light reflection for computer synthesized pictures. In ACM SIGGRAPH, pages 192- 192-19811 1977.
[3] Yeonjin Chang, Yearim Kim, Seunghyeon Seo, Jung Yi, and Nojun Kwak. Fast sun sun-alinnee aligned outdoor scene relighting based on TensoRF. In WACV. pages 3626-3638 January 2024.
[4] Jiaxin Chen, Xiao-Ming Wu, Yanke Li, Qimai Li, Li-Ming Zhan, and Fu-lai Chung, A closer look at the training strategy for modern meta-learning. NewrlPS, 33:396-406, 2020.
[5] Ziang Cheng, Jumxuan Li, and Hongdong Li. WildLight: In-the-wild inverse rendering with flashlight. In CVPR, pages 4305-4314 14, 2023.
Jiahui Fan, Fujun Luan, Jian Yang, Miloš Hašan, and Beibei Wang. RNG: Relightable neural Gaussians. arXiv preprint arXiv:2409. 19702, 2024.
[7] Jiahui Fan, Fujun Luan, Jian Yang. Milos Hasan, and Beibei Wang. Rng: Relightable neural gaussians. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages
2025.
[8] Jian Gao, Chun Gu. Youtian Lin, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3D Gaus- sian: Real-time point cloud relighting with BRDF decomposition and ray tracing. arXiv preprint arXiv:2311. 16043, 2023.
[9] Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang, and Yuexin Ma. SaadeianShader: 3D Gaussian splatting with shading functions for reflective surfaces. In CVPR, pages 5322 5332, 2024.
[10] Bernhard Kerbl, Georgios Kopanas. Thomas Leimkühler, and George Drettakis. 3D Gaussian splatting for real-time radiance field rendering. TOG, 42(4):1-14, 2023.
[11] Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014.
[12] Zhiyi Kuang, Yanchao Yang, Siyan Dong, Jiayue Ma, Hongho Fu, and Youyi Zheng. Olat gaussians for generic relightable appearance acquisition. In SIGGRAPH Asia 2024 Conference Papers, pages 1-11, 2024.
[13] Ruofan Liang. Huiting Chen, Chunlin Li, Fan Chen, Selvakumar Panneer, and Nandita Vijaykumar. ENVIDR: Implicit differentiable renderer with neural environment lighting. ICCV, pages 79-89, 2023.
[14] Zhihao Liang, Qi Zhang, Ying Feng, Ying Shan, and Kui Jia. GS-IR: 3D Gaussian splatting for inverse rendering. In CVPR, pages 2 21644-21653, 2024.
[15] Lingjic Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Thcobalt. Neural sparse voxel fields. NewrIPS, 33:15651-15663, 2020.
[16] Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3D Gaussians: Tracking by persistent dynamic view synthesis. In 3DV, pages 800-809. IEFF, 2024.
[17] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-11) 106, 2021.
[18] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. NewrIPS, 32, 2019.

---

[19] Bui Tuong Phong. Illumination for computer generated pictures. In Seminal gruphics: pioneering efforts that shaped the field. pages 95-101. 1998.
[20] Kripasindhu Sarkar, Marcel C Bühler, Gengyan Li, Daoye Wang, Delio Vicini, Jérémy Riviere, Yinda Zhang. Sergio Orts-Escolano, Paulo Gotardo, Thabo Beeler, et al. LitNeRF: Intrinsic radiance decom- position for high-quality view synthesis and relighting of faces. In ACM SIGGRAPH Asia, pages 1-11, 2023.
[21] Pratul P. Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, and Jonathan T. Barron. NeRV: Neural reflectance and visibility fields for relighting and view synthesis. In CVPR, pages 7495-7504, June 2021.
[22] Jia-Mu Sun, Tong Wu, Yong-Liang Yang. Kun Lai, and Lin Gao. SOL-NeRF: Sunlight modeling for outdoor scene decomposition and relighting. In ACM SIGGRAPH Asia, pages 1-11, 2023.
[23] Marco Toschi, Riccardo De Matteo, Riccardo Spezialetti. Daniele De Gregorio. Luigi Di Stefano. and Samuele Salti. Relight My Nerf: A dataset for novel view synthesis and relighting of real world objects. In CVPR, pages 20762 20772, June 2023.
[24] Dongqing Wang. Tong Zhang, and Sabine Süsstrunk. NEMTO: Neural environment matting for novel view and relighting synthesis of transparent objects. In ICCV, pages 317-327, October 2023.
[25] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600-6) 612) 2004.
[26] Guanjun Wu, Taoran Yi, Jiemin Fang. Lingxi Xie, Xiaopeng Zhang. Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4D Gaussian splatting for real-time dynamic scene rendering. CVPR, pages 20310 20310-20320, 2024.
[27] Tong Wu, Jia-Mu Sun, Yu-Kun Lai, and Lin Gao. DE-NeRF: Decoupled neural radiance fields for view- consistent appearance editing and aee-frryyyeccy environmental relighting. In ACM SIGGRAPH, pages 1- 2023.
[28] Tong Wu, Jia-Mu Sun, Yu-Kun Lai, Yuewen Ma, Leif Kobbelt, and Lin Gao. DeferredGS: Decoupled and editable Gaussian splatting with deferred shading. arXiv preprint arXiv:2404.09412 2024.
[29] Vingyan Xu, Gaspard Zoss, Prashanth Chandran, Markus Gross, Derek Bradley, and Paulo Gotardo. ReNeRF: Relightable neural radiance fields with nearfield lighting. In ICCV, pages 22581-22591, 2023.
[30] Siqi Yang, Xuanning Cui, Yongjie Zhu, Jiajun Tang, Si Li, Zhaofci Yu, and Boxin Shi. Complementary intrinsies from neural radiance fields and CNNs for outdoor scene relighting. In CVPR, pages (6600-16609) June 2023.
[31] Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3D Gaussians for fidelityy monocular dynamic scene reconstruction. In CVPR, pages
2024.
[32] Lior Yariv. Jiatao Gu. Yoni Kasten, and Yaron Lipman. Volume rendering of neural implicit surfaces. NewrIPS, 34:4805-4815, 2021.
[33] Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sautle,, and Andreas Geiger. MonoSDF: Exploring monocular geometric cues for neural implicit surface reconstruction. NeurlPS, 35:25018-25032, 2022.
[34] Chong Zeng, Guojun Chen, Yue Dong, Pieter Peers, Hongzhi Wu, and Xin Tong. Relighting neural radiance fields with shadow and highlight hints. ACM SIGGRAPH, pages 1-11, 2023.
[35] Kai Zhang, Fujun Luan, Zhengqi Li, and Noah Snavely. IRON: Inverse rendering by optimizing neural sdfs and materials from photometric images. In CVPR, pages 5565-5574, 2022.
[36] Kai Zhang, Fujun Luan, Qianqian Wang, Kavita Bala, and Noah Snavely. PhySG: Inverse rendering with spherical Gaussians for physics-based material editing and relighting. In CVPR, pages 5453-5462. June 2021.
[37] Kai Zhang, Geroot Riegler, Noah Snavely, and Vladien Koltun. NeRF++: Analyzing and improving neural radiance fields. arXiv preprint arXiv:2010007492 2020.
[38] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metrie. In CVPR, pages 386-595. 2018.
12

---

[39] Xiuming Zhang, Pratul P Srinivasan, Boyang Deng. Paul Debevec, William T Freeman, and Jonathan T Barron. NeRFactor: Neural factorization of shape and reflectance under unknown illumination. TOG. 40(6);1-18, 2021.
[40] Ruichen Zheng, Peng Li. Haoqian Wang. and Tao Yu. Learning visibility field for detailed 3D human reconstruction and relighting. CVPR. pages 216-226, June 2023.
[41] Zuo-Liang Zhu, Beibei Wang, and Jian Yang. Gs-ror: 3d gaussian splatting for reflective object relighting sdf priors. arXiv preprint arXiv:2406. 18544, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly summarize the main contributions, which are consistent with the methods and results presented the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: See Section 6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
See Section 4.1 and 3.3 for the setup of the setting and experimental details.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We give a detailed explanation of our model in Section and plan to release our code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification: We include the code in the supplementary an anonymous pattern, and we will soon release our latest version of the code publicly before the decision of the paper acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (kttps://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
7 /pips.cccppblic//uu.dde http://gguudee//eeeuububmnii
ssinisolicyyyyyc for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See Section
3.3 Section and supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: See for the discussion of statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative eтror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Section 3.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: httprrpps../ppbbbcc/cccccccieeee
Answer: [Yes]
Justification: The research conducted in the paper adheres to the NeurIPS Code of Ethics in all aspects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The ability to synthesize realistic lighting effects and relight scenes from novel viewpoints could enhance the quality of visual media, including movies, video games, and virtual reality experiences. This could lead to more immersive and engaging entertainment, thereby benefiting society by providing richer visual experiences. However, the ability to manipulate lighting and scenes convincingly could be exploited to create deceptive or misleading content, including deepfakes falsified evidence in legal political contexts. This misuse could contribute to the spread of misinformation and undermine trust in visual media.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
17

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: This paper does not involve the responsible release data or models that have high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
Our dataset includes assets modified from licensed assets acquired from Adobe Stock. According to Adobe's 's Terms License, we have obtained the full Adobe Stock license for dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing experiments and research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLM is only used for writing, editing, or formatting purposes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
20


Reviews:
[{"review_id": "gTqGcmZaV9", "reviewer": "Reviewer_Wg3c", "summary": "This paper introduces MetaGS, a novel method for 3D scene relighting under out-of-distribution (OOD) lighting conditions using 3D Gaussian Splatting. Unlike previous OLAT (One-Light-At-a-Time) methods that struggle with generalization to unseen lighting, MetaGS addresses this by combining a differentiable Blinn-Phong reflection model to disentangle shading components and a bilevel meta-learning framework to learn lighting-independent scene representations. The approach enhances the model's ability to generalize across varying lighting conditions, even with spatially biased training data, and shows strong performance on both synthetic and real-world datasets without requiring retraining for new lighting distributions.", "questions": "1. The paper doesn’t clarify how many distinct light positions are used during training. To better assess OOD robustness, could the authors evaluate performance when training with very few or clustered lights? This would test the method under more realistic and challenging conditions. Strong results in such cases would significantly strengthen the paper’s impact.\n2. Since OLAT images are captured under individual point light sources, it is feasible to synthesize ground truth renderings for environment map relighting by linearly combining OLAT images according to the lighting distribution in the environment map. However, the paper provides only qualitative results for this experiment, without reporting any quantitative metrics or comparisons with ground truth images. Including such ground truth-based evaluations (e.g., PSNR, SSIM, LPIPS) would significantly strengthen the claim of generalization to complex lighting. Could the authors provide comparisons with ground truth images to further validate the results?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. MetaGS effectively tackles the challenge of out-of-distribution (OOD) lighting in 3D relighting. By leveraging a bilevel meta-learning strategy, the model simulates unseen lighting conditions during training, enabling it to learn lighting-invariant geometry and reflectance attributes. This allows robust relighting under novel lighting setups without retraining.\n2. The method incorporates a differentiable Blinn-Phong reflection model into Gaussian splatting, introducing physically grounded priors. This enables the model to explicitly disentangle ambient, diffuse, and specular components, enhancing the realism and interpretability of relighting results.\n\nWeaknesses:\n1. This work focuses on out-of-distribution (OOD) lighting scenarios from biased OLAT data. However, real OLAT captures use a single camera with one moving point light, limiting lighting diversity and making extreme OOD cases rare in practice. Thus, the OOD problem here is largely artificial and not representative of typical real-world conditions.", "was_revised": true, "final_justification": "The rebuttal, along with the additional experiments, has alleviated many of my concerns to a certain extent.", "rebuttal": "We thank the reviewer for the valuable comments and address them below.\n\n>  *Q1. The OOD problem here is largely artificial and not representative of typical real-world conditions.*\n\nWe appreciate the reviewer's comment regarding the nature of the OOD lighting setup in our work. While we agree that some OLAT setups (e.g., using a handheld flashlight) may not always exhibit extreme OOD lighting conditions, we respectfully clarify that our OOD lighting definition is not artificial, but rather a practical abstraction of real-world limitations. We address this concern from three perspectives:\n\n#### 1. Real-world OLAT captures are often spatially constrained \nEven with a mobile light source, physical constraints (e.g., tripod positioning, occlusions, user-defined capture paths) typically result in non-uniform and biased light distributions. For example, OLAT captures using smartphone flashlights (like in [5, 34]) or mounted light rigs are frequently constrained to a subset of hemisphere directions. Unlike controlled lab environments, real-world conditions often involve relighting from previously unobserved directions (e.g., when user illumination changes or devices rotate). In such cases, generalization to unseen or poorly sampled lighting directions becomes necessary, which is exactly the scenario our OOD setup simulates.\n\nTherefore, our OOD lighting split is designed to test the generalization capability of relighting models, which is a crucial requirement for practical deployment. As shown in Fig. 1, 3, and 4, existing methods suffer noticeable degradation even under moderate lighting distribution shifts, while MetaGS generalizes well.\n\n#### 2. OOD OLAT is only a training/test protocol; Generalization is the goal\nWhile we train on OLAT, our method evaluates under broader settings, including environment map relighting and camera-light-colocated setups. These results (Tab. 4, Fig. 6) demonstrate that our method's benefits are not limited to synthetic OOD cases but also transfer to practical test-time distributions beyond the training configuration.\n\n#### 3. Broader applications\nTo better simulate realistic lighting biases, we conduct additional experiments restricting training lights to a fixed elevation, mimicking a planar circular light path, while the testing lights are placed at different heights. We also evaluate it under OOD clustered-light and few-light training setups, as detailed in our **Responses to Q2**. These experiments further support the method's applicability beyond synthetic extremes.\n\nFinally, we would like to discuss another practical scenario--**OLAT relighting for dynamic scenes**. In such settings, where a scene is captured by a single camera and illuminated by a single light source (common in casual or consumer-grade setups), the scene’s motion is typically non-reproducible, preventing the dense sampling of illumination conditions. As a result, illumination coverage is inherently sparse over time, introducing a severe OOD relighting problem. While our current work does not explicitly target dynamic scenes, this scenario highlights the real-world significance of our OOD formulation. We see this as a promising direction for future work.\n\nWe thank the reviewer again for raising this important concern and hope this clarifies our motivation and rationale behind the OOD setup.\n\n\n> *Q2. (1) The paper doesn’t clarify how many distinct light positions are used during training. (2) To better assess OOD robustness, could the authors evaluate performance when training with very few or clustered lights?*\n\n#### 1. Number of light positions\nIn our OLAT setup, each image features a distinct light position and a distinct camera viewpoint. Specifically, we use 500 images (i.e., 500 unique point-light positions) for each synthetic scene and 600 images for each real-world scene. These capture details are provided in Appendix B (Lines 404–406).\n\n\n#### 2. Training with few or clustered lights\nTo evaluate MetaGS under constrained illumination conditions, we additionally design three settings that vary in light distribution, coverage, and sparsity, simulating more realistic capture scenarios:\n- **Clustered Lights:** We construct lighting configurations with 1, 2, or 3 clusters on the illumination hemisphere. Each configuration is evaluated under two distinct settings:\n    * *Interpolation lighting:* The cluster centers of the training lights are uniformly distributed across the hemisphere; test lights are placed between clusters.\n    * *Extrapolation lighting:* Training lights are sampled from one side of the hemisphere, with cluster centers uniformly distributed (e.g., for two clusters, their centers share the same elevation of $45^\\circ$; for three clusters, they form a triangular configuration). Test lights are randomly sampled from the opposite side.\n- **Few Lights:** We sample 3 training lights in this setting, each paired with 200 camera views (i.e., one-light–multi-camera). Training light positions match the cluster centers used in the *Clustered Lights* setting.\n    * *Interpolation:* Training lights are uniformly distributed; test lights sampled from the full hemisphere.\n    * *Extrapolation:* Training lights are confined to one side; test lights from the opposite side.\n- **Fixed-elevation Lights:** All training lights are placed at a fixed elevation of $45^\\circ$, uniformly sampled along a horizontal ring. Test lights are drawn elevation bands outside the training range (i.e., $\\leq 35^\\circ$ or $\\geq 55^\\circ$). This simulates fixed-height capture trajectories.\n\nAll settings (except Few Lights) use 500 training images and 100 test images rendered under OLAT conditions, with unique light-camera pairs. We report average results on three synthetic scenes (Blender-rendered) in terms of PSNR / SSIM / LPIPS.\n\nClustered Lights:\n\n|  | Single cluster (Interp) | Single cluster (Extrap) | Two clusters (Interp) | Two clusters (Extrap) | Three clusters (Interp) | Three clusters (Extrap) |\n|---------|:--------------:|:--------------------:|:------------:|:------------------:|:--------------:|:---------------------:|\n| NRHints | 21.14/0.9256/0.0580 | 18.81/0.9049/0.0705 | 22.55/0.9391/0.0512 | 20.45/0.9232/0.0606 | 26.45/0.9773/0.0325 | 22.17/0.9370/0.0540 |\n| GS$^3$  | 19.95/0.9184/0.0663 | 18.32/0.8895/0.0647 | 20.51/0.9286/0.0627 | 19.19/0.9117/0.0680 | 25.17/0.9640/0.0494 | 20.20/0.9223/0.0630 |\n| Ours    | **24.09/0.9350/0.0493** | **23.29/0.9282/0.0539** | **26.13/0.9492/0.0387** | **25.18/0.9417/0.0489** | **28.13/0.9678/0.0316** | **26.85/0.9550/0.0427** |\n\nFew Lights \\& Fixed-elevation Lights:\n\n|         | Fixed-elevation | Few lights (Interp) | Few lights (Extrap) |\n|---------|:---------------:|:----------:|:----------------:|\n| NRHints | 25.76/0.9633/0.0402 | 18.05/0.8823/0.0911 | 18.62/0.8960/0.0681 |\n| GS$^3$  | 22.93/0.9393/0.0580 | 19.92/0.9165/0.0693 | 18.06/0.8781/0.0920 |\n| Ours    | **27.48/0.9590/0.0371** | **24.90/0.9476/0.0462** | **23.94/0.9369/0.0533** |\n\n**Observations and insights:** As shown in the above results, our method consistently achieves strong performance across various lighting settings. In contrast, both baselines degrade significantly under sparse or biased lighting (e.g., Single Cluster, Two Clusters, and Few Lights). In particular, NRHints fails to reconstruct the Ball scene under the Three Lights setting, producing a collapsed geometry, which is similar to the failure case described in the paper's OOD setting. These results demonstrate that MetaGS not only generalizes well to unseen lighting conditions, but also remains robust under extremely limited or clustered illumination. This highlights its practical applicability in real-world relighting scenarios where lighting is sparse, biased, or expensive to capture.\n\n> *Q3. Quantitative result of envmap relighting.*\n\nWe note that this setting poses a particularly challenging generalization task, as our model is trained exclusively on OLAT data without any access to global illumination or multi-light supervision. While environment map relighting is not the primary target of our design, the results still reflect the model’s capacity to generalize beyond the OLAT setting. To this end, we supplement the following experiments:\n\n**Data construction:** Constructing ground truth via weighted combinations of real OLAT images often leads to errors, due to limited coverage of light directions and colors from importance-sampled environment maps. To ensure reliable evaluation, we generate envmap-illuminated scenes synthetically using Blender, and conduct this comparison only on synthetic scenes.\n\n**Baselines for comparison:** Most OLAT relighting methods (e.g., GS$^3$, NRHints, WildLight) do not support environment relighting. Methods that do support it (e.g., BiGS, PRTGaussian) typically require additional supervision such as all-light-on captures or multi-camera, multi-light setups, which fall outside our assumed constraints. We therefore use Relightable 3DGS as the primary baseline, since it supports environment map rendering under comparable supervision.\n\n|                  | Ball | PlasCup | RubCup |\n|------------------|:----:|:-------:|:------:|\n| Relightable 3DGS | 20.88/0.8432/0.1488 | 22.07/0.8644/0.1294 | 22.15/0.8390/0.1366 |\n| Ours             | **22.70/0.9018/0.0975** | **23.10/0.9335/0.0651** | **23.55/0.9306/0.0736** |\n\nAs shown, our method significantly outperforms the baseline in SSIM and LPIPS, with smaller gains in PSNR. We attribute this to imperfect brightness alignment, since environment maps are not seen during training and their HDR intensity values are manually calibrated at test time, potentially introducing global color shifts. Nonetheless, the strong perceptual scores and structural consistency highlight the robustness and generalization capability of our method to novel lighting conditions."}, {"review_id": "ZNi8OOWbDi", "reviewer": "Reviewer_m2RQ", "summary": "This paper tackles the challenging problem of out-of-distribution (OOD) 3D relighting, where the goal is to synthesize novel views under lighting conditions not seen during training. The authors propose MetaGS, a method that combines meta-learning with 3D Gaussian Splatting to improve generalization across diverse lighting environments. The approach encourages the learning of geometry and appearance features that transfer well to unseen lighting, even when the training data is biased. In addition, MetaGS incorporates a differentiable Blinn-Phong reflection model into the Gaussian framework to better disentangle shading components and improve physical realism. Experiments on both synthetic and real-world datasets show that MetaGS consistently outperforms existing methods in OOD relighting scenarios, demonstrating strong generalization to new light sources, including complex environment maps.", "questions": "- As mentioned in the weaknesses, it would be helpful to know whether the proposed training framework can be applied to other existing relighting models. If so, showing the results of such experiments could further support the claim that the meta-learning approach is model-agnostic and broadly applicable.\n\n- According to line178, the implementation of MetaGS is done in PyTorch. Is there a specific reason why CUDA was not used for performance optimization? A brief explanation would clarify whether there are any trade-offs or constraints involved in the current implementation.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "> Strengths\n- The paper is well-organized with a clear and logical flow, making it easy to follow.\n- Tackling a more challenging relighting scenario through a multi-task learning approach seems effective.\n- While the main focus is on the OLAT relighting problem, the method also shows strong performance under environment map-based relighting, highlighting its generalization capabilities.\n- The proposed method consistently outperforms existing baselines, both quantitatively and qualitatively.\n\n> Weaknesses\n- One of the core contributions, the meta-learning scheme, feels relatively general and could potentially be applied to other relighting models as well. While the implementation details may differ, showing that this training paradigm can benefit other architectures would have made the case for MetaGS being model-agnostic even stronger.", "was_revised": true, "final_justification": "The authors have effectively addressed my concern regarding the generalizability of the proposed meta-learning-based method to other models, providing appropriate experimental results. Therefore, I will maintain my initial rating of accept.", "rebuttal": "We appreciate the reviewer’s insightful questions and provide our responses below.\n\n\n> *Q1. Apply meta-learning to other relighting models.*\n\nYes, the proposed meta-learning scheme is inherently applicable to other rendering frameworks. To evaluate its generality, we integrate it into several alternative relighting models across diverse families, including both NeRF-based and 3DGS-based methods, and spanning from BRDF-based to fully implicit formulations:\n* **Meta-learning + NRHints** (implicit geometry + implicit lighting): This method does not rely on any parametric BRDF model, instead modeling full light transport via an MLP.\n* **Meta-learning + Neural Relightable Gaussians**$^{[1]}$  (explicit geometry + implicit lighting): Each Gaussian is assigned a learned feature vector decoded into color via an MLP (completely free of explicit shading assumptions). *We apply meta-learning in both of its two stages*.\n* **Meta-learning + GS$^3$** (explicit geometry + explicit lighting with MLPs): This method uses a simplified BRDF formulation, along with an MLP-based shadow refinement module, and a residual MLP to compensate for unmodeled lighting effects.\n\n\n\nThe PSNR results are summarized below (*NaN* indicates model failure during RNG’s second stage):\n\n| Method        |        Ball         |       PlasCup       |       RubCup        |         Cat         |      Catsmall       |      CupFabric      |        Fish         |      FurScene       |       Pikachu       |        Pixiu        |   Avg.    |\n|:------------- |:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:---------:|\n| **Ours**      |      **26.76**      |      **27.54**      | $\\underline{27.95}$ |      **26.45**      |      **26.44**      |      **27.29**      |      **24.68**      |      **24.82**      |      **25.54**      |      **25.65**      | **26.31** |\n| NRHint    |        17.25        |        23.92        |        27.44        |        18.04        |        24.63        |        24.65        |        22.57        |        21.55        |        24.00        |        23.03        |   22.71   |\n| NRHint + meta |        17.70        | $\\underline{24.57}$ |      **28.18**      |        19.75        | $\\underline{25.44}$ |        25.83        |        24.04        | $\\underline{21.90}$ |        24.51        | $\\underline{24.18}$ |   23.61   |\n| RNG       |        20.22        |        22.72        |        24.94        |        *NaN*        |        24.20        |        25.09        |        *NaN*        |        20.81        |        23.55        |        *NaN*        |   23.08   |\n| RNG + meta    | $\\underline{22.65}$ |        23.93        |        25.88        | $\\underline{21.94}$ |        25.07        |        26.52        |        23.66        |        21.75        |        24.90        |        22.75        |   23.91   |\n| GS$^3$    |        18.84        |        20.30        |        24.37        |        17.66        |        23.34        |        25.04        |        21.12        |        17.34        |        24.11        |        19.63        |   21.18   |\n| GS$^3$ + meta |        21.54        |        23.18        |        26.15        |        21.55        |        25.16        | $\\underline{26.96}$ | $\\underline{24.38}$ |        21.22        | $\\underline{25.02}$ |        22.98        |   23.81   |\n\n\nThese results show that while meta-learning consistently enhances OOD generalization across various rendering models, its benefits are most pronounced when paired with explicit geometric modeling and shading methods. Notably, the meta-learning approach yields the largest improvement on our Gaussian-Phong rendering model. This is because the Phong model’s simple yet physically grounded design introduces strong inductive biases that enable meta-learning to capture generalized scene structural information even under limited lighting conditions.\n\n[1] Fan, Jiahui, et al. \"Rng: Relightable neural gaussians.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n\n> *Q2. Is there a specific reason why CUDA was not used for performance optimization?*\n\nThank you for your question. Our implementation is fully GPU-accelerated and leverages PyTorch's built-in CUDA operations throughout. We did consider writing custom CUDA kernels, but found that the built-in operations were already highly optimized and sufficient to meet our performance needs. Moreover, our rasterization pipeline builds directly upon the same CUDA-accelerated primitives used in 3DGS, which are sufficient to achieve efficient training. As such, introducing additional low-level kernels would have increased implementation complexity without yielding significant performance gains in our setting."}, {"review_id": "sRHVDCcqnp", "reviewer": "Reviewer_N4oh", "summary": "The paper introduces MetaGS, a novel method for out-of-distribution (OOD) 3D scene relighting, which addresses the challenge of generating realistic novel views under unseen lighting conditions. Existing relighting methods often fail in OOD scenarios due to overfitting to training lighting distributions. MetaGS proposes two key innovations: a meta-learning framework and a differentiable Blinn-Phong model to tackle this issue. Evaluations on synthetic and real-world datasets show MetaGS outperforms baselines in OOD relighting, supports efficient point-light relighting, and generalizes to unseen environment maps.", "questions": "The experimental results obtained from 3DGS exhibit unexpectedly and unreasonably poor performance. Notably, given that the output of 3DGS is in the format of OLAT, it is surprising that even a straightforward convolution operation applied to any novel environment map would yield a significantly superior outcome compared to the results presented in the paper.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. OOD Generalization. MetaGS explicitly addresses the OOD relighting challenge, where test lighting conditions differ from training. Its meta-learning scheme simulates unseen lighting during training, reducing overfitting and improving generalization to novel illuminations.\n2. Physical Priors Integration. By incorporating the Blinn-Phong model, MetaGS decouples shading components (ambient, diffuse, specular), enabling more accurate and physically grounded relighting. This structural decomposition enhances the model’s ability to adapt to new lighting scenarios.\n\n**Weaknesses**\nThe incorporation of parametric material modeling, like the Blinn-Phong model used here, is not a judicious approach in this context. In contrast, 3DGS is a potent and universally applicable 3D representation capable of accommodating any material theoretically. Notably, the adoption of parametric models will inevitably impose limitations on the representational capabilities of 3DGS.", "was_revised": true, "final_justification": "The authors' response has adequately addressed my concerns about the potential limitations imposed by the employed Blinn-Phong model and the unexpectedly poor performance of GS3.", "rebuttal": "We appreciate the reviewer’s time and effort in reviewing our manuscript and have responded to the points raised as follows.\n\n> *Q1. The adoption of parametric models, like Blinn-Phong model, will inevitably impose limitations on the representational capabilities of 3DGS.*\n\nWe thank the reviewer for raising this question. This touches on a fundamental trade-off between expressiveness and generalization ability. While we agree with the reviewer that introducing parametric reflectance models like Blinn-Phong may limit the representational capabilities of 3DGS, our design choice is motivated by the need to improve generalization, particularly in OOD scenarios.\n\nWe fully acknowledge that 3DGS is, in theory, capable of modeling arbitrary appearance and reflectance. However, in practice, especially under limited and biased OLAT data, standard 3DGS comes with significant risk of **overfitting to training light distributions** and **entangled specular/diffuse representations**, as we observe in our baseline comparisons (e.g., GS$^3$, WildLight).\n\nThe goal of incorporating a simple parametric model like Blinn-Phong is not to improve the rendering quality of 3DGS under rich and fixed lighting conditions, but to introduce a **lightweight, physically grounded inductive bias** that encourages the disentanglement of ambient, diffuse, and specular reflections, thereby enabling more robust and interpretable modeling of light transport.\n\nIn OOD relighting tasks, generalization is often more crucial than raw representational power. Despite its simplicity, the Phong model offers physically meaningful priors that help constrain the learning space under sparse and varied lighting conditions. Importantly, this inductive bias synergizes with our meta-learning algorithm, promoting the separation of global scene factors (e.g., geometry and material properties) from lighting-specific variables (e.g., direction and intensity). This significantly enhances generalization performance, as demonstrated in our experiments.\n\nIn summary, while we agree that parametric models like Blinn-Phong impose certain representational constraints, we believe that in the context of OOD relighting, these constraints serve as useful inductive priors. Our work takes a first step toward integrating physically inspired models into 3DGS to improve generalization, and we see great potential in future work exploring more powerful models that retain these desirable physical properties.\n\n\n\n> *Q2. The experimental results obtained from 3DGS exhibit unexpectedly and unreasonably poor performance. Notably, given that the output of 3DGS is in the format of OLAT, it is surprising that even a straightforward convolution operation applied to any novel environment map would yield a significantly superior outcome compared to the results presented in the paper.*\n\nWe appreciate the reviewer's comments and are committed to addressing all concerns thoroughly. However, we would like to kindly ask for some clarification to ensure we fully understand the points raised.\n\nCould you please specify which particular experiment or result you refer to when mentioning the \"*unexpectedly poor performance*\" of 3DGS? Additionally, we are unclear about the phrase \"*a straightforward convolution operation applied to any novel environment map*\". Could you please elaborate on what this refers to? For example, is this referencing a known baseline or a particular technique?\n\nYour further explanation will greatly assist us in providing a precise and constructive response. We thank you in advance for your guidance. We are also happy to provide a more detailed explanation during the subsequent author-reviewer discussion phase."}, {"review_id": "SuI69zw69H", "reviewer": "Reviewer_v8Ud", "summary": "The paper presents the MetaGS model, a meta-learning approach for improving out-of-distribution 3D GS scene relighting. \nIn the task, the 3D object's is first captured One Light At a Time (OLAT), in other words, the light source and camera positions vary at the same time. OLAT is a convenient capture setup, but lighting and geometry is more entangled. Prior work often fails to truly uncover the material / lighting interaction, and as a result leads to poor generalizability under novel lighting condition. \n\nThe method tackles this issue using two primary ways:\n1. It employs a meta-learning framework using bilevel optimization, which treats each capture sequence as an individual learning task, and use an existing meta learning framework to initialize and update the learnable parameters.\n2. It integrates physical priors from the Blinn-Phong model to decouple shading components. This makes the GS rendering model more physically based and more constrained, avoiding overfitting.", "questions": "While the meta learning framework is novel and effective, I wish the authors did more experiments on the effect of the Blinn-Phong model. Namely, can you still achieve comparably high OOD test performance if you used another rendering model from prior work? The reason being Blinn-Phong model is still quite restrictive in terms of shadow effect. Neural Relightable Gaussians and several other work learned a MLP decoder instead of physically-based and interpretable material parameters. I'm curious if Blinn-Phong model is a contribution or a limiting factor.", "limitations": "The main limitation seems to be the expressiveness of the Blinn-Phong model. \nCurrently the authors claimed contribution for the use of this model, however, it's not clear from experiments how and if at all this simplistic modeling actually improves the optimization result. \nI'm curious if the meta learning itself is robust enough or it has to be coupled with this Blinn-Phong model. \nSome experiments to explore the two contributions independently would clarify my question.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- High impact: the paper tackles a very common challenge in a highly relevant application, relightable GS asset.\n- The proposed solution with meta training seems very effective, yielding large PSNR gains in difficult OOD test cases.", "was_revised": true, "final_justification": "The new experiments in the author's rebuttal excellently answered my concerns. Please add it to the main script.", "rebuttal": "We thank the reviewer for the comments and note that the main concern lies in the effectiveness of the Blinn-Phong model used in our rendering pipeline. Specifically, whether it remains necessary given the presence of an effective meta-learning framework. \n\nWe argue that both the meta-learning framework and the Blinn-Phong shading model play essential and complementary roles in MetaGS, and that their combination is key to achieving strong generalization under OOD lighting. Below, we provide further results and discussion to clarify the respective contributions of the meta-learning component and the Gaussian-Phong renderer.\n\n#### 1. Meta-Learning with Different Relighting Models\n\n> *Can you still achieve comparably high OOD test performance if you used another rendering model from prior work? ...I'm curious if the meta learning itself is robust enough or it has to be coupled with this Blinn-Phong model.* \n \nTo investigate the general applicability of our meta-learning method, we incorporate it into several alternative relighting frameworks across different model families—including both NeRF-based and 3DGS-based methods, and ranging from BRDF-based to fully implicit formulations:\n* **Meta-learning + NRHints** (implicit geometry + implicit lighting): This method does not rely on any parametric BRDF model, instead modeling full light transport via an MLP.\n* **Meta-learning + Neural Relightable Gaussians**$^{[1]}$ (explicit geometry + implicit lighting): Each Gaussian is assigned a learned feature vector decoded into color via an MLP (completely free of explicit shading assumptions). *We apply meta-learning in both of its two stages*.\n* **Meta-learning + GS$^3$** (explicit geometry + explicit lighting with MLPs): This method uses a simplified BRDF formulation, along with an MLP-based shadow refinement module, and a residual MLP to compensate for unmodeled lighting effects.\n\nThe PSNR results are summarized below (*NaN* indicates model failure during RNG’s second stage):\n\n\n| Method        |        Ball         |       PlasCup       |       RubCup        |         Cat         |      Catsmall       |      CupFabric      |        Fish         |      FurScene       |       Pikachu       |        Pixiu        |        Avg.         | $\\Delta$ (Meta$-$Base) |\n|:------------- |:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:-------------------:|:----------------------:|\n| **Ours**      |      **26.76**      |      **27.54**      | $\\underline{27.95}$ |      **26.45**      |      **26.44**      |      **27.29**      |      **24.68**      |      **24.82**      |      **25.54**      |      **25.65**      |      **26.31**      |           -            |\n| NRHint     |        17.25        |        23.92        |        27.44        |        18.04        |        24.63        |        24.65        |        22.57        |        21.55        |        24.00        |        23.03        |        22.71        |           -            |\n| NRHint + meta |        17.70        | $\\underline{24.57}$ |      **28.18**      |        19.75        | $\\underline{25.44}$ |        25.83        |        24.04        | $\\underline{21.90}$ |        24.51        | $\\underline{24.18}$ |        23.61        |          0.90          |\n| RNG       |        20.22        |        22.72        |        24.94        |        *NaN*        |        24.20        |        25.09        |        *NaN*        |        20.81        |        23.55        |        *NaN*        |        23.08        |           -            |\n| RNG + meta    | $\\underline{22.65}$ |        23.93        |        25.88        | $\\underline{21.94}$ |        25.07        |        26.52        |        23.66        |        21.75        |        24.90        |        22.75        |        23.91        | 1.31 (excluding *NaN*) |\n| GS$^3$     |        18.84        |        20.30        |        24.37        |        17.66        |        23.34        |        25.04        |        21.12        |        17.34        |        24.11        |        19.63        |        21.18        |           -            |\n| GS$^3$ + meta |        21.54        |        23.18        |        26.15        |        21.55        |        25.16        | $\\underline{26.96}$ | $\\underline{24.38}$ |        21.22        | $\\underline{25.02}$ |        22.98        | $\\underline{23.81}$ |          2.63          |\n\n\n\nThese results demonstrate that while meta-learning consistently improves OOD generalization across different rendering models, its effect is especially pronounced when combined with explicit geometric modeling and shading methods, and is most significant with our Gaussian-Phong formulation. \n\n\n\n[1] Fan, Jiahui, et al. \"Rng: Relightable neural gaussians.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n\n\n#### 2. Why the Blinn-Phong Model Works Well with Meta-Learning\n> *The reason being Blinn-Phong model is still quite restrictive in terms of shadow effect... I'm curious if Blinn-Phong model is a contribution or a limiting factor.*\n\nFrom the above results, we observe that while the meta-learning scheme is compatible with various model designs, its benefits are most pronounced when combined with the physically structured Blinn-Phong model. Below, we outline two key reasons why this combination performs best:\n- **Simplicity \\& stable convergence:** Despite its simplicity, the Blinn-Phong model defines a low-dimensional parameter space (ambient, diffuse, specular) for meta-learning optimization. Since meta-learning aims to capture shared structures across tasks (e.g., different lighting conditions in our case), this compact parameterization facilitates smoother and more stable gradient flow during meta-training, resulting in faster convergence and improved generalization of the learned scene geometries.\n- **Straightforward inductive bias of disentanglement:** The physically grounded formulation naturally encourages a separation of global scene representations (e.g., geometry, materials) from lighting-specific illumunation factors. Compared to neural-based alternatives such as **Neural Relightable Gaussians (RNG)**, the Gaussian-Phong model injects an explicit physical prior into the rendering process, constraining illumination components to follow interpretable, physics-based rules. Rather than embedding all lighting effects within a high-dimensional latent space, our method promotes the disentanglement of shared global attributes (e.g., scene geometry and material properties) from lighting-specific factors (e.g., direction and intensity) during meta-training.\n    \nIt is worth noting that the full potential of the Gaussian-Phong formulation is only unlocked through meta-learning. While its low-dimensional structure provides strong interpretability and regularization, it does come at the cost of expressiveness, as pointed out by the reviewer. When trained in isolation (see our ablation in Table 3, “w/o meta-learning”), the Phong model can indeed lead to suboptimal results. The bilevel meta-learning loop addresses this by validating each inner-loop update on held-out lighting directions, effectively turning the strong physical prior of Phong into a practical advantage in OOD generalization.\n    \n#### 3. Summary: A Balanced Trade-Off\nIn summary, OOD relighting poses a fundamental trade-off between in-domain expressiveness and OOD generalization. Our experiments demonstrate that combining a lightweight, physically grounded rendering model with meta-learning strikes this balance well. While the proposed Gaussian-Phong model is not the most physically accurate model, it functions as both a reasonably expressive renderer and a compact, differentiable prior that facilitates robust generalization through meta-learning across diverse lighting conditions."}]

Author rebuttal:

Meta-review: This submission receives 1 borderline accept and 3 accepts. As recognized by the reviewers during the rebuttal discussion period, this submission has several contributions:
1. tackle the challenging task, out-of-distribution relighting of 3d scenes. 
2. propose a novel meta-learning paradigm that is agnostic to the specific relighting model, leading to general improvements.
3. the proposed method consistently outperforms existing baselines both quantitatively and qualitatively, yielding large PSNR gains in difficult OOD test cases.

Although the method is only evaluated at asset level rather than scene level, current evaluation is sufficient and comprehensive, and contribution is significant for an important and challenging task in relevant community. Therefore, the recommendation is acceptance with spotlight.
Decision: Accept (spotlight)