Title: MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relighting
Venue: NeurIPS 2025
Accepted: yes
Keywords: Neural rendering

3D relighting

meta-learning
Abstract: Out-of-distribution (OOD) 3D relighting requires novel view synthesis under unseen lighting conditions that differ significantly from the observed images. Existing relighting methods, which assume consistent light source distributions between training and testing, often degrade in OOD scenarios. We introduce **MetaGS** to tackle this challenge from two perspectives. First, we propose a meta-learning approach to train 3D Gaussian splatting, which explicitly promotes learning generalizable Gaussian geometries and appearance attributes across diverse lighting conditions, even with biased training data. Second, we embed fundamental physical priors from the *Blinn-Phong* reflection model into Gaussian splatting, which enhances the decoupling of shading components and leads to more accurate 3D scene reconstruction. Results on both synthetic and real-world datasets demonstrate the effectiveness of MetaGS in challenging OOD relighting tasks, supporting efficient point-light relighting and generalizing well to unseen environment lighting maps.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents the MetaGS model, a meta-learning approach for improving out-of-distribution 3D GS scene relighting. \nIn the task, the 3D object's is first captured One Light At a Time (OLAT), in other words, the light source and camera positions vary at the same time. OLAT is a convenient capture setup, but lighting and geometry is more entangled. Prior work often fails to truly uncover the material / lighting interaction, and as a result leads to poor generalizability under novel lighting condition. \n\nThe method tackles this issue using two primary ways:\n1. It employs a meta-learning framework using bilevel optimization, which treats each capture sequence as an individual learning task, and use an existing meta learning framework to initialize and update the learnable parameters.\n2. It integrates physical priors from the Blinn-Phong model to decouple shading components. This makes the GS rendering model more physically based and more constrained, avoiding overfitting.\n\nSTRENGTHS AND WEAKNESSES:\n- High impact: the paper tackles a very common challenge in a highly relevant application, relightable GS asset.\n- The proposed solution with meta training seems very effective, yielding large PSNR gains in difficult OOD test cases.\n\nQUESTIONS:\nWhile the meta learning framework is novel and effective, I wish the authors did more experiments on the effect of the Blinn-Phong model. Namely, can you still achieve comparably high OOD test performance if you used another rendering model from prior work? The reason being Blinn-Phong model is still quite restrictive in terms of shadow effect. Neural Relightable Gaussians and several other work learned a MLP decoder instead of physically-based and interpretable material parameters. I'm curious if Blinn-Phong model is a contribution or a limiting factor.\n\nLIMITATIONS:\nThe main limitation seems to be the expressiveness of the Blinn-Phong model. \nCurrently the authors claimed contribution for the use of this model, however, it's not clear from experiments how and if at all this simplistic modeling actually improves the optimization result. \nI'm curious if the meta learning itself is robust enough or it has to be coupled with this Blinn-Phong model. \nSome experiments to explore the two contributions independently would clarify my question.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces MetaGS, a novel method for out-of-distribution (OOD) 3D scene relighting, which addresses the challenge of generating realistic novel views under unseen lighting conditions. Existing relighting methods often fail in OOD scenarios due to overfitting to training lighting distributions. MetaGS proposes two key innovations: a meta-learning framework and a differentiable Blinn-Phong model to tackle this issue. Evaluations on synthetic and real-world datasets show MetaGS outperforms baselines in OOD relighting, supports efficient point-light relighting, and generalizes to unseen environment maps.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. OOD Generalization. MetaGS explicitly addresses the OOD relighting challenge, where test lighting conditions differ from training. Its meta-learning scheme simulates unseen lighting during training, reducing overfitting and improving generalization to novel illuminations.\n2. Physical Priors Integration. By incorporating the Blinn-Phong model, MetaGS decouples shading components (ambient, diffuse, specular), enabling more accurate and physically grounded relighting. This structural decomposition enhances the model’s ability to adapt to new lighting scenarios.\n\n**Weaknesses**\nThe incorporation of parametric material modeling, like the Blinn-Phong model used here, is not a judicious approach in this context. In contrast, 3DGS is a potent and universally applicable 3D representation capable of accommodating any material theoretically. Notably, the adoption of parametric models will inevitably impose limitations on the representational capabilities of 3DGS.\n\nQUESTIONS:\nThe experimental results obtained from 3DGS exhibit unexpectedly and unreasonably poor performance. Notably, given that the output of 3DGS is in the format of OLAT, it is surprising that even a straightforward convolution operation applied to any novel environment map would yield a significantly superior outcome compared to the results presented in the paper.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the challenging problem of out-of-distribution (OOD) 3D relighting, where the goal is to synthesize novel views under lighting conditions not seen during training. The authors propose MetaGS, a method that combines meta-learning with 3D Gaussian Splatting to improve generalization across diverse lighting environments. The approach encourages the learning of geometry and appearance features that transfer well to unseen lighting, even when the training data is biased. In addition, MetaGS incorporates a differentiable Blinn-Phong reflection model into the Gaussian framework to better disentangle shading components and improve physical realism. Experiments on both synthetic and real-world datasets show that MetaGS consistently outperforms existing methods in OOD relighting scenarios, demonstrating strong generalization to new light sources, including complex environment maps.\n\nSTRENGTHS AND WEAKNESSES:\n> Strengths\n- The paper is well-organized with a clear and logical flow, making it easy to follow.\n- Tackling a more challenging relighting scenario through a multi-task learning approach seems effective.\n- While the main focus is on the OLAT relighting problem, the method also shows strong performance under environment map-based relighting, highlighting its generalization capabilities.\n- The proposed method consistently outperforms existing baselines, both quantitatively and qualitatively.\n\n> Weaknesses\n- One of the core contributions, the meta-learning scheme, feels relatively general and could potentially be applied to other relighting models as well. While the implementation details may differ, showing that this training paradigm can benefit other architectures would have made the case for MetaGS being model-agnostic even stronger.\n\nQUESTIONS:\n- As mentioned in the weaknesses, it would be helpful to know whether the proposed training framework can be applied to other existing relighting models. If so, showing the results of such experiments could further support the claim that the meta-learning approach is model-agnostic and broadly applicable.\n\n- According to line178, the implementation of MetaGS is done in PyTorch. Is there a specific reason why CUDA was not used for performance optimization? A brief explanation would clarify whether there are any trade-offs or constraints involved in the current implementation.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces MetaGS, a novel method for 3D scene relighting under out-of-distribution (OOD) lighting conditions using 3D Gaussian Splatting. Unlike previous OLAT (One-Light-At-a-Time) methods that struggle with generalization to unseen lighting, MetaGS addresses this by combining a differentiable Blinn-Phong reflection model to disentangle shading components and a bilevel meta-learning framework to learn lighting-independent scene representations. The approach enhances the model's ability to generalize across varying lighting conditions, even with spatially biased training data, and shows strong performance on both synthetic and real-world datasets without requiring retraining for new lighting distributions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. MetaGS effectively tackles the challenge of out-of-distribution (OOD) lighting in 3D relighting. By leveraging a bilevel meta-learning strategy, the model simulates unseen lighting conditions during training, enabling it to learn lighting-invariant geometry and reflectance attributes. This allows robust relighting under novel lighting setups without retraining.\n2. The method incorporates a differentiable Blinn-Phong reflection model into Gaussian splatting, introducing physically grounded priors. This enables the model to explicitly disentangle ambient, diffuse, and specular components, enhancing the realism and interpretability of relighting results.\n\nWeaknesses:\n1. This work focuses on out-of-distribution (OOD) lighting scenarios from biased OLAT data. However, real OLAT captures use a single camera with one moving point light, limiting lighting diversity and making extreme OOD cases rare in practice. Thus, the OOD problem here is largely artificial and not representative of typical real-world conditions.\n\nQUESTIONS:\n1. The paper doesn’t clarify how many distinct light positions are used during training. To better assess OOD robustness, could the authors evaluate performance when training with very few or clustered lights? This would test the method under more realistic and challenging conditions. Strong results in such cases would significantly strengthen the paper’s impact.\n2. Since OLAT images are captured under individual point light sources, it is feasible to synthesize ground truth renderings for environment map relighting by linearly combining OLAT images according to the lighting distribution in the environment map. However, the paper provides only qualitative results for this experiment, without reporting any quantitative metrics or comparisons with ground truth images. Including such ground truth-based evaluations (e.g., PSNR, SSIM, LPIPS) would significantly strengthen the claim of generalization to complex lighting. Could the authors provide comparisons with ground truth images to further validate the results?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-05"}