PI-Light: Physics-Inspired Diffusion for Full-Image Relighting
Authors: Zhexin Liang
Zhaoxi Chen
Yongwei Chen
Tianyi Wei
Tengfei Wang
Xingang Pan
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion model
Relighting
Inverse rendering
Neural forward rendering
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Full-image relighting remains a challenging problem due to the difficulty of collecting large-scale structured paired data, the difficulty of maintaining physical plausibility, and the limited generalizability imposed by data-driven priors. Existing attempts to bridge the synthetic-to-real gap for full-scene relighting remain suboptimal. To tackle these challenges, we introduce **P**hysics-**I**nspired diffusion for full-image re**Light** ($\pi$-Light, or PI-Light), a two-stage framework that leverages physics-inspired diffusion models. Our design incorporates (i) batch-aware attention, which improves the consistency of intrinsic predictions across a collection of images, (ii) a physics-guided neural rendering module that enforces physically plausible light transport, (iii) physics-inspired losses that regularize training dynamics toward a physically meaningful landscape, thereby enhancing generalizability to real-world image editing, and (iv) a carefully curated dataset of diverse objects and scenes captured under controlled lighting conditions. Together, these components enable efficient finetuning of pretrained diffusion models while also providing a solid benchmark for downstream evaluation. Experiments demonstrate that $\pi$-Light synthesizes specular highlights and diffuse reflections across a wide variety of materials, achieving superior generalization to real-world scenes compared with prior approaches.

Published as a conference paper at ICLR 2026
PI-LIGHT: PHYSICS-INSPIRED DIFFUSION FOR FULL- IMAGE RELIGHTING
Zhexin Liang', Zhaoxi Chen', Yongwei Chen', Tianyi Wei', Tengfei Wang?, and Xingang Pan L-Laa, Nanyang Technological University Tencent
image
Image Intrinsic Decomposition
(b) Shading Prediction and Relighting
Careo
Light conditon
Light condiiu
Stndia Frodecue
Shuckin Pred Cto
Nomal Estimation
Materal Estimation
Relighoed Image
Relighted Image
Figure I; Light enables full-image relighting through physics-inspired diffusion models. achieves favorable results in both a) inverse rendering and b) neural forward rendering stages, en- abling precise control over lighting.
ABSTRACT
Full-image relighting remains challenging problem due the difficulty of col- lecting large-scale structured paired data, the difficulty of maintaining physical plausibility, and the limited generalizability imposed by data-driven priors. Exist- ing attempts to bridge the synthetic-to-real gap for full-scene relighting remain suboptimal. To tackle these challenges, we introduce Physics- Physics-Inspired diffu- sion for full-image reLighting (n-Light, or PI-Light), two-stage framework that leverages physics-inspired diffusion models. Our design incorporates (i) batch- aware attention, which improves the consistency of intrinsic predictions across a collection of images, (ii) a physics-guided neural rendering module that en- forces physically plausible light transport, (iii) physics-inspired losses that regu- larize training dynamics toward physically meaningful landscape, thereby en- hancing generalizability real-worlh image editing, and (iv) carefully curated dataset of diverse objects and scenes captured under controlled lighting condi- tions. Together, these components enable efficient finetuning pretrained diffu- sion models while also providing a solid benchmark for downstream evaluation. Experiments demonstrate that r-Lighh synthesizes specular highlights and diffuse reflections across wide variety of materials, achieving superior generalization to world scenes compared with prior approaches. Code will available at http:://jihubbcom//heeiiniiggg
INTRODUCTION
Recent advances in computer graphics and vision have highlighted image relighting as critical
task for film production, augmented reality, and digital content creation. Existing works can be grouped into object-centric approaches (Zeng et al. 2024a| Mei et al. 2023 Zhang et al.] 2025]
iPonglertnapakonn al. 2023 He et al. 2024 Futschik
2023), which often ignore replace

---

Published as a conference paper at ICLR 2026
the background, and scene-level methods (Choi al. 2024: Koesis et al. 2024a), which attempt full-image relighting but remain largely data-driven and struggle to generalize beyond the training distribution without extensive diverse data, and cannot maintain self-
objects' lighting. While state-of-the-arr foreground relighting methods (Zhang et al.) 2025) have shown impressive results, they still suffer from albedo inconsistency and imprecise lighting control. For example, as shown in Fig. 2 carpet colors are altered (1 row) and the intended leftward light appears to come from the left rear in the absence of internal light source (2nd row). These limitations highlight the lack of a decomposed scene interpretation and the inherent limitations of data-driven priors, which are essential for controllable and physically plausible full-image relighting.
While prior methods have advanced the field, their limitations highlight three major challenges for full-image relighting. First, the main chal- lenge lies in the difficulty of acquir- ing datasets with diverse and complex scene illuminations: obtaining high- quality images under diverse and controlled lighting conditions across full scenes is both resource-intensive and technically challenging. In par- ticular, it is infeasible to obtain real- world datasets that capture the same scene under multiple lighting con- ditions. Second, ensuring physical plausibility remains challenging, as purely data-driven approaches often generate results that violate basic light transport principles. Finally, limited data-driven priors restrict generalizability, leading to poor robustness when facing more unseen materials, lighting conditions, realistic scene compositions.
Inpue Image
Foreground
Righti ighh
Light Conditio::
Input Image
Forog ound
Frompt: Loft Light
Light Condrion:
IC-Ught
Ours
Figure Comparisons between IC-Light (Zhang et al. 2025) and Ours. The state-of-the-an art foreground relight- ing method still suffers from inconsistencies maintain- ing albedo uniformity and lacks physically plausible control over lighting direction.
To alleviate these challenges, we propose Light,, a novel diffusion- based neural relighting pipeline designed to tackle the full-image relighting problem. Similar to the previous PBR approach (Zeng jet al. 2024b Liang et al.] (025)), our approach also adopts a two-stage framework of inverse neural rendering followed by neural forward rendering, but differs from prior works in the following key aspects. First, inspired by Wonder3D (Long et al. 2024), during both inverse neural rendering and neural forward rendering stages, we extend standard self-attention layers to be global-aware, en- abling communication across batches. This design improves both efficiency and consistency among predicted intrinsics. Second, we design a physics-inspee insiise neural forward rendering module, where the physics- based loss serves as an efficient learning mechanism. Note that, our physics-inspired loss function is non-trivial. It regularizes the training dynamics into physically plausible landscape, en- abling easier convergence and allowing the model to learn more correct light transport with less data and computation. As a result, our method achieves competitive performance and generalizes well to with fewer training samples than previous works (Zeng al. 2024b Liang ct al. 2025) and without access to fully realistic datasets. We further propose a simple yet effective lighting representation that uses only the front hemisphere of the environment map. This design avoids interference from self-luminous objects and built-in scene lighting, enabling effective control of light direction and intensity while preserving background consistency.
real-world scenes,
To address the data scarcity that hinders full-image relighting research, we constructed a new dataset featuring diverse objects and scenes, all captured under controlled lighting conditions. This dataset not only supports supervised learning but also enables comprehensive downstream benchmarking. By training on this dataset, combined with the image prior from pretraining, T-Litht demonstrates strong generalization across diverse objects and scenes, as extensively validated in our experiments.
Our contributions can be summarized as follows:
We propose Lightt a diffusion based neural relighting framework that achieves physically- aware full-image relighting.
2

---

Published as a conference paper at ICLR 2026
We impose ahysics-inspireds light transport priors onto the neural forward rendering module as a regulation to help the model converge toward physical principles and enhance generaliz- ability.
We contribute a new, high-quality dataset of diverse objects and scenes rendered under con- trolled lighting conditions, providing a valuable resource for advancing full-image relighting research.
Extensive experiments demonstrate the superiority of our method over previous models.
RELATED WORK
Neural Rendering. PBR simulation has recently emerged research focus, encompassing both inverse rendering and forward rendering processes. Many recent works (Zhu et al.) 2022; Zhang the
et al. 2024 Zeng et al. 2024at Kocsis al. 2024b Li et al. 2020) have concentrated inverse rendering process, also referred to as image intrinsic decomposition. However, due to the difficulty of acquiring large-scale training data, these methods have limited applicability and primarily designed for specific scenarios, such as indoor scenes (Zhu et al.) [022] Kocsis et al.)
2024b [i et al., 2020) or object-level inverse rendering (Zhang al. 2024| Zeng et al. [024a] Jin et al. 2024). MAIR (Choi et al., (2023) improves inverse rendering with multi-view attention and spatiall-varying lighting estimation but is sensitive to viewpoint inconsistencies.
Other approaches focus on the inverse rendering of illumination maps (Enyo & Nishino 2024
Lyu et al. 2023). aiming to infer scene lighting properties directly from images. Additionally, some methods (Bae & Davison, 2024; Bae et al. [02T]; Fu et al.) (024) specialize in geometry estimation, particularly in normal and depth estimation. Furthermore, unsupervised approaches and those leveraging large-scale model priors (Pan et al. [2000) Paaantonnou et al. 2023 Chen et al., 2024 introduce various constraints to recover multiple intrinsic components.
RGB++X (Zeng al., 2024b) successfully employs diffusion model to simulate both the inverse
and forward rendering processes. However, its forward rendering relies on irradiance as the input lighting condition, which makes precise lighting control challenging. Additionally, its application is limited to indoor scenes. More recent work (Liang et al., (025) leverages video diffusion model to perform both inverse and forward rendering of vidcos. We further discuss the differences between our method and these two approaches in Appendix AA.11.
2D Image Relighting, In addition advances in 3D relighting, such as Poirier-Ginme et al. (2024); Zhao et al. al. (2024), 2D relighting has also begun to attract increasing attention in recent years. Recent work (Zhang al., (025) has achieved relighting effects for foreground objects and can generate different scenes based on prompts. However, as shown in Fig. 2 its data-driven approach without physical representations still fails to preserve the albedo and background, lacking precise controls
over lighting direction.
Many previous studies focus on portrait relighting (Papantoniou al. 2023 Pandey et 2021
Tajima et al. 2021 Futschik et al., 2023 Ponglertnapakom et al. 2023 Mei et al. 2023 or ob-
ject relighting (Bashkirova et al.] (2023), achieving practical performance foreground relighting. ScribbleLight (Choi et al. al. (024) a model that enables indoor scene relighting using scribble-based controls. However, applying scribbles to control lighting directly on images still lacks precision and it's application is limited to indoor scenes. While LightIt (Kocsis et al., 2024a) and OutCast (Grif- fiths et al.] 2022) enable outdoor full-image relighting, LightIt requires normals as input, and both methods rely directional lighting models that generalize poorly to indoor scenes.
PRELIMINARIES
Physically-Basee Rendering (PBR) is rendering approach that simulates interactions of light with materials in a physically accurate manner. The Principled BRDF (based on the Disney Prin- cipled Shader (Burley & Studios] 2 is user-friendly BRDF model, which is based micro- facet BRDF models, including the Lambertian reflection model for diffuse surfaces and the Cook- Torrance microfacet model for specular reflections. Here, only introduce its formula to
3

---

Published as a conference paper at ICLR 2026
Plysical based
Denpyl
Devi
Emibeddiing
Convh
Recsnstuution
8: VAE dccoder
J" VAF eecadde
Reflection
Caloulaned Difluso loss
Stage Forward Bendering
Figure 3: Overview of n-Lighh. Stage I: Inverse Neural Rendering. Given a 2D image as input, this stage repurposes a pretrained image diffusion model to simultaneously predict four intrinsic components: albedo, normal, roughness, and metallic. Stage 2: Neural Forward Rendering, Given the input image, the physical intrinsies from Stage 1, and a target lighting condition, this stage tames the image diffusion model to generate the relit image along with its diffuse and specular shading, guided by physics-insping light transport prior.
compute the diffuse map:
D(p) = L(w) max(0, (p) w)dw / L(w) max(0, w)siin dddø,
(1)
where p is a pixel location in image, Ω is the hemisphere above the surface, L(w) is the environment map lighting in the incoming light direction w, (p)) is the surface normal at p.
Latent Diffusion Model (LDM). LDM (Rombach et al. 2022) is proposed to the diffusion process in latent space instead of pixel space, saving computational costs. LDM introduces an autoencoder model for image reconstruction. The encoder & encodes the input image in 2D latent space z. The decoder 2 then reconstructs the image from z. Thus, the overall LDM framework consists of denoising network (U-Net) and an autoencoder. We calculate the overall loss using
V-prediction (Salimans & Ho 2022):
Lv.peed = Exo,,r [110( 1) - Us
where,
Va + vI ãie, Vv /1 0,
(9(2,, t) the model's predicted V-prediction, and the expectation is taken over data samples 20s Gaussian noise and timestep t.
4 METHODOLOGY
In this section, we present our dataset construction pipeline along with our two-stage method. First, Sec. 1.1 we introduce our data construction pipeline, which encompasses both object- and scene- level data. The dataset consists of RGB images paired with their corresponding ground-truth in- trinsic properties, as well as light conditions, rendered using Blender (Foundation) 2024). Unlike previous methods (Kocsis al. 2024b Zeng et al. 2024b) that represent illumination using irra- diance, we employ a rendered gray ball to model lighting. allowing for more precise and intuitive control. Next, in Sec. 4.2 leveraging the constructed dataset, describe inverse neural rendering pipeline, which decomposes an input RGB image into multiple intrinsic components, Fi- nally, in Sec. [4.3 utilizing the decomposed intrinsics from the previous stage, we finetune Stable Diffusion (Rombach et al.) 2022) to relight the image based on user-provided lighting conditions. An overview of our method illustrated in Fig.
4.1 DATASET CONSTRUCTION
To train model capable of accurately decomposing an input RGB image, need to collect objects and scenes that contain rich material information. At the object level, we filter objects from Obja- verse (Deitke al. 2023b a), selecting those with BRDF material properties. For scene data, while some existing datasets (ocsss et al. 2024b) Zeng et al., 2024b) provide intrinsic components, they primarily focus indoor scenes and lack object masks exclude translucent transparent objects,
4

---

Published as a conference paper at ICLR 2026
which can hinder accurate material estimation. Moreover, these datasets do not render images under varying lighting conditions- a critical factor for our task. To address these limitations, we curate a scene dataset from online repository (BlenderKit) 2024) and render intrinsic components along with diverse lighting conditions for training.
Object Data. For object-centric data, we randomly select over 10,000 objects from filtered Obja- verse (Deitke et al. 2023b a) that contain BRDF materials and render them under varying viewpoints and lighting conditions. Each object rendered with 10 views and 10 lighting conditions, totaling 100 images per object. Lighting is sampled with preference toward the upper hemisphere, and in- cludes both point lights and HDRI maps randomly selected from over 700 environment maps from Poly Haven (Haven 2025).
BlenderProc [Denninger et al. 2023) utilizes Blender's composition layer to generate intrinsic prop- erties. However, we observed that certain albedo values fluctuate with the light source, and for trans- parent or semi-transparent materials, the composition layer often outputs entirely white or black val- ues, which are inaccurate. To mitigate these issues, we extract all intrinsics directly from Blender's Principled BRDF nodes while generating a mask to exclude non- con-Principeed regions.
Different from all previous methods (Zhang al. 2025 Bashkirova et al. 2023 Zeng et al. 2024a) that either condition on global HDRI lighting information (Zeng et al.) (2024) or infer lighting from background to relight foreground (Zhang et al. 2025 Bashkirova al. 2023),
method constructs the lighting condition using only the light sources directly facing the image: the front
hemisphere of gray ball rendered from the HDRI. This representation mitigates the influence of self-luminous objects and built-in light sources (e.g.. light from windows) present in the original scene.
Scene Data. We curate 300 high-quality indoor and outdoor scenes in total from BlenderKit (BlenderKit (024)) manually filtered to remove scenes with excessive special effects, such as fog and rain. Camera views are sampled via constrained random perturbations to maintain valid viewpoints. Lighting is augmented by adding a point light source behind the camera, ensur- ing diverse shadow and highlight patterns. To maintain consistency in normal rendering, we fix the Blender version per scene.
Details of the sampling strategies and rendering settings are provided in Appendix A.I
4.2
STAGE 1: INVERSE NEURAL RENDERING
4.2.1 PIPELINE
As shown in Fig. 3 (Left), inspired by Wonder3D (ongg et al. 2024) and GeoWizard (Fu et al., 2024 our first stage takes four batchwise concatenated mppt images In RCx as condition channelwisely concatenating with noise and simultaneously predicts the albedo A E [ normal N E C roughness R E RCxH*W metallic M RCCH*W for the given in- put. The CLIP image embedding of the condition inputs go into the cross-attention layers well. The standard self-attention layers are extended to be global-aware, enabling communication across batches. This design improves both efficiency and consistency among predicted intrinsics.
Attention links across the four batches ensure structural consistency among the predicted intrinsic components while enabling the sharing of global information, thereby improving prediction accu- racy. Our model uses four one-hot labels J. E N1 to control which batch outputs which intrinsic component. As shown in Fig. 5 compared the latest and most relevant 2D rendering work, RGB++X (Zeng et al.] (2024b), our model can generate all four intrinsic components simultaneously while maintaining structural consistency.
4.2.2 TRAINING OBJECTIVE
As mentioned in Sec. some intrinsic components are either inaccurate or difficult to obtain during data collection, inc includigg albedo for translucent/transparent objects, normal for the sky, and intrinsic components of objects rendered without the Principled BRDF model. To address this issue, we generate a corresponding mask alongside each dataset sample to annotate these unreliable re- gions. Here we use this mask to calculate mask loss. It is important to note that the latent space and
5

---

Published as a conference paper at ICLR 2026
RGB space are not strictly pixel-aligned, though we have not yet found better alternative. Please refer to Appendix A..1..1 for further analysis.
Specifically, given the latent V-prediction output of any intrinsic component, denoted Upred R4x Wh the masked latent loss applied directly the V-prediction space. The corresponding mask m downsampled by a factor of 8, yielding my E IR * w which is then element-wise multiplied with both the predicted and ground-truth latents. The loss defined as follows:
Laggel MSE M2y Vaaget m22)
where Garget E
121 # denotes the V-prediction of the latent code obtained by adding ground
truth noise to the ground truth intrinsic latent.
4.3 STAGE NEURAL FORWARD RENDERING
4.3.1
PHYSICS-INSPIRED PIPELINE
As shown in Fig. (Right), our relighting pipeline is designed based on the Surface Reflection Model, formulated in equation 5 within the framework of Physically-Basdd Rendering (PBR):
Trenteeed = @ D++ S,
(5)
where rendered represents the rendered image, and A, D, represent the albedo, diffuse and specular maps, separately.
Our model follows the Principled BRDF (Zeng al. 2024a). Both the input and output of our relighting model consist of three components: described in equation g our model separately outputs the diffuse and specular components, and the final rendered relighted image is obtained through a loss constraint which is illustrated in Sec. 4.3.2
According to the Principled BRDF in Sec. the diffuse component is modeled using Lambertian reflection, which depends solely on the lighting conditions and surface normal. In contrast, the spec- ular component follows the Cook-Torrane microfacet model, which is influenced by a combination of lighting conditions, surface normal, roughness, and metallic properties.
To account for these dependencies, our relighting model takes three batchwise concatenated input conditions, denoted as:
In psuges [I Iin2: Ima)
Im (Jm A), Inz (N, L, m). Ins (N, L, R, m),
(7)
where Iin, R2C*XxW represents the concatenation of the input image and albedo. Im2 E R33*HXW corresponds to the surface normal, lighting condition, and mask. and I BRCCHHW includes the surface normal, lighting condition, metallic, roughness and mask.
Each of these input conditions concatenating with noise maps to a corresponding batchwise con- catenated output, denoted as:
oou [Leeitt Dpred- Sprea),
where Trein represents the final rendered relighted image, Dpred corresponds the diffuse component, and Spred denotes the specular component.
Thus, the input and output for each batch
Lnnt (Lin, A) + Iretit, Janz (N, L, + Dpredt Lin3 L,, - Spred
This formulation ensures that the second and third batches of the model focus on the strength and structure of lighting and shadow, while the first batch focuses more on the final rendered color, allowing the outputs of the second and third batches to more closely follow the properties provided in those batches. For instance, the third batch enables the specular component to directly learn the correspondence between the specular map and the metallic information provided in the third input channel. As a result, our method effectively reconstructs highlights under the given conditions, shown in Fig. | Furthermore, by incorporating our Physics-inspired loss, shown in Sec. 4.3.2 we enforce physical consistency in the final rendered output, ensuring that it adheres to the given lighting conditions.

---

Published as a conference paper at ICLR 2026
4.3.2 TRAINING OBJECTIVES
Since physical laws lose their meaning when computed in the latent space, our physics-based losses for the relighting model are applied entirely in the RGB space, i.e., it is computed after the VAE decoder reconstructs the output, as mentioned Sec.
Diffuse Shading Loss. Since obtaining ground-truth diffuse reflectance is challenging in certain cases and some scene lighting con- ditions may be inaccurate, we incorporate manually-computed loss based on the normal map and environment map. This loss encour- ages the model to better capture the relation- ship between the normal and lighting condi- tions. The computation process is as follows:
Glayball
Enviooment Map
Refuction
Mop
(without casl slastun)
Image
Figure 4: Diffuse Shading Loss. We incorporate a manually computed loss using the Lambertian model, without requiring ground truth annotations of diffuse shading.
As shown in Fig. 4 given a gray ball represent- ing the light source G, convert it into an vironment map E E 32 16x3, The normal map N is then normalized the range [-1,
Since the gray ball rendered in our scene already corresponds to a diffused version of the lighting representation, unfolding directly yields the energy-conserving diffuse environment map Eait which is the discretized form of the diffuse map described in equation |
By lookup-bascd sampling using the normal map, we obtain the diffuse map that does not include cast shadows:
N [ng, ny no] N 3R3xHxW
(9)
UV = tan tan
nr n
cos 1 (ny) - 1}, Dealculated = grid samplee Eaitt, UV),
(10)
π
Finally, compute the calculated diffuse loss between the calculated diffuse map Dealeulated and the predicted diffuse map Dpred decoded from latent prediction:
Los MSE(D Prred)
(11)
Physical-based Shading Loss. To ensure that the rendered results are shaded according to the given lighting conditions and adhere to physical principles, we define physics-based loss as shown
Eq. 12 based on the formulation in Eq. 5
Lps = MS (Treti A @ Dpred + Spred);
(12)
Reconstruction Loss. To ensure that the overall content of the image remains unchanged before and after relighting, we introduce a reconstruction loss. This loss is computed as a perceptual loss by extracting features from both images using the DINO feature extractor and measuring the difference between them. The specific formulation is given as follows:
Tnec = | Treti) o( ((Tinuu)
(13)
where @ denotes the DINO feature extractor; Lee and In denote the relighted and input images.
Final loss function for the training:
Lstage2 Lvvpreed (A1 + A2 + Are.
(14)
where \1, A2, A are three constants weight the losses, is the timestep the diffusion noise. We use 1/1 to mitigate the errors introduced by directly inferring 2g from the predicted zt.
5
EXPERIMENTS
IMPLEMENTATION DETAILS
Training. We trained for 80k iterations in stage and 90k iterations in stage two using eight 40GB A100 GPUs, with a batch size of and a learning rate le 1e-5, optimized using Adam. The initial weights of the U-Net were derived from GeoWizard (Fu at. 2024

---

Published as a conference paper at ICLR 2026
Input
Zhu el.
Kocsis
RGB+XX
Ours
Ground truth
regions ground truth represent masked areas, indicating that the shader used these regions not based Principled BRDF model and bas therefore been masked. (top button: Albedo, Normal, Roughness, Metallie)
Figure 5: Qualitative comparisons on our Scene200 test dataset. From top bottom, the results correspond to albedo, normal, roughness, and metallic comparisons. Our method produces more detailed and accurate results, even specular reflection regions.
MERLK
POCKT
USINT
21.50
133337
S3RITT
(9j-19)
Thaut
16 0.
8975
B1601
17:07
15.30
3
[11011
17:29
0.0178
111
17.43
BGB--X
30.00
23.75
15.97
0,5084
111226
38.53
BJ093
1351
03315
$ame2000
7lacet
13.55
6459
0.2055
16.36
66.55
8.30%
0.64
5385
0.3050
5.40
6.115
0.9972
11.29
8.3435
14.46
0.925
8:224
1L280
7.00
0,300
13.85
0.666
03341
01010
V888 22
41
M0H
Table I: Quantitative comparisons on inverse neural rendering task. Our method outperforms most of the metrics both object and scene level test dataset.
Input Image
Lighting Condition
RGB-X
Gaffer
DiLightNet
Ours
(reconstruction)
Figure 6: Qualitative comparisons with the state-of-the-art object relighting methods.
Evaluation Protocols. We randomly select 50 objects from Objaverse Deitke et al. 2023b a) and 20 scenes from BlenderKit (BlenderKit 2024), which not included in the training dataset. For each object, we rendered images under 6 different lighting conditions and 4 camera views, yielding total of 6 x 50 images. For the scene data, we rendered 200 images from various viewpoints.
8

---

Published as a conference paper at ICLR 2026
Methed RGBB++ Neural Gaffer Dil iethitt
Dataset
PSNRT
SSIMF
LPIPS.
Dataset
PSNRT
SSIMF
LPIPS]
9.64
0.8878
0.0785
Object50
11.72
0,9069
0.0844
Object50 (reconsimiction)
13.69
0.9186
0,0765
12.21
0.8938
0.0890
12.72
0.9035
0.0816
Ours
14.09
0,9211
0.0553
15,06
0,9332
0.0458
Table 2: Quantitative comparisons on forward rendering task. Our method outperforms other state- of-the-art methods on object50 test dataset. (reconstruction) means the input image, output image and lighting condition are well aligned, that is, the inputs are identical to the ground-truth outputs.
Promptd ighting Proference: "Left Light"
"Right Light
Input image
IC-Light results (seed: 12345)
Our results under different lighting conditions
Figure 7: Qualitative comparisons with the state-of-the relighting method IC-Light (Zhang al. 2025
demonstrating the generalization ability of our model on real images.
For all quantitative results in our paper, we set CFG to 1.0 in stage 1 and 1.5 in stage 2, with these values selected empirically. The influence of the CFG scale is further discussed Appendix A.3
5.2 COMPARISONS FOR INVERSE NEURAL RENDERING
For inverse rendering, we compare four state-of-the art inverse neural rendering methods; Intrinsic- Anything (Chen et al.] 2024). Zhu (Zhu et al.) 2022). Kocsis et al. (Kocsis et al. 2024b),
and RGB (Zeng et al. 2024b), using our test datasets mentioned in Sec. 5.1 We evaluate performance using 3 metrics: PSNR, SSIM (Wang al. 2004). LPIPS (Zhang et al. 2018).
As in Tab. [ our model achieves state-of-the-an performance on most metrics. Notably, the resolu- tion Zhu al. (2022) and Kocsis al. (2024b) relatively low, which may contribute to higher pixel-supervised scores. As in Fig. our method produces more precise and clearer results. For instance, in the first row, our model successfully predicts the albedo of objects reflected in a mirror, whereas other methods incorrectly treat them part of the mirror surface.
5.3 EVALUATIONS NEURAL FORWARD RENDERING
As shown in Tab. and Fig. 6 our model outperforms existing object relighting methods under our evaluation setting. Since the second stage of RGB+x (Zeng et al., (022bb) does not involve environment map control, we report metrics under the reconstruction setting. As observed in the first row, DiLightNet (Zeng et al. 2024a) tends to bake the input shadow into the output. Similarly, in the second and third rows, Neural Gaffer (Jin et al. (024) also exhibits shadow baking when the shadow covers large area. In contrast, RGB+>X (Zeng et al. 2024b) is trained only indoor scenes, resulting in limited generalization capability. In comparison, our method does not suffer from these issues and achieves more faithful relighting results.

---

Published as a conference paper at ICLR 2026
Table Quantitative comparison between
pipelines with and without decomposition, well as with and without the two shading loss functions
proposed in method. All models are trained
for 30k iterations to ensure fairness.
Method
Metries
Eye Map
Decomposition
Lps
PSNRT
SSIM
LPIPS]
Figure 8: Qualitative comparisons in the abla- tion study for the effects of decomposition and the two shading loss functions proposed in our method.(zoom in for better view)
10.88
0.8964
0.0712
11.95
0.9072
0.0616
11.97
0.9054
0.0617
12.18
0.9070
0.0620
12.68
0.9139
0.0572
As shown in Fig. 7 our model accurately generates specular highlights, particularly on highly metal-
lic objects such the desk lamp in the fourth row. Additionally, successfully produces shadows on the ground or the wall, as seen with the basketball and water cup in the second row. While IC- Light (Zhang et al., 2025) exhibits two main limitations: 1) It fails to preserve the object's albedo and roughness, as highlighted in the red box, where both the object's color and glossiness have changed. 2) lacks precise lighting control; for instance, in the second row, despite providing right-light prompt, the illumination still originates from the left.
5.4 ABLATION STUDIES
The Necessity of Image Intrinsic Decomposition. As shown Fig. and Tab. 3 (w/o decompo- sition), we conducted an end-to-end training experiment using only the lighting condition input and evaluated the results on the object50 test dataset. We observed that without decomposition, as shown in Fig. [ (bottom row), the model fails accurately perceive the object's normal, leading to unrealistic shading effects that do not align with the object's 's true orientation. As shown in Fig. [g (top row), the albedo varies with lighting changes, further deviating from physically consistent behavior.
The Effectiveness of the Diffuse Shading Loss and Physical-based Shading Loss. As shown in Fig. 8 and Tab, 3 we evaluate the effectiveness of the two shading loss functions on the object50 test dataset. Tab. 3 indicates that both losses significantly enhance model performance. Moreover, a comparison between the third and fourth columns in Fig. reveals that the mhysically-inspiree shading loss effectively corrects highlight and shadow directions.
CONCLUSION
We have introduced novel neural rendering diffusion-based approach for full-image relighting. To support this, we constructed a dataset containing objects and scenes under diverse lighting condi- tions. Our method incorporates physical light transport, enabling controllable illumination while preserving the intrinsic physical properties of objects. Experimental results demonstrate the supe- riority of approach in predicting physical attributes and the effectiveness and controllability of our relighting results.
ACKNOWLEDGMENT
This research is supported by the National Research Foundation, Singapore, under its NRF Fellow- ship Award as cash and innd funding from NTU S-Lab and industry partner(s).
This research is also supported by NTU SUG-NAP, as well
REPRODUCIBILITY STATEMENT
To ensure that all results, including both visual and quantitative ones, reproducible, fixed the random seed to 42 for all experiments. For the proposed datasets, a complete description of the data processing steps is provided in Appendix A.T
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Gwangbin Bae and Andrew J Davison. Rethinking inductive biases for surface normal estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 2535-9545, 2024.
Gwangbin Bae, Ignas Budvytis, and Roberto Cipolla. Estimating and exploiting the aleatoric uncer- tainty in surface normal estimation. In Proceedings of the IEEEJCVF International Conference on Computer Vision, pp. 13137-13146, 2021.
Dina Bashkirova, Arijit Ray. Rupayan Mallick, Sarah Adel Bargal, Jianming Zhang, Ranjay Kr- ishna, and Kate Saenko. Lasagna: Layered score distillation for disentangled object relighting. arXiv preprint arXiv:2312.00833, 2023.
BlenderKit. Blenderkit.
https: http://wwwwblleedeeee. www.derkit.ccom h.tp:::/ww.bleeerrrtt.o
2024. URL https: // /ww.
blenderkite com/
Brent Burley and Walt Disney Animation Studios. Physically-based shading at disney. Acm Sig- graph, 2012(2012):1-7,22) 2012.
2012.
Xi Chen, Sida Peng. Dongchen Yang. Yuan Liu, Bowen Pan, Chengfei Lv, and Xiaowei Zhou. Intrinsicanything: Learning diffusion priors for inverse rendering under unknown illumination. In European Conference on Computer Vision, pp. 450-467, 2024.
Jun Myeong Choi, Annie Wang. Pieter Peers, Anand Bhattad, and Roni Sengupta. Scribblelight: Single image indoor relighting with scribbles. arXiv preprint arXiv:2411. 17696, 2024.
Jun Yong Choi, Seok Yeong Lee, Haesol Park, Seung-Won Won Jung, Ig-Jae Kim, and Junghyun Cho. Mair: multi-view attention inverse rendering with 3d spatially-varying lighting estimation. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8392- 8401. IEEE, 2023.
Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Newral Information Processing Systems, pp. 35799-35813, 2023a.
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: universe of anno- tated 3d objects. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 13 13142-13153, 2023b.
Maximilian Denninger, Dominik Winkelbauer, Martin Sundermeyer, Wout Boerdijk, Markus Knauer, Klaus Strobl, Matthias Humt, and Rudolph Triebel. Blenderproc2: procedural pipeline for photorealistic rendering. Journal of Open Source Software, 8882):4901, 2023. doi: 10.21 100/000s..0000....... URL htttps::///dee org/ 10. 211111 joss. 04901
Yuto Enyo and Ko Nishino. Diffusion reflectance map: Single-image stochastic inverse rendering of illumination and reflectance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11873-11883, 2024.
Blender Foundation. Blender 3d modelling and rendering package, 2024. URL https://
www.blender.cm
Version 4.0, retrieved from https:/wwww.blender.rrg///
Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. Geowizard: Unleashing the diffusion priors for 3d geometry estimation from single image. In European Conference on Computer Vision, pp. 241-258, 2024.
David Futschik. Kelvin Ritland, James Vecore, Sean Fanello, Sergio Orts-Escolann Brian Curless, Daniel Sýkora, and Rohit Pandey. Controllable light diffusion for portraits. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8412-8421, 2023.
David Griffiths, Tobias Ritschel, and Julien Philip. Outcast: Outdoor single-image relighting with cast shadows. In Computer Graphics Forum. Wiley Online Library, 2022.

---

Published as a conference paper at ICLR 2026
Poly Haven. Poly haven. bttpp::/ppolhhaven.cco 2025. URL httph::tptoppoaaaaeeeee com/
Mingming Hc, Pascal Clausen, Ahmet Levent Taşel. Li Ma, Oliver Pilarski, Wenqi Xian, Laszlo Rikker, Xueming Yu, Ryan Burgert, Ning Yu, al. Diffrelight: Diffusion- based facial perfor- mance relighting. In SIGGRAPH Asia 2024 Conference Papers, pp. 1024.
Haian Jin, Yuan Li, Fujun Luan, Yuanbo Xiangli, Sai Bi, Kai Zhang, Zexiang Xu, Jin Sun, and Noah Snavely. Neural gaffer: Relighting any object via diffusion. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Peter Kocsis, Julien Philip. Kalyan Sunkavalli, Matthias Nießner, and Yannick Hold-Geoffroy. Lightit: Illumination modeling and control for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2359-9369, 2024a.
Peter Kocsis, Vincent Sitzmann, and Matthias Nießner. Intrinsic image diffusion for indoor single- view material estimation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 3198-5208 2024b.
Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chan- draker. Inverse rendering for complex indoor scenes: Shape, spatially-varying lighting and svbrdf from a single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2475-2484, 2020.
Ruofan Liang. Zan Gojeic, Huan Ling, Jacob Munkberg, Jon Hasselgren, Chih-Hao Lin, Jun Gao, Alexander Keller, Nandita Vijaykumar, Sanja Fidler, et al. Diffusion renderer: Neural inverse and forward rendering with video diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 26069-26080, 2025.
Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Mare Habermann, Christian Theobalt, et al. Wonder3d: Single image to 3d using cross cross-domain diffusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9970-9980, 2024.
Linjie Lyu, Ayush Tewari, Mare Habermann, Shunsuke Saito, Michael Zollhöfer, Thomas Leimkühler, and Christian Theobalt. Diffusion posterior illumination for ambiguity- in- verse rendering. ACM Transactions on Graphics (TOG), pp. 1-14, 2023.
Yiqun Mei, He Zhang, Xuaner Zhang, Jianming Zhang. Zhixin Shu, Yilin Wang, Zijun Wei, Shi Yan, HyunJoon Jung, and Vishal M Patel. Lightpainter: Interactive portrait relighting with freehand scribble. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 195-205, 2023.
OpenAl. Chatgpt-4o viisio language model], 2025. URL hhttts::// www.aat.ppnnac.ccm
Xingang Pan, Bo Dai, Ziwei Liu, Chen Change Loy, and Ping Luo. Do 2d gans know 3d shape? unsupervised 3d shape reconstruction from 2d image gans. arXiv preprint AXXi::001.00084/ 2020.
Rohit Pandey, Sergio Orts-Escolano, Chloe Legendre, Christian Haene, Sofien Bouaziz, Christoph Rhemann, Paul E Debevec, and Sean Ryan Fanello. Total relighting: learning to relight portraits for background replacement. ACM Trans. Graph., 2021.
Foivos Paraperas Papantoniou, Alexandros Lattas, Stylianos Moschoglou, and Stefanos Zafeiriou. Relightify: Relightable 3d faces from a single image via diffusion models. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 8806-881 2023.
Yohan Foirier-Ginted Alban Gauthier, Julien Phillip. J-F Lalonde, and George Drettakis. A diffusson approach to radiance field relighting using multi-ille mulii-lllmmination synthesis. In Computer Graphics Forum, pp. e15147, 2024.
Puntawat Ponglertnapakomn, Nontawat Tritrong, and Supasorn Suwajanakom. Difareli: Diffusion face relighting. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 22646-22657, 2023.
12

---

Published as a conference paper at ICLR 2026
Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter models. SC20: International Conference for High Perfor- mance Computing, Networking. Storage and Analysis, pp. 1-16. 2020.
Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System opti- mizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. -505- 2020.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF confer- on computer vision and pattern recognition, pp. 10684-10695, 2022.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022.
Daichi Tajima, Yoshihiro Kanamori, and Yuki Endo. Relighting humans in the wild: Monocular full-body human relighting with domain adaptation. In Computer Graphics Forum, pp. 205-216 2021.
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600-612, 2004.
Chong Zeng, Yue Dong, Pieter Peers, Youkang Kong, Hongzhi Wu, and Xin Tong. Dilightnet: Fine-grained lighting control for diffusion- image generation. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-12, 2024a.
Zheng Zeng, Valentin Deschaintre, Iliyan Georgiev, Yannick Hold-Geofiie Geoffryr Yiwei Hu, Fujun Luan, Ling-Qi Qi Yan, and Miloš Hašan. Rgb <> Image decomposition and synthesis using material- and lighting-awarr diffusion models. SIGGRAPH '24, pp. 1-11. ACM, 2024b.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Scaling in-the- whld training for diffusion baseed illumination harmonization and editing by imposing consistent light transport. In The Thirteenth International Conference on Learning Representations, 2025.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In CVPR, 2018.
Yuqing Zhang, Yuan Liu, Zhiyu Xie, Lei Yang, Zhongyuan Liu, Mengzhou Yang, Runze Zhang, Qilong Kou, Cheng Lin, Wenping Wang, et al. Dreammat: High-quality pbr material generation with geometry-am light-aware diffusion models. ACM Transactions on Graphics (TOG), pp- 1-18, 2024.
Xiaoming Zhao, Pratul Srinivasan, Dor Verbin, Keunhong Park, Ricardo Martin Brualla, and Philipp Henzler. Illuminerf: 3d relighting without inverse rendering. Advances in Neural Information Processing Systems, 37:42593-42611 2024.
Jingsen Zhu, Fujun Luan, Yuchi Huo, Zihao Lin, Zhihua Zhong, Dianbing Xi, Rui Wang, Hujun Bao, Jiaxiang Zheng, and Rui Tang. Learning-basedd inverse rendering of complex indoor scenes with differentiable monte carlo raytracing. In SIGGRAPH Asia 2022 Conference Pupers, pp. 1-8,
2022.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 1 DATASET CONSTRUCTION
To train a model capable of accurately decomposing an input RGB image, we need to collect objects and scenes that contain rich material information. At the object level, we filter objects from Obja- verse (Deitke et al. 2023b a selecting those with BRDF material properties. For scene data, while some existing datasets Koesis et al. (2024b); Zeng et al. (2024b) provide intrinsic components, they primarily focus on indoor scenes and lack object masks to exclude translucent or transparent objects, which can hinder accurate material estimation. Moreover, these datasets do render images under varying lighting conditions- a critical factor for our task. To address these limitations, we curate scene dataset from an online repository BlenderKit 2024) and render intrinsic components along with diverse lighting conditions for training. The following section details our dataset construction pipeline.
A.1.1 OBJECT DATA
For object-centric data, we randomly select over 10,000 objects from filtered Objaverse (Deitke al. 2023bba a) that contain BRDF materials and render them under varying viewpoints and lighting conditions. Each object is rendered with 10 different views and 10 different lighting conditions, resulting in a total of 100 images per object. The first four views are captured at 70% elevation angle from the front, back, left, and right, while the remaining views are randomly sampled.
The lighting direction is randomly sampled. The horizontal angle is uniformly distributed in [0 3.0%). while the vertical angle follows a normal distribution centered around 60 with a standard deviation of 40", clipped to the range [20"; 160"). This sampling strategy biases the light direction toward the upper hemisphere, reflecting a higher probability of natural top-down lighting. The sun- light strength is randomly sampled in the range [5, 20). with slight random variation, and is reduced when HDRI lighting is used. A small random ambient light is added to avoid extremely dark areas. In 80% of cases, a point light source is used, while the remaining 20% incorporate both point light source and an environment map. The environment map is randomly selected from a set of 700 maps sourced from Poly Haven (Haven 2025).
BlenderProc DDnninger et al. 2023) utilizes Blender's ' composition layer to generate intrinsic prop- erties. However, we observed that certain albedo values fluctuate with the light source, and for trans- parent or semi-transparent materials, the composition layer often outputs entirely white or black val- ues, which are inaccurate. To mitigate these issues, we extract all intrinsics directly from Blender's s Principled BRDF nodes while simultaneously generating mask to identify regions that are not rendered using the Principled BRDF model.
To effectively model the lighting of full image, we propose novel light representation different from all previous methods (Zhang et al. 2025 Bashkirova et al. 2023 Zeng et al. 2024a). Most
existing methods either condition on global aa HDRI lighting information (Zeng al. 2024a) infer lighting from the background to relight the foreground (Zhang et al., 2025 Bashkirova et al. 2023). In contrast, our method constructs the lighting condition using only the light sources directly facing the image, leveraging only the front hemisphere of a gray ball rendered from the HDRI. This repre- sentation effectively mitigates the influence of self-luminous objects and built-in light sources (e.g.. light from windows) present in the original scene.
Specifically, we position camera the same location as the input image and render a gray ball with 50% roughness in Blender under the given lighting condition. The rendered image is saved as an RGB image, E TR. Cx where the sphere is tangent to the image boundaries.
A.1.2 SCENE DATA
For scene data, we downloaded 400 scenes from BlenderKit (BlenderKit) 2024), consisting of 300 indoor and 100 outdoor environments. After manual filtering, we retained 300 scenes by removing those containing excessive special effects, such as fog and rain. The rendering process for these scenes is largely similar to that of object data, with slight modifications in view selection. Specifi- cally, we applied parallel translations within a range of [-0.5, 0.5] along the planar direction perpen- dicular the camera orientation, random forward backward translations within range of [-0.2,
14

---

Published as a conference paper at ICLR 2026
0.5] along the camera direction, and minor rotations within a 5 elevation angle. These adjustments prevent the camera from penetrating walls or moving outside the scene boundaries.
The lighting conditions scene illumination setup. The additional point light was placed behind the camera, with an angular deviation of [-30%, 30"] relative the camera's viewing direction. Notably, we observed that normal map rendering varies across different Blender versions. To ensure consistency, we filtered scenes suitable for training and manually specified the Blender version used for each scene.
generated by randomly adding point light source to the original
We construct the Object50 and Scene200 datasets using the same procedure as the training set con- struction. To prevent data leakage, the object IDs and scenes in the evaluation from those in the training set.
strictly disjoint
A.2 VALIDATING PHYSICAL CONSISTENCY IN RELIGITING
As shown in Fig. 9 when we modify the normal map by flipping the R channel, which corresponds inverting the X-coordinate in camera space, we observe that under the same lighting condition, the generated illumination and shadows change accordingly.
Normal1
Relighted result1
Normal2 (horizontal flip)
Relighted result2
Figure 9: Our stage-2 model successfully perceives changes in light direction caused by variations in surface normals.
A.3 ABLATION STUDIES ON THE WEIGITT OF CLASSIFIER-FRES GUIDANCE (CFG)
As shown in Tab. we analyze the impact of different CFG weights the results using the object test dataset. We observe that increasing the CFG value leads to outputs that are more closely aligned with the conditioning input. For instance, in the case of albedo estimation, a larger CFG encourages the model to preserve more details from the input image, inadvertently baking shading effects into the albedo, which degrades the quality of the results. To mitigate this issue, the first stage, that is, disable CFG, and selected a relatively small CFG value in the second stage to ensure that the generated results appear more physically accurate.
CFG 1.0 in
We provide qualitative comparisons of different CFG weights in Fig. 10 Consistent with our ob- servations the main paper, increasing the CFG weight causes the model produce intrinsics that deviate from physical properties, indicating a stronger resemblance to the condition image. As re- sult, higher CFG values lead to more shading baked into the albedo, while the normal maps become sharper but exhibit more artifacts.
Elemeet
Scale
PSNK
SSIXE
Albedo
EPIPS.
PSNN
SSIMI
LEIPS)
PSNK
Knopluess SSINF
LPIPST
PENK
SSIME
LPJPS:
PSNK
Aseeae
SSIAT
LPIPS]
1.5
22.19
0.3300
0.8502
24.97
(092LT
00031
2139
0.896]
0.0736
1396
0.33T.
0.1316
20.53
0.0761
CEG
1.5
20.56
21.42
0.9053
0.9153
0.06 16
0.0637
34.94
24/24
0.9207
0.933
0.000
00522
19.28
1805
0.8721
0,8855
0.094
0.881
13.54
13.42
03338
03834
0.1228
0.8895
010772
0.0155
0.3966
0.0691
24.55
00538
17.27
0.8645
0.816
0.341
0.1063
18.73
0,8783
00852
Table 4: Ablation studies of the influence of the Classifier Free Guidance (CFG).
15

---

Published as a conference paper at ICLR 2026
Albedo Estimation
Normal Estimation
Input
CFG=1.0
CFG-2.0
CFG=3.0
CFG=1.0
CFG=2.0
CFG=3.0
Figure 10: Qualitative comparisons on different CFG weights. Higher CFG values lead to more shading baked into the albedo,, while the normal maps become sharper but exhibit more artifacts.
A.4 ATTRIBUTION OF PERFORMANCE GAINS
As shown in Tab. 5 we reproduce RGB+X (Zeng al.) 2024b) following the training details in their paper and retrained RGB+X on the same dataset used by our method. Even after retraining RGB+*X on our dataset, our method consistently achieves better performance, demonstrating that the methodological design itself also contributes to the performance gain.
As shown in Tab, 6 we conducted the additional evaluation on a representative subset of our dataset to show that the advantage is not only gained from the dataset but indeed benefits from the design of our physics-ime ahysics-inspired module. Specifically, we retrained Neural Gaffer (Jin et al., 2024 on subset, which is part of our dataset with both the grey ball and environment maps rendered, for 80k iterations following the original protocol in paper. We also retrained our framework on the same subset for 30k iterations. Despite the shorter training schedule, our model still outperforms Neural Gaffer on this shared training set, demonstrating that the performance gains are attributable not only to the dataset but also to the effectiveness of our proposed architecture. This also verifies that our loss, as a regularization term, guides the model to converge more quickly toward physically plausible principles.
MESTE
RCBB
Zengal 2Ynil
30,99
09125
64737
23.75
0 6.090
0.00 0,053
1507
13336
19.33
0 5505
Oryaz00
0.8800
AL0903
BJ062
2459
03851
33.00
53861
13.6
0.834
10083
132-1
1430
198-3
14:14
0.100
1.30
10.8
62025
8.2286
8.770
14.6
BA3K
732
0.000
0.5666
11.343
Table 5: Quantitative comparisons inverse neural rendering task. Our method outperforms most of the metrics on both object and scene level test dataset.
Method
PSNRT
SSIM
LPIPS
Neural Gaffer (Jin et al. 2024
TO.31
0.8909
0.0937
Ours
11.74
0.9060
0.0637
Table 6: Quantitative comparisons of the forward neural rendering task.
A.5 LIGHT INVARIANCE OF THE RECONSTRUCTION LOss
Here, provide additional details and discussion regarding the reconstruction loss introduced in Sec. 4.3.2 of the main paper. We use the feature from the last hidden layer of the DINOv2-base model for reconstruction. This layer was selected empirically based on performance in experi- ments.
As shown in Fig 11 the features from this layer more sensitive to texture while being largely invariant to lighting. For example, as illustrated in the table on the right of Fig when the lighting conditions differ but the scene texture remains the same, the feature similarity remains above 97%.
16

---

Published as a conference paper at ICLR 2026
Imaue
Ingge I (light
Imge vlgiineci
21 Inge 1., cxciiie 2)
loage 4 41..g contikan
legadatin)
Figure 11: Light invariance of the DINO reconstruction loss.
Specular
Diffuse
Output
Specular
Diffuse
Output
Figure 12: More Explanation on the Physical-basee Shading Loss.
In contrast, when the lighting is held constant but the undergoes degradation (simulated by using a high CFG value), the similarity drops and the MSE loss increases significantly.
Therefore, leverage features from this layer to construct a light-invane ientriinvariant reconstruction loss that enforces scene consistency under varying lighting conditions and helps reduce artifacts.
A.6 MORE EXPLANATION ON THE CEYSICAL-BASEH SHADING LOSS
Without the Physical- based Shading Loss, when encountering highly out-of-domain data, the re- lighting output tends to remain consistent with the input and does not change in response to vari- ations in the diffuse and specular components, even the generated diffuse and specular maps are correct and vary with lighting conditions. However, after introducing the Physical-based Shading Loss, as shown in Fig. 12 the final output follows the diffuse and specular maps and becomes consistent with them.
A.7 MORE EXPLANATION ON THE SXTCH-AWARD SELF-ATTENTION
a) Self-attention
b) Cross-batch Self-attention
Figure 13: Illustration of standard self-attention and the cross-batch self-attention used in our method.
As shown in Fig. 13 our batch-aware self-attention enforces stronger consistency among albedo, normal, roughness, and metallic predictions. In the cross-batch self-attention layers, the queries, keys, and values are computed as follows:
qa = 8 ha = (a @ 2d
0 e "")) ) = W. + 2" @
Wy2", , kn WA(2" 20000020 Un = W.(2" @ 2 0 e 2²)
4. W = Wa(2" +
W.(:"
)m)
17

---

Published as a conference paper at ICLR 2026
9m W g Km (x + 2" 2" 3 22"),,, Um W.(2nM 2 2")
where, 29, ,", 2m are latent albedo, normal, roughness and metallic embeddings in transformer blocks. The operator @ denotes concatenation, and Wú: Wa and Wa are the projection matrices used to compute the query, key, and value embeddings. The cross-batch features are Att(qs, ks, vz), {a, n, m), where Att() denotes the standard softmax attention.
A.8 MORE RESULTS
A.8.1
DECOMPOSITION AND RELIGHTING
As shown in Fig. 25 26 27 and 28 provide more results for both of our inverse and forward ren- dering pipeline on real images. Our results output reasonable intrinsic components and controllable relighted images under different lighting conditions.
A.9 MORE COMPARISONS
A.9.1 QUALITATIVE ABLATIONS ON OUT-OF-DISTRIBUTEE IMAGE RELIGHTING
w/o hps
w/ Lps
w/o lps
w/ Lps
w/o Lps
w/ Lps
Figure 14: Qualitative comparisons out-of-distribution inputs, with and without the physics- inspired loss functions. Incorporating the physics-inspired loss produces more realistic and detailed results, demonstrating its contribution to improved generalization.
Here include qualitative comparisons out-of-distribution inputs, with and without the physics- inspired loss functions. As shown in Fig. 14 incorporating the physics-inspired loss produces more realistic and detailed results, demonstrating contribution improved generalization.
A.9.2 MORE QUALITATIVE COMPARISONS ON REAL IMAGE RELIGHTING
we provide additional qualitative comparisons on real image relighting. We compare our method with two widely used approaches, IC-Light (Zhang et al. 2025) and ChatGPT- 4o (OpenAl 2025). As observed in Fig. 15] both IC-Light and ChatGPT-44 struggle to preserve the original foreground albedo. Notably, the monkey appears aged, and the bicycle color is inconsistent with the input image. While ChatGPT-40 produces relatively natural results in the overall scene, fails to accurately control the lighting direction. For example, even when instructed to apply lighting from the right, it still illuminates from the left. This suggests that incorporating chain-of-the of-touht (CoT) reasoning might be necessary to guide and refine its output through iterative correction. In contrast, our method consistently produces relighting results under precisely controlled lighting con- ditions while effectively preserving the intrinsic properties of the original images.
As shown in Fig. 15
A.9.3 QUALITATIVE COMPARISONS ON SCENE IMAGE RELIGHTING
As shown in Fig. 16 we provide qualitative comparisons on scene image relighting with the state- of-the-art methods RGB ZZeg et al. 2024b), Neural Gffer (Jin et al. 2024), DiLLigtNet (Zeng) al. 2024a).
A.9.4 INTRINSIC DECOMPOSITION ON REAL IMAGE
As shown in Fig. and 18 provide qualitative comparisons with RGB+xX (Zeng et al. 2024b)
on real image intrinsic decomposition task. As shown in Fig. 23 and 24 we also provide extended versions of Fig. from the main paper.
18

---

Published as a conference paper at ICLR 2026
Left Light
Right Lighs
Left Light
Right Light
Input
IC-Light
ChaGPT-4o
Figure 15: Additional qualitative comparisons on real image relighting. Both IC-Liight and ChatGPT-4o struggle to preserve the original foreground albedo. and ChatGPT-40 fails to accurately follow the specified lighting conditions. In contrast, our method consistently produces relighting results under precisely controlled lighting directions while effectively preserving the intrinsic prop- erties the original images.
Lighting Condition
RGB-XX
Neural Gatter
DiLightNet
Ours
GT
Figure 16: Qualitative comparisons on scene image relighting.
19

---

Published as a conference paper at ICLR 2026
RGB e-x
Input
Ours
Albedo
Normal
Roughness
Metallic
Figure 17: Qualitative comparison with RGB+X real image. Our method produces more de- tailed and physically consistent intrinsics instance, our model successfully predicts the high metallicity of the iron basket the picture.
A.9.5 NORMAL ESTIMATION
As shown in Fig. 20] our method can generate more reasonable and detailed normal map even compared with the state-of-the-at normal estimation methods, DSINE (Bae & Davison 2024) and GeoWizard (Fu et al. 2024).
We also evaluated the predicted surface normals using the Mean Angular Error (MAE) metric. Quan- titative comparisons based on MAE are reported in Tab.
A.9.6 EXTENDED TEST DATASETS
We have expanded our test split from 50 objects/ 20 scenes 500 objects 36 scenes, that is, obj500 contains 500 objects resulting in 12000 images, and the updated quantitative results are reported in Tab.
A.10
ADDITIONAL SETUP FOR TRAINING.
Since the loss function in stage two requires loading the VAE decoder, which significantly increases memory consumption, we employed DeepSpeed (Rasley et al, 2020) with ZeRO (Rajbhandari al.) 2020) stage 2 and mixed precision set to 'bfl6' to optimize memory usage during training.
A.11
DISSCUSIONS
A.11.1 FRAMEWORK DISTINCTIONS FROM PRIOR METHODS
Our framework design differs from RGB ZZng et al. 2024b) and DiffusionnRederer (Liang ctt al., 2025). the inverse neural rendering stage, both prior methods generate one intrinsic com- ponent at time using prompt-based textual inputs, resulting in lower efficiency and limited consis- tency across outputs. In contrast, our method predicts multiple intrinsic components simultaneously. facilitated cross-batch self-attention mechanism that allows information exchange between dif- ferent intrinsics. Specifically, following Wonder31 Long et al. 2024). we extend standard self-
Object50
Scene200
Method
11.25
22.5 1
30"
11,25
22.5
30
Kocsis et al.
Kocsii et al. 2024b
34.8
60.7
72.4
59.4
74.2
80.7
Zhu et al.
(Zhu et al. 2022
10.6
31.3
44.7
43.1
60.1
68.6
RGB+*X +
Zeng et al.
2024b
19.5
42.5
53.8
44.5
60.7
67.2
Ours
36.0
64.0
74.1
52.1
68.3
75.8
Table Quantitative comparisons of normal estimation based on MAE metric.
20

---

Published as a conference paper at ICLR 2026
2558
PSNDE
1.21255
PONELT
LIPSS
13.55
0.117
01y-130
14:0
0.881
1063
13.35
3 1973
17.25
35534
011333
B3773
0,8912
15.00
00111
03335
23a m-
1.:0
0 16
16.33
1300
6.38
6447
11.00
0.3654
0.425
8.329
037
5700
033907
Table Quantitative comparisons on inverse neural rendering task. Our method outperforms most of the metrics both object and scene level test dataset.
RGB +X
Input
Ours
Albedo
Normal
Roughness
Metallic
Qualitative comparison with RGB+>X on real images. Our method produces more de- tailed and physically consistent intrinsics.
Figure 18:
attention layers to be global-aware, enabling communication across batches. This design improves both efficiency and consistency among predicted intrinsics.
In the neural forward rendering stage, the intrinsic components obtained from the first stage are fur- ther injected as conditions through cross-batch concatenation and the same cross-batch self-attention mechanism. Note that, our physics-inspired loss function is non-trivial. The physics- pyssccs-based loss serves as an efficient learning mechanism, which regularize the training dynamics into a physical plausible landscape, allowing the network to converge faster and learn correct light transport while training on less data and compute, compared to the two mentioned works. The ablation study pro- vided in Tab. B] of the main paper and the study provided in Sec. A.4 prove the effectiveness of our loss design. These design choices further differentiate our approach from the aforementioned methods.
A. LIMITATION
A.12.1 SPATIAL MISALIGNMENT BETWEEN LATENT AND RGB REPRESENTATIONS
We conducted diagnostic experiment where encoded an image into the VAE latent space, masked out part of the latent representation, and then decoded it back to the RGB space. If the latent space were strictly pixel-aligned with the RGB space, the unmasked regions should reconstruct cleanly, unaffected by the masked areas. However, as shown in Fig, [9] we observed that even within the unmasked rectangular region, the reconstruction showed color shifts and blurring artifacts. not only at the boundaries but across the entire retained region.
This suggests that the latent representation not strictly pixel-aligned: high-frequency information is not localized spatially in one-to-on correspondence with the input pixels. As result, masking operations in the latent space can inadvertently corrupt seemingly unrelated regions during decoding. This misalignment introduces subtle but systematic enror when computing masked losses in latent space, which can propagate into the final output during training. Although the resulting artifacts may not be highly visible, they nonetheless degrade overall performance. As mentioned in the main paper, we have not yet found a better alternative to mitigate this issue. This could be a limitation for further improvement.
21

---

Published as a conference paper at ICLR 2026
Original Image
Raconstruction with latent mask
Reconstruction with Iatent mask?
Figure 19: Latent-space masking diagnostic. From left to right: original image; decoded recon- struction after masking the VAE latent with Mask I; decoded reconstruction after masking the VAE latent with Mask 2; zoomed-in in reconstruction details.
A.12.2 LIMITATIONS ON IMAGE INTRINSIC DECOMPOSITION
Image intrinsic decomposition is ambiguous task with multiple possible solutions, making inac- curacies in the first-stage results inevitable. While our second-stage model has some tolerance for such errors, severe inaccuracies lead to distortions in the final rendering or deviations from the original image.
The predicted metallic maps may sometimes appear inconsistent with the ground-truth labels. For example, the wooden floor shown in the last row of Fig- in our scene is predicted as highly metallic despite being a non-metallic material. This behavior can be explained by the limitations of the current material representation.
In Blender's Principled BSDF model, in addition the primary attributes such albedo, normal, metallic, and roughness, there are many other factors that influence surface reflectance, such as sheen, coat, and the index of refraction (IOR). For example, non -etelell surfaces with high IOR values can still exhibit strong specular reflections. Since our model does not explicitly model these additional properties, their effects may be implicitly absorbed into the predicted metallic values. This does not imply that the model failed to learn the correct behavior; rather, it reflects the limitation that certain reflective factors, like IOR, are not part of the model's representation space. As shown in the last row in Fig. of our paper, the floor is predicted metallic, while in the original Blender scene its IOR is set to 1.450. Without modeling IOR explicitly, its reflective effect is captured by the metallic prediction.
A possible improvement is to extend the material representation to explicitly incorporate IOR or train explicit residual prediction enable broader material support without losing physical in- terpretability the other outputs, which may further enhance accuracy in distinguishing between different types of unformulated materials.
A.1 LIMITATIONS ON RELIGHTING
When wall is present behind the sides, the front-hemine front-hemisphere light sources may be obstructed, leading to results that do not align with the expected lighting conditions.
Under current setting and design, it is not possible modify add visible light emitters in the rear hemisphere of the environment light. mination conditioning is restricted to the frontal hemisphere and does not alter the lighting already present in the rear hemisphere. We view this as a more challenging setting and plan to explore further in future work.
out of our current scope. This because our illu-
A.13 BOARDER IMPACTS
Positive impact. This work presents a physically grounded relighting framework based intrinsic decomposition and environment-aware light modeling. The proposed method enables controllable and high-fidelity relighting of objects and scenes, which has potential applications in computer graphics, augmented/virtual reality (AR/VR), digital content creation, and scene editing. On the positive side, our technique may facilitate more realistic rendering and editing tools for creatives, improve lighting consistency in AR/VR applications, and support visual restoration or data augmen- tation tasks in computer vision.
22

---

Published as a conference paper at ICLR 2026
Potential negative impact. There exists potential risk of misuse, such as manipulating visual evidence or contributing to deceptive media (e.g., by relighting real-world scenes to imply different environments or time-of-day). To mitigate such risks, we plan to release model and dataset under a research-only license with appropriate usage guidelines.
A.14 THE USE OF LLM
Large Language Models (LLMs) were used solely for minor grammar correction and stylistic pol- ishing of the manuscript text. They were not involved in the design of the methodology, execution of experiments, analysis of results, or any other aspect the scientific contribution.
Input
DSINE
GeoWizard
Ours
Figure 20: Comparisons of the generalization ability with state-of-the-ee methods, DSINE Bac
& Davison (0044) and GeoWizard (Fu al. 2024) in normal estimation task on real images. Our method can generate more reasonable and detailed normal maps.
23

---

Published as a conference paper at ICLR 2026
Input
Zhu et al.
Kocsis et al.
RGB-X
Ours
Ground truth
"Part the black regions the ground truth represent masked areas, Indicating that the shader used these regions not based the Principled BRDF model and has therefore been masked. (top button: Albedo, Normal, Roughness, Matallio)
Figure 21: Qualitative comparisons on our Scene200 test dataset. From top to bottom, the results correspond to albedo, normal, roughness, and metallic comparisons. Our method produces more detailed and accurate results, even in specular reflection regions.
Input
Zhu et al.
Kocsis et.
RGB-X
Ours
Ground truth
*Part the black regions the ground truth represent masked areas, indicating that the shader used in these regions is not based on the Principled BRDF model and has therefore been masked. (top to button: Albedo, Normal, Roughness, Metallic)
Figure 22: Qualitative comparisons on our Scene200 test dataset. From top to bottom, the results correspond to albedo, normal, roughness, and metallic comparisons. Our method produces more detailed and accurate results, even in specular reflection regions.
24

---

Published as a conference paper at ICLR 2026
Input
Zhu at.
Kocsie et al.
RGB-X
Ours
Ground truth
*Part of the black regions in the ground truth represent masked areas, indicating that the shader used in these regions is not based on the Principted BRDF model and has therefore been masked. (top to button: Albedo, Normal, Roughness, Metallic)
Figure 23: Qualitative comparisons on our Scene200 test dataset. From top bottom, the results correspond to albedo, normal, roughness, and metallic comparisons. Our method produces more detailed and accurate results, even specular reflection regions.
Input
Zhu
Kocsis
RGB-X
Ground
black regions ground represent areas, indicating based the Principled BRDF model and has therefore been masked. (top button: Albedo, Normal, Roughness, Metallic)
regions
Figure 24: Qualitative comparisons on our Scene200 test dataset. From top to bottom, the results correspond to albedo, normal, roughness, and metallie comparisons. Our method produces more detailed and accurate results, even in specular reflection regions.
25

---

Published as a conference paper at ICLR 2026
Input Image
Image Intrinsic Decomposition (Inverse Rendering)
Shading Prediction (Forward Rendering)
Relighting Results (Forward Renderingl
Lighting Condition
Figure 25: More results showcase the performance of our whole pipeline. Our method successfully relights the glass and accurately predicts the specular highlights.
Input Image
Image Intrinsic Decomposition (Inverse Rendering)
Shading Prediction (Forward Rendering)
Relighting Results (Forward Rendering)
Lighting Condition
Figure 26: More results showcase the performance of our whole pipeline. Our method can also generalize to human relighting.
26

---

Published as a conference paper at ICLR 2026
Input Image
Image Intrinsic Decomposition (Inverse Rendering)
1
Shading Prediction (Fowaard Renderingg
Relighting Results (Forward Rendering)
Lighting Condition
Figure 27: More results showcase the performance of our whole pipeline. Our method successfully generates shadow projections on the tabletop.
Input Image
Image intrinsic decomposition (Inverse Rendering)
Shading Prediction (Forward Rendering)
Relighting Results (Forward Rendering)
Lighting Condition
Figure 28: More results showcase the performance of our full pipeline. Our method also generalizes well to indoor scenes with strong built-in light sources.
27


Reviews:
[{"review_id": "eexugFeKg2", "reviewer": "Reviewer_wQ6F", "summary": "This paper proposes PI-Light, a framework to relight images via the extraction of intrinsic properties of a scene and diffusion-based rendering for the final result: in the inverse rendering stage, surface normal and materials are estimated from a customized diffusion model; in the neural forward rendering stage, another diffusion model is used to synthesize the image under target lighting conditions. To facilitate training of the models, the authors constructed a new dataset featuring rendered data using objects from Objaverse. In the experiment section, the authors compare PI-Light with both intrinsic decomposition models and relighting methods. The proposed method shows reasonable performance.", "questions": "- Why is the full model performance in Table 3 (last row) different from Table 2 (last row)? If I understand it correctly, they should be the same model and tested on the same Object50 test set.\n\n- How does the proposed method work with HDR image-based environment maps? It seems by design the pipeline should not be limited to directional lights.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The motivation for having physics-inspired scene properties in the relighting pipeline is solid. With more and more frameworks becoming end-to-end, it is good to have a variety of approaches with competitive performance. In particular, this paper shows that it is possible to combine intrinsic decomposition with diffusion-based relighting for a good result. \n\n- The paper focuses on full image relighting, which is an under-explored direction in image relighting, although currently both the dataset and the experiment in this paper are still mainly object-centric. \n\n- The paper is easy to follow, and the design of the framework is well described. \n\n- The proposed dataset could be valuable to researchers studying both image relighting and intrinsic decomposition tasks.", "weaknesses": "- Intrinsic decomposition using neural networks is a well-studied problem. There is little new insight from this paper (Section 4.2). \n\n- It seems the relighting results shown in the paper are mostly directional lighting. In other relighting work, such as Neural Gaffer, HDR image-based environment maps are used as lighting conditions which provide more natural and complex lighting conditions. Since the proposed method is targeting full image relighting, single directional lighting could be a limitation. \n\n- Evaluation is done on 50 objects and 20 scenes. It might be better to try with more objects and scenes, as well as on other relighting datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LdytT0UTNg", "reviewer": "Reviewer_GfkN", "summary": "This paper tackles the problem of full-image relighting. To enable realistic relighting results, the authors propose to first estimate intrinsic properties, e.g., albedo, normal, and roughness, based on a diffusion model. Later, these intrinsic properties are fed into another diffusion model that produces a relit image as well as diffuse and specular maps. The inputs for these outputs are inspired by physically-based rendering. Specifically, one takes the source image and albedo; one takes normal, lighting, and mask; the other uses normal, lighting, metallic, roughness, and mask. Further, with a simplified physically based rendering equation, i.e., albedo $\\cdot$ diffuse + specular, the directly-produced relit image is compared against the physically-rendering-composed image. Experiments on various datasets verify the effectiveness.", "questions": "See \"Weakness\"", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- originality-wise: the idea of closely following physically-based rendering to construct the input for the diffusion model is interesting.\n- quality-wise: the qualitative and quantitative results are promising.\n- clarity-wise: the presentation is good in general but can be further improved.\n- significance-wise: the realistic relighting is important for various downstream applications, e.g., AR/VR.", "weaknesses": "## 1. Clarification on image intrinsics estimation\n\nCan authors clarify why the model in the paper works better than baselines, e.g., IntrinsicAnything in Tab.1? Is it because of carefully curated data or different model designs?\n\n## 2. About efficiency\n\n**For training**. As mentioned in L302:\n> Since physical laws lose their meaning when computed in the latent space, our physics-based losses for the relighting model are applied entirely in the RGB space, i.e., it is computed after the VAE decoder reconstructs the output\n\nThis makes the training extremely costly as each data in the training set needs to go through the full diffusion process to obtain the final latent. How to make the training affordable?\n\n**For inference**, can the author provide some runtime analysis for the inference?\n\n## 3. Presentation\n\n1. There is no lighting condition in Figure 3's \"Stage 2\"\n\n2. For Eq. (6) and (8), the input and output orders are not aligned. Either the authors can have separate equations, or the authors can make the order aligned. Currently, it isn't very clear. I do not know which input corresponds to each output.\n\n## 4. References\n\nMissing related works for generative relighting\n\n[a] Ginter et al., A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesis. EGSR 2024.\n\n[b] Zhao et al., IllumiNeRF 3D Relighting without Inverse Rendering. NeurIPS 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qACyfKpPgl", "reviewer": "Reviewer_Wtey", "summary": "This paper proposes a physics-inspired diffusion for full-image relighting that integrates light transport priors into neural forward rendering to achieve more precise relighting. Additionally, the authors introduce a new dataset comprising diverse objects and scenes.", "questions": "- About the batch-aware attention: Lines 237-238 state that “the standard self-attention layers are extended to be global-aware, enabling communication across batches. This design improves both efficiency and consistency among predicted intrinsics.” Can you provide a more intuitive explanation of why and how the self-attention can lead to such improvements in consistency and efficiency? And why can the observed performance gains be attributed to self-attention layers? Please provide more details on the implementation of batch-aware attention and how information is exchanged across batches.\n- The paper mentions that normal and roughness are fed into the frozen VAE encoder pretrained on RGB images to extract latent features. Given that the encoder was trained on RGB data, have the authors considered the potential domain gap between the intrinsic maps (e.g., normal and roughness) and the RGB domain? How can you ensure the extracted features are practical?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "+ Experimental performance: The proposed method demonstrates superior results compared to existing methods in terms of consistency, maintaining, and rendering quality.\n+ Clarity and readability: The manuscript is well-structured and clearly written, making the methodology and contributions easy to follow. \n+ Attribution: This paper provides a physical perspective to solve the relighting problem, and a meaningful dataset is collected.", "weaknesses": "- The authors claim that existing methods struggle to generalize to out-of-distribution data (Line 51). It remains unclear how the proposed approach addresses this challenge. The authors are encouraged to elaborate on the mechanisms or design choices that contribute to improved generalization beyond the training distribution.\n- As shown in Fig. 2, the proposed method appears to misinterpret the light reflections of “moonlight scattered on the sea surface” in the background painting and treats it as the light source. The authors may improve albedo estimation accuracy by incorporating a deeper understanding of scene semantics under such challenging conditions.\n- The paper should provide more details regarding the trainable components of the proposed network. In particular, it remains unclear which layers (e.g., convolutional layers in Fig. 3) are trainable and what their specific architectures or configurations are.\n- Comparison with $RGB \\leftrightarrow X$: Since the $RGB \\leftrightarrow X$ was trained only on indoor scenes, it results in limited generalization performance. A performance comparison with a version of $RGB \\leftrightarrow X$ trained on the same datasets as the proposed method is needed. This would offer a fairer evaluation of the proposed approach’s advantages.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)