IntrinsiX: High-Quality PBR Generation using Image Priors
Authors: Peter Kocsis
Lukas Höllein
Matthias Nießner
Venue: NeurIPS 2025 poster ()
Keywords: Image Synthesis
Diffusion
Intrinsic Images
Rendering
Texturing
Deep Learning
Primary area: applications
TL;DR: We introduce a text-to-intrinsic (albedo, rough, metallic, normal) model, which can be used rerendered under arbitrary lighting condition or even distilled into room-scale PBR textures.

Abstract: We introduce IntrinsiX, a novel method that generates high-quality intrinsic images from text description.
In contrast to existing text-to-image models whose outputs contain baked-in scene lighting, our approach predicts physically-based rendering (PBR) maps.
This enables the generated outputs to be used for content creation scenarios in core graphics applications that facilitate re-lighting, editing, and texture generation tasks. 
In order to train our generator, we exploit strong image priors, and pre-train separate models for each PBR material component (albedo, roughness, metallic, normals).
We then align these models with a new cross-intrinsic attention formulation that concatenates key and value features in a consistent fashion. 
This allows us to exchange information between each output modality and to obtain semantically coherent PBR predictions.
To ground each intrinsic component, we propose a rendering loss which provides image-space signals to constrain the model, thus facilitating sharp details also in the output BRDF properties. 
Our results demonstrate detailed intrinsic generation with strong generalization capabilities that outperforms existing intrinsic image decomposition methods used with generated images by a significant margin.
Finally, we show a series of applications, including re-lighting, editing, and for the first time text-conditioned room-scale PBR texture generation.
We will release the code and the pre-trained model weights.

IntrinsiX: High-Quality PBR Generation using Image Priors
Peter Kocsis
Lukas Höllein
Matthias Nießner
Technical University of Munich
peter- kocsis github..h..... so/IntrinsiX/
'An astronaed riding
unicom on mooe.
Albréo
Bough
Beadering
Brlighting
Naterial Bilting
Figure 1: IntrinsiX. We present a text-guided intrinsic image generator. Given text prompt, our method produces high-quality albedo, roughness, metallic, and normal maps which can rerendered under any lighting conditions. Our model enables downstream applications, such as relightable object or scene generation, and material or lighting editing.
Abstract
We introduce IntrinsiX, a novel method that generates high-quality intrinsic images from text description. In contrast to existing text-to-image models whose outputs contain baked-in scene lighting, our approach predicts physically-based rendering (PBR) maps. This enables the generated outputs to be used for content creation scenarios in core graphics applications that facilitate re-lighting, editing, and texture generation tasks. In order to train generator, exploit strong image priors, and pre-train separate models for each PBR material component (albedo, roughness, metallic, normals). We then align these models with new cross-intrinsic attention formulation that concatenates key and value features in consistent fashion. This allows us exchange information between each output modality and obtain semantically coherent PBR predictions. To ground each intrinsic component, we propose rendering loss which provides image-span signals to constrain the model, thus facilitating sharp details also in the output BRDF properties. Our results demonstrate detailed intrinsic generation with strong generalization capabilities that outperforms existing intrinsic image decomposition methods used with generated images by significant margin. Finally, we show series of applications, including re-lighting, editing, and for the first time text-conditioned room-scale PBR texture generation. We will release the code and the pre-trained model weights.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Text-to-image (T2I) models have revolutionized 2D content creation, by generating high-quality RGB images from just text description [50] 48). They are used in widespread upplications, including extensions for controllable generation beyond text [74 44]. personalization and stylization of
generated images [52 25], and 3D asset or scene generation 46 7 22 However, in all cases the
content is typically generated in shaded RGB space, that contains baked-in lighting effects (e.g.. reflections, shadows, specular highlights). This limits the usability of T2I models for many content creation scenarios such as gaming or VR applications, that requires PBR maps (albedo, roughness, metallic, normal) to render or relight scenes realistically.
Existing methods perform intrinsic image decomposition on RGB images 776 31 72 5]. However, finding the correct decomposition to a given input image is constrained task, usually causing over-smoothed or simplified predictions on out- out-of-domain samples. These methods are trained with synthetic conditioning input 766 33], leading to low-quality decompositions for of-diiiiittiitnn
inputs, limiting their effectiveness on diverse real-world images. Similarly, methods that generate 3D)
PBR content from T2I models [58] 61 47 26
are trained on object-scale datasets 10 9
making
them unsuitable for large-scale 3D scenes.
We take different approach for PBR map generation. For the first time, we directly generate PBR maps from text as input in a probabilistic diffusion process. Since our method does not rely on an input image, it is more self-contained,, enabling better generalization capabilities. We can use the generated PBR maps for downstream tasks, such as physically- based rendering, relighting, or material editing (Figure []. We also showcase that our method can generate PBR textures for entire 3D scenes, for the first time to the best of our knowledge, making it directly usable for gaming/VR applications (Figure 5). Our method leverages the strong image prior of pretrained T21 models and converts it into a PBR map generator. This way, our model can generate PBR content from diverse, distrriiittiitttinn text prompts, similar to existing T2I models that operate in RGB space. Concretely, we first train intrinsic priors for each material property and for normal map generation separately (Section(3.) 3.I.. We leverage small, curated datasets and the established LoRA (25) extension for T2I models. Then, we fine-tune all priors jointly by employing cross-intrinsic attention in the diffusion transformer network (Section 3.2). This allows intrinsic properties to interact, enabling their joint and coherent generation. We also introduce rendering objective with importance based lighting sampling to ground the intrinsic components. This image-space signal encourages sharp and semantically meaningful decompositions, In summary, our contributions:
We introduce the first method, that directly generates PBR images from text as input in a proba- bilistic diffusion process. In comparison to baselines, our PBR maps are of higher quality and can be used for various downstream tasks, including physically-basee rendering, editing/relighting, and room-scale 3D scene PBR texturing.
We decompose the strong image prior of pretrained T2I models into intrinsic components in two-stage training process. This allows us to generate PBR maps from diverse text prompts, that are not limited to the distribution of existing, synthetic datasets.
We combine cross-intrinsic attention with novel rendering objective using importance-based light sampling to jointly generate semantically coherent PBR maps.
2 Related Work
Text-to-Image Models Text-to-image (T2I) models have emerged as powerful tools for 2D content
creation; they create high-quality, diverse images from only text as input [50) [4| 48) Since their
inception, several models further increased the visual quality of generated images [45 32 67 73
These models are trained on datasets consisting of billions of images, like [55]. This makes them strong 2D prior for arbitrary content generation. They typically model the diffusion process following Ho et al. [21 or Lipman et al. (37] with U-Net [51] or diffusion transformer (DiT)
41 62] architectures. Many downstream applications leverage T2I models, including controllable content generation [74] (7) 44 30 [66 as well as personalization and stylization of generated images We leverage pretrained T21 models as prior for our task, the generation of PBR maps
[2 25 63 59
from text.

---

Task-specific Finetuning of Text-to-Image Models In order to use T2I models for downstream tasks, different modifications to the model architecture exist and can be applied 774 44
38 30
In particular, LoRA layers [25] can be used to teach T21 models about specific "styles" (e.g., artistic paintings). Additional low-rank linear layers are trained in every attention block, which keeps the generalized prior of the T21 model, while finetuning on smaller-scale datasets.
We similarly finetune multiple LoRAS to teach a T2I model about the distribution of intrinsic images.
Other tasks generate multi-view image outputs, such as video generation [65 70] multi-view image generation 23 39 60) or multi-modal generation [68] They augment the attention operation in the transformer architecture to jointly process multiple images in a batch with the same model. Related to these tasks, we perform cross- coos-intrinnsic attention to generate aligned PBR maps in a single denoising forward pass with our finetuned model.
T21 models are also applied to 3D tasks, like object generation 66] or scene generation 22 7 Some methods finetune T2I models on synthetic 3D objects datasets, like 10 to generate object-scale 3D assets 33
58 13 In contrast, we utilize score distillation 46 18
generate PBR textures of
entire 3D scenes following Chen et al.
Material Reconstruction
Completely decoupling lighting from material properties requires multi- ple surface observations under difffrrnnt lighting conditions. Pret-trained models can enable material acquisition from sparse observations. [T] uses a feed-forward model to predict the texture of single material and similarly use lighting sampling with the reflected view direction. Orthogonal to this, we aim to generate the PBR properties of complex scenes in image space to enable downstream applications. We introduce rendering loss to the diffusion framework using a Noughness-weightee importance sampling for the lighting direction.
The field of intrinsic image decomposition focuses on obtaining PBR maps from a single RGB image. Early approaches focus on separating the reflectance from shading 33 24 heuristics, such as sparsity in reflectance properties [14 57
64 using various
17 75,, or smoothness 2 Later, deep- learning methods [2 31 72 61 36) train decomposition networks on synthetic datasets, such as (76). However, the decomposition of an RGB image into its intrinsic properties is a constrained task, making it hard to generalize to -dist--diitiiuution input images. In contrast, we directly generate all PBR components from text input. This drastically improves the performance in thel settings.
Material Generation Recent works use diffusion models for single material generation, condi- tioned on text or image inputs [40 42] The work of [69] concurrently addresses text-conditional complex PBR generation. They train a shared ControlNet [74] and use a diffusion renderer for editability. In contrast, our method first learns prior the PBR properties independently. then aligns them with cross-intrinsic attention using fixed renderer to ensure compatibility with standard rendering engines.
3 Method
Our method generates the intrinsic properties of an image given a text prompt as input (Figure ] top). Specifically, leverage the strong prior of a pretrained text-to-image model and turn it into a PBR map generator. First, we learn the distribution of intrinsic properties (albedo, roughness, metallic, normal) by finetuning LoRA layers on each modality separately (Section 3.1). Then, we learn the joint distribution by leveraging cross-intrinsic attention and by minimizing a novel rendering objective (Section 3.2). Our method generates multiple images corresponding the different PBR maps, allowing for various downstream applications (Figure Figure 5). We summarize our method in Figure
3.1 PBR Prior Training
In order to generate PBR maps of an image, we model the distribution of the individual intrinsic image properties. Specifically, we model the probability distribution po (Xp) over data XXo). where (,=(x, 3XP X, X.. ER. ERR'P)) P:=H x W shorthand for the image size, m, n refer to the albedo, roughness, metallic, and normal intrinsic properties. and the suffixes a, m,m, respectively. In other words, we learn the join! probability distribution of all intrinsie properties through the parameters 0 of neural network.
3

---

I'aupl
Inl Inaiuns
Triting:
Inforare
An idon iccniion the noon"
Vibingal a
Aladda bitt
lveile
Ronehh 'Ketal Lati
Dracesn
Noiss
Intrinsix o
Normal LaRk
VUE
I&T llrkk
Intrinsix
Nibsite
Iecendriing las
Light Samping
L
$225
Demvkeing
Irakring
UN
Figure 2: Method Overview. We generate the intrinsic properties of an image given text input. Left: we train different LoRAs for a pretrained, latent text-to-image model, corresponding to the intrinsic properties (albedo, normal, and roughness metallic) curated synthetic datasets (Section (.T). We facilitate communication between all 4 modalities through cross-intrinsic attention to predict PBR maps corresponding to the same image (Section B.2.1. A novel rendering loss using importance-bast based light sampling ensures that we can render high-quality RGB images from physically realistic PBR maps (Section 3.2.2). Right: after training. we jointly denoise and decode all 4 PBR maps and can prompt our model with diverse, out-of-distribution descriptions.
Unfortunately, existing datasets, such as Openrooms
35 InteriorVerse 766 or Hypersim 49
contain either only synthetic examples of intrinsic decompositions or are limited in size. Thus, models trained on such datasets exhibit limited generalization to arbitrary, real-world examples. On the other side, recent text-to-image diffusion models [50] 45, 67) are able to generate high-quality and diverse image samples. These models learn the probability distribution Px(xo c)== J Po Px(oorrl |c c))xx:r where is text condition, Xy Xx 9rgb 9rgb(Xo) is sampled from billions of RGB images [55] and the latent variables X1,--11,- Xy gradually add more Gaussian noise to the data, following (21). We leverage this strong image prior by turning pretrained diffusion models into PBR map generators.
In the first stage, we model the intrinsic image properties separately. That is, we learn Pa.e, (xa) and Po.a., (xn) corresponding to the albedo and normal maps, respectively. Since roughness and metallic are both 1-channel properties, we concatenate them together with an additional O-channel and learn Po.0.... Xm.) This concatenation makes samples compatible with the VAE, similarly in 31. Here,
the pretrained weights of the Flux. 1-ue. model 32 and are the parameters of LoRA layers [25] injected into the MLP layers before and after the attention (to h. to_k, to__, to out) module of all DiT blocks of the diffusion transformer model architecture [41] This an established way to teach large text-to-image models about new concepts (e.g.. our PBR map distribution), while retaining the ability to generate diverse samples (19]. To this end, we curate paired dataset of prompts and intrinsic properties and train the LoRA layers, while keeping the rest of the pretrained model frozen. Precisely, we minimize the conditional flow matching loss [37]
(CF = 1E4-20001). (215 1) (x.;
where x, xx((x,,, z, tte the noisy data at timestep t, u.=c-x, the ground-truth vector field, and ú X, its network prediction.
Dataset for albedo and normals Thanks to utilizing pre-trained image prior, our method does not require extensive PBR datasets, which are generally not available. We collect as little as 20 synthetic examples of albedo and normal maps from the Interior dataset (76). Then, generate captions for each image with the Florence-2 model [66] using the respective rgb renderings. We train the LoRAs 0. and 9, on these text-image pairs and obtain high-quality results for diverse, out-of- dittr--distribution prompts. This follows previous works, in which text-to-image models learn a
new "style" of generated images given only few example images 63 59
53
We refer to the
supplementary material for more details.

---

Dataset for roughness and metallic Similarly, we collect and caption samples for roughness and metallic properties. However, we observe that training on small dataset does not teach the model intricate details about the distribution of these PBR maps. We hypothesize that this is because the data distribution of roughness/metallie is drastically different from RGB images and therefore requires more observations to learn. To this end, we curate large dataset of 20K roughness/metallic samples using the Interior dataset [76.. The resulting LoRA 0r.m exhibits worse generalization capabilities than Og and 0z, i.e., it overfits to the indoor scene setup. However, in Section 3.2 we show how we can still turn Or.m into a generalized PBR generator by combining with 0g and On.
3.2 PBR Prior Alignment
After training the LoRAs separately in the first stage, we finetune all LoRA parameters together to learn the joint distribution Po.00.0..00..f. (Xo). At inference time, this allows us sample aligned PBR maps across all modalities. First, we replace self-attention with cross-intrinsic attention in every DiT block to facilitate communication between the different PBR maps. Second, we propose novel rendering objective that uses all generated PBR maps to create an RGB output image. In the following, we detail both components.
3.2.1 Cross-Intrinsic Attention
we leverage cross-attention in the DiT blocks facilitate communication between batch elements. We employ batch-size of 3 and use one of the intrinsic LoRAs from the first stage training for each of the images, while sharing weights for all the other parts of the model. We denote q k va as the query, key, and value features of the i-th DiT' block for the batch element corresponding to the albedo image and similarly for the other intrinsic properties. Then, we calculate cross-intrins. attention as:
Inspired by multi-view diffusion methods 39
23
60
[6
4% [K r.m Kn, k.
h softmax(
Iv no v
vd
where [.-]] denotes concatenation along the sequence dimension and we omit the text feature for clarity. We similarly calculate hi.m and hi, Finetuning all LoRA layers jointly with cross-ing cross-intrinsic attention allows us generate aligned PBR maps of the same image.
Additionally, we employ dropout regularization to preserve the learned prior of the intrinsic LoRAs. That is, with probability pt 0.25 we calculate regular self-attention instead of cross-intrinsic attention in the i-th DiT block during training. We show in Figure that this yields PBR maps of higher quality with sharper details.
3.2.2 RGB Rendering Loss
W
WE
Cross-intrinsic attention allows us to generate aligned PBR maps of the same image. However, individual intrinsics can still be of low quality (see Figure 8). This is because all LoRAs are finetuned jointly, which encourages similar fea- ture distributions during attention, i.e., the dif- ferences between the PBR maps are "averaged out". To this end, we incorporate novel ren- dering loss in the finetuning stage. Its goal is to provide semantic guidance to the intrinsic properties, that is, it teaches how the PBR maps combined, encouraging their distinct feature distributions.
Inneers
Narmal
Bongh
Wright
Figure 3: Importance- Light Sampling. We render RGB images (bottom) from our generated PBR maps and sampled light source as input (top). We employ multinomial importance sam- pling based using the inverse roughness to select less rough pixels more often (red squares). The light direction is then the viewing direction to the pixel reflected by its normal. The rendered images thus contain more specular effects, which provides better gradients during training.
Concretely, we render an RGB image from the predicted PBR maps. First, we obtain the clean data samples as - 10:2,--ü((2 Li(2;; 1), where de- notes the batched data of all PBR maps. Then, decode them from the latent space with the
5

---

315
Relightngg
Albedo Edit
Specular Edit
Figure 4: Editable Image Generation. Our generated PBR maps can be edited and utilized in standard physically-based rendering frameworks produce RGB renderings. Here, we place a light source top of the scene at constant elevation and rotate it around the vertical axis. From top to bottom we show, (1): RGB renderings with different light source positions; (2): manual edit of the albedo (desaturate the moon color); (4): lower roughness and higher metallic value (more glossy, mirror- like reflections).
VAE to obtain Xg. We use the simplified Disney BRDF model 4 and interpret X, the screen- space buffers of albedo, roughness, metallic, and normal properties. Assuming single directional light source, we can use deferred shading to obtain an RGB image as:
= f(wo: Xo) L, (#
where f is the BRDF evaluation value, wo the viewing direction, and {bi wi} the intensity and direction of single light source. We determine the viewing direction wo using the camera intrinsics of the dataset (we find this still yields good results during inference). Similarly, we obtain the ground-truth RGB image by using the same light, but the PBR maps of the dataset. Then, calculate the rendering loss:
Ligh (i.1)
= - 111 + 0.1 LPIPS(
(4)
where LPIPS denotes the perceptual loss (28)
We require light samples [L, } to render RGB images, following Equation 3 In practice, we sample single directional light source per image and always use constant intensity L,= We employ importance sampling to obtain the direction of the light w; (see Figure 3). That is, we invert the generated roughness x, (0, 1] and use the weights for multinomial sampling of pixel the image. Thus, pixels with lower roughness are selected more often. Then, we obtain the light direction as the reflected view direction w;=28,, Xx, wo) -w,: where wo the viewing direction and &,, the normal vector corresponding to the sampled pixel. This way, we produce RGB images that contain specular highlights and therefore we obtain better gradients for the roughness and metallic LoRAs. This helps to increase the quality of those PBR maps (Figure {).
During the second finetuning stage, we sample directional light sources in every iteration and render a separate RGB image with each of them. The final loss then becomes C= E Light (1,, I,). We do not backpropagate Ligh to the parameters O, of the normal LoRA, we find it stabilizes the rendering quality by avoiding ambiguities between material and geometry.
Applications
Since directly generate PBR maps, utilize standard computer graphics pipelines for physically- rendering to produce RGB renderings. This allows for various downstream tasks.

---

View
View 2
Input Scene Mesh
Displaced Normal
Alhedo
R/M
Rendering
Figure 5: Scene Texturing. We can use our method for scene texturing using score distillation 77. Given a scene geometry, first, condition method the rendered normal maps to produce the remaining PBR maps. Through iterative optimization, we obtain realistic PBR textures for the whole scene. Then, similarly optimize for normal map textures to obtain fine geometric details, conditioned on rendered material maps. This showcases the potential of direct PBR map generation democratize scene texturing from only text as input.
Editable Image Generation We select directional light source during rendering of an RGB image from our PBR maps (see Equation (3)). Since our model produces PBR maps, we can vary the direction of the light source arbitrarily and render them under numerous lighting conditions. Similarly, we can manually edit the individual PBR maps, e.g., by changing the albedo color of individual objects or by making them
specular. We show two examples in Figure and Figure Note that our PBR maps are not restricted to a single lighting direction. This can enable artists to precisely tune the appearance of our generated images their individual needs and therefore make the generations more useful for practical applications.
PBR Scene Texturing We use our method perform 3D scene PBR texturing (Figure 5 Recently, pretrained text-to-image models have been used as prior to distill information in 3D We apply the SceneTex approach 77), but use our fine-tuned PBR model instead of an RGB model. This enables us to distill uv-textures for given geometry corresponding to the individual intrinsic components. We can then render, relight, and edit an entire 3D scene according to physically-based rendering frameworks (see Figure This shows the potential of direct PBR map generation using Al-generated environments for games or VR applications.
46.
58 7.
Scene'Tex 77 requires a conditional generator. To this end, we finetune our model for 4K iterations after the first stage as described in Section 3.2 Additionally, we randomly (with probability p=0.25) set one of the PBR maps to the ground truth and the corresponding timestep to t=0. This enables our model be conditioned on any PBR input, similar
23 We render normal maps of the geometry from different viewpoints to condition our PBR generation. In the first stage, we optimize for the material properties, conditioned on the rendered normal. Since our model is based on Flow Matching 32]. we also modify the distillation in Scene Tex (7]. Concretely, we use the VFDS loss [18 in image space. To avoid over-smoothed results, we use CFG=10 and normalize the flow direction. We backpropagate to separate uv-textures for each property and follow the weighting scheme of 27). We weight the loss with the observation frequency and represent the textures with a regularized multi-resolution Laplacian-pyramae
to stabilize the updates for sparsely observed regions. In the second stage, we similarly optimize normal textures for fine geometric details, conditioned on the already obtained material properties. We represent the normal map in tangent space and regularize with the original geometry. For more samples and details, see the supplement.
Experiments
Training and Testing Details the first stage, we train the LoRAs separately for 2K iterations with batch size of 10, which takes 5h on single NVIDIA A100 (80GB) GPU. In the second stage, we finetune for another 2.5K iterations with batch size of 30 (10 aligned PBR maps), which takes 4 in both stages. The LoRA layers use rank of 64, which gives total of 224M additional parameters. For inference, we use a single NVIDIA A6000 (48GB) GPU. Sampling single image takes around 12 seconds.
21h. We employ the Prodigy optimizer
7

---

ITD 3
RGBX
Normal
Material
Lighting
Lighting
Normal
Material
Lighting
Lighting 2
Figure 6: Rendering comparisons. We show sample PBR maps of our method and baselines as well as rendered RGB images under two different lighting conditions. We use a diverse set of text prompts to produce our PBR maps, well as the imput RGB images for the baseline methods. This highlights our models' capability to retain the generalized prior of the pretrained text-to-image model. Our method better captures the semantic meaning of the individual intrinsic properties. For example, there are no baked-in lighting effects in the albedo, and the metallic/roughnnss. maps are sharper with more intricate details. This leads more realistic renderings and lighting effects.
Table Baseline comparisons. We compare the albedo quality for in-distribution (A-ID-FID) and ditt----dittibttiinn (A-OOD-FID) settings well perceptually with study (A-PQ). We evaluate the material quality with a user study focusing on the rendering quality (R-PQ), specularity quality (S-PQ), and prompt coherence (PC). Our method produces the best quality and it is preferred by most of the participants.
A-ID-FID A-OOD-FID4
A-PQ1
R-PQ1
S-PQ1
PCT
78.77
98.77
14.24%
2.95
2.82+1.13
4.47
RGBX 72
61.36
90.12
15.63%
2.961098
2.57
4.332003
ColorfulShading
91.10
86.48
2.77%
N/A
N/A
N/A
IID 31 w/ FLUX-LORA
103.36
79.29
N/A
N/A
N/A
N/A
w/o Rendering
78.77
72.23
N/A
3.422mm2
2.73
4.522008
w/o CIA-Dropout
71.47
75.54
N/A
3.68
3.21 2118 4.52
Ours
72.09
71.39
67.36 %
3.93 3.62 4.62-2077
Rendering Images During inference, we render RGB images following Equation (3) to obtain use slightly higher lighting intensity (L,=e)) than during training. Then we add an ambient color term: Lamb = (1-a)I ax, with a=0.2. Afterwards, we apply the tone mapping from [29]: + loge==
+ with 64 We empirically find this creates visually more pleasing RGB images. This also demonstrates the advantages of generating intrinsic image properties, i.e., can arbitrarily render them post-generation. We list the used text prompts in the supplemental.
the first method perform direct PBR map generation (from only text as input). Therefore, we compare our method against recent methods that perform intrinsic image decomposition, namely IID [1I, RGBX [72), and ColorfulShading (5]. In contrast to our method, these works require an RGB image as input from which the PBR maps are generated. Unless noted otherwise, we generate the RGB image for baselines by prompting our pretrained text-to-image model 32) We only compare albedo quality against ColorfudShading (5), since they are decomposing an image into albedo and shading components, which does not allow for complete relighting (including specular effects) editing effects.
Baselines To the best of our knowledge,

---

Metries We measure the quality of generated PBR maps through various metrics. First, we calculate the FID score [20] on an-disribbttinn and out-of-distribution albedo images. For in-distribution (A- ID-FID), we use all 2595 albedo images from the ViterioVeerse [76] test set and caption them based on the corresponding renderings with Florence-2 [66]. For each caption, we generate an albedo image, creating a total of 2595 generated albedo images. For out-of-distributte
(A-OOD-FID), we evaluate on the pre-processed G-Buffer renderings [47] of Obja Verse [I0] (GObja Verse). We take 1000 samples from the diverse "Daily-Used" category. As before, we generate an albedo map for each of the prompts, creating a total of 1000 generated albedo images. In both cases, we calculate FID against the respective ground ground-truth albedos.
Evaluating generated PBR maps remains a hard problem. To this end, we also conduct a user study and ask to rate the quality of albedo (A-PQ), specularity (S-PQ), rendered images (R-PQ), and the prompt coherence (PC). total, we collect 2,274 data points from 36 participants and report averaged results (we refer to the supplementary material for more details).
5.1 Intrinsic Image Generation
We show qualitative comparisons against IID 31. RGBX (72 and (5) in Figure 6 uuiing text prompts from [6], LLM-ggnerated ones, and our own prompts. The baselines receive an RGB image as input, which was created with our pretrained text-to-image model, whereas we directly generate the PBR maps from only text as input. For a fair comparison, we also
IID
RGBX (3
Colertul Shading 1
train another variant of IID on the same small dataset and same architecture as ours, i.e. we use LoRA fine-tuning of FLUX [32]. All methods showcase similar diversity, i.e., the generated images align well with the out-of-distribution text prompts. This showcases that finetuned model still retains the generalized prior, which is also confirmed in the user study (Table I PC). Furthermore, our generated PBR maps are of higher quality, semantically more meaningful, and they closer resemble the expected distribu- tion for physically- based rendering. This is be-
Figure 7: Albedo comparisons. We show albedo images of our method and baselines correspond- ing the same text prompt in each column. Our albedo images have less baked-in shadows and re- flections, which is desirable for downstream tasks, such as physically-basee rendering. We provide more samples in the supplemental.
cause the baseline methods are trained on synthetic, indoor scenes 76 and are not designed to generalize their decomposition tion is constrained to match the appearance of the input image, making it difficult to rather focus on the PBR distribution for out-of-domain samples. Additional albedo comparisons in Figure as well as the quantitative comparisons in Table [ confirm this observation. Our generated albedos are not oversmoothed, showing sharp details with flat colors. We provide more samples in the supplemental.
out-of-distribution setups. Furthermore, intrinsic image decomposi-
5.2 Ablations
The main technical contributions of our method are the cross-intrinss attention (Section 3.2.1) and the rendering loss (Section 3.2.22. In the following, we highlight the importance of each component. We provide additional ablations in our supplementary material.
How important is the rendering loss? The rendering loss improves the quality of all PBR maps (see Figure and T.. The additional supervision of Equation (4) provides more diverse gradients to the LoRA weights than the L2 loss of Equation (I). This way, the influence of the loss the individual PBR maps is different and becomes grounded in image space through the rendering function, Equation (3). This leads to a better separation of the intrinsic properties, giving meaningful normal maps, detailed albedos without baked-in lighting effects, and sharper roughness/metallic maps without undesired texture lighting patterns. Our importance-based light sampling strategy further improves the sharpness of roughness and metallic maps. In comparison, sampling light directions uniformly renders specular effects less often. This results in less pronounced PBR maps in Figure g

---

How important is the dropout in Cross- Intrinsic Attention? Without cross-intrinsic attention, we cannot create aligned PBR maps, because then there no communication be- tween batch elements during inference (see sup- plementary material), Additionally, we utilize dropout regularization tention. This technique motivates the model to preserve the prior of each intrinsic component during the 2nd stage alignment training. As can be seen in Figure 8 and Table [T] this increases the quality of both the rendered images and the PBR maps. The generated samples are sharper and do not suffer from noisy artifacts.
Normal
Albedo
R/M Lighting
Lighting 2
wiO W Dra.Dr . wio Light Sampling w/o Rendering
cross-intrinsic at-
Can a generalized PBR prior help intrinsic image decomposition? We believe that in- trinsic image decomposition methods are con- strained to match the appearance of the input image, Thus, the model can learn to rely more on the input, instead of focusing staying in the PBR distribution; therefore, facing an out- of-distribution input image poses challenge to these models to produce faithful PBR maps, corresponding to the respective distributions. On the contrary, our model is trained with the key constraint to produce faithful PBR maps enabling better generalization. To verify this hypothesis, we fine-ture our model make it image-conditional. We extend the set of in- put modalities with the rgb rendering and apply dropout similarly how we achieve a normal con- ditional variant for the room-scale scene textur- ing section 4 The resulting model can be condi- tioned on an input image during inference, sim- ilar to the baselines. This variant achieves 70.51 FID on the out-of-domain GObjaverse evalu- ation set (A-OOD-FIL FID), outperforming direct intrinsic image decomposition training (IID w/ FLUX-LoRA), and even our base model. This shows that pre-trained prior that directly mod- els the underlying distribution is beneficial for generalization.
Ours
Figure 8: Ablations. We compare our full method against ablations that do not use the rendering loss (w/o Rendering), instead of importance-basee light sampling (w/o Light Sampling), and do not use dropout in the cross-intrinsic attention (w/o CIA-Dropout). With- out the rendering loss (Section 3.2.2p. the PBR maps lose their semantic meaning, e.g., there are baked-in shadows in the albedo and the generated images appear "averaged out".. Importance- based light sampling (Section 3.2.2p and CIA dropout (Section 3.2.T) both increase the sharpness of in- dividual PBR maps, e.g., the roughness/metallic images have realistic details without baked-in tex- tures. Overall, all components improve the quality of rendered images under varied lighting condi- tions. We provide more samples the supplement.
uniform light sampling
6 Conclusion
We have presented IntrinsiX, the first method for direct generation of intrinsic image properties from text input. We leverage the strong image prior of pretrained text-to-image models and convert it into a PBR map generator. We have introduced cross-intrinsic attention to produce semantically aligned PBR maps. Furthermore, we have shown that using our novel rendering loss with tailored light sampling provides important signal for the model to better ground each intrinsic component. Our approach allows us to generate high nigh-quality, diverse results that go beyond the distribution of existing. synthetic datasets. Our method enables several downstream applications, such as physically-base rendering, material editing, relighting, and for the first time 3D scene PBR texture generation. We believe this showcases the potential that text-to-image models like ours can have on gaming and VR applications. Instead of generating content shaded RGB space, we produce the PBR maps that can be directly used in standard computer graphics pipelines.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by the ERC Consolidator Grant Gen3D (101171131) of Matthias Nießner, the German Research Foundation (DFG) Grant "Making Machine Learning Static and Dynamic 3D Data Practical", and the German Research Foundation (DFG) Research Unit "Learning and Simulation in Visual Computing". We thank Angela Dai for the video voice-over.
References
[1] Maria Teresa Baldassarre, Danilo Caivano, Berenice Fernández Nieto, Domenico Gigante, and Azzurra Ragone. The social impact of generative Al: an analysis on chatgpt. CoRR, ftss/2003.04667, 2024.
[2] Sean Bell, Kavita Bala, and Noah Snavely. Intrinsic images in the wild. ACM Trans. Graph., 33 (4):159:1-159:12, 2014.
[3] Raphael Bensadoun, Tom Monnier, Yanir Kleiman, Filippos Kokkinos, Yawar Siddiqui, Mahen- dra Kariya, Omri Harosh, Roman Shapovalov, Benjamin Graham, Emilien Garreau, Animesh Karnewar, Ang Cao, Idan Azuri, Jurii Makarov, Eric-Tuan Lc, Antoine Toisoul. David Novotný, Oran Gafni, Natalia Neverova, and Andrea Vedaldi. Meta 3d gen. CoRR, abs/2407.02599 2024.
[4] Brent Burley and Walt Disney Animation Studios. Physically-basee shading at disney. In Acm Siggraph, pages 1-7. vol. 2012, 2012.
[5] Chris Careaga and Yagiz Aksoy. Colorful diffuse intrinsic image decomposition in the wild. ACM Trans. Graph., 43(6):17::1-17: 2024.
[6] Dave Zhenyu Chen, Yawar Siddiqui, Hsin- Ying Lee, Sergey Tulyakov, and Matthias Nießnnr. Text2tex: Text-driven texture synthesis via diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 18558-18568 2023.
[7] Dave Zhenyu Chen, Haoxuan Li, Hsin- Hsin-Ying Lee, Sergey Tulyakov, and Matthias Nießner. Scenetex: High-quality texture synthesis for indoor scenes via diffusion priors. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 21081-21091. IEEE, 2024.
[8] Blender Online Community. Blender 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam, 2018
[9] Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo. Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, Eli VanderBilt, Aniruddha Kembhavi, Carl Vondrick, Georgia Gkioxari, Kiana Ehsani, Ludwig Schmidt, and Ali Farhadi. Objaverse-xl: A universe of 10m+ 3d objects. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
[10] Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated objects. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 13142-13153. IEEF, 2023.
[11] Valentin Deschaintre, Miika Aittala, Frédo Durand, George Drettakis, and Adrien Bousseau. Single-image SVBRDF capture with a rendering-awan deep network. ACM Trans. Graph., 37 14): 2018
[12] Xiaodan Du, Nicholas I. Kolkin, Greg Shakhnarovich, and Anand Bhattad. Generative models: What do they know? they know things? let's find out! CoRR, abs/2311.171337 2023.
[13] Xiang Feng, Chang Yu, Zoubin Bi, Yintong Shang, Feng Gao, Hongzhi Wu, Kun Zhou, Chenfanfu Jiang, and Yin Yang. ARM: appearance reconstruction model for relightable 3d generation. CoRR, abs/2411.1082 2024.

---

[14] Graham D. Finlayson, Mark S. Drew, and Cheng Lu. Intrinsic images by entropy minimization. In Computer Vision ECCV 2004, 8th European Conference on Computer Vision, Prague, Czech Republic, May 11-14, 2004. Proceedings, Part III, pages 582-595. Springer, 2004.
[15] Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 3d-future: 3d furniture shape with texture. International Journal of Computer Vision, pages 1-25, 2021.
[16]
Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo
Pratul P. Srinivasan, Jonathan T. Barron, and Ben Poole. CAT3D: create anything in 3d with multi-view diffusion models. In Advances in Neural Information Processing Systems 38; Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10- 15, 2024, 2024.
[17] Roger B. Grosse, Micah K. Johnson, Edward H. Adelson, and William T. Freeman. Ground truth dataset and baseline evaluations for intrinsic image algorithms. In IEEE 12th International Conference on Computer Vision. ICCV 2009, Kyoto, Japan, September 27 October 4, 2009, pages 2335-23422 IEEE Computer Society, 2009.
[18] Jun Guo, Xiaojian Ma, Yikai Wang, Min Yang, Huaping Liu, and Qing Li. Flowdreamer: A RGB-D world model with flow-based motion representations for robot manipulation. CoRR, 1bs/2505.10005, 2025.
[19] Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey. Trans. Mach. Learn. Res., 2024, 2024.
[20] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 6626-6637, 2017.
[21] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[22] Lukas Höllein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner. Text2room: Extracting textured 3d meshes from 2d text-to-image models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7909-7920, 2023.
[23] Lukas Höllein, Aljaž Božič, Norman Müller, David Novotny, Hung- Yu Tseng, Christian Richardt, Michael Zollhöfer, and Matthias NieBner. Viewdiff: 3d-consistent image generation with text-to-image models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5043-5052, 2024.
[24] Berthold KP Horn. Determining lightness from an image. Computer graphics and image processing, 34))2777-299 1974.
[25] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29,
2022. OpenReview.net 2022.
[26] Xin Huang, Tengfei Wang, Ziwei Liu, and Qing Wang. Material anything: Generating materials for any 3d object via diffusion. CORR, abs/2411.15111 2024.
[27] Yukun Huang, Jianan Wang. Yukai Shi, Xianbiao Qi, Zheng-Jun Zha, and Lei Zhang. Dream- time: An improved optimization strategy for text-to-3d content creation. CoRR, abs/2306. 12422, 2023.
[28] Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In Computer Visio-ECCVV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 694-711. Springer, 2016.
[29] Nima Khademi Kalantari and Ravi Ramamoorthi, Deep high dynamic range imaging of dynamic scenes. ACM Trans. Graph.,
2017.
12

---

[30] Peter Koesis, Julien Philip, Kalyan Sunkavalli, Matthias Nießner, and Yannick Hold holf-Geoffroy. Lightit: Illumination modeling and control for diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 9359-9369. IEEE, 2024.
[31] Peter Kocsis, Vincent Sitzmann, and Matthias Nießner. Intrinsic image diffusion for indoor single-view material estimation. IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 5198-5208. IEEE, 2024.
[32] Black Forest Labs. Flux. https://gitthb..... com hoooo//lack--oosttttee labs/flux 2023.
[33] Edwin H Land and John J McCann. Lightness and retinex theory. Josa, 61(1):1-11, 1971.
[34] Zhengqin Li, Mohammad Shafici, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chandraker. Inverse rendering for complex indoor scenes: Shape, spatially-vayying lighting and SVBRDF from single image. In 2020 IEEE/CVF Conference Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 4482-2481. Computer Vision Foundation IEEE, 2020.
[35] Zhengqin Li, Ting-Wei Yu, Shen Sang, Sarah Wang, Meng Song. Yuhan Liu, Yu-Ying Yeh, Rui Zhu, Nitesh B. Gundavarapu, Jia Shi. Sai Bi, Hong- Yu. Zexiang Xu, Kalyan Sunkavalli, Milos Hasan, Ravi Ramamoorthi, and Manmohan Chandraker. Openrooms: An open framework for photorealistic indoor scene datasets. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 7190-7199. Computer Vision Foundation IEEE, 2021.
[36] Ruofan Liang, Zan Gojcic, Huan Ling. Jacob Munkberg, Jon Hasselgren, Zhi- Hao Lin, Jun Gao, Alexander Keller, Nandita Vijaykumar, Sanja Fidler, and Zian Wang. Diffusionrenderer: Neural inverse and forward rendering with video diffusion models. arXiv preprint arXiv: 2501.18590. 2025.
[37] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
[38] Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Von- drick. Zero-1-to-3: Zero-shot one image to 3d object. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1 1-6, 2023, pages 9264-9275. IEEE, 2023.
[39] Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang- Syncdreamer: Generating multiview-consisten images from a single-view image. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.
[40] Ivan Lopes, Fabio Pizzati, and Raoul de Charette. Material palette: Extraction of materials from a single image. In IEEESCVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 3379-4388 IEEE, 2024.
[41] Nanye Ma. Mark Goldstein, Michael S. Albergo, Nicholas M. Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-basse generative models with scalable interpolant transformers. In Computer Vision ECCV 2024 18th European Conference, Milan, Italy, September 29- 29-October 4, 2024, Proceedings, Part LXXVII, pages 23-40. Springer, 2024.
[42] Xiaohe Ma, Valentin Deschaintre, Milos Hasan, Fujun Luan, Kun Zhou, Hongzhi Wu, and Yiwei Hu. Materialpicker: Multi-modal bateb material generation. ACM Trans. Graph., 44 (4):133:1-133:12 2025.
[43] Konstantin Mishchenko and Aaron Defazio. Prodigy: An expeditiously adaptive frremeter-free learner. In Forty-first International Conference on Machine Learning, ICML. 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024.
13

---

[44] Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 0296-4304 2024.
[45] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: improving latent diffusion models for high-resolution image synthesis. CoRR, abs/2307. (a//0/337.0195,, 2023.
[46] Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
[47] Lingteng Qiu, Guanying Chen, Xiaodong Gu, Qi Zuo, Mutian Xu, Yushuang Wu, Weihao Yuan, Zilong Dong, Liefeng Bo, and Xiaoguang Han. Richdreamer: A generalizable normal-depth diffusion model for detail richness in text-to-3d. In IEEE/CVF Conference Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 9914-9925. IEEE, 2024.
[48] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with CLIP latents. CoRR, abs/2204.061 2022.
[49] Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M Susskind. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10912-10922, 2021.
[50] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 10684-10695, 2022.
[51] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention-MICCA 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings. part III 18, pages 234 241. Springer, 2015
[52] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In IEEF/CVE Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, 22500022210..2 10. IEEE, 2023.
BC, Canada, June 17-24, 2023, pages 22500-22511
[53] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 22500-22510. IEEE, 2023.
[54] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
[55] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaezmarczyk, and Jenia Jitsev. LAION-5B: an open large-scale dataset for training next generation image-text models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
14

---

[56] Prafull Sharma, Varun Jampani, Yuanzhen Li. Xuhui Jia, Dmitry Lagun, Frédo Durand, Bill Freeman, and Mark J. Matthews. Alchemist: Parametric control of material properties with diffusion models. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 24130-24141. IEFE, 2024.
[57] Li Shen, Ping Tan, and Stephen Lin. Intrinsic image decomposition with non-local texture cues. In 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008), 24-26 June 2008, Anchorage, Alaska, USA. IEEE Computer Society, 2008.
[58]
Yawar Siddiqui, Tom Monnier, Filippos Kokkinos, Mahendra Kariya, Yanir Kleiman, Emilien Garreau, Oran Gafni, Natalia Neverova, Andrea Vedaldi, Roman Shapovalov, and David Novotný. Meta 3d assetgen: Text-to-mesh generation with high-quality geometry, texture, and PBR materials. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver. BC, Canada, December 10- 15, 2024, 2024.
[59] Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, Yuan Hao, Irfan Essa, Michael Rubinstein, and Dilip Krishnan. Styledrop: Text-to-image generation in any style. CoRR, abs/2306.00983, 2023.
[60] Shitao Tang. Fuyang Zhang, Jiacheng Chen, Peng Wang, and Yasutaka Furukawa. Mvdif- fusion: Enabling holistic multi-view image generation with correspondence-awarc diffusion. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10- 16, 2023, 2023.
[61] Shimon Vainer, Mark Boss, Mathias Parger, Konstantin Kutsy, Dante De Nigris, Ciara Rowles, Nicolas Perony, and Simon Donné. Collaborative control for -ondttoy- PBR image generation. In Computer Vision ECCV 2024 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part XIII, pages 127-145. Springer, 2024.
[62] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998-6008, 2017.
[63] Zhouxia Wang, Xintao Wang, Liangbin Xie, Zhongang Qi, Ying Shan, Wenping Wang, and Ping Luo. Styleadapter: A single-pass lora-free model for stylized image generation. CoRR, abs/2309.01770, 2023.
[64] Jiaye Wu, Sanjoy Chowdhury, Hariharmano Shanmugaraja, David Jacobs, and Soumyadip Sengupta. Measured albedo in the wild: Filling the gap in intrinsics evaluation. In IEEE International Conference on Computational Photography, ICCP 2023, Madison, WI. USA, July 28-30, 2023, pages 1-12. IEEE, 2023.
[65] Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tunc-a-video: One-shot tuning of image diffusion models for text-to-video generation. IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pages 7599.7599. IEEE, 2023.
[66] Bin Xiao, Haiping Wu, Weijian Xu, Xiyang Dai, Houdong Hu, Yumao Lu, Michael Zeng, Ce Liu, and Lu Yuan. Florence-2: Advancing a unified representation for a variety of vision tasks. In IEEE/CVF Conference Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 48 4818-4829, IEEE, 2024.
[67] Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, and Song Han. SANA: efficient high-resolution text-to-image synthesis with linear diffusion transformers. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025.
15

---

[68] Xiaogang Xu, Hengshuang Zhao, Vibhav Vineet, Ser-Nam Lim, and Antonio Torralba. Mt- former: Multi-task learning via transformer and cross-task reasoning. In Computer Vision ECCV 2022 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Purt XXVII, pages 304-321. Springer, 2022.
[69] Bowen Xue, Giuseppe Claudio Guamerr, Shuang Zhao, and Zahra Montazeri. Diffusion-basee g-buffer generation and rendering. CoRR, abs/2503.15147, 2025.
[70] Shuai Yang. Yifan Zhou, Ziwei Liu, and Chen Change Loy. Rerender A video: Zero-shot text-guided video-to-video translation. In SIGGRAPH Asia 2023 Conference Papers, SA 2023, Sydney, NSW, Australia, December 12-15, 2023, pages 95:1-95:11. ACM, 2023.
[71] Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. CoRR, abs/2308.06721, 2023.
[72] Zheng Zeng, Valentin Deschaintre, Iliyan Georgiev, Yannick Hold-Geoffroy, Yiwei Hu, Fujun Luan, Ling-Qi Yan, and Milos Hasan. Rgbe>x: Image decomposition and synthesis using material- and lighting-aware diffusion models. In ACM SIGGRAPH 2024 Conference Papers, SIGGRAPH 2024, Denver, CO, USA, 27 July 2024- 1 August 2024, page 75. ACM, 2024.
[73] Chenshuang Zhang, Chaoning Zhang. Mengchun Zhang. and In So Kweon. Text-to-image diffusion models in generative AI: A survey. CoRR, abs/2303.0799 077999, 2023.
[74] Lymin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In JEEF/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pages 3813-3824. IEEE, 2023.
[75] Qing Zhang, Jin Zhou, Lei Zhu, Wei Sun, Chunxia Xiao, and Wei-Shi Zheng. Unsupervised intrinsic image decomposition using internal self-similarity cues. IEEE Trans. Pattern Anal. Mach. Intell., 44(12):9669-9688 2022.
[76] Jingsen Zhu, Fujun Luan, Yuchi Huo, Zihao Lin, Zhihua Zhong, Dianbing Xi, Rui Wang, Hujun Bao, Jiaxiang Zheng, and Rui Tang. boaming-bbsedd inverse rendering of complex indoor scenes with differentiable monte carlo raytracing. SIGGRAPH Asia 2022 Conference Papers, SA 2022, Daegu, Republic of Korea, December 6-9, 2022, pages 6: 6:1-6:8. ACM, 2022.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The key contributions are summarized in the abstract as well as in the last paragraph of the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Appendix SeanndixCcessciii desccibes the limitations of our method.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
17

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
We will release the training and testing codes along with our trained model weights upon acceptance. should also be possible to reproduce the model based on our description in the paper and the supplementary.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
18

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We will release the training and testing codes along with our trained model weights upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
Sublic/guidess Beaiii/gguides//deubbniii
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to reproduce the results. See the NeurlPS code and data submission guidelines (https
cttpss.cccpubblci gwwdes/ideeeee www.oooeeewmmomooooooooooooo
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Sections 3 and and Appendices and G describe the the necessary training and testing details.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the standard deviation for our user study results (table T The FID metrics are evaluating statistical similarity between two distributions (between large amount of generated samples), thus already providing information about the statistical significance of the experiments. Additionally, we show multiple generated samples with different seeds in Figure
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
19

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The first section of Section describes the required resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Our work is in accordance with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We provide a discussion about the societal impact in Appendix
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
20

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: We will add section about ethical and out-of-scope usage in our code release similarly to 32 to ensure responsible usage.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We describe the licenses of all the assets in Appendix
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
21

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We will release the code and the trained model weights with training and testing scripts. We provide details about the training in Section We describe the licenses of all the assets in Appendix
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: We conduct user-study with human participants. We describe the details in Appendix Fnd provide one example of the provided instructions in Figure T8
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: We conduct user-study with human participants. We describe the details in Appendix F Our user-study is anonymous and does not raise any risks for the participants.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
22

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurips: hett::///eerrrps.ccc//ooeeereeee cc///onfereeeee//222///.... for what should or should not be described.
23

---

IntrinsiX:
High-Quality PBR Generation using Image Priors
Supplementary material
Peter Kocsis
Lukas Höllein
Matthias Nießner
Technical University of Munich
peter github.............. io/Intrinsix/
A Additional Ablations
How important is the dataset size and diversity? In the first stage of training, we train separate LoRAs, corresponding to the different intrinsic properties. We curate synthetic indoor scene examples from the Interior Verse dataset [76]. We empirically find that we need large dataset size for the roughness/mettalic PBR maps to achieve reasonable understanding of the corresponding intrinsic distribution. In contrast, the albedo/normal maps can be learned from a much smaller dataset of only 20 samples. This is important to retain the generalizable prior of the pretrained text-to-image model (see Appendix A We confirm this with additional experiments in Table 2 that compare the quality and diversity of generated albedo images for different dataset sizes. The in-distribution FID (A-ID-FID) measures the quality of the albedo (calculated
subset of 100 test images of InteriorVerse 76]. similar as in main paper). The diversity metric (A-Diversity) compares FID between the generated set of all images and the mean of the generated set. This measures if the distribution is collapsed and therefore signals how diverse the generated samples are. We can see that dataset consisting of 20 samples does the best in terms of diversity, while still having reasonable albedo quality. Importantly, albedos trained on larger datasets also start to include baked-in lighting effects (see Figure TOp. This motivates choice to not increase the dataset size further. The final dataset consists of sampled images from the Interior Verse dataset [76). We sample images from the following room-types to curate 20 samples: 5 bedrooms, 5 kitchens, 5 livingrooms, kidroom, offices, cabinet, bathroom.
LoRA Rank We ablate the rank of the LoRA 25 modules in fig. 9 The rank determines the total number of trainable parameters. Therefore, with a too low rank, the number of trainable parameters are too low in order to achieve the domain shift. On the other hand, with too high rank, are introducing too many parameters, which will lead to forgetting; thus, negatively effecting the generalization. As middle-ground, we chose rank 64.
Rank A-OOD-FID
32
64 (Ours) 67.25
128
256
68.52
70.07
68.48
71.13
Figure 9: Qualitative and quantitative comparison of albedo quality for different LoRA 25 ranks. Too low rank fails to change the domain from rgb the target albedo modality. On the other hand, a too high rank negatively impacts the generalization, resulting in unrealistic composition.

---

Table Quantitative comparison albedo quality for different dataset sizes. We observe that training with larger dataset might lead to slightly better albedo quality (A-ID-FID); however, the diversity (A-Diversity) and thus the generalization capabilities degrade. This motivates our choice for small, curated dataset of 20 samples for the first stage finetuning of the altedo/normal LoRAs.
Dataset size
A-ID-FID 4
A-Diversity 1
10
220.28
284.93
20 (Ours)
187.83
398.36
100
161.51
369,43
1k
154.58
366.35
20k
155.64
352.04
(Ours)
Figure 10: Qualitative comparison of albedo quality for different dataset sizes. Datasct sizes of 100 more images tend to generate albedos with baked-in lighting effects, which is undesirable for physically-based rendering. dataset that only consists albedos. This motivates usage of 20 curated samples in the albedo/normal LoRA training, which balances both extrema. We show multiple samples per column, corresponding to different generations from the same text prompts. This highlights, that model creates diverse images.
images shows less details in generated
Can we maintain sample diversity? We show multiple samples using the same text prompt in Figure [1 Our method manages to maintain the generalization capabilities of the T2I model and generates diverse samples even for -uitt--distributioon prompts (see also Figure and the supplementary material).
M
!:I0
More samples We show additional qualitative comparisons in Figure
Individual PBR Priors In the first stage of training, we train separate LoRAs, correspond- ing the different intrinsic properties. We curate synthetic indoor scene examples from the Interior Verse dataset [76]. We show in Fig- ure [3 that this leads to high--qulity and di- verse albedo and normal map generations. This confirms our choice of training these PBR maps on small-scale datasets, i.e., we retain the gener- alized prior of the pretrained text-to-image model during the first stage finetuning.
Normal
Albedo
Roughness
Metallic
Figure 11: Sample diversity. We show 3 gen- erated samples using the same text prompt. Our model predicts different samples and maintains the diversity of the T2I backbone (numerous chairs were not seen during training).

---

Normal
Albedo
R/M
Lighting
Lighting
w/o Rendering
w/o Light Sampling
w/o CIA-Dropout
Ours
Figure 12: Additonal ablations. We compare our full method against ablations that do not use the rendering loss (w/o Rendering), use uniform light sampling instead of importance-based light sampling (w/o Light Sampling), and do not use dropout in the cross-intrinsic attention (w/o CIA- Dropout). Without the rendering loss (Section 3.2.2), the PBR maps lose their semantic meaning, e.g.. there are baked-in shadows in the albedo and the generated images appear "averaged out". Importance-basee light sampling [Section[3.2.2 and CIA dropout (Section (.2.1] both increase the sharpness of individual PBR maps, c.g., the roughness/metallic images have realistic details without baked-in textures. Overall, all components improve the quality of rendered images under varied lighting conditions.
In contrast, the roughness/metallic LoRAs fail to generalize to out-of-distribution scenarios. This because a larger dataset for training this LoRA. However, Figure 13 (bottom) shows that second stage alignment training turns this LoRA to an equally-well generalizable PBR map generator. In other words, the generalizability of the albedo/normal LoRAs can be combined with the understanding of the intrinsic distribution the roughness/mmetallic LoRA. Together, we can still produce high-quality, diverse PBR maps.
B
Additional Results
Lighting Direction Sampling We sample light directions that maximize specular highlights to provide strong gradients about reflectance. As an alternative, we train a BRDF-sampled variant (Disney model) for more diverse lighting directions. This yields slightly worse results on the in- domain dataset (75.26 A-ID-FID), but further improves generalization (68.87 A-OOD-FID), showing

---

Normal
Roughness Albedo
YORK
Metallic
Normal
Metallic Roughness Albedo
Figure 13: Comparison between stage and stage 2 samples. In the first stage, train LoRAs separately corresponding to the different PBR maps (albedo, normal, roughness+metallie synthetic indoor-scene examples. In the second stage, align these PBR priors through cross- intrinsic attention and the rendering loss. Top: generated images in the first stage (independently for each modality) show good quality for the albedo and normal maps. However, the roughness/metallic predictions are only reasonable for in-distribution scenarios (e.g. the 4th column) and become less detailed for -attr--distribution prompts. Bottom: after alignment training, all PBR maps have meaningful structure and exhibit sharp, high-quality content.
that lighting direction sampling is crucial for our task. Exploring other sampling strategies is great avenue for future research.
Baseline comparisons
We show additional comparisons to the baselines in Figure 14
Albedo comparisons
We show additional albedo comparisons the basclines in Figure 15
We used Blender X to render the scene with uniform white environment map lighting and a single spherical light source. To enhance geometric details, we used an approximation of the displacement map by thresholding the normal textures.
Scene Texturing Results
We show more scene texturing results in Figure 16
To achieve room-scale scene texturing, we apply the SceneTex method 77] in two-stage manner with the conditional variant of our model. First, we render normal maps from multiple views and generate material (albedo, roughness and metallic) textures, conditioned on the rendered normals. Then, we render the material properties for the given views and generate fine- mne-grained normal details, conditioned on the rendered material maps. We use VFDS [88] [88 loss in image space. To balance updates between over- and under-sampled texels, we weight the lostt with the inverse obervations frequency. As pre-processing step, we create texture, which stores the texel observation frequency. During the optimization, we render this texture together with the other components apply the weighting pixel-wise. We found that too low CFG value causes smoothed results, while too high

---

ROBX
Ohrs
Narmal
Material
Lighting
Lighting 2
Normal
Material
Lighting
Lighting 2
ITD
RGBX
OhiEs
Figure 14: Additional rendering comparisons. We show sample PBR maps of our method and baselines as well rendered RGB images under two different lighting conditions. We diverse of text prompts to produce our PBR maps, as well as the input RGB images for the baseline methods. This highlights our models' capability retain the generalized prior the pretrained lext-to-image model. Our method better captures the semantic meaning of the individual intrinsic properties. For example, there are no baked-in lighting effects the albedo, and the metalllc/roughness maps are sharper with more intricate details. This leads to more realistic renderings and lighting effects.
values can break the generated images in case of Flux [32]. To solve this issue, normalize the flow direction to keep the norm of the text-conditional prediction, but use the direction towards the extrapolated flow direction.
C Limitations
We use a screen-space renderer similar to 76 For better 2D results, neural/diffusion renderer can optionally be trained on top of our method. Our method maintains generaliza- tion far beyond the minimal training set, thanks formulating the task directly in PBR space. Since FLUX does not inherently know about intrinsic components, we sacrifice some compo- sitional diversity to enable our task using LoRA modules (see Figure [7p. Ultimately, training with diverse billion-scale PBR dataset would further improve generalization, but does not exist.
31
Albedo
Normal
FLUX 32
Figure 17: Limitations. Since FLUX 32 does inherently know about the intrinsic properties and we cannot train on a similarly large dataset as the model was originally trained, we sacrifice details during the fine-tuning. Therefore, our PBR maps do contain as much details as an image generate by FLUX and sometimes the generated properties can be incorrect (e.g. normals).

---

II 3I
RGBX 72
Colorful
Shuding
T
w/ FLUX-LX LoRA S2 25
SmallData
Ours w/ In-Cond
Ours
Figure 15: Additional albedo comparisons. We show albedo images of our method and baselines corresponding the same text prompt in each column. Our albedo images have less baked-in shadows and reflections, which is desirable for downstream tasks, such physically-based rendering.
D Societal Impact
Generative models impact the society in general. Next accelerating and democratizing creative content creation, they can also raise ethical concerns. Potential misuse for generating misinformation or deepfakes can become major threat for naive users. These risks needs to be discussed and made public as soon as possible with open-sourcing and publishing results in the field to show the limits of current state-of-the-art, These challenges have been widely discussed in the recent years, such in M. Our method enables decomposed generation, which coupled with photo-realistic rendering can produce realistic-looking results.
E
Licenses
Table Sshows a summary about the licenses the used components.
Table 3: Licenses. We provide a summary about the licenses of the used resources.
Type
Source
License
Code
Flux 1.0 Dev
Apache v2.0
Code
SceneTex
CC BY-CC--AA- 3.0 MIT License
Data
InteriorVerse
Data
3D-FRONT
Custom, research-only
Data
GUbjaVerse
Apache v2.0

---

View
View 3
Normal
Albedo
Roughness
Metallic
Rendering
View
25
View 3
Normal
Albedo
Roughness
Metallic
Rendering
View
View 3
Normal
Albedo
Roughness
Metallic
Rendering
View
View 3
Normal
Albedo
Roughness
Metallic
Rendering
Figure 16: Scene Texturing. We show more scene texturing results on multiple 3D-Front scenes with multiple prompts. Continues on the next page.

---

View
View 3
Normal
Albedo
Roughness
Metallic
Rendering
View
View 3
Normal
Albedo
Roughness
Metallic
Rendering
View
View 3
Normal
Albedo
Roughness
Metallic
Rendering
View
View 3
Normal
Albedo
Roughness
Metallic
Rendering
Figure 16: Scene Texturing We show more scene texturing results on multiple 3D-Front scenes with multiple prompts. Continues on the next page.

---

View
View
Normal
Albedo
Roughness
Metallic
Rendering
View 1
View 3
Normal
Albedo
Roughness
Metallic
Rendering
View
View 3
Normal
Albedo
Roughness
Metallic
Rendering
AMA
View
View 3
Normal
Albedo
Roughness
Metallic
Rendering
Figure 16: Scene Texturing We show more scene texturing results on multiple 3D-Front scenes with multiple prompts. Continues on the next page.

---

View
View
Normal
Albedo
Roughness
Metallic
Rendering
View
View 3
Normal
Albedo
Roughness
Metallic
Rendering
View
View
Normal
Albedo
Roughness
Metallic
Rendering
View
View 3
Normal
Albedo
Roughness
Metallic
Rendering
Figure 16: Scene Texturing. We show more scene texturing results on multiple 3D-Front scenes with multiple prompts.
10

---

Select masted the bodo for ovalaation Abodo gritig ofeica Ranodddr bukedie
shudonn
Rendering/ Lighting Quality, Prompt Coherence Albedo Preference
theido cevepent
catiiin clogee murameer esuor guine antonatio
susa bo new soste agr
hightght and
eg deo
cfficre ucnaalenic whicch
Figure 18: Sample questions in the user study. Users are presented with two types of questions. Top: users select the best albedo among all methods. Bottom: users rate the specular and rendered quality as well as the prompt coherence on a scale of 1-5 for rendered video example.
F
User Study
To better evaluate the quality of our generated PBR maps, conduct user study. The participation anonymous and no personal data collected. We summarize Figure T8 the questions we asked by the participants. In the following, we explain how each metric is caleulated.
A-PP: calculate the perceptual preference of albedo images (see Figure 18 top). Users choose
one of the images and we calculate in percentage how often each method was preferred.
S-PQ: calculate the quality of specularity of the rendered video under varying lighting conditions
(see Figure 18 bottom). Users rate on a scale of 1-5 how good the specular quality is.
R-PQ: we calculate the general quality of the rendered video under varying lighting conditions (see
Figure T8 bottom). Users rate on a scale of 1-5 how good the general quality is.
PC: calculate the prompt coherence, i.e, how well the text prompt matches the rendered video
(see Figure 18 bottom). Users rate on a scale of 1-5 how good the coherence is.
G
Prompts
We used the following prompts in main results. We used own, L.-M-geeerated prompts, and prompts from Gao et al. [16]:
Figure
"An astronaut riding unicom on the moon'
Figure
"An astronaut riding unicorn on the moon"
Figure "Astronaut in front of landscape space alien planet"
Figure 5
"An industrial-style room with exposed brick walls, and reclaimed wood furniture, The
room features leather sofa, coffee table made from metal frame, and modern decor that complements its edgy vibe"

---

Figure 6 from left to right and top to bottom: "A wooden treasure chest reinforeed with golden bands, lid slightly ajar to reveal glittering jewels and coins, with faint beams of light spilling out from inside", "3d cartoon folk singers character music guitar animation",
Figure 7 exploration nature trees hiking woodland outdoor", "Adventurous teddy bear explorer travel outdoor", "Alpaca wearing a suit animal clothing formal wool", "Anime character lab coat scientist cartoon drawing japanese style".
from frrm left right: "3d cartoon boy character animation", "Adventurer standing in forest
Figure g
"A vintage pocket watch with its cover open, revealing a complex arrangement gears and springs, some of which are glowing faintly, surrounded by engraved floral patterns."
Figure 9
"An astronaut riding a unicomn on the moon"
Figure I0
"An astronaut riding unicorn on the moon"
Figure
11
"A wooden chair"
Figure T2
"A sportcar"
Figure 13 from left to right: "An astronaut riding a unicorn on the moon", "3d cartoon folk singers
character music guitar animation", "Alien merchant extraterrestrial market fantasy science fiction", "Alley city urban narrow passage architecture outdoor", "Astronaut in front of landscape space alien planet", "A majestic castle made entirely of ice, perched atop snowy hill with shimmering pink and golden light reflecting off its towers. Below, a frozen lake mirrors the grandeur of the scene". "A sprawling library with towering bookshelves reaching to the ceiling, glowing orbs floating mid-air -ir to provide light, books that seem to fly on their own, and a spiral staircase made of golden wood.". "A house in a forest", "New York", "A wooden chair"
Figure [4 from left to right and top to bottom: "A rusted sword with glowing blue rune etched into the blade, its hilt wrapped in weathered leather, and a faint aura of light surrounding it as if imbued with ancient magic", "An astronaut riding a unicorn on the moon", "A large, ornate key made of silver, with intricate vine-like patterns etched along the shaft and a glowing emerald embedded in the handle", "Taj Mahal"
Figure from left to right: "Arches national park nature rock formations desert travel outdoor", "Astronaut in colorful cave exploration adventure discovery geology outdoor". "An epic battlefield where knights in shining armor clash with dragon-riding warriors under a stormy sky. A massive fire-breathing dragon is mid-flight, casting shadows over the chaos below", "A massive with a forest on its back swims through crystal-clear waters, accompanied by schools of colorful fish. A small sailing ship navigates beside it, dwarfed by the turtle's size", "A sleek, metallic helmet with a reflective visor that glows neon blue, featuring angular designs and small vents that emit a soft, white mist"
turtle
Figure 17 "A very strange burger food unusual creative"
Figure 16 from top bottom: "An opulent Baroque-style room with intricate details, Walls are decorated with elaborate molding, in shades of cream, gold, and soft pastels, A plush velvet sofa, A richly patterned Persian rug covers the marble floor". "An industrial-style room with exposed brick walls, and reclaimed wood furniture, The room features a leather sofa, a coffee table made from a metal frame, and modern decor that complements its raw, edgy vibe". "A Tuscan-style room with warm earthy tones, terracotta tiles, and wrought iron details, The furniture features rich wood frames and soft eushions, complemented by Mediterranean -nnspired decor". "A breathtaking Greek- style room with intricate details, featuring serene blue-and- color scheme, Majestic marble columns with ornate Corinthian capitals support high, coffered ceiling adorned with classical frescoes, The walls showcase elegant friezes and gold-accented moldings, reflecting the grandeur ancient Greece, Large arched windows allow soft, natural light to flood the space, enhancing the contrast between crisp white walls and rich blue decorative elements, A luxurious chaise lounge with blue upholstery sits, accompanied by a marble-topped table with delicate carvings, The floor is adorned with intricate mosaic patterns"
12


Reviews:
[{"review_id": "J6rNI1hzv3", "reviewer": "Reviewer_3G9g", "summary": "The paper introduces a framework for generateing PBR maps directly from text prompts.  It leverages a pretrained flow model (Flux.dev1) and employs LoRAs for effective fine-tuning.  To further enhance the quality of generated PBR maps, the authors use a cross-view/multi-view attention which enables effective data interaction across different attributes of PBR maps (e.g., albedo, roughness/metallic, normal) on batch dimension. The paper also apply a rendering loss to enhance the modeling of PBR map distributions.", "questions": "- Exactly how many images are used for training the albedo and normal maps? The mention in Line 130 of \"20 synthetic examples of albedo and normal maps\" is somewhat unclear.\n- Does the cross-view/multi-view attention mechanism introduce new attention layers (e.g., new linear layers for queries, keys, and values)? If not, reusing the existing attention layers might interfere with pretrained priors due to altered attention behavior, potentially complicating the second-stage training process that still only employs LoRAs.", "limitations": "Please refer to Weaknesses.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n- The paper is well-written and easy to follow. The figures are intuitive and helpful for understanding the method.\n- The proposed method shows promise in generating high-quality PBR maps from text prompts, especially showing improvements in generating clean albedo maps.\n\n### Weaknesses\n\n- Although the proposed method produces cleaner albedo maps, it still faces challenges in accurately handling reflections. Additionally, the generated normal maps frequently appear noisy. Consistency among different attributes (albedo, roughness/metallic, normal) is also lacking, as each attribute often presents disparate details.\n\nThe paper is motivated by the limitations of existing methods in decomposing RGB images into distinct PBR maps, a task inherently ambiguous and ill-posed. The authors opt for generating PBR maps directly from text prompts instead. However, for practical real-world applications, decomposing or generating PBR maps from RGB images aligns more closely with common workflows, such as image editing and relighting. Therefore, the primary advantage of the paper, generating cleaner PBR maps, is simultaneously its principal limitation regarding practicality in real-world scenarios. Nonetheless, the overall approach shows promising results, particularly given the limited training data. The proposed solution is logically sound and well-justified. Considering these points, I am leaning towards borderline acceptance.", "was_revised": true, "final_justification": "The authors’ rebuttal addresses most of my concerns, and training an image-conditional IntrinsiX resolves my main issue about the motivation of the paper. So, I recommend accepting this paper.", "rebuttal": "## Motivation for Image-Space PBR Priors\n\nText-based generative models can act as a generic prior over images, which has shown great value for numerous downstream tasks: improved intrinsic image decomposition(IID \\[29\\], RGBX \\[67\\]), depth estimation (Marigold \\[Ke et al CVPR 2024\\]), etc. Our goal is to establish a similar *generic prior* for PBR materials to support downstream applications like UV-based texturing. \n\nThe key challenge of the field of PBR generation is the lack of large-scale, diverse datasets of PBR UV textures. Unlike RGB data, material datasets are scarce, and crucially, consistent UV parametrization (mapping from object coordinates to texture coordinates) across assets practically does not exist. This makes training direct generative models in UV texture space infeasible.\n\nInstead, we learn a prior in image space. To distill image-space information into texture-space, we use SDS, which aggregates image-space signals from multiple views. This enables PBR UV texture generation for 3D assets, even though trained with relatively small amounts of image-space PBR data. In contrast, decomposition methods require an image as input and it is not obvious how one would synthesize consistent PBR UV textures for novel 3D assets. \n\n## Accurate Reflections and Normal Maps\n\nWe propose an image-space rendering loss with our lighting direction sampling to improve the specular reflections. We show in Fig. 8 that without this rendering loss all modalities are averaged and washed away. The rendering loss helps to ground the individual PBR properties and leads to more details. Our lighting direction sampling further enhances this effect by strengthening the gradient signal coming from the specular highlights. Compared to the baselines (Fig. 6), our method exhibits much more details in the rough and metal maps, which ultimately facilitates higher-quality renderings. That being that we fully agree that there is still room for further improvements (e.g., with more diverse datasets); however, at the same time we argue that our approach makes a significant step forward.\n\n## Implementation Details\n\nWe curate 20 views from the InteriorVerse dataset across different scenes. Then, we select the corresponding albedo and normal maps for training our LoRA layers. Please also refer to the supplementary material L942 for more details.  \nOur cross-view attention mechanism utilizes the new LoRA layers inside of the attention blocks. Concretely, we add LoRA in every attention block for the to\\_q, to\\_k, to\\_v, to\\_out linear layers in our first stage training. In the second stage, we fine-tune these new layers with our cross-intrinsic attention. In both stages, we do not modify the pretrained prior weights, thus preserving the generalization capabilities. We are happy to elaborate further in the final revision."}, {"review_id": "nDl0ITuXzJ", "reviewer": "Reviewer_PKpj", "summary": "This paper introduces IntrinsiX, a novel approach for generating high-quality intrinsic images directly from text descriptions. Unlike traditional text-to-image models that produce RGB images with baked-in lighting effects, IntrinsiX generates physically-based rendering (PBR) maps (albedo, roughness, metallic, and normal maps) using a probabilistic diffusion process. The method involves a two-stage training process: first, training intrinsic priors for each PBR component separately using LoRA layers on curated datasets, and then aligning these models via cross-intrinsic attention and a new rendering loss with importance-based light sampling. This ensures semantically coherent and detailed PBR predictions. The core contributions of this work include being the first to directly generate usable PBR maps from text, decomposing strong image priors into intrinsic components, and combining cross-intrinsic attention with a novel rendering objective.", "questions": "How was the decision made to use a rank of 64 in the LoRA training? Were other values explored during the experimentation process? If so, how did they compare in terms of performance and efficiency? If not, what was the reasoning behind choosing 64 without exploring other options?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. It's the first method to directly generate PBR maps from text, addressing the limitations of existing text-to-image models that only output RGB images with baked-in lighting effects.\n2. The two-stage training process with cross-intrinsic attention and novel rendering loss is well-designed and effectively combines the strong image prior of pretrained T2I models with intrinsic component generation.\n3. The paper provides comprehensive evaluations with quantitative metrics and qualitative comparisons against baselines. The user study further supports the claims about the quality of the generated PBR maps.\n\nWeaknesses\nThe method requires specific datasets for training each PBR component and may struggle with more diverse or complex scenarios beyond the training data. The paper only uses 40 images for out-of-distribution evaluation, which may not reflect the true generalization ability of the method.", "was_revised": true, "final_justification": "The authors basically addressed my concerns in the rebuttal. The significance of this research lies in its advancement of text-to-image generation for practical graphics applications, such as gaming and VR, by producing outputs suitable for re-lighting and editing. I remain postive about the paper.", "rebuttal": "## FID Metrics\n\nWe extend our test set and re-evaluate all the baselines and our method. For in-domain experiments, we use the full InteriorVerse test set (2595 samples, as done in IID \\[29\\]). Evaluating the out-of-domain performance is challenging since there exists no large-scale PBR dataset of complex scenes from various domains. To evaluate one specific domain, we use the single-object G-Buffer ObjaVerse \\[Zuo et. al. ECCV 2024\\] dataset. We take 1000 samples from the diverse \\`\\`Daily-Used\" category. To show that these numbers are representative, we evaluate the FID with a different number of samples. The relative improvement over baselines align with the original FID numbers in the main paper and the metrics are consistent. Our method achieves better generalization to out-of-distribution domains. \n\nWe provide three additional experiments to better support our claims (as also requested by the reviewers). \n\n* *Ours w/ Image-cond:* we make our model image-conditional similarly, as we did for our normal-conditional model during the SDS-based texturing (L228). This variant maintains sharp results and outperforms baselines on OOD data, demonstrating the benefit of a learned PBR prior.   \n* *Ours w/ BRDF-sampling:* we sample more diverse lighting directions using BRDF-sampling during our rendering loss to provide more gradient signal about the BRDF behavior. This variant slightly improves our generalization results, showing the importance of lighting direction sampling.   \n* *IID+FLUX LoRA:* we re-train the IID baseline with FLUX as backbone and use LoRA layers similar to our method. This improves the fidelity, but this still does not reach the quality of our method, showing that learning a generalizable prior over PBR distributions is beneficial for generalization. For more details, we refer to our rebuttal to reviewer 48oZ.   \n\nWe are happy to include these detailed results.\n\n**\\# Samples vs FID on InteriorVerse**\n\n|  | 100 | 500 | 1000 | 1500 | 2000 | 2595 |\n| :---- | :---: | :---: | :---: | :---: | :---: | :---: |\n| Ours | 186.60 | 113.96 | 92.21 | 83.32 | 77.13 | 72.08 |\n\n**\\# Samples vs FID on G-Buffer ObjaVerse**\n\n|  | 100 | 300 | 500 | 700 | 800 | 900 | 1000 |\n| :---- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| Ours | 186.60 | 123.54 | 99.14 | 85.44 | 80.46 | 75.28 | 71.39 |\n\n**FID Metrics**\n\n|  | In-domain InteriorVerse | Out-of-domain G-Buffer ObjaVerse |\n| :---- | :---: | :---: |\n| IID | 78.77 | 98.77 |\n| RGBX | **61.36** | 90.12 |\n| ColorfulShading | 91.10 | 86.48 |\n| *IID+FLUX LoRA* | 103.36 | 79.29 |\n| *Ours w/ BRDF-sampling* | 75.26 | **68.87** |\n| *Ours w/ Image-cond* | 98.08 | 70.51 |\n| Ours | 72.09 | 71.39 |\n\n## LoRA Rank\n\nWe explored different LoRA ranks and noticed differences in (a) image quality and (b) generalizability to out-of-distribution samples. A too low LoRA rank fails to produce high-quality images from the target domain. In contrast, a too high LoRA rank negatively impacts generalizability in a similar fashion as our analysis of dataset size for LoRA finetuning (Fig.9/Tab.2). Thus, we chose rank 64 as the best middle-ground that produces high-quality images and still generalizes well to out-of-distribution domains. We'll include these findings into our Fig.9."}, {"review_id": "xcrj8Y9AFJ", "reviewer": "Reviewer_48oZ", "summary": "The paper addresses the problem of text-conditioned PBR generation using a pretrained generative model. The authors adapt an image generation model with LoRA for the text-based generation of different PBR components. This results in three sets of LoRA weights: one for albedo, one for normals, and one for specularity (channel-packing roughness and metalness). Naturally, these models are not aligned and produce inconsistent outputs. To address this, the authors introduce a second stage where traditional self-attention is replaced by cross-task attention. In this setup, the activation for a given component is computed based on its correlation with the activations of other components (including itself). Additionally, a rendering loss is applied in image space between the re-rendered ground truth and the rendering of the generated samples, using a single point light source. The model produces high-quality, sharp results that are consistent across PBR properties. Finally, the authors demonstrate that their method can be successfully applied to scene texturing via score distillation sampling.", "questions": "- Wouldn't you say there are also inherent limitations to relying on text-based conditioning? I remain puzzled about the practical need for such a prompt-based approach. Text prompts in diffusion models have been shown to be relatively imprecise, as they tend to map to broad regions in the appearance space; as shown in Material Palette (Lopes et al., CVPR 2024) and TexSlider (Guerrero-Viu et al., SIGGRAPH 2024).\n\n- Regarding the importance sampling, here the strategy used is rather simplistic, shouldn’t directions other than the reflected direction also be sampled? It seems natural that sampling this direction exclusively would prevent the model from learning rougher properties.\n\n- On line 294, authors say that baselines perform worse because they rely on indoor datasets. Again, I am convinced that the problem tackled by these methods is a different that the one proposed in this paper. Anyhow, I think this still raises the question of whether the qualitative results stem from model size or from the fact that the model is learning the joint distribution. I wonder if the generative capability depends primarily on model size. If the authors had compared their approach with a smaller model, such as Stable Diffusion (v1 or v2), we could better understand whether the improvements come from model capacity or the problem formulation.\n\n- Have the authors investigated which layers are most critical for adapting the model to the downstream task? Is fine-tuning all layers necessary, and how effective is this relative to the model size? In the current setup, how many parameters are trained when applying LoRA?\n\n- Regarding the dataset size ablation (line 937), the fact that larger dataset sizes yield worse results sounds counterintuitive. Do you have an explanation for why this occurs? Isn’t this a limitation if the model’s learning capacity doesn’t scale well with more data? How does the model behave when a new domain is added to the training set, especially for the two tasks trained with fewer examples (normals and albedo)?\n\n- In the scene texture examples, I find the roughness and metallic values rather flat, that is, the values seem clustered around an average. Do you have any idea why this might be?\n\n- Regarding the image-space loss, how is backpropagation handled through all the steps? Also, have you tried rendering with multiple light directions or even an environment map during inference?", "limitations": "The limitations section does not provide an in-depth analysis of the method’s failure modes. Instead, it mainly explains how a larger dataset could improve results. I believe a deeper understanding of the method’s real limitations and how future research could address them would be very beneficial, and this aspect is definitely missing from the paper.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Paper strengths\n- I find the explanation of why joint learning (with cross-talk between different material properties) leads to unsatisfactory results quite interesting. As the authors hypothesize, since attention mechanisms are correlation-based, they tend to encourage similar features, which may hinder diversity among outputs.\n\n- The quality of the results are good and material property maps remain sharp.\n\n- I also appreciate the extension to more complex scenes which does not look like a trivial addition.\n\n## Task definition\n- The formulation of the problem is a bit clumsy. For example, the phrase “we formulate PBR generation as a generative approach” (l.91) is redundant, PBR generation is inherently a generative task. And since the downstream task is not material decomposition, it's already framed as a conditional generation task by definition.\n\n- The authors claim that generating PBR maps “from scratch” helps alleviate the ambiguity of the task. First, “from scratch” is a somewhat odd choice of words, especially considering the model is not unconditional. More importantly, it's unclear how this approach resolves ambiguity. One could argue that the task is simply easier, since it does not rely on an RGB reference. In fact, any prediction in the parameter space can result in a plausible output under the BRDF analytic model. In contrast, I would argue that decomposition methods (conditioned on RGB images) tackle a more complex problem and suffer from inherent bias, precisely because they are forced to meet specific constraints. The task chosen in this paper, text-conditioned generation, is less constrained: the model is free to generate any plausible decomposition, as long as it aligns with a coherent scene. It’s not about explaining the properties of a specific image, but rather about sampling a plausible material configuration without regard to a particular appearance. In that sense, the comparison with decomposition methods is not entirely fair.\n\n## Novelty and scope\n- The authors claim: “For the first time, we directly generate PBR maps from text as input in a probabilistic diffusion process.” This is a rather bold statement. Indeed, text prompts have already been used for PBR generation in \"Diffusion-based G-buffer Generation and Rendering\" (Xue et al. arXiv 2025). In the context of single-material generation, this has been explored in works such as MatFuse (Vecchio et al., CVPR 2024), ControlMat (Vecchio et al., 2023), and DreamPBR (Xin et al., arxiv 2024).\n\n- The idea of modeling the joint distribution has also been addressed before, notably in ControlMat, although it targets single materials. In fact, they first train a text-conditioned model and then add a ControlNet on top. If joint learning of the distribution is indeed so beneficial, this naturally raises the question of whether adding a ControlNet, or any other form of image conditioning on the resulting model would in fact yield a more powerful decomposition. Compared to a model trained directly for decomposition from the start.\n\n## Related works\n- I am wondering whether the cross-task attention is similar to how Tune-A-Video (Wu et al., ICCV 2023) models attention between the input frames? I think it’s important to position this work with respect to such approaches, which also rely on cross-task mechanisms to model interactions. See also Material Picker (Ma et al., SIGGRAPH 2025) which uses a video diffusion model to learn interactions between PBR maps. There currently seem to be no discussion positioning this part of the contribution. Also, it would be valuable if the authors positioned their approach relative to the literature on cross-task attention mechanisms. This is common mechanism found in the multi-task literature: for example in MTFormer (Xu et al., ECCV 2022). \n\n- On line 201, I don’t think it is accurate to say the light direction is “obtained”; rather, it is chosen as the reflected direction with respect to the outgoing direction. It would be good to mention that a similar rendering loss has been applied in prior works, e.g., SVBRDF capture (Deschaintre et al., Siggraph 2018), as well as the fact that they enforce sampling of the reflected direction. \n\n## Qualitative results\n- Overall, the generations look good, but my concern is whether the model is simply outputting random properties that remain internally consistent within image segments. Since the model is conditioned on text prompts alone, what needs to be verified is whether it has actually learned the correct distribution of object properties: whether it understands which objects should be metallic or non-metallic, and what their reflectance characteristics should be (glossy or rough).\n\n- Based on the results in Figure 12, I'm not so convinced it does so very well. Plausible roughness values are difficult to assess and quite subjective. However, the metallic maps can be more easily judged. For instance, several object parts are labeled with high metalness values despite being clearly non-metallic: the horse’s tail, the guitar soundhole, the spacesuit helmet visors (both), the guitar player’s eyes, and the windows of the wooden cabin. Similarly, in Figure 13, the water and tree in the Taj Mahal scene are predicted as metallic, while the sword and key are assigned rather low metalness values.\n\n## Quantitative results\n- Regarding the metric in Table 1, I am not very convinced by the results of the user study (particularly for the rendered images, specularity assessment, and coherence with the input prompt). I notice that the uncertainty in the results is greater than the difference between the baseline and the proposed method. Therefore, I do not fully agree with the conclusions drawn in Section 5.1.\n\n- Regarding the FID metric: since the albedo maps are generated via text prompts, they are naturally not aligned with the ground-truth albedo annotations. As a result, the authors resort to using the FID score. However, for the FID evaluation, only 100 images are used for the ID set and 40 for the OOD set. Is this sample size truly representative for a metric like FID? I believe that such distribution-based measures typically require larger sample sizes to be reliable.\n\n## Minor comments:\n- On line 97, authors said they \"align them [each modality] by leveraging cross-intrinsic\". The term align is a bit vague; perhaps learning the joint distribution would be more accurate\n\n- On line 201, the sentence \"Then, we obtain the light direction as the reflectance \\omega_i\", shouldn't it be \"as the reflected direction \\omega_i\" instead?\n\n## Typos:\n\n- line 54: \"pysically\"\n- line 67: It is grammatically incorrect to use citation numbers within sentences like \"They typically model the diffusion process following [19] or [36]\". Same on line 84 with \"generate PBR textures of entire 3D scenes following [7].\" and line 108 with \"Unfortunately, existing datasets, such as [34, 71, 46]\".\n\n- Line 124: I find \"paired datasets for each intrinsic property\" to be imprecise\n\n- Line 129: Simplifying the grammatical structure would help fix the sentence \"Our method with utilizing a pre-trained image prior does not require extensive PBR datasets\"", "was_revised": true, "final_justification": "The paper addresses the problem of text-conditioned PBR generation using a pretrained generative model. It delivers high-quality results with sharp material property maps and successfully extends the approach to handle more complex scenes, which is a non-trivial achievement. The authors have addressed the various questions and concerns I raised in my review. In particular, they provided an extended evaluation of the FID scores and detailed their positioning relative to prior works. As mentioned in my response to the authors, I have therefore decided to raise my score.", "rebuttal": "## Related works\n\nThe work of Xue et al. (Diffusion-based G-buffer Generation and Rendering, arXiv 2025\\) addresses text-conditional complex PBR generation. They train a shared ControlNet for Stable Diffusion v2 and use a diffusion renderer for editability. In contrast, our method first learns a prior over the PBR properties independently, then aligns them with cross-intrinsic attention using a fixed renderer to ensure compatibility with standard rendering engines. We will include a thorough discussion; however, at the same time we want to highlight that this work is concurrent (almost appeared at the same time on arXiv as our submission).  \n\n### Single-material generation and capture\n\nSingle-material generation methods focus on texture generation; they cannot handle complex environments. \\`\\`SVBRDF capture” uses a feed-forward model with a similar rendering loss using the reflected viewing direction of a random surface point for lighting. Orthogonal to this direction, we aim to generate the PBR properties of complex scenes in image space to enable downstream applications. We introduce a rendering loss to the diffusion framework using a roughness-weighted importance sampling for the lighting direction. \n\n### Cross-frame attention\n\nWe discuss related approaches that use cross-frame attention in L77-80. Tune-A-Video similarly uses cross-frame attention, but as other multi-view image generation methods, they have the same modality across the \\`\\`frames\" (=batch elements); thus, they can share the LoRA weights. MTFormer applies cross-task attention on the dense prediction heads of a deterministic transformer model. In contrast, our method uses cross-intrinsic attention, where all the LoRAs are different, but still tuned together for a generative task. \n\nWe will include these discussions. \n\n## Comparison against Decomposition\n\nTo the best of our knowledge, there is no other text-conditional PBR map generation method that operates on complex, unconstrained scenes in image space; therefore, we compare to decomposition methods. We agree that PBR generation is less constrained than decomposition, and our results demonstrate this: direct PBR generation is better suited for high-quality PBR generation (Tab.1, Fig.6,7).\n\n## Additional Results\n\nWe extend our test set and re-evaluate all the baselines and our method. For in-domain experiments, we use the full InteriorVerse test set (2595 samples, as done in IID \\[29\\]). Evaluating the out-of-domain performance is challenging since there exists no large-scale PBR dataset of complex scenes from various domains. To evaluate one specific domain, we use the single-object G-Buffer ObjaVerse \\[Zuo et. al. ECCV 2024\\] dataset. We take 1000 samples from the diverse \\`\\`Daily-Used\" category. To show that these numbers are representative, we evaluate the FID with a different number of samples. The relative improvement over baselines align with the original FID numbers in the main paper and the metrics are consistent. Our method achieves better generalization to out-of-distribution domains. We are happy to include these detailed results.\n\n**\\# Samples vs FID on InteriorVerse**\n\n|  | 100 | 500 | 1000 | 1500 | 2000 | 2595 |\n| :---- | :---: | :---: | :---: | :---: | :---: | :---: |\n| Ours | 186.60 | 113.96 | 92.21 | 83.32 | 77.13 | 72.08 |\n\n**\\# Samples vs FID on G-Buffer ObjaVerse**\n\n|  | 100 | 300 | 500 | 700 | 800 | 900 | 1000 |\n| :---- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| Ours | 186.60 | 123.54 | 99.14 | 85.44 | 80.46 | 75.28 | 71.39 |\n\n**FID Metrics**\n\n|  | In-domain InteriorVerse | Out-of-domain G-Buffer ObjaVerse |\n| :---- | :---: | :---: |\n| IID | 78.77 | 98.77 |\n| RGBX | **61.36** | 90.12 |\n| ColorfulShading | 91.10 | 86.48 |\n| *IID+FLUX LoRA* | 103.36 | 79.29 |\n| *Ours w/ BRDF-sampling* | 75.26 | **68.87** |\n| *Ours w/ Image-cond* | 98.08 | 70.51 |\n| Ours | 72.09 | 71.39 |\n\n### Image-conditional IntrinsiX\n\nOur model can be made image-conditional in a similar fashion as for our downstream task of normal-conditioned PBR generation (Fig. 5, 15): we add the RGB image as input and apply input dropout during fine-tuning (as in L228). Our image-conditional method produces sharper results and outperforms decomposition baselines on OOD data (see *Ours w/ Image-cond* above). This demonstrates the benefits of learning a generative prior over PBR modalities.\n\n### Lighting Direction Sampling\n\nWe sample light directions that maximize specular highlights to provide strong gradients about reflectance. We train a BRDF-sampled variant (Disney model) for more diverse lighting directions (denoted as *Ours w/ BRDF-sampling* above*)*. This yields slightly worse results on the in-domain dataset, but improves generalization, showing that lighting direction sampling is crucial for our task.\n\n### Model Inconsistency\n\nWe re-trained the IID \\[29\\] baseline using the FLUX backbone, both with full fine-tuning (as in IID) and with LoRA (matching our model, 300M trainable parameter). Stronger backbone improves results. Full fine-tuning performs better in-domain (InteriorVerse) but generalizes worse (*IID+FLUX Full*). LoRA helps toretain the original prior and improves generalization (*IID+FLUX LoRA*). While sample sharpness improves, IID still shows artifacts (e.g., washed or misaligned textures). We believe image-conditioned models struggle with OOD generalization due to over-reliance on the input image, whereas our method learns a prior over PBR distributions that generalizes better. We are happy to include an extensive version of these experiments. \n\n**IID \\+ FLUX**\n\n|  | ID | OOD | In-domain InteriorVerse | Out-of-domain G-Buffer ObjaVerse |\n| :---- | :---: | :---: | :---: | :---: |\n| IID | 188.34 | 224.83 | **78.77** | 98.77 |\n| IID+FLUX Full | **178.05** | 245.41 | 127.41 | 83.58 |\n| IID+FLUX LoRA | 185.11 | **224.27** | 103.36 | **79.29** |\n\n## Rough and Metal Results\n\nWe agree that evaluating material properties alone is challenging, which is why we include renderings and relighting videos and also a user study assessing specular and rendering quality (Tab. 1). The first four rows of Fig. 12 show our first-stage model, trained without rendering loss or cross-intrinsic attention, hence the predictions are incoherent and do not understand the rough and metal properties well. After stage two (bottom four rows), we can see clear improvements. The horse's tail and the guitar soundhole are not metallic anymore (guitar strings correctly remain metallic). The helmet visor, guitar player's eye and the windows of the cabin should have glass-like reflections, which can be best matched with metallic (mirror-like) reflection. Training with more diverse and high-quality PBR maps should improve the generalization. \n\n## User Study\n\nSince PBR quality is difficult to assess, user studies naturally exhibit a large uncertainty. However, we rate each method separately, and uncertainty of the different methods are close to each other. Thus comparing the mean values is fair.\n\n## Details on LoRA Finetuning\n\nEach LoRA has \\~75M params (0.625% of the base model), giving a total of 225M params. It is not necessary to fine-tune all layers, we insert LoRAs only into attention blocks, augmenting the to\\_q, to\\_k, to\\_v, and to\\_out layers. While we explored skipping some blocks to reduce parameters, we found that cross-intrinsic attention performs best when LoRA is applied throughout.\n\n## Dataset Size\n\nLoRA finetuning can adapt T2I models effectively with few high-quality samples (see LoRA paper Sec. F.3). However, available larger PBR datasets are domain-specific (indoor). While images might become more diverse, the prompts do not vary that much, so the LoRA links style mainly to that domain. Therefore, out-of-distribution prompts (Fig. 9\\) don’t adopt the new style and remain shaded RGB images. The ideal solution is a large PBR dataset, with diverse, complex domains, which currently does not exist. Adding a new domain to the training improves the generalization, as discussed above (Rough and Metal Results).\n\n## Rendering\n\nDuring training, we process only a single diffusion timestep, avoiding backpropagation through all steps, which enables training with a rendering loss. Starting from a noisy sample, a single-step (x0) prediction produces clean PBR maps that receive useful gradients when rendered. At inference, multiple lights can be rendered using a fixed forward renderer, as demonstrated in our scene texturing results with complex lighting (supplementary video at 1:25).\n\n## SDS\n\nFlat colors often occur in SDS methods due to averaging predictions. To avoid this, high guidance scales are used but cause oversaturation (ProlificDreamer \\[Wang et. al.\\] Fig.3). This is unacceptable for rough/metal as it alters reflectance. Thus, we use lower guidance and apply an observation-frequency weighted loss and normalized flow direction to preserve details (L983-990). Other works that overcome this limitation can be additionally applied to our downstream application (e.g., LoRA adaption in ProlificDreamer, consistent noising in SyncDreamer \\[Liu et. al.\\]).\n\n## Limitations\n\nWe’ll expand Sec. C and add a figure showing failure cases (flat decompositions, ambiguous metallic, noisy normals). T2I models have shown value as generic image priors for numerous tasks, e.g. improvements in intrinsic image decomposition. Our goal is to get a similar prior for PBR maps, i.e. focusing on broad domain coverage, and not on exact prompt precision.\n\n## Writing\n\nWe adopt the suggestions to improve the clarity of our exposition/method details and fixed the typos in L91,97,201,124,129,201.\n\n## Formatting\n\nWe will fix the formatting issues. Since other NeurIPS papers (e.g., “Light Field Networks” \\[Sitzmann et al., NeurIPS 2021\\]) use half-column figures, we believe this is allowed."}, {"review_id": "LgveSNE3It", "reviewer": "Reviewer_1FNt", "summary": "This paper proposes a method to generate intrinsic properties images (including albedo, roughness & metallic, and normal) from a given text description. The method uses the pretrained FLUX.1-dev model as the generative backbone and finetunes it using 3 separate LoRAs for each intrinsic property. After separate finetuning, the method finetunes the LoRAs jointly using a cross-intrinsic attention and a re-rendering loss with importance-based lighting direction sampling. The paper showcases the applications of the proposed method and conducts extensive experiments to show the superiority of the proposed method against baseline methods and ablation studies to show the effectiveness of the proposed techniques.", "questions": "Corresponding to the weaknesses above, I summarize the questions below:\n1. The paper uses FID to compare the quality of the generated albedo image across different baselines. Is it a common practice to use 100 and 40 images to compute FID? If not, why not use more images?\n2. Could the authors provide a more in-depth discussion of \"in-distribution\" and \"out-of-distribution\", as the ability to handle \"out-of-distribution\" is stated as a major difference between the proposed method and baseline methods (L33, L44, L90, L297)?\n3. Why \"the inherent ambiguity of decomposing an RGB image into its intrinsic properties makes it difficult to match the PBR distribution for out-of-domain samples\" (L90, L298) and how does the proposed method overcome this difficulty using an even more ambiguous task?\n4. Could the authors compare the usefulness between the proposed intrinsic image generation task and the traditional intrinsic image decomposition task?\n5. What is the possible outcome of using the proposed techniques for intrinsic image decomposition (i.e., changing the input from text to image)?\n\nI will raise my rating once the authors provide clarifications to these questions.", "limitations": "Regarding the rendered image: The method does not discriminate between foreground and background regions. Sometimes relighting the background region like a plain surface feels weird. Also, the renderer cannot render cast shadows as the depth information is absent. These can be left for future work to make the rendered image more realistic.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The proposed two-stage training strategy with the cross-intrinsic attention (though this attention is alike the commonly-used cross-view or cross-frame attention and lacks true novelty) and the re-rendering loss is somewhat novel: Though the re-rendering loss is vastly used in previous CNN-based intrinsic image decomposition methods such as [33], it is the first time (as far as I know) that it is used in a diffusion-based framework. The proposed importance-based lighting direction sampling for training is also interesting and effective.\n2. The paper proposes an efficient pipeline that takes only 26h to train on a single A100 GPU and 12s to generate an image on a single A6000 GPU, and also gives state-of-the-art results in the task of intrinsic image generation (though it is a new task proposed by this paper).\n3. Extensive quantitative and qualitative comparisons and ablation studies.\n4. The authors provide an entire section to showcase the applications of the proposed method, including relightable image generation and PBR texturing.\n5. Good presentation and easy-to-follow paper.\n\nWeaknesses:\n1. The quantitative evaluation may not be very rigorous. \n  - Firstly, the paper uses FID to compare the quality of the generated albedo image across different baselines. However, the testing dataset contains only 100 (for in-distribution) + 40 (for out-of-distribution) images, which are far less than the common requirements of computing FID, i.e., 10,000 images. The small amount of images for computing FID may render the evaluate unreliable. Is it a good practice to use 100 and 40 images to compute FID? If not, why not use more images?\n  - Secondly, I am a little bit confused about the definition of \"in-distribution\" and \"out-of-distribution\" in L263-L272. While the sampled images are \"in-distribution\" regarding the proposed method as it is trained on InteriorVerse, are they still \"in-distribution\" regarding the comparing methods, including IID, RGBX, and ColorfulShading, as these baselines may not be trained on InteriorVerse? Could the authors provide a more in-depth discussion of \"in-distribution\" and \"out-of-distribution\", as the ability to handle \"out-of-distribution\" is stated as a major difference between the proposed method and baseline methods (L33, L44, L90, L297)?\n2. The statement about the inherent ambiguity of intrinsic image decomposition is not persuasive. Intuitively, generating PBR images from mere text inputs is more ambiguous than from an input shaded RGB image, as even the scene structure is not determined yet. So I fail to understand why \"the inherent ambiguity of decomposing an RGB image into its intrinsic properties makes it difficult to match the PBR distribution for out-of-domain samples\" (L90, L298) and how does the proposed method overcome this difficulty using an even more ambiguous task.\n3. The above weakness also motivated me to question the proposed task, intrinsic image generation. I think in essence it is similar to the traditional intrinsic image decomposition, except that the input is changed to a text description instead of an image. Personally, I prefer to use an image as the input in many cases as I may want to edit an exisiting image instead of generating a new image from scratch. Thus, I think the proposed task is not very useful (the texturing task in Section 4 is also actually achieved by finetuning the model to enable it to condition on an image). Could the authors compare the usefulness between the proposed intrinsic image generation task and the traditional intrinsic image decomposition task? Also, what is the possible outcome of using the proposed techniques for intrinsic image decomposition (i.e., changing the input from text to image)?\n\nTrivia:\n1. Typos:\n  - L129: \"with utilizing\" -> \"with\" or \"utilizing\". \n  - L274: \"A-PP\" -> \"A-PQ\".\n  - L337: \"the preserve\" -> \"to preserve\".\n2. Eq. (3) seems incorrect. E.g., the BRDF evaluation should be related to the incoming lighting direction $\\omega_i$ and the cosine term should be related to the surface normal.", "was_revised": true, "final_justification": "The rebuttal thoroughly addresses my question 1 (about the size of evaluation dataset), question 2 (about the concrete definition of ID and OOD), question 4 (about the application scenario of text-to-PBR image compared to image-to-PBR image methods), and question 5 (about using the proposed method for intrinsic image decomposition). I really appreciate the efforts made by the authors during the rebuttal period that largely increase the dataset set by more than 10 times and gives comprehensive additional experimental results. I will raise my rating to \"Accept\" as promised.", "rebuttal": "## Motivation for Image-Space PBR Priors\n\nText-based generative models can act as a generic prior over images, which has shown great value for numerous downstream tasks: improved intrinsic image decomposition(IID \\[29\\], RGBX \\[67\\]), depth estimation (Marigold \\[Ke et al CVPR 2024\\]), etc. Our goal is to establish a similar *generic prior* for PBR materials to support downstream applications like UV-based texturing. \n\nThe key challenge of the field of PBR generation is the lack of large-scale, diverse datasets of PBR UV textures. Unlike RGB data, material datasets are scarce, and crucially, consistent UV parametrization (mapping from object coordinates to texture coordinates) across assets practically does not exist. This makes training direct generative models in UV texture space infeasible.\n\nInstead, we learn a prior in image space. To distill image-space information into texture-space, we use SDS, which aggregates image-space signals from multiple views. This enables PBR UV texture generation for 3D assets, even though trained with relatively small amounts of image-space PBR data. In contrast, decomposition methods require an image as input and it is not obvious how one would synthesize consistent PBR UV textures for novel 3D assets. \n\n## Inherent Ambiguity\n\nWe believe that intrinsic image decomposition methods are constrained to match the appearance of the input image. Thus, the model can learn to rely more on the input, instead of focusing on staying in the PBR distribution; therefore, facing an out-of-distribution input image poses a challenge to these models to produce faithful PBR maps, corresponding to the respective distributions.\n\nL90 and L298 are also grounded on our empirical observations of our experiments. Concretely, the decomposition baselines perform well on in-domain samples (indoor scenes), but struggle to generalize out-of-distribution. In contrast our method does not have such limitations (see Tab. 1, Fig.6,7). Thus, we argue that decomposition is inherently more ambiguous than our task of directly generating the PBR maps.\n\n## ID vs OOD\n\nOur diffusion-based baselines are only trained on indoor rooms, partially or fully on the InteriorVerse dataset. These methods perform well on indoor scenes, since this is in-domain for them. Our goal is to generate unconstrained scenes in diverse domains (e.g. single objects, composited scenes, outdoor, nature, etc.); therefore, we curated a dataset of complex scenes from various domains (OOD dataset). Since the baselines are image-conditional, when facing an out-of-domain input, their performance decreases. In contrast, our method does not depend on an input image, but rather learns the distribution of the PBR properties while maintaining the prior knowledge of the base model; thus, it can better generalize, as our qualitative and also quantitative results show. \n\n## Additional Results\n\nWe extend our test set and re-evaluate all the baselines and our method. For in-domain experiments, we use the full InteriorVerse test set (2595 samples, as done in IID \\[29\\]). Evaluating the out-of-domain performance is challenging since there exists no large-scale PBR dataset of complex scenes from various domains. To evaluate one specific domain, we use the single-object G-Buffer ObjaVerse \\[Zuo et. al. ECCV 2024\\] dataset. We take 1000 samples from the diverse \\`\\`Daily-Used\" category. To show that these numbers are representative, we evaluate the FID with a different number of samples. The relative improvement over baselines align with the original FID numbers in the main paper and the metrics are consistent. Our method achieves better generalization to out-of-distribution domains. \n\nWe provide three additional experiments to better support our claims (as also requested by the reviewers). \n\n* *Ours w/ BRDF-sampling:* we sample more diverse lighting directions using BRDF-sampling during our rendering loss to provide more gradient signal about the BRDF behavior. This variant slightly improves our generalization results, showing the importance of lighting direction sampling.   \n* *IID+FLUX LoRA:* we re-train the IID baseline with FLUX as backbone and use LoRA layers similar to our method. This improves the fidelity, but this still does not reach the quality of our method, showing that learning a generalizable prior over PBR distributions is beneficial for generalization. For more details, we refer to our rebuttal to reviewer 48oZ.   \n\nWe are happy to include these detailed results.\n\n### Image-conditional IntrinsiX\n\nOur model can be made image-conditional in a similar fashion as for our downstream task of normal-conditioned PBR generation (Fig. 5, 15): we add the RGB image as input and apply input dropout during fine-tuning (as in L228). Our image-conditional method produces sharper results and outperforms decomposition baselines on OOD data (see *Ours w/ Image-cond* below). This demonstrates the benefits of learning a generative prior over PBR modalities.\n\n**\\# Samples vs FID on InteriorVerse**\n\n|  | 100 | 500 | 1000 | 1500 | 2000 | 2595 |\n| :---- | :---: | :---: | :---: | :---: | :---: | :---: |\n| Ours | 186.60 | 113.96 | 92.21 | 83.32 | 77.13 | 72.08 |\n\n**\\# Samples vs FID on G-Buffer ObjaVerse**\n\n|  | 100 | 300 | 500 | 700 | 800 | 900 | 1000 |\n| :---- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| Ours | 186.60 | 123.54 | 99.14 | 85.44 | 80.46 | 75.28 | 71.39 |\n\n**FID Metrics**\n\n|  | In-domain InteriorVerse | Out-of-domain G-Buffer ObjaVerse |\n| :---- | :---: | :---: |\n| IID | 78.77 | 98.77 |\n| RGBX | **61.36** | 90.12 |\n| ColorfulShading | 91.10 | 86.48 |\n| *IID+FLUX LoRA* | 103.36 | 79.29 |\n| *Ours w/ BRDF-sampling* | 75.26 | **68.87** |\n| *Ours w/ Image-cond* | 98.08 | 70.51 |\n| Ours | 72.09 | 71.39 |\n\n## Writing\n\nWe fixed the typos as suggested and clarify eq. (3): $$\\\\hat{\\\\mathbf{I}} \\= f(\\\\omega\\_o; \\\\omega\\_i; \\\\hat{\\\\mathbf{X}}\\_0) \\\\cdot L\\_i \\\\cdot(\\\\hat{x}\\_n^T \\\\omega\\_i)$$"}]

Author rebuttal:

Meta-review: This submission introduces a novel method for generating high-quality, text-conditioned PBR maps using a diffusion-based framework. The reviewers initially raised concerns about the novelty of cross-intrinsic attention, the practicality of text-based conditioning, and the robustness of quantitative evaluations (e.g., limited FID sample sizes). The authors provided a comprehensive rebuttal, clarifying that their approach uniquely combines independent PBR priors with cross-intrinsic attention and a rendering loss to ensure semantic coherence. The expanded evaluations (e.g., testing on 2595 in-domain and 1000 out-of-domain samples) demonstrated image-conditional variants and justified design choices (e.g., LoRA rank, BRDF-sampling). While some weaknesses remain (e.g., noisy normals, reflections), the method’s technical soundness, strong empirical results, and potential for applications like relighting and texture generation were widely acknowledged. After discussion, all reviewers upgraded their ratings, recognizing the paper’s contributions. Acceptance is suggested.
Decision: Accept (poster)