LiTo: Surface Light Field Tokenization
Authors: Jen-Hao Rick Chang
Xiaoming Zhao
Dorian Chan
Oncel Tuzel
Venue: ICLR 2026 Poster (2026)
Keywords: generative model
3D vision
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We propose a 3D latent representation that jointly models object geometry and view-dependent appearance. Most prior works focus on either reconstructing 3D geometry or predicting view-independent diffuse appearance, and thus struggle to capture realistic view-dependent effects. Our approach leverages that RGB-depth images provide samples of a surface light field. By encoding random subsamples of this surface light field into a compact set of latent vectors, our model learns to represent both geometry and appearance within a unified 3D latent space. This representation reproduces view-dependent effects such as specular highlights and Fresnel reflections under complex lighting. We further train a latent flow matching model on this representation to learn its distribution conditioned on a single input image, enabling the generation of 3D objects with appearances consistent with the lighting and materials in the input. Experiments show that our approach achieves higher visual quality and better input fidelity than existing methods.

Published as a conference paper at ICLR 2026
LITO: SURFACE LIGHT FIELD TOKENIZATION
Jen-Hao Rick Chang Apple
Xiaoming Zhao
Dorian Chan
Oncel Tuzel
Figure 1: LiTo tokenizes surface light fields into a latent representation. It models 3D geometry and view-dependent appearance such as specular reflection. The figure shows reconstructions (first columns) and Langle-imae--e results (last two columns). Mesh credit: Anthony Schmidt (2016); @ @anyybeeast (2021); LLOYDO (2019); brysew (2015); Osho (2018). See more on the project page.
ABSTRACT
We propose 3D latent representation that jointly models object geometry and view-dependent appearance. Most prior works focus on either reconstructing 3D geometry or predicting view-independent diffuse appearance, and thus struggle to capture realistic view-dependent effects. Our approach leverages that RGB-depth images provide samples of a surface light field. By encoding random subsamples of this surface light field into a compact set of latent vectors, our model learns to represent both geometry and appearance within unified 3D latent space. This representation reproduces view-dependent effects such as specular highlights and Fresnel reflections under complex lighting. We further train latent flow matching model on this representation to learn its distribution conditioned on a single input image, enabling the generation 3D objects with appearances consistent with the lighting and materials in the input. Experiments show that our approach achieves higher visual quality and better input fidelity than existing methods.
INTRODUCTION
The world is filled with objects that vary widely in shape and material. Some are smooth and reflective, while others are rough, detailed or even translucent. Even familiar objects can appear differently from different viewpoints as light creates reflections and subtle color changes across their surfaces. Capturing this richness is important for building generative models of realistic objects. To do so, we need representations that can model both the underlying 3D geometry of real-world objects as well as their view-dependent appearance.
However, today in machine learning, most existing 3D representations tackle only part of this problem. Many methods are designed to capture geometry alone (He et al., 2025; Li et al., 2025a; Chang
Indicates equal contribution. See Sec. G for detailed breakdown of individual contributions.

---

Published as a conference paper at ICLR 2026
et al., 2024), aiming to recover the overall shape of objects. Other approaches (Xiang et al., 2025) include appearance information, but treat it as view-indepee new-independent diffuse color. As a result, these models struggle to represent view-dependent effects such as reflections, highlights, subtle changes in shading that are important for realistic appearance.
In this work, aim to model both the 3D geometry and the view-dependent appearances of objects. We introduce a 3D latent representation that encodes a surface light field into a compact set of latent vectors. In summary, rather than encoding geometry and color only, e.g. with an input RGB point cloud, we additionally input viewing direction along with surface points and color, to capture how realistic materials change appearance with angle. Because a full surface light field contains highly dense information, we instead provide random subsample of the surface light field- captured from RGB-depth multiview images-and and rely on an encoder to interpolate the missing samples. This approach allows the model to reproduce view-dependent effects such as highlights and Fresnel reflections, that can be visualized via a decoder that outputs Gaussian splats with higher-order spherical harmonics (Kerbl et al., 2023). We evaluate our method by comparing its reconstruction quality against the state-of-the-art 3D latent representations (Xiang et al., 2025; Li et al., 2025a; He et al., 2025; Chen et al., 2025b; Chang et al., 2024), and find that modeling these view view-dependent effects improve visual quality without significant degradation in geometric accuracy.
Building the proposed representation, train latent flow matching model that learns the distribution of our 31) latent representations conditioned on a single input image. The generative model learns infer both geometry and riew-dependent appearance from images under different lighting conditions. Given an input image, the model generates full 3D object whose shape matches the object in the image from the input viewpoint and whose appearance reflects the lighting and view-dependent material properties present in the input. Our approach connects 2D observations to 3D object generation, enabling controllable synthesis of realistic, hie-ddependent materials from diverse image inputs.
Our work makes the following contributions.
We introduce 3ID latent representation that captures both geometry and view-dependent appear- ances by encoding surface light field information into a compact set of latent vectors.
We design a training framework that jointly supervises geometry and appearance using random subsamples of surface light field data from RGB-depth multiview images, enabling the model to reproduce view-dependent effects such as highlights and fresnel reflections via Gaussian splats with higher-order spherical harmonics.
We develop latent flow matching model that learns the distribution of these latent representations conditioned on images, allowing the generation of full 3D objects whose appearances reflect the lighting and materials in the input.
Together, these components enable more accurate reconstruction and better separation of geometry and appearance than existing methods.
2 RELATED WORKS
growing number of recent approaches have explored learning latent 3D representations. In Tab. SI, we summarize and compare their properties, including geometry and appearance modeling mech- anisms, data requirements, latent dimensionality, encoder inputs, and training sets. For clarity, we review geometry-only approaches and those that jointly model geometry and appearance separately.
Geometry-only latent. A large body of work focuses on latent representations that model geometry alone. These approaches differ primarily in the underlying 3D signal they encode. PointFlow (Yang et al., 2019), ShapeGF (Cai et al., 2020), and Shape'Token (Chang et al., 2024) learn to model 3D surfaces as 3D distributions. TripoSG (Li et al., 2025a), and Hunyuan3D (Zhao et al., 2025), instead model shapes occupancy or signed distance functions (SDF). Direct3D (Wu et al., 2024). XCube (Ren et al., 2024), LT3SD (Meng al., 2025), and Make-A- Ahhape (Hui et al., 2024) embed geometry into dense sparse voxel grids containing occupancy or SDF values at vertices. While grid-based methods offer structured latents, they face inherent trade-offs between spatial resolution and memory efficiency. A common limitation
JDShape2VecSee (Zhang et al., 2023), CLAY (Zhang et al., 2024),
2

---

Published as a conference paper at ICLR 2026
5D seano
encoder
key, value
flow-matching
supervise geomeity
velucity
voud sel atth.
linear
quary
view-depenn niw-dependennt 3D Gaussian decoder
supervise view-dependent radiance
latent
surface light field samples ((surface loc. view dit,, colot))i-I.N
Figure 2: Overview of the 3D latent representation. Given samples of the surface light field of the scene, we learn a latent representation that reconstruct the full surface light field information. The encoder (pink block) condenses input information into the latent representation. We jointly supervise the latent representation to contain full 3D geometry and view-dependent radiance information beyond the input samples. In the architectures, design localized attention pattern improve efficiency and support million input tokens.
when relying on occupancy or SDE, however, is the reliance on significant preprocessing of the training data. Many methods require watertight meshes (Zhang al., 2023; 2022: 2024), expensive mesh-to-field conversions, or optimization- based radiance-fiee field fitting in order to define consistent supervision signals. Moreover, these methods capture only geometry, without appearance, texture, or view-dependent effects.
Geometry and appearance latent. More recently, smaller set of works has begun extend latent 3D representations beyond pure geometry to also encode appearance. Two of the most relevant are 3DTopia-XL (Chen et al., 2025b) and TRELLIS (Xiang et al., 2025).
3DTopia-XL XL introduces the PrimX representation, where each primitive encodes not only geometry through signed distance but also material properties such RGB color, roughness, and metallicity. This design allows the model to generate textured 3D assets that are ready for physically based rendering. However, PrimX requires optimization step construct the primitive representation from meshes before training, making data preparation more demanding.
TRELLIS introduces Structured LATent (SLAT) representation: sparse voxel grid fused with dense multiview visual features extracted by a foundation vision model (DINOv2) to provide both geometry and appearance cues. Given the coarse geometry an object, SLAT is constructed by averaging projected DINOv2 features from all input views. The model decodes SLAT into multiple output 3D formats, including 3D Gaussians, meshes, and radiance fields. To handle the sparsity SLAT efficiently, TRELLIS employs transformers with windowed attention and sparse 3D convolution. and is trained at scale on roughly 500K assets from Objaverse-XL and related datasets.
TRELLIS has several limitations relative to our approach. First, SLAT requires coarse occupancy information to be known in advance, so generation is performed in two stages, whereas our latent directly encodes complete object information and supports single-stage generation. Second, TRELLIS encodes only indeenndeenddntt appearance: multiview features are meeln-ooole,, discarding angular variation and preventing modeling of view-dependent effects. Finally, TRELLIS generates objects in a canonical coordinate system (i.e., their dataset orientation), which necessitates post- post-processing to align them with input images. This restriction arises from its reliance preconstructed axis aligned voxel grids, which makes coordinate transformations like rotation during training difficult. In contrast, model takes points input, which allows apply coordinate transformations during training, ensuring generated objects are consistently oriented with respect to the input view (see Fig. and 6).
METHOD
3.1 PRELIMINARY AND NOTATION
The surface light field jointly models both the 3D surfaces of scene as well as the outgoing radiance from each point on the surface toward every viewing direction. In theory, if the surface light field is perfectly represented, any image captured by camera at any arbitrary location and
3

---

Published as a conference paper at ICLR 2026
orientation can be directly reconstructed (Wood al., 2000). We represent the surface light field a 5D function f(x, d) TR3 x g RR, where E 0Ω is any 3D location on surfaces 02, e 5² [v/v E IR, ||v|| = 1} is the viewing direction, and E R is the color of the outgoing radiance from x toward d.
We use bold lowercase symbols (e.g., v) to denote vectors, bold lowercase symbols with hats (e.g., v) for unit-norm directions, capital letters (e.g., A) for matrices or transformations, and calligraphic symbols (e.g., S) for sets.
3.2 TOKENIZER OVERVIEW
Our goal is to learn a 3D latent representation that models the surface light field of an object-centric scene with a compact set å 1 where S, E IR' is a d-dimension latent vector. Fig. 2 shows an overview of our latent representation. Our encoder outputs S after taking N samples of the surface light field defined in the following as input:
X [x,, d, c, = ((x,, d
where x,, di C,, and f(--)) are defined in Sec. 3.1.
To learn meaningful representation of the surface light field, must supervise both the decoded 3D geometry as well as view-dependent radiance. A trivial solution would utilize an autoencoder formulation that directly reconstructs the input X' However, in practice we only have sparse, discrete samples of the surface light field (e.g., as rendered from multiview images of a training object), and thus such an approach may not meaningfully represent the entire continuous function É. Thus, rather than directly supervising with the surface light field, we instead opt for indirect supervision with carefully-designee loss functions on decoded geometry and view-dependent appearance (as well as the regularization in Sec. E.4):
Geometry supervision. We utilize prior work (Chang et al., 2024), which models 3D surfaces as 3D probabilistic density function that is aligned with the actual surfaces via flow matching. This formulation enables us to model 3D surfaces beyond the imput 3D locations. Specifically, the latent S trained to parameterize 3D distribution P(x(S) that approximates dirac delta function lying on 3D surfaces in the scene, i.e., P(x[S) R 8(x E a2). The flow matching formulation also optionally allows us sample p(x[S) and get point cloud lying on surfaces during inference, and zero-shot estimate surface normals. The loss function follows that used by Chang et al. (2024):
Cyco (
(e
(X(x;; t) - (x -ee-el =)]l dt,
where 0 is all parameters in the encoder and the decoder, t is the fow-matching time, (O, 1) is the uniform distribution between 0 and 1, is noise sampled from standard normal distribution, Vo(xi; is the flow-matching decoder that estimates the velocity at x, = L- + (1 (1- () 6, and x is sampled from the surface light field.
View-dependent radiance supervision. The supervision of the view-dependent radiance is through rendering multi-view images. Specifically, we convert the latent S into a of 3D Gaussians, which models view- -ew-deeendent color by spherical harmonics, and we render the 3D Gaussians from random viewpoints and compare with ground -truth images. The loss is
Cradiance (0) EH, Igull + > lpips (g))
where Test = Render 0), H, E) is the rendered image from 3D Gaussians at camera pose H. and intrinsic E, 1et = Render r (object, H, E) the ground-truth image, D is the Gaussian decoder that will be detailed below, D(S, O) are the estimated 3D Gaussians given the latent S and low-resolution sparse occupancy grid O constructed from the sampled point cloud or an occupancy estimator, and 0 is all parameters in the encoder and the decoder. In all experiments, we use X = 0.2.
In the rest of this section, we discuss the architectures for our surface light-field encoder, geometry decoder and Gaussian decoder in more detail.
3.3 ENCODER
We first describe how we sample surface light field to obtain the input to the encoder and the samples for the loss in Eq. (2). Then we detail our encoder architecture
4

---

Published as a conference paper at ICLR 2026
Input. To sample from the surface light field ((x, d) in Eq. (1), need to sample random surface locations and view directions. We achieve this by densely rendering multi-view RGBD images. Since focus object-centric scenes, the cameras are placed uniformly object. The surface location can be obtained by baoe-projecting the depth map, view direction d, is derived from the pinhole camera model, and C; from the pixel color'. This operation densely samples both the surfaces and viewing directions and returns X' = {(x,, d, c,) in Eq. (1).
sphere surrounding the
In our experiments, we box-normalize the scene [-1, 1], and we render 150 images of resolution x036x I 1036 with 40 degree field of view, uniformly on sphere of radius 3.5. This provides 160 million samples of light field introduced in Sec. 3.1, of which we randomly sample N as our input to the encoder and the rest to serve as the ground-truth to supervise Eq. (2).
Architecture. We use Perceiver IO (Jaegle et al., 2022) as our encoder, which is widely used in prior latent 3D representations (Zhang et al., 2023; Chang et al., 2024; Li et al., 2025a). The encoder contains cross and self attention blocks, and the number of initial queries of the first cross attention block determines the number of output latent tokens, i.e., k 8192 in case discussed in Sec. 3.2. The output of the Perceiver 10 is passed to a linear layer to reduce the latent dimension to d = 32. Our latent S is thus a set of k tokens of d dimension (see Sec. 3.2).
To capture enough information from light field introduced in Sec. 3.1, we use N = 20 (~1 million) samples input. However, the large number of input makes the typical cross atten- tion in Perceiver 1O computationally expensive. We are inspired by the non-overlapping patchi- fication in Vision Transformers (Dosovitskiy al., 2021), which converts dense pixels into coarse tokens. Instead of using a convolution layer to aggregate information from individual 16 x 16 patches into tokens, use cross attention. However, our inputs are scattered points on 3D surfaces instead of pixels on a regular grid, and it is non-trivial to patchify 3D surfaces. We design an approximation of 2D patchification on 3D sur- faces with K-nearest neighbor. Specifically, given the input samples X' in Eq. (I), we first randomly select samples as the query Q to the first cross attention layer, similar to Zhang et al. (2023). The number of samples is equal to the number of latent tokens k. To patchify 3D surface, for each sample x E X' we find its closest point Q in terms of le distance of x and assign the index of the closest point the sample. Finally, during the cross attention, a query only attends to input samples that have its index. This operation can be implemented by standard libraries like xformers (Lefaudeux et al., 2022) or PlashAttentiin (Dao, 2024).
query;
key/value for query;
Figure 3: 3D patchification
An illustration is shown in Fig. 3. Note that this is an approximation because we use é2 distance of x instead of geodesic distance. Thus, when there are more than one surface lie in the neighborhood, the query will attend across surfaces. As & distance is much faster to compute than the geodesic distance, we think is a good trade-off.
For self attention, we use voxel- attention mechanism. Specifically, tokens that lie within the same voxel in predefined coarse grid attend to each other, and the coarse voxel grid shifts by a half cell width every layer. Unlike TRELLIS (Xiang et al., 2025), whose tokens lie a voxel grid, our tokens have continuous coordinates and are not grid-aligned We use a voxel grid only to organize self-attention. Overall, the encoder has 59.2 million parameters (see Fig. S2). Together with decoders below, the model is trained with 256 batch size for 90k iterations on 64 GPUs for days.
3.4 DECODER
Flow-matching velocity decoder. We utilize the same flow dow-matching velocity decoder used by Chang et al. (2024). Specifically, takes the latent S, 3D location, and flow-matching time as input, and it predicts the flow-matching velocity at the 3D location. To ensure we model a 3D distribution, i.e., S)) the decoder processes each 3D point independently (only cross attention and piint-wise operations used). The decoder has 8.8 million parameters. See details in Fig. S3
We assume the depth map measures the distance to the first intersection point of the scene, regardless of transparency. For example, in blender, this he achieved by setting the alpha threshold be 0.
5

---

Published as a conference paper at ICLR 2026
tripoSG
3DTopia-XL
trellis
ourss
ground truth
Figure 4: Reconstruction results on various lighting conditions. Boxes ground-truth highlight specular and Fresnel reflection. Please refer to Tab. for quantitative results. Mesh credit: Digital- Souls (2019); 3Dji (2025); Virtual Museums of Matopolska (2020).
View-dependent Gaussian decoder. Similar to our encoder, use Perceiver IO architecture (Jae- gle et al., 2022) for our Gaussian decoder. We use a low-resolution sparse occupancy grid for our initial queries, and cross attend the predicted latent S. We use small MLP output 64 3D Gaussians for each occupied voxels (see Sec. E.3). Unlike past work that only uses Gaussians with view-independent color (Xiang al., 2025), our decoder predicts Gaussians of spherical harmonics degree 3 for viwwddependent radiance. We observe that different harmonic degrees encode distinct appearance characteristics (see Sec. F.1). The decoder has 77.3 million parameters (see Fig. S4).
At training time, we use ground-truth occupancy for the decoder queries, like recent work leveraging structured latent representations (Xiang et al., 2025; He et al., 2025; Wu al., 2025). After learning the representation, we can either points sampled from the aforementioned flow-mate matching geometry decoder or alternatively train downstream occupancy decoder (see Sec. E.5 and Fig. S5), to directly predict sparse occupied voxels from the encoded latent. Thus, at generation time, our approach does not require a second generative model to predict occupancy as done in structured latent- based approaches (Xiang et al., 2025; He et al., 2025; Wu et al., 2025), simplifying the overall pipeline.
3.5 GENERATIVE MODEL
To demonstrate our latent representation, we train flow-matching model that generates 3D latents conditioned on an image of an object. We rely on a standard Diffusion Transformer (DiT) architec- ture (Peebles & Xie, 2023), with a zero-initialized learnable positional encoding for each latent token. The input image is encoded by DINOv2-large image embeddings (Oquab et al., 2024) and a learnable patchification layer. While we originally considered using more explicit camera geometry encoding, e.g., Plucker ray embeddings, we found in practice that such an approach reduced overall performance (see Tab. S6 for an ablation). In total, the model has 623 million parameters (see Fig. S7).

---

Published as a conference paper at ICLR 2026
TRELLIS
Ours
TRELLIS
Ours
Figure 5: Single image to 3D results. The input image is shown at the center of each set with black border. The rendering at the input view is shown with the input image. Please refer to Tab. 3 for quantitative results. Mesh credit: Eleanie (2025); Rigsters (2017); 3d-coat (2015); 3Dji (2025).
For each training sample, we rotate the world coordinate system the input view's camera pose is set to the identity orientation, removing the need for the model to infer 3D orientation. As result, the trained model's outputs align with the input view identity orientation. We train the model for 600k iterations on the tokenizer lokenizer-training set (effective batch size 256 on 128 H100 GPUs for 20 days).
4 EXPERIMENTS
We first train the latent representation, and once learned, we then train latent flow now-matching model conditioned on an input image. See Sec. E for more implementation details. We discuss the training and evaluation of our latent representation in Sec. 4.1, and our image-to-3d model in Sec. 4.2.
4.1 RECONSTRUCTION
Datasets. We train the encoder-decoder on the 500k high-quality object subset of Objaverse- XL. (Deitke al,, 2023) selected by TRELLIS (Xiang al., 2025). Unlike TRELLIS. instead
7

---

Published as a conference paper at ICLR 2026
Table Reconstruction on Toys4k. We provide imput needed by individual methods. TRELLIS (Xi- ang et al., 2025) takes the ground-truth mesh and 150 distrr-dittributee renderings. Ours uses RGBD images from 150 evenly distributed views. For appearance evaluation, we render each model's output from 100 random cameras, varying difficulty by adjusting camera radius. Please refer to Fig. 4 for
qualitative results and Sec. C for comprehensive quantitative results. The better one is highlighted.
Simple, Camera Radius [3, 4]
Hard, Camera Radius [1. 3]
Method
PSNR1
SSIM
LPIPS
PSNR1
SSIM
LPIPS
TRELLIS
31.1
0.974=0.022
0.034+0.022
27.57:33.38
0.941=0.050
0.090=000055
Ours
34.16+3.339
0.985=0.016
0.023++00188
32.36++3.77
0.967=00000
0.055=00046
Table 2: Geometric reconstruction evaluation. We report Chamfer distances multiplied by 104 for readability, computed using 100k sampled points each from ground-truth and reconstruction. As 3DTopia-XL XL (Chen et al., 2025b) and TripoSG (Li ct al., 2025a) can be sensitive to imput geometry, we also list variants with their 10% worst-performing objects removed. We separate based on those that require ground-truth coarse geometry for decoding the latent representation, and those that do not utilize this information. Our method outputs the best geometry among the approaches in the latter category, and it is competitive with the techniques in the former while using
tested approaches
10x smaller latent space. Best and 2nd-Best methods in each category are highlighted.
Method
Appearance
Latenl size
PHR Ohjjvereee
TTys44
GSO
GT
82.49=21.33
76.03++24.30
87.50++21.4
Requires coarse geometry oracle:
TripoSF (He et al., 2025)
x
2 244k X 11
13.11=21.45
76.994224:95
87.63421.66
2
TRELLIS (Xiang et al., 2025)
RC 20k x 11
20.17
2:2.99
105.5 1::25:1:22.44
3-1
3DTopia-XL (Chen et al., 2025b)
2048 x 64
412.5=1129.
153.9+3005 8
98.50=19.66
3-2
(worst 10% removed)
2048 x 64
135.4=95.75
90.61+:23.96
93.62++12886 86
Ours (oracle, mesh decoder)
8192 x 32
87.02
80.32.1.7.50
94.87-123.42
Does not utilize coarse geometry oracle:
5-1
TripoSG (Li et al., 2025a)
2048 x 64
269.2+220.0
299.7++265.9
301.3+300022
5-2
(worst 10% removed)
X
2048 x 64
199.62125.3
2:3055::122
119.7:17736
6
Shape Tokens (Chang et al., 2024)
X
1024 x 16
126.0=23.20
119.8 + 28.02
130.5++-2072
7-1
Ours (no mesh decoder)
8192 x 32
94.08 22.000
88,30 25.22
98.66.1.1.55
7-2
Ours (mesh decoder)
8192 X 32
27.17=24.29
80.55:227.59
95.19:223.64
of using all 500k objects for training, we divide the data into training, validation, and test sets in an 8:1:1 ratio. For each object, we pair it with 3 lighting conditions: 1) fixed smooth area lighting (matching TRELLIS)². 2) an all-white environment map, and 3) randomly placed lights. For each configuration, we render using Blender from 150 viewpoints uniformly distributed on a sphere, to sample the surface light field as input for our encoder. We render from 100 random viewpoints to supervise our view-dependent Gaussian decoder.
We evaluate the models on Toys4k (Stojanov et al., 2021), GSO (Downs et al., 2022), and Objaverse- XL (Deitke et al., 2023). For Objaverse-XI. we select subset of 200 objects with PBR materials, which we dub PBR-Objaverse.
Fig. 4 shows a few objects with view-dependent appearance, including specular reflections from metallic surfaces and Fresnel reflections when viewed at grazing angles.
Qualitative results:
Quantitative results (appearance): To evaluate appearance quality, we render the 3DGS from 100 random views on sphere and measure PSNR, SSIM (Wang et al., 2004), and LPIPS (Zhang et al., 2018). Tab. shows reconstruction metrics under different zoom-in levels on the Toys4k dataset rendered with TRELLIS's training lighting condition. Our surface light-field representation outper- forms competitor appearance representations across all the tested metrics. More evaluations on other datasets and lightings are described in Sec. C.
Quantitative results (geometry): To evaluate the quality of reconstructed 3D geometry, we estimate ground truth point clouds by unprojecting the rendered depth of target object from 100 uniformly distributed views on the sphere and randomly selecting 100k reference points. We then compute
Pt / /i waw.kitssbbleddr_ssrrrre
http://ooroooftttnnni////////oa 650d64751 655d ad544933/ddatttto /atasset_
8

---

Published as a conference paper at ICLR 2026
Input View
Ours (same view)
TRELLIS (same view)
Figure 6: Fidelity to input view. Our image-to-3d generative model respects the coordinate system of the input view. In contrast, existing state-of-the techniques, e.g., TRELLIS (Xiang et al., 2025), do not. Mesh credit: Virtual Museums of Matopolska (2016); animanyarty (2022).
Chamfer distance in See. C.1 between these ground truth point clouds and reconstructed ones. For LiTo and Chang et al. (2024), we sample 100k points from the flow-matching velocity decoder to produce the output point cloud, To fairly compare to baselines that output meshes (Xiang et al., 2025; Li et al., 2025a; He et al., 2025), also train mesh decoder (Sec. E.6, Fig. S6, and Fig. S1). Similar to the ground truth points, we unproject rendered depths of the mesh from another set of 100 views on the sphere and select 100k points for the Chamfer calculation.
Tab. 2 shows geometry evaluation when the input is lit with TRELLIS's s training lighting. Our method (row 7-1 and 7-2) outperforms most geometry-only latent representations, despite additionally rep- resent appearance information and do not utilize additional ground truth coarse geometry information that other state-of-the-eeee approaches require (Xiang al., 2025; He et al., 2025).
Table 3: Single-image-to-3D generation on Toys4k. KID reported by x x100. CFG scale for both models are 3.0. The best is highlighted. See Fig. 5, for qualitative results.
Conditioning View
Novel View
Method
CLIPt
FID.
KID4
FIDamo
KIDaino
FIDJ
KIDJ
FIDaino
KID
TRELLIS
0.899
12.84
0.088
84.692
2.311
7.600
0.100
67.458
3.166
Ours
0.005
6.219
0.009
41.621
1.333
6.216
0.058
66.530
3.522
4.2 GENERATION
Fig. contains qualitative results. Our model generates complex geometry and view-dependent appearance, despite being trained on other lighting types. We also visualize our model's input view fidelity compared to TRELLIS in Fig. 6 to verify our training strategy in Sec. 3.5.
We quantitatively evaluate generation results with the same fixed area lighting as TRELLIS allow a fair comparison. We calculate two distribution-wise metrics. First, to evaluate the fidelity of the generative model the input content, we render the generated 3D asset at the same pose as the conditioning view. As shown in Tab. 3, our approach produces significantly improved FID (Heusel al., 2017) and KID (Binkowski et al., 2018) scores in this setting compared to TRELLIS. Second, to measure the overall quality of the generated asset, we render from four novel views distributed around the object pitch of 30" following the evaluation setup of TRELLIS (Xiang et al., 2025). As shown in Tab. despite model's increased faithfulness to the input view, the overall generation performance does not significantly degrade. Please refer to Sec. for more studies.
CONCLUSION
We propose an autoencoder that learns a compact latent space for 3D assets with view-dependent appearance. In particular, we build an encoding of the surface light field, that can be easily produced via multi-view RGBD rendering. With a flow- fow-matching geometry decoder (or separately-trane -rained mesh decoder) and view-dependent Gaussian decoder, our representation can be easily applied with an off-the-shelf DiT for generating view-dependent 3D assets. We validate the performance of our view-dependent 3D representation in both reconstruction and generation.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
We thank Muhammed Kocabas for creating the LiTo demo. We are grateful to Miguel Angel Bautista Martin, Hadi Pouransari, Josh Susskind, Barry Theobald, Yuyang Wang, and the reviewers for their valuable feedback on our paper. We also thank Denise Hui, David Koski, and the broader Apple infrastructure team for maintaining the computing resources that supported this work. Names listed in alphabetical order by last name.
REFERENCES
18 1panorama.. Drummer the revel infantry regiment, 2019. URL https://kkk: 1y/ 6Xq6W. Accessed September 2025. Licensed under Creative Commons Zero Public Domain. 17
3d-coat. Robot steampunk 3d-coat 4.5 pbr, 2015. URL https: / September 2025, Licensed under Creative Commons Attribution.
Accessed
3Dji. Mechanical beast, 2025. URL https: / http//k.b.ll/// 1y/p/Fo.. Accessed September 2025. Licensed under Creative Commons Attribution. 7
3Dji. Coffee grinder, 2025. URL bttps://skff wttp:/f..lyy.pgg. 1y/p2pn7. Accessed September 2025. Licensed under Creative Commons Attribution. 6
a108082046. Telephone, 2022. URL https: /skkb..b../////////////////// 1y/ovVBJJ, Accessed September 2025. Licensed under Creative Commons Attribution. 17
AdamJonesCGD. Conrad carriage, 2020. URL https://sskke //ts::://kkb... 1ot.ff/.ly/yyy......m 9t Accessed September 2025. Licensed under Creative Commons Attribution. 17
Alienor.org. Conseil des musées. La grand' goule, 2016. URL wtt.httss///kfb..////0 Ac- cessed September 2025. Licensed under Creative Commons Auribuioo--aanCCmmrrraa--oo 17
alzarac. Hypostomus / coroncoro, 2019. URL https: /ttp///.bb.///// httpss://ssbb..y.///... 11W.b.2....88........ 2025. Licensed under Creative Commons Attribution. 17
6WD2. Accessed September
animanyarty. Motorcycle, 2022. URL https:: :/tps:://skfat 1y0/HMPH, Accessed September 2025. Licensed under Creative Commons Attribution. 9
Anthony Schmidt. Spartan Helmet, 2016. URL https::/ llteff..lyl/ll /Moyw. Accessed September 2025. Licensed under Creative Commons Attibution-NonCommercd..
Mikolaj Binkowski. Danica J. Sutherland, Michal Arbel, and Arthur Gretton. Demystifying MMD GANs. In International Conference on Learning Representations (ICLR), 2018.
brysew. Cartoon Tractor T40, 2015. URL htt.httss///kkt..l//// Licensed under Creative Commons Attribution-NonCoomeriic
Accessed September 2025.
Ruojin Cai, Guandao Yang, Hadar Averbuch-Elor, Zekun Hao, Serge Belongie, Noah Snavely, and Bharath Hariharan. Learning Gradient Fields for Shape Generation. In European Conference on Computer Vision (ECCV), 2020. 16
Jen-Hao Rick Chang, Yuyang Wang. Miguel Angel Bautista Martin, Jiatao Gu, Xiaoming Zhao, Josh Susskind, and Oncel Tuzel. 3D Shape Tokenization via Latent Flow Matching. arXiv, 2024. 1, 2,
4, 5, 8, 9,9,11 16
Rui Chen, Jianfeng Zhang, Yixun Liang, Guan Luo, Weiyu Li, Jiarui Liu, Xiu Li, Xiaoxiao Long, Jiashi Feng, and Ping Tan. Dora: Sampling and Benchmarking for 3D Shape Variational Auto- Encoders. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2025a. 16
Zhaoxi Chen, Jiaxiang Tang. Yuhao Dong. Ziang Cao, Fangzhou Hong, Yushi Lan, Tengfei Wang, Haozhe Xie. Tong Wu, Shunsuke Saito, et al. 3DTopia-XL: Scaling high-quality 3d asset generation via primitive diffusion. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025b. 2, 8, 16
10

---

Published as a conference paper at ICLR 2026
Gene Chou, Yuval Bahat, and Felix Heide. Diffusion-sdf: Conditional generative modeling of signed distance functions. In IEEE International Conference on Computer Vision (ICCV), 2023. 16
Tri Dao. FlashAttention-22: Faster attention with better parallelism and work partitioning. In International Conference on Learning Representations (ICLR), 2024. 5
Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, Eli VanderBilt, Aniruddha Kembhavi, Carl Vondrick, Georgia Gkioxari, Kiana Ehsani, Ludwig Schmidt, and Ali Farhadi. Objaverse-xl: A universe of 10m+ 3d objects. arXiv, 2023. 7.
DigitalSouls, Delicious red apple, 2019. URL https: htt.//ktbb...t 6Rxtt. Accessed September 2025. Licensed under Creative Commons Ciabutioo-NonCommmreee 6, 28
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations (ICLR), 2021.
Laura Downs, Anthony Francis, Nate Koenig, Brandon Kinman, Ryan Michael Hickman, Krista Reymann, Thomas Barlow McHugh, and Vincent Vanhoucke. Google Scanned Objects: High- Quality Dataset of 3D Scanned Household Items. In International Conference on Robotics and Automation (ICRA), 2022.
Eleanie. Fox metal statues, 2025. URL htt.t::tt////ccc //sk U/L/tttp////ttbbbb/////.. Accessed September 2025. Licensed under Creative Commons Attribution. 7,28
Zhengyang Geng, Mingyang Deng, Xingjian Bai, Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. In Advances in Neural Information Processing Systems (NeurIPS), 2025. 20
GJ2012. Toaster kitchenaid artsan, 2013. URL http:::///////.. htwp://wwwwwttt www.wwwp.dww......m.. com hommmbbendd 8552. Accessed September 2025. Licensed under Creative Commons Zero. 17
Xianglong He, Zi-Xin Zou, Chia-Hao Chen, Yuan-Chen Guo, Ding Liang, Chun Yuan, Wanli Ouyang, Yan-Pei Cao, and Yangguang Li. SparseFlex: High-Resolution and Trbirrary-Tppolog 3D Shape Modeling. arXiv, 2025.
2, 6, 8, 16
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by Two Time-Scale Update Rule Converge Local Nash Equilibrium. In Advances in Neural Information Processing Systems (NeurIPS), 2017. 9
Ka-Hei Hui, Aditya Sanghi, Arianna Rampini, Kamal Rahimi Malekshan, Zhengzhe Liu, Hooman Shayani, and Chi- Chi-Wing Fu. Make-a-shape: a ten-million-scale 3d shape model. In International Conference Machine Learning (ICML), 2024. 2, 16
Andrew Jaegle, Sebastian Borgeaud, lean-Baptise Alayrac, Carl Doersch, Catalin Ioneseu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, et al. Perceiver IO: a General Architecture for Structured Inputs & Outputs. In International Conference on Learning Representations (ICLR), 2022. 5,
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuehler, and George Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Transactions on Graphics (TOG), 2023.
Benjamin Lefaudeux, Francisco Massa, Diana Liskovich, Wenhan Xiong, Vittorio Caggiano, Sean Naren, Min Xu, Jieru Hu, Marta Tintore, Susan Zhang, Patrick Labatut, Daniel Haziza, Luca Wehrstedt, Jeremy Reizenstein, and Grigory Sizov. xformers: A modular and hackable transformer modelling library. https:/ ///p::///// //
thub. com/ racebookreses weeeerrrchhh wwwaarch//ooreeereeeeeeeeeeeeeee formers, 2022. 5
Yangguang Li, Zi-Xin Zou, Zexiang Liu, Dehu Wang, Yuan Liang, Zhipeng Yu, Xingchao Liu, Yuan-Chen Guo, Ding Liang. Wanli Ouyang, et al. TripoSG: High-Fidelity 3D Shape Synthesis using Lcrge-Scale Rectified Flow Models. arXiv, 2025a. 1, 5, 9,
11

---

Published as a conference paper at ICLR 2026
Zhihao Li, Yufei Wang, Heliang Zheng. Yihao Luo, and Bihan Wen. Sparc3D: Sparse representation and construction for high-resolution 3d shapes modeling. arXiv, 2025b. 16
LLOYDO. Love, Death + Robots, small orange bot, 2019. URL https: /skfb.k....../ 1y/ 6VVUM. Accessed September 2025. Licensed under Creative Commons Attribution.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations (ICLR), 2019. 23
Luo and W. Hu. Diffusion probabilistic models for 3d point cloud generation. In IEEE Conference on Computer Vision and Pattemn Recognition (CVPR), 2021. 16
Quan Meng, Lei Li, Matthias NieBner, and Angela Dai. LT3SD: Latent trees for 3d scene diffusion. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 2, 16
nastasyas. Gart_220 _en__220_eenturr, 2019. URL https: / sk fb. 1y/ 6WR7H. Accessed September 2025. Licensed under Creative Commons Attribution. 17
Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generating 3d point clouds from complex prompts. arXiv, 2022. 16
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Q. Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, AlaacIdin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojeiech Galuba, Russ Howes, Po-Yao (Bernie) Huang. Shang-Wen Li, Ishan Misra, Michael G. Rabbat, Vasu Sharma, Gabriel Synnaeve, Huijiao Xu, Hervé Jégou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning Robust Visual Features without Supervision. Transactions Machine Learning Research (TML.R), 2024.
Osho. Allan Pinkerton electric horse, 2018. URL https: /ttp//.b../.. bl..fblly//oe PI Accessed Septem- ber 2025. Licensed under Creative Commons Attribution. 1
William Peebles and Saining Xic. Scalable diffusion models with transformers. In IEEE International Conference Computer Vision (ICCV), 2023. 6
Xuanchi Ren, Jiahui Huang. Xiaohui Zeng, Ken Museth, Sanja Fidler, and Francis Williams. XCube: Lcrge-Scale 3D Generative Modeling using Sparse Voxel Hicrarchies. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2, 16
Rigsters. Lion crushing a serpent, 2017. URL https: /swfb...b.////// wt...l//////// Accessed September 2025. Licensed under Creative Commons Attribution. 7
@sanyabeast. [Archive] SM Vintage Scooter 01 A, 2021. URL https://skfb.... 1l/to..L...//.L... Accessed September 2025. Licensed under Creative Commons Attribution. 1
Tianchang Shen, Jacob Munkberg, Jon Hasselgren, Kangxue Yin, Zian Wang. Wenzheng Chen, Zan Gojcic, Sanja Fidler, Nicholas Sharp, and Jun Gao. Flexible Isosurface Extraction for Gradient- Based Mesh Optimization. ACM Transactions on Graphics (TOG), 2023. 23
Stefan Stojanov, Anh Thai, and James M. Rehg. Using Shape to Categorize: Low-Shot Learning with an Explicit Shape Bias. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Zhicong Tang. Shuyang Gu, Chunyu Wang, Ting Zhang, Jianmin Bao, Dong Chen, and Baining Guo. Volumediffusion: Flexible text-to-3d generation with efficient volumetric encoder. arXiv, 2023. 16
Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, Karsten Kreis, et al. Lion: Latent point diffusion models for 3d shape generation. Advances in Neural Information Processing
Systems (NeurIPS), 2022. 16
Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkorcit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is All You Need. In Advances in Neural Information Processing Systems (NeurIPS), 2017. 23
Virtual Museums of Małopolska. Black and white "belweder", 2016. URL httpt::t/s/tetttc 1y/ NnEr Accessed September 2025. Licensed under Creative Commons Zero Public Domain. 9
12

---

Published as a conference paper at ICLR 2026
Virtual Museums of Malopolska, Metal "kontusz" knob, 2020. URL htttps://stt /ttpt//k...././///// .l 6Vy Accessed September 2025. Licensed under Creative Commons Zero Public Domain.
Bruce Walter, Stephen R. Marschner, Hongsong Li, and Kenneth E. Torrance. Microfacet models for refraction through rough surfaces. In Eurographics Conference Rendering Techniques, 2007.
24
Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with metric scale and sharp details. arXiv, 2025. 18
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing. 2004.
webdataset development team. webdataset: A Perf-Paefee Python I/O System for Deep Learning, 2026. URL https: /ttps:///iittti com http://bbaaatttte htwpseeeeeeeweeeeeeeeeeeeeeeeeee Accessed September 2025. 24
Daniel N. Wood, Daniel Azuma, Ken Aldinger, Brian Curless, Tom Duchamp, David H. Salesin, and Werner Stuetzle. Surface Light Fields for 3D Photography. In ACM SIGGRAPH, 2000. 4
Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Jingxi Xu, Philip Torr, Xun Cao, and Yao Yao. Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer. In Advances in Neural Information Processing Systems (NeurIPS), 2024. 2, 16
Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Yikang Yang, Yajie Bao, Jiachen Qian, Siyu Zhu, Xun Cao, Philip Torr, et al. Direct3d-s2: Gigascale 3d generation made easy with spatial sparse attention. In Advances in Neural Information Processing Systems (NeurIPS), 2025. 6, 16
Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen. Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 2, 3, 5, 6, 7, 8, 9, 16, 18, 19, 20, 23
Guandao Yang. Xun Huang, Zekun Hao, Ming- Yu Liu, Serge Belongie, and Bharath Hariharan. PointFlow: 3D point cloud generation with continuous normalizing flows. IEEE International Conference Computer Vision (ICCV), 2019. 2, 16
Jiayu Yang, Taizhang Shang, Weixuan Sun, Xibin Song, Ziang Chen, Senbo Wang, Shenzhou Chen, Weizhe Liu, Hongdong Li, and Pan Ji. Pandora3D: A Comprehensive Framework for High-Quality 3D Shape and Texture Generation. arXiv, 2025. 16
Lior Yariv, Omri Puny, Oran Gafni, and Yaron Lipman. Mosaic-sdF for 3d generative models. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 16
Biao Zhang, Matthias Nießner, and Peter Wonka. 3DILG: Irregular Latent Grids for 3D Generative Modeling. In Advances in Neural Information Processing Systems (NeurIPS), 2022. 3, 16
Biao Zhang, Jiapeng Tang, Matthias Nießnner and Peter Wonka. 3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models. ACM Transactions on Graphics (TOG), 2023. 3. 5, 16
Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang. Wei Yang, Lan Xu, and Jingyi Yu. CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets. ACM Transactions on Graphics (TOG), 2024. 2, 3, 16
Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 8
Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng. Rui Wang, Pei Cheng, BIN FU, Tao Chen, Gang YU, and Shenghua Gao. Michelangelo: Conditional 3D Shape Generation based Shape-Image-Tee Text Aligned Latent Representation. In Advances in Neural Information Processing Systems (NeurIPS), 2023. 16
13

---

Published as a conference paper at ICLR 2026
Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang. Sheng Zhang, Xianghui Yang. et al. Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation, arXiv, 2025. 2, 16
14

---

Published as a conference paper at ICLR 2026
APPENDIX LITo: SURFACE LIGHT FIELD TOKENIZATION
This supplement is organized as follows:
1. Sec. A discusses more on related works;
2. Sec. B discusses limitations;
comprehensive reconstruction quantitative results; comprehensive generation quantitative results; 5. Sec. E introduces more implementation details;
3. Sec. C provides
4. Sec. D provides
6. Sec. F showcases studies;
7. Sec. G breaks down full contributions.
A MORE RELATED WORKS
Tab. provides an overview of related works with respect 1) how they model the geometry; 2) how they model the appearance; the requirements the data preparation enable the model training; 4) the compactness of the latent size; 5) the input to the encoder and the training dataset.
B LIMITATIONS
We utilize 3D Gaussians with spherical harmonics to model surface light field. While we show that the improved reconstruction quality as we increase the degree of the spherical harmonics, we are constraint by the 3DGS implementation that supports up degree 3, which limits our capability to faithfully reconstruct transparent or high-frequency specularities.
C COMPREHENSIVE RECONSTRUCTION RESULTS
We provide comprehensive quantitative results for reconstruction in Tab. S2, S3, and S4. As discussed Sec. 4.1, we pair each dataset with three distinct lighting conditions thoroughly evaluate the appearance modeling capabilities of our method. Unlike previous approaches, which primarily assess performance on zoomed-out views, we additionally evaluate appearance modeling under close-up settings. Close-up views demand greater fidelity in capturing high-freyuency details, where all methods face challenges; nevertheless, LiTo consistently demonstrates the most robust performance.
Further, provide qualitative results for reconstructed mesh in Fig. S1.
C. METRICS
We use the following definition for Chamfer distance for any specific 3D asset reported in the quantitative results:
1
CD(Xar, Mprea)
min ||xer Xprit E
Xpreal|2
Karl
X
+ (Xprodl
min NareAor
NGT
Xprod
(S1)
where Xer and Xpred denote the ground-truth and predicted sets of points respectively.
C.2 ABLATIONS ON MODEL DESIGNS
As far know (see Tab. S1), the first to utilize 1) viewing directions in the encoder; and 2) higher order spherical harmonics in the decoder during 3D asset tokenization training. Thus, we are mainly interested in understanding the effects of these design choices.
15

---

Published as a conference paper at ICLR 2026
Table S1: Recent latent 3D representations. The table provides a summary of recent 3D repre- sentations and their properties. We compare the properties that are relevant to machine learning applications. Minimal preprocessing indicates how easy is to utilize a 3D dataset (e.g., do we need to convert data to watertight meshes, do we need optimization radiance fields to acquire the actual training dataset). Continuous latent indicates whether the 3D representation is fully differentiable (e.g., no graph topology sparsity patterns). Total latent dimension indicates the total size to represent one scene. Note that there may be multiple variants of the same method with different latent dimensions. We choose the representative one in each paper. indicates second generative model is used in the paper to add texture to a texture- meshes.
geometry
appearunce
data requirements
total latent dimension
Inpur to encoder
training dataset
Poal (2021)
play4)
point cloud
256
cloud
StupeNet
PoinFlow (2019)
p0gg))
point cloud
512
point cloud (X)
ShapeNee
ShupeGF (2020)
poay)
point cloud
256
poizt cloud (X)
ShapeNet
Stupe Token (2024)
point cloud
1004
point cloud (x)
Objwzzee
Ours
poys)
view dep. 1008
nultiviow RGBD
8192 32
(s. d)
surnac light Reld
Objavense ObjaverseXL
Puint-E (2022)
Encd sisc
diffise RGB
puint cloud (x)
prpprittrry dutacc
LION (3022)
fixed size pount
point cloud
128 8192
cloud
ShapeNett
3DShupe2Veceet (2023)
occupancy
watertight mesh
poist cloud
ShupeNet wusenight
30IL (301))
occupuncy
wateright
point cloud
ShupeNae-woorighh
Michelangeto
occupancy held
watertight mech
point cloud
ShapeNet, cartocn
CLAY (2094)
orcrgoncy fickd
watenight mosh
2048 64
point clooa
Objoenz:
Dom (30054)
occugancy held
wateright mech
1230 bi
point cloud (X)
Objwone
PardseaDD (2025)
oxcupancc fild
watenigha mesh
2048
point cloud (x. i0
Ohjozns; ObjascrseXX.. ABD,, HuildingNet.
HSSI) Try ii. polygons datsect, propricurr
Dine::3 (2004)
oecuppancy grid
walenight mesh
3xx 32 x 32 x 16
chout cloud (s. io
prigisttry dusact
Diros 10=2 (210)
SIDP grid
watenight mush
(126* x 16)
puint cloud (s.
Objecnze ObjaannaX).
XCabe
occupancy arid
watertight mesh
more (2x3+2)x
dccupancy gaid
ShupeNet, Objavense
EUSSI (3005)
UDB grid
wateright mech
(5+14 4144%x4 36%h)4
UDF grid
3D Feor
Diiissien-SDP (1033)
SDPP fildd
watenight mosh
7608
point cloud (8)
StopeNia-wwerrigh. YCB
MOSAIC.SDF (2004)
SDF Geld
watenight mesh and oprimization
1024 (341477)
StupeNa-wwaerigl.
scalable 30 cxptioning dataset
TripoSG (2005))
SDF Reld
watertight mech
2048
potat cloud
Objarense ObjavereeL.
Huryun 2.0 (2025)
SDF hold
watenight mush
9072
point cloud
Objocnee Objosamell
Make-A-Shrp (2014)
SDM grid
wateright mech
MM
1% dumeets
x (8+ 1-1 1500, 2014
307 (20255)
Prinx (SDF held) RGB, PER
PrimX optimmcation
Prixx
Objwwome
Spare (10055)
grid
watoright gsd opiinization
unknown
grid
Volame Dillaaion (2023)
rudiaane fold
diffus: RGB
nus inferme nziwock
mulliriww imuges
Objacze
TREELIS (2025)
occupancy grd
diftuse 3DUS
multiview DINUX2
grid))
spanse feature
Ohjyrese, Objavessel.
3D-futnnc
TripuSP (2025)
SDD grid
uulliviow dapth noemal
3,0000000000000 (206" grid)
point Objecnss ObjaoassclI.
When examining LPIPS across Tab. S2, S3, and S4, we observe: 1) increasing the degree of spherical harmonics from 0 to 3 improves the capacity consistently, e.g., from row 1-3 to 1-6 (or row 2-3 to 2-6, 3-3 to 3-6) in all three tables; and 2) simply adding ray information does not directly enhance appearance modeling performance, e.g., row 1-2 vs. 1-3 (or row 2-2 vs. 2-3, 3-2 vs. 3-3). We hypothesize that this is because zero-degree spherical harmonics cannot capture view-dependent effects, which then becomes bottleneck. preventing the model from fully leveraging the information contained in the view directions. To verify, we ablate by removing the ray information from our encoder when using 3-degree spherical harmonics. The improvement row 1-6, which incorporates ray information, from 1-7 (or row 2-6 vs. 2-7, 3-6 vs. 3-7) corroborates our hypothesis.
C.3 ABLATIONS ON NUMBER OF INPUT VIEWS IN INFERENCE
We are interested in understanding to what extent our approach is robust to the discrepancies between the number of input views during training and inference. Quantitative evaluations are in Tab. S5.
16

---

Published as a conference paper at ICLR 2026
Ground truth
Ours
Ours (oracle)
TRELLIS
Figure $1: Mesh comparisons. We demonstrate the qualities of our mesh decoder results to TRELLIS. As highlighted, our produced mesh maintains more details. Mesh credit: alzarac (2019); Alienor.org, Conseil des musées (2016); 1812panorama (2019); AdamJonesCGD (2020); nastasyas (2019); a108082046 (2022); GJ2012 (2013).
17

---

Published as a conference paper at ICLR 2026
Table S2: Reconstruction on Toys4k. For 3D assets, we adapt inputs per model. TRELLIS (Xiang et al., 2025) takes the ground-truth mesh and 150 sphere-distrbuted renderings. Ours uses RGBD images from 150 evenly distributed views. For appearance evaluation, we render each model's output from 100 random cameras, varying difficulty by adjusting camera radius. Each model is further evaluated under three distinct lighting conditions. Importantly, no separate models are trained; all evaluations are conducted on the same model. As result, we conduct evaluations at the scale of (lightings) x 1.8 million images. We
over 3000 (objects) x 100 (views) x 2 (difficulties) report Chamfer distances multiplied by 10 for readability, computed using 100k sampled points each from the ground truth and reconstruction. We report in the format of mean where the standard deviation is computed across objects.
Method
SII Deg
Ene
Pred Occ
Mesh
Simple, Radius
4]
Camena Radius [1,3]
CD (100k)
Ray
PSNR
SSIM
LPIPS]
PSNR
SSIM
LPIPS)
Uniform Lighting
1-1
TRELLIS
0
X
28.1744009
0.970+00024
0.039m0:22
24.63+4401
0.934400054
0.098:00009
95.19426,44
Ours
0
34.40:13.22
0,98410017
0.02510:019
32.19:1.95
0.96510:022
0.059110047
90.22:12546
Ours
0
/
34.44:1.447
0,984110177
0.026:::020
32.18:1..79
0.964100022
0.060110.048
89.24:125.34
14
Ours
1
/
X
x
35.12#3339
0.986
0.023+0:117
33.17=3.76
0,9684:0000
0.0545:0.044
28.42#25.21
1.5
Ours
2
/
35.32::445
0,98640016
0.023400117
33.29
0,969:00000
0.055
88, 18.944225.34
Ours
3
35.32:13338
0,986110015
0.022::0017
33.39
0.96911.009
0.05310.00
88, 13:125.5
1-7
Ours
3
X
X
35.54
0.986:00015
0.023=0:117
33.37=3.97
0,969::0000
0.055
89.63=22516
188
Ours
3
35.27
0,98640015
0.022:00177
33.3843.71
0,969:00000
0.052220.04
28.133225.9
Ours
3
35.27:1336
0,986110055
0.022:::017
33.38.13.1
0.9691100100
0.05211.00
80.42:1:27.00
1-10
Oraele
3
35.26:13.34
0.986:10:055
0.022:0.017
33.42:13.69
0,970
0.051
80.17:127.58
TRELLIS Lighting
2-1
TRELLIS
0
X
31.12:1.339
0.994:1:0022
0.034:0:022
27.37:13.38
0.941::0000
0.090:00.055
92.21
2-2
Ours
0
32.47:4385
0,980+0:20
0.029=0422
30.21::4.19
0,958:00066
0.0677+0.083
90.00=
2-3
Ours
0
32.47:1369
0,980100000
0:029:0:022
30.21:14006
0.957:100046
0.068110052
89.1 12:12.35533
24
Ours
1
V
X
X
34.00:13.38
0.984:10006
0.025:0:019
32.03:13.74
0.965:1:0000
0.05911.044
88.46:125.13
2.5
Ours
2
/
34.06:3.40
0,984100116
0.024=0.019
32.122:3.79
0,966#0041
0,058+000447
88.82++25.30
2-6
Ours
3
34.1911.39
0.985
0.02410:019
32.36:13.77
0.967100000
0.05610.046
88, 25.30:125.23
2-7
Ours
3
X
X
X
34.16.1668
0.985110017
0.025:00019
32.11:1:404
0.96611::01
0.058110044
89.35:125.12
2-8
Ours
3
34.16:1339
0.023=00m1 0,065
32.36:+3.77
0,967#0040
0.055
88.30-+25.25
Ours
34.16:3.39
0.985400016
0.02340.015
32.16::3.77
0.967::0000
0.055
80.55:227.59
2-10
Oraele
3
34.1411.37
0.985
0.023
32.38:1374
0.967:100000
0.054:10.045
80.32:127.30
Random Lighting
3-1
TRELLIS
0
X
27.9413.77
0.9661.0005
0.038
24.37:1166
0.927:100054
0.098:20.058
93.95:125.89
3-2
Ours
0
X
32.12+3225
0,981
0.026:20:21
30.08++3.67
0.961+00043
0.062++0.01
90.15:+:25.9
Ours
32.18:3.112
0.981
0.026:0:021
0.960+-0044
0.063++0.022
34
Ours
1
/
X
X
33.02:12:2
0.984110177
0.023:00019
31.20:11.39
0.965110:41
0.057:10.047
88. 25.65
3-5
Ours
2
V
33.13+2299
0.023:000119 0.984
31.34-+3.49
0,966+0041
0.058:+0.048
89.18
Ours
33.22#295
0.984:00117
0.02340.019
31.50-:3.41
0.966400041
0.056+20.44
3-7
Ours
3
X
X
X
33.23:11322
0,984:10017
0.024:0:019
31.30-11.83
0.96510044
0.058:20.049
89 29..99125.19
3-8
Ours
3
/
/
33.18:1293
0,984:10077
0.022:00019
31, 31.49:13.99
0.9662:0041
0.0551:0048
1
Ouns
33.18#292
0.984100017
0.02240.019
31.50
0.966400011
0.055
80.39:227.95
3-10
Oracle
3
33.15:2290
0.984:10006
0.02210:019
31.50-13.36
0.967:10:00
0.054:20.047
80.1 1::7.55
D COMPREHENSIVE GENERATION RESULTS
As demonstrated in Fig. 6, we are interested in aligning the generation with the input view faithfully. To achieve this, for each sample used during training, we carefully rotate the world coordinate system such that the input view's s corresponding camera poses are at the identity orientation. This relieves the model from the burden of inferring the orientation of 3D space during training. Further, we consider utilizing the view direction during the generative model training as well to enable the model be aware of 3D orientation. Since make the orientation identity, ray information essentially means the availability of camera intrinsics. Then, during inference, we use an off-the- thelf intrinsic estimator (Wang al., 2025) to obtain the intrinsics. However, as shown in row 3 vs. 2 in Tab. S6, it seems like the intrinsic information is unnecessary. Thus we use the generative model trained without any ray information to report our qualitative and quantitative results in the paper.
18

---

Published as a conference paper at ICLR 2026
Table S3: Reconstruction on GSO. For 3D assets, we adapt inputs per model. TRELLIS (Xiang et al., 2025) takes the ground-truth mesh and 150 sphere-distributed renderings. Ours uses RGBD images from 150 evenly distributed views. For appearance evaluation, we render each model's output from 100 random cameras, varying difficulty by adjusting camera radius. Each model is further evaluated under three distinct lighting conditions. Importantly, no separate models are trained; all evaluations are conducted on the same model. As result, we conduct evaluations at the scale of over 1000 (objects) x 100 (views) x (difficulties) x 3 (lightings) s 600 thousand images. We report Chamfer distances multiplied by 10 for readability, computed using 100k sampled points each from the ground truth and reconstruction. We report in the format of mean where the standard deviation is computed across objects.
Method
SII Deg
Ene
Pred Oce
Mesh
Simple, Radius
4]
Camena Radius [1,3]
CD (100))
Ray
PSNR
SSIM
LPIPS.
PSNR
SSIM
LPIPS)
Uniform Lighting
1-1
TRELLIS
0
X
27.34
0.947
0.053=0:229
23.72+3.66
0.883+0008
0.139:00.05
108.2
Ours
0
34.27:1.25
0,975
0.034:0:025
31.391161
0,937:10046
0,093:10055
101.4112.211
Ours
0
/
x
34.04:1.:23
0,974110002
0.034:1:025
31.15:1.59
0.935:100048
0.093100055
100.0:21:41
14
Ours
1
/
X
x
34.55#3.18
0.976:00021
0.02
31.75+3360
0.9394:0045
0.087::0.033
99.21
1.5
Ours
2
/
34.62#324
0,97640021
0.024
31.77::3.65
0,939:00046
0.087440003
99.68:22.75
Ours
3
34.69:1.22
0,976100001
0.031
31.88:13665
0.9401100066
0.086100053
98.911121.55
1-7
Ours
3
X
X
34.93#3224
0.977
0.031
32.00+3.63
0,942400044
0.087::0.03
101.0=22.20
188
Ours
3
34.67#321
0,97640:211
0.03
31.88:3365
0,940:0046
0,086:40.03
98.92421.67
Ours
3
34.6711.21
0.976110001
0.031
31.88
0.940100016
0.08610.053
92.95:1.2419
1-10
Oraele
3
34.66.13.20
0.976:10021
0.030
31.92:1366
0.041
0.08510.003
92.70:121.05
TRELLIS Lighting
2-1
TRELLIS
0
X
30.81:2:.77
0.958:1:0008
0.047:0:026
27.21:12.56
0.907:1:0055
0,12610.058
105.5:1:2244
2-2
Ours
0
33.99+2554
0,978+0017
0.033=0:22
31.65++2.77
0.94840066
0.089400002
101.2:22196
2-3
Ours
0
33.71:1243
0,97811008
0.033:::024
31.40:1262
0.947:10007
0.088100051
99.62:121.28
24
Ours
1
V
X
X
34.75:22.0
0.980110016
0.030:0:022
32.50:12887
0.952:100005
0.08010.048
98.
98.93:121.33
2.5
Ours
2
/
34.87:#2.68
0,980+0017
0.030=0422
32.58+-2995
0.952
0.081
99.28:22155
2-6
Ours
3
34.91:22:6
0,980110016
0.02910:022
32.67:1.95
0.952:10.006
0.080
28.66.221.50
2-7
Ours
3
X
X
X
35.19:2..2
0.981
0.030:0:022
32.79:11997
0.953:1:0004
0.081
100.6:221.9
2-8
Ours
3
34.89+26.4
0,980
0.029:0:22
32.68:+2.94
0.952
0.079::0049
98.64
Ours
34.894264
0.98020006
0.029:0:222
32.68
0.952440066
0.079:00349
2-10
Oraele
3
34.871263
0.981
0.029
32.70:12.94
0.953:1:0006
0.078:10.048
94.87:12344
Random Lighting
3-1
TRELLIS
0
X
27.6611.26
0.050 0.948
24.11::3.08
0.886:10064
0.13320.002
107.5::12.66
3-2
Ours
0
X
33.09++2.77
0,977+0018
0.031 +0:02
30.97:+281
0,945:+0009
0,086:+0.022
101.34422.24
Ours
32,97+2241
0,97610018
0.031
30.82+2.77
0.943-+00000
0.087+00.52
34
Ours
1
/
X
X
33.46:2241
0.978110077
0.028:0:023
31.41:2881
0.947:100008
0.08010.0449
99.29 99.29:12144
3-5
Ours
2
V
33.61+22477
0,978+0017
0.029:0:022
31.55++2.88
0.947
0,081
99.67#221.81
Ours
33.67#246
0,97940017
0.028=0:022
31.65++2.9
0.948
98.93-221.68
3-7
Ours
3
X
X
X
33.98:12.53
0.980.10016
0.028:0:02)
31 31.84:11.93
0.949:100006
0.001
100.9:112.77
3-8
Ours
3
/
/
X
33.6422243
0,97910017
0.028:0:022
31 31.64:1887
0,948:10.008
0,080
98 98.93:221.68
Ouns
33.64#245
0.979:0017
0.028=0:022
31.64
0.948
0.080+0.000
92.94-224.25
3-10
Oracle
3
33.61:2242
0.979:10077
0.028:0:023
31.65:12.86
0.949:100008
0.079:00049
92.67:124.05
D. ABLATIONS ON ODE NUMERICAL INTEGRATION
We study the effect of ODE numerical integration used when sampling from our generative model. Specifically, we ablate the algorithms (Euler and Heun), the step size (or equivalently the number of steps) used during the numerical integration, and the numerical precision of the model (float and bfloat 6) during sampling. We provide quantitative results in Sec. S7. The results suggest our generative model is robust to numerical integration - we observe small change in performance when switching from the second-order method Heun with 100 steps using float 32 (conditioning view FID 6.6) 6.6), to a relatively cheaper first-order Euler with 25 steps using bf foottt 166 (conditioning view FID 6.7).
19

---

Published as a conference paper at ICLR 2026
Table S4: Reconstruction on PBR-Objaverse. For 3D assets, we adapt inputs per model. TREL- LIS (Xiang et al., 2025) takes the ground-truth mesh and 150 sphere-distribute renderings. Ours uses RGBD images from 150 evenly distributed views. For appearance evaluation, we render each model's output from 100 random cameras, varying difficulty by adjusting camera radius. Each model further evaluated under three distinct lighting conditions. Importantly, no separate models are trained; all evaluations are conducted on the same model. As result, we conduct evaluations at the scale of 200 (objects) x 100 (views) X 2 (difficulties) x (lightings) x 120 thousand images. We report Chamfer distances multiplied by 10 for readability, computed using 100k sampled points each from the ground truth and reconstruction. We report in the format of mean where the standard deviation is computed across objects.
Meshod
SII Deg
Ene
Pred Occ
Mesh
Simple, Radius
4]
Camena Radius [1,3]
CD (100k)
Ray
PSNR
SSIM
LPIPS]
PSNR
SSIM
LPIPS)
Uniform Lighting
1-1
TRELLIS
0
X
28.63#3.09
0.046=0a22
25.06+2993
0,902:00077
0.121 +0,002
98.09422.21
Ours
0
32.95:1287
0,97410008
0.03320:020
30.0711002
0,939100022
0.087:10051
95.08:12261
Ours
0
/
33.14:12.68
0,974110088
0.034:1:021
30,21:1885
0.937:10002
0.089110053
94.16.122.55
14
Ours
1
/
X
x
34.35#2377
0.978
0.028=0011
31,67+267
0,947:40088
0.076::0.046
93,48:222.5
1.5
Ours
2
/
34.47:22455
0,978
0.028=00m
31.74
0,947:00099
0.077
94.01
Ours
3
34.62:1.33
0,979110016
0.028:10018
31.98:1266
0.948:10009
0,07511.007
92.92.1.22.8
1-7
Ours
3
X
X
24.66:2.62
0.979
0.029=0:m15
31,83++899
0,948400099
0.077::0047
94,71
188
Ours
3
34.63#233
0,979:0016
0.027=0017
32.01=+2:64
0,94840.09
0.074+00047
92.89
Ours
3
34.63.2.33
0,979110006
0.027::0017
32.01:12.64
0.94810.009
0.07511.047
85.82:125.06
1-10
Oraele
3
34.64.1..11
0.979:10.066
0.027
32.07:12.2
0,949:100008
0.074:10.016
85.57:124.80
TRELLIS Lighting
2-1
TRELLIS
0
X
29.69:1..59
0.958:10:00
0.04420:023
26.03:12.50
0.904:1:0053
0,118:200588
15.16:22
2-2
Ours
0
30.35+3011
0,965
0.039:0022
27.39
0.102240006 0,921
96.06::21.91
2-3
Ours
0
30.37:3.04
0,965 100023
0.040:0:0233
27.41:3.21
0.919:10.000
0,102:0.056
95.11
24
Ours
1
V
X
X
32.52:12445
0.975
0.031
29.87:12.0
0.939:10042
0.084:10.049
94.59:21.75
2.5
Ours
2
/
32.47:22455
0,975
0.031
29.90+-22.3
0.940:00022
0.083450.049
94.984221.11
2-6
Ours
3
32.63:2.38
0,976100117
0.030:10:018
30.14:269
0.941100012
0.081
94.08.1.2.01
2-7
Ours
3
X
X
X
32.56:12.72
0.975
0.031
29.89:1197
0.939110042
0.084:10.049
95.38:121.99
2-8
Ours
3
32.63+2377
0,976:00117
0.030=0.01
30.16++2669
0,942#0042
0.080#00049
94.11#22201
Ours
32.624237
0,97610017
0.030=0.015
30.16:2669
0.94240042
0.080+0.449
2-10
Oraele
3
32.61:2.37
0.97610017
0.029
30.20:11669
0.942:10042
0.08010.048
87.02:124.19
Random Lighting
3-1
TRELLIS
0
X
26.29
0.939110:08
0.05220:030
22.74:11.37
0.869110005
0,134:20.070
99.60:12434
3-2
Ours
0
X
28.58+3365
0,957
0.043:0:225
25.66-t:3.77
0,904+:0066
0,107++0.065
95.14-:22.72
Ours
28.88#3611
0.95610002
0.043:0::25
0.903-40007
0.109::0.066
34
Ours
1
/
X
X
30.36:1.15
0.000
0.03620:024
27.60:1144
0.920:10009
0.095::00058
93.98:112.70
3-5
Ours
2
/
30.39::.008
0,965
0.036::0:024
27.65
0,920=50000
0,095
94.72-422.66
Ours
30.59
0.966200077
0.036:0::24
27.9243342
0.922:40.09
0,093-+0.099
3-7
Ours
3
X
X
X
30.11:13.48
0.964-10:007
0.037
27.27:13775
0.917:100000
0.0961:0059
94.74:112.64
3-8
Ours
3
/
/
X
30.59
0,966:20027
0.035:0:024
27.94:13:43
0.922:10.060
0,092:10059
93.39:112.82
Ouns
30.60
0.966+0007
0.035=0:024
27.95
0.922-400009
0.092++0.09
25.73
3-10
Ornele
3
30.59
0.966:10:006
0.035
27.97:13.42
0.922:100059
0.092:10.058
85.56:224.65
D.2 RUNTIME AND MEMORY ANALYSIS
We analyze the runtime for both TRELLIS and our generative models in Tab. S8. Our model's latent sampling costs 9.3 seconds on while all decoders' feedforward passes cost less than 100 milliseconds on a single NVIDIA H100 80GB HBM3 GPU. In comparison, for TRELLIS, sampling SLAT (both coarse voxel and feature) takes 11.8 seconds. Utilizing one-step fow-matching models like Mla (Geng et al., 2025) can further improve the speed of our generative model and is left future work.
20

---

Published as a conference paper at ICLR 2026
Table S5: Ablation on number of input views for reconstruction during inference. We choose TRELLIS lighting setup on Toys4k dataset. Our model is the same as "ours" in Tab. 1. Both TRELLIS and ours are trained with 150 views. For appearance evaluation, we render each model's output from 100 random cameras, varying difficulty by adjusting camera radius. We report in the format of mean where the standard deviation is computed across objects. We report Chamfer distances multiplied by 10a for readability, computed using 100k sampled points each from the ground truth and reconstruction. Note, we re-render the evaluation data for this ablation, thus row (row 2) differs slightly from row 2-1 (row 2-9) in Tab. S2.
Simple, Camera Radius [3,
Hard, Camera Radius [1, 3]
Method
CD (100k)
PSNR
SSIM
LPIPS
PSNRT
SSIM
LPIPS
150 input views
TRELLIS 31.55913.509
0.9740:100224
0.000110.0217 27.94823559 0. 0.092810.089 25.65::25.13
Ours 33.909 0.9841
0.0260=0.0189 32.073=33521 0.9658+00000
0.0585+0.0058
80.54+27.62
120 input views
TRELLIS 31.518+3.509 0.9738++0.0225
0.0363=#0.0188 27.912+3.541 0,9404+00.050
0.0932++00041
90.88++25.19
Ours
33.90813158 0.9841100022
0.026010.0188 32,072235222 0,9658110000
0.05851000577
20.53:227558
90 input views
TRELLIS 31.431 0.9734400027
27. 27.83333540 0.0266=00221 0.9397200514 0.093820:0045
91.19:22.11
Ours
33.910+3.157 0.9841+000162
0.0260=100119 32.074+3.522 0,9658+00002
0.0585++0.055
80.53++27.60
60 input views
TRELLIS 31.27011496 0.9726.100211
0.0372.10.224 27.688 0.93831000520
0.0952.10.5522
91.92.1.2516
Ours
33,909#3115 0.9841:00010
0.0260=000188 32.073=33119 0.9658:20:000
0.0585=00:457
80,53:227.60
30 input views
TRELLIS
30.69211441 0.9699.10.244 0.0396_10.0238
3.44
0,9336-100541
0.1002:10.057
94.58.125.50
Ours
13.90 0.98411000162 0.0060 32.072 1.5221 0,9658
0.0585:100457
80.56.127.61
Table S6: Singleinnge-conditooeedd generation Toys4k with TRELLIS lighting. KID is reported by x x100. CFG scale 3.0. The best is highlighted.
Train w/ Ray
Infer w/
Train
Conditioning View
Novel View
Method
CLIPT
GT Ray
Iters
FID
KIDI
FIDam
KIDam-4
FID,
KID|
FIDai..
KID
TRELLIS
400k
0,899=0015
12.84
0.088
84,692
2,311
7,600
0.100
67.458
3,166
2-1
Ours
280k
0.906=00000
8.193
0.012
48.117
0.461
6.648
0.064
75.814
4.321
Ours
400k
0.906=0041
7,741
0.010
44.555
0.392
6.413
0.064
71,436
3.997
2-3
Ours
X
600k
0.905=0041
6,219
0.009
41.621
1.333
6.216
0.058
66.530
3,522
0.900=00040
10.78
65.644
2.281
8.076
92.915
6.698
Ours
290k
0.904=0.039
10.13
0.053
61,342
1.665
7.831
0.097
86.091
5.826
Table S7: Ablation on DiT sampler for singl=-image-oondiioonee generation. The experiments conducted Toys4k with TRELLIS lighting. The generative model is trained for 600k iterations. Note, row is copied from "ours" in Tab. KID is reported by x x100. CFG scale is 3.0. Our generative model's performance is robust across various numbers of sampling steps and numerical integration algorithms.
Occ Pred
Conditioning View
Novel View
Data Type
Method
Step
CLIPt
FID4
KID|
FiDaim
KIDain
FID
KID|
FIDain
KIDains
float32
Heun
100
0.905=00041
6.219
0.009
41.621
1.333
6.216
0.058
66.530
3.522
/
float32
Heun
100
0.905=0.041
6.622
0.021
42.197
1.391
6.270
0.064
66 69.699
3.534
/
6
Heun
100
0.905-0.041
6.661
0.020
43.992
1.741
6.270
0.063
68.025
3.906
/
6
Heun
50
0.905-10:04
6.659
0.020
45.533
2.105
6.266
0.062
68.319
4.185
biloar16
Heun
0.904.10.41
6.644
0.019
54,231
4,011
6.251
0.060
77.148
5.879
biloat16
Buler
100
0.906:10004
6.656
0.022
42.472
1.476
6.365
0.066
67.856
3.848
bfloat16
Buler
0.905+000M1
6.688
0.023
42.363
1.430
6.384
0.066
68.987
3.958
8
/
bfloat16
Fuler
25
0.905
6.733
0.025
43.034
1.280
6.833
0.074
75.687
4.484
21

---

Published as a conference paper at ICLR 2026
Table S8: Generative model runtime analysis. All results are reported with torch. forch.profile profiler across three runs. TRELLIS uses 50 Euler steps for both its sparse structure and structured latent generations. We use 50 Euler steps for generating the latents, corresponding row 7 in Tab. S7.
Cond Proc (ms) Structure Gen (x) Lalent Gen (5) Oce Pred (mss) 3DGS Dec (ms)) Mesh Dee (ms)) Total (5) Memory (GB)
NVIDIA S100-XXM4-80G
TRELLIS
68.9 49
4.89
6.
58.70
67.33=13.98
12:76
12.70
Our
68.78=0..0 41
17.32+1.50
36.07+3.67
14.3
90.78=29.75
17.55
15.95
NVIDIA 80GB
TRELLIS
=1.01
3.95+1.17
7.868
15.03+6.119
46.81=13.31
11.91
12.69
Ours
10.5
9.266100336
27.1616.87
30.96114.33
79.15=31.71
9.426
15.93
E IMPLEMENTATION DETAILS
E. ARCHITECTURES
We provide detailed network architectures in Fig. S2 to S7. These include our encoder (Sec. 3.3) in Fig. S2, velocity decoder and Gaussian decoder (Sec. 3.4) in Fig. S3 and S4, occupancy decoder in Fig. S5, mesh decoder in Fig. $6, and generative model's DiT (Sec. 3.5) in Fig. S7.
E.2 POSITION ENCODING
We have the following position encoding function applied on each channel of the input data:
(sin(wo), sin(u--1) cos( (o.(uu)......
..... cos( (oo[UFF1)) where = x M Mrsy
(S2)
Main
(S3)
is the value at the corresponding channel where the position encoding applied. We use 32,
Mmin 0, Mmux = 12, 8, and 8 in position encoding functions for 3D location x,, viewing direction d, color in Eq. (1) respectively. For time step / in flow matching (Bq. (2)), we use F 16, Mmin logg 2n, and Mmax = Mmin + F-
E.3 3D GAUSSIAN PREDICTION
In Fig. S4, the output position of 3D Gaussian is predicted with respect to normalized space centered around the occupied voxel's world coordinates, and is then translated to the world coordinate system using the voxel's information. Specifically, we predict 3D Gaussian's 's position as Xoutput [-1, 133. Assume the corresponding voxel's center located at X E R in the world coordinate system. The final 3D Gaussian's 's position in the world coordinate system is computed as X = Xvexel + Xoutput where s is hyperparameter experiments, we set s = 0.05. Note, s = 0.05 is actually larger than the voxel size we consider. This intentional provides more flexibility, such that the predicted 3D Gaussian can go across the voxel boundaries.
define the size of the normalized space mentioned above. In our
E.4 TOKENIZER TRAINING
Our tokenizer is trained with the following loss:
Lrokenizer Egeo (0) + Cradiance (0) 10-1 KL (d(S)x))p(S))
(S4)
where and Lradiance (0) are from Eq. (2) and Eq. (3), respectively.
We the KL-divergence KL (g(S) (PS)) to regularize the latent space, where p(S) and q(S X) represent prior and posterior distribution for the latent representation S. Ideally, this should be imposed the joint distribution of the & latent vectors (Sec. 3.2). In practice, we simplify and assume each element in the latent space independent. Thus we have
k=8192 d=32
KL (g(()))))))()))
Σ
KL (2(ss)))))((((s)))))
(S5)
22

---

Published as a conference paper at ICLR 2026
where $1 is the j-th element of the i-th latent vector 8, E S. Further, assume p(s.,) follows (0011) 1) while 2(81,) (2)) is ( 10-9). Thus, KI. (1(stgl=))((81s)) boils down to || I|?
In practice, we find that Eq. (S5) effective. When computing the mean and standard deviation of the latent representations S across 1000 objects, we obtain values of 1.1300 and 19.8604. In contrast, with the proposed regularization, the latent space becomes more compact: the mean and standard deviation decrease to 0.0931 and 1.7800, without compromising reconstruction performance.
E.5 OCCUPANCY DECODER TRAINING
We adopt the pretrained sparse- structure VAE from TRELLIS (Xiang et al., 2025) to compute the occupancy grid. Specifically, given LiTo latent code, we first generate low-resolution continuous latent representation (in our case, 162). We then leverage the TRELLIS decoder to upsample this representation and predict occupancy values higher-resolution grid (in case, 64%). Our model is specified in Fig. S5. The details of the upsampling decoder is specified in "3D convolutional U-net"
in Sec. A. from Xiang al. (2025).
The training loss defined as per-element Huber loss between 1) our model predicted low-resolution representation, and 2) the representation encoded from the ground-truth occupancy grid with the pretrained encoder from TRELLIS.
E.6 MESH DECODER TRAINING
We train the mesh decoder in Fig. S6 primarily by penalizing the discrepancy between renderings generated from the ground-truth mesh and those from the estimated mesh. For each mesh encountered during training, we randomly sample 12 views for supervision. Specifically, we use the following loss:
Lmesh = mmssk + 10 Lx... + Lisce_normal normall + L normal + Crea_ + frog
(S6)
fmusk the l distance between 1) mask rendered from the ground-truh mesh; and 2) mask rendered from the generated mesh.
fx... for each pixel's corresponding XYZ in the camera coordinate system. Concretely, we compute the fa distance between the ground-truth coordinates and the predicted ones. We only apply this loss to pixels within the ground- -ouund-ruuhh object mask area. Compared to using depth loss alone, this provides a stronger supervisory signal, which we found significantly improves training performance.
Lrace_norm nor_nn and Eventer nornal supervise the predicted face normals and vertex normals, respectively. They are computed as 1 minus the cosine similarity (i.e. negative cosine distance) between the predicted normal maps and the corresponding ground -trth normal maps. Similar to Lxam, we only apply this loss to pixels within the ground-truie -truh object mask area.
Since we use FlexiCubes (Shen et al., 2023) to produce the mesh, we inherit its regularizations during training. Lreg dev penalizes deviations between each dual vertex and the edge crossings that define the face containing (see Eq. (8) in (Shen et al., 2023)). Leg sdr penalizes sign changes of the SDF across all grid edges." 4
E.7 TRAINING SETUPS
We use A (Loshchilov & Hutter, 2019) for all our training.
For our tokenizer training, i.e., Eq. (S4), that involves encoder (Fig. S2), velocity decoder (Fig. S3), and 3D Gaussians decoder (Fig. S4), we use 3n = 0.9, and 32 = 0.98 without weight decay for A. We use the following learning rate scheduler copied from Sec. 5.3 in Vaswani al. (2017):
Ir(step) 0.4 0.44.-0.5 amadd min
step 0.5
step
(S7)
warmuupstepsst
http://.....ooootaaaaa/a//aeeeae optimi ze- pylt.
http://hww.....o.oaaaacccaaaaaaa es/ loss.
f.96
23

---

Published as a conference paper at ICLR 2026
where dmedel 512, which is latent dimension for Perceiver IO, and warmup carmup_steps 4000 in our case. The model is trained with an effective batch size 256 for 90k iterations on 64 H100 GPUs for 9 days.
For the occupancy decoder training that involves Fig. S5, we use the same optimizer and learning rate setup tokenizer training. We train the model for 58k iterations with effective batch size 256 on 64 H100 GPUs for 1.5 days.
For the mesh decoder training, i.e., Eq. (S6), that involves Fig. S6, learning rate setup as tokenizer training. We train the model for 100k iterations with an effective batch size 128 on 64 H100 for 3 days.
the same optimizer and
For the generative model training that involves Fig. S7, we use 8 = 0.9, and 82 = 0.999, and a weight decay of 0,01 for AdamW. We use the following linear warmup scheduler for the learning rate:
10-6 6 + step
(10- 4 - 10000 step S warmup_steps
In(step) = 10
stops
(S8)
otherwise
where have warmup_sterd 5000 in case. We train the model for 600k iterations with an effective batch size 256 on 128 H100 GPUs for 20 days.
F
MORE STUDIES
F.1 STUDYING SPHERICAL HARMONICS DEGREES
Our Gaussian decoder outputs Gaussians with spherical harmonics up to degree three. We study what information is captured by individual spherical harmonics degrees. Fig. S8 and Fig. S9, we render the 3D Gaussians from both reconstruction and generation by clipping the degree of the spherical harmonics (i.e., we use only S degrees during rendering). We observe that zeroth-degree renderings are mostly nie-iindependent and have little lighting baked in, whereas higher-degree renderings illustrate lighting effects. This is contrast to TRELLIS's results whose zeroth-degree renderings contain both baked lighting and inaccurate view-dependent appearance produced using micro-surface geometry (Walter al., 2007). The results suggest that our model able to represent view-dependent effects using the higher-degree spherical harmonics, and to the veroth-degre rendering for view- new-inddeendent,, diffused, appearance. This separation is an interesting finding, and provides potential opportunity for future investigation of relighting using our representation.
G AUTHOR CONTRIBUTIONS
All authors contributed to writing this paper, designing the experiments, and discussing results at each stage of the project.
Framing. Oncel Tuzel led the research direction, including research framing and question identifica- tion. All authors contributed to setting project priorities.
Writing, Jen-Hao Rick Chang and Xiaoming Zhao completed the majority of the writing while Dorian Chan refined and polished the presentation.
Data. Jen-Hao Rick Chang and Xiaoming Zhao developed the data rendering scripts convert 3D assets into the surface light fields required for Eq. (1). Xiaoming Zhao developed the data preprocessing pipeline to clean and curate the dataset to ensure high quality inputs for model training. Jen-Hao Rick Chang implemented the efficient dataloader based webdataset (webdataset development team, 2026).
Model design. Jen-Hao Rick Chang and Xiaoming Zhao were primarily responsible for developing the encoder (Sec. 3.3), decoder (Sec. 3.4), and image-to-3D generative model (Sec. 3.5). Jen-Hao Rick Chang designed and implemented the data structures that enable the efficient 3D patchification process in Fig. 3, designed encoder, velocity decoder, and iterates on Gaussian decoders, and he built the 3D library, p1 bs.
24

---

Published as a conference paper at ICLR 2026
latent tokens (k, d)
Inear
velocity (1. 3xyz)
MLP (SWIGLU)
linear
layenorm
lincar
alleeinn block
layemorr
linoar d
Yuse bused sol mime atlention
mastoon
gtling 2
+6
tokens (k, df)
quay rry. WaTR
modulation
Ingernom
MLP (SWGGLL
giing
layenoorm
attention
choss allentioo
block
qucn
kay. volto
softimax
modulation
lincer
[iner
Inner
SILU
layernami
[ayenoor
Iayernorm KV
Tineer
linear q
linear kv
linear
suery'
key
valse
initial tokens (k. d')
position-encoded points (N, d)
position-cnceded
position-cncoded point (1. d')
Intent tokens (K. d)
flow matching time (d'.)
Figure S2: Encoder architecture. The model uses a feature dimension of df = 512, while the hidden layer in MLP uses feature dimension of 2048. The number of heads for cross-attention and self-attention is 16. The input dimension = 396, which includes 3D location, position- encoded 3D location. RGB, position-encoded RGB, and Plucker coordinates. Our latent has 8192 and 32. Please refer to Sec. for position encoding details.
Figure S3: Velocity decoder architecture. The model uses feature dimension of 512, while the hidden layer in MLP uses feature dimension of 2048. The number of heads for cross-attention is 8. Our latent has A 8192 and d 32. We have d' = 195, which includes 3D location and position-encoded 3D location. Mcanwhile, d" 64, which is obtained by applying a linear layer to time-step position encoding in Eq. (S2). Please refer to Sec. E for position encoding details.
Model training. Jen-Hao Rick Chang and Xiaoming Zhao led the training of all models de- seribed in this study, including both tokenizers and the generative models presented in the paper. Specifically, Jen-Hao Rick Chang trained the tokenizer (Fig. S2, Fig. S3, and Fig. S4), occupancy prediciton decoder (Fig. S5), and generative model (Fig. S7). Xiaoming Zhao trained the tokenizer (Fig, S2, Fig. S3, and Fig. S4), mesh decoder (Fig. S6), and generative model (Fig. S7). Jen-Hao Rick Chang implemented the training backbone.
Evaluation. Jen- Hao Rick Chang and Xiaoming Zhao led the evaluation strategy and experimental design. Xiaoming Zhao developed the evaluation pipeline and frameworks used generate all quantitative results reported in the paper. Dorian Chan conducted the geometric evaluations for the baseline methods presented Tab. 2 while Xiaoming Zhao completed the remaining tables.
25

---

Published as a conference paper at ICLR 2026
Structured Latent (res, res, res, d'))
lincer
position, rotation, scaling
SH coeff, opacity
linear
Tinear
MI. (Swici.
MLP (SWIGLU)
MLP (liee GELU, lineur)
MLP (SwiGLL
Styomorm
self ottiattoon blok
self attemion black
lincer d
5octmax attention
attention
[imaer
lnncte
lincar
misnom
Tyymmomm
iincr
Inner
tokens (k, di)
syymoorm
tokens (I., df)
MLP (linear, GELL lssar)
Jayernom
MLP (SWiGLU)
CLOSS attension
layemorm
attention block
Emsnorr
minorr
linear
[inear
lineur
liner
plobal softmax attention
layemonn
layernoem
Misnoono
missom
Taany
Bay,
[iinerr
lincar
lincar
Lyyeronn
layernorm kv
learnable
(res, res, res,
key
valez
linerr
Innar
position encoded
position encoded
locaiton
latent tokens
vexel grid indices latent tokens (res, res, d')
(a. d')
(k, d)
(k. d)
Figure $4: 3D Gaussian decoder architecture.
Figure $5: Occupancy decoder architecture.
The model uses a feature dimension df = 512, while the hidden layer in MLP uses feature di- mension 2048. The number heads for cross- attention and self-attention is 8. Our latent has = 8192 and d = 32. We have d' = 195, which includes 3D location and position-encoded 3D location. Please refer to Sec. E for position encod- ing details.
The model uses a feature dimension of df = 512, while the feature dimension for QKV in cross/self- attention is 1024. The hidden layer in MLP uses feature dimension of 2048. Our latent has = 8192 and = 32. The number of heads for cross-attention and self-attention is 8. We have d' = 771, with Mmin 0, Mmax = 5, and F 128 in Eq. (S2) for encoding 3D location. We use resolution of 16, i.e., res = 16. Please refer to Sec. E for position encoding details.
26

---

Published as a conference paper at ICLR 2026
FlexiCubes parameters
convid
proppoom SILU
convid
Subdivide
cortid Subdivide
groupnorm, SILU
(LP(S
Sayemorm
SILU
JUn mole
self attemion block
linear
voitel votte-hhsee softmas attentinn
bamalao gaag
mistum
missionss
lincar
luncar
incor
syy UTDUETI
an
ML llieen GELD linears
modetaion
tokens (n, di)
(S.
crois ottention book
laymoorm
cross attention block
linear o
gJ0bal sofumax ariension
self aturiius
missomm
mesoum
lincar
lineer
linear
eprry silee
Layensoem
lavernorm
modilation
linzur kv
H6
an A wake
Byoman
-28
leannbbe embedking
Finzur
(k, di)
lnca
Vll Prinnor Car3.l), Inver)
position-encoded location (n, d')
latent tokens (k,
position encoded flow matching time (e".)
noised latest tokens conditioned image feature (1.d)
(k.d)
Figure S6: Mesh decoder architecture. The model
Figure S7: Generative model DiT architecture. The model uses feature dimension of df 1152, while the hidden layer in MI.P uses a feature di- mension of 4608. Our latent has d = 32. The number of heads for self-attention and cross-attention is 16. The feature dimension for conditioning image d' 2048. We have d" 64, with Mmin 0. M 12, and F = 32 in Eq. (S2) for encoding flow match- ing time step. Please refer to Sec. for position encoding details.
feature dimension of df 512, while the hidden layer in MLP uses feature di- mension of 2048. Our latent has d = 32. The number of heads for cross-attention and self-attention is 16. We have d' 195, which includes 3D location and position-encoded 3D location. Please refer to Sec. E for position encod- ing details.
8192 and
8192 and
27

---

Published as a conference paper at ICLR 2026
Ours (SII
Ours (SII
Ours s
Ours (SII 3)
TRELLIS
Ground truth
Figure $8: Rendering with various spherical harmonics degrees reconstruction. When restricted to zeroth-order spherical harmonics, our 3D Gaussians produce view-independent appear- ance and avoid the over-exposed regions observed in TRELLIS's renderings. As we progressively incorporate higher-order spherical harmonies, our method yields increasingly pronounced view- dependent effects. Mesh credit: DigitalSouls (2019).
Ours
Ours (
Ours (
Ours (SH 3)
TRELLIS
(SH
Input Image
Figure S9: Rendering with various spherical harmonics degrees in generation. When restricted to zeroth-order spherical harmonics, our 3D Gaussians produce a view-independent appearance and avoid the over-exposed regions observed in TRELLIS's renderings. As progressively incorporate higher-order spherical harmonics, our method yields increasingly pronounced view-dependent effects. Mesh credit: Bleanie (2025).
28


Reviews:
[{"review_id": "oidyP9lSrU", "reviewer": "Reviewer_2CdX", "summary": "This paper propose a 3D latent representation that jointly models object geometry and view-dependent appearance from RGB-D inputs, capturing both within a unified latent space. This approach reproduces realistic effects like specular highlights and Fresnel reflections and enables high-fidelity 3D generation consistent with the input image’s lighting and materials.", "questions": "please see weakness", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The writing is clear and easy to follow.\n\n2. The proposed pipeline appears novel; however, the motivation could be made more convincing or better justified.", "weaknesses": "Unclear motivation:\nThe motivation behind the proposed method remains unclear. Existing approaches typically avoid incorporating view-direction information because doing so simplifies subsequent relighting tasks. If all lighting- and view-related information are modeled jointly, it becomes questionable how the proposed model can perform relighting and be naturally integrated into a scene without introducing inconsistent illumination or lighting variations. The authors should clarify the motivation and explain how their approach maintains relighting consistency under such conditions.\n\nLack of ablation studies:\nThe current version of the paper does not include any ablation studies in the main text. A detailed ablation analysis is essential to demonstrate the contribution of each component and validate the effectiveness of the proposed method. The absence of such studies significantly weakens the paper. The authors should provide comprehensive ablation results to substantiate their design choices; without them, the paper’s claims are difficult to evaluate.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C0WSOLVEqr", "reviewer": "Reviewer_42pJ", "summary": "The paper presents a novel autoencoder that learns a latent representation capturing the view-dependent appearance of 3D assets in addition to geometry. Besides reconstruction, the latents can be used to supervise a DiT model, and in turn can be used to generate 3D assets from a single image.", "questions": "Q1. Is the flow matching decoder only used to predict the coarse occupancy grid, which then guides the Gaussian decoder to predict the splat geometry (including xyz) and appearance? More details of the Gaussian decoder need to be provided (similar to Q3 below). Are the splat mean and covar predicted directly in world space, and are they scaled or clamped to lie within scene bounds?\n\nQ2. What is the k used for selection of nearest neighbors for cross-attention/3D patchification?\n\nQ3. For the generative model, what is the structure/dimensions of the learnable positional encoding? What is the patch size and architecture of the learnable patchifier? For reproducibility, authors should include a table of dimensions and structure for each module, instead of only reporting the number of parameters.\n\nQ4. What are the inference-time memory and runtime costs for image to 3D generation?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is well written and mostly easy to follow.\n- Modeling of view-dependent appearance while retaining performance on geometry modeling is a novel and valuable contribution.\n- Experiments and ablations are sufficient to evaluate architecture and performance. LiTo consistently achieves strong empirical results.", "weaknesses": "- Some architectural details are missing for reproducibility (see questions).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xU1iZTfk4q", "reviewer": "Reviewer_2eED", "summary": "This paper proposes a 3D latent representation that jointly encodes both geometry and view-dependent color. The model takes surface locations, view directions, and colors as inputs, and encodes them into $k$ tokens of $d$ dimensions. A decoder then reconstructs geometry via predicted velocities and view-dependent Gaussian parameters. To assess the quality of the latent space, the authors train a generative model that synthesizes 3D latents from a single image. Experiments on multiple benchmarks demonstrate that the proposed representation achieves high fidelity, particularly in high-frequency details.", "questions": "1. How does the inference time compare to TRELLIS?\n\n2. In Table 3, why does the method show worse performance on novel-view metrics (FID_dino and KID_dino)\n\n3. Since this representation encodes view-dependent color, I’m curious how it performs on more challenging benchmarks such as Shiny Blender and NeRFactor, which involve complex lighting and reflectance conditions.\n\nRef-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields\n\nNeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Comprehensive experiments. The experimental section is thorough, with well-designed ablation studies that clearly justify key architectural and training choices.\n2. High reconstruction fidelity. The method achieves superior fidelity in input images, which is an important aspect of 3D generation quality.", "weaknesses": "- The view-dependent color entangles the representation with environment lighting. While this improves reconstruction fidelity, it limits the method’s applicability for tasks requiring relighting or lighting-invariant representations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KH8qnpMunN", "reviewer": "Reviewer_wBM6", "summary": "This paper presents a 3D latent representation that simultaneously models object geometry and view-dependent appearance. Unlike previous methods that separately reconstruct geometry or predict view-independent appearance, this approach encodes RGB-depth samples of a surface light field into compact latent vectors. The unified latent space effectively captures realistic visual phenomena, including specular highlights and Fresnel reflections, under complex lighting conditions.\nThe main contribution lies in introducing a unified 3D latent representation that jointly learns geometry and view-dependent appearance from surface light field samples. Additionally, the authors propose a latent flow matching model conditioned on a single image, enabling the generation of 3D objects with lighting- and material-consistent appearances. Experimental results demonstrate superior visual realism and fidelity compared to existing approaches.", "questions": "-1. In this work, the authors utilize an RGB-D dataset. I wonder whether using an RGB-only dataset augmented with depth predicted by a large model could achieve reconstruction results comparable to those obtained with an RGB-D dataset. In other words, how significant is the impact of accurate depth information on the reconstruction quality?\n\n-2. The manuscript mentions that LiTo uses 150 views during training. It would be informative if the authors could provide a brief discussion or evaluation of the reconstruction results when using fewer input views. Such an analysis could help assess the robustness of the method to reduced view availability.\n\n-3. The paper appears to address two aspects: jointly modeling object geometry and view-dependent appearance, and single-view 3D reconstruction. These seem to correspond to two distinct tasks. Could the authors clarify which of these is the primary focus of the work? My understanding is that single-view 3D reconstruction is more akin to a downstream application of the learned model, so a clear explanation would help resolve this point of confusion.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Originality: It is the first work to unify 3D geometry and view-dependent appearance in a single framework by encoding surface light fields sampled from multi-view RGB-D images. It employs 3D Gaussians with 3rd-order spherical harmonics (SH) to reproduce view-dependent effects such as specular highlights and Fresnel reflections, which traditional methods fail to capture. Additionally, it introduces a k-NN-based 3D patchification and voxel-aware attention structure, overcoming the efficiency bottleneck in processing million-scale surface light field samples.\n\nQuality: The experimental design is rigorous and validated on datasets including ObjaverseXL and Toys4k. In reconstruction tasks, it achieves a PSNR of 36.45±4.00 (surpassing TRELLIS by 3.27), while in generation tasks, it reaches an FID of 8.193 at the conditioning view (better than TRELLIS’s 12.84). The dual-decoder supervision (flow-matching geometry decoder and 3D Gaussian appearance decoder) balances geometric accuracy and appearance realism, producing reproducible results.\n\nClarity: The framework follows a clear sampling–encoding–decoding–generation pipeline. Core formulas (e.g., the flow-matching loss and the radiance loss) are clearly annotated, and key concepts (e.g., surface light field, 3D Gaussians) are explained consistently. The connections between symbol definitions, method descriptions, and experimental procedures are well aligned, ensuring easy comprehension.\n\nSignificance: It fills the gap of “separated geometry and appearance modeling” in 3D latent representations, providing an efficient framework for 3D content generation and virtual-real fusion. Its ability to avoid precomputed coarse geometry reduces data dependency, and the view alignment of single-image-to-3D generation promotes the application of downstream tasks (e.g., game asset creation, AR modeling).", "weaknesses": "-1. In the expression $I_{gt}=Render(scene,H,E)$ presented below Eq. 3, the term scene is used, but its definition or corresponding description is not provided. Could you clarify what it represents?\n\n-2. The results presented in Table 2 indicate that LiTo still exhibits a performance gap compared to TripoSF and TRELLIS. I understand that this may be due to the differences in the types of input data. The authors could frame this as a deliberate design choice, prioritizing appearance over geometry; however, this trade-off is a key characteristic of the method that should be more clearly articulated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)