SceneTransporter: Optimal Transport-Guided Compositional Latent Diffusion for Single-Image  Structured 3D Scene Generation
Authors: Ling Wang
Hao-Xiang Guo
Xinzhou Wang
Fuchun Sun
Kai Sun
Pengkun Liu
Hang Xiao
Zhong Wang
Guangyuan Fu
Eric Li
Yang Liu
Yikai Wang
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Scene Generation
Part-aware 3D Generation
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We introduce SceneTransporter, an end-to-end framework for structured 3D scene generation from a single image. While existing methods generate part-level 3D objects, they often fail to organize these parts into distinct instances in open-world scenes. Through a debiased clustering probe, we reveal a critical insight: this failure stems from the lack of structural constraints within the model's internal assignment mechanism. Based on this finding, we reframe the task of structured 3D scene generation as a global correlation assignment problem. To solve this, SceneTransporter formulates and solves an entropic Optimal Transport (OT) objective within the denoising loop of the compositional DiT model. This formulation imposes two powerful structural constraints. First, the resulting transport plan gates cross-attention to enforce an exclusive, one-to-one routing of image patches to part-level 3D latents, preventing entanglement. Second, the competitive nature of the transport encourages the grouping of similar patches, a process that is further regularized by an edge-based cost, to form coherent objects and prevent fragmentation.  Extensive experiments show that SceneTransporter outperforms existing methods on open-world scene generation, significantly improving instance-level coherence and geometric fidelity. Code and models will be publicly available at \url{https://2019epwl.github.io/SceneTransporter/}

Published as a conference paper at ICLR 2026
SCENETRANSPORTER: OPTIMAL TRANSPORT-GUIDED COMPOSITIONAL LATENT DIFFUSION FOR SINGLE- IMAGE STRUCTURED 3D SCENE GENERATION
Ling Wang 1.2+
Haoxiang Guo
Xinzhou Wang Fuchun Sun
Kai Sun Eric Li
Pengkun L Liu
Hang X,a
Zhong Wang' Guangyuan Ful
Yang Liu Yikai Wang
Xii'n Research Institute of Hi-Tech
Tsinghua University, Skyyork AI,
"Tongji University, FFudan University,
"Bejjing Normal University yikaiw@bnu. edu.on
Ecsunêmail mrsunemaiL.tsingua.. edu. en
ABSTRACT
We introduce SceneTransporter end-to-end framework for structured 3D scene generation from a single image. While existing methods generate part-level 3D objects, they often fail organize these parts into distinct instances in open-world scenes. Through a debiased clustering probe, we reveal a critical insight: this fail- ure stems from the lack of structural constraints within the model's internal as- signment mechanism. Based on this finding, we reframe the task of structured 3D scene generation a global correlation assignment problem. To solve this, Scene- Transporter formulates and solves an entropic Optimal Transport (OT) objective within the denoising loop the compositional DiT model. This formulation im- poses two powerful structural constraints. First, the resulting transport plan gates cross-attention to enforce an exclusive, one-to-one routing of image patches to part-level 3D latents, preventing entanglement. Second, the competitive nature of the transport encourages the grouping of similar patches, a process that is further regularized by edge-based cost, to form coherent objects and prevent fragmen- tation. Extensive experiments show that Scene Trenssranprrrerr outperforms existing methods open-world scene generation, significantly improving instance-level coherence and geometric fidelity. Code and models will be publicly available at https://2019epte github...........g io/ /tssceneTrrasppoter//
INTRODUCTION
The capacity to generate high-quality, scalable 3D scenes is cornerstone for the next generation of immersive technologies and embodied AI. While advancements in generative Al promise scalable synthesis, the vast majority of scene generators still produce monolithic, unstructured meshes (Vo- drahalli et al., 2024; et al., 2025; Xiang et al., 2025). For real-world pipelines, fused 3D shell functionally inert. Downstream tasks tasks-including material assignment, realistic physics simulation, asset retrieval and placement, and fine- ime-grained editing editing-require a structured scene mesh with ex- plicit, instance-level object-context disentanglement, A common "divide and conquer" solution at- tempts this by first segmenting an input image, then generating a 3D model for each part, and finally assembling them into scene (Huang al., 2025; Chen et al., 2024b; Ardelean et al., 2024). This multi-stage pipeline, however, is inherently brittle; its heavy reliance on 2D segmentation makes incapable of handling heavily occluded objects, and transforms even minor 2D segmentation flaws into significant 3D geometric artifacts.
In recent years, end-to-end structured generation has emerged as a promising alternative (Lin al., 2025b; Tang et al., 2025; Chen et al., 2025a; Yang et al., 2025b), enabled by the development of large-scale repositories with explicit part annotations (e.g., Objaverse (Lin et al., 2025a)). In this paradigm, a scene is no longer represented as a single, indivisible latent code but as collection of
"Equal contribution
1 "orresponning authors

---

Published as a conference paper at ICLR 2026
(a) Compositional Generation Methods
(b) With Multi-Instance Diffusion
(6) End-to-end Generation Method
Cljject Image
Objicd Image
Cojecl
Single Scene Image
Image
Insainsing
Inaziniing
Multi-istance 3iffusion
Composttional 3D Latent Dithasion
3D Ob ect Ceneration
Object Generction
Osject Generation
Ofject Generation
Fart level 3D Object Ceneration
Layout Optinization
Multi-instance Attention
OT Plan-Gated Aroose-Atteetton
Figure 1: Comparison between our end-to-end scene generation pipeline in (c) with compositional 3D latent diffusion and existing "divide and conquer" methods.
disentangled latent tokens, where each set of tokens corresponds to distinct 3D part. Although these methods show great promise for generating structured objects and indoor scenes, open- world structured scene generation remains underexplored. As illustrated in Figure 4, when naively applied to large, complex open-world scenes, uncovers two persistent 3D pathologies: (i) Structural
iittaatttion-smmmnti instances within the scene fail to form disjoint parts; and (ii) Geometric Redundancy-mulle multiple latents "compete" to describe the same geometric area.
To address these challenges, first perform Probing Latent Structure with Debiased Clustering. This probe reveals the primary bottleneck: a lack of structural constraints within the model's 's assign- ment mechanism prevents the formation of stable instances. To this end, we introduce SceneTrans- porter, a framework that reframes the task as an Optimal-Transport-- Guided Correlation Assignment problem. To solve this, define principled entropic Optimal Transport (OT) formulation that cal- culates a globally optimal transport plan between the set of image patch features and the part-level tokens. This optimization imposes powerful structural constraints through two key components. First, an OT Mlan-Gated Cross-Attention module uses the plan to enforce a hard, one-to-one rout- ing, ensuring each patch contributes only one part, thus preventing feature entanglement. Second, the competitive nature of the transport incentivizes patches with high feature similarity to be as- signed to the same token, naturally forming coherent structures. To further refine this grouping and ensure sharp object boundaries, izes assignments that cross salient image edges within the transport objective. Extensive experiments show that our approach outperforms existing methods, setting new framework for structured 3D) scene generation.
introduce an Regullazed Assignment Cost, which penal-
In summary, contributions are follows:
We design Debiased Clustering probe based on Canonical Correlation Analysis (CCA) to investigate the latent structure of part-level generators. The probe reveals that the core failure lies in the assignment mechanism due to a lack of structural constraints.
We reframe the task an Optimal-Transport (OT)-Guided Correlation Assignment prob- lem. To solve this, we propose Scene'rransporre Transporter, which leverages an entropic OT frame- work to impose two powerful structural constraints: OT Plan-Gaaeed Cross-Attention module for exclusive one-to-one routing, and an Edge-Regularized Assignment Cost for coherent structures grouping.
Transprra achieves of-the-rrt performance on open-world 3D scene genera- tion, demonstrating significantly improved instance- coherence and geometric fidelity.
RELATED WORK
3D Scene Generation. Early approaches to scene synthesis were dominated by retrieval-base methods, which compose scenes by aligning assets from 3D database with an input image (Feng
2

---

Published as a conference paper at ICLR 2026
et al., 2023; Gao et al., 2024; Langer al., 2024). These methods, however, are fundamentally limited by the scope of their database and often suffer from alignment errors from a single view. The advent of large-scale generative models has shifted the focus towards creating novel content. One dominant approach is 2D-prior distillation, a multi-stage process that first generates consistent multi-view images (Chen et al., 2025b; Li et al., 2024), videos Sun et al. (2024); Wang et al. (2025); Liang et al. (2025), or panoramic images Yang et al. (2025a); Zhou et al. (2024) via powerful dif- fusion models, and then reconstructs a 3D scene from these views using techniques like Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (3DGS). To address the geometric inconsisten- cies of 2D-lifting, another line of research develops models that operate directly in native 3D latent spaces (Meng et al., 2025; Wu et al., 2024; Lee et al., 2025; Ren et al., 2024; Lee et al., 2024; Liu et al., 2024b). While this improves multi-view consistency, it requires large-scale 3D datasets and generalizes poorly to in-the-wild scenes. Crucially, both of these generative avenues converge on the same limitation: their output is a single, unstructured monolithic mesh, which lacks the explicit object-level separation needed for most downstream tasks.
3D Structured Scene Generation. A common "divide and conquer" strategy for structured scene generation is to process scene piece by piece (Chen al., 2024b; Ardelean al., 2024; Han al., 2024). This typically involves a pipeline that segments the input, generates 3D models for the seg- ments, and then arranges them. Such methods benefit from modularity but suffer from two key weaknesses: the accumulation of errors across stages and a failure to maintain global consistency. While more integrated approaches like MIDI (Huang et al., 2025) use a multi-instane mechanism to capture inter-object interactions, they remain fundamentally limited by their reliance on segmenting visible content, preventing the reconstruction of occluded parts. Another line of work explores end-to-end compositional generation using 3D diffusion models that bind latent to- ken subsets to semantic parts (Lin et al., 2025b; Tang et al., 2025). While these models excel at generating structured objects, they fail to generalize to complex open-world scenes due to a scarcity of scene-level part annotations. This generalization gap manifests as two distinct geometric patholo- gies (Figure 4): structural mispartition and geometric redundancy. Our work offers the first direct solution, introducing a method to guide the compositional cross-attention of pretrained generators and explicitly correct these failure modes.
attention
Attention control in diffusion models. Attention maps in image and video diffusion models have become a compact, training-free control interface: by manipulating or augmenting self- and cross- attention at sampling time, practitioners can steer layout and composition (Chen et al., 2024a; Hertz et al. 2022; Patashnik et al., 2023), transfer fine-grained texture and style (Hertz et al., 2024), enforce multi-prompt alignment (Chefer al., 2023) and temporal coherence for videos (Cai et al., 2025; Liu et al., 2024a; Qi et al., 2023), and perform a variety of image-editing tasks (Parmar et al., 2023; Yang et al., 2023; Mokady et al., 2023; Park al., 2024; Tumanyan al., 2023). Despite its success in 2D, attention-based control has seen little application to 3D latent diffusion-largely because 3D representations differ fundamentally. In 2D, attention queries live explicit positions. By contrast, 3D data commonly takes irregular forms. For instance, in the vecset- based latent representation (Zhang et al., 2023) where a shape is encoded an unordered set of latent vectors, positional information is not explicitly structured. The absence of a canonical spatial index therefore makes attention manipulation for 3D editing substantially more difficult and still largely unexplored.
regular spatial grid with
3 METHODOLOGY
3.1 PRELIMINARIES: COMPOSITIONAL 3D GENERATORS
A typical compositional 3D generator consists of two key components: (i) a Transformer- Variational Autoencoder (VAE), such as 3DShape2VecSet, which encodes 3D mesh into set of latent vectors and decodes them back into a geometric field (e.g., SDF), and (ii) a denoising network that operates within this latent space to reverse diffusion process and synthesize clean latents from noise. Prior to going further, we first introduce the notations and preliminaries used in our paper, especially those for the attention mechanism, as its core lies in modeling the geometric dependencies among shape components. At denoising step t, the model maintains N part-specific token blocks (1) IN N= where each 2, KxD contains K tokens (width D) for part i. A common practice is
3

---

Published as a conference paper at ICLR 2026
Image
PartPacker
Cluster
Cluster with CCA
VAE
Figure 2: Qualitative Results on Vecset-based Latent Probing. Cluster and Cluster with CCA probes that perform in the compositional latent space of PartPacker; VAE clusters the latent obtained by encoding the fused geometry produced by PartPacker into the VAE. Colors denote part assignments.
to add learnable part identity embedding e; E RR all tokens of part i, where 1 is a column vector of ones:
(1) + 1kee
( concat (z)
E (NK)xD
(1)
Conditioning comes from single RGB image encoded by frozen DINOV2 d(x) = I R L.Dss. Linear projections yield queries, keys, and values (omitting head indices for brevity):
Q lQ E (R(N))xd
K = 4x E Lxddd
V= = (I) E Rxdd
(2)
Row-normalized cross-attention fuses image evidence into the latent tokens. In practice, to increase expressiveness, multi-head attention is used: the projections split into H heads (Q,, K,, Va), attention is computed per head M, Softmax Q / /vd), the head outputs are concatenated, and learned output projection produces the final update,
Z (M, Wo.
The updated sequence proceeds through the DiT block and the denoising schedule. At inference, the final clean sequence is split back into { and each subset decoded by the pretrained VAE to obtain part mesh; parts are then fused to form the full scene.
3.2 PROBING LATENT STRUCTURE WITH DEBIASED CLUSTERING
Part-level generators carry a strong inductive bias toward a part-level organization that models cor- relations among parts of the same object (e.g., chair legs and back share style and structure). In contrast, scene synthesis models an instance-level organization, where distinct entities are largely conditionally independent.
Applying the former to the latter gives rise to two observable failure modes (Figure 4); Structural Mispartition, where geometry for single object is scattered across multiple part-tokens, and Geo- metrie Redundancy, which leads to the geometry overlap between objects. Intriguingly, despite this incoherent part-level assignment, the union of all parts often reconstructs the overall ably well. This raises a concrete question: Can we recover a coherent, instance-level organization from these part-level latents? To probe this, introduce debiased clustering procedure with three steps: identify the shared subspace via canonical correlation analysis (CCA) on the part-level latent sets; suppress by projecting tokens onto the orthogonal complement isolate object-specific vari- ation; and regroup the residual tokens to obtain semantically coherent sets. Implementation details are provided in Appendix B.
reason-
An illustrative example is provided in Figure 2. Our probe offers a clear signal: clustering raw part tokens fails to produce stable instance groupings, whereas clustering the CCA-debiased residual tokens reliably succeeds. This striking contrast reveals critical flaw in current part-level generators: while their learned features implicitly contain the necessary information for correct associations, the models fail to establish these associations explicitly. As result, the leamned groupings are left weak, fragmented, and entangled with scene context. Based on this insight, we argue that a paradigm shift required. Instead of hoping for organization to emerge from implicit learning process, must introduce explicit structural constraints to guarantee a coherent, instance- structure by design.
4

---

Published as a conference paper at ICLR 2026
Tcaneoranee
Paizhlly
mg
img
ims
ing
incoder
Part Errees,
OTT
Q KV
Q KV
Q KV
KV
2
2
22
Part
Optimal-Trnssport Guided Correlation Assignment
Transport in the Attention Map
Pirt Avaragel
Edgo Gublod
Pooling
Part W
Orat Natns
Seehhom
Figure 3: Overview of the Scenspranpporter pipeline. At each denoising step t, our Optimal- Transport-Guided Correlation Assignment framework formulates a global OT problem between image patches and part-level tokens within the compositional latent DIT. We compute part-patch cost from Q/K similarity, regularized by image edges, and solve for an optimal transport plan using Sinkhorn iteration. The OT plan gates the cross attention to enforce an explicit patch-to-part routing, and the resulting gated attention map updates the latent Attention maps transport over time, showing assignments becoming sharper and more instance-comsistent.
3.3 OPTIMAL- TIIMAL-TRAAA TATTRAAPPOR-GUDDER CORRELATION ASSIGNMENT
To impose the explicit structural constraints suggested by our probe, we reframe the routing of visual evidence to part tokens globally optimal correlation assignment problem. We propose solve this problem at each denoising step using the principled framework Optimal Transport (OT). OT provides the exact constraints needed to combat the failures observed. By enforcing one-to- one assignment, it ensures exclusivity, preventing the feature entanglement that causes objects to blend. Simultaneously, by requiring each part-latent to meet coverage budget, it encourages the aggregation of large, coherent patch regions, directly mitigating semantic fragmentation. This OT- guided mechanism, which we detail next, transforms the generation from unconstrained process into a structured, well-posed optimization.
OT Problem Setup. The core challenge of routing is to ensure globally consistent allocation:
image patches should not be double-counted by multiple 3D parts, and 3D parts should not compete for the same patch information. To address this, we model the assignment as a single optimization problem with constraints. We compute an entropic OT plan A, RN to allocate the mass from L image patch features to the latent tokens of N 3D parts:
A arg min (C,, A) & H(A) s.t. A1 pI Al=V, A20
where the matrix C, E NxL measures the incompatibility between patches and parts, will be detailed in Eq. (12). Crucially, the constraints enforce a budget for each part through μa E N which prevents any part from being "starved" of information'. They also ensure that each patch contributes an equal amount of mass by + 1,,, meaning each patch contributes 1/L of the total mass. H ((A) ) the entropy of the transport plan A, and e, the entropic regularization term, which helps to smooth the solution. The resulting transport plan A, minimizes the assignment cost while adhering to the capacity constraints.
OT Plan-Gated Cross-Attention. Given the entropic OT plan A, E NXL from Eq. (4), we inject this global assignment into the native cross-attention mechanism by using it to gate the in- coming visual information multiplicatively. This approach enables us regulate the amount of evidence each image patch can contribute to each 3D part, while preserving the standard Softmax attention to decide which the available patches each token should focus on. To achieve this, we
A N
{x R.N R N z 1} is probability simplex. /a E A satisfies N Pi with μ4
5

---

Published as a conference paper at ICLR 2026
first convert the transport plan A, into per-part patch weights w, A. through row- www-nomaaizatton.
We then design a bounded, identity-preservie preserving gating function 2b(-) that transforms these weights into a gating signal:
M (w) = Eg + (1- w
E [0,1], 20 Eg [0,
(5)
where the guidance strength 7, determines how quickly the gate closes for patches with low weights, while small floor Eg prevents any patch from being entirely starved of attention. Critically, when guidance is disabled (A, = 0), the function outputs exactly I, guaranteeing that the mechanism reverts to standard cross-attention. This gating signal is subsequently applied to modulate both the keys and values in the attention computation for each part. For a specific head ht, the keys K₁, and values Vn are scaled row-wise by the gating signal derived from the part-specific weights wi())- This results in a unique, gated "view" of the image memory for each part i:
K G.:) ('xess(Wi(ii) Kn(i;;,, G.:) (xx.s.lWili)) Vn(i.)).
(6)
The queries for the 3D part focus solely on this gated memory. By gating both keys and values, we ensure that routes suppressed by the OT plan contribute neither large logits nor large feature values, resulting in a clean, cppssity-consistett routing with low leakage. The final representations for each part computed via standard multi-heat attention on their respective gated inputs and then stitched together:
Q (K(
R = Softmax
H
M V
Vdd.
(S.,)) Concat Wo: i= 1
2 E (NN))xd
(8)
Edge-Regularized Assignment Cost. In cluttered scenes, patch features near contact boundaries often seem compatible with multiple parts. As a result, the transported patch tends to "leak" in- formation across adjacent objects. To address this issue, we introduce a weak but spatially precise prior: an edge map E E [0, x obtained from the conditioning image (e.g.. Canny/Sobel or learned edge detector). The goal is to encourage region-wise consistency the affinities between parts and patches while discouraging the spread of information across image edges.
(t) Let {q, Jim1 represent the aggregated token of the i-th part. Meanwhile, let { denote the patch keys. We first compute the raw cosine similarities between these prototypes and keys,
Su
cos(q!") k,) E [-1,1].
We downsample the edge map match the patch grid, resulting in E2 E (0, H),,,,, where H,W,, L. and construct a 4-neighborhood graph among the patches. For a patch j and its neighboring patch f (EN()), we define an edge-aware coupling weight:
Whe
exp
Yedge max (i), ((O)).
Yedge > 0,
(10)
which close to 1 in smooth regions and decays near image edges. We then perform a single edge-aware smoothing step the affinities,
B St Enen Si,e
Aedge 20. 0.
(11)
+ AdDDEnENN))
Eq. (1I) facilitate the spread of evidence within regions characterized by low edge strength, while simultaneously inhibiting the spread across edges. The process results in affinities that are piecewise smooth and respect boundaries. To further intensify the competition among parts for each patch, we implement a contrast normalization
per-patch basis and obtain St,, from St.y
Finally, our OT cost defined as the margin-enhanced dissimilarity measure that incorporates the edge map to guide the assignment cost, ensuring that the affinities respect the boundaries and main- tain region-wise consistency,
C((i,)) = 1 (1 (1.g).. 2
(12)

---

Published as a conference paper at ICLR 2026
Method
Instance Mask
Geometry Fidelity
Disentanglement
Inference Time
ULIPT
ULIP-21
Uni3D
JoUnnx + JoUneen
MIDIHuunng al. (2025)
0.1397
0.2763
0.2518
0.0458
0.1642
149.68
PanCCratterL.in (2025h)
X
0.3096
0.2635
157.97
PartPackerTang et al. (2025)
0.1417
0,3083
0.2887
0.0319
0.2142
47.41
Ours
X
0.1466
0.3220
0.3021
0.0101
0.0926
54.99
Table 1: Quantitative Comparison on Structured 3D Scene Generation across Methods. Bold values indicate the best scores, while underlined values indicate the second- -est scores among the fair comparison.
When Aedge = 0 (or the edge map consists entirely of zeros), Eq. (12) simplifies to the standard cosine-backe based cost with contrast normalization. A positive Audge suppresses cross- bousdary transport without requiring any semantic masks: patches opposite sides of strong edges receive weak mutual support Eq. (11), so the subsequent All operations differentiable and aaaadppat-agnosiie and the hyperparameters (Aedge Todge) can be annealed across denoising steps (stronger in carly steps, weaker later) to promote clean separation first and fine detail later.
solver more reliably assigns them to different parts.
EXPERIMENTS
4.1
EXPERIMENTAL SETUP
Baselines We compare approach against recent ofate-of-the-art methods for part-level 3D gen- eration- PARTCRAFTER (Lin et al., 2025b), PARTPACKER (Tang et al., 2025), and MIDI (Huang al., 2025). We ensure fairness in our evaluation by using the officially released source code and checkpoints for each baseline.
Metries. Following prior work (Tang et al., 2025; Zhao et al., 2025), we evaluate geometry fidelity with ULIP (Xue et al., 2023; 2024) and Uni3D (Zhou al., 2023). These models learn unified rep- resentations across text, image, and point cloud modalities. Since both ULIP-2 and Uni3D require colored point clouds input, we assign uniform white color to all mesh outputs before computing the metrics. For part disentanglement, we voxelize the canonical space into a 64 grid, binarize oc- cupancies {O: Jezz for the N' generated parts, and compute pairwise Intersection-over- Union (IoU). We report the mean of the top-20 largest IoUs, and the maximum IoU; lower values indicate better disentanglement (less inter-part overlap).
Implementation details. We build SceneTTransporter the open-source part-level 3D) generator of Tang et al. (2025), which uses a rectified-flow DiT with 24 attention blocks and supports arbitrary part counts via dual-volume packing strategy. In our setup we instantiate dual volumes, yielding a compositional latent of size 4096 x 2 = 8192 with channel width 64. For the OT solver, we use stabilized log-domain Sinkhorn with 40 iterations. We enable OT planggtte attention in the first half of the DiT blocks, and use standard cross-attentin in the remaining blocks to refine the global geometry. All other inference settings follow Tang al. (2025).
4.2 COMPARISONS WITH STATE-OF-THEE-RRSS
To demonstrate the effectiveness and breadth of our approach, we evaluate on an open- world set of 74 high-quality scene images collected from the Web, spanning diverse styles. As shown in Table 1, our method achieves the highest geometry fidelity and the second-lowest inter-part overlap. The absolute lowest IoU is reported by PartCrafter, largely because it discards background/groumm re- gions during generation, which trivially reduces overlap but also compromises scene completeness. Although our runtime is slightly slower than PartPacker, deliver substantially better geometry and disentanglement, while remaining much faster than MIDI and PartCrafter.
method produces coherent object-level parts (e.g., complete houses, sofas, trees, lamps), whereas PartPacker shows semantic fragmentation (c.g., roofs tree canopies split across parts) and feature entanglement (e.g., ground features leaking into adja-
Figure 4 provides qualitative comparisons:

---

Published as a conference paper at ICLR 2026
Image
MIDI
PartCrafter
PartPacker
Figure 4: Qualitative Comparison on Structured 3D Scene Generation across Methods. Dif- ferent colors indicate different parts the generated 3D scene.
Method
Geometry Layout 1 Segmentation
MIDI (Huang al., 2025)
2.61
1.82
2.29
ParCrrftter al., 2025b)
2.17
PurtPacker (Tang ol., 2025)
2.81
295
1,97
Ours
3,09
3,3
3,22
Table 2: User Study. Human evaluation of dif- ferent structure 3D scene generation methods multiple aspects. Scores range from 1 to 4, with higher scores indicating better perfor- mance. Bold values represent the best perfor- mance within each metric.
Image
Camry Edgge
All Edge Regulatration
Edipe Regulaizztion
Figure 5: Qualitative Ablation Studies the Adge-Reeullarieed Assignment Cost.
cent buildings). Trained primarily on indoor assets, MIDI further requires additional instance masks inference; it performs reasonably on simple indoor scenes but degrades on outdoor cases, exhibit- ing spatial layout distortions and weaker instance separation. PartPacker does not use masks and can perform well on objects that are well separated from their surroundings, but performance deteriorates in complex spatial layouts.
We invite 30 participants to evaluate three baselines and our method, considering three criteria: geometry quality, layout coherence, and segmentation plausibility. We employ Forced Ranking Scale, where items are ranked from 1 to 4, with the highest rank receiving a score and the lowest rank receiving score of 1. As clearly indicated in Table 2, method receives the highest preference across all three criteria, indicating more coherent object-level parts, reduced feature leakage, and better wide layout consistency.
4.3 ABLATION STUDY
Effects of OT Plan-Gated Cross-Attention As shown in Figure 6 (a), our OT Plan-Gated Cross-Attentinn method produces highly structured and focused attention maps. Notice how A atten. map clearly isolates the ground. while the other (B atten. map) concentrates exclusively on the houses. This clean separation of duties, visualized the hard affinity map, results in distinet, non-overlapping regions. Consequently, each part is generated object (A-geo, for the ground, B B.geo. for the houses). When combined, they form a perfectly orga- nized scene Uni.geo. with sharp boundaries. In contrast, the standard cross-attention in (b) is noisy and chaotic. The attention maps are diffuse, sending mixed signals about which part is responsible for which region. This confusion leads corrupted geometry. This result validates the efficacy of our OT Plan-Gated Cross-Attentin module, proving that its enforcement of one-to-om constraints effectively prevents feature entanglement.
complete and clean geometric
8

---

Published as a conference paper at ICLR 2026
Image
B_atton.
Hard attinity
geo. B_ge0.
Uni_goo.
Figure 6: Qualitative Ablation Studies on the OT Plane-gated Cross Attention. Here, A attn. and attn, denote the dual-volume soft attention probability maps, reshaped to the image patch grid (brighter means higher affinity). Hard affinity visualizes the argmax(A,B) patch assignments overlaid on the input image (blue>A, red-+B). A.geo. and B_geo. are the geometries decoded from dual volumes, respectively, and Uni is their fused scene mesh. Row (a) shows our OT plan-gated cross attennn, row (b) shows the standard cross- cross-attention.
Image
increasing Denoising
Uni. geo.
Figure 7: Qualitative Ablation Studies on the OT Plan Progression over Denoising Steps. Each map visualizes the hard OT plan at a given denoising step: every cell an image patch assigned to one volume (dark blue A, light cyan = B). Left->right shows the OT plan's 's evolution; later steps mostly stabilize with only local refinements.
OT Plan Progression over Denoising Steps. As shown in Figure 7, increases, the OT plan quickly stabilizes: after roughly 5x540/600 the global partition changes little. In the late denoising stage, entire objects (c.g.. buildings, furniture, trees) are already routed into a single volume, with only fine adjustments thereafter. This temporal behavior explains the coherence of our object-level parts: coarse, semantic routing decided early and preserved, while later steps polish details without flipping the global assignment.
Effects of Edge-Regularized Assignment Cost. Our correlation assignment operates at the patch level. Because image features are locally smooth, neighboring patches usually prefer the same part, which tends to pull an entire connected region into single part. While desirable within an object, this behavior can mistakenly merge spatially adjacent bui semantically distinct objects at contact zones (e.g., furniture touching walls or fences touching posts). As shown in Figure 5, adding the edge regularizer cleanly separates objects that are contiguous in the image- the sofa from the corner side table in the top row, and the wooden posts from the surrounding fence in the bottom row. Compared to the version without edge regularization, the edge-aware plan yields crisper inter- object boundaries, fewer mixed parts, and improved structural fidelity, while requiring additional instance mask supervision.
5 CONCLUSION
In this paper, we introduced Scene'Transporte a novel framework for structured 3D scene generation from a single image. By reframing the task as a global correlation assignment problem and solving with Optimal Transport layer, method imposes powerful structural constraints directly on the generative process, effectively resolving the critical issues of structural mispartition and geometric redundancy found in existing models. Experimental results demonstrate that method achieves state-of-thh--an performance, generating complex open-world scenes with significantly improved geometric fidelity and instance-level coherence.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is supported by the National Science and Technology Major Project of the Ministry of Science and Technology of China (No. 20252D12063011 the National Natural Science Foundation of China (No. 62576043), and the Natural Science Foundation of Beijing, China (No. L233006).
REFERENCES
Andreea Ardelean, Mert Özer, and Bernhard Egger. Gen3dsr: Generalizable 3d scene reconstruction via divide and conquer from single view. arXiv preprint arXiv:2404.0344 03421, 2024.
Minghong Cai, Xiaodong Cun, Xiaoyu Li, Wenze Liu, Zhaoyang Zhang, Yong Zhang, Ying Shan, and Xiangyu Yue. Ditctrl: Exploring attention control in multi-modal diffusion transformer for tuning-fiee free multi-prompl longer video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp.
2025.
Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-exciiee Attention-base semantic guidance for text-to-image diffusion models. ACM transactions on Graphics (TOG), 42(4):1-10, 2023.
Minghao Chen, Iro Laina, and Andrea Vedaldi. Training-free layout control with cross-attention guidance. In Proceedings of the IEEE/CVF winter conference applications of computer vision, pp. 5343-5353, 2024a.
Minghao Chen, Jianyuan Wang, Roman Shapovalov, Tom Monnier, Hyunyoung Jung, Dilin Wang, Rakesh Ranjan, Iro Laina, and Andrea Vedaldi. Autopartgen: Autogressive 3d part generation and discovery. arXiv preprint vrXiv:2507.13340 2025a.
Yongwei Chen, Tengfei Wang, Tong Wu, Xingang Pan, Kui Jia, and Ziwei Liu. Comboverse: Com- positional 3d assets creation using spatially-awar diffusion guidance. In European Conference Computer Vision, pp. 128-146. Springer, 2024b.
Zheng Chen, Chenming Wu, Zhelun Shen, Chen Zhao, Weicai Ye, Haocheng Feng, Errui Ding, and Song-Hai Zhang. Splatter-360: Generalizable 360 gaussian splatting for wide-baseline panoramic images. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 21590- 21599, 2025b.
Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang- Layoutgpt: Compositional visual planning and gen- eration with large language models. Advances in Neural Information Processing Systems, 36: 18225-18250, 2023.
Daoyi Gao, Dávid Rozenberszki, Stefan Leutenegger, and Angela Dai. Diffcad: Weakly-supervised probabilistic cad model retrieval and alignment from rgh image. ACM Transactions Graph- ics (TOG), 43(4):1-15, 2024.
Haonan Han, Rui Yang. Huan Liao, Jiankai Xing, Zunnan Xu, Xiaoming Yu, Junwei Zha, Xiu Li, and Wanhua Li. Reparo: Compositional 3d assets generation with differentiable 3d layout alignment. arXiv preprint N.X.:22005.18555 2024.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yacl Pritch, and Daniel Cohen-Or. Prompt-to-promp image editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022.
Amir Hertz, Andrey Voynov, Shlomi Fruchter, and Daniel Cohen-Or. Style aligned image generation via shared attention. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4785- 2024.
Zehuan Huang, Yuan-Chen Guo, Xingqiao An, Yunhan Yang, Yangguang Li, Zi-Xin Zou, Ding Liang, Xihui Liu, Yan-Pei Cao, and Lu Sheng. Midi: 3d scene generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 23646-23657, 2025.
diffusion for single image to
10

---

Published as a conference paper at ICLR 2026
Florian Langer, Jihong Ju, Georgi Dikov, Gerhard Reitmayr, and Mohsen Ghafoorian. Fastcad: Real-time cad retrieval and alignment from scans and videos. In European Conference on Com- puter Vision, pp. 60-77. Springer, 2024.
Han-Hung Lee, Qinghong Han, and Angel X Chang. Nuiscene: Exploring efficient generation of unbounded outdoor scenes. arXiv preprint 113755 2025.
Jumin Lee, Sebin Lee, Changho Jo, Woobin Im, Juhyeong Seon, and Sung-Fui Yoon. Semcity: Semantic scene generation with triplane diffusion. In Proceedings of the IEEF/CVE conference computer vision and pattern recognition, pp. 28337-28347, 2024.
Haoran Li, Haolin Shi, Wenli Zhang, Wenjun Wu, Yong Liao, Lin Wang. Lik-hang Lee, and Peng Yuan Zhou. Dreamscene: 3d gaussian-based text-to-3d scene generation via formation pattern sampling. In European Conference on Computer Vision, pp. 214-230. Springer, 2024.
Yangguang Li, Zi-Xin Zou, Zexiang Liu, Dehu Wang, Yuan Liang, Zhipeng Yu, Xingchao Liu, Yuan-Chen Guo, Ding Liang, Wanli Ouyang, et al. Triposg: High-fidelity 3d shape synthesis using large-scale rectified flow models. arXiv preprint arXiv:2502. 066088 2025.
Hanwen Liang, Junli Cao, Vidit Goel, Guocheng Qian, Sergei Korolev, Demetri Terzopoulos, Kon- stantinos N Plataniotis, Sergey Tulyakov, and Jian Ren. Wonderland: Navigating 3d scenes from single image. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp- 798-810, 2025.
Chendi Lin, Heshan Liu. Qunshu Lin, Zachary Bright, Shitao Tang. Yihui He, Minghao Liu, Ling Zhu, and Cindy Lc. Objaverse++:: Curated 3d object dataset with quality annotations. arXiv preprint ar04v:2504. 20v::2504407334, 2025a.
Yuchen Lin, Chenguo Lin, Panwang Pan, Honglei Yan, Yiqiang Feng, Yadong Mu, and Katerina Fragkiadaki. Partcrafter: Structured 3d mesh generation via compositional latent diffusion trans- formers. arXiv preprint arXiv:2506.05555 2025b.
Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, Xuanmao Li, Xingpeng Sun, Rohan Ashok, Aniruddha Mukherjee, Hao Kang, Xiangrui Kong, Gang Hua, Tianyi Zhang, Bedrich Benes, and Aniket Bera. D13dv-10k: A large-scale scene dataset for deep learning-batt based 3d vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 22160-22169, June 2024.
Shaoteng Liu, Yuechen Zhang. Wenbo Li, Zhe Lin, and Jiaya Jia. Video-p2p: Video editing with cross-attention control. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8599-8608 2024a.
Yuheng Liu, Xinke Li, Xueting Li, Lu Qi, Chongshou Li, and Ming-Hsuan Yang. Pyramid diffusion for fine 3d large scene generation. In European Conference on Computer Vision, pp. 71-87. Springer, 2024b.
Quan Meng, Lei Li, Matthias Nießner, and Angela Dai. Lt3sd: Latent trees for 3d scene diffusion. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 650-660, 2025.
Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6038-6047, 2023.
Dong Huk Park, Grace Luo, Clayton Toste, Samaneh Azadi, Xihui Liu, Maka Karalashvili, Anna Rohrbach, and Trevor Darrell. Shape-guiee guided diffusion with outside attention. Proceed- ings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 4198-4207, 2024.
Gaurav Parmar, Krishna Kumar Singh, Richard Zhang. Yijun Li, Jingwan Lu, and Jun- Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 conference proceedings, pp- 1-11, 2023.
11

---

Published as a conference paper at ICLR 2026
Or Patashnik, Daniel Garibi, Idan Azuri, Hadar Averbuch- and Daniel Cohen-Or. Localiz- ing ubject-level shape variations with text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference computer vision, pp.
2023.
Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fusing attentions for zero-shot text-based video editing. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 15932-15942, 2023.
Xuanchi Ren, Jiahui Huang, Xiaohui Zeng, Ken Museth, Sanja Fidler, and Francis Williams. Xcube: Large-scale 3d generative modeling using sparse voxel hicrarchies. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 4209-4219, 2024.
Wenqiang Sun, Shuo Chen, Fangfu Liu, Zilong Chen, Yueqi Duan, Jun Zhang, and Yikai Wang- Dimensionx: Create any 3d and 4d scenes from single image with controllable video diffusion. arXiv preprint arXiv:2411.04922 2024.
Jiaxiang Tang, Ruijie Lu, Zhaoshuo Li, Zekun Hao, Xuan Li, Fangyin Wei, Shuran Song, Gang Zeng, Ming-Yu Liu, and Tsung- Yi Lin. Efficient part-level 3d object generation via dual volume packing. arXiv preprint arXiv:2506. 099800 2025.
Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to- amage-to-image to-image translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1921-1930, 2023.
Kiran Vodrahalli, Santiago Ontanon, Nilesh Tripuraneni, Kelvin Xu, Sanil Jain, Rakesh Shivanna, Jeffrey Hui, Nishanth Dikkala, Mehran Kazemi, Bahare Fatemi, al. Michelangelo: Long con- text evaluations beyond haystacks via latent structure queries. arXiv preprint arXiv:2409.1200 2024.
Hanyang Wang. Fangfu Liu, Jiawei Chi, and Yueqi Duan. Videoscene: Distilling video diffusion model to generate 3d scenes in one step. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16475-16485. IEEE, 2025.
Zhennan Wu, Yang Li, Han Yan, Taizhang Shang, Weixuan Sun, Senbo Wang, Ruikai Cui, Weizhe Liu, Hiroyuki Sato, Hongdong Li, et al. Blockfusion: Expandable 3d scene generation using latent tri- tri-plane extrapolation. ACM Transactions on Graphics (ToG), 43(4):1-17, 2024.
Janfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 21469-21480, 2025.
Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martin, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning unified representation of language, images, and point clouds for 3d understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1179-1189, 2023.
Le Xue, Ning Yu, Shu Zhang, Artemis Panagopoulou, Junnan Li, Roberto Martin-Martin, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, et al. Ulip-2: Towards scalable multimodal pre-training for 3d understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 27091-27101, 2024.
Fei Yang, Shiqi Yang, Muhammad Atif Butt, Joost van de Weijer, et al. Dynamic prompt learning: Addressing cross-attention leakage for text-based image editing. Advances in Neural Information Processing Systems, 36:26291-26303, 2023.
Shuai Yang, Jing Tan, Mengchen Zhang, Tong Wu, Gordon Wetzstein, Ziwei Liu, and Dahua Lin. Layerpano3d: Layered 3d panorama for hyper-immersive scene generation. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers, pp. 1-10, 2025a.
Yunhan Yang. Yuan-Cher Guo, Yukun Huang, Zi-Xin Zou, Zhipeng Yu. Yangguang Li, Yan- Pei Cao, and Xihui Liu. Holopart: Generative 3d part amodal segmentation. arXiv preprint MKKv:25044009443 2025b.
12

---

Published as a conference paper at ICLR 2026
Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neural fields and generative diffusion models. ACM Transactions On Graphics (TOG), 42(4):1-16, 2023.
Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang, Sheng Zhang, Xianghui Yang, et al. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation. arXiv preprint arXiv:2501.12002 2025.
Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang. and Xinlong Wang. Uni3d: Exploring unified 3d representation at scale. arXiv preprint arXiv:2310.06773, 2023.
Shijie Zhou, Zhiwen Fan, Dejia Xu, Haoran Chang, Pradyumna Chari, Tejas Bharadwaj, Suya You, Zhangyang Wang, and Achuta Kadambi. Dreamscene360: Unconstrained text-to-3d scene gen- eration with panoramic gaussian splatting. In European Conference Computer Vision, pp- 324-342. Springer, 2024.
13

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
The writing of this paper was assisted by Large Language Models (LL.M). Specifically, the L.M was utilized for the following tasks:
Improving grammar, clarity, and academic tone throughout the manuscript.
Rephrasing and restructuring sentences and paragraphs to enhance the logical flow of our arguments.
In accordance with ICLR policy, the human authors directed all content generation, critically re- viewed and edited all model outputs, and take full and final responsibility for the claims, accuracy, and integrity of this work.
B METHOD DETAILS
Debiased Clustering Probe To quantitatively investigate the flawed latent organization of the baseline, we designed a diagnostic probe based on unsupervised clustering. The key idea is that the constituent information for complete objects indeed present within the generated latents, but is merely disorganized and entangled by the model's component-based prior. Specifically, let ZA Zg E R xxx be the doll-volume latents generated by Tang et al. (2025) for a given scene. We stack them and apply mild whitening:
A ZB
2K
2K
Z
ž 2=
1
Z,
E=
2K > ZZ
(Z, 14) (13)
2K
where E-1/2 is computed from eigendecomposition with small ridge for stability. Denote the whitened halves by A. nn-
Directly clustering the raw token latents often fails because the process is dominated by strong, shared nuisance such as the ground plane global style-that non-diagnostic for ob- ject identity. To suppress these pervasive cross-volume trends, we first estimate a shared subspace between the two latent volumes AŽ 2R) using canonical correlation analysis (CCA). CCA iden- tifies paired directions (u,, V5) that maximize the correlation between the volumes" projections. We retain all directions whose canonical correlation Pj exceeds a threshold r to define this shared subspace
(es, uy Vol CCA(ZA: ZB),
[ >T>}} Ugtaard span (u, v, :J:
(14)
where {-anf-} denotes the set of all linear combinations of the listed vectors. Let U be a column- orthonormal basis of Uitturd and P UUT the orthogonal projector. We obtain debiased tokens by removing their shared component:
2, ZA P,P.
2B = 2B ŽgP.
(15)
Intuitively, (15) down-weights the high-variance global modes while preserving object-specific vari- ation. We then cluster the debiased tokens Ż = A Zn) with flexible Gaussian mixture
Th NeN Ed).
C=
(16)
and denote by Ka the posterior responsibility of component e for token To improve robustness, to- kens with low maximum confidence (max, THic 5) are reassigned to the nearest centroid computed from high-confidence members (max, The 2 8). Finally, grouped tokens are decoded independently with the frozen VAE decoder to visualize the resulting object-level organization.
C EXPERIMENT SETTINGS
C.I HYPERPARAMETERS
Table 3 provides the hyperparameters needed replicate experiments.
14

---

Published as a conference paper at ICLR 2026
Table Hyperparameters of OT plan-gated cross-attention used in all experiments.
Symbol
Description
Value
Entropic regularization weight
0.10
Aedge
Edge regularization strength
0.8
Vedge
Edge sensitivity
8.0
A
Guidance strength
2.5
Eg
Floor term
0.02
Kot
Number of Sinkhorn iterations
40
C.2 HUMAN EVALUATION.
In our user study, we compare our method with three baselines (MIDI (Huang et al., 2025), PartCrafter (Lin al., 2025b), and PartPacker (Tang et al., 2025)) on perceptual quality. For each reference image, all four methods generate a structured 3D scene, which we export as gllb meshes. These meshes are loaded into web-based 3D viewer with identical lighting and shading settings. The four methods are assigned to labels A-D in single random permutation, and the correspond- ing meshes are displayed side-by-side under these labels. Participants can freely rotate, zoom, and inspect each mesh interactively.
We recruit a total 30 participants (graduate students and researchers in computer vision/graphics not involved in this project). For each reference image they are shown, participants evaluate the four methods along three dimensions. Specifically,, they are asked to rank the four scenes (A-D) from (lowest) to 4 (highest) for each of the following questions:
Geometry Quality: "Please rank the overall geometry quality of each scene." " This metric evaluates how detailed, precise, and faithful to the reference image the geometry of each scene is, including fine- finegrained structures and overall shape fidelity.
Layout Coherence: "Please rank the overall spatial layout and arrangement of objects in each scene relative to the reference image." This measures how coherent the scene layout is, and how well object positions, scales, and composition align with the input image.
Segmentation Plausibility: "Please rank the overall plausibility of the object-level part decomposition in each scene." This assesses what extent each scene exhibits clear, rea- sonable instances with minimal overlaps, missing regions, or mixed parts.
For each method and each criterion, we compute the average rank over all evaluated images and participants. These averaged scores reported in Table 2 of the main paper, where higher values indicate stronger human preference. Our method achieves the highest average rank across all three dimensions, suggesting more coherent object-level parts, reduced feature leakage, and better scene- wide layout consistency.
D ADDITIONAL EXPERIMENTS
D.1 REAL-WORLD RESULTS
In our current setting, all methods (including the baselines) are trained on synthetically rendered scenes, which allows us to construct large and diverse training set with consistent part-level anno- tations. To test the generalization ability of our model natural photographs, we perform zero-shot evaluation on real-world images from the DL3DV-10K dataset (Ling et al., 2024).
As expected, directly applying the synthetic-trained model to raw photographs leads notice- able performance drop due to the appearance domain gap. Following the strategy proposed in PartCrafter (Lin et al., 2025b), we therefore explore transferring the style of real-world images to make them look more like images rendered from graphics engine using recent image editing models, such as GPT-5. Specifically, we use prompts of the form: "Preserve all details and perform image-to -ii-age style transfer convert the image into the style of 3D rendering (Objaverse rendering)." The style-transferree images preserve the original scene layout and object identities, while matching the rendering statistics of synthetic training data.
15

---

Published as a conference paper at ICLR 2026
Image
Result
Stylized
Stylized-Result
Figure 8: Qualitative Results on Structured 3D Scene Generation from Real- World Images. We GPT-5-based image editing model to transfer the style of real-world images, making them look like images rendered from a graphics engine.
Under this simple, purely test-time preprocessing, our method produces significantly more faithful and coherent 3D scene reconstructions: objects are better separated, part boundaries align more closely with image evidence, and the overall layout matches the input photograph more accurately. As shown in Figure 8, this simple strategy works surprisingly well on a variety of indoor and outdoor real-world scenes.
D.2 CONVERGENCE ANALYSIS OF THE ENTROPIC OT SOLVER
In each OT-gated cross-attention layer, we solve small entropic optimal transport (OT) problem between two "rolls" of part queries and M image tokens. Given a cost matrix C' 222M row marginals y E IR2 and column marginals
RM we minimize
min (C, A) + =H(( S.L. Al A1 A1= A20
(17)
where A E R2xM is the transport plan, 0 is the entropic regularization strength, and H En Aiy log Ai is the negative-entropy regularizer.
Following the standard dual formulation of entropic OT, we introduce dual potentials f f RR (for the row constraints) and R.M (for the column constraints). At Sinkhorn iteration k the corre-
16

---

Published as a conference paper at ICLR 2026
sponding transport plan has the form
A exp
+g.
-Gs
(18)
followed by normalization step enforce the marginal constraints A μ and (A()) T s v. A fixed point of the Sinkhorn updates corresponds to a stationary solution of the entropic OT problem equation 17.
Monitored residuals. To assess optimization stability and convergence in practice, we instrument our entropic solver and log convergence statistics for all OT-gated cross-attention layers. For each Sinkhorn solve, at each iteration k we record the following residuals:
Dual updates: the & norms of the changes in the dual variables
[[ (())
((k)
(kg 1
(19)
g
These measure how much the dual potentials still move between two successive iterations. Transport-plan update: the Frobenius norm of the change the transport plan
(e) = | - A
(20)
which quantifies how much the routing plan still being updated.
Marginal-constran violation: the deviation the current plan from the prescribed marginals,
2 EI(44)) mal, t=1
(k) col
1 M j=1
Σ
((A(4))T1),
1/y.
(21)
and we report their sum
marg
(R)
= -
row
(k)
+
Cool
(22)
(k) Intuitively, Tmarg measures how well the current (k) satisfies the OT marginal constraints.
In words, r' ..' (k) and r tell us whether the dual variables and the transport plan have stabilized,
while rhnny indicates how strictly the mass conservation constraints are enforced. Figure. 9 visualizes these quantities for three OT-gated cross-attention layers. For each layer, we run the Sinkhorn solver with fixed number of iterations (40 in all our experiments) and plot the residuals rf T
(k) (k) (k)
(k) and m'arr as function of the iteration index k.
Across all OT-gated layers and across different denoising steps, we observe a consistent pattern:
the dual and plan residuals v .rg and (k) drop by 3-5 orders of magnitude within only 3-5
(k)
Sinkhorn iterations and then remain numerically flat, while the marginal-constraint violation rmarg quickly converges
very small value and stays stable without oscillation or divergence. This indicates that the entropic OT subproblem in setting is well-conditioned and that solver converges rapidly and stably under the default choice of 40 iterations used in all experiments.
D.3 ADDITIONAL ABLATION STUDIES
Table presents quantitative ablations regarding the core components and hyperparameters of our OT-guided routing mechanism. The key observations are summarized below.
Impact OT Plan-Gated Attention and Gddge-Regularized Cost. Rows (a. (a.3) isolate the contributions of the proposed OT modules. The removal of the OT Plan-Gated Cross-Attention (a.1)
precipitates a discernible drop in Geometry Fidelity (e.g., ULIP decrease) and a marked deterioration in IOU metrics. This empirical evidence substantiates capacity-constraie patch-to-pan routing is critical for mitigating feature leakage and suppressing redundant geometry. Furthermore, while retaining OT gating but omitting the Regulaeegllarized
hypothesis that enforcing one-to-one,
17

---

Published as a conference paper at ICLR 2026
Convergence of the entropic OT solver layer
Deciverlehe
Dusf vatti
Hangrnll oor incint stten
54300
5400
137
13
177
23
Convergence of the entropic OT solver layor
1371
137
Convergance of the entropic OT solver of lnyer
Dusl v
Coal cerible
Tessprtt Sank
Pari controke alte
Figure 9: Convergence of the entropic OT solver across OT-gated cross-attention layers. We plot the residuals of the dual variables and transport plan, as well as the marginal-constrain viola- tion, for three representative OT-gated cross-attention layers.
Assignment Cost (a.2) improves upon the baseline, the IoU metrics remain suboptimal compared to the full configuration. The full model (a.3) attains superior performance across all metries, confirm- ing that edge-aware smoothing effectively refines object separation without compromising global geometric fidelity. In terms of efficiency, the inclusion of OT modules introduces a manageable computational overhead (increasing inference time from 47.4, to 55.0,s) while memory consump- tion remains negligible.
Effect of OT Hyperparameters. We further analyze the sensitivity of our method to key hyper- parameters in Rows (d). First, varying the entropic regularization & (Rows b.1-b.4) reveals that = 0.10 yields the optimal trade-off. Deviating to lower or higher values disrupts the bal- ance between transport plan sparsity and smoothness, leading to a degradation in both geometric fidelity (ULIP/Uni3D) and part disentanglement metrics. Second, regarding the edge sensitivity Yodge (Rows c.1-c.4), a lower value (6.0) relaxes boundary constraints, which marginally benefits global geometry (highest ULIP) but causes leakage across object boundaries (increased IoUmax)) Conversely, excessive sensitivity (2 10.0) over-constrains the routing, harming all metrics. Our default "edge 8.0 strikes the best balance. Finally, for the ddge-smoothing weight Aadge (Rows d.1-d.3), we observe that sufficient smoothing is required to enforce intra-part coherence. Setting Asdge too low (0.6) or too high (1.0) results in suboptimal segmentation, evident from the sharp rise in loUmax. The default Agdge = 0.8 consistently achieves superior performance, demonstrating that the method is robust within a reasonable range around the optimal settings.
Number of OT-Gated DiT Blocks. Finally, Rows (e..3) investigate the optimal density of OT Gate- Cross-Attention within the DiT architecture. Restricting OT integration to only one- third of the blocks (c.1) yields some geometric improvement over the bascline but proves insufficient for effective part separation, as evidenced by suboptimal IoU metrics. Increasing the coverage to half of the blocks (e.2, our default) precipitates a substantial gain in both Geometry Fidelity and Part Disentanglement. Crucially, this setting maintains a moderate runtime, incurring only rea- sonable overhead compared to the baseline. Applying OT to all blocks (e.3) provides marginal gains in specific geometry metrics (e.g.. ULIP-2) but leads to diminishing returns- or even slight degradation in part metrics, while imposing significant latency penalty. These results indicate that integrating OT into approximately half of the layers offers the most favorable trade-off between structural disentanglement and computational efficiency.
18

---

Published as a conference paper at ICLR 2026
Setting
Goometry Fidelity
Part Disentangloment
Inference Time (5)
ULIPf
ULIP-27
Uni3D"
loU
loUnen
Memory (M)
(a. () OT Plan-Gared Cross-AAtention
0.1417
0.3083
0.0319
47.41
(a.2) wlo Edge-Regularie Regularized Assignment Cost
0.1452
0.3164
0.2916
0.0241
0.1136
54.61
9068
(a.3) Full model
0.1466
0.3220
0.3021
0.0101
0.0926
54.99
9070
(b.1) 0.08
0.1460
0.3184
0.3003
0.0117
0.0914
55.30
9070
(h.2) 0.10"
0,1466
0.3220
0.3021
0.0101
0.0926
54.99
9070
(b.3) 0.12
0,1437
0.3185
0.2970
0.1239
0.1119
55.12
9070
(b.4) 0.14
0.1424
0.3178
0.2937
0.0159
0.0936
55.02
9070
(c.1) Tedgs 6.0
0,1488
0.3239
0.0243
56.17
(c:2) Todas
0.1466
0.3220
0.3021
(c.3) Tedas
0,1457
0.3182
0.3029
0.0570
58.17
(c:4) Tway 12.0
0.1426
0.3156
0.3003
0.1149
0.0994
53.89
9070
(d.1) Avess 0.6
0.1456
0.3236
0.3003
0.0609
0.1128
53.59
9070
(d.2) Ave 0.8*
0.1466
0.3220
0.3021
0.0101
0.0926
53.97
9070
(d.3) Avas 1.0
0,14.99
0.3204
0.3017
0.1482
0.0952
55.17
9070
(e.1) 1/3 DiT blocks
0.1465
0.3170
0.3004
0.0992
0.1061
51.89
9070
(c:2) DiT blocks'
0.1466
0.3220
0.0101
54.99
(o.3) blocks
0.1426
0.3262
0.0207
Table 4: Comparison of metrics for ablation. Bold values indicate the best scores, while under- lined values indicate the second-best scores among the fair comparison. Asterisk (+) indicates the default settings in method.
Image
PartPacker
Ours
Ours Ours_residual
Figure 10: Effect of adding residual vanilla cross-attention branch. Left right: input image, PartPacker baseline, our OT-guided routing, and our OT-guided routing with an additional residual cross-attention branch (Ours_resia) residuus which recovers the small crowded instances (e.g.. boats) while preserving the improved global layout and part separation.
D.4 FAILURE CASES AND RESIDUAL DIAGNOSTIC
Crowded tiny instances. In scenes with very dense clusters of small, similar objects that share only a few image patches (e.g., tightly packed boats or trees), the OT-guided routing can allocate most capacity to the strongest responses and effectively merge a few weak instances into their neigh- bours. This may slightly under-count very small repeated objects, while the global layout (buildings, roads, docks, etc.) remains correct. We find that adding small residual branch of vanilla cross- attention on top of the OT-gated attention largely mitigates this issuc, keeping the OT plan as a low-frequency structural prior and using the residual attention to recover high-frreuency local de- tails. Figure 10 visualizes representative example, comparing (i) the bascline, (ii) SceneTransporte with pure OT-guided routing, and (iii) SceneTransponne Transporter with the residual pross-attention branch: the residual variant recovers the missing tiny instances while preserving the cleaner global structure of OT.
Geometry artifacts. We occasionally see nonsmooth surfaces or floating parts when the denoising schedule is too short the number of tokens is too small. Increasing the number of diffusion steps or the token budget alleviates these cases, and the resulting artifacts are typically local and do not affect the overall scene layout.
19

---

Published as a conference paper at ICLR 2026
Out-of-distribution appearance. Since the model is trained on synthetic rendered images, strongly -ustr--distributiin real images (e.g., unusual lighting or textures) can lead to degraded geometry and part grouping. As discussed in Appendix D.I, applying automated style transfer to convert real images into a 3D-rendering style significantly improves the results, and Scene'Trans porter still produces plausible structured scenes many challenging real-world inputs.
20


Reviews:
[{"review_id": "mlBLHaTv9a", "reviewer": "Reviewer_13P1", "summary": "This paper proposes SceneTransporter, an end-to-end framework for structured 3D scene generation from a single image. By introducing a debiased clustering probe, the authors identify a key limitation of existing methods—the lack of structural constraints in internal feature assignment. They reframe the structured scene generation task as an Optimal Transport (OT)–guided global correlation assignment problem, solving the OT objective at each denoising step to enforce one-to-one routing between image patches and part-level 3D latents. The approach further introduces an OT Plan–Gated Cross-Attention mechanism and an Edge-Regularized Assignment Cost to prevent semantic entanglement and object fragmentation.", "questions": "See the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper presents a promising analysis of current 3D scene generation methods and proposes a simple yet effective solution to the structural inconsistency problem. The overall methodology is well-motivated, technically sound, and the experimental results demonstrate clear improvements over existing baselines.", "weaknesses": "The work lacks a detailed discussion on the optimization stability and convergence of the OT formulation, as well as the influence of hyperparameters (λ_edge, ε_t, γ_edge) on the generation quality and computational efficiency. The computational cost of solving OT during inference should be analyzed more thoroughly. Compared to other compositional latent diffusion models such as PartPacker, the innovation mainly lies in adding an OT-based constraint rather than proposing a fundamentally new architecture. It is also recommended to clarify the setting and methodology of the user study.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WAtBqoRcNg", "reviewer": "Reviewer_VxXm", "summary": "This paper introduces SceneTransporter by reformulating the task of structured 3D scene generation as a global correlation assignment problem.", "questions": "1. How many GPU hours does the training take?\n\n2. What is the GPU memory usage for generation?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Strong performance: better experimental results have been observed against baselines like PartPacker, PartCrafter, MIDI. The generation speed is also comparable. \n\n2. The proposed Debiased Clustering probe can produce stable instance groupings, leading to better generation in the following stages.", "weaknesses": "1. No apperance: It seems that all methods, including baselines, only generate meshes without textures, which might limit the real-world applications. Can the authors provide more details about this?\n\n2. Lack of quantitative ablations: all ablations are qualitative, can the authors provide numeric results? If not, please explain it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "b64x7FqZtb", "reviewer": "Reviewer_vKGo", "summary": "This paper introduces SceneTransporter, an end-to-end method for generating 3D scenes at the part level from a single image. This work builds upon PartPacker [Tang et al. 25]. By analyzing the latent structure of part-level generators, the authors observe that part tokens lack structure constraints, leading to mixed patch-to-part assignments in the DiT architecture. The authors thus propose to solve optimal transport (OT) problems to guide the assignment of image patches to part tokens in attention maps. In the OT formulation, the authors also consider the image edge map in the assignment cost to promote coherent structures grouping. Experimental results show that SceneTransporter outperforms prior methods on structured 3D scene generation from a single image.", "questions": "- Have the authors trained and tested SceneTransporter, for example, on real-world indoor scene images? If so, how well does the method work on such images?\n- In the generated results, the proposed method struggles with fine detail generation, however, the evaluation metrics (Geometry Fidelity) do not seem to reflect this, which is concerning. \n- In the ablation study, only qualitative results are provided. It is unclear, for example, how much improvement the edge-regularized Assignment cost brings. Other ablations like the authors enable OT plan-gated attention in the first half of the DiT blocks, but it is unclear whether this is the optimal choice and how it affects the geometry fidelity to input images.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- I like that the authors first present an interesting motivation for their work by analyzing the latent sets of part-level generators through canonical correlation analysis (CCA). They find that the learned features implicitly encode part structures, but the networks do not establish part associations explicitly. This analysis motivates their method of enforcing explicit patch-to-part constraints in attention maps.\n- The authors propose to use entropy-regularized optimal transport to guide the assignment of image patches to part tokens in attention maps. To promote region-wise consistency and reduce information exchange across image regions, the authors incorporate image edge maps into the assignment cost of the OT problem, which I find interesting.\n- The authors compare their method with baselines including MIDI, PartCrafter, and PartPacker on structured 3D scene generation from a single image. Results show that the proposed method has better geometry fidelity and improved part disentanglement.", "weaknesses": "- The results shown in the paper does not include any real-world image inputs. While these cartoon-style scene images show the effectiveness of the proposed method, it is unclear how well the method can generalize to real-world images with more complex structures.\n- While this explicit patch-to-part assignment approach improves the generation of coherent object-level parts, it seems that the method struggles to generate fine-grained details. For example, in Fig. 6 (Uni geo), the boats in the bottom part of the image are not well generated, whereas the standard cross-attention method correctly generates the five boat shapes. This makes me wonder if the proposed method may overconstrain the information exchange and thus reduce the context information from other image patches needed for generating fine details.\n- Several implementation details (like hyperparameters for the OT solver, training details) are missing. This makes it hard to reproduce the results. Quantitative ablation results are also not included.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fL8wDNTUVW", "reviewer": "Reviewer_1EWa", "summary": "This paper proposes SceneTransporter, a novel framework for generating structured 3D scenes from a single image using compositional latent diffusion guided by Optimal Transport (OT). The authors first diagnose a key failure in prior part-level 3D generators—namely, the lack of explicit structural constraints leading to structural mispartition and geometric redundancy. They introduce a Debiased Clustering Probe to reveal this issue and then reframe scene generation as a global correlation assignment problem. The proposed OT-guided module imposes two constraints within the diffusion model’s cross-attention mechanism: (1) an OT Plan–Gated Cross-Attention enforcing exclusive patch-to-part routing, and (2) an Edge-Regularized Assignment Cost ensuring spatial coherence across image regions.\nEmpirical results show substantial improvements in instance-level coherence and geometric fidelity over state-of-the-art methods such as PartPacker, PartCrafter, and MIDI.", "questions": "1. How sensitive is performance to the hyperparameters of the OT solver (e.g., entropy regularization, number of Sinkhorn iterations)?\n2. Can SceneTransporter be adapted for text-conditioned or multi-view inputs?\n3. Does the OT guidance improve consistency when compositing previously unseen object categories?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The motivation of this paper is clear.  Extending the end-to-end structured generation pipeline from the object level to the scene level is an interesting endeavor, which simplifies the common \"divide and conquer\" solution and provides bigger potential for scaling up. \n\n2. Recasting feature-to-part assignment as an entropic OT problem is conceptually elegant and mathematically well-grounded. The OT-guided attention gating is seamlessly embedded in the denoising process, preserving end-to-end differentiability.\n\n3. Quantitative benchmarks (ULIP, Uni3D, IoU metrics) and qualitative visualizations convincingly show better object separation and geometric consistency.", "weaknesses": "1. The computational cost of solving the OT problem (e.g., Sinkhorn iterations) is briefly mentioned but not thoroughly analyzed in terms of training or inference scalability for large scenes.\n2. The experimental dataset (74 web images) seems small and lacks diversity benchmarks such as Objaverse or large-scale indoor/outdoor test sets. Generalization beyond curated examples remains unclear.\n\n3. The qualitative ablation figures are insightful, but quantitative ablation should also be conducted (e.g., removing OT gating or edge regularization) to make the contribution breakdown stronger.\n\n4. Although the OT can improve the global consistency in allocating image patches to 3D parts, it is practically hard to guarantee due to the occlusion, semantic ambiguities, etc. Did you observe failure cases?\n\n5. All the experiments are conducted on synthetic data. Have you ever tried to deploy the proposed method to real-world imaegs to check the generalization ability?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)