AssetFormer: Modular 3D Assets Generation with Autoregressive Transformer
Authors: Lingting Zhu
Shengju Qian
Haidi Fan
Jiayu Dong
Zhenchao Jin
SiweiZhou
Dong Gen
Xin Wang
Lequan Yu
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Generation
Autoregressive Transformer
Modular 3D Assets
Primary area: generative models

Abstract: The digital industry demands high-quality, diverse modular 3D assets, especially for user-generated content (UGC). In this work, we introduce AssetFormer, an autoregressive Transformer-based model designed to generate modular 3D assets from textual descriptions. Our pilot study leverages real-world modular assets collected from online platforms.  AssetFormer tackles the challenge of creating assets composed of primitives that adhere to constrained design parameters for various applications. By innovatively adapting module sequencing and decoding techniques inspired by language models, our approach enhances asset generation quality through autoregressive modeling. Initial results indicate the effectiveness of AssetFormer in streamlining asset creation for professional development and UGC scenarios. This work presents a flexible framework extendable to various types of modular 3D assets, contributing to the broader field of 3D content generation. The code is available at https://github.com/Advocate99/AssetFormer.

Published as a conference paper at ICLR 2026
ASSETFORMER: MODULAR 3D ASSETS GENERATION WITH AUTOREGRESSIVE TRANSFORMER
Lingting Zhu'
Shengju Qian
Haidi Fan Jiayu Dong" Zhenchao Jin'
Siwei Zhou² Gen Dong² Xin Wang² Lequan Yu
'The University of Hong Kong LIGHTSPEED
1tzhu99econnect. hku.hk, 1qyu@hku.hX
ABSTRACT
The digital industry demands high-quality, diverse modular 3D assets, especially for user-generated content (UGC). In this work, we introduce AssetFormer, an autoregressive Transformer-based model designed to generate modular 3D assets from textual descriptions. Our pilot study leverages real-world modular assets collected from online platforms. AssetFormer tackles the challenge of creating assets composed of primitives that adhere to constrained design parameters for various applications. By innovatively adapting module sequencing and decoding techniques inspired by language models, our approach enhances asset generation quality through autoregressive modeling. Initial results indicate the effectiveness of AssetFormer in streamlining asset creation for professional development and UGC scenarios. This work presents a flexible framework extendable to various types modular 3D assets, contributing the broader field of 3D content generation. The code is available at (nttps::///// /tgtttu...uuggggg com/ http///vooccaaeeeoa/m wtw.eeee////seestsoeeeeeooo
INTRODUCTION
3D asset generation has garnered significant attention due to its potential impact on digi- tal creativity across various domains. Recent advancements have explored a variety of rep- resentations, including voxels (Brock al. 2016; Wu et al., 2016). point clouds (Luo and Hul 2021 Vahdat et al. 2022). neural fields (Gao et al., 2022 Chen and Zhang 2019), and meshes (iidiiqui et 2024 Nash al. 2020). However, despite progress in so- phistrcated geometry and texture, these tradi- tional representations face critical limitations in real-world upplications, particularly within
r xx)) 22
Compase
Deployment
Figure 1: Illustration of modular 3D assets. Modu- lar assets can be decomposed into primitives, each possessing its own attributes, e.g., the orientation r and the position The modular asset can be ren- dered with configurations to enable 3D deployment.
the game industry. In professional game development, existing methods often struggle to meet the high-quality standards demanded by modern games, resulting in a time-intensive workflow for artists who may spend hundreds of hours meticulously designing and refining each asset. Meanwhile, in user-generated content (UGC) scenarios (Epic) 2017 Duan et al. 2022) and online gaming, these representations frequently yield large file sizes, which present substantial challenges for storage and transmission in efficiencc-driveer environments. Such issues can strain server infrastructure and hinder seamless sharing and real-time interaction crucial elements in UGC platforms and multi- player online games. Furthermore, the inherent complexity of these representations often restricts non-professional users from easily creating, modifying, and sharing their content, thereby limiting the potential for diverse and engaging user-generated game assets.
In digital production, artists frequently employ modules and constrained design spaces as foundational elements for complex assets. This approach, drawing concepts from Constructive Solid Geome-
Project Lead.
Corresponding author.

---

Published as a conference paper at ICLR 2026
try (CSG) (Voelcker and Requicha 1977: Laidlaw et al., 1986) in Computer-Aides Design (CAD), offers several advantages. It facilitates rapid prototyping, ensures asset consistency, and enables seamless integration into game engines. Utilizing CSG principles, artists efficiently combine and manipulate basic shapes to create intricate forms with precision. This modular methodology not only streamlines the asset creation process but also lowers the barrier to entry for non-professional users, fostering broader participation (Krumm et al. 2008 Rymaszewskij (007)) and enhanced scalability in content creation. Moreover, this approach enables transmission efficiency in user-generated content (UGC) and online gaming environments.
While other 3D modalities benefit from the availability of growing public datasets (Deitke al. 2023; Wu et al., (223). modular 3D assets suffer from a significant scarcity of publicly available training data, leaving automatic modular asset generation an understudied field. This deficiency stems from the proprietary nature of most modular asset libraries, which are often closely guarded by game studios and content creators. To address this challenge, our research leverages modules and data collected from online user generated content (UGC) platform, where players create intricate 3D homestead assets by manually arranging pre-defined construction materials.
Illustrated in Fig. L modular representation of 3D assets exemplifies the potential for complex asset creation from basic components but also highlights the demand for tools that can automate and enhance creation. Building on these insights, our study aims to develop model capable of generating diverse modular 3D assets with customization on textual descriptions.
In this work, we propose novel framework that leverages autoregressive modeling with modular 3D assets. Composed of primitive elements, each asset can be viewed as a series of modules, as well as proper decisions about their placement and orientation. This sequential nature aligns perfectly with autoregressive models, which excel at capturing and generating ordered sequences. Meanwhile, mirrors the step-by-step process of human construction, leading to more intuitive and controllable asset generation. Unlike text or image generation, where the sequence order is often inherent (left-to- right for text, pixel-by-pixe for images). 3D assets pose a unique challenge in determining the optimal order of modular components. This ordering is crucial as it affects both the coherence of the generated structure and the model's 's ability to capture complex spatial relationships. By carefully analyzing the connectivity among primitives, we design improved tokenization algorithms and decoding strategies that capture the hierarchical and spatial relationships within assets. In summary, our contributions are as follows:
We propose an autoregressive generation framework for modular 3D asset generation, which shows promising results compared other 3D modalities.
We introduce a large-scale dataset of modular 3D assets, collected and cleaned from the UGC platform of an online game. To our knowledge, this the only real-world modular 3D dataset of high quality.
We analyze the impact of module tokenization order and decoding strategies on the quality and diversity of generated assets, offering insights that can be extended to other 3D sequential generation tasks.
Our model demonstrates the ability to generate high-quality, contextually appropriate 3D assets, providing a practical guide for the application of 3D generation.
2 RELATED WORK
Generative Visual Modeling, The recent years have seen a continuous pursuit of advanced generative models, including generative adversarial networks (GANs), autoregressive models (ARs). flows, and variational autoencoders (VAEs), and their crown battle for visual creation in image, video, and 3D
applications (Goodfellow et al. 2014 Ho et al. 2020 Van Den Oord et al. 2016 Vaswani 2017
Rombach et al. Chang et al.
Kingma and Dhariwal 2018 Kingma 2013] Singer
ct al.) 2022; Hong et al., 2022; Poole et al. 2022: Hong et al.) 2023 ). Inspired by the scalability demonstrated by autoregressive models in language modeling (Achiam et al. 2023 Brown 2020
Chowdhery et al. 2023). recent efforts have focused on extending the capabilities of AR models to
mixed-modal modeling
challenging the dominance of diffusion models in visual generation (Zhu
et al. 2023 Liu et al. 2024b) Team 2024; Driess et al. 2023 Liu et al. 2024a; Wang et al.) 2024a
Sun al. 2024). For instance, Emu3 (Wang et al. 2024a) posits that next-token prediction is all you
2

---

Published as a conference paper at ICLR 2026
need for achieving tfate-of-the-arr performances in multimodal tasks, demonstrating robust results in the understanding and generation of images, text, and videos. Built upon autoregressive transformers, our work delves deeper into design rationales tailored to downstream visual creation for 3D assets, e.g., modular 3D generation.
3D Generation. Recent advancements in 3D generation have demonstrated significant progress. creating complex 3D representations from textual descriptions or sparse images (Gao et al. 2022: Lin
et 2023 Poole al. 2022; Tang et al.] 2023; Zhang al. 2024 Liu et. 2023b) Long al. 2024).
These methods have explored various 3D representations, including voxels, point clouds, neural fields, and meshes. Notably, autoregressive Transformer-baas
models for mesh generation (Nash)
et al., 2020; Siddiqqu et al., [024; Chen et al. [024c|bb [Tng et al.] (2244) have garnered attention due their potential to synthesize detailed 3D structures. Despite these breakthroughs, existing methods face several challenges in real-world applications, including meeting high-quality standards, managing large file sizes, and providing accessibility for non-professional users. Some studies have adapted generative models for specific applications like CAD models (Wu et al. 202 Li et al. 2022
Xu et al., 2024b Ritchie et al., 2023) and human garments (Korosteleva and Lee| |022]] et al. 2023a He et al. 2024), aiming to address domain-specifie challenges. Our work builds upon these
advancements while specifically targeting the challenges modular 3D asset generation, aiming to address the limitations of existing methods in terms of quality, efficiency, and accessibility.
Autoregressive Modeling. Autoregressive transformers have demonstrated remarkable success in language modeling and visual generation (Achiam al. 2023 Liu et al. 2024b] Team 2024 Liu
et al. 2024a), benefiting from their scalability and ability to capture complex dependencies. However, adapting these models to visual and 3D domains presents unique challenges, particularly in data tokenization and sequence ordering. For instance, VQGAN (Esser el al., 202T) employs codebook for images, while MAR (Li et al. (0244) learns a mattiuous-vvllues space using diffusion based probability distribution modeling. In the 3D domain, methods for mesh generation (Tang et al. 2024
Chen et al. 2024d) have explored compact mesh tokenization to effectively represent complex 3D structures. -Mesh Wang et al. (2024b) and Mesh-I.LM Fang et al. (2025) integrate L.LM's strong prior knowledge and enable the generation of sext-serialized 3D meshes, but struggle producing very complex 3D meshes. A recent work YYe et al. 2025) uses AR model for decomposing complex shapes into 3D primitive, with part-level understanding (Mo et al. 2019 Gao et al. 2021 Hertz et al.
2022 Li et al., 2024a). Furthermore, decoding strategies (Holtzman al. [019 Leviathan et al.
2023 Chen et al. 2023a TTng et al. 2024) for improving generation quality and inference speed
remain active area of research in visual models.
3 METHOD
3.1 PROBLEM FORMULATION
We collect the intricate 3D assets from an online UGC platform, which the users manually create with the provided modular materials. As the data represents distinctive homesteads, each asset comprises a sequence of building primitives, e.g., roof and floor patches, where the building primitive has its attributes including class E C, rotation r E R (vertical axis), and position æ E X with their finite sets of discrete values. To be specific, i-th sample can be characterized as N, primitives { and P3 = (cj: r, ,. ). Our goal is to learn a generative model G capable of synthesizing samples from (P)
textual deseription t: G
The dataset source is obtained and cleaned from real user-created assets, which are of high complexity and variety. One advantage of the modular 3D representation its easy compatibility with traditional Procedural Content Generation (PCG) methods. To better study the influence of different data sources, we formed another data source with PCG in addition to the real data we collected. Building on procedural generation (Short and Adams 2017 Raistrick et al. (202),, we use random generators for attributes such as the number of storeys and the positions of key modules. The details of the Algorithm can be found in Appendix. To prepare the text prompt, use GPT-4o (OpenAl 2024) to produce phrase bundles such as (apartment, multi-story flat roof, few windows), characterizig the global features of the asset based on the rendered images.
3

---

Published as a conference paper at ICLR 2026
Stage 1: Data Curation
Sage 2: Autoragrass Modaling
Stago 3: Post- Enhancement Deployment
Textual Embedder
Diserete Tokenization & Re-Ordering
Model-Basee Enhancement
Auceleration
Validation Check
Application Deployment
AssetForment
Modular Assets (e.g.. Buildings)
Modular Texturing
4P Labeling
Decoding & Rendering
Modular Editing
Token Set Modeling
Figure 2: Overview of the AssetFormer Framework. Given the modular assets, e.g., the building, we first render the assets in digital engines and produce the images for querying GPT-4o. The cleaned captions, pre-filled with re-ordered token set, serve as input for the autoregressive modeling. After training, AssetFormer autoregressively produces modular assets that are ready be integrated into industrial environments, with model-based based enhancement and application-drive deployment.
3.2 AUTOREGRESSIVE TRANSFORMER MODELING
To model the sequence distribution of tokens, our AssetFormer built on a Decoder-only Transformer, using standard cross-entropy loss for next-token prediction:
(Shifntttpy(Siif(((( (okenize((P)))
(1)
where (S) denotes shifted result of predicted tokens sequence $ and (P} represents the asset comprising primitives. We adopt Llama (Touvron et al.) 2023) the Transformer backbone with our vocabulary and model configurations, and use ID rotary positional embeddings (Su et al. 2024). The text features are projected to tokens and pre-filled to the token sequence during training and inference.
Discrete Tokenization. Modular 3D assets typically consist of primitives with discrete attributes and fixed decision spaces. This inherent discreteness allows us to leverage a more efficient representation without resorting to complex graph encoders like those used in MeshGPT (Siddiqui et al. 2024). Our approach utilizes finite sets of discrete values, directly modeling pre-defined vocabulan laries for each attribute type in a lossless manner. Each asset represented as a sequence of token tuples, where the i-th sample has a primitive length of N; and a token length of 5N, reflecting the five parameters required for each attribute tuple. Following common practice (Team 2024 Liu et al., [024a] Wu et al. [202] He et al., (024), these sequences are padded with <EOS> tokens to indicate the end of the prediction.
Token Set Modeling, Fach primitive is defined by 5 parameters and jointly modeled with trans- former, necessitating the maintenance of distinct vocabularies for different attributes. The combined set of attribute vocabularies, along with the <EOS> token, forms the token vocabulary V:
V = C V R V X, v X v X v << EOS >}
(2)
| = |C| R + X
X + |2/++1,
where C, R, No, X1, Xr denote the vocabulary of primitive class, rotation, and positions, respec- tively. Consequently, the raw token sequence T is expressed as:
T= [c (c"
7 21. 22.....
-- 200
22 EOS).
where denotes the number of primitives. While this joint vocabulary approach does not affect training, as we can naively treat the periodic token sequences conventionally for next-token prediction, it requires special consideration during inference. To achieve diversity and randomness, from the logits distribution for each token. However, this can potentially produce tokens that do not belong the current attribute's 's vocabulary. For instance, after generating token C (primitive class), the next token should be drawn from (rotation). To ensure valid token set decoding, we filter out unwanted logits and re-normalize the remaining non-zero distribution.
sample
4

---

Published as a conference paper at ICLR 2026
Token Re-Ordering, Token order plays a crucial role, emphasized by recent studies in Transformer- based visual models YYu et al. 2024; Tang et al. 2024 Chang et al. [022 Chen et al. (024d). 3D assets contain rich structural intormation both globally and locally. To capture this hicrarchical and spatial relationship, we design traversal methods based on depth-first search (DFS) and breadth-fing search (BFS). These methods ensure modular connectivity locally while maintaining first-to-end sequential order globally. In industrial practice, the graph traversal is often used in validation check where the key nodes are checked with designed connectivity rules. This scenario eligible for checking the validity of the generated building in post-proe post-processing where the popped nodes of the stack in DFS are required follow certain rules with the neighborhood nodes.
In practice we start at the lower corner of the asset and traverse all primitives using a graph searching method. This produce permutation order A = {To, TI,... Ia-1) for a primitive set of length n, where 7; denotes the original index of the i-th element in the raw primitive sequence. Consequently, the re-ordered token sequence T is given by:
T' = ReOrrde(TT) = {
To T0 11
r, 1, 1,20
, Ta-1 ,OOS.. (4)
While both DFS and BFS can capture local features with modular connectivity, it is not immediately clear which method leads to better data normalization. Empirically, we have found that DFS performs slightly better as the primitive re-ordering method. This re-ordering facilitates the training of the token set modeling, and remains agnostic to asset deployment in rendering.
Classifier-Fres Guidance. Inspired by the widely used Classifier-Fres Guidance (CFG) Ho and Salimans 2022) in text-to-image diffusion models (Saharia et al. [022 Xue al. 2024 Chen et al. 2023b), which enhances generation fidelity and text alignment, recent research on generative visual Transformers has also adopted it achieve similar goals. We follow the methodology outlined in (Liu et al. [2024a] (Sun et al. (2024), randomly dropping control signals in training and utilizing unconditional logits additionally during inference. The decoding process is based on logits calculation: lefg = 1' + 8. (l I'), where I and l' denote the conditional and unconditional logits, and 8 denotes the CFG scale.
3.3 AUTOREGRESSIVE TRANSFORMER DECODING
As large language models advance, generative visual Transformers can significantly benefit from shared techniques adapted for visual tasks. We aim present preliminary analysis of sampling techniques that affect AssetFormer's quality. Furthermore, since our modular representation allows for seamless integration into game engines or rendering pipelines, without the necessity for post- processing steps like vertex merging as required in MeshGPT (Siddiqui et al. 2024), we implement decoding techniques to significantly enhance on-the-fly asset generation.
Sampling Strategies. AssetFormer generates modules sequentially to form complete assets, starting with pre-filled text tokens and continuing until the <EOS> token is generated. While we' we've explored various sampling strategies including greedy search, beam search, and top-k sampling (Fan et al. 2018), we find that top-k sampling offers a good balance between asset quality and diversity.
SlowFast Decoding, To address the computational challenges of autoregressive decoding, we introduce SlowFast decoding, our adaptation of speculative decoding (Leviathan al. [023]] Chen al. 2023a) for 3D asset generation which accelerates decoding without compromising quality and requires minimal additional training. Our SlowFast decoding employs two models:
The draft model with smaller capacity to quickly predict easy tokens.
The target model with larger capacity to handle more complex token predictions.
The effectiveness of SlowFast decoding in modular 3D asset generation stems from the varying complexity of different parts of the asset. Many modular locations, especially those following common patterns or simple structures, can be accurately predicted by the smaller, faster model. The larger, slower model is then used to decode more challenging tokens that require a deeper understanding of context or complex spatial relationships. This approach is particularly suited to modular representation, as it allows for efficient prediction of common simple components while ensuring accurate generation of more intricate or context-dependent parts of the asset. Our implementation includes modifications to filter out unwanted logits of other token types (Nash et al.
5

---

Published as a conference paper at ICLR 2026
Trb220
Treba
Hiyaarbzz
Our
Figure Qualitative comparison with comparison methods. (a) While PCG can synthesize high-quality building models, it requires meticulous algorithm design for complex buildings and can only produce simple assets that are difficult to control with text. (b) Compared with 3D generation methods, which typically yield dense meshes, struggle to accurately capture intricate geometries (the internal structure of buildings), and produce imperfect textures, our methods follow the design rationales of preferred rules (e.g., with standard primitives of plain faces) and deliver precise texture in real-world pipelines with primitive-texture mapping.
2020) during reject sampling, similar to our token set modeling approach. The detailed SlowFast decoding algorithm is presented in Algorithm 2 in the Appendix.
EXPERIMENTS
4.1 DATASET
Our dataset is derived from two sources: procedurally-synthess data using PCG (detailed in Algorithm in the Appendix) and real user-created 3D assets collected from the game. We streamlined the user data by removing extraneous long-tail information and mapping the assets to a set of 25 basic primitives. To ensure dataset quality, we employed a combination of automatic GPT-4o (OpenAI 2024) queries and manual review to filter out overly simple and duplicate samples. This process resulted in a high-quality dataset comprising 16,000 real samples and 4,000 synthesized samples. The average token length of the data sample is larger than 4,000. For DFS and BFS, we select the random node the next query if multiple nodes available which is equivalent to randomly sorting the data sample and query the first node in multiple choices. Note that our focus is on learning the modular arrangement of 3D assets, with texture considerations typically left for post-processing during production. We employ a total of 25 primitives, which can broadly categorized into three types: roof primitives, wall primitives, and other component primitives.
To enable text control over asset generation, we utilize GPT-4o to generate phrase bundles that indicate the global type of assets and highlight key features. It's worth noting rendering data exhibits a significant domain gap compared to natural images, making it challenging to caption discriminative global types for buildings using multimodal language models. Nevertheless, the generated captions provide probabilistic guidance for our model. Detailed information about the modular primitives, prompt templates, and phrase statistics can be found in the Appendix.
4.2 IMPLEMENTATION DETAILS
The joint vocabulary serves as the discrete token space for our Transformer model, with a total vocabulary size [V] of 214. This comprises |C] 25 (primitive classes), |R] = 4 (rotations), |.t = 59, | 44, and | 81 (3D positions). Complex data samples contain up to 1,000 primitives each. To enable text control, we follow (Sun et al.) 2024) to use FLAN-T5 XL (Chung ct al. 2024) as the encoder and project the features through an MLP (Chen et al., 2023b). To support CFG, we implement condition dropout ratio of 0.1 during training. Our primary model, AssetFormer-B B (312M), uses a Llama-base backbone (no pre-trained weights) consisting of 24 Transformer layers. To facilitate SlowFast decoding, AssetFormer-S (87M) with 12 Transformer layers. For inference, we employ a CFG scale of 2.0 and temperature of 0.7, using top-k sampling with k=10 for all comparisons.
additionally train smaller draft model,

---

Published as a conference paper at ICLR 2026
Order
Cflieiec Dala
Figure 4: Qualitative ablation analysis. (a) Ablation on token orders. With improper token order, the model struggles to fit and generate the distribution accurately. (b) Ablation data sources. The models fail to cover a wide range of diverse building types and exhibits a higher ratio of failure cases when trained single data source. The artifacts are indicated in red rectangles.
4.3 COMPARISON WITH THE BASELINES
PCG techniques have long been the cornerstone of game production pipelines, creating assets using PCG. While these methods are well-established, they lack more free-form control with challenges from generative methods. In this section, we include procedural generation method as baseline, using the same algorithm employed in our data synthesis stage. This baseline randomizes modular features such orientation and position but lacks complex modeling and textual control. Fig. 3 (a) demonstrates AssetFormer's ability to generate a variety of assets in a data-driven manner, controlled by text conditions, which is not present in the PCG method.
We also compare our method with state-of the-ahterrr art general 3D generation approaches, specifically SF3D (Boss et al., 20024), Tripo 2.0 (Tripo) 2024), Trellis (Xiang al.) (2024), and Hunyuan3D 2.0 (Zhao et I 2025) which are designed to generate dense meshes for open-domain objects. We use the prompt "A high-quality 3D model of a building". We use flux 1.1 [pro] I their official image generation integration to generate images for amage-to-3D pipelines. Fig. 3(b) highlights the visual results of these methods alongside our approach. By adopting a primitive- basse representation, Asset avoids the generation of low-quality, dense meshes that are difficult to integrate into industry pipelines. Although recent 3D generation methods have achieved significant improvements in producing high- quality geometry, they continue to exhibit noticeable texturing artifacts. These issues primarily stem from the suboptimal performance current texturing techniques (Zhao et al., (0225) Zhu et al.) 2025; Youwang et al.) 2024). In contrast, primitive- based generation methods benefit from the more advanced development of primitive- mapping, which results in more refined texturing outcomes.
Table 1: Quantitative results compared with baselines. We show comparison results on generation quality, indicated by FID and CLIIP score.
Methods
FID
CLIP 1
True Data
/
0.322
PCG (Algorithm 1)
108.476
0.319
AssetFormer Greedy Search
63.351
0.319
AssetForner Heam Search
63.333
0.321
AssetFormer Top-K K Sampling
55.186
0,320
To perform quantitative evaluation, assess generation quality using Fréchet Inception Distance (FID) (Heusel et al. [017]] Parmar et al.) 2022) and CLIP score (Radford et al. (021). Our evaluation procedure involves synthesizing 500 assets using sampled test prompts and rendering 500 images from a fixed viewpoint that properly captures the global structure. FID is computed between these rendered images and the full training set, with clean-FID (Parmar al., (022). which indicates the visual quality of generated assets. Due to the difficulties of large-scale rendering, the FID values are much higher than those typically seen in text-to-image works (Liu et al. 2024a Chang et al. 2023) as
hhaps//bleccfforsttabs..i
7

---

Published as a conference paper at ICLR 2026
they computed with the smaller sets, but the relative values faithfully indicate similarity and thus the quality the generated buildings since the numbers are large enough and controlled the same. CLIP score is computed between rendered image features and the text feature of fixed prompt, i.e,, "A high-quality 3D model of a building". " We opt not to use CLIP scores between images and generation prompts due challenges in obtaining informative results (originated from the domain gap) in our unusual image and text domains, i.c., all settings produce fluctuating results near 0.29, yet these relative performances poorly align with human validation.
Table D presents the quantitative results. While the PCG method can synthesize compact modular assets, struggles to cover the full breadth of the data distribution and generate richly detailed outputs with sophisticated structures. This is reflected in the FID scores, given that our training data includes both simple and complex assets. Regarding sampling strategies, our quantitative results indicate that top-k sampling outperforms both greedy search and beam search.
We further compare our method with MeshGPT (Siddiqui al. 2024), which utilizes mesh represen- tation and leverages Transformer as the decoder. Please check the detailed analysis and discussion the Appendix.
4.4 ABLATION STUDIES
4.4.1 ABLATION STUDY ON TOKEN ORDERS
The ablation study on token orders demonstrates the effectiveness of our proposed primitive token re-ordering method. Table 2 presents a comparison of different ordering operations. The results clearly indicate that re-ordering methods, specifically DFS and BFS, yield superior results compared to learning sequences in their raw order.
Table Quantitative ablation analysis token orders. We compare the results of models trained on different token orders and we also implement recent token randomized training method design for autoregressive modeling of image generation.
Ordering Techniques
FID |
CLIP
Raw Order
65.215
0.318
RAR Yu al. 2024
83.561
0.313
Sreadth-First-eeare
61.620
0.319
Depth-First-See
55.186
0.320
We also implement RAR Yuu et al. 2024 recent work focusing token randomization in training text-to-image autoregressive models. RAR employs an annealing strategy and tailored positional embedding to outperform standard raster-order-based AR image generator training. To adapt RAR to our setting, we use hierarchical operation to re-order tokens in an annealing manner, accommodating our token set modeling for building primitives. Specifically, given primitives in DFS order, we randomly permute them while freezing the second-stage permutation, maintaining the original order of attribute tokens within each primitive. Interestingly, our results indicate that RAR does not perform well in our task. We hypothesize that, unlike images which benefit from token disturbance for better bidirectional learning, the challenges in leveraging local details of 3D> structures hinder the efficient learning capabilities.
Note that while CLIP scores may be similar in absolute values, the visual results of the baselines can be poor, illustrated in Fig. a). Clear artifacts, highlighted in red rectangles, can be observed. A notable phenomenon is the presence of isolated generated parts in results obtained with raw order, reinforcing conclusion that re-ordering helps grasp local structures and ensure modular connectivity.
4.4.2 ABLATION STUDY ON DATA SOURCES
Our method incorporates data from both procedural generation and human creation. Table 3 presents metrics for models trained with different data sources, revealing an intriguing phenomenon. Models trained solely collected data show substantial improvement those using only synthesized data,
8

---

Published as a conference paper at ICLR 2026
with FID scores of 63.381 and 113.560, respectively. However, the most striking result emerges from the combination of both data sources, yielding a superior FID of 53.186. This improvement likely stems from the complementary nature of the two data types. Synthesized assets, generated through PCG, tend to be more compact and structured. While they may perform poorly in isolation due to limited diversity, they provide beneficial scaffolding for the model's learning process. In contrast, user-created data offers greater diversity and randomness, which, when combined with the structured synthesized data, enhances the model's ability to generalize.
Table 3: Ablation analysis on data sources. We train models on different configurations of data sources, and show the distribution difference of data generation.
Training Data Types
FID J.
CLIP
Synthesized Data Only Collected Data Only
113,560
0.320
63.381
0.321
Synthesized Collected
0.320
Fig. 4b) illustrates this synergy. showcasing rendered results from various generated assets. The visualization highlights how the integration of both data sources enables the model to capture a wider spectrum of architectural styles and structures, overcoming the limitations observed when relying on a single data type. Our findings underscore the importance of leveraging multiple, complementary data sources in training generative models for modular assets. The structured nature of synthesized data provides solid foundation, while the diversity of collected data expands the model's s creative range. This balanced approach not only improves the quality and variety of generated buildings but also enhances the model's robustness in meeting diverse user preferences.
4.4.3 ANALYSIS ON FLOWFAST DECODING
We implement SlowFast decoding for autoregressive asset generation, which required training an additional draft model. This draft model, with its smaller capacity and reduced number of parameters. enables accelerated decoding through meticulously designed algorithms (Leviathan et al. 2023 Chen al., 2023a). Table 4 presents the generation quality and decoding speed for models with varying parameters, controlled by the number of Transformer layers, heads, and feature dimensions. AssetFormer-B is the base model we have trained and the smaller AssetFormer-S the draft model. These results demonstrate that our tailored SlowFast decoding method successfully accelerates the generation process without sacrificing performance.
The FlowFast decoding is particularly effective for modular 3D asset generation, where prediction difficulty varies significantly. Simple primitives and standard components are swiftly handled by the draft model, while complex, context-dependent elements benefit from the larger model's 's nuanced predictions.
Table 4: Analysis on SlowFast Decoding. We train models of different parameters and perform SlowFast decoding. The generation quality and decoding speed are evaluated.
Model Configurations
FID
Speed (token/s)
AssetFoorer--S (87M)
60.420
151.31
AssetFormer-B B (312M)
55.186
80.62
SlowFast Decoding
55.831
119.02
4.4.4 ANALYSIS ON MODULAR REPRESENTATION VERSUS NATIVE 3D REPRESENTATIONS
In our work, a key design and exploration lies in the modular representation, which is well-suited for autoregressive modeling and UGC deployment. Recent text-to-3I methods have demonstrated promising native 3D generation capabilities. leveraging representations such as VecSet (ecSee/Zhangg et al. (2023); ; Zhao al. b; Zhang al. (2024); Chen et al. (2024a) 2025) or sparse voxel grids Li et al. (2025): : Wu et al. (005);; Xiang et al. (024): He et al. (2025). Nevertheless, the state of the arts still still face challenges in generating high-quality structural details cacillssartccularly for internal structures primarily stemming from limited training data and suboptimal data preprocessing pipelines. A critical bottleneck is that most existing 3D generative methods require watertight

---

Published as a conference paper at ICLR 2026
(a)
Our Asset and the Asset Watertightened
(c) Output of Fine-tuned Hunyuan3D 2.1
(b)
Details Changed After Watertightening
(d) Detalls Fine-tuned Hunyuan3D
Figure 5: Qualitative analysis on fine-tuning native 3D generative models. (a) After Watertight conversion, the modular information is lost and the geometry erroneous (e.g., the ladder). (b) The geometry details are actually changed (zoom in to see the vertices and faces). (c) The fine-tuned Hunyuan3I 2.1 produces overall inferior assets and (d) the details poor.
geometry step, which introduces additional complexity to geometry processing and prevents VAEs from recovering high-fidelity details. While native 3D generation demands more advanced data curation strategies to enable finer-graie annerggrinedd control, the modular representation we propose serves as practical alternative for real-world applications. Further details and comparisons provided in Appendix A.2 and Table
In this section, further demonstrate that the modular representation facilitates high-fidelity asset production: it preserves geometric details and yields visually consistent, high-quality results. To highlight the limitations existing text-to-3D approaches downstream tasks, we design controlled toy experiment. Specifically, we use our dataset structured as modular representations to export object geometries (vertices and faces), then apply watertight preprocessing data for native 3D generative models. Visual comparisons of assets before and after the watertight step are presented in Fig. ().). It is observed that object conversion and watertight processing lead to loss of modular information, as individual primitives are merged into single unstructured mesh. Additionally, this process alters fine- sine-graieed details (see Fig. 5(b)), distorting the geometry of primitives sourced from asset library-whert structural integrity is intentionally preserved. Furthermore, we argue that the existing native 3D models (e.g.. Hunyuan3D 2.1) struggle to generate high-quality outputs when trained on modular data particularly for complex objects with internal structures. To validate this, we conduct an overfitting experiment: we fine-tune Hunyuan3D 2. on small subset of 10 modular samples, then evaluate its performance using the exact training conditions for inference. Fig. 5 c) shows the output meshes from the fine-tuned model, while Fig. d) highlights the poor reconstruction of training sample details. These results confirm that even with overfitting, the base model fails to capture the complex structure of modular data: it corrupts individual primitives, ultimately limiting the practical utility of native 3D generation for modular-based applications.
generate training
CONCLUSION
In this work, we introduce AssetFormer, novel autoregressive Transformer-base framework designed for modular 3D) asset generation. Our approach emphasizes the modeling of assets from primitives and the learning their distribution for generative applications. The framework is meticulously tailored to accommodate both potential applications and user-generated content (UGC), ensuring versatility and adaptability in various contexts. We innovatively adapt token sequencing and decoding techniques inspired by language models, achieving high-fidelity asset generation through autoregressive modeling. We anticipate that AssetFormer will contribute significantly to the evolving landscape of 3DD content creation and enable widespread real-world applications.
10

---

Published as a conference paper at ICLR 2026
Limitations. Currently, AssetFormer is designed to accept only text input for asset generation. The ability to incorporate image-bar based conditioning remains uncertain and unexplored. Meanwhile, model relies on fixed discrete vocabularies, necessitating additional design considerations to accommodate varing design spaces.
ACKNOWLEDGEMENTS
This work was supported in part by the Research Grants Council of Hong Kong (C5055-24G, and T45-401/22-N), and the Hong Kong Innovation and Technology Fund (GHP/31822GD),
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
Mark Boss, Zixuan Huang, Aaryaman Vasishta, and Varun Jampani. Sf3d: Stable fast 3d mesh recon- struction with uv-umwrapping and illumination disentanglement. arXiv preprint arXiv:2408.00653 2024.
Andrew Brock, Theodore Lim, James M Ritchie, and Nick Weston. Generative and discriminative voxel modeling with convolutional neural networks. arXiv preprint arXiv:1608. 04236, 2016.
Tom B Brown. Language models are few-shot learners. arXiv preprint arXiv:2005.14166 2020.
Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William Freeman. Maskgit: Masked generative image transformer. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1 1135--1 1325, 2022.
Huiwen Chang. Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hssun Hsuan Yang, Kevin Murphy, William Freeman, Michael Rubinstein, et al. Muse: Text-to-image generation via masked generative transformers. arXiv preprint arXiv:2301.00704. 2023.
Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean- Caan-Baptiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling. arXiv preprint arXiv:2302.01318, 2023a.
Jinnan Chen, Lingting Zhu, Zeyu Hu, Shengju Qian, Yugang Chen, Xin Wang, and Gim Hee Lee. Mar-3d: Progressive masked auto-regressor for high-resolution 3d generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 1108--11 11092, 2025.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, al. Pixart-a: Fast training of diffusion transformer for photorealistic text-to-ine text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023b.
Rui Chen, Jianfeng Zhang, Yixun Liang, Guan Luo, Weiyu Li, Jiarui Liu, Xiu Li, Xiaoxiao Long, Jiashi Feng, and Ping Tan. Dora: Sampling and benchmarking for 3d shape variational auto- encoders. arXiv preprint 2rXiv:2412.178888 2024a.
Sijin Chen, Xin Chen, Anqi Pang, Xianfang Zeng, Wei Cheng, Yijun Fu, Fukun Yin, Yanru Wang, Zhibin Wang, Chi Zhang, et al. Meshxl: Neural coordinate field for generative foundation models. arXiv preprint ArXhv:2455.2085.. 2024b.
Yiwen Chen, Tong He, Di Huang, Weicai Ye, Sijin Chen, Jiaxiang Tang, Xin Chen, Zhongang Cai, Lei Yang, Gang Yu, al. Meshanything: Artist-created mesh generation with autoregressive transformers. arXiv preprint arXiv:2406. 10163, 2024c.
Yiwen Chen, Yikai Wang, Yihao Luo, Zhengyi Wang, Zilong Chen, Jun Zhu, Chi Zhang, and Guosheng Lin. Meshanything v2: Artist-created mesh generation with adjacent mesh tokenization. arXiv preprint 4r0iv:2408.0m 02555, 2024d.
11

---

Published as a conference paper at ICLR 2026
Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. Longlora: Efficient fine-tuni finettuning long-contest large language models. arXiv preprint arXiv:2309. 12307, 2023c.
Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5939-5948, 2019.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung. Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(24))1--13,, 2023.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetueed language models. Journal of Machine Learning Research, 25(70):1 2024.
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: universe of annotated 3d objects. In CVPR, 2023.
Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm-e: embodied multimodal language model. arXiv preprint arXiv:2303.03378, 2023.
Haihan Duan, Yiwei Huang, Yifan Zhao, Zhen Huang, and Wei Cai. User-generated content and editors in video games: Survey and vision. In 2022 IEEE conference games (CoG), pages 536-543. IEEE, 2022.
Epic. Fortnite. Epic Games, 2017.
Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873-12883, 2021.
Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. arXiv preprint 1805.0880004833 2018.
Shuangkang Fang, Shen, Yufeng Wang, Yi-Hsuan Tsai, Yi Yang, Shuchang Zhou, Wenrui Ding, Yang, al. Meshllm: Empowering large language models to progressively understand and generate 3d mesh. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14061-14072, 2025.
Takeo Igarashi,
Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojeic, and Sanja Fidler. Get3d: A generative model of high quality 3d textured shapes learned from images. Advances In Neural Information Processing Systems, 55531841-3188 2022.
Lin Gao, Tong Wu, Yu-Jie Yuan, Ming-Xian Lin, Yu-Kun Lai, and Hao Zhang. Tm-net: Deep generative networks for textured meshes. ACM Transactions on Graphics (TOG), 40(6):1-15, 2021.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Kai He, Kaixin Yao, Qixuan Zhang, Jingyi Yu, Lingjie Liu, and Lan Xu. Dresscode: Autoregressively sewing and generating garments from text guidance. ACM Transactions on Graphics (TOG), 43 (4):1-13, 2024.
Xianglong He, Zi-Xin Zou, Chia- Chen, Yuan-Chen Guo, Ding Liang, Chun Yuan, Wanli Ouyang, Yan-Pei Cao, and Yangguang Li. Sparseflex: High-resolution and trbitrary 3d shape modeling. arXiv preprint 2rX1 2025.
12

---

Published as a conference paper at ICLR 2026
Amir Hertz, Or Perel, Raja Giryes, Olga Borkine-Hormund and Daniel Cohen-Or. Spaghetti: Editing implicit shapes through part aware generation. ACM Transactions on Graphics (TOG), (1(4) 2022.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 2022.
Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. arXiv preprint
2019.
Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for rext-to-vides generation via transformers. arXiv preprint arXiv:2205. 15868, 2022.
Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint Stti:23111004000 2023.
Diederik Kingma, Auto-encoding variational bayes. arXiv preprint arXiv: 1312.61 2013.
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018
Maria Korosteleva and Hee Lee. Neuraltailor: Reconstructing sewing pattern structures from
3d point clouds of garments. ACM Transactions on Graphics (TOG), 41(4):1-16, 2022.
John Krumm,, Nigel Davies, and Chandra Narayanaswami. User-generated content. IEEE Pervasive Computing, 104) 2008
David H Laidlaw, W Benjamin Trumbore, and John F Hughes. Constructive solid geometry for polyhedral objects. In Proceedings of the 13th annual conference on Computer graphics and interactive techniques, pages 161-170, 1986.
Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274-19286. PMLR, 2023,
Changjian Li, Hao Pan, Adrien Bousseau, and Niloy J Mitra. Free2cad: Parsing freehand drawings into cad commands. ACM Transactions on Graphics (TOG), 41(4):1-16, 2022.
Songlin Li, Despoina Paschalidou, and Leonidas Guibas. Pasta: Controllable part-aware shape generation with autoregressive transformers. arXiv preprint trXiv:2407.13677 2024a.
Tianhong Li. Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. arXiv preprint arXiv:2406.11883 2024b.
Zhihao Li, Yufei Wang, Heliang Zheng, Yihao Luo, and Bihan Wen. Sparc3d: Sparse representation and construction for high-resolution 3d shapes modeling, arXiv preprint arXiv:2505. /4521, 2025.
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming- Yu Liu, and Tsung- Yi Lin. Magic3d: High-resolution text-to-3d content creation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 300-309, 2023.
Dongyang Liu, Shitian Zhao, Le Zhuo, Weifeng Lin, Yu Qiao, Hongsheng Li, and Peng Gao. Lumina- mgpt: Illuminate flexible photorealistic text-to-image generation with multimodal generative pretraining. arXiv preprint MXXi:2448..02677 2024a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024b.
13

---

Published as a conference paper at ICLR 2026
Lijuan Liu, Xiangyu Xu, Zhijie Lin, Jiabin Liang, and Shuicheng Yan. Towards garment sewing pattern reconstruction from a single image. ACM Transactions on Graphics (TOG), 42(6))1-15, 2023a.
Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Generating multiview-consisten images from a single-view image. arXiv preprint 3xXi::2309.04553. 2023b.
Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma. Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Single image to 3d using cross-domain diffusion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9970-99988 2024.
Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2837-2845, 2021.
Kaichun Mo, Shilin Zhu, Angel X Chang, Li Yi, Subarna Tripathi, Leonidas Guibas, and Hao Su. Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 909-918, 2019.
Charlic Nash, Yaroslav Ganin, SM Ali Eslami. and Peter Battaglia. Polygen: An autoregressive generative model of 3d meshes. In International conference on machine learning, pages 7220-7229. PMLR, 2020.
OpenAl. Hello gpt-4o. https:/ppeua..ccmimdeeeeelllooop 2024a., 2024.
Gaurav Parmar, Richard Zhang, and Jun-Yan Zhu. On aliased resizing and surprising subtleties in gan evaluation. In CVPR, 2022.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. /4988, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8766 PMLR, 2021.
Alexander Raistrick, Lahav Lipson, Zeyu Ma, Lingjie Mei, Mingzhe Wang, Yiming Zuo, Karhan Kayan, Hongyu Wen, Beining Han, Yihan Wang, et al. Infinite photorealistic worlds using procedural generation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 12630-12641, 2023.
Daniel Ritchie, Paul Guerrero, R Kenny Jones, Niloy Mitra, Adriana Schulz, Karl DD Willis, and Jiajun Wu. Neurosymbolic models for computer graphics. In Computer graphics forum, volume 42, pages 545-568. Wiley Online Library, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pages 10684-10695, 2022.
Michael Rymaszewski. Second life: The official guide. John Wiley & Sons, 2007.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily I. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479-36494, 2022.
Tanya Short and Tarn Adams. Procedural generation in game design. CRC Press, 2017.
Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Tatiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, and Matthias Nießner. Meshgpt: Generating triangle meshes with decoder-only transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19615-19625, 2024.
14

---

Published as a conference paper at ICLR 2026
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.1477 14792, 2022.
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 127063, 2024.
Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06555. 2024.
Jiaxiang Tang, Jiawei Ren. Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatring for efficient 3d content creation. arXiv preprint arXiv:2309. 2023.
Jiaxiang Tang, Zhaoshuo Li, Zekun Hao, Xian Liu, Gang Zeng. Ming- Yu Liu, and Qinsheng Zhang. Edgerunner: Auto-regressive auto-encoder for artistic mesh generation. arXiv preprint arXiv:2409. 181 14, 2024.
Chameleon Team. Chameleon: Mixed-modas early-fusion foundation models. arXiv preprint arXiv:2405.098/8. 2024.
Yao Teng, Han Shi, Xian Liu, Xuefei Ning, Guohao Dai, Yu Wang, Zhenguo Li, and Xihui Liu. Ac- celerating auto-regressive text-to-image generation with training-free speculative jacobi decoding. arXiv preprint arXiv:2410.01699, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv: 13971, 2023.
Tripo. Tripo 2.0. htmps:/www..rrppo...aoooootttppp aadp:////r--p-----------ddieiied 2024., 2024.
Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, Karsten Kreis, et al. Lion: Latent point diffusion models for 3d shape generation. Advances in Neural Information Processing Systems, 35:10021-10039 2022.
Aäron Van Den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks.
In International conference on machine learning, pages 1747-1756. PMLR, 2016.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Herbert Voelcker and Aristides Requicha. Constructive solid geometry. 1977.
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409. 2024a.
Zhengyi Wang, Jonathan Lorraine, Yikai Wang, Hang Su, Jun Zhu, Sanja Fidler, and Xiaohui Zeng. Llama-mesh: Unifying 3d mesh generation with language models. arXiv preprint
arXiv:2411.09595 2024b.
Jiajun Wu, Chengkai Zhang, Tianfan Xue, Bill Freeman, and Josh Tenenbaum. Learning proba- bilistic latent space of object shapes via 3d generatiie-aavversriil modeling. Advances in neural information processing systems, 29, 2016.
Rundi Wu, Chang Xiao, and Changxi Zheng. Deepead: A deep generative network for computer- aided design models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6772-6782. 2021.
Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Yikang Yang, Yajie Bao, Jiachen Qian, Siyu Zhu, Xun Cao, Philip Tom, et al. Direct3d-s2: Gigascale 3d generation made easy with spatial sparse attention. arXiv preprint arXiv:2505. 2in.22555./112. 2025.
Tong Wu. Jiarui Zhang, Xiao Fu, Yuxin Wang, Jiawei Ren, Liang Pan, Wayne Wu, Lei Yang, Jiaqi Wang, Chen Qian, et al. Omniobject3d: Lorge-vocabulary 3d object dataset for realistic perception, reconstruction and generation. In CVPR, 2023.
15

---

Published as a conference paper at ICLR 2026
Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong. and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation. arXiv preprint arXiv:2412.01100 2024.
Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv2244.007191, 2024a.
Xiang Xu, Joseph Lambourne, Pradeep Jayaraman, Zhengqing Wang, Karl Willis, and Yasutaka Furukawa. Brepgen: A b-rep generative diffusion model with structured latent geometry. ACM Transactions on Graphics (TOG), 15(4):1-14 2024b.
Zeyue Xue, Guanglu Song, Qiushan Guo, Boxiao Liu, Zhuofan Zong, Yu Liu, and Ping Luo. Raphael: Text-to-image generation via large mixture of diffusion paths. Advances in Neural Information Processing Systems, 36, 2024.
Jingwen Ye, Yuze He, Yanning Zhou, Yiqin Zhu, Kaiwen Xiao, Yong-Jin Liu, Wei Yang, and Xiao Han. Primitiveanything: Human-crafter 3d primitive assembly generation with auto-regressive transformer. arXiv preprint arXiv:2505. 04622, 2025.
Kim Youwang. Tae- Hyu- Oh, and Gerard Pons-Moll. Paint-it: Text-to-texture synthesis via deep convolutional texture map optimization and physically-based rendering. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 4347 4356, 2024.
Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and I. Liang-Chieh Chen. Randomized autoregressive visual generation. arXiv preprint arXiv:2411.00770 2024.
Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neural fields and generative diffusion models. ACM Transactions On Graphics (TOG), 42(4))116, 2023.
Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creating high-quality 3d assets. ACM Transactions Graphics (TOG), 43(4):1-20, 2024.
Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang. Sheng Zhang, Xianghui Yang, et al. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation. arXiv preprint arXiv:2501, 122222 2025.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing vision-language understanding with advanced large language models. arXiv preprint MrXi:2230410552. 2023.
Lingting Zhu, Jingrui Ye, Runze Zhang, Zeyu Hu, Yingda Yin, Lanjiong Li, Jinnan Chen, Shengju Qian, Xin Wang, Qingmin Liao, al. Muma: 3d pbr texturing via multi-channel multi-view generation and agentic post-processing. arXiv preprint arXiv: 18461, 2025.
16

---

Published as a conference paper at ICLR 2026
A
APPENDIX
A.1
USER STUDY
We conducted a user study to better validate the qualitative performance of our method. The study involved 6 participants aged between 22 and 28 years. The participants were asked to grade the buildings based on four criteria: compactness, diversity, aesthetic, and complexity. The grading was done in batches, each consisting of six building samples. We included ground truth data, PCG generated data, and the synthetic data from AssetFormer. The results, shown in Table are rated on scale of to 5, with 5 being the highest. Our participants widely acknowledged that our method produces high-fidelity results in terms of diversity, aesthetic, and complexity. It is worth noting that the PCG method, which generates buildings with simpler structures, received higher grades for compactness from participants, even surpassing the ground truth due to the different domains.
Table 5: User study results. The ratings of compactness, diversity, aesthetic, and complexity, a scale of 1-5.
Method
Compactness
Diversity
Aesthetic
Complexity
Ground Truth
3.83
4.00
3.67
4.42
PCG
4.47
2,42
3,33
2.08
AssetFormer
3.42
3.50
3.50
3.92
A.2 COMPARISON WITH MESHGPT
We select MeshGPT (iiddiqui et al., 2024), which utilizes mesh representation and leverages a Transformer the decoder, baseline for qualitative comparison. We further discuss the charac- teristics both mesh representation and modular representation, shown in Table Additionally, we attempted directly fine-tune language models with supervised fine-tuning our building JSON data. Given that the building data can comprise up to 1000 primitives, requiring more than 3K tokens when tokenized from the raw JSON data which includes the primitives types and attributes, we adopted LongLoRa (Chen et al. 2023c) to fine-ture Llama-2 (Touvron et al. (2023). However, the results are far poorer than AssetFormer, with qualitative winrate below 5%. We account for the inherent complexity and implicit nature of the representation, which pose significant challenges for language-bar based models to comprehend and create long sequences.
We present the comparison results with MeshGPT in Fig. 6 Although we generate results in modules with AssetFormer, is important to note that these results can be seamlessly converted to triangle meshes if needed, the modules are compact, as shown in Fig. 6 For this comparison, we first converted all building data to triangle meshes and extracted the vertex and face information required by MeshGPT. We then trained the autoencoder and Transformer our data using MeshGPT. We present both non-transparent and transparent rendered results. While MeshGPT encodes faces and vertices and learns mesh generation based on face and vertex representation, becomes evident that as the task complexity increases, i.e., generating complex buildings with numerous vertices and faces, the training becomes challenging and the decoding often fails. We do not include subsequent works like MeshAnything (Chen et al. 2024c) and MeshXL (Chen et al. 2024b) as they adopt the same representation. Additionally, since modules can be decomposed into triangle meshes, modular representation is more efficient and requires fewer tokens compared to mesh-based generation methods. Even recent works focusing on compact tokenization of meshes, such as EdgeRunner (Tang et al., 2024), typically handle meshes with fewer than 4K faces, whereas our data can comprise more than 30K faces in triangle meshes.
Table 6: Comparison of mesh representation and modular representation.
Representation
Lossless
Ready for Engines
Triangle Meshes
Efficient
No Post-Processing
User-Friendly
Mesh
X
X
Modular
17

---

Published as a conference paper at ICLR 2026
MeshCPT AssetFormer
Non-Transparent
Transparent
Transparent Triangle
Vieupoint
Non-Transparent
Transparent
Non-Transparent
Transparent
View
Figure 6: Qualitative comparison with MeshGPT. We present the non- con-transparent and transparent results of MeshGPT and ours. Our method can produce compact arrangement of primitives, as demonstrated by the transparent rendering images and the viewpoint from the inside.
Table presents the characteristics of mesh-based and modular representations. Meshes benefit from not requiring representation conversion in 3D generation methods that adopt Triplane and implicit representations (Xu et al., 2024a| Poole et al. 2022), which has recently increased their popularity, It is worth noting that modular representation also inherits the crucial strengths of meshes. Built upon primitive modules, the representation is lossless, ready for game engines, and can be directly converted into triangle meshes.
Furthermore, Fig. showcases the X-Ray results, revealing internal structures buildings. The results demonstrate that AssetFormer is capable of synthesizing buildings with not only impressive appearances but also intricate internal structures. It is important to note that the internal structure of game assets is crucial for real-world applications. While AssetFormer excludes explicit texture information, the modular nature of our generated geometry allows for versatile streamlined applica- tions. We showcase the versatility of modular representation in Fig. by mapping the primitives to diverse set of textured intricate modules. This flexibility aligns with industry practices and enables seamless integration with specific scenes game aesthetics. Moreover, such modular representation supports both procedural and generative texture rendering techniques, allowing for dynamic and diverse visual outcomes.
Mesh representation can hardly compete special needs in real-world scenarios. Using mesh representation to train generative models presents key issue: the token length can be extremely long, especially for real-world objects with details. Additionally, after decoding, mesh representa- tion requires post-processing to merge close points in 3D space, whereas modules are compactly connected. Furthermore, although mesh representation is ready for artists, it is not yet user-frient geer-friendly. In contrast, modular representation- based generation serves as a powerful technology integrated for user-generated content (UGC), thanks its user-friendly manipulation.
A.3 ALGORITHM DESCRIPTION
Procedural Content Generation. PCG is effective for quickly synthesizing simple buildings and can produce data samples without artifacts. However, this method struggles adapt to variety of complex buildings, resulting in a mismatch with user preferences and a gap in representing intricate building distributions. We randomly set attributes such width, length, and floor height. Using roof primitives, wall primitives, and component primitives, we can randomly generate walls, floors, and roofs, and decorate each aspect with special primitives, such doors and stairs.
18

---

Published as a conference paper at ICLR 2026
Non-Transparent Transparent
waretoias, story, windous
carfie multi-ttony
apartment: muite-atory.
cour tyand, single story.
pliched
Figure 7: Ray results the generated buildings. We further present transparent results that highlight the complex and compact structures of the generated samples. For supplementary illustration, we provide results from different viewpoints of the buildings in Fig.
w/o Texture
DST CETTER
w/ Texture
Figure 8: Visualization of generated buildings with textured intricate modules mapping. Our generated assets can be seamlessly integrated into engine runtime, mapped with different textured modules of dewal-of-details. We show various viewpoints for reference.
Algorithm 1 Procedural Content Generation
1: width Randint(11 .MXX_ WIDTH)
2: length Randint(11 (1, MAX LENGTH)
floorHeight Randint(L, MAX FLOOR
4: // Decorate with floor primitives and stair primitives
5: SetlWal(width length, floor Height)
6: // Decorate with floor primitives and stair primitives
7: for floor_id shor_iidin in Range(MAX._ WTLOOR_HHEIOO
do
8:
HaPPlaeewidth length, floor id)
9: end for
10: // Decorate with roof primitives
11: SetRoof width, length, floor Height)
19

---

Published as a conference paper at ICLR 2026
Algorithm 2 SlowFast Decoding
1: Require the target model AssetFormer-B which produces q(-1-), the draft model AssetFormer-S which produces p(-l-)
2: Input the text prompt, which gives pre-filled tokens prefix, the lookahead K, and target sequence length
3: Set token number
4: while πt < 'I' do
5: Sample from draft model
6:
in Range(K) do
7:
3r ~ p(alprefiaz To St-1))
8:
end for
9:
Forward target model
10:
Compute logits (alppre/fer fix, 20...., un- ,.,, O....,K
Drop with probability 1-4/p prpeet_pos--a
12:
reject pos = Random GronomDDoo(p logils, logils)
13:
I/ Get the primitive types for the rejected tokens
14:
reject_fype
reject_pos.
Draw from q as Speculative Sampling with primitive token type awareness resampled tokens Sample(rejeet, prepleefreeeeeoos.eeeoooee pos, reject__eeppp 9 loogiss,, ,, p logits)
17:
Sample Tn-K from if needed Update
18:
19:
end while
Return [r0.".",, 2T
SlowFast Decoding, The SlowFast Decoding method, is adapted from Speculative Decoding (Chen al. 2023a Leviathan et al., 2023), which utilizes two models of different sizes to accelerate the sampling of large language models. Following this key insight, train draft model, AssetFormer-B BB, to quickly produce draft tokens. After decoding with the draft model, the target model processes the token sequences to obtain logits, which are used to reject existing tokens with defined probability. Notably, since our modular representation requires meaningful token orders, we also need to track the vocabulary types for each token. With the tracked types, filter out the logits that do not belong to the current token and sample within the re-normalized distribution. Experiments have clearly demonstrated the effectiveness of SlowFast Decoding, achieving acceleration without compromising performance.
A.4 EMERGENT EDITING CAPABILITIES
Continue Building
(b) Add Roof
Figure 9: Illustration of emergent editing of AssetFormer. We showease that without further training, the model is able to edit the modular buildings. The case (a) and case (b) show that the model can continue building and add roof. The two prompts are "small building, single-story, flat roof, minimal windows" and "modern building, multi-story, pitched roof, lots of windows".
We further demonstrate that our model enables zero-shot editing of modular buildings, as illustrated in Fig. [9] Framed as a sequence inpainting task, this application showcases the model's 's ability to extend existing modular structures and incorporate roof components. By pre-training the model on text-to-modular building data, it learns both the structural constraints and semantic relationships inherent to modular architectures. In practice, given a modular building representation, we first perform based token reordering preprocessing, using these reordered tokens the initial sequence. Additionally, unwanted primitives (c.g., existing roof structures) can be removed, with the remaining tokens serving the target for inpainting. Notably, despite not being explicitly trained for
20

---

Published as a conference paper at ICLR 2026
this editing task nor exposed to the distinct token order patterns of the inpainting setup the model successfully predicts the missing target primitives and completes the editing task.
A.5 DIVERSITY FOR THE SAME PROMPT
castle, single-story. flat roof, simple design
apartment, multt-story. pitceed roof, lots windows
The generated results with the same prompts. We showcase that the generated assets with the same prompts to show the diversity. The two rows are the cases two different prompts.
Figure 10:
We present generated results with the same prompts in Fig. 10 The cases show that with the same prompts, benefitting from the sampling of the Transformer, the generated samples are diverse.
A.6 MORE ASSETS: GALLERY IN UNREAL ENGINE
Assets w/ Texture Mapping
Optional Genmatry Mapping and Lighting Conditions
Assets Gallery in UE
Editng
Lighting Control
Figure 11: Asset gallery in UE. We showcase that the generated assets be easily edited and seamlessly integrated into Unreal Engine (UE), enabling the assembly of cohesive and production- ready gallery collections.
We present additional generated results in Fig. The modular representation natively enables texture mapping and even geometry mapping. Notably, the generated samples-equipped with customizable textures, optional geometry mapping, and adjustable lighting be seamlessly integrated into Unreal Engine, directly supporting real-world 3D content production workflows.
A.7 MORE INFORMATION ON THE DATA
Modular Primitives. Fig. 12 illustrates the primitives utilized in our data. These categories are displayed in three separate columns. Additionally, we provide statistics the primitives Fig [3] We present the distributions of the PCG data and the collected real data to show the distribution differences in the dataset.
Prompt Curation. To prepare the text conditions for the building samples, use GPT-4o (OpenAl 2024) to generate text descriptions based on rendered images from a fixed viewpoint. To control the flexibility of the text conditions,
curated prompts, presented in Table 7 Additionally, provide statistics of the text phrases Fig [4] We present the distributions of the PCG data and the collected real data to show the distribution differences in the dataset.
21

---

Published as a conference paper at ICLR 2026
Concave Roof
Beam
Lowi Concaue Roof
Low Wall
Floor
Conuex Roof
Window Wall
Railing
Low Concex Roof
Right-Angle Wall
Rectangular Column
Sloped Roof
Inverted Right-Angle Wall
Sloped Ladder
High Sloped
Low Right-Angle Wall
Low Sloped Ladder
Lou Sloped Roof
Triangular Wall
Straight Ladder
Triangular Roof
Door
Lowi Triangular Roof
Wide Door
Figure 12: Descriptions of primitives in building data. We showcases roof primitives, wall
primitives, and other component primitives in three columns.
Table 7: The prompt of querying GPT-4o.
I will provide an image of building and you should generate one string, which comprises of several phrases.
The first phrase describes building type: Example phrases and rules:
"castle": often with features like towers or battlements.
"skyscraper': tall, rectangular building.
'courtyard': high, often with an open space the middle.
'mansion", : large, impressive, featuring multiple stories high-end architectural details. "townhouse': : usually multi-story, often in row with similar houses.
'apartment": multi-story, often with lots of windows. If all the above types are precise. you
other types.
The second phrase should describe the height of the building in terms of floors: Example phrases: "single-story', 'multi-story', 'high-rise". If the building is higher that 5 stories, can be classified as 'high-rise'.
remaining two phrases, giving most precise features of building. You do not need to always use some phrases if they are ambiguous. For example, if there are more than 15 windows, then you can use 'lots of windows' but do not treat doors as windows. Example phrases: 'pitched roof", 'flat roof", 'lots of windows', 'magnificent', 'dull', 'weird", etc.
A.8 VIDEO VISUALIZATION
We include video demo of AssetFormer in our supplementary materials, including the generation process of the method, the comparison of token orders, and the integration of generated game assets into game engines.
To further compare the token order, which is a key factor in our methodology design, we present the video visualization demonstrating the construction sequence of results generated by different token orders. We construct the building primitives according to the decoding sequence and show the construction videos for raw orders, BFS, and DFS.
Additionally, we demonstrate the seamless integration of modular representation- -aased generated building data into game engines. To this end, we develop a lightweight software on Unreal Engine, a mature game engine in the gaming industry. By placing the modular building in a scene, we can control digital human to traverse the scene, surrounding the building and entering it, as if the building is part of the normal environment in games. This is especially appealing when we map the building assets to have complex geometry and texture details with asset mapping. The video demo clearly demonstrates that our generated game assets are effective, convenient, clean, and ready for integration into game engines.
22

---

Published as a conference paper at ICLR 2026
PCG Data Primitive Type Distribution
Real Data Primitive Type Distribution
0.30
0.30
0.25
Wobbabityy
0.25
0.20
0.15
810
0.10
0.55
0.00
0.00
Runyy
Wall
Low sear
sloped Rool
Rellin
Suppel Latder
High sirpeed fooo
Conves Reet
Deerr
Wall
Ratiin
senato Walf
Oour
Skaped Root
Low Wall
Lew Sloped Road
Low stapet Loditn
Windine
iniine type
Prirstive 7pp
Figure 13: Phrases statistics of primitives. We show the histograms on primitives of PCG data and real data.
(a) Phrases Distribution of PCG Data
Data Cenributin
Deta
homaning Phrees Ditrbktter
0.0-
Eabiing
M0 goge
(b) Phrases Distribution of Real Data
Dicviution
0331
02.3
083
0809
Prises
Figure 14: Phrases statistics of text phrases. We show two histogram sets in (a) and (b) on text phrases of PCG data and real data. The three histograms in set show the distribution of the first phrases, the second phrases, and the remaining phrases, used in our dataset.
A.9 LLM USAGE DISCLOSURE
We used GPT-4o exclusively for grammar checks and light editing. The LLM acted as writing aid and was not involved in generating ideas or content, thus not qualifying as an author.
23


Reviews:
[{"review_id": "F1clsJQI7c", "reviewer": "Reviewer_jBah", "summary": "In this work, the authors introduces AssetFormer, which is a new type of autoregressive Transformer based framework. The authors design this for generating 3D assets in a modular way. Their approach focus on how assets are model from primitives and how they learn the distribution of these primitives for generative uses. The authors innovatively adapted token sequencing and decoding methods that is inspired by language models.", "questions": "What makes autoregressive modelling better than diffusion-based generation?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Encouraging results.\n- Theoretically well-written paper", "weaknesses": "- Can have more ablations.\n- Can be good for the paper to have more expressive examples.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "btEz25OKN3", "reviewer": "Reviewer_Kioq", "summary": "The paper proposes a modular text-to-3D generation approach by leveraging an auto-regressive transformer model. Specifically, it introduces a new dataset comprising both real-world and procedurally generated homestead modular models, along with text descriptions derived from renderings of the 3D models using GPT-4o. This curated dataset is used to fine-tune a decoder-only transformer model, such as Llama, by applying techniques including discrete tokenization, token reordering, and classifier-free guidance. To decode the tokens generated by the model during inference, the paper proposes a method called SlowFast decoding.", "questions": "- The paper lacks detailed dataset statistics, particularly regarding the distribution between real-world and procedurally generated (PCG) data. Providing separate breakdowns, similar to those shown in Figure 9, for each data source would help understand the diversity and representativeness of the training data.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The use of large language models (LLMs) to generate sequential, modular 3D primitives enhances both efficiency and interpretability.\n- If released, the dataset could be a valuable contribution to the community, addressing the current gap in large-scale modular 3D assets.\n- The ability to directly integrate the generated 3D models in game engines unlocks numerous real-world applications.\n- AssetFormer demonstrates strong generation quality through its proposed techniques, including token ordering, token sampling, and decoding strategies.", "weaknesses": "- The paper is similar in spirit to existing approaches [1,2] that utilize different 3D primitives with autoregressive transformer models for sequential 3D asset generation. While the method is adapted to a homestead-specific dataset, the core technical contribution appears incremental. Additionally, the paper would benefit from including and discussing these relevant prior works.\n- Quantitative evaluation is limited, with comparisons restricted to MeshGPT. Broader benchmarking against other LLM-based 3D generation methods (e.g., [1] or [2]) would help clarify whether observed performance improvements are related to the backbone LLM-model or to the way the 3D data is preprocessed for fine-tuning the model.\n- The qualitative comparisons in Figure 3(b) may not be entirely fair, as the baseline text-to-3D models are trained on more diverse 3D datasets. A more rigorous evaluation would be to fine-tune these models on the proposed homestead dataset to ensure a fair comparison.\n- While the paper discusses the diversity of generated 3D assets, it does not provide qualitative results demonstrating diversity for the same prompt. Including such examples would help assess the model’s ability to produce varied outputs under identical inputs.\n\nAdditional references:\n\n[1] MeshLLM: Empowering Large Language Models to Progressively Understand and Generate 3D Mesh\n\n[2] LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Zpm7Matv4R", "reviewer": "Reviewer_mESg", "summary": "This paper introduces AssetFormer, an autoregressive Transformer model designed to generate modular 3D assets from text descriptions. Addressing the limitations of traditional 3D representations for user-generated content and gaming—such as large file sizes and difficulty in editing—the authors propose representing assets as a collection of discrete primitives. The core contributions include: 1) a novel framework that models the generation of primitives (class, position, rotation) as a sequential, next-token prediction task; 2) the creation of a large-scale, high-quality hybrid dataset combining procedurally generated assets with real user-created content; 3) an innovative token re-ordering strategy based on graph traversal (DFS) to capture spatial relationships, which is shown to be crucial for generating coherent structures; and 4) the adaptation of efficient decoding techniques like SlowFast decoding for practical application. Experiments demonstrate that AssetFormer generates high-quality, diverse, and structurally sound assets that are directly usable in game engines, outperforming both procedural baselines and showing clear advantages over general-purpose mesh generation models for this task.", "questions": "In a real UGC platform, the library of available primitives might expand over time. How would AssetFormer handle the addition of new primitive types to its vocabulary? Would it require a full retraining, or could methods like vocabulary expansion be adapted?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1: The decision to focus on modular assets is highly practical and directly addresses key pain points in UGC and game development. This representation is efficient, easy to edit, and perfectly suited for an autoregressive framework. \n\n2: The creation and combination of a procedural dataset and a real-world user-generated dataset is a major strength. The ablation study convincingly shows that this hybrid approach is superior to using either source alone, providing both structure and diversity. This dataset could be a valuable resource for the community.", "weaknesses": "1: The model relies on a fixed, discrete vocabulary for primitive types, positions, and rotations. This fundamentally limits its creative potential to the predefined set of components and a grid-based layout. It cannot generate novel primitive shapes or place them with continuous precision, which will be a constraint for more organic or complex designs.\n\n2: The method is demonstrated on buildings composed from a specific set of architectural parts. It is unclear how well the approach would scale to a much larger vocabulary of primitives or to entirely different asset categories that may have different structural rules.\n\n\n3: The text prompts are \"phrase bundles\" describing high-level features. While effective, this may limit the model's ability to understand more complex, compositional, or spatial instructions in natural language. The quantitative CLIP scores do not show a large separation between methods, suggesting text alignment could be further improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RK5483Tjpy", "reviewer": "Reviewer_QQ43", "summary": "This paper presents a method for generating game assets as a sequence of pre-modeled 3D modules.  The experiments utilize a new dataset including 20k building models with text annotations from GPT-4o, probably the first of its kind.  The model is a decoder only transformer built on a Llama backbone.   \n\nThe paper contains interesting comparisons for the ordering in which the modules are generated in the sequence.  Qualitative and quantitative results demonstrate that this ordering plays an important role in generation quality.   A speculative decoding scheme is also presented which is shown to speed up generation with minimal loss of quality.", "questions": "I'm interested to learn more about the DFS and BFS algorithms employed.   \n- Given a node with k neighbors which have not yet been visited, how do you choose which of these to add next?  Perhaps you used an off the shelf graph sorting algorithm where the behavior is not well defined.\n- I would be very interested to know if some kind of lexicographical ordering in x0, x1, x2 would give a significantly different result over making this choice randomly.  I suspect the way this choice gets made would have a large impact on the results.  Perhaps the DFS is making this decision in some non-random way.  A figure showing an example of the ordering would help spot any pattern.\n\nLine 208\n> To ensure valid token set decoding, we filter out unwanted logits and re-normalize the remaining non-zero distribution\nIt's interesting that this was necessary.  87M parameters should be enough for the syntax of the sequence to be learned with very few invalid sequences, however 16k-20k data is very low and may not have been enough.   I would be very interested to know if this filtering actually had a substantial impact.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The dataset used in the experiment is a major contribution if made available to the community.  \n\nThe investigation into the ordering of the blocks is very interesting.  Understanding how best to order elements like this has applications to other problems.  It’s very interesting to see that DFS outperformed BFS.  The failure case in the qualitative figure gives a hint as to what goes wrong.  \n\nThe “SlowFast speculative decoding” is well motivated and the timings show this is effective.\n\nThe ablation studies and comparisons with baselines are sound.  Evaluation of generative models is always tricky.  To the best of my knowledge the authors have done a good job evaluating their model and comparing to other methods.", "weaknesses": "I would have liked more details on the DFS and BFS algorithms.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)