ShapeGen4D: Towards High Quality 4D Shape Generation from Videos
Authors: Jiraphon Yenphraphai
Ashkan Mirzaei
Jianqi Chen
Jiaxu Zou
Sergey Tulyakov
Raymond A. Yeh
Peter Wonka
Chaoyang Wang
Venue: ICLR 2026 Poster (2026)
Keywords: 4D reconstruction
generative model
Primary area: generative models

Abstract: Video-conditioned 4D shape generation aims to recover time-varying 3D geometry and view-consistent appearance directly from an input video. 
In this work, we introduce a native video-to-4D shape generation 
framework that synthesizes a single dynamic 3D representation end-to-end from the video.
Our framework introduces three key components based on large-scale pre-trained 3D models:  (i) a temporal attention that conditions generation on all frames while producing a time-indexed dynamic representation; 
(ii) a time-aware point sampling and 4D latent anchoring that promote temporally consistent geometry and texture; 
and (iii) noise sharing across frames to enhance temporal stability. 
Our method accurately captures non-rigid motion, volume changes, and even topological transitions without per-frame optimization. Across diverse in-the-wild videos, our method improves robustness and perceptual fidelity and reduces failure modes compared with the baselines.

Published as a conference paper at ICLR 2026
SHAPEGEN4D: TOWARDS HIGH QUALITY 4D SHAPE GENERATION FROM VIDEOS
Jiraphon Yenphraphai 1,2 Ashkan Mirzaei'
Jianqi Chen? Jiaxu Zou'
Sergey Tulyakov' Raymond A. Yeh Peter Wonka 1.3
Chaoyang Wang'
SSna
Purruue University KAUST
https: attpe://shhapggenddd..hhbhbb io/
Figure 1: ShapeGen4D generates high-quality mesh sequences from input monocular videos.
ABSTRACT
rideo-conditioned 4D shape generation aims recover time-varying 3D geom- etry and view-consistent appearance directly from an input video. In this work, we introduce native video-to-4D shape generation framework that synthesizes a single dynamic 3D representation end-to-end from the video. Our framework introduces three key components based on large-scale pre-trained 3D models: (i) a temporal attention that conditions generation on all frames while producing a time-indexed dynamic representation; (ii) a time-aware point sampling and 4D latent anchoring that promote temporally consistent geometry and texture; and (iii) noise sharing across frames to enhance temporal stability. Our method ac- curately captures non-rigid motion, volume changes, and even topological tran- sitions without per-frame optimization. Across diverse in-the-wild videos, our method improves robustness and perceptual fidelity and reduces failure modes compared with the baselines.
INTRODUCTION
Video-conditioned 4D shape generation aims to produce dynamic objects and their textures from an input video. With the rapid progress of generative models for images (BlackForestl.abs) 2024), 2025a), this task has seen
videos (Team, 2025), and 3D geometry (Tencent/unyyuannDTTaaa markable advances over the past two years. One line of work builds on score distillation sampling (SDS) (Poole et al., 2023) to optimize 4D shapes (Singer al., 2023). However, SDS methods are fragile and computationally expensive. Subsequent approaches improve robustness and efficiency by adopting a two-stage pipeline (Zhang al., 2024a; Xie et al., 2025; Wu al., 2025a: Wang et al., 2025b;a): first generating multi-view videos with image video diffusion model, then reconstructing 3D or 4D geometry from these views. While faster and more stable than SDS, these
Work done during internship Snap

---

Published as a conference paper at ICLR 2026
methods remain limited by reconstruction errors and imperfections accumulated during multi-view generation, leading to suboptimal quality and efficiency.
More recently, inspired by the success of large-scale latent-diffusion transformers for 3D genera- 2025a; Xiang et al., 2025; Zhang et al., 2024b), two preliminary approaches have attempted adapt pretrained 3D generative models for video-to-4D tasks. How- ever, these methods either do not natively generate meshes in 4D or rely on optimization-ban pipelines rather than fully feedforward generation. V2M4 (Chen et al., 2025) applies 3D gen- erative model independently to each video frame, followed by optimization to improve temporal smoothness and texture consistency. Despite these efforts, the method remains prone artifacts in geometry, motion, and texture. In contrast, Gaussian Variational Field Diffusion (GVFD) (Zhang et al., 2025) generates the first frame with Trellis and trains a model to deform this initial geom- etry. This strategy has two key drawbacks: (a) geometry and texture are conditioned only on the first frame, ignoring new information revealed later frames, and (b) the reliance on limited 4D) training data,c.g.. (Deitke et al., 2023), restricts the framework to rigid or near-isometric deforma- tions, preventing it from handling topological changes or large volume variations such as growth or shrinkage.
tion (TencentHuunuann3DDi
In this work, we propose the first video-to-4D generation framework that directly produces dy- namic 3D meshes. Building recent advances in 3D generative models (TencentHonyuan33DTe 2025a; Xiang et al., 2025; Li et al., 2025a), our approach differs from concurrent work GVFD: rather than introducing new modality such deformation offsets of Gaussian particles, we generate sequence of 3D meshes-a capability already learned by base 3D models-while explicitly address- ing temporal consistency. This design accommodates variations in topology and relaxes constraints on the types of possible animations. More importantly, leveraging state-of-the-art pretrained 3D generators enables effective transfer of knowledge from large-scale 3D datasets, which are far more abundant than 4D datasets. With the generated mesh sequences, we can orthogonally apply reg- istration and texturization to build animatable assets, i.e., simpler and more stable process than generating animations from scratch, as in GVFD.
To realize this framework, we adapt 3D generative models to the 4D setting through three key design choices: (a) incorporating spatiotemporal attention in the shape diffusion transformer to cap- ture temporal dependencies; (b) redesigning point sampling in the shape encoder to improve latent consistency and diffusibility; and (c) leveraging shared noise across frames enhance temporal sta- bility. We empirically verified that these design choices lead to high-quality 4D generation results; See teaser in Fig. 1.
In summary, our main contributions are:
We propose the first video-to-4D generation framework that directly produces dynamic 3D meshes. Our method extends the architecture of a 3D shape generation method. The model is fine-tuned, rather than used black-box and extended with separate network, like (GVFD) (Zhang et al., 2025), or combined with optimization, like V2M4 (Chen et al., 2025).
To extend an SOTA 3D generation model 4D generation, we propose several novel architec- tures and training innovations, including a spatio-temporal attention, redesigned surface point sampling, and noise sharing across frames.
RELATED WORK
bDS-based 4D generation. Score Distillation Sampling (SDS) (Poole et al., 2023) has emerged as a widely adopted approach for lifting 2D image diffusion models into 3D and 4D domains, with sub- sequent work advancing resolution, stability, and efficiency (Lin et al., 2023; Wang et al., 2023a;b; Tang et al., 2024b; Chen et al., 2023). A major research direction is the integration of heterogeneous diffusion priors (Qian et al., 2024; Shi et al., 2024), where text-to-image, multi-view, and video dif- fusion models are combined to promote both appearance fidelity and temporal coherence (Bahmani et al., 2024; Jiang et al., 2024b; Zhao et al., 2023; Jiang et al., 2024b). Other works (Zheng et al., 2024: Yuan et al., 2024; Ling et al., 2024; Li et al., 2024c; Yu et al., 2024) focus instead enhanc- ing the 4D representation itself, leveraging deformation fields, Gaussian splatting, and normal/depth priors to improve geometry quality and disentangle material properties. While SDDSbased methods
2

---

Published as a conference paper at ICLR 2026
Animated Asset
Pount Sampling
Subsample
Latcals
Meshes
Vidco Frannnes
Time
Noise
Eoc. SYV
Dec.
Antimate
Sampling
BuilStngle Sinai Black
Noise o
Velecity
Eoc.
Dec.
Time
Noisy Shape Latents
:
Animate
:
:
Sameling
Dec.
3D VAE with Temporally- Aligned Latenis
(b) Spatiotemporal Dittusion Translormer
Figure 2: ShapeGen4D employs flow-based latent diffusion transformer to generate sequence of meshes from an input video. (a) A 3D VAE encodes shapes into latents by cross-attending subsam- pled query points with a dense point cloud. To encode a sequence of animated assets, query points are subsampled from the first-frane frst-frame point cloud and then propagated through the animation to ob- tain query points for subsequent frames-yielding temporally-aligne
latents. The decoder maps these latents to signed distance fields, which are then converted into meshes via marching cubes. (b) The spatiotemporal diffusion transformer interleaves frozen dual/single-stres from the base 3D generative model, which process hidden states for each frame independently, with learnable spatiotemporal attention layers that capture cross- cross-frame frame dependencies and enforce tempo- ral consistency in the denoised latents.
transformer blocks
produce promising results, they remain computationally intensive and susceptible to artifacts such as Janus artifacts and oversaturation.
Multi- View diffusion then reconstruction. To bypass costly iterative optimization, recent 3D gen- cration works (Li et al., 2024b; Zhuang et al., 2025; Xu et al., 2024; Shi et al., 2024; Huang et al., 2025) adopt two-stage paradigm: first generating multi-view images, then reconstructing 3D rep- resentations with a feedforward model. Inspired by this, several recent 4D generation methods also follow two-stage design. A key challenge lies in building multi-view video diffusion models that can generate frames across both time and viewpoints simultaneously. This is difficult due to lim- ited training data. Early attempts (Zeng et al., 2024; Jiang et al., 2024b; Yang al., 2025; Sun et al., 2025a) adapt multi-view image diffusion in training-free manner to improve temporal con- sistency, but remain limited in robustness and the scale of motion they can handle. With advances in camera and motion control for video diffusion, works such as DimensionX (Sun et al., 2025b) and CAT4D (Wu et al., 2025a) show that multi-view videos can be generated via sophisticated pro- cedures that alternate diffusion steps across time and view axes. A more fundamental solution is to train models that directly generate synchronized multi-view video grids. Research progress has moved from dynamic objects (Zhang et al., 2024a; Jiang et al., 2024a; Li et al., 2024a; Xie et al., 2025; Yao et al., 2025) and humans (Shao et al., 2024) to general scenes, with notable works such as 4Real- (Wang et al., 2025b;a) and SynCamMaster (Bai et al., 2025). Despite these advances, the two-stage paradigm still suffers from accumulated errors due to inconsistencies in the generated video frames, making it difficult to reconstruct high-quality geometry.
Direct 3D generation. An alternative paradigm is to train generative models that directly produce 3D representations (Jun & Nichol, 2023). With the growing scale of 3D training datasets, diffusion- transformer-based direct 3D generators have emerged as the new state of the art. These models mainly differ in how they tokenize 3D shapes into diffusable latents. One line of work, following 3DShape2Vecthe VectSet (Zhang et al., 2023), encodes unstructured point clouds into sets of unordered latent vectors. Large-scale foundation models such as Hunyuan3D 2.0 (TencentHunyunn33 2025a;b), XteD (Li et al., 2025a), and others (Zhang et al., 2024b; Li et al., 2025b) build on this representation, achieving generalizable, high-quality geometry and texture with semantic- and UV- aware enhancements (Hunyuan3D, 2025). Another line of work leverages voxel grids and structured latents that decode into meshes. Gaussians, or radiance fields, as proposed by TRELLIS (Xiang al., 2025) and extended by follow- ups (Wu et al., 2025b; Li et al., 2025c). Among these models, StepIX-3D (Li et al., 2025a) places particular emphasis on fidelity and openness, making it a suitable foundation for our work.
Direct 4D generation. While current direct 3D generators emphasize scalability, flexibility, and ed- itability, extending them to direct 4D generation remains underexplored, largely due the scarcity
3

---

Published as a conference paper at ICLR 2026
of 4D data. L4GM (Ren et al., 2024), inspired by large multi-view Gaussian model (LGM) (Tang et al., 2024a), predicts multi-view images where each pixel represents a Gaussian particle condi- tioned on an input video. However, constrained by its non-diffusion backbone, small training set, and image-based representation, L4GM suffers from limited quality, generalizability, and relatively weak geometry. V2M4 (Chen et al., 2025) leverages per-frame meshes generated by state-of-the- art 3D models such as TRELLIS, but itself is not a direct method, relying instead on complex and fragile pipelines for mesh registration and geometry optimization. Concurrently, GVFD (Zhang et al., 2025) trains a diffusion model to generate deformation fields that drive Gaussian particles pre- generated by off-the-shelf 3D generators. Yet, due to limited 4D training data and restrictions on the possible deformations, learning deformation fields as a new modality yields weak generalization; in our experiments, GVFD generalized poorly. In contrast, our work deeply integrates with strong pretrained 3D generators to inherit their generalization ability, while leaving texturing to a simpler mesh registration step.
3 DYNAMIC MESH GENERATION
We present a flow-based latent diffusion model that generates mesh sequences capturing dynamic object motion, conditioned on a monocular video (see Fig. 2). Specifically, this involves extracting temporally-aligned latents by querying at the "same" surface location and introducing a spatiotem- poral transformer for processing the sequence of frames.
3.1 TEMPORALLY-ALIGNE LATENTS FOR DYNAMIC SHIPP VAE
A compact, expressive, and well-regularized latent space is essential for training diffusion mod- els (Skorokhodov et al., 2025). We build upon the VAE architecture adopted in recent 3D generative models, including Step1 X-3D (Li et al., 2025a) and Humyuan3D (TencentHunyuan33DD which encodes a sequence of meshes into a sequence of latent codes. The decoder then maps these latents into continuous geometric representations, ie.. truncated signed distance fields. We first review the original VAE in preliminary and then describe our extension in aligning latents.
2025a),
Preliminary. The VAE architecture adopt follows the SDShape2VectSet design (Zhang al., 2023), which has been utilized in several state-of-the-ar 3D generation models, including Step 3D (Li et al., 2025a) and Hunyuan3D TTencentHunyuannnDDe.
2025a).
Each mesh is first converted into a watertight form, and a dense set of points P is sampled from its surface. To create fixed-length length representation for the unstructured point cloud, the encoder cross-attends P with set of query points Q = FPS(P), obtained via farthest point sampling (FPS). The resulting tokens are then processed by self-attention layers produce latent codes. To better capture high-frequency details, P is augmented with salient points sampled along sharp edges, and each point concatenated with surface normal features.
Encoder:
The decoder first processes the latent codes through self-attention layers to extract shape features. For any 3D position ar, the truncated signed distance value is predicted by cross-attending the positional embedding of æ with the decoded shape features.
Decoder:
Aligning latents. Naively encoding a mesh sequence [M,,..., pendently produces temporally jittery latents. See Fig. 3 for illustration. As the sparse query set Q sampled independently from Pi at each frame t, this leads to inconsistent query points across time. In our preliminary experiments, we found that such jittered latents make it harder for the diffusion model to learn smooth temporal dynamics.
by encoding each frame inde-
To address this, we introduce temporal structure into the sequence of query sets (Qi..., 2r). In- stead of independent samples, we first sample Q1 from mesh M1, and then obtain subsequent sets by warping Q1 with the corresponding animation, i.e. Q, (Q1), where w, denotes the defor- mation at the t-th frame. This ensures temporal alignment: each latent sequence corresponds to the same physical point on the deforming surface. Empirically, we find that aligned latents substantially reduce jitter and improve diffusion training.
In more detail, we sample query points directly from the original non-wanne waterrgttt mesh. where the animation is defined, rather than from the processed watertight mesh. Otherwise, establishing correspondences requires costly mesh registration.
4

---

Published as a conference paper at ICLR 2026
K.V
K.V
Point Sumpling
Subsample
Point Sampligg
Subcample
Point Sampling
Antronne
Point Simpling
Animane
23
54
13
160
125
( w/ alignment
(b) wio aligament
(c) Lateni Analysis
Figure 3: Illustration of latents with and without aligning query points across frames in (a) and (b). In (c), we visualize the average normalized L2 difference between latents at the closest 3D positions across neighboring frames. We observe that with alignment, the L2 difference smaller, indicating that the latents are more consistent, i.e., less jittery compared to non-aligned latents.
3.2 SPATIOTEMPORAL DIFFUSION TRANSFORMER
We extend the pretrained diffusion transformer architecture used in StepIX-3D (Li et al., 2025a) and Hunyuan3D TenncentHunyuan3DTeam 2025a), originally designed to generate a single 3D latent shape from a single image, to instead produce a sequence of latents representing dynamic shapes inferred from video. The decision to leverage pretrained shape diffusion model is that was trained on a much larger dataset than the available 4D data. This provides a valuable source for achieving generalization. To adapt the 3D model the dynamic setting, introduce temporal extensions that enable consistent generation across time.
The pretrained rectified-flow diffusion transformer (Li et al., 2025a) takes a conditioning image embedded with D2 (Oquab et al., 2024), then applies a sequence of dual-stream and single- stream transformer blocks (BlackForestLabs, 2024). These blocks jointly attend to the hidden states of the image features and the noised shape latents. In the following, we propose two extensions.
Inserting spatiotemporal attention layers. Motivated by prior extensions of text-to-image diffu- sion models to video (Guo et al., 2023; Blattmann et al., 2023), we insert spatiotemporal transformer layers after each block of the pretrained model to capture temporal dependencies. These layers mir- ror the base model's 'ss single-stream blocks, but their self-attention operates jointly across shape and image hidden states from all frames. We also explored alternative variants, including attending only to shape hidden states and applying ID temporal attention that excludes fame-frame interactions. Empirically, both alternatives degraded generation quality. To encode frame indices, we add 1D RoPE embeddings (Su et al., 2024) to the hidden states. During training, only the spatiotemporal layers are updated, while the base model remains frozen to avoid forgetting, given the limited 4D data. Each spatiotemporal layer's output projection is zero-initialized stabilize training.
Sharing noise across frames. diffusion models, the additive Gaussian noise is sampled inde- pendently. However, in our setup, independent noise across frames leads unstable motion. We suspect that this instability occurs because the base model is trained to generate 3D shapes without regard the viewpoint. As result, different noise samples push the model toward different poses and scales, causing visible flickering between frames.
In contrast, image and video diffusion models work on strututructured data where each hidden state has explicit positional embeddings. These embeddings give attention layers exact spatiotemporal coordinates, so independent noise can be used per frame without breaking temporal consistency. By comparison, VDShape2VectSet-ssy/l VectSet-style models operate on irregular structures without explicit po- sition embeddings. They must infer position implicitly, which makes them more sensitive to vari- ations in the noise. Inspired by practices in early image-to-vides diffusion (Ge et al., 2023) and video-to- toteo- editing (Burgert et al., 2025), we enforce temporal correspondence by replicating the same noise across frames during both training and inference. This simple yet effective strategy substantially improves temporal smoothness, yielding more consistently aligned shapes even before additional training. A visual comparison is provided in Fig. 4.
5

---

Published as a conference paper at ICLR 2026
Before Training
After 4D Training
Input
w/o Shared Noise
w/ Shared Noise
w/ Shared Noise
w/ Shared Noise
Figure 4: Qualitative comparison of noise sharing. The base 3D model generates object shapes in arbitrary orientations agnostic to the input image viewpoint, often causing pose changes across sequence (e.g. the hippo in the first row). We observe that sharing noise across frames reduces flickering and further improves shape quality in challenging cases such as the flag example.
MESH REGISTRATION AND TEXTURIZATION
Once the sequence of meshes generated, we apply a two-stage pipeline to ensure alignment with the input video and produce consistent textures. This process consists of global pose registration and global texturization.
Global pose registration. Our mesh generation network is trained on animated objects in a canon- ical coordinate system, i.e., with object geometries aligned to the coordinate axes. Consequently. the generated 4D geometry may match the pose observed in the input video if the latter is not canonicalized. To address this, we adopt the pose registration strategy from V2M4 (Chen et al., 2025) to re-pose the generated 4D geometry so that aligns with the input image. Owing to the stability and temporal consistency of our generated 4D geometry, we only need to estimate the pose for the first frame and then apply the same transformation globally to all subsequent frames, which achieves effective alignment with the input video.
Global texturization. A straightforward approach to texturizing 4D geometry is to apply off-the- shelf 3D mesh texturization methods H 2025a; Li et al., 2025a) indepen- dently to each frame. However, this often leads to inconsistency across frames due to the random- ness in texture generation, especially in occluded unseen regions such as the object's backside. To overcome this, we employ the pairwise mesh registration strategy by Chen et al. (2025) to con- vert 4D geometry into copology-consisee 4D mesh. We then apply off-the-shelf texturization methods to the first-frame mesh and propagate the resulting texture across all frames by leveraging the consistent topology.
Further details of the texturization steps are provided in the Appendix.
EXPERIMENTS
5.1 IMPLEMENTATION DETAILS
Data curation. We curated a set of 14k high-quality animated 3D assets from Objaverse. Since model builds on StepIX-3D (Li et al., 2025a) and Hunyuan3D-2.1 (TencentHunnyuannDDTeam 2025a), we process the data to match its distribution. Each mesh converted into watertight version to remove internal surfaces. Root-body motion is removed from the animation, and objects

---

Published as a conference paper at ICLR 2026
Table I: Quantitative comparison with baselines, evaluated the held-out Objaverse test set.
Representaion
Feedforward
Chamfer]
IoUt
F-Score 1
Time 4
X-3D (Li al., 2025a)
SDF SDF
0.1356
0.3033
0.2617
min
StepIX-3D shared noise
/
0.1368
0.3149
0.2817
min
DreamMesh4D (Li et al., 2024c)
mesh MV-3D GS mesh deform 3D GS + deform SDF SDF
0.2108
0.2422
40 min
L4GM (Ren et al., 2024)
0.1576
0.1932
25
V2M4 (Chen et al., 2025)
0.1233
0.3023
0.2814
30 min
GVFD (Zhang et al., 2025)
0.3978
0.0699
10 min
ShapeGen4D (StepIX-3D)
0.1220
0.3276
0.2934
3 min
ShapeGen4D (Hunyuan3D-2.11
0.0827
0.4155
0.3971
15 min
rescaled that all shapes in sequence are centered within a unit bounding box. We retain the original non-watertight mesh, where the animation defined, to extract aligned query points for encoding, but filter out inner vertices based their projected distance to the watertight mesh. For each sample, fixed-view videos are rendered from randomly sampled azimuth angles. To ensure consistent object size, videos are rendered with a channel and rescaled that the object occupies a fixed portion of the image.
Training details. The diffusion model is trained to generate 16 frames, each represented by 1024 latents. For every frame, we sample 32k-point cloud from the watertight mesh as input to the en- coder, paired with 1024 query points from the animated non-watertight mesh. Training is conducted with a batch size of 64 across 16 A 100 GPUs, using a learning rate of 5 x 10-5, We train for 25k iterations 2 days) after which the validation loss indicates negligible quality improvement.
5.2 ANALYSIS AND COMPARISONS
Test datasets. We collect two evaluation sets: (1) a held-out set of 33 animated samples from Ob- javerse, each with significant object motion and high-quality textures, used to evaluate geometric accuracy across methods; and (2) 20 video sequences from Consistent4D (Jiang et al., 2024b), cov- ering both in-the-wild and synthetic subjects. Since ground-truth geometry unavailable in the latter, evaluate using rendering-basee metrics instead.
Baselines. We conduct a comparison with state-of-the-art feedforward 4D generation methods, Le.. L4GM (Ren et al., 2024) and GVFD (Zhang et al., 2025), both of which output Gaussian particles. For geometric evaluation, we threshold particle densities to retain only those near the surface and treat the particle centers points. Since the resulting point clouds exhibit noisy surface artifacts, converting them into watertight meshes for loU evaluation is technically challenging and therefore omitted. As reference baseline, also evaluate StepIX-31 applied independently to each frame.
Metrics. Geometric metrics: Since StepIX-33 and GVFD do not generate shapes aligned with the input video frames, pose registration required for comparison with ground-truth geometry. We first estimate a global similarity transformation from the first frame of each sequence by grid-searching over azimuth angles, scaling, and translation (based on truncated Chamfer distance). followed by iterative closest point (ICP). This transformation is applied to the full sequence, with additional per-frame translations to ensure alignment while preserving original frame-wise rotation and scale changes from the prediction, allowing temporal consistency to be properly evaluated. We then follow Zhang et al. (2023) to report Chamfer distance and F-Score between point clouds, and IoU on occupancy voxel grids at 128 resolution.
Rendering metrics: We render the textured shapes from the target camera view and evaluate with rendering metrics, including LPIPS (Zhang et al., 2018) and DreamSim (Fu et al., 2023) for percep- tual similarity, CLIP score (Radford et al., 2021) for conceptual alignment between text and rendered images, and FVD (Unterthiner et al., 2019) for video quality.
Quantitative comparison. On geometry: As shown in Tab. 1, our method achieves more precise geometry compared to the baselines as indicated by the consistent gain across all three metrics.
On rendering: We show results in Tab. 2 on the Consistent4D dataset. Our method produces more consistent results than other generative methods, StepIX-3D and GVFD, yielding noticeably better renderings. Interestingly, L4GM still achieves higher scores, despite its geometry being qualitatively worse than ours. This advantage comes from its strong bias towards reconstructing the input views
7

---

Published as a conference paper at ICLR 2026
Table 2: Quantitative comparison of rendering qual- ity. L4GM has unique ad vantage because its pre- dictions are inherently aligned with the input im- age views, whereas other methods are not. Refer
Table 3: Ablation study by removing one component at a time.
Chamfer
loUtt
F-Score
w/o aligned latents
0.1348
0.3230
0.3002
to Fig. in the Appendix for visual comparisons.
w/o shared noise
0.1186
0.3137
0,2962
Aligned LPIPS CLIPt FVD. DreamSim)
temp. atin.
0.2118
0.1503
0.1462
w/o image hidden states
0.1196
0.3332
0.3084
X-3D
x 0.1524
0.9040
940
0,1106
w/o time shift
0.1374
0.3087
0.2861
L4GM
V
0.0988
0.9397
302
0,0487
Full method
0,1096
0,3346
0.3190
GVFD
x
0.1691
0.8601
916
0.1467
Ours
0.1359
0,9009
796
0.0966
at the expense of generating meaningful 4D shapes that also look plausible in other views. L4GM's outputs align closely with input viewpoints, reducing all metrics (LPIPS, DreamSim, CLIP, FVD). In contrast, X-3D, GVFD, and our method are visadvantaged by camera-coordinate misalignment and the fact that they balance reconstruction of input views with generating meaningful 4D shapes that are also valid in other views.
Qualitative comparison. Fig. 5 and Fig. 6 further shows the limitation of baseline approaches. L4GM suffers from two main issues: (1) its multi-view image- baage representation yields Gaussian particles that do not seamlessly merge, leading to imperfect geometry and ghosting artifacts in ren- derings; and (2) due to its limited model scale, it often produces incorrect shapes on challenging test cases. GVFD, on the other hand, generates jittery motions where objects distort significantly and fails to capture fine surface dynamics, such as the motion of a flag. StepIX-311 applied indepen- dently to each frame produces reasonable per-frame geometry but lacks temporal consistency, with poses drifting across frames. Using shared noise per sequence with StepIX-3D reduces global pose jitter but still results in noticeable shape fluctuations. In contrast, our method produces high- meshes, maintains consistent poses, and exhibits substantially less temporal jitter.
Ablations. We analyze each design choice through controlled ablation experiments by removing one component at a time from the full method. To reduce training cost, we use frames instead of 16. Results on our held-out Objaverse dataset are reported in Tab. 3.
Aligned latents: We find that learning with aligned latents, obtained by using query points from the animated original wot-tttttt meshes, plays central role. Replacing them with independently sampled query points per frame leads to quality degradation and more flickering.
Shared noise: We find that applying shared noise across frames noticeably reduces flickering and shape artifacts compared to independent noise. Visual comparisons are shown in Fig. 4. Without noise sharing, StepIX-3D (before training) produces random orientations of the hippo; even after training with spatiotemporal attention, often yields inconsistent poses. By sharing noise, our model resolves this issue. Furthermore, observe that training with shared noise can also improve geometry in challenging cases, such as the flag example, compared to baselines.
Spatiotemporal attention architecture: We explore alternative architectural designs for spa- tiotemporal attention. (1) Instead of applying self-attention across all hidden states, we restrict it to ID temporal attention: each hidden state only attends to the sequence of the same query point, with- out attending to other states within the same frame. While this design is common in video generative models (Guo et al., 2023), we find it leads to catastrophic failure. We hypothesize that, unlike image or video models where hidden states are explicitly embedded with spatial coordinates, our latent representation lacks such information; thus, intra-frame attention necessary for hidden states to infer their spatial positions. (2) We also remove the image hidden states from the spatiotemporal attention layer and observe a performance drop, indicating that cross-frame dependencies among image features contribute positively.
Denoise time shift: With more latents and shared noise, the 4D diffusion transformer faces reduced prediction difficulty at the same noise level compared to 3D generation. Similar to strategies used in training multi-resolution image diffusion models (Esser et al., 2024), we apply a time shift to the denoising scheduler to allocate more steps at mid-to-high noise levels. We find that applying time shift during denoising significantly improves the stability of our results, though training with a time shift has little effect.
8

---

Published as a conference paper at ICLR 2026
Ingut Video
ShapeGenlD (Hunyyum)
ShapeGoodD
V2M4
DreamMesh4D
Step1X-3D SteD
shared cuise
L4GM
GVED
Time
Time 2
Time
Time 2
Time
Time
Figure 5: Qualitative comparison with baselines the held-out Objaverse test set.
CONCLUSION
We propose feedforward model that generates sequence of temporally consistent 3D shapes from an input video. Our model builds on state-of-the-art 3D generation methods, extending them with spatiotemporal attention layers and careful design choices that preserve strong priors while enabling training with relatively small 4D datasets. Compared to recent baselines such as L4GM and GVFD, our approach demonstrates stronger generalization and produces higher-quality geometry than their 3D Gaussian-splattnng representations.
Nonetheless, our method can be further improved with the following limitations in mind: (a) Inher- ited from the base 3D generation model, our framework is agnostic to the viewpoint of input video frames. As a result, struggles to capture global motions such as rigid object rotations. Viewpoint- aligned reconstruction requires additional pose registration, which might be mitigated by finetuning with paired, viewpoint-aligned images and 3D shapes.
(b) create fully animatable textured as- sets, additional steps such as pose registration and texture propagation still necessary. (c) Local temporal jitter remains visible some results. Following lessons from video diffusion, we believe spatiotemporal 3D VAE may help reduce flickering.
ACKNOWLEDGMENTS
We thank Avalon Vinella for assistance in preparing the Objaverse dataset. We are also grateful to Willi Menapace for his advice on training flow matching. Finally, we thank Gleb Dmukhin, Hao Zhang, Michael Vasilkovsky, Sergei Korolev, and Vladislav Shakhray for their valuable discussions and support.

---

Published as a conference paper at ICLR 2026
REFERENCES
Sherwin Bahmani, Ivan Skorokhodov, Victor Rong. Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, and David B. Lindell. 4D-fy: Text-to-4D generation using hybrid score distillation sampling. In Proc. CVPR, 2024. 2
Jianhong Bai, Menghan Xia, Xintao Wang, Ziyang Yuan, Xiao Fu, Zuozhu Liu, Haoji Hu, Pengfei Wan, and Di Zhang. Syncammaster: Synchronizing multi-camera video generation from diverse viewpoints, In Proc. ICLR, 2025. 3
Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. MultiDiffusion: Fusing diffusion paths for controlled image generation. In Proc. ICML,, 2023. 15
BlackForestLabs, FLUX. https: : /tt//ggtttuu www.o///lak--oe Baeaack--orrstte
fflx, 2024. 1,
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable video diffusion: Scaling latent video diffusion models to large datasets.
arXiv:2311.15127, 2023. 5
Ryan Burgert, Yuancheng Xu, Wenqi Xian, Oliver Pilarski, Pascal Clausen, Mingming He, Li Ma, Yitong Deng, Lingxiao Li, Mohsen Mousavi, Michael Ryoo, Paul Debevec, and Ning Yu. Go- With-the-Flowe Motion-controllable video diffusion models using real-time warped noise. In Proc. CVPR, 2025.
Jianqi Chen, Biao Zhang, Xiangjun Tang, and Peter Wonka. V2M4: 4D mesh animation reconstruc- tion from single monocular video. In Proc. ICCV, 2025. 4, 6, 7, 14
Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fantasia3D: Disentangling geometry and appearance for high-quality text-to-3ID content creation. In Proc. ICCV, 2023. 2
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Osear Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of anno- tated 3D objects. In Proc. CVPR, 2023.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion En- glish, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. Proc. ICML, 2024. 8
Stephanie Fu, Netanel Tamir. Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. DreamSim: Learning new dimensions of human visual similarity using synthetic data. In Proc. NeurlPS, 2023. 7, 14
songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming- Yu Liu, and Yogesh Balaji. Preserve your own correlation: A noise prior for video diffusion models. In Proc. ICCV, 2023. 5
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. AnimateDiff: Animate your personalized text-to-image diffu- sion models without specific tuning. In Proc. ICLR, 2023. 5, 8
Zehuan Huang, Yuanchen Guo, Haoran Wang, Ran Yi, Lizhuang Ma. Yan-Pei Cao, and Lu Sheng. MV-Adapter: Multi-view consistent image generation made easy. In Proc. ICCV, 2025.
Tencent Hunyuan3D. Hunyuan3D Studio: End-to-end Al pipeline for game-ready 3D asset genera- tion. arXiv:2509. /2815, 2025. 3
Yanqin Jiang, Chaohui Yu, Chenjie Cao, Fan Wang, Weiming Hu, and Jin Gao. Animate3D; Ani- mating any 3D model with multi-view video diffusion. Proc. NeurIPS, 2024a.
Yanqin Jiang, Li Zhang, Jin Gao, Weiming Hu, and Yao Yao. Consistent4D: Consistent 360 dy- namic object generation from monocular video. In Proc. ICLR, 2024b. 2, 3,
10

---

Published as a conference paper at ICLR 2026
Heewoo Jun and Alex Nichol. arXiv:230S.02463, 2023. 3
Shap-E:
Generating conditional 3D implicit functions.
Bing Li, Cheng Zheng, Wenxuan Zhu, Jinjie Mai, Biao Zhang, Peter Wonka, and Berard Ghanem. Vivid-ZOO: Mulw video generation with diffusion model. arXiv:2406.08655 2024a. 3
Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant31D: Fast text-to-3D with sparse-view gener- ation and large reconstruction model. In Proc. ICLR, 2024b.
Weiyu Li, Xuanyang Zhang. Zheng Sun, Di Qi, Hao Li, Wei Cheng, Weiwei Cai, Shihao Wu, Jiarui Liu, Zihao Wang, et al. XteplX-3D: Towards high-fidelity and controllable generation of textured 3d assets. arXiv:2505.07747, 2025a. 2, 4, 5, 6,
Yangguang Li, Zi-Xin Zou, Zexiang Liu, Dehu Wang, Yuan Liang, Zhipeng Yu, Xingchao Liu, Yuan-Chen Guo, Ding Liang, Wanli Ouyang, et al. TripoSG: High-fidelity shape synthesis using large-scale rectified flow models. arXiv:2502.06666 2025b. 3
Zhihao Li, Yufei Wang, Heliang Zheng, Yihao Luo, and Bihan Wen. Sparc3D: Sparse representation and construction for high-resolution 3D shapes modeling. arXiv:2505. 14521, 2025c. 3
Zhiqi Li. Yiming Chen, and Peidong Liu. Dreammesh4D: Video-to-4D generation with sparse- controlled gaussian- hybrid representation. In Proc. NeurIPS, 2024c. 2,
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming- Yu Liu, and Tsung- Lin. Magic3D: High-resolution text-to-3D con- tent creation. In Proc. CVPR, 2023. 2
Huan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fidler, and Karsten Kreis. Align your gaussians: Text-to-4D with dynamic 3D gaussians and composed diffusion models. In Proc. CVPR, 2024. 2
Baptiste Nicolet, Alec Jacobson, and Wenzel Jakob. Large steps inverse rendering of geometry. ACM Transactions Graphics, 2021. 15
Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Po-Yao Huang. Hu Xu, Vasu Sharma, Shang- Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nico- las Ballas, Gabriel Synnaeve, Ishan Misra, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features without supervision. TMLR, 2024.
Ben Poole. Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. DreamFusion: Text-to-3D using 2D diffusion. Proc. ICLR, 2023. 1,2
Guocheng Qian, Jinjie Mai, Abdullah Hamdi, Jian Ren, Aliaksandr Siarohin, Bing Li, Hsin-Yin Ying Lee, Ivan Skorokhodov, Peter Wonka, Sergey Tulyakov, and Bernard Ghanem. Magic123: One image high-quality 3D object generation using both 2D and 3D diffusion priors. In Proc. ICLR, 2024.7 2
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. In Proc. ICML, 2021. 7
Jiawei Ren, Kevin Xie, Ashkan Mirzaei, Hanxue Liang, Xiaohui Zeng, Karsten Kreis, Ziwei Liu, Antonio Torralba, Sanja Fidler, Seung Wook Kim, and Huan Ling. L4GM: Large 4D gaussian reconstruction model. In Proc. NeurIPS, 2024. 4,7
Ruizhi Shao, Youxin Pang, Zerong Zheng, Jingxiang Sun, and Yebin Liu. 360-degree human video generation with 4D diffusion transformer. ACM Transactions on Graphics, 2024. 3

---

Published as a conference paper at ICLR 2026
Yichun Shi, Peng Wang, Jianglong Ye, Long Mai, Kejie Li, and Xiao Yang. MVDream: Multi- view diffusion for 3D generation. Proc. ICLR, 2024. URL https::/
Eorum?id=FUgr j42pbB. 2,
Uriel Singer, Shelly Sheynin, Adam Polyak, Oron Ashual, Iurii Makarov, Filippos Kokkinos, Naman Goyal, Andrea Vedaldi, Devi Parikh, Justin Johnson, and Yaniv Taigman. Text-to-4D dynamic scene generation. In Proc. ICML, 2023. 1
Ivan Skorokhodov, Sharath Girish, Benran Hu, Willi Menapace, Yanyu Li, Rameen Abdal, Sergey Tulyakov, and Aliaksandr Siarohin. Improving the diffusability autoencoders. In Proc. ICML, 2025. 4
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: En- hanced transformer with rotary position embedding. Neurocomputing, 2024. 5
Qi Sun, Zhiyang Guo, Ziyu Wan, Jing Nathan Yan, Shengming Yin, Wengang Zhou, Jing Liao, and Houqiang Li. EG4D: Explicit generation of 4D object without score distillation. In Proc. ICLR, 2025a. 3
Wenqiang Sun, Shuo Chen, Fangfu Liu, Zilong Chen, Yueqi Duan, Jun Zhang, and Yikai Wang. DimensionX: Create any 3D and 4D scenes from single image with controllable video diffusion. In Proc. ICCV, 2025b.
Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. LGM: Large multi-view gaussian model for high-resolution 3D content creation. In Proc. ECCV, 2024a.
Jaaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. DreamGaussian: Generative gaussian splatting for efficient 3D content creation. In Proc. ICIR, 2024b. 2
Wan Team. Wan: Open and advanced large-scale video generative models. arXiv:2503.20314. 2025.
TensentHunyuannDDraam Hunyuan3D 2.0: Scaling diffuusion models for high resolution textured 3D assets generation. https: //tp:i//aivvvv/// ottppp/f//51..12222222 2025a. 1,2, 4, 5,
TencentHlunyann3DTeal Hunyuan3D 2.5: Towards high-fidelity 3D assets generation with ulti- mate details. https:// http::://axxxvv.rrrr abs 22002 16504, 2025b. 3
Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. FVD: A new metric for video generation. ICLRW, 2019.
Chaoyang Wang. Ashkan Mirzaei, Vidit Goel, Willi Menapace, Aliaksandr Siarohin, Avalon Vinella, Michael Vasilkovsky, Ivan Skorokhodov. Vladislav Shakhrai, Sergey Korolev, Sergey Tulyakov, and Peter Wonka. 4Real-Video-V2. Fused view-time attention and feedforward reconstruction for 4D scene generation. In Proc. NeurIPS, 2025a. 1,3
Chaoyang Wang, Peiye Zhuang, Tuan Duc Ngo, Willi Menapace, Aliaksandr Siarohin. Michael Vasilkovsky, Ivan Skorokhodov. Sergey Tulyakov, Peter Wonka, and Hsin-Yine Ying Lee. 4Real- Video: Learning generalizable photo-realistic 4D video diffusion. In Proc. CVPR, 2025b. 1,
Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2D diffusion models for 3D generation. In Proc. CVPR, 2023a. 2
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. VGGT: Visual geometry grounded transformer. In Proc. CVPR, 2025c. 14
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolific- Dreamer: High-fidelity and diverse text-to-3I) generation with variational score distillation. In Proc. NeurIPS, 2023b. 2
Rundi Wu, Ruiqi Gao, Ben Poole, Alex Trevithick, Changxi Zheng, Jonathan T. Barron, and Alek- sander Holynski. CAT4D: Create anything in 4D with multi-view video diffusion models. In Proc. CVPR, 2025a. 1,3
12

---

Published as a conference paper at ICLR 2026
Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Yikang Yang, Yajie Bao, Jiachen Qian, Siyu Zhu, Philip Torr, Xun Cao, and Yao Yao. Direct3D-S2: Gigascale 3D generation made easy with spatial sparse attention. In Proc. NeurIPS, 2025b. 3
Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3D latents for scalable and versatile 3D generation. In Proc. CVPR, 2025. 3,3
Yiming Xie, Chun- Yao, Vikram Voleti, Huaizu Jiang, and Varun Jampani. SV4D: Dynamic 3d content generation with multi-frame and multi-view consistency. In Proc. ICLR, 2025. 1,
Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. InstantMesh: Efficient 3D mesh generation from a single image with sparse-view large reconstruction models. 000::2404.07911, 2024. 3
Zeyu Yang, Zijie Pan, Chun Gu, and Li Zhang. Diffusion": Dynamic 3D content generation via score composition of video and multi-view diffusion models. In Proc. ICLR, 2025. 3
Chun-Han Yao, Yiming Xie, Vikram Voleti, Huaizu Jiang, and Varun Jampani. SV4D2.0: Enhancing spatio-temporal consistency in multi-view video diffusion for high-quality 4D generation. arXiv preprint 1.X.i:25033.63366 2025. 3
Heng Yu, Chaoyang Wang, Peiye Zhuang, Willi Menapace, Aliaksandr Siarohin, Junli Cao, László Jeni, Sergey Tulyakov, and Biin Lee. 4Real: Towards photorealistic 4D scene generation via video diffusion models. In Proc. NewrlPS, 2024. 2
Ye Yuan, Xueting Li, Yangyi Huang, Shalini De Mello, Koki Nagano, Jan Kautz, and Umar Iqbal. GAvatar: Animatable 3D gaussian avatars with implicit mesh learning. In Proc. CVPR, 2024. 2
Yifei Zeng, Yanqin Jiang, Siyu Zhu, Yuanxun Lu, Youtian Lin, Hao Zhu, Weiming Hu, Xun Cao, and Yao Yao. STAG4D: Spatial-temporal anchored generative 4D gaussians. In Proc. ECCV, 2024. 3
Biao Zhang, Jiapeng Tang, Matthias Nießner, and Peter Wonka. 3DShape2VecSet. A 3D shape representation for neural fields and generative diffusion models. ACM Transactions on Graphics, 2023. 3, 7,
Bowen Zhang, Sicheng Xu, Chuxin Wang, Jiaolong Yang, Feng Zhao, Dong Chen, and Baining Guo. Gaussian variation field diffusion for high-fidelity video-to-4D synthesis. In Proc. ICCV, 2025. 2, 7,7
Haiyu Zhang, Xinyuan Chen, Yaohui Wang. Xihui Liu, Yunhong Wang, and Yu Qiao. 4Diffusion: Multi-view video diffusion model for 4D generation. In Proc. NeurlPS, 2024a. 1,3
Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang. Haoran Jiang, Wei Yang. Lan Xu, and Jingyi Yu. CLAY: A controllable large-scale generative model for creating high-quality 3D assets. ACM Transactions on Graphics, 2024b.
Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In Proc. CVPR, 2018. 7
Yuyang Zhao, Zhiwen Yan, Enze Xie, Lanqing Hong, Zhenguo Li, and Gim Hee Lee. Animate 124: Animating one image to 4D dynamic scene. arXiv:231 146003, 2023. 2
Yufeng Zheng, Xueting Li, Koki Nagano, Sifei Liu, Otmar Hilliges, and Shalini De Mello. A unified approach for text-and image-guided 4D scene generation. In Proc. CVPR, 2024. 2
Peiye Zhuang, Songfang Han, Chaoyang Wang, Aliaksandr Siarohin, Jiaxu Zou, Michael Vasilkovsky, Vladislav Shakhrai, Sergey Korolev, Sergey Tulyakov, and Hsin- King Lee. GTR: Improving large 3D reconstruction models through geometry and texture refinement. In Proc. ICLR, 2025. 3
13

---

Published as a conference paper at ICLR 2026
APPENDIX
A MESH REGISTRATION AND TEXTURIZATION DETAILS
In this section, we present the detailed implementations of mesh registration and texturization.
Let the input video sequence of length T be denoted as Vie {Ved,, and let the generated untextured 4D geometry Minit (M, M...M...,MMy.. Mr). We align these meshes to the poses in the reference video and subsequently apply texturization through two-step process: Global Pose Registration and Global Texturization, inspired by V2M4 (Chen et al., 2025).
A.1 GLOBAL POSE REGISTRATION
To align the generated geometry with the video input, we reformulate the task from mesh pose estimation camera pose estimation. The key idea is that if determine the camera pose under which a rendered view is semantically similar to the corresponding video frame, then by inverting this pose and applying the transformation to the geometry, we can register the mesh the input video pose.
We parameterize the camera pose C R {yaw, pitch, radius, lookat for each untex- tured mesh MM. The camera pose estimation procedure consists of three stages:
1) Candidate sampling. We first sample a large number of candidate camera positions around the object and select the top n positions that yield the highest semantic simisartyymmasuure by DreamSim (Fu al., 2023) to the reference video frame Viett-
2) Pose prediction via VGGT. We then employ VGGT (Wang et al., 2025c) to process the rendered views from these candidate cameras together with the reference frame Viet, obtaining predicted point clouds for each view. Let PCref and (I...n. denote the predicted point clouds for Viet, and (I....(......... respectively. Since the camera models, poses, and mesh Mi used for rendering Vrend. (e.... are known, we can compute the corresponding ground-trum point clouds in our coor- dinate system, denoted as PCrend, using a global similarity transformation (rotation, translation, and scale) optimized via the Chamfer distance. Applying this transformation to PC yields PC,er. Ideally, under the correct camera pose Gre,s. the 3D coordinates of PCrer should project consistently onto the 2D pixel coordinates of Viet.- We therefore optimize the camera extrinsic parameters enforce this projection, resulting in the initial estimate VGGT.
We align the predicted point clouds with the ground truth
This estimated camera pose is then added to the pool of candidate poses obtained in step 1. We further refine it using Particle Swarm Optimization (PSO), which searches for a more accurate pose Cro by comparing the rendered images with the reference frame.
3) Pose refinement via gradient descent. Starting from Cref, we further refine the camera pose through gradient-based optimization. Specifically, we minimize the discrepancy between the mask of the rendered view dee-ned as the pixel region occupied by the object during rasterization-ann and the foreground region Viet.- This yields the final camera pose estimate Cre,.--
Once Creft is determined, we compute the corresponding camera motion, invert it, and apply the transformation to Mr, obtaining the pose-registered mesh Mrepose.t
A.2 GLOBAL TEXTURIZATION
As discussed the main paper, independently texturizing each frame's ' geometry leads to appear- ance inconsistencies due to topology variations across frames. To address this, we first convert the 4D geometry into a topology-consister consistent representation and then perform texturization.
1) Topology-consiseent geometry conversion. For each pair of consecutive meshes along the time dimension, M, and M₁-1, we treat M, as rigid body. We first optimize its global transformation (rotation, translation, and scale) to align it with M using a combination Chamfer distance and differentiable rendering loss. After achieving global registration, we further refine the alignment
14

---

Published as a conference paper at ICLR 2026
Method
Chamfer
IoUt
Method
Chamfer
loUt
F-Scoref
KVQ (watertight) 0.0246
0.8680
Baseline
0.0929 0.0934
0.3739
0.3557
Q (original)
0.0369
0.8401
MultiDiftusion
0.3779
0.3499
Table 4: Watertight vs. original query points.
Table 5: Baseline vs. MultiDiffusion.
using the preconditioned optimization method of Nicolet al. (2021), which enables fast and stable convergence during the inverse reconstruction process. To enhance local registration, we also apply As-Rigid-As-Possibbe (ARAP) constraint conjunction with the Chamfer distance.
Through these global and local alignment steps, we obtain a deformed version Mí, with vertices V and faces Fy, that shares identical topology (V) Vi, Fi = Fi) while closely matching the shape of M We then replace the original M with the deformed mesh Mi. In practice, we start at M1 and iteratively apply this registration process across consecutive frames, yielding sequence of hopology-consistee meshes M ((,....TT
2) Texturization. After obtaining apploogy-oonisste geometry, first apply existing texture generation method texturize the initial mesh M The resulting texture map and UV coordinates are then directly reused for subsequent meshes M (?. ensuring appearance consistency across the entire sequence.
B ADDITIONAL RESULTS
Fig. 6 visualizes the colored reconstructions from each method. Our approach produces the most temporally consistent shapes and captures fine-grained motion details from the input images. StepIX-3D can capture motion but suffers from jitter and random object poses. GVFD often fails to model motion, highlighting its limited ability to generalize deformations. L4GM generates high- quality results in some cases (e.g.. cats), but fails catastrophically on challenging examples such as water splashes, where misfused 3D Gaussian splats lead to ghosting artifacts.
C TOPOLOGY CHANGE RESULTS
Figure shows results topology changes which are hardly modeled by using mesh and defor- mation field. Each group shows three time steps (rows) for one sequence, with the input images on the left and our generated meshes rendered parts (e.g., the appearance of a gun barrel on a pistol), object fusion and splitting (c.g.. an egg merg- ing into a cup), tearing events (e.g., shirt ripping), morphing (e.g., Lego brick transforming into a cat), growing objects (e.g., a flower blooming), and object shattering (e.g., the explosion of gift box).
the right. The examples
follows: emerging
D VAE RESULTS
Both Step1 X-3D and the Hunyuan VAE are trained on watertight meshes. A concern that sampling query points directly the original meshes might degrade performance introduce artifacts. We evaluate this in Table 4 where we can see marginal degradation in reconstruction quality compared to sampling points from the watertight mesh. Upon visual inspection, there no noticeable difference in the reconstructed surfaces as seen in Fig 8. We hypothesize that this discrepancy may be due to small surface offsets introduced by the different sources of query points.
X-3D
E MULTIDIFFUSIO RESULTS
Here, we show how to extend our method to longer videos by leveraging MultiDiffusion (Bar-Tal al,, 2023) techniques from the long video generation literature. MultiDiffusion [1], a commonly
15

---

Published as a conference paper at ICLR 2026
Input
L4GM
GVFD
Step1X-31 w/o Shared Noise
Step Meep1x X-3D w/ Shared Noise
Ours w/ Step X-3D) Texturing
Figure 6: Qualitative comparison on Consistent4D test set.
16

---

Published as a conference paper at ICLR 2026
Time 1
Time 2
Time 3
Time
Time 2
Time 3
Figure 7: Topology Change Results. Our method can handle topology changes: emerging parts, tearing, growth, fusion and splitting, morphing, and object shattering. Each column shows the input view (left) and our reconstructed mesh (right) across three time steps.
used technique for panoramic image long video generation. The method applies the 16-frame diffusion model in diding-window manner: at each diffusion step, noise predictions are computed for all overlapping windows in parallel, and the predictions in the overlapping regions are fused by averaging. We note that MultiDiffusion is theoretically suboptimal, information propagation between distant temporal windows is inefficient and may result in reduced global consistency. In Table 5, we see small drop in performance when using MultiDiffusion. Visual results can be seen Figure 9.
17

---

Published as a conference paper at ICLR 2026
Input
KQV from Watertight
Q from original
Input
KQV from Watertight
from original
Time I
Time 2
Time
Time 2
Time
Time 2
Figure 8: Qualitative comparison of VAE reconstructions using query points from the watertight mesh versus the original mesh. The first column shows the input point clouds, while the second and third columns show the reconstructed surfaces when using KQV from the watertight mesh and Q directly from the original mesh. Both settings produce visually similar geometry across time.
18

---

Published as a conference paper at ICLR 2026
Time 0
Time
Time 12
Time
Time 24
Time 30
Baseline
MultiDiffusion
Figure 9: Qualitative comparison of our baseline vs MultiDiffusion. Each column is time step sam- pled from each sequence. The baseline is evaluated on frames 0-16, while MultiDiffusion extends the sequence to frames 0-32 using overlapping windows. Despite being trained only on 16-frame videos, it produces visually coherence and temporally consistent geometry over the longer sequence.
19


Reviews:
[{"review_id": "dY3dzoIOKB", "reviewer": "Reviewer_uhdb", "summary": "The paper works on 4d shape generation from monocular video. This is a new task and the authors aim to generate per-frame yet temporally coherence mesh instead of static mesh and deformation field to accommodates variantions in topology and relaxe constrains on the type of possible animations. Specifically, they construct the 4d shape generation model on a 3d shape generatoin work, and add temporal attention layers to ensure temporal consistency. Besides, they use global pose registration and global texturation as post-processing steps to better present the generated results and evaluation. Experiments on public and collected datasets validate the performance of the proposed model.", "questions": "It is unclear why the proposed method does not incorporate data augmentation using geometric or spatial transformations during training, which might have alleviated or removed the requirement for global registration.\nIf my concerns are solved, I'll raise the recommendation.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1) the generated shape seems to be of high-quality in terms of single-frame results, probably benifiting from high-quality pretrained 3D generation model weight\n\n2) the paper is well-written and present its contribution in a clear way\n\n3 )the authors work an new task, and propose a new pipeline of 4D shape generation", "weaknesses": "1) The generated results show noticeable flickering artifacts in both geometry and texture, where the texture flickering may stem from the instability in geometry.\nWPOwpo\n2)  While the authors claim that per-frame mesh generation is intended to capture variations in topology and enable a wider range of animations, the paper provides only a single example illustrating this capability (the BANG case in the supplementary material). The remaining examples appear to be rendered from skeleton-based animation models without topology changes and with a limited diversity of animation types. The authors are encouraged to present examples of topology-changing animations include: object shattering, characters growing extra limbs, soft-body fusion or splitting, cloth tearing, and morphing into an entirely different mesh structure.\n\n3) Lack of comparison with dynamic mesh generation method, for example DreamMesh4D and DriveAnyMesh, both are video-4d mesh generation method. Besides, the authors are encouraged to compare the generation result with general video-4d method beyond L4GM.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VpW3dY2GT4", "reviewer": "Reviewer_pzzF", "summary": "The paper introduces ShapeGen4D, a direct, feedforward framework for producing 4D shape sequences from monocular video input. The method extends an existing 3D shape diffusion model into the temporal domain through several architectural modifications. Firstly, the authors obtain temporally aligned shape latents through a time-aware point sampling strategy. Then, the pretrained shape DiT is finetuned with incorporation of spatiotemporal attention. During training, all shape frames share same noise to enhance temporal stability. Extensive experiments on diverse benchmarks demonstrate the strong qualitative and quantitative results.", "questions": "* Current design only handles short clips (e.g., 16 frames), have the authors considered scaling the model to longer videos?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Conceptual originality: the first to adapt pretrained 3D shape diffusion generator for 4D shape sequence generation.\n* End-to-end pipeline: the proposed method produces temporally coherent mesh sequences directly from video, avoiding costly optimization.\n* Simple but effective modification: the model builds on a well-known 3D backbone with several straightforward and effective architectural modifications, achieving quantitative and qualitative improvements compared to baseline methods.", "weaknesses": "* Underlying gap with respect to pretrained prior: the queries of latents are sampled from non-watertight mesh, while the base 3D backbone is pretrained on watertight queries. Although the authors have explained the reason to do so (to avoid costly mesh registration), the potential performance drop still exists due to this gap.\n* No explicit motion modeling: the generated mesh under each frame is independent with each other. The lack of explicit motion constraint cannot guarantee physically plausible, leading to minor jitter and limiting the application to continuous motion interpolation.\n* Technically lack of novelty: the main components of this framework are mostly common practices in this field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Q6XHBe6NvA", "reviewer": "Reviewer_6RuC", "summary": "This paper introduces ShapeGen4D, a feedforward framework for high-quality 4D shape generation from monocular videos. The method builds upon a pre-trained 3D generative model (Step1X-3D) and adapts it to the dynamic setting through three key innovations:\n\n1. Spatiotemporal attention layers to capture cross-frame dependencies;\n2. Temporally-aligned latent encoding to reduce jitter and improve consistency;\n3. Shared noise across frames to enhance temporal stability.\n\nShapeGen4D directly outputs a sequence of 3D meshes, supporting non-rigid motion, volume changes, and topological transitions without per-frame optimization.", "questions": "1. Since the meshes for consecutive timesteps are generated independently from discrete latents, the resulting 4D sequence lacks an explicit, continuous deformation field. This may lead to non-smooth interpolations and visually incoherent dynamics when rendered at frame rates higher than the generation rate. Do you have plans to incorporate a post-processing step or an intermediate representation (e.g., a neural deformation field) to enable truly continuous, smooth morphing between the generated key meshes? How might this be integrated into your current pipeline?\n2. The current framework generates a fixed number of frames (e.g., 16) in a single feedforward pass. What is the potential of extending ShapeGen4D to handle arbitrarily long videos? For instance, could an autoregressive approach be adopted, where the generation of a subsequent clip is conditioned on the final frames of the previous clip, similar to the strategy employed by L4GM? If so, what would be the main technical challenges, such as error accumulation or maintaining global consistency across segments?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. End-to-end 4d generation framework. The paper presents a direct, feedforward approach for generating 4D mesh sequences from a single video, which is a simplification over more complex optimization-based or multi-stage pipelines.\n2. Handling of complex dynamics. The framework demonstrates the capability to generate a range of dynamic phenomena, including non-rigid motion and topological changes, which are challenging for methods restricted to simpler deformations.\n3. Clear written. The paper is clearly written and provides a well-organized summary of related work, helping to situate the contributions within the field of 4D generation.", "weaknesses": "1. Inherent Limitations in Temporal Geometry Consistency. While the proposed techniques of latent alignment and noise sharing effectively reduce temporal jitter, they do not establish an explicit, parametric model of motion (e.g., a deformation field). The framework still generates each frame's mesh independently from a sequence of latents. This inherently discrete representation may struggle to guarantee as-rigid-as-possible or physically plausible transitions over time, potentially leading to subtle topological inconsistencies or unnatural deformations that are not explicitly regularized. This limitation is indirectly acknowledged by the authors, who note that \"local temporal jitter remains visible in some results.\"\n2. Limited Scalability to Long Video Sequences. The model is designed to generate a fixed-length sequence (e.g., 16 frames) in a single forward pass, constrained by the memory and architecture of the underlying diffusion transformer. This fixed-horizon generation prevents the method from processing arbitrarily long videos, a common requirement in real-world applications. The paper does not explore mechanisms for temporal auto-regressive generation or sliding-window inference, which would be necessary to scale to longer durations, potentially at the cost of error accumulation across segments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)