Track, Inpaint, Resplat: Subject-driven 3D and 4D Generation with Progressive Texture Infilling
Authors: Shuhong Zheng
Ashkan Mirzaei
Igor Gilitschenski
Venue: NeurIPS 2025 poster ()
Keywords: Subject-driven Generation
Personalization
3D and 4D Generation
Primary area: applications
TL;DR: We present TIRE, a novel method for subject-driven 3D/4D generation that well preserves the identity.

Abstract: Current 3D/4D generation methods are usually optimized for photorealism, efficiency, and aesthetics. However, they often fail to preserve the semantic identity of the subject across different viewpoints. Adapting generation methods with one or few images of a specific subject (also known as Personalization or Subject-driven generation) allows generating visual content that align with the identity of the subject. However, personalized 3D/4D generation is still largely underexplored. In this work, we introduce TIRE (Track, Inpaint, REsplat), a novel method for subject-driven 3D/4D generation. It takes an initial 3D asset produced by an existing 3D generative model as input and uses video tracking to identify the regions that need to be modified. Then, we adopt a subject-driven 2D inpainting model for progressively infilling the identified regions. Finally, we resplat the modified 2D multi-view observations back to 3D while still maintaining consistency. Extensive experiments demonstrate that our approach significantly improves identity preservation in 3D/4D generation compared to state-of-the-art methods. Our project website is available at https://zsh2000.github.io/track-inpaint-resplat.github.io/.

Track, Inpaint, Resplat: Subject-driven 3D and 4D Generation with Progressive Texture Infilling
Shuhong Zheng 1.2
Ashkan Mirzaei
Igor Gilitschenski
University of Toronto
Vector Institute
Snap Inc.
{shuhong, ashkan, gilitschenski oor
Abstract
Current 3D/4D generation methods are usually optimized for photorealism, ficiency, and aesthetics. However, they often fail to preserve the semantic iden- tity of the subject across different viewpoints. Adapting generation methods with one or few images of specific subject (also known as Personulization or Subject-driven generation) allows generating visual content that aligns with the identity of the subject. However, personalized 3D/4D generation is still largely underexplored. splat), novel method for subject-driven 3D/4D generation. It takes an ini- tial 3D asset produced by an existing 3D generative model as input and uses video tracking to identify the regions that need to be modified. Then, adopt a subject-driven 2D inpainting model for progressively infilling the identified regions. Finally, we resplat the modified 2D multi-view observations back to 3D while still maintaining consistency. Extensive experiments demonstrate that approach significantly improves identity preservation in 3D/4D gener- ation compared to state-of-the-ar methods. Our project website available at https://zsh2000 gi thub. io to/track- aappannt-esppppe weappaa.ggttu............ wit.ub.ito///oo. io/.
this work, we introduce TIRE (Track, Inpains, RE-
Introduction
Improving the personalization quality of 3D/4D generation is core challenge to enable impact and improve user experience. Current generation methods, however, are mostly guided by text prompts [11, 46, 68, 79, 96, 107, 119] or single images/videos [51, 88, 123, 129] to determine the front-facing appearance of the generated assets. Although these methods provide users with certain amount of control on the content of the generated scene, they fall short in delivering identity outputs that are desired for personalized generation. As illustrated in Fig. (()), the -fate-of-the-art 4D generation model L4GM [73] fails to preserve the identity for the side and back views in the generated 4D asset. In the given example, it results in blueish tone on the originally occluded regions of the cat. These limitations highlight the need for methods to enable subject-driven, 3D/4D generation for personalized applications.
Identity preservation in 3D/4D generation, though tempting, is challenging accomplish. single image- or video- sideo-ggided 3D/4D generation. the model has few cues to infer the appearance of unobserved viewpoints and is forced to hallucinate. One line of research adopts score distillation sampling [1, 5, 12, 26, 29, 59, 68, 91] to optimize the appearance of the novel viewpoints. However, the time-consuming optimization process prevents the paradigm from being widely used in real- world applications. Moreover, during optimization, the appearance and motions of the 3D/4D assets often become averaged out [2, 132], which can reduce the quality of the generated content. More recently,
"Wor done while at University of Toronto and Vector Institute.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Solution: Progressive Texture Intiling
Source View
Can we progressively infill the 30 assel to make look like the given subject...
Rotate
Infili Unseen Regions
Target Views
Challenges Solutions
How identify unseen regions that need to be infilled other views?
infill mesked region while keeping the ideniity?
views allke the givan subject?
How unproject the 20 observations back to 3D and consistency?
Resplat
(a) L4GM
(b) Ours
Figure 1: saate-of-the-ass 4D generation model L4GM [73] and solution. (a) L4GM performs video-to-4D generation. The side and back views of the generated 4D asset does not look alike the subject in the given source view. (b) Our proposed solution TIRE (Track, Inpaint, REsplat) adopts the progressive texture infilling paradigm to inpaint the 3D asset to achieve subject-driven 3D/4I) generation, which preserves the identity of the generated assets when observing from the novel views.
multi-view diffusion models are deployed in 3D/4D generation pipelines [54, 60, 78, 84, 92, 95, 106, 115] to hallucinate appearance of certain number selected novel views. However, these models suffer from systematic errors in color and appearance for novel views. This is likely due to bias in the training data, which is difficult to address without careful dataset filtering. The most recent advancements [27, 42, 50, 102, 104, 110, 114, 124, 130] achieve superior efficiency with native 3D generation, i.e., directly generating the 3D representation without per-scene optimization multi-view observations intermediate results. However, they still fail produce results that can satisfactorily preserve the identity of the given reference as further demonstrated through the experimental analysis in Sec. 4.2 and App. F.
To effectively handle the challenges in 3D/4D generation better personalization, we propose to perform subject-driven 3D/4D generation with progressive texture infilling, shown in Fig. 1(b). Our proposed method, TIRE, named after three key components of the pipeline: Track, Inpaint, and REsplat. These three key components handle specific subtasks to achieve progressive texture infilling in a coordinated way: (1) Track identifies regions in other views that need infilling using long-video tracking. (2) Inpaint uses customized 2D inpainting model to progressively infill unseen regions identified by Track, and ensures that the infilled content matches the subject's 's identity from the given source view. (3) Resplat unprojects the multi-vies view 2D infilled observations from Inpaint back to 3D while still maintaining consistency across multiple views. With these three components collaborating together in cascaded manner, achieve subject-driven 3D/4D generation while preserving the subject's 's identity.
We conduct extensive qualitative and quantitative evaluations, and find that our method serves as an effective general solution for enhancing the identity preservation in 3D/4D generation results upon the baseline methods. Moreover, direction to current reed-forward approaches that utilize multi-view foundation models or native 3D/4D generation pipelines, which can be complementary to other advancements made in 3D/4D generation to collaboratively push forward the research field.
solution takes the exploration step in orthogonal
To summarize, our key contributions are threefold: First, we propose to solve subject-driven 3D/4D generation enhance identity preservation of the generated 3D/4D assets for more personalized experience, which still remains challenge for the most recent advancements an orthogonal while complementary effort towards high-quality generation from the foundation 3D/4D generation models. Second, to achieve the goal, propose innovative three-stage method,
3D/4D generation.

---

TIRE: Track, Inpaint, Resplat. We first adopt video tracking for identifying regions that need infilling. Then, a customized 2D inpainting model is applied to infill the unseen regions. Afterwards, the 2D infilled observations are reprojected back to 3D to create the identity-peseerinng 3D/4D assets. Finally, comprehensive experimental results on our constructed DreamBooth-Dyye Dynnmmic benchmark and wn-the-wild data showcase the superior performance of TIRE on subject-driven 3D/4D generation.
2 Related Works
3D Generation. Diffusion models [18, 80] have recently advanced and greatly improved 2D image/video generation [74]. To apply 2D diffusion model knowledge to 3D generation, Dream- Fusion [68] introduced score distillation sampling (SDS) to transfer knowledge to 3D. While some works [19, 25, 35, 39, 45, 69, 96, 99, 121] improved the quality and efficiency of optimization-br based SDS, 3DiM [98]. Zero- 1-to-3 [51], and their successors [48, 49, 77] offered different approach. They synthesized novel view images using diffusion models and then reconstructed 3D models from these synthetic views. Later works [7, 9, 10, 23, 28, 41, 52-55, 92, 97, 101] further enhanced the correctness and consistency of multi-view diffusion models. To accelerate generation, LRM [21] and concurrent works [40, 112] directly generate 3D representations using
networks. Subsequent works [86, 111, 126] applied this feed-forward approach to 3D Gaussians [36], a com- mon 3D representation. Recent advancements [27, 100, 102, 110, 127] including Mesheormer [50], TRELLIS [104], and the Hunyuan3D series [1 14, 130] achieve fast native 3D generation that directly produce 3D representations after training on massive data. However, the goal of personalized cus- tomization is largely neglected as the field evolves, and even the most recent statc-of-the-ant struggle to generate 3D content that satisfactorily preserve the identity.
4D Generation. Similar to 3D generation, early 4D generative methods [1, 2, 32, 72, 132] used a video version of SDS to transfer knowledge from video generation models to 3D space. Subsequently, multi-view video generation models [37, 108, 123, 125] were introduced. These models enabled 4D optimization using synthetic multi-view videos. Later research [31, 43, 44, 73, 82, 83, 94, 103, 109, 116, 118, 120, 122, 128] further improved the geometry, consistency, and efficiency of generated 4D assets. In contrast efficiency-focusee 3D/41) generation, we foeus on subject-driven generation to preserve identity in generated assets, offering more personalized option for 3D/4D content creation.
Subject-driven Generation. To personalize generated content for users, subject-driven genera- tion became a prominent topic. Textual inversion [14] optimized a special text prompt token to represent specific subject. DreamBooth [75] enabled text-to-image diffusion models to generate customized content for subjects by finetuning pre-trained models with a small number of example images. RealFill [89] extended this idea to subject- driven inpainting and adopted LoRA [22] for parameter-efffiient finetuming. Subject-driven model personalization was also studied in multi-subject compositional generation [105] and videos [17, 30). Beyond prior work in subject-driven 2D gen- eration, DreamBooth3D [70] used image-to-image translation with personalized 2D models on multi-view observations to enhance identity in generated 3D objects. Customize-It-311 [24] proposed using a subject-specified prior to guide SDS optimization of generated assets. Boke-YYur-3D [47] finetuned a multi-view generation model with identity-aware optimization. Unlike previous works, we proposed leveraging the powerful 2D video tracking and inpainting tools to progressively infill the occluded regions of 3D/4D assets, preserving identity in the generated assets with the knowledge in 2D models.
3 Method
Given a single image or video of a certain subject, our goal is to generate a 3D (for images) or 4D (for vidcos) asset that faithfully represents the identity of this specific subject. Our proposed method, TIRE, consists of three stages: Track, Inpaint, Resplat. Track aims at providing the masks indicating the infilling regions from other viewpoints beyond the given source view (Sec. 3.2). Inpaint targets at progressively infilling the unobserved regions in other viewpoints with the infilled contents preserving the identity, while the regions are identified by the previous Track step (Sec. 3.3). Resplat is responsible for unprojecting the 2D infilled observations back to 3D (Sec. 3.4).
Our algorithm starts from a rough 3D/4D representation generated by existing models, shown in the leftmost part in Fig. about the setup stage. We render multi-view observations from the viewpoints within azimuth angle 1 and clevation angle 0%, following the practice in [73, 86).
3

---

Setup
Stage I: Track
Consecutive Frames
Input
Backward Video
Visiblity Map
Video Tracking Model
Stack
30 Generation
Stage Inpaint
Model
Rotate
Rough 3D Asset
Augmentation
Inpaint
Inpaint
Add Training
Add for Training
Render 360 Views
Training Process Proceeds
Stage III: Resplat
Infilled Observations
Pixel Splatted to 3D Gaussian
(x.s.,
Figure 2: Pipeline TIRE. TIRE starts from rough 3D asset created by existing models and its rendered multi-view observations. Afterwards, the three stages Track, Inpaint, Resplat target at identifying the inpainting masks, infilling the occluded regions, and unprojecting back to 3D, respectively.
These initial rendered results will be used in our three-stage pipeline for infilling mask calculation, inpainting, and unprojection as discussed the following subsections.
3.1 Preliminary
Diffusion Models [18] are generative models that learn given data distribution by learning denoising process that starts with random (typically Gaussian) noise and transforms this noise over multiple steps to the data distribution. In order to reduce the computational cost of learning distributions of image data, latent diffusion models [74] perform the diffusion and denoising processes in latent space. Pretrained autoencoders are used for the conversion between latent and image spaces.
In the diffusion process, convert clean latent =o to noisy latent 2r of arbitrary timestep T as
2T ~ q(2 20)) = N((r; Vārzo. (1 )r)
(1)
where the notation oT = 1 37 and ãT II T ag simplify the formulation with Br representing the schedule of the strength of the noise added in timestep I". When I' + 0o, is close to being equivalent to sampling from an isotropic Gaussian distribution.
The denoising process is the inversed operation to the diffusion process. The denoised latent at timestep f 1 can be estimated with the latent timestep
(0(21-122) (2 1: pe (2. t), Ee(,,
(2)
where the parameters (z t), Eg (z, t) of the Gaussian distribution are estimated from the diffusion model. As revealed in [18]. (2 t) only has negligible contribution on the results from the experiments. Therefore, the main objective of the denoising framework is to estimate μi (Zz, t), which reparameterized with
Be
( f)
co(2),
(3)
Vas
a2
where cp(2,, t) is the denoising network to predict the added noise for at timestep

---

The training objective for the denoising network is
L Re-N(O,1),. [|le co (2z, 1)13]
With the well-trained denoising network E 1) and the deterministic sampling schedule in DDIM [81], the denoising process can be represented as
v1 a160(=1.))) + VI- a1-aa-ieo 1).
1
vo--1
Vos
Large Reconstruction Models (LRM) [21] are foundation models for 3D reconstruction that predicts triplane representation in a feed-forward manner. Later, large Gaussian model (LGM) [86] achieves feed-forward prediction from multi-view observations to 3D) Gaussians, inspired by the previous works [6, 85] that utilize U-Net to splat each pixel into 3D Gaussian in space. To be more concrete, taking 4 multi-view images as input, LGM functions as
f: ::Re xx + ( x
(6)
where the output is a total of (4 x 2 the center position x E RR', scaling S E R rotation quaternion q E IR, opacity a E R, and color IRS of the 3D Gaussians. Follow-up work L4GM [73] extends the feed-forward 3D Gaussian generation to 4D with the additional time dimension, generating a sequence of 3D Gaussians that are consistent across both the spatial and time dimension.
x W 2
3D Gaussians, each has 14 parameters representing
3.2 First Stage: Track
Goal. As shown in Fig. 1(b), our proposed method adopts progressive texture infilling to achieve dlentity-preservingg 3D/4D generation results. The first subtask we need to solve is to identify the regions that need to be infilled in viewpoints not observed in the original data. Obtaining a decent mask for infilling, however, poses non-trivial challenge.
Identifying Infilling Masks with Video Track- ing. With the multi-view observations rendered from the setup stage, we can stack the consecu- tive frames together in the order of the camera movement to form video. Then, we leverage the video tracking model CoTTracker [34] to find the correspondence between the source view and the target views. The underlying principle is that, the tracking result of point on the target view is still within the valid mask of the target image, and with its visibility flag still active, we con- sider the point the source view to be visible on the target view. An intuitive implementation is to start tracking from the given view, and see how the valid 2D pixels get propagated to the target views. However, as shown in the first row in Fig. 3, many small inpainting regions appear in the infilling mask, resulting in subop- timal grainy inpainting performance. Although this issue may be mitigated by performing other post-processing operations like dilations on the mask, it would require case-specific parameters to control the post- post-processing operations. Instead, we design a wiser approach to obtain the mask with backward tracking, which performs video tracking from the target views to the given source view. As displayed in the second row in Fig. 3, the masks obtained with backward tracking is more accurate and also more suitable for the following inpainting process. The insight behind backward tracking is that the given source view contains the richest information about the subject's 's identity. Therefore, we start tracking from the target views to establish as many correspondences as possible with the given view, to effectively leverage the subject information present in the given source view. Our proposed solution a model-agnostic approach that is generally applicable to any types of 3D representations, and also leverages the power of 2D models that are trained massive video data.
Forward Tracking
Backward Tracking
Source View
Infilling Mask
Infilled Result
Figure 3: Comparison between forward tracking and backward tracking when identifying the in- painting mask. Forward tracking, which means that the tracking process starts from the given source view the target views, though being more intuitive, leads to grainy inpainting results. In con- trast, backward tracking produces more accurate masks in better shapes, which benefits the follow- ing inpainting process.
5

---

3.3 Second Stage:
Inpaint
Challenges. After obtaining the infilling masks for novel views in the Track stage, we need to wisely inpaint these regions to maintain the identity of the subject. We are facing two substantial challenges: (1) How to faithfully preserve the identity in the infilling regions: (2) How to impaint the viewpoints that are far away from the given source view, as the reference appearance in the source view may be very different and is unable to provide direct guidance. To address challenge (1), inspired by RealFill [89], we propose to personalize the pretrained stable diffusion [74] inpainting model to be subject-driven, aiming to preserve the identity of the inpainted region. To handle challenge (2), we perform the inpainting process in progressive manner, we first start inpainting from the viewpoints that are close to the given source view. As the training proceeds, the model progressively learn to inpaint the viewpoints that are farther away from the original source view.
Solution. With the pretrained stable diffusion inpainting model in hand, we inject LoRA [22] weights in the pretrained model and finetune with randomly generated binary mask m, for inpainting. The loss calculation will only be conducted on the valid regions my as
m. [ea t, P. mi, (1 m;) O x) E),
where my E {0.1) 1 is the valid mask in which 1 indicates the foreground of the image, while 0 indicates the background of the image. The valid mask is obtained by the same background removal tool used in recent 3D/4D generation works [54, 73, 86]. p is a fixed language prompt "A photo of sks". "o" denotes the element-wise multiplication and therefore (1 - (1-m;) O is the masked image. The other notations follow the same convention as Eq. 4.
At the beginning of the tuning process, since only single image/video from the source view is available, we perform horizontal flipping and small-scale rotations within 15s on the original image for data augmentation. After training the inpainting task on the original image together with its augmented counterparts, we perform inpainting of azimuth angle = +00 which we name it a sweet spot. This viewpoint serves an anchor viewpoint for later processes when inpainting the viewpoints that are farther away from the source viewpoint. To be more concrete, when inpainting the farther viewpoints within +900, the similar operation of backward tracking in Sec. 3.2 will be applied to track from the queried viewpoints to this anchor viewpoint. It helps further reduce the area that needs to be infilled, therefore lowering the difficulty for the model to inpaint the far away viewpoints. The reasons for choosing +:20 as the sweet spot for the anchor viewpoint in tracking are (1) compared the source viewpoint at 0%, it has certain exploration on unseen regions that can provide larger known regions for the farther viewpoints; (2) when inpainting the +:200 viewpoint, since it does not have significant shift from the original training image, the inpainted results are relatively decent and reliable. Therefore. this sweet spot strikes a balance between "exploration and exploitation" " of the given source view observation. After the =90 viewpoint is inpainted, it serves as the next anchor point for inpainting the rest of the viewpoints within +90 ~ =180" As we do not want significant change on the original structure, we perform denoising with the first 30% of the denoising schedule following similar practice as [33].
3.4 Third Stage: Resplat
The goal of this stage is to unproject the inpainted 2D observations back 3D. As the frames are infilled separately during the inpain/ stage in Sec. 3.3, there may exist inconsistency across the inpainted frames. Thus, before lifting to 3D, we propose to the multi-view diffusion model [92] to refine the consistency of the multi-view observations. More specifically, inspired by the previous work on mask-aware image editing [62], our multi-view denoising process only updates the latents on the unseen viewpoints as
2t-1 2-11 = ž O M + 2, (1-
(8)
where 21-1, E [[V+))XXXXXX, _ is the predicted latent at T = t--1 1 during the denoising process obtained by Eq. 5. 2,-1 is the noisy latent obtained by Eq. with the forward diffusion process. V y== 4 is the number of views, while c, ht, w are the channel number, height and width of the latents. As the multi-view diffusion model is conditioed, there are (V + 1) entries on the first dimension of the latents, as the additional one being the latent of the conditional image. M E (0, 1] (1)x)x elsewhere. Similar to the practice of the Inpaint stage, only the first 30% of the denoising schedule
the mask for refinement with value 0 for the source view and

---

Sourca View
SV3D
Wonder30
LBM
MeshFormer
TRELLIS
Hunyuan3D-ve
Durs
Figure 4: Qualitative comparison on image-to-3D generation with SV3D [90]. Wonder3D [54], LGM [86], McshFormer [50], TRELLIS [104], and Hunyuan3D-v2.5 [38]. Compared against other method in the image-to-3D setting, our method better preserves the identity of the reference image, and also reaches superior quality on geometry. It is noticeable that even for the most recent
advancements in image-to-3D like TRELLIS and Hun2 the challenge of producing 3D assets is still not well solved.
applied. The mask-aware latent update strategy reinforces the identity preservation of the front view of the final 3D/4D assets. After refining the multi-view observations with Eq. 8, we resplat pixels in every viewpoint to Gaussians with [73, 86]. Note that this multi-view-to sawwwto-Gaussian process is also adaptable to different large reconstruction models, and thus can be applied to different 3D representations beyond 3D Gaussians.
Experiments
4.1 Experimental Settings
Datasets. Starting from the original DreamBooth [75] dataset that focuses on subject-driven image generation, we construct a DreamBooth-Dynamiic Dynamic dataset. It is based on animatable subjects in the original DreamBooth dataset and will be used for subject-driven image-to-3D and video-to-4D generation. More details about the dataset are provided in App. B in the appendix. Besides the qualitative and quantitative evaluations on the constructed DreamBooth- Dyyh- dataset, we also demonstrate that our method works for in-the-wild data that are displayed on the official project webpage of L4GM [73] in App. F in the appendix due to space issue.
Baseline Methods. We select Wonder3D [54], SV3D [90], LGM [86], MeshFormer [50], TREL- LIS [104], and H--v [38] for the comparison on the image-to-3D task. For video-to-4D, we select recent 4D generation methods STAG4D [123] and SV4D [108]* along with L4GM [73]. As few existing works focus on subject-driven 3D/4D generation [24, 47, 70], we choose Customize- It-3D [24] for comparison, as it is the most recent open-source option.
4.2 Qualitative Evaluation
We show visualizations of our results and the compared methods in Fig. 4 for maage-to-3D generation and Fig. and Fig. for video-to-4D generation, rendered from different viewpoints and timesteps. From the qualitative comparisons, we observed the following:
Identity-preservinn Appearance. Fig. and Fig. 5 demonstrate that the 3D/4D assets generated by our method achieves significantly better identity preservation compared to the baselines. Notably, even the most recent advancements in image-to-3D generation, such as TRELLIS [104] and Hunyuan3D- v2.5 [38], still face substantial challenges in producing identity-preservie that the task of subject-driven 3D/4D generation is a valid and important problem to investigate, highlighting the necessity of addressing it to achieve better identity preservation for 3D/4D generation.
3D assets, demonstrating
CComparison included in App. in the supplementary material due different pose settings.

---

Reference Video
Reference Video
Customize-It-3D
L4GM
Customize-It-3D
STAG4D
L4GM
Figure Comparison between our method and the baselines Customize-It33D [24] (additional feed-forward operation from L4GM is applied after obtaining multi-view observations to allow it to generate dynamic 3D assets), STAG4D [123], and L44M [73].
Reference
Reference Video
L4GM
Ours
L4GM
L4GM
L4GM
Ours
Comparison between our method and L4GM [73]. Although the main objective of our TIRE is to preserve the identity of the generated assets, the geometry the generated assets also gets improved because the three stages in our pipeline collectively promote cross-view consistency.
Figure 6:
Enhanced Quality on Geometry, Although TIRE mainly aims improve subject-driven appearance modeling of the generated 3D/4D assets, it also demonstrates better geometry quality than the baseline methods. As shown Fig. 6, our results outperform L4GM on the geometry of the generated assets. The rendered viewpoints from our method have fewer ghosting artifacts, which are caused by cross- view inconsistency. The reason that TIRE can fix the geometry is that the Truck and Inpain! stages collaboratively propagate pixels seen in source views to the target views. Also, the Resplat stage includes a mask-aware halfway juffusion-denoising process in Eq. 8. These designs within TIRE implicitly refine the consistency of the multi-view observations and the generated 3D/4D assets.
General Solution 3D/4D Generation Methods. As discussed in Sec. 3.4, the demonstrated results are generated with LGM [86] and L4GM [73] as the base model for our method. As our progressive texture infilling process only needs perform on rendered 2D frames for tracking and inpainting, our method has the advantage of being generally applicable to all types of 3D/4D generation methods, no matter what representation the method adopts. To support this, also include the results of how our TIRE can further enhance identity preservation for even the most advanced models like H-vv [38] in App.
4.3 Quantitative Evaluation
As there are no standard evaluation protocols for benchmarking the quality subject-driven 3D/4D generation, we first follow [75] to adopt DINO [4] feature similarity to measure the subject fidelity of

---

the generated assets. More specifically, the source view is the reference image of the subject, and without loss of generality, 4D generation is selected as a case study.
For each generated asset, we render every 10 timesteps from 8 views around the assets, with a 45 interval in azimuth angle, to calculate the DINO similarity score with the reference image.
Table 1: Quantitative comparisons the DINO feature similarity metrics from DreamBooth [75]. Best: bold, 2nd best: italics.
We report the results in Tab. 1, which is the average similarity score actoss all rendered im- ages in all the scenes. Although TIRE outper- forms L4GM and STAG4D in subject fidelity according to the DINO similarity metric, which commonly used in the literature [13, 75), it is surprising that Customize-It-3D ranks highest. However, the qualitative comparisons in Sec. 4.2 clearly show that Customize-It-31 performs worse than other methods. This suggests that the original DINO similarity metric, while being a standard metric for subject fidelity, is not the most suitable quantitative measure for the nuances of the subject-driven 3D/4D generation task. We further discuss this limitation in App. C in the appendix.
Method
DINO (WIT-S/16) (f)
DINO (VIT-B/1) (T)
Customize-It-3D [24]
0,5773
0.60887
[14D]
0.5213
0.5426
STAGAD [123]
0.5287
0.5592
L4GM [73]
0.5506
0.5694
TIRE (Ours)
0,5665
0,5815
Having recognized the challenges of conducting quantitative evaluations with purely vision-based models, we turn to vision- Laiionlanguage models (VLMs) for more comprehensive assessment across multiple dimensions including identity preservation and visual quality. Specifically, we follow DreamBench++ [66] to adopt VI.M-hased evaluation on the identity preservation quality of the generated assets. Following the subject similarity evaluation option in DreamBench+++, VLMs to give an overall score from 0-4 (0 is the worst, while 4 is the best) on subject consistency between the reference image and the generated images rendered from the assets, considering the following aspects: shape, color, texture, and facial features. Detailed implementations including the used prompts for VLMs are described in App. D. For the choice of VLMs, besides GPT-4o [64] which is adopted in DreamBench++, we also use five other VLMs (OpenAl o4-mini [65], Gemma 27B [15]. Gemini 2.0 Flash [16], Qwen2.5-VL-7B [3], Sestral-Small3.111 1...3.....111. -nB--nnttuct [63]] with different architectures and sizes to foster the reliability of the evaluation. For each generated asset, we evenly render views for each generated asset for evaluating the quality. The results in Tab. show that our method achieves the best subject consistency when taking multiple aspects of consistency into account, demonstrating the effectiveness of our method for enhancing identity preservation for the generated 3D assets. Nevertheless, we can observe that the scores for all the methods still remain distant from the perfect score of 4, indicating that the task of subject-driven 3D/4D generation is still far from being solved.
ask the
Table 2: Quantitative comparisons on the VLM-based scores on the views rendered from the generated assets from different methods. Best is marked as bold.
Methods
VLM-based scores
GPT.40
OpenAl o4-mini
Gemma
Qwen2.5.VL.P7
Mistral--Sall-3.1-222
Average
TRELLIS
1.332
1.426
1,870
1,402
1.596
1.228
1.476
Hunywn3DDa225
1.614
1,600
2,098
1,533
1.780
1.501
1.703
TIRE (Ouns)
1.777
2.103
1.739
4.4 User Study
To further calibrate the identity preservation quality with human perception, we conduct user study on the 4D assets generated by Customize- It-3D [24]. L4GM [73], and our method. We ask the volunteers to score the generated assets in scale of 1-10 on the overall quality, where the participants are guided focus on subject fidelity, cross-view consistency, etc. that are con- sidered important factors for 3D/4D generation quality. We randomly select 10 samples from the DreamBooth- Dynamic dataset. Note that we
Custemize-R-30
1.72
L4GM
5.67
6:37
3
5
Figure 7: Results of our user study. Our method scores the highest in overall quality without explic- itly informing the users that we are focusing on subject-driven generation.
do not explicitly tell the participants that working improving subject-driven generation,

---

Appeorance by Tradking
Progreeeve Intling
Prograssize
Intling
Source Veew
Source View
Irpanting Meak
30%
50%
Figure 8: Ablation study on the progressive learn- ing strategy in TIRE. Without adopting progres- sive learning, the model tends to consistently infill the appearance of the given source view regard- less of the current pose, which results in wrongly infilling the textures on the side and back views.
Figure 9: Ablation study on different denoising schedules for inpainting. A smaller schedule may leave some regions unchanged, while a larger one may overly distort the textures.
which fosters the fairness in user study and reduces the underlying bias of the subjective preference towards our method. Details the instructions and interface of our user study can be referred in App. A in the appendix. With 18 volunteers participating in the user study, we collect a total of 540 scores from participants. The results of the user study are reported in Fig. 7, which shows that our method is more subjectively preferred from the user experience.
4.5 Ablation Study
Progressive Texture Infilling is core idea of our pipeline to infill the texture of unseen regions in the generated assets. Therefore, to prove the validity of our progressive infilling strategy, we demonstrate the ablation results illustrated in Fig. 8. More specifically, for the variant "w/o Progressive Inpainting". we use the inpainting model finetuned only on the original image the source view and augmented counterparts. When the progressive infilling strategy is not applied, the model tends to inpaint the appearance corresponding to the given source view the object. We can observe that the model fills in textures of cat's face with pair of cat whiskers in the target view, even if the target view is 60 off the source view and actually displays the side view of the cat.
Denoising Schedule for Inpainting, As discussed in Sec. 3.3, we empirically adopt the first 30% of the denoising schedule in the Inpaint stage. To provide more intuitive demonstration for guiding real practice, we examine the impact of larger or smaller denoising schedule in Fig. 9. We can observe that when we adopt a smaller denoising schedule of 15%, some of the regions where the textures need to be refined remain unaltered, like the regions in the blue bounding box. On the other hand, when we choose larger denoising schedule of 50%, we may have the risk of having an overly intense change on the textures, such that the appearance becomes less realistic.
Due to the limit of space, more ablation study can be found in App. E in the appendix.
5 Conclusions
We present TIRE, which aims at subject-driven 3D/4D generation that progressively infill the textures of the occluded regions. TIRE contains three important steps: Track, Impaint, Resplat. Track helps identify the regions that need to be infilled via video tracking tools. Inpaint aims at infilling the occluded regions while preserving the identity of the subject. Resplat unprojects the multi- view infilled observations back to 3D with cross-view consistency. Extensive experimental results demonstrate that the 3D/4D assets obtained from our method achieve superior performance on both more identity-presee preserving appearance and refined quality on geometry. We believe that our solution an important exploration on the direction complementary to the current research on efficient feed-forwal 3D/4D generation pipelines, and can serve as a useful tool for expressing the creativity and enhancing the personalization on subject-driven 3D/4D generation. More discussions can be referred in App. C in the appendix.
10

---

References
[1] S. Bahmani, X. Liu, W. Yifan, Skorokhodov, V. Rong. Z. Liu, X. Liu, J. J. Park, S. Tulyakov.
G. Wetzstein, A. Tagliasacchi, and D. B. Lindell. TC4D: Trajectory text-to-4D generation. ECCV, 2024. 1,3
[2] Bahmani, I. Skorokhodov, V. Rong. G. Wetzstein, L. Guibas, Wonka, S. Tulyakov, Park. A. Tagliasacchi, and B. Lindell. 4D-fy: Text-to-4D generation using hybrid score distillation sampling.
In CVPR, 2024. 1,3
[3] Bai, K. Chen, X. Liu, Wang. Ge, Song, K. Dang, P. Wang, Wang. J. Tang, Zhong, Zhu, M. Yang. Z. Li, J. Wan, P. Wang. W. Ding, Z. Fu, Y. Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and J. Lin. Qwen2.5-VL. technical report. arXiv preprint arXiv:2502. 13923, 2025. 9
[4] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin. Emerging properties
in self-supervised vision transformers. In ICCV, 2021. 8
[5] I. Chachy, G. Yariv, and S. Benaim. RewardSDS: Aligning score distillation via reward-weighted sampling. arXiv preprint WXXv:2503.09601 2025.
[6] D. Charatan, S. Li, A. Tagliasacchi, and V. Sitzmann. pixelSplat: 3D gaussian splats from image pairs for scalable generalizable 3D reconstruction. In CVPR, 2024. 5
[7] C. Chen, X. Yang. F. Yang. C. Feng. Z. Fu, C.-S, Foo, G. Lin, and F. Liu. Sculpt3D: Multi-view consistent text-to-3D generation with sparse 3D prior. In CVPR, 2024. 3
[8] D. Z. Chen, Y. Siddiqui, H.-Y. Lee, S. Tulyakov, and M. Nießner. Texi2Tex: Text-driven texture synthesis via diffusion models. In /CCV, 2023. 26
[9] Н. Chen, B. Shen, Y. Liu, R. Shi, L. Zhou, C. Z. Lin, J. Gu, H. Su, G. Weizstein, and L. Guibas. 3D-Adapter: Geometry-consister multi-view diffusion for high-quality 3D generation. arXiv preprint arXiv:2410.188 189744 2024. 3
[10] H. Chen, R. Shi, Y. Liu, B. Shen, J. Gu, G. Wetzstein, H. Su, and L. Guibas. Generic 3D diffusion adapter using controlled multi-view editing. arXiv preprint 0rXiv:243..12333 2024. 3
[11] Z. Chen, F. Wang, Y. Wang, and H. Liu. Text-to-31 tD using gaussian splatting. In CVPR, 2024.
[12] K. Do and B.-S. Hua. Text-to-3D generation using Ceenenn-Shnnoon score distillation. arXiv preprint arXiv:2503. 10660, 2025. 1
[13] Z. Fan, Z. Yin, G. Li, Y. Zhan, and H. Zheng. DreamBoothh+:: Boosting subject-driven generation via region-level references packing. In ACM MM, 2024. 9
[14] R. Gal, Alaluf, Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-Or -rh An image worth one word: Personalizing text-to-image generation using textual inversion. In ICLR, 2023, 3
[15] Gemma Team. Gemma 3 technical report. arXiv preprint arXiv:2503. 19786, 2025.
[16] Google.
Introducing Gemini 2.0: our new Al model for the agentic era, 2025.
[17] Y. Guo, C. Yang. A. Rao, Z. Liang, Y. Wang. Y. Qiao, M. Agrawala, D. Lin, and B. Dai. AnimateDiff: Animate your personalized text-to-image diffusion models without specific tuning. ICLR, 2024. 3
[18] J. Ho, A. Jain, and Abbeel. Denoising diffusion probabilistic models. In NewrIPS, 2020. 3.4
[19] S. Hong. D. Ahn, and S. Kim. Debiasing scores and prompts of 2D diffusion for view-consistent coxt-to-3D generation. In NeurlPS, 2023. 3
[20] W. Hong, M. Ding, W. Zheng, X. Liu, and J. Tang. CogVideo: Large-scale pretraining for text-to-video generation via transformers. In ICLR, 2023. 24
[21] Y. Hong, K. Zhang, J. Gu, S. Bi, Y. Zhou, D. Liu, F. Liu, K. Sunkavalli, T. Bui, and H. Tan. LRM: Large reconstruction model for single image to 3D. In ICLR, 2024. 5,
[22] E. J. Hu, Y. Shen. P. Wallis. Z. Allen-Zhu, Y. Li. S. Wang. L. Wang, and W. Chen. LoRA: Low-rank adaptation of large language models. In /CLR, 2022. 3, 6, 24
[23] H. Hu, T. Yin, F. Luan, Y. Hu. H. Tan, Z, Xu, S. Bi. S. Tulsiani, and K. Zhang. Turbo3D: Ultra-fast text-to-3D generation. In CVPR, 2025. 25,

---

[24] N. Huang, T. Zhang. Y. Yuan, D. Chen, and S. Zhang. Customize-It-3D High-quality 3D creation from a
single image using subject-specific knowledge prior. arXiv preprint arXiv:23/2.11585 2023. 3, 3,7, 8, 37
[25] Huang. Y. Zeng. Zhang. W. Xu, H. Xu, S. Xu, R. W. H. Lau, and Zuo. DreamControl: Control-based text-to-31 3D generation with 3D self-prior. In CVPR, 2024. 3
[26] Huang, J. Wang, Y. Shi, B. Tang, X. Qi, and L. Zhang. DreamTime: An improved optimization strategy for manuiiio-guidedd 3D generation. In ICLR, 2024. 1
[27] Z. Huang, M. Boss, A. Vasishta, J. M. Rehg, and V. J.Jampnni. SPAR3D: Stable point-aware reconstruction of 3D objects from single images. In CVPR, 2025. 2, 35,
[28] K.-H. Hui, A. Sanghi, A. Rampini. K. R. Malekshan, Z. Liu, H. Shayani, and C.-W. Fu. Make-A-Shape: ien-million-ssale 3D shape model. In ICML, 2024. 3
[29] C. Jiang. Y. Zeng. T. Hu, S. Xu, W. Zhang. W. Xu, and D.-Y. Yeung. JointDremmr: Ensuring geometry consistency and text congruence text-to-3D generation via joint score distillation. ECCV, 2024.
[30] Jiang, Wu, S. Yang. C. Si, Lin. Y. Qiao, C. Loy. and Z. Liu. VideoBooth: Diffusion-based video generation with image prompts. In CVPR, 2024.
[31] Y. Jiang. C. Yu, C. Cao, F. Wang, W. Hu, and J. Gao. Animate3D: Animating any 3D model with multi-view video diffusion. In NeurlPS, 2024. 3
[32] Y. Jiang, L. Zhang, J. Gao, W. Hu, and Y. Yao. Consistent4D:: Consistent 360 dynamic object generation from monocular vidco. In ICLR, 2024. 3
[33] Y. Kant, A. Siarohin, M. Vasilkovsky, R. A. Guler, J. Ren, S. Tulyakov, and I. Gilitschenski. iNVS: Repurposing diffusion inpainters for novel view synthesis. In SIGGRAPH Asia, 2023. 6, 28
[34] N. Karaev, I. Rocco, B. Graham, N. Neverova, A. Vedaldi, and C. Rupprecht. CoTracker: it better track together. In ECCV, 2024. 5
[35] O. Katzir, O. Patashník, D. Cohen-Or Or, and D. Lischinski. Noise-free free score distillation. In ICLR, 2024.
[36] Kerbl, G. Kopanas, Leimkühhler. and G. Drettaiis, 3D gaussian splatting real-time radiance field rendering. In SIGGRAPH, 2023. 3
[37] J.-G. Kwak, E. Dong. Y. Jin, H. Ko, S. Mahajan, and K. M. Yi. Vivid-I-to-3: Novel view synthesis with video diffusion models. In CVPR, 2024. 3
[38] Z. Lai, Y. Zhao, H. Liu, Z. Zhao, Q. Lin, H. Shi, X. Yang, M. Yang, S. Yang. Y. Feng, S. Zhang, X. Huang, D. Luo, F. Yang, F. Yang, L. Wang. S. Liu, Y. Tang, Y. Cai, Z. He, T. Liu, Y. Liu, J. Jiang, Linus, J. Huang, and C. Guo. Hunyuan 2.5: Towards high-fidelity 3D assets generation with ultimate details. arXiv preprint arXiv:2506. 16504, 2025. 7, 8, 31,
[39] K. Lee, K. Sohn, and J. Shin. DreamFlow: High-quality text-to-3D generation by approximating probability flow. In ICLR, 2024.
[40] J. Li, H. Tan, K. Zhang, Z. Xu, F. Luan, Y. Xu, Y. Hong, K. Sunkavalli, G. Shakhnarovich, and S. Bi. Instant3D: Fast text-to-33 3D with sparse-view generation and large reconstruction model. In ICLR, 2024. 3
[41] P. Li, Y. Liu, X. Long. F. Zhang, C. Lin, M. Li. X. Qi, S. Zhang, W. Xue, W. Luo, P. Tan, W. Wang. Q. Liu, and Y. Guo. Fra3D; High-resolution multiview diffusion using efficient row-wise attention. In NeurIPS, 2024. 3
[42] W. Li, Liu, H. Yan, Chen, Y. Liang. X. Chen, P. Tan, and X. Long. CraftsMan3D: High-fidelity mesh generation with 3D native generation and interactive geometry refiner. In CVPR, 2025. 2
[43] Li, Chen, and P. Liu. DreamMesh4D: Video-to-4D generation with sparse-controlled gaussian-mest hybrid representation. In NeurlPS, 2024.
[44] H. Liang, Y. Yin, D. Xu, H. Liang, Z. Wang, K. N. Plataniotis, Y. Zhao, and Y. Wei. Diffusion4D: Fast opatial-temporal consistent 4D generation via video diffusion models. In NeurIPS, 2024. 3
[45] C.H.. Lin, J. Gao, L. Tang, T. Takikawa, X. Zeng. X. Huang, K. Kreis, S. Fidler, M.-Y. Liu, and T.-Y. Lin. Magic3D: High-resolution text-to-3D content creation. In CVPR, 2023. 3
12

---

[46] H. Ling, S. W. Kim. A. Torralba, S. Fidler, and K. Kreis. Align your gaussians: Text-to-4D with dynamic 3D gaussians and composed diffusion models. In CVPR, 2024. 1
[47] F. Liu, H. Wang. W. Chen, H. Sun, and Y. Duan, Make Your-3D: Fast and consistent berbbecc-driieen 3D content generation. In ECCV, 2024. 3.7
[48] M. Liu. Xu, H. Jin, L. Chen, M. Varma T, Xu, and H. Su. One-2-3-45: Any single image to 3D mesh in seconds without per-shape optimization. In NewrPPS, 2023.
[49] M. Liu, R. Shi, L. Chen, Z. Zhang, C. Xu, X. Wei, H. Chen, C. Zeng. J. Gu, and H. Su. One-2-3-45+++ Fast single image to 3D objects with consistent multi-view generation and 3D diffusion. In CVPR, 2024. 3, 32, 35
[50] M. Liu, C. Zeng, X. Wei, R. Shi, L. Chen, C. Xu, M. Zhang, Z. Wang X. Zhang, I. Liu, H. Wu, and H. Su. MeshFormer. High-quality mesh generation with 3D-guided reconstruction model. In NewrIPS, 2024. 3, 7, 25
[51] R. Liu, R. Wu, V. Hoorick, P. Tokmakov, Zakharov, and C. Vondrick. Zero- Zero-1-to-3- Zero-shot one image to 3D object. In ICCV, 2023. 1,3
[52] Y. Liu, C. Lin, Z. Zeng, X. Long, L. Liu, T. Komura, and W. Wang. SyncDreamer: Generating multiview- consistent images from single-view image. In ICLR, 2024. 3
[53] Y. Liu, M. Xie, H. Liu, and T.T. Wong. Text-guided texturing by synchronized multi-view diffusion. In SIGGRAPH Asia, 2024.
[54] X. Long, Y--. Guo, C. Lin, Y. Liu, Z. Dou, L. Liu, Y. Ma. S.-H Zhang. M. Habermann, C. Theobalt, and W. Wang, Wonder3D: Single image 3D using cross-domain diffusion. In CVPR, 2024. 2, 6, 7, 28, 31, 32, 36
[55] Y. Lu, J. Zhang. S. Li. T. Fang. D. McKinnon, Y. Tsin, L. Quan, X. Cao, and Y. Yao. Direct2.5: Diverse text-to-3D generation via multi-view 2.5D diffusion. In CVPR, 2024. 3
[56] A. Lukoianov, H. S. de Ocáriz Borde, K. Greenewald, V. C. Guizilini, T. Bagautdinov, V. Sitzmann, and
J. Solomon. Score distillation via reparametrized DDIM. In NeurlPS, 2024, 25
[57] J. Ma, J. Liang, C. Chen, and H. Lu. Subject-Diffusion Open domain personalized text-to-image to-tooi-gge generation without test-time fine-tuning. In SIGGRAPH, 2024. 28
[58] Ma, Y. Wei, Y. Zhang, X. Zhu, Z. Lei, and L. Zhang. ScaleDreamer: Scalable text-to-3D synthesis with asynchronous score distillation. In ECCV, 2024.
[59] D. McAllister, Ge, J.-B. Huang, W. Jacobs, A. A. Efros, A. Holynski, and A. Kanazawa. Rethinking score distillation as a bridge hetween image distributions. In NeurlPS, 2024. 1
[60] L. Keriaz-K Laina, C. Rupprecht, N. Neverova. A. Vedaldi, O. Gafni, and F. Kokkinos. IM-3D: Iterative multiview diffusion and reconstruction for high-quality 3D generation. In ICML, 2024. 2
[61] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. NeRF: Representing scenes neural radiance fields for view synthesis. In ECCV, 2020. 26
[62] A. Mirzaei, T. Aumentado-Armstronnggg M. A. Brubaker. J. Kelly, A. Levinshtein, K. G. Derpanis, and I. Gilitschenski. Watch your steps: Local image and scene editing by text instructions. In ECCV, 2024.
[63] Mistral AI. Mistral small 3.1, 2025,
[64] OpenAI. GPT-4o system card. arXiv preprint urXiv:2410.21276, 2024.
[65] OpenAI. OpenAl o3 and 04-mini system card, 2025.
[66] Peng, Cui, Tang, Z. Qi. R. Dong. Bai, C. Han, Z. Ge, Zhang, and S.-T. Xia. DreamBench++:
A human- aligned benchmark for personalized image generation. In ICLR, 2025. 9,29
[67] D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. Müller, J. Penna, and R. Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. In /CLR, 2024. 24
[68] B. Poole, A. Jain, J. T. Barron, and B. Mildenhall. DreamFusion: Text-to-3D using 2D diffusion. In ICLR, 2023. 1.3
13

---

[69] G. Qian, J. Mai, A. Hamdi, J. Ren. A. Siarohin, B. Li. H.-Y. Lee. I. Skorokhodov, P. Wonka, S. Tulyakov. and B. Ghanem. Magic123: One image high-quality 3D object generation using both 2D and 3D diffusion priors. In ICLR, 2024. 3
[70] A. Raj. S. Kaza, B. Poole, M. Niemeyer, N. Ruiz, B. Mildenhall, S. Zada, K. Aberman, M. Rubinstein, J. Barron, Y. Li, and V. Jampani. DreamBooth3D: Subject-driven text-to-3D generation. In ICCV, 2023.
3,7
[71] N. Ravi, V. Gabeur, Y.-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, Mintun, J. Pan. K. Alwala, N. Carion, C.-Y. Wu, R. Girshick, P. Dollár, and C. Feichtenhofer. SAM
2: Segment anything in images and videos. In ICLR, 2025. 27
[72] J. Ren, L. Pan, J. Tang, Zhang, Cao, G. Zeng, and Z. Liu. DreamGaussianDD: Generative 4D gaussian splatting, arXiv preprint arXiv:3312.17142, 2023. 3
[73] J. Ren, K. Xie, A. Mirzaei, H. Liang, X. Zeng. K. Kreis, Z. Liu, A. Torralba, S. Fidler, S. W. Kim, and Ling. L4GM: Large 4D gaussian reconstruction model. NewrIPS, 2024. 1, 2, 5, 6, 7, 8, 9, 31, 33,37
[74] R. Rombach, A. Blaumann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022. 3, 4,
[75] N. Ruiz, Y. Li. V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman. DreamBooth: Fine tuning lext-to-image diffusion models for subject-driven generation. CVPR, 2023. 3, 7, 8, 9, 24, 27
[76] J. Shi, W. Xiong, Z. Lin, and H. J. Jung. InstantBooth: Personalized text-to-image generation without test-time finetuming. In CVPR, 2024, 28
[77] R. Shi, H. Chen, Z. Zhang, M. Liu, C. Xu, X. Wei, I. Chen, C. 7eng, and H. Su. 7ero123+4 a single image consistent multi-view diffusion base model. arXiv preprint arXiv:2310.15110, 2023. 3
[78] Y. Shi, P. Wang. J. Ye, L. Mai, K. Li, and X. Yang. MVDream: Multi-view diffusion for 3D generation. In ICLR, 2024.
[79] U. Singer, S. Sheynin, A. Polyak, O. Ashual, I. Makarov, F. Kokkinos, N. Goyal, A. Vedaldi, D. Parikh, J. Johnson, and Y. Taigman. Text-to-4D dynamic scene generation. In /CML, 2023. 1
[80] J. Sohl-Dickstein, E. A. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In JCML, 2015. 3
[81] J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. In ICLR, 2021. 5
[82] Q. Sun, Z. Guo, Z. Wan, J. N. Yan, S. Yin, W. Zhou, Liao, and H. Li. EG4D: Explicit generation of 4D object without score distillation. In ICLR, 2025. 3
[83] W. Sun, S. Chen, F. Liu. Z. Chen, Y. Duan, J. Zhang, and Y. Wang. DimensionX: Create any 3D and 4D scenes from single image with controllable video diffusion. arXiv preprint arXiv:2411.04925, 2024.
[84] Z. Sun, T. Wu, P. Zhang, Y. Zang, X. Dong, Y. Xiong, D. Lin, and J. Wang. Bootstrap3D: Improving multi-view diffusion model with synthetic data, arXiv preprint arXiv:2406.0000 00093, 2024.
[85] S. Szymanowicz, C. Rupprecht, and A. Vedaldi. Splatter image: Ultra-fast single-view 3D reconstruction. In CVPR, 2024.
[86] J. Tang, Z. Chen, X. Chen, T. Wang, G. Zeng, and Z. Liu. LGM: Large multi- gaussian model for high-resolution 3D content creation. In ECCVV 2024. 3, 6, 7, 31, 32, 36
[87] J. Tang, R. Lu, X. Chen, X. Wen, G. Zeng, and Z. Liu. InTeX: Interactive text-to-texture synthesis via unified depth-aware inpainting, arXiv preprint arXiv:2403. / 1878, 2024. 26
[88] J. Tang, J. Ren, H. Zhou, Z. Liu, and G. Zeng. DreamGaussian: Generative gaussian splatting for efficient 3D content creation. In ICLR, 2024. 1
[89] Tang, N. Ruiz, Q. Chu, Y. Li, A. Holynski, D. E. Jacobs, B. Hariharan, Pritch, N. Wadhwa, K. Aberman, and M. Rubinstein, RealFill: Reference-driven generation for authentic image completion.
ACM TOG, 43(4), 2024. 3, 6, 28
[90] V. Voleti, C.-H. Yao, M. Boss, A. Letts, D. Pankratz, D. Tochilkin, C. Laforte, R. Rombach, and V. Jampani. SV3D: Novel multi-view synthesis and 3D generation from single image using latent video diffusion. In ECCV, 2024. 36
14

---

[91] H. Wang, X. J. Li. R. A. Yeh, and G. Shakhnarovich. Score jacobian chaining: Lifting pretrained 2D diffusion models for 3D generation. In CVPR, 2023. 1
[92] P. Wang and Y. Shi. ImageDream: Image-prompt prompt multi-view diffusion for 3D generation. arXiv preprint arXiv:2312 02201, 2023. 2, 3,6 6
[93] Wang. D. Xu, Z. Fan, D. Wang. S. Mohan, F. landola, R. Ranjan. Li, Q. Liu. Z. Wang. and V. Chandra. Taming mode collapse in score distillation for text-to-3D generation. In CVPR, 2024, 25
[94] Y. Wang. X. Wang. Z. Chen, Z. Wang. F. Sun, and Zhu, Vidu4D: Single generated video to high-fidelity
4D reconstruction with dynamic gaussian surfels. In NewrIPS, 2024. 3
[95] Y. Wang. F. Hong. S. Yang, L. Jiang, W. Wu, and C. C. Loy. MEAT: Multiview diffusion model for
human generation on megapixels with mesh attention. In CVPR, 2025. 2
[96] Z. Wang, C. Lu, Y. Wang, F. Bao. C. Li. H. Su, and J. Zhu. PromificDreammr: High-fidelit and diverse text-to-3D generation with variational score distillation. In NeurlPS, 2023. 1,
[97] Z. Wang. Y. Wang, Y. Chen, C. Xiang, S. Chen, D. Yu, C. Li, Su, and J. Zhu. CRM: Single image to 3D textured mesh with convolutional reconstruction model. ECCV, 2024.
[98] D. Watson, W. Chan, R. Martin-Brualla, J. Ho, A. Tagliasacchi, and M. Norouzi, Novel view synthesis with diffusion models. In ICLR, 2023. 3
[99] M. Wei, J. Zhou, J. Sun, and X. Zhang. Adversarial score distillation: When score distillation meets GAN. In CVPR, 2024. 3
[100] S.-T. Wei, R.-H. Wang. C.-Z. Zhou, B. Chen, and P.-S. Wang. OctGP: Octree-based multiscale autoregressive models for 3D shape generation. In SIGGRAPH, 2025. 3
[101] K. Wu, F. Liu, Z. Cai, R. Yan, H. Wang. Y. Hu, Y. Duan, and K. Ma. Unique3D: High-quality and efficient 3D mesh generation from single image. NeurlPS, 2024. 3
[102] Wu, Y. Lin, Y. Zeng, Zhang, Xu, Torr, X. Cao, and Yao. Direct3D: Scalable image-to-3D generation via 3D latent diffusion transformer. In NeurlPS, 2024. 2,3, 25, 32, 35
[103] Z. Wu, C. Yu, Y. Jiang, C. Cao, F. Wang. and X. Bai. SC4D: Sparse-controlled video-to-4D generation and motion transfer. In ECCV, 2024. 3
[104] J. Xiang. Z. Lv, S. Xu, Y. Deng, R. Wang, B. Zhang, D. Chen, X. Tong, and J. Yang. Structured 3D latents for scalable and versatile 3D generation. In CVPR, 2025. 2, 2, 7, 25
[105] G. Xiao, T. Yin, W. T. Freeman, F. Durand, and S. Han. FastComposer: Tuning-free free multi-subject image generation with localized attention. IJCV, 2024. 3
[106] D. Xie, Li, Tan, X. Sun, Z. Shu, Zhou, S. Bi, Pirk, and A. Kaufman. Carve3D: Improving multi-view reconstruction consistency for diffusion models with RL finetuning. CVPR, 2024.
[107] K. Xie, J. Lorraine, T. Cao, J. Gao, J. I Lucas, A. Torralba, S. Fidler, and X. Zeng. LATTE3D: Large-scale amortized texh-to-enhanced3D synthesis. In ECCV, 2024. 1
[108] Y. Xie, C.HH. Yao, V. Voleti, H. Jiang, and V. Jampani. SV4D: Dynamic 3D content generation with multi-frame and multi-view consistency. In ICLR, 2025. 3, 9, 38
[109] D. Xu, H. Liang, N. P. Bhatt, H. Hu, H. Liang, K. N. Plataniotis, and 7.. Wang. Comp4D: L.L.-guided compositional 4D scene generation. arXiv preprint arXiv:2403.16993, 2024. 3
[110] J. Xu, W. Cheng, Y. Gao, X. Wang. S. Gao, and Y. Shan. InstantMesh: Efficient 3D mesh generation from a single image with sparse-view large reconstruction models, arXiv preprint arXiv:2404.0719), 2024. 2, 3, 25
[111] Y. Xu, Z. Shi, W. Yifan, H. Chen, C. Yang. S. Peng, Y. Shen, and G. Wetzstein. GRM: Large gaussian reconstruction model for efficient 3D reconstruction and generation. In ECCV, 2024. 3
[112] Y. Xu, H. Tan, F. Luan, S. Bi, P. Wang, J. Li, Z. Shi, K. Sunkavalli, G. Wetzstein, Z. Xu, and K. Zhang, DMV3D: Denoising multi-view diffusion using 3D large reconstruction model. In ICLR, 2024.
[113] R. Yan, Y. Chen, and X. Wang. Consistent flow distillation for text-to-3D generation. In ICLR, 2025. 25
15

---

[114] X. Yang, Н. Shi. B. Zhang, F. Yang. J. Wang, Н. Zhao. X. Liu, X. Wang. Q. Lin, J. Yu. L. Wang. J. Xu, Z. He, Z. Chen, S. Liu, J. Wu, Y. Lian, S. Yang. Y. Liu, Y. Yang, D. Wang, J. Jiang, and C. Guo. Hunyuan3D 1.0: A unified framework for text-to-3D and image-to-3D generation. arXiv preprint arXiv:2411.02293, 2024, 2, 3, 25, 32,
[115] Yang. Y. Huang, X. Wu, Y.-C. Guo, S.-H. Zhang, H. Zhao, T. He, X. Liu. DreamComposer: Controllable 3D object generation via multi-view conditions. In CVPR, 2024. 2
[116] Yang, Z. Pan. C. Gu, and L. Zhang. Diffusion": Dynamic 3D content generation via score composition of video and nuew diffusion models. In ICLR, 2025. 3
[117] Yang, J. Teng, W. Zheng, M. Ding, S. Huang. J. Xu, Y. Yang. W. Hong. X. Zhang, Feng, Yin. X. Gu, Y. Zhang, W. Wang, Y. Cheng. T. Liu, B. Xu, Y. Dong. and Tang. Cog VogVideoX: Text-to-video diffusion models with an expert transformer. In ICLR, 2025. 24, 27
[118] C.-H. Yao, Y. Xie, V. Voleti, H. Jiang, and V. Jampani. SV4D 2.0: Enhancing spatio- temporal consistency in multi-view video diffusion for high-quality 4D generation. arXiv preprint arXiv:2503. 16396, 2025. 3
[119] J. Ye, F. Liu, Q. Li, Z. Wang, Y. Wang. X. Wang, Y. Duan, and J. Zhu. DreamReward: Text-to-3 generation with human preference. In ECCV, 2024. 1
[120] H. Yu, C. Wang, P. Zhuang. W. Menapace, A. Siarohin, J. Cao, L. A. Jeni, S. Tulyakov, and H. Lee. 4Real: Towards photorealistic 4D scene generation via video diffusion models. In NewrlPS, 2024. 3, 25
[121] X. Yu, Y.-C. Guo, Y. D. Liang, S.-H. Zhang, and X. Qi. Text-to-3D with classifier score distillation. In ICLR, 2024. 3
[122] B. Zeng, L. Yang, S. Li, Liu, Z. Zhang, J. Tian, K. Zhu, Y. Guo, F-Y. Wang, M. Xu, S. Ermon, and W. Zhang. Trans4D: Realistic geometry- transition for compositional text-to-4D synthesis. arXiv preprint WxXv:2410.00755. 2024. 3
[123] Y. Zeng, Y. Jiang. S. Zhu, Y. Lu. Y. Lin, H. Zhu, W. Hu. X. Cao, and Y. Yao. STAG4D: Spatial-temporal anchored generative 4D gaussians. In ECCV, 2024. 1,3, 8, 3.
[124] B. Zhang, J. Tang, M. Nießner, and P. Wonka. 3DShape2VecSet: A 3D shape representation for neural fields and generative diffusion models. ACM TOG, 42(4), 2023. 2
[125] H. Zhang, X. Chen, Y. Wang, X. Liu, Y. Wang, and Y. Qiao. 4Diffusion: Multi-view video diffussio model for 4D generation. In NeurlPS, 2024.
[126] K. Zhang, S. Bi, H. Tan, Y. Xiangli, N. Zhao, K. Sunkavalli, and Z. Xu. GS-LRM: Large reconstruction model for 3D gaussian splatting. In ECCV, 2024. 3
[127] L. Zhang, Z. Wang, Q. Zhang, Q Qiu, A. Pang. H. Jiang, W. Yang, L. Xu, and J. Yu. CLAY: A controllable large-scale generative model for creating high-quality 3D assets. ACM TOG, 43(4), 2024. 3, 25
[128] Y. Zhao, 7. Yan, E. Xie, I. Hong, 7. Li, and G. H. Lee. Animate124: Animating one image to 4D dynamic scene. arXiv preprini urXiv:2311.144
2023. 3
[129] Z. Zhao, W. Liu, X. Chen, X. Zeng, R. Wang, P. Cheng, B. Fu, T. Chen, G. Yu, and S. Gao. Michelangelo: Conditional 3D shape generation based on shape-image-text aligned latent representation. In NeurlPS, 2023.
[130] Z. Zhao, Z. Lai, Q. Lin, Y. Zhao, H. Liu, S. Yang. Y. Feng, M. Yang. S. Zhang. X. Yang. H. Shi, S. Liu, J. Wu, Lian, F. Yang. R. Tang, Z. He, X. Wang, J. Liu, Zuo, Z. Chen, Lei, H. Weng, J. Xu, Zhu, X. Liu, L. Xu, C. Hu, S. Yang. S. Zhang, Y. Liu, T. Huang, L. Wang. J. Zhang, M. Chen, L. Dong, Y. Jia, Y. Cai, J. Yu, Y. Tang, H. Zhang, Z. Yc, P. Hc, R. Wu, C. Zhang, Y. Tan, J. Xiao, Y. Tao, J. Zhu, J. Xue,
K. Liu, C. Zhao, X. Wu, Z. Hu, L. Qin, J. Peng, Z. Li, M. Chen, X. Zhang, L. Niu, P. Wang. Y. Wang. Fan, X. Zheng, W. Zhuang.
Hc, Liu. Yang. Wang. Y. Liu, J. Jiang. Huang. and C. Guo. Hunyuan3D 2.0: Scaling diffusion models for high resolution textured 3D assets generation. arXiv preprint arXiv:2501.12202. 2025. 2, 3, 25,
H. Kuang,
[131] C. Zheng, Y. Lin, B. Liu, X. Xu, Y. Nic, and S. Hc. RecDreamer: Consistent text-to-3D generation via uniform score distillation. In ICLR, 2025. 25
[132] Zheng, X. Li, K. Nagano, S. Liu, O. Hilliges, and S. D. Mello. unified approach for text- and image-guided 4D scene generation. In CVPR, 2024. 1,3
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the work have been discussed in App. Cn the appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
17

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have discussed all the details to reproduce experimental results in our paper.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
18

---

Answer: [No]
Justification: While the current submission does not contain code implementation, we have included detailed instructions the appendix to provide guidance for reproducing the main experimental results. Also, we promise that we will open-source the data and code after paper acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: ctspps.ccppullcc//eeecc http:pppllccguucceeeeep guides//eeeeee /CodeSSmmmi ssi aeionnolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The training and test details have been specified in the main paper and the appendix, which are enough understand the results.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[No]
Justification:
The paper mainly focuses on qualitative comparisons.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
19

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The compute resources required for reproduce the is specified in App. the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: htttss:/neurt: //tp:r//nne...eeeeeeee www.eers...ppuullllleeeeee cctppblic//aieecceceeeecceeeeeee htt.:///tthiiiuuieeeeeeee ines?
Answer: [Yes]
Justification: respect.
The conducted research conforms with the NeurlPS Code of Ethics in every
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Both positive and negative societal impacts have been discussed in App. G in the appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
20

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not have such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have properly credited and listed the license in App. B in the appendix. The assets are also properly used under the terms of use.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
21

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com /atasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: At the submission time, we have not released the new assets used in the paper. We will well document our data and model at the time we release our data after acceptance.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: We have included the full text of instructions given to the participants in App. A in the appendix, as well as the screenshots of the user study.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: There is no potential risks in the crowdsourcing research conducted for this work.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
22

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: / //nerrurpp.... htt..///eeeeennee//00000000 for what should or should not be described.
23

---

Track, Inpaint, Resplat: Subject-driven 3D and 4D Generation with Progressive Texture Infilling
Technical Appendices and Supplementary Material
In the appendix, we first provide more details on the instructions and interface used in our user study in App. A to show how we foster the faimess and soundness of our user study. Next, in App. B, we illustrate the pipeline of how we construct our dataset for subject-driven 3D/4D generation. In App. C, we carry out more detailed discussions on how our proposed TIRE can co-exist and co- develop with other 3D/4D generation methods, the insights of our method design, together with limitations of our approach. Then, more ablation study is presented in App. E. Afterwards, we provide additional qualitative results in App. F on in-the-wild data, more state-of-the-ee advancements on 3D generation, etc. Moreover, in App. D, we elaborate more implementation details to better support the reproducibility of the proposed method. Finally, the socictal impact of our work is discussed in App. G.
A
User Study
Below we present more details of our user study setup. We use the following instructions for participants at the start of the user study.
For each scene, we use three different methods to generate 4D assets, which are rendered in the forms of videos and placed random orders. We also provide reference image show the appearance of the subject. We would like to invite you to give an overall score from to 10 to measure the quality of the generated results.
There are a few points to consider when providing the scores:
Whether the videos are consistent across different views
Whether different viewpoints of the generated results look alike the subject in the given reference image
The visual quality of the videos (e.g., whether has blurry artifacts, whether the rendered views look realistic)
From the above instructions, note that we do not explicitly tell the participants that we improving on subject-driven 4D generation. Instead, we ask the users to rate the generation results based on the overall quality. fosters the fairness user study and reduces the underlying bias of the subjective preference towards our method. We also include screenshots of our user study interface in Fig. A and Fig. B.
B More Details on Datasets
We show an overview of our dataset construction pipeline in Fig. C. More specifically, train customized text-to-image generation model on each subject following DreamBooth [75], with the backbone being an SDXL [67] model finetuned with LoRA [22]. The link of the dataset is https://githi: tion 4.0 International (CC-BY-.0), Then, we use manually created prompts to generate images with the subjects performing specific activities. Afterwards, we adopt the image-to- -iieeoo model Cog VideoX [20, 117] create an animated video capturing the scene. Altogether, we curate a of 23 animable subjects in the original DreamBooth dataset performing diverse activities.
com/ google/dn hetee//rrebbottheeeeeeooo and the license is Creative Commons Attribu-

---

User Study on 4D Generation
Thanks for participating on the user study of 4D generation!
For each of the scene, we use three different methods to generate 4D assets, which are rendered the forms of videos and placed random orders. We also provide reference subject. would
image the appearance score from (worst quality) 10 (best quality) measure the quality of the generated results.
you give
There are few points to consider when providing the scores:
Whether the videos are consistent across different views
Whether different viewpoints of the generated results look alike the subject in the given reference image
The visual quality of the videos (e.g., whether has blurry artifacts, whether the rendered look realistic)
There 10 questions total, the estimated time for finishing this study 5-6 mins.
Figure A: Screenshot of the instructions of user study. From the instructions, we do not explicitly tell the participants that our project is targeting at subject-driven 4D generation (e.g., in the title we only mention "User Study 4D Generation"). Instead, the users are asked rate the generated results based on the overall quality, which enhances the fairness and reduces the underlying bias of the potential subjective preference towards method.
C Discussions and Limitations
Position of proposed TIRE relative to existing models. Existing state-of-the-art 3D/4D generation models [21, 23, 50, 58, 102, 104, 110, 114, 127, 130] are mostly feed-forward models with native 3D generation achieve superior generation efficiency. Therefore. is no denying that their feed-forward generation schema pioneers the "foundation models" for 3D/4D generation. Per-scene optimization-bans
methods [56, 93, 96, 113, 120, 131]. do suffer from the limitation of efficiency as discussed in the training efficiency part in this section. However, we consider our method as an orthogonal effort towards subject-driven generation which more customized. From the perspective of customers, it is valuable to have an option, supported by our method, to further preserve the identity and refine the generated results produced by the other feed-forward 3D/4D generation methods.
From a methodology perspective, some may argue that, resolve the issues like the blueish shade shown in Fig. 1 in the main paper, we could heuristically design specific rules to filter out problematic data before training our model. However, it is infeasible enumerate all the issues that may lead to data bias during the data filtering stage, as the issue shown in Fig. 1 is only one specific example of the many underlying issues that undermine the robustness of the feed-forward generation models. For example, if we place the rendered views of the generated assets from specific model side by side, as shown Fig. D, it becomes obvious to see that each model suffers from a specific bias pattern for side views and back views. For L4GM, the generated assets suffer from the unrealistic blueish or whitish color patterns similar to the case in Fig. For SV3D, there are whitish tone and blurry texture at the generated viewpoints, which may originate from the imbalance object properties of the dataset used for training the model, or the bias from the network design. For Hunyuan3D-v2.55 although the shape and color tones looks generally reasonable, the appearance is over-smoothed and looks unrealistic. This could be still the consequence using certain rules for data filtering and preprocessing, and the rest of the data gets biased towards the opposite direction. Therefore, it is extremely challenging and laborious to design data filtering principles to address the issue.
Insights and innovations of the proposed TIRE pipeline. Our solution is not simple concatenation of existing methods, as seamlessly combining these works (some of them like video tracking methods are even seldom used in 3D generation) itself is already non-trivial task. Our insight for devising the TIRE paradigm is that we can convert the problem of progressive inpainting for 3D assets to 2D problems, with Track and Inpaint identify and fill the occluded regions step by step with 2D
25

---

Video 4: cat playing with a ball of yam
Reference
4
O
O
Mideo
2nd Video
3rd Video
Figure B: Screenshot of an example question, containing videos generated by three methods placed side by side random order, the reference front view, and the questions score the three generated results.
models, followed by Resplat to fix the consistency in 3D. As a result, pipeline is able to leverage the powerful 2D foundation models that are trained on substantially abundant and more diverse 2D data. Also, compared with other inpainting based texture completion methods like Text2Tex [8] and InTeX [87], our pipeline can perform on any 3D representation including the implicit representations like neural radiance fields [61], while Text2Tex and InTeX require the adopted 3D representation to support surface modeling. Moreover, the three-stage pipeline well orchestrated in logical cascade. For example, the Track stage not only identifies the regions that need to be inpainted, but also
26

---

Text Prompt
DreamBooth
dog Ustening
headsel"
Customized Text-to-Image Model
Image-to-vied video with CogVideoX
Segment
Figure C: Pipeline of constructing data for our DreamBBoth--Dyaant dataset. First, DreamBooth- finetuned [75] customized text-to-image model is trained on several casually collected images of use manually created text prompts generate images with the specific
subject. Afterwards, subjects, Afterwards, a powerful image-to-video model [117] is applied to animate the static images into videos. Finally, we use SAM 2 [71] ]]] to segment the foreground subjects form the source view videos used in our evaluation process.
L4GM
SV3D
Hun2
Figure D: Side and back viewpoints generated by specific models. It is obvious each model suffers from a bias pattern for generating the appearance of the occluded viewpoints.
propagate the corresponding pixels along the tracking path to novel views to mitigate the difficulty of the following Inpaint stage.
Quantitative metrics for 3D/4D subject-driven generation. As shown in Sec. 4.3 the main paper, the quantitative metric adopted in DreamBooth leads to unreasonable results. We visualize an example in Fig. E to demonstrate the failure case of the DINO similarity metric. We can observe that the compared methods Customize-It-33 It-3D and STAG4D have obviously incorrect geometry for this viewpoint which is 135" away from the source view. However, as their shape or appearance looks similar to the reference which is the source view image, they score significantly higher than L4GM and our method, which at least correctly display the back view of the subject. Therefore, we believe that currently there are limitations in the quantitative evaluation generation. better evaluation metric for subject-driven 3D/4D generation needs to be proposed for properly benchmarking the performance of the methods. Potential solutions could be integrating the geometric evaluation with the appearance evaluation that is conducted by the current DINO feature similarity metric. The geometric assessment may pertain to the structural characteristics of the foreground of the generated images. We leave the design of proper evaluation metrics on subject-driven 3D/4D generation future work.
the subject fidelity of 3D/4D
Training efficiency. Our method takes around 100 mins on single NVIDIA A100 GPU (when considering the memory consumption, an NVIDIA Quadro RTX 6000 RTX 4090 with 24GB
27

---

Reference
Customize-It-30
STAG4D
L4GM
Ours
DINO Similarity (t)
0.4730
0.6844
0.3306
0.3853
Figure E: An example of DINO (ViT-S/16) similarity scores of rendered viewpoint which is 135 away from the source view across different methods. The baseline methods Customize-lt-3D and STAG4D are obviously incorrect regarding geometry but get higher scores for DINO similarity.
will suffice), which is not very efficient. This issue is inherited from RealFill [89] which our method's pipeline based on. However, since our solution focuses more on subject-driven 3D/4D generation, work is orthogonal effort to existing methods like L4GM [73], Turbo3D [23], etc. that work towards increasingly faster feed-forward 3D/4D generation pipelines. Moreover, personalization models are also in the trend of becoming feed-forward models [57, 76], without the need of finetuning on every subject like DreamBooth. Therefore, we believe that there will be more efficient subject-driven 2D inpainting models in the future that greatly improve the efficiency of our approach.
Why there are no specific operations designed for 4D generation in the temporal dimension? Subject-driven generation, in the current context, mainly refers to the point that the appearance the generated asset needs to conform to the reference image, which is often a static attribute. Therefore, temporal reasoning is not needed in most cases. Also, in our current implementation, the method L4GM [73] that builds upon for 4D generation essentially generates sequence of 3D Gaussians as mentioned in Sec. 3.1, which makes 4D generation, in the current situation, can be understood do acknowledge that
as the composition of series of 3D generation processes. Nevertheless, there exist significant challenges specifically for 4D generation compared with 3D generation, and there more complicated cases that customization in 4D generation also becomes important and especially challenging. For example, the reference objects shown in the given video may contain temporal patterns that change their appearance over time, and want to generate 4D assets that faithfully preserve this property. We leave this as an interesting future direction to explore.
D
Additional Implementation Details
D.1 Training Details
As mentioned in the main paper, during the inpainting process, as do not want to significant change the original structure of the observations from novel views, we perform denoising in the first 30% of the denoising schedule following similar practice as [33]. Therefore, the original structure of the images will not be destroyed after the inpainting process. Since we set the denoising schedule to only 30%, the original color of the images from the other views cannot be too deviant, which sometimes may not be satisfied with the original multi-view observations obtained by IrageDream. Therefore, we leverage another multi-view diffusion model Wonder3D [54], which empirically has inferior geometry but superior color than ImageDream, to provide guidance for color. Specifically, splat multi-view observations from both ImageDream and Wonder3I) 3D Gaussians with LGM,, and concatenate color features from Wonder3D and other features from ImageDream, followed by another halfway diffusion-deenoisngg process with ImageDream. For progressive inpainting, except for the sweet spot - and the secondary anchor viewpoint +900, the other viewpoints are (+400, + + - +130", 1500 +170%. Each training stage consists of 300 training steps before advancing to the next stage. When adding the newly inpainted sample to the training data, we perform the same augmentation strategy when there only one single source view image in the training set. We follow the other training parameters in RealFill [89] for finetuning the stable diffusion inpainting model (e.g., batch size 16, LoRA rank learning rate is 2e-4 for U-Net and 4e-5 for the text encoder). For the halfway diffusion-denoising process used after fixing the color of the rough assets with Wonder3D and before resplatting the pixels back to 3D, we find that the same denoising schedule of 0.3 the inpainting stage generally works well in both cases.
28

---

D.2 VLM-based Evaluation Details
We follow the concept preservation evaluation proposed in DreamBench++ [66] to prepare the prompts for VLMs to assign scores for the identity preservation quality between the reference image and the rendered view of the generated asset. The prompts are shown follows.
### Task Definition
You will be provided with an image generated based on reference image.
As an experienced evaluator, your task is to evaluate the semantic consistency between the subject of the generated image and the reference image, according to the scoring criteria.
### Scoring Criteria
It is often compared whether two subjects are consistent based on four basic visual features:
1. Shape: Evaluate whether the main body outline, structure, and proportions of the gener- ated image match those of the reference image. This includes the geometric shape of the main body, clarity of edges, relative sizes, and spatial relationships between various parts composing the main body.
2. Color: Comparing the accuracy and consistency of the main colors generated in the image with those of the reference image. This includes saturation, hue, brightness, and whether the distribution of colors is similar to that of the subject in the reference image.
3. Texture: Focus on the local parts of the RGB image, whether the generated image effectively captures fine details without appearing blurry, and whether possesses the required realism, clarity, and aesthetic appeal. Please note that unless specifically mentioned the text prompt, excessive abstraction and formalization of texture not necessary.
4. Facial Features: If the evaluation is of a person or animal, facial features will greatly affect the judgment of image consistency, and you also need to focus judging whether the facial area looks very similar visually.
### Scoring Range
You need to give specific integer score based on the comprehensive performance of the visual features above, ranging from to
Very Poor (0): No resemblance. The generated image's subject has no relation to the reference.
Poor (I): Minimal resemblance. The subject falls within the same broad category but differs significantly.
Fair (2): Moderate resemblance. The subject shows likeness to the reference with notable variances.
Good (3): Strong resemblance. The subject closely matches the reference with only minor discrepancies.
Excellent (4): Near-identical. The subject of the generated image is virtually indistinguish- able from the reference.
### Input Format
Every time you will receive two images, the first image is a reference image, and the second image is the generated image.
Please carefully review each image of the subject.
### Output Format
Score: |Your Score]
You must adhere the specified output format, which means that only the scores need to be output, excluding your analysis process.
29

---

Reference
10
20 30
Figure F: Ablation study on the degree of progressiveness during the Inpaint stage. Our current choice of degree of progressiveness (209) strikes balance between inpainting quality and efficiency. Smaller degree of progressiveness (10%) yields decent results but slows down the whole inpainting process, while larger degree of progressiveness (30") brings noticeable degradation the inpainting quality.
Reference
Base Model
w/o Inpainting
Ours
Figure G: Ablation study on the necessity of having the inpainting process. The color and texture of assets without our inpainting process would be far from having satisfactory quality and decent identity preservation.
E
More Ablation Study
E.1 Degree of Progressiveness for Inpainting
We empirically choose 20 as the degree of progressiveness the inpainting stage as described in Sec. 3.3 for striking balance between the inpainting quality and the algorithm efficiency. We display the ablation on selecting smaller or larger degree of progressivenes: (10" and 30") in Fig. F to show the effect of different choices on progressiveness. Basically, for the degrees of progressiveness smaller than the current choice, the visual results are very similar to the current results, but we need additional steps to calculate the inpainting masks for the increased number of training stages. For the degrees of progressiveness larger than the current choice, we could observe that the regions (especially for those that are far away from the reference view, e.g., the regions that are close to the back view for the target views of +-90") are inpainted worse than the current implementation. Our current choice of 20" of progressiveness balances between the number of frames that need for tracking, and the difficulty of inpainting the unseen regions that are far away from the source view.
30

---

Table A: Quantitative comparisons the assets generated by Hunyuan3D-vv.55 [38] and our enhance- ment built upon it. Best is marked as bold.
Methods
VLM--used soores (1)
GPT-4o
OpenAl oé mini
Gemma 27B
Gemini 2.0 Flash Qwen2,5-VL-
Mistral Small-3, 3-1- Instract
Avenge
1ummuunn3-vv x2.5 TIEE on I 52.5 (Oun)
1.614
1
2,096
1.533
1,780
1.501
1:703
1,750
1,712
2,092
1,099
1.99
1.707
1.795
Table B: Quantitative comparisons the assets generated by L4GM [73] and our enhancement built upon it on the in-the-will wild data displayed on the L4GM official website. Best is marked bold.
scures
GTT--o
OpenAl of-mini
Gemma 3 27B
Gemini 2.0 Flush
Qwen2.5-VL-7B
Mistral-Sman 1.1ell..---11 211B-Insrrctt
Average
L4GM TIRE on L4GM (Our)
2.743
2.441 2,478
2.600 2,684
2,022 2,110
2.104
2.125
2.346
2.601
2,191
2.163
2.371
E.2 Necessity of the Inpainting Process
As mentioned in App. D, leverage another multi-view diffusion model Wonder3D [54] which empirically has superior color than the base 3D generation model LGM [86] that we mainly use in the paper, to roughly fix the color first before applying necessity of our method by demonstrating how much identity is restored with our inpainting process. that without inpainting, there are many obvious flaws from the results. For
method. We would like show the
In Fig. G. example, on the right view of the scene, the dog's nose has a large grayish patch. The dog's body is also covered with messy dark brown patterns. Therefore, is still far away from good 3D asset for decent identity preservation. After the inpainting is done, the large grayish patch on the dog's s nose gets infilled with reasonable color, and the textures on the dog's 's body gets more realistic and coherent with the given source view.
F
More Experimental Results
For an overview of this section, first we show the comparison on in-the-wild data to show the robustness of our method in Sec. F.1. Then, we illustrate that our method is generally applicable all types of 3D/4D generation methods by showing its application on one of the most recent image-to-3D methods. Next, comparisons with more state-of-the-arr advancements are demonstrated in Sec. F.3 to show that recent advancements are generally not robust enough to produce personalized 3D assets that well preserve the identity the subjects. Afterwards, more comparisons on image-to-3D and video-to-4D generation are presented in Sec. F.4. Finally, the comparisons with SV4D is displayed in Sec. F.5 due to different pose settings.
F1 Comparisons on In-the-wild Data
We additionally demonstrate the performance of our model on in-the-wild data, which are the examples displayed on the official wehpage L44MM [73], their validation dataset has not been released. The comparison of our method against L4GM is shown in Fig. H and Tab. B. We can observe that our method more faithfully preserves the subject fidelity in the generated 4D assets and achieves superior overall quality compared with the baseline.
F.2 General Solution to 3D/4D Generation Methods
Our solution is plug-in solution that is generally applicable all types of 3D/4D representations and methods. This benefit is a natural outcome from our method design of fully leveraging 2D tools to solve this 3D task. To apply on any 3D/4D methods, it is as simple as replacing the process of generating the initial assets in our pipeline with the 3D/4D generation that we want to improve upon. We show the improvement
our method upon the current state-of-the-art advancement in 3D generation --v2 [38] in Fig. I and Tab. A. We can observe that our method makes refinement to the texture of the generated 3D assets of Hunyuan3D-vv.55 yielding results that better match the identity of the reference images.
31

---

F.3
Comparisons with More State-of-the-art Methods
We showcase the comparison with more saate-ofthhe-ee advancements in Fig. J. including Hunyuan3D. v1.0 [1], SPAR3D [27], and commercial models like Sudo AI' (built upon One-2-3-45++ [49]) and Neural4D (built upon Direct3D [102]). Together with the results of MeshFormer [50], TREL- LIS [104], and Hunyuan3ID-v255 [130] in Sec. 4.2, we can observe that even the most recent advance-
ments in 3D generation still struggle with producing well-rounded personalized 3D/4D contents that well preserve the identity of the subject.
F4 More Qualitative Comparisons on 3D/4D Generation
We include more comparisons on image-to-3D generation with methods Wonder3D [54], SV3D [90]. and LGM [86] in Fig. K. The comparisons demonstrate that our method outperforms existing image- to-3D approaches with better identity preservation and superior geometry accuracy. Additional visual comparisons on video-to-4D generation in Fig. I. to demonstrate the preeeevviieeee appearance and the enhanced quality on geometry produced from our method, indicating the effectiveness of our approach.
F.5 Comparisons with SV4D
We display the comparison with SV4D [108] Fig. M. The reason that we show the SV4D results separately in the supplementary material is that the official code of SV4D has only released the multi-view video generation part, without the following 4D optimization step. Therefore, only certain number of selected views can be rendered with its official code. For simplicity, the viewpoints selected for our method are close viewpoints but not the identical ones. However, we can still observe from the qualitative comparisons that SV4D fails to produce decent results regarding both texture and geometry.
G Societal Impact
We expect work to have meaningful and positive impact the society. We sincerely wish that our method can ignite the creativity inside people to design personalized 3D/4D assets that serve versatile purposes. Also, hope that our work highlights critical and often overlooked challenge during the evolution of 3D/4D generation: the task of generating identity-preserved 3D/4D assets for personalized applications still remains unsolved. By bringing attention this gap, aim to raise awareness within the research community and encourage further exploration in this direction.
Potential negative societal impact. Our work is likely the same as other research on data generation regarding potential negative societal impact with the risk digital forgery. Also, the technique is mistakenly used, copyright and ethical issues may occur.
http::/wwww....rr...cococ//oat// www.s/tdo///oamggeoooooooo to-
32

---

Reference Video
L4GM
Ours
L4GM
Ours
L4GM
Ours
Figure H: Qualitative comparisons on the in-the-wild videos which are displayed on the official website of L4GM [73]. The visualizations demonstrate that our results more faithfully reflect the identity the given subjects, yielding better overall quality.
33

---

Reference
Hun2
Ours
Hun2
Ours
Figure I: Qualitative results showing the improvement of our method when applying the most advanced image-to-3D method Hunyuan3D-v2.5 [38]. We can observe that our method produces superior texture of the generated assets which better match the identity of the reference image.
34

---

Sudo Al
Ours
Neural4D-v2.0 (Bullt upon Direct30)
Ours
(Built upon One-2-3-45+.)
Source View
Source View
SPAR3D
Hunyuan3D-v1.0
Ours
Source View
Source View
Figure J: Qualitative comparison with more advanced methods including Hunyuan3D-vI.0 [114], SPAR3D [27]. Sudo AI (built upon One-2-3-45++ [49]), and Neural4D (built upon Direct3D [102]). Even the recent advancements in 3D generation cannot satisfactorily handle the personalized 3D
generation challenge.
35

---

Source View
SV3D
Wonder3D
LGM
Ours
Figure K: Qualitative comparison with SV3D [90], Wonder3D [54], and LGM [86]. Compared against other method in the image-to-3D setting, our method achieves better preserves the identity of the reference image, and also reaches superior quality on geometry.
36

---

Reference Video
Customize-It-3D
STAG4D
L4GM
Ours
Reference Video
L4GM
Ours
L4GM
Ours
Figure L: Additional qualitative results of the comparison between our method and the baselines Customize-It-31 [24], STAG4D [123], and L4GM [73]. Compared with other methods, our solution achieves superior subject fidelity along with improved geometry on the generated assets, due to our design in our three-stage stage framework that fosters cross-vies consistency.
37

---

Reference Video
SV4D
Ours
SV4D
Ours
SV4D
Ours
Figure M: Qualitative comparison with SV4D [108]. Since only a certain number of selected views be rendered with the official code of SV4D, we choose the close viewpoints but not the identical ones for comparison for the sake of simplicity. Nevertheless, it is still obvious that SV4D has inferior performance compared with our method in the perspectives of both appearance and geometry.
38


Reviews:
[{"review_id": "mEwcHCHsTM", "reviewer": "Reviewer_jNcb", "summary": "This paper tries to address the problem of 3D and 4D generation while preserving the subject identity. They first use off the shelf point tracker to find out regions that need to be inpainted and then rerun 3D reconstruction to obtain better 3D/4D generation. They compare the proposed pipeline with some existing 3D/4D reconstruction methods and show that it can indeed improve the unnatural back side views.", "questions": "- I am mainly concerned with the limited scope of this paper.. Authors motivates this method by showing that some methods generate unrealistic backside views which are not faithful to the input views. The example comes from LGM, but what is the real reason for this? As far as I know, this is due to the multi-view diffusion stage of ImageDream. A simple way to address this is to fine tune ImageDream with better images. If full model fine tuning is too expensieve, LoRA based fine tuning could also be considered. In this regard, the pipeline is designed to address a problem specific to one particular model. In order to actually show this method is general, can the authors demonstrate: \n1. The backside artifact/identity not preserving problem is a general problem that lots of single image to 3D/4D generation models have? e.g. if the root cause of LGM is due to ImageDream, then one can simply address it by using another multi-view diffusion models. e.g. Zero123++. In fact Gaussian reconstruction model (GRM, Xu et al.) can take multi-views output from different models. \n2. If the problem does exist in lots of models, can the authors show that the proposed pipeline is general to fix this type of problems? e.g. can this method also take other multi-view/3D generation models as input and improve their artefacts? \n\n\n\n- Experiment evaluation is not solid. The paper claims to address both 3D and 4D generation. Both tasks are not properly evaluated and the results are not convincing. \n1. For 3D, why is there only qualitative result? The selected examples from TRELLIS and HY3D do not look like the actual performance one would see for object reconstruction. Quantitative evaluation with at least 100 examples is needed, i.e. via CLIP score or user study, or more advanced 3D generation metrics (e.g. MET3R, CVPR'25). \n2. For 4D, why is SV4D not compared is the user study, and also table 1? This is related to the first point, does SV3D/SV4D actually have the problem of artifacts in the backside views?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 1, "originality": 1, "strengths_and_weaknesses": "**Strength**\nThe paper is overall well written, it is easy to follow the whole pipeline. \n\n**Weakness**\n- Significance: the problem this paper tries to address seems to be very specific to LGM -- the backside views do not look natural and faithful to frontview, same for LG4M. The root cause of this is actually comes from ImageDream which generates blueish images in the backside views. I am not sure this is a really meaningful problem, as this might be largely due to data issue. hence can be trivially solved with better image renderings and fine tune the model. \n- Originality. The whole pipeline is basically combining several blocks together and making the system work. There is not much technical contribution. \n- Quality. The evaluation is not solid. it is not convincing that the image to 3D is actually better than TRELLIS or HY3D. only a few qualitative examples are shown.", "was_revised": true, "final_justification": "Before the rebuttal stage, I was mainly concerned that the approach from this paper focuses on a small problem and the pipeline is not general. Author response show that it is a common problem for lots of models and their pipeline does improve from them, by showing quantitative evaluation results. Therefore, I am willing to improve my score and urge the author to add the additional explaination and results to the revision.", "rebuttal": "We sincerely appreciate the detailed feedback from you for our work! We provide the following clarifications in response to your concerns:\n\n---\n\n**W1 & Q1-1: Significance and the scope of the paper**\n\nWe would like to point out that the problem we are solving is ***not*** specific for certain models of LGM/L4GM. This issue still exists for other models, and even for the most recent 3D methods like Hunyuan3D-2.5. The blueish effect, as shown in Fig. 1, is just an example of how the generated views can look different from the identity of the source view. Solving the dataset bias, as suggested by the reviewer, is actually very challenging to accomplish. We have some arguments in Lines 651-655 about the infeasibility of enumerating every bias issue during dataset filtering. \n\nTake the most recent method Hunyuan3D-2.5 as an example, they already have careful dataset preprocessing steps for optimizing the appearance for back views and side views. However, if playing with the Hunyuan3D-2.5 demo with some in-the-wild images, we can observe the issue of the generated 3D assets having unnaturally smooth and textureless appearances, along with a whitish tone. It may be a result of the data filtering strategy used in Hunyuan3D-2.5, as the data cleaning algorithm filters out \"bad\" data according to certain rules. At the same time, it inevitably gets biased towards other issues that are the opposites of the data properties being filtered out. \n\nWe have prepared intriguing qualitative results for the reviewers. However, due to the policy of the rebuttal not allowing for PDF or other types of media, we choose to describe the qualitative comparisons in detail to the reviewer. *The qualitative comparisons requested for the following questions are addressed in the same way.*\n\n\n> ***Description of Visual Comparisons***\n\n> For the most recent advancement of Hunyuan3D-2.5, we place 5 samples generated by the model side by side, and it becomes obvious to see that for the side views and back views of the object, although the shape and color tones looks generally reasonable, the appearance is over-smoothed and looks unrealistic. This could be the consequence of using certain rules for data filtering and preprocessing, and the rest of the data gets biased towards the opposite direction. \n\n\n\n\nBesides the dataset bias issues which are already very challenging to solve as mentioned above, there are other issues that result in identity mismatch of the generated back views and source views. For example, **(1)** the prior or bias introduced in the optimization target when training the 3D/4D generation model, and **(2)** the inaccurate boundaries of the reference view of the object, when segmenting the object out from an in-the-wild image/video. Therefore, resolving the issue of unfaithful generated novel view appearance is very challenging. Our paper provides a \"use-as-needed\" solution that can be applied whenever the user feels that the back and side views of the generated assets need to be enhanced. Thus, we believe that our paper can be broadly useful for various types of 3D generation methods to enhance their quality regarding identity preservation. \n\n\n---\n\n**Q1-2: Can our model serve as a general solution to fix the problem?**\n\nYes! This is actually a strength of our proposed method, as our solution is ***representation-agnostic***. It is because our progressive texture infilling is completely performing in 2D, leveraging the powerful 2D tools for the 3D task. Therefore, for handling any 3D generation methods with any 3D representations, our pipeline is the same, which demonstrates the general applicability and broad scope of our method. More specifically, we will first render views from the 3D representation, and then use *Track* to identify the infilling mask, *Inpaint* to infill the unseen regions, and *Resplat* to unproject the 2D observations back to 3D. Specifically, we choose Wonder3D which uses 3D mesh as the representation to demonstrate the general capability of our model. The description of the qualitative results is shown below:\n\n\n> ***Description of Visual Comparisons***\n\n> We use the \"dog eating apple\" example shown in Fig. 3 for easy reference. For the results from Wonder3D, the back and side views have weird dark brown patterns mixed with the orange color of the dog. We render from different viewpoints and apply our method to progressively inpaint the scene. Actually, 3D mesh as representation would even make the *Track* stage easier as 3D mesh generally has less floating artifacts than other representations like NeRFs or 3D Gaussians (more discussions on this point can be referred in **Reviewer AB6n's W1 & Q1: Which 3D representations are more suitable?**). After our method is applied, the weird dark brown pattern issue gets mitigated, and thus improves the identity preservation of the generated asset.\n\n\n \n---\n\n\n\n**W2: Originality**\n\nThe originality of our paper comes from our innovation to leverage the powerful 2D models like tracking and inpainting tools for 3D generation tasks. 2D models are trained on massive images and videos, which can come to rescue for the 3D field that has relatively scarce data. Also, as acknowledged by other reviewers, the utilization of strong 2D generative models (Reviewer aG8d) is novel and interesting (Reviewer HPe1). Specially, our creative backward tracking (Reviewer AB6n) and progressive personalized inpainting is a solid and interesting idea that can also be useful in other contexts (Reviewer aG8d). \n\nAlso, seamlessly combining these methods is a non-trivial job. For example, the *Track* stage not only identifies the regions that need to be inpainted, but also propagates the corresponding pixels along the tracking path to novel views to mitigate the difficulty of the following *Inpaint* stage, and our backward tracking is designed for better inpainting quality in the *Inpaint* stage. Therefore, our three-stage pipeline is delicately designed and well orchestrated.\n\n\n\n---\n\n**W3 & Q2-1: Quality comparison with existing methods**\n\nThanks for the suggestions! Also suggested by Reviewer HPe1, we follow DreamBench++ [a] to adopt MLLM-based evaluation on identity preservation and the generation quality. Besides GPT-4o which is adopted in DreamBench++, we also use five other vision-language models (VLMs) with different architectures and sizes to foster the reliability of the evaluation. The results are reported below with 184 view samples:\n\n\n|Methods|GPT-4o ($\\uparrow$)|OpenAI o4-mini ($\\uparrow$)|Gemma 3 27B ($\\uparrow$)|Gemini 2.0 Flash ($\\uparrow$)|Qwen2.5-VL-7B ($\\uparrow$)|Mistral-Small-3.1-24B-Instruct ($\\uparrow$)|Average ($\\uparrow$)|\n|:---|:---:|:----:|:----:|:----:|:----:|:----:|:---:|\n|TRELLIS|1.332|1.426|1.870|1.402|1.596|1.228|1.476|\n|Hunyuan3D-2.5 | 1.614 | 1.690 | 2.098 | 1.533 | 1.780 | 1.501| 1.703  |\n|TIRE (Ours) | **1.777** | **1.834** | **2.103** | **1.793** | **1.880** | **1.739** |**1.854**  |\n\nWe can observe that **(1)** our method consistently outperforms the state-of-the-arts TRELLIS and Hunyuan3D-2.5 in the quantitative evaluation regarding identity preservation and generation quality; **(2)** different VLMs share consistent scoring rank for different models, demonstrating the robust generalization of the adopted evaluation protocol.\n\n[a] Peng et al. DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation. ICLR 2025.\n\n---\n\n**Q2-2: Comparison with SV4D**\n\nThe reason for us to exclude SV4D in the user study and Tab. 1 in our original manuscript is that the official code of SV4D has only released the multi-view video generation part with 21 pre-defined viewpoints, without the following 4D optimization step. Therefore, only a certain number of selected views can be rendered with its official code, so we choose to pick the closest viewpoints only for qualitative comparison in the appendix. To resolve the concern, we also evaluate the DINO-I scores in the same setting as Tab. 1 based on these closest viewpoints.\n\n|Methods|DINO-I (ViT-S/16) ($\\uparrow$)|DINO-I (ViT-B/16) ($\\uparrow$)|\n|:---|:---:|:---:|\n|SV4D| 0.5213 |0.5426|\n|Ours| **0.5665** | **0.5815** |\n\nWe can see that SV4D still has inferior performance than our method regarding identity preservation. For qualitative comparison with SV3D/SV4D, we have also prepared informative visual results for the reviewer, and the description of the visualizations is shown below: \n\n\n> ***Description of Visual Comparisons***\n\n> From a group of SV3D/SV4D results on side views and back views placed side by side, we can observe that they often suffer from whitish tone and blurry texture at the generated viewpoints, which may still originate from the imbalance object properties of the dataset used for training the model, or the bias from the network design. It indicates that the problem still exists for SV3D/SV4D.\n\n---\n\nWe would like to sincerely thank you once again for your valuable comments, which have greatly helped improve our work. We hope that the explanation above can well address your concerns!"}, {"review_id": "36FuxhHCJk", "reviewer": "Reviewer_aG8d", "summary": "This paper introduces a pipeline for enhancing the appearance preservation (personalization) in image(video) to 3D(4D). First a 3D(4D) is initialized using ImageDream (and Wonder3D), then three steps are performed: Track - visibility masks are created for the 3D/4D representation with respect to the source view using a video tracker. Inpaint - the masks are progressively inpainted using a personalized inpainting model (similar to RealFill). Resplat - the multi-view images are resplatted using LG(4)M to obtain a 3D(4D) gaussian splatting representation. The pipeline can be seen as an extension for LG(4)M. Experimentally, they compare their method with 6 img-3D models and 3 vid-4D models and also conduct a user study.", "questions": "+ Could you clarify whether the observed appearance improvements are due to the proposed inpainting module or simply the use of Wonder3D for 3DGS color features? An ablation without inpainting would help isolate its effect and increase the score.\n\n+ Can you provide specific examples of inconsistencies that arise without progressive inpainting?\n+ Given that each frame at timestep t is processed independently in the pipeline, how do you address potential temporal inconsistencies across frames in video sequences?\n+ Does the “30% of the denoising schedule” correspond to setting the inpainting strength parameter to 0.3? If so, could this be clarified explicitly in the appendix?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The method improves the shadowy bias of L(4)GM (Figs. 4-5), and corrects some geometry artifacts (Fig. 6), without heavy training, relying only on light inpainter fine-tuning and off-the-shelf models. It can be applied on top of any image-to-multiview (the initialization) and multiview-to-3D (the resplatting) models.\n2. Since 3D/4D generation typically relies on small and scarce 3D/4D datasets (excluding slow SDS-like methods), this approach effectively combines geometry from methods trained on 3D/4D data with appearance from strong 2D generative models.\n3. Progressive personalized inpainting, combined with visibility masks from a video tracker, is a solid and interesting idea that could be useful in other contexts.\n\n**Major weaknesses:**  \n1. Appendix F states that the authors use Wonder3D for 3DGS color features and ImageDream for geometry features, due to ImageDream’s poor color quality. This makes it unclear whether appearance improvements stem from the inpainting module - the main contribution, or simply from using Wonder3D as the image-to-multiview backbone for color. An ablation without inpainting is needed to isolate its impact.\n2. From my understanding, progressive inpainting should not introduce inconsistencies in an image-to-3D setting. Could the authors provide examples of such inconsistencies and clarify how the 30% denoising steps in the image-to-multiview model addresses them? Additionally, since each video frame at timestep t is processed independently, wouldn’t this lead to temporal inconsistencies across frames?\n\nMinor Weaknesses:\n1. Regarding reproducibility: most inpainting pipelines use a strength parameter, and I believe “30% of the denoising schedule” corresponds to setting this parameter to 0.3. It would be helpful to clarify this explicitly in the appendix.\n2. Line 202: “A photo of sks” — It would be more appropriate to denote the rare token as $\\hat{V}$ in the main text, and provide the exact token in the appendix. Additionally, in prior work, rare tokens are often prepended to a general class name. Have you considered extracting a general class (e.g., via BLIP or an LLM) to guide the inpainter more effectively?\n\nTypos:\n           Eq. 7, I believe that term should include a squared norm, please verify.", "was_revised": true, "final_justification": "The rebuttal was able to address my main concerns and i'm happy to raise the score accordingly. Please include that \" the inpainting part indeed plays an indispensable role for achieving good results with decent identity preservation\" in the final revision.", "rebuttal": "We are sincerely grateful for your constructive feedback! We provide the following clarifications in response to your concerns:\n\n---\n\n**W1 & Q1: Ablation without inpainting**\n\nThanks for expressing the concern! Without inpainting, the 3D assets would largely be unready, which are caused by mainly two reasons: \n\n**(1)** The color component from Wonder3D mismatches with the underlying geometry of the 3D assets, which results in incorrect color fulfilling and messy textures for many parts of the 3D assets.\n\n**(2)** The color from Wonder3D is only for guidance and still away from good identity preservation quality. Even for the most recent 3D methods like Hunyuan3D-2.5, the identity preservation still needs to be improved. \n\n\nFor this ablation, we have prepared very interesting qualitative results for the reviewers. However, due to the policy of the rebuttal not allowing for PDF or other types of media, we choose to describe the qualitative comparisons in detail to the reviewer.\n\n\n> ***Description of Visual Comparisons***\n\n> We use the \"dog eating apple\" example shown in Fig. 3 for easy reference. Without inpainting, there are many obvious flaws from the results. For example, on the right view of the scene, the dog's nose has a large grayish patch. The dog's body is also covered with messy dark brown patterns. Therefore, it is still far away from a good 3D asset for decent identity preservation. After the inpainting is done, the large grayish patch on the dog's nose gets infilled with reasonable color, and the textures on the dog's body gets more realistic and coherent with the given source view.\n\nTherefore, the inpainting part indeed plays an indispensable role for achieving good results with decent identity preservation. We will present this ablation result in the final version of our paper.\n\n\n---\n\n\n**W2-1 & Q2: Inconsistencies introduced during inpainting and temporal consistency across frames**\n\nThanks for the insightful comments! Your intuition is mostly correct! With our designed solution of progressive inpainting, we indeed manage to avoid the inconsistencies to a great extent. The remaining inconsistency is mainly caused by the inaccurate tracking results from the tracker. Once the tracker yields a false negative (a position in the target view should be tracked to a valid position in source view, but the tracker gets lost), the inconsistency emerges. Basically, we fail to propagate the pixel values from the source view to the positions of the false negative pixels in the target view. Consequently, since the pixel loses track from the source view, it will be wrongly marked as \"inpainting needed\", which results in inconsistent inpainted appearance. \n\nThe false negatives are most commonly seen at the boundaries of the objects. As we can observe from Fig. 6 in the main paper and Fig. I in the appendix, the inconsistent geometry is most obvious around the object boundaries. \n\n---\n\n**W2-2 & Q3: Temporal consistency for video-to-4D**\n\nIn this work, we are mostly focusing on identity preservation of the generated 3D/4D assets, which is often a static attribute of the subjects regarding their appearance. Therefore, we do not have special designs tailored for temporal consistencies, but we observe that it can be decently handled with the base 4D generation models we adopt. The multiview-to-4D lifting model that we are using, L4GM, enforces temporal consistency from its training process. Specifically, for a given video sequence from the source view, together with the multi-view observations for the first frame generated by our method, the base 4D generation model is capable of generating multi-view dynamic videos with decent temporal consistency.\n\nEssentially, what we propose is a unified solution for both 3D and 4D generation, while for 4D generation we only need to generate multi-view observations for static assets, and pass the tasks of handling temporal consistency to the 4D base models. This unified and general solution grants our method more compatibility with the ever-evolving feed-forward 4D generation models (e.g., Splat4D [a], 4D-LRM [b]) in the future, and can benefit from their strong capability of maintaining temporal consistencies.\n\n\n[a] Yin et al. Splat4D: Diffusion-Enhanced 4D Gaussian Splatting for Temporally and Spatially Consistent Content Creation. SIGGRAPH 2025.\n\n[b] Ma et al. 4D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Time. arXiv:2506.18890.\n\n---\n\n**W3 & Q4: Clarification on denoising schedule**\n\nYes, the understanding from the reviewer is correct! 30% of the denoising schedule corresponds to setting the strength parameter in the inpainting pipeline to 0.3 in practice. We will clarify this explicitly in the final version of the paper. Thanks for the suggestion to make our paper more clear!\n\n\n---\n\n**W4: Text prompts for inpainting**\n\nThanks for the thoughtful comment! We believe that it is definitely a good idea to extract a general class name for guiding the inpainter. The reason for us to not have the general class name in our current version is that we follow the exact solution in RealFill [c], which also did not extract a general class name for guiding the inpainting process. We find this solution also generally works well in our setting, so we keep this simple and general solution in our work. Nevertheless, your comment is indeed very inspiring, as including the scene content or class name for the text prompt would definitely have the potential to further improve the inpainting performance! For the notation of the rare token, we will revise the corresponding text in our final version. \n\n[c] Tang et al. RealFill: Reference-Driven Generation for Authentic Image Completion. ACM TOG 2024.\n\n---\n\n**T1: Typo**\n\nThanks for the careful reading and spotting the typo! Yes, in Eq. 7, a squared norm should be included in the loss term!\n\n\n---\n\n\nWe would like to sincerely thank you once again for your valuable comments, which have greatly helped improve our work. We hope that the explanation above can well address your concerns!"}, {"review_id": "mP4DSTtmNQ", "reviewer": "Reviewer_HPe1", "summary": "The paper proposes a method for generating a 3D model from a single RGB image such that the identity of the subject is preserved. This is done in a progressive manner by first creating a coarse 3D representation using a pre-trained 2D-to-3D model; rendering this representation from novel viewpoints; tracking regions of the rendered views that are not visible in the source image; inpainting these regions using an identity-aware diffusion-based inpainter; and finally, combining all rendered+inpainted views into a single Gaussian representation after enforcing consistency among views.", "questions": "1. The paper does not provide any details about how the iniital 3D model is generated. In particular, what method is used to lift the image from 2D to 3D? \n2. It is not clear to me how the inpainting model can be successfully finetuned on an identity with a prompt such as \"a photo of sks\" which does not include a class (\"a photo of an sks dog/cat/vase\") of the subject. Some more details and justification for the choices made here would be very helpful.\n3. Why don't the authors compare all baselines in the quantitative evaluation stage? Specifically, the SOTA methods Trellis and Hunyuan3D are not included in Table 1.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The method shows fairly good results using pre-trained and fine-tuned components.\n2. The idea of fine-tuning an identity-preserving inpainting model for view synthesis is novel and interesting.\n\nWeaknesses:\n1. The motivation for the task the paper is solving, and what exactly \"identity preservation\" means is not clearly defined in the introduction. This is especially important since \"personalization\" and \"identity preservation\" are terms more commonly used by generative models which use text prompts to generate subjects in novel settings (\"an sks dog in an astronaut suit\"). But such text-based control doesn't seem to be present in the current model. \n2. The qualitative evaluation is limited solely to pictures of cats and dogs. In these cases it is difficult to judge how effectively identity is preserved. Furthermore, the title of the paper implies applicability to general objects, not just animals. Therefore, this should be supported with additional qualitative results on diverse objects. \n3. The paper claims to work for 4D reconstruction based solely on the assumption that all its steps translate to video. But no results on 4D reconstruction are shown. This claim should be toned down. \n4. The evaluation section does not support the claims of the paper. Specifically, the qualitative results of the method, while good, are not clearly better than past work. In fact, they may arguably be worse than Trellis and Hunyuan3D. The quantitative results are also not better. If, as the authors claim, the quantitative metrics are incapable of capturing the improvement of the method, then they should explore or devise metrics that better capture the contribution of identity preservation. Perhaps the authors can refer to the \"VBench\" paper to get a comprehensive list of metrics for generative tasks. Furthremore, the user study does not ask participants to judge identity preservation, which is the main contribution of the paper.\n5. Section 3.1 provides far too much detail about diffusion models that is not directly related to the task at hand. Diffusion models are common enough now for this information to be known beforehand by readers. If absolutely necessary, it should be moved to a supplemental. The additional space can be used for more qualitative results as suggested above.", "was_revised": true, "final_justification": "The paper answered most of my concerns and have shown a clear path towards addressing the remaining questions I had.", "rebuttal": "We sincerely appreciate the constructive feedback you have given for our work! Below, we provide the following clarifications to address your concerns:\n\n---\n\n**W1: More clarification on the task setting**\n\nThanks for the comments! It is true that our setting is a bit different from the traditional personalization setting of using generative models to generate subjects with novel text prompts. In comparison, our setting aims at generating 3D/4D assets that correspond to the identity of the subjects when observing from different views. Both settings are generating something of the specific subject in certain conditions. The traditional personalization setting is \"under certain text prompts\", while our setting is \"under certain viewing directions\", which is the reason for us to name our task \"subject-driven 3D/4D generation\". We will make it more clear in the final version of our paper.\n\n---\n\n**W2: More diverse samples**\n\nWe indeed have evaluations on in-the-wild data as mentioned in Lines 247-249, and the results are shown in Fig. F and the vase example in Fig. G (though only the vase is a non-animal object). The results seem to be biased towards animals because animals are the most common objects that are used for dynamic scenes, as for the examples shown in the L4GM website, animals also take the majority of their chosen objects. However, our work can definitely work on other subjects beyond animals. First, many of our examples are scenes of animals interacting with certain objects like apples and violins, which demonstrates that our method is applicable to other objects to certain extent. Second, we have prepared interesting visual comparisons and want to show the reviewer of how our method is capable of enhancing identity preservation for more general objects. Due to the policy of the rebuttal not allowing for PDF or other types of media, we choose to describe the qualitative comparisons in detail to the reviewer.\n\n> ***Description of Visual Comparisons***\n\n> We would like to describe two cases for demonstrating the capability of our method. Both cases are objects from in-the-wild images. The first case is a house covered with snow on the roof. One obvious issue for other 3D generation methods like SV3D and LGM is that they fail to preserve the property of the covering snow on top of the house, while our method is capable of inpainting the snow pattern on the roof from the back view. The second case is a traffic cone with red and silver stripes. Other 3D generation methods like SV3D and LGM mess up the stripe patterns and produce unnatural colors. With the assistance of our method, the texture and appearance of the traffic cone get well preserved. \n\n\n---\n\n**W3: 4D results**\n\nWe do have 4D reconstruction results in the paper. We would like to kindly point out to the reviewer about our 4D results as follows. Figs. 5, 6 in the main paper and Figs. F, I, J in the appendix show the snapshots of different timesteps from different views for the 4D assets. Also, the user study is conducted in the 4D setting, as presented in Sec. A in the appendix. Participants are shown with a series of videos showing the comparisons of the 4D scenes for scoring. For animable demonstrations, there are also examples in the video in the supplementary material (03:15 - 04:12) for reference. Therefore, we do have abundant 4D results and demonstrations for the paper.\n\n---\n\n**W4-1: Evaluation on identity preservation** \n\nThanks for the suggestion from the reviewer! After checking the VBench paper, we get inspired to use vision-language models (VLMs) for evaluating the identity preservation capability of the models. We follow the setting of DreamBench++ [a] for evaluation, and use a variety of VLMs with different architectures and sizes to ensure the reliability and soundness of the evaluation. The results are shown as follow:\n\n\n|Methods|GPT-4o ($\\uparrow$)|OpenAI o4-mini ($\\uparrow$)|Gemma 3 27B ($\\uparrow$)|Gemini 2.0 Flash ($\\uparrow$)|Qwen2.5-VL-7B ($\\uparrow$)|Mistral-Small-3.1-24B-Instruct ($\\uparrow$)|Average ($\\uparrow$)|\n|:---|:---:|:----:|:----:|:----:|:----:|:----:|:---:|\n|TRELLIS|1.332|1.426|1.870|1.402|1.596|1.228|1.476|\n|Hunyuan3D-2.5 | 1.614 | 1.690 | 2.098 | 1.533 | 1.780 | 1.501| 1.703  |\n|TIRE (Ours) | **1.777** | **1.834** | **2.103** | **1.793** | **1.880** | **1.739** |**1.854**  |\n\nWe can observe that **(1)** our method consistently outperforms the state-of-the-arts TRELLIS and Hunyuan3D-2.5 in the quantitative evaluation regarding identity preservation and generation quality; and **(2)** different VLMs share consistent scoring rank for different models, demonstrating the robust generalization of the adopted evaluation protocol.\n\n\n[a] Peng et al. DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation. ICLR 2025.\n\n\n---\n\n**W4-2: User study on identity preservation**\n\nIdentity preservation is actually already taken into account in the user study. In the instruction of the user study, as shown in Sec. A in the appendix, we can see that there is the instruction *\"Whether different viewpoints of the generated results look alike the subject in the given reference image\"* (Lines 623-624) that informs the participants that whether the identity of the generated novel views match the given source view is a crucial factor for the scores. Therefore, we do ask the participants to focus on identity preservation, it is just we ***avoid to explicitly*** ask them to specifically focus on this point. \n\nThe reason for us to not explicitly emphasize this point is that we want to foster the fairness of the user study. If we explicitly mention that our project is about identity preservation, the outcome will be obvious because our method is designed to fix this issue. It will make participants consciously feel in preference of our method. Therefore, our current user study setting is more fair, and can reflect the overall quality of all the methods. \n\n---\n\n**W5: Diffusion model details in Section 3.1**\n\nThanks for the suggestion! The original reason for us to have a relatively long Sec. 3.1 is because we want to present the equations for diffusion models, especially Eqs. 4 and 5, that are needed for reference when introducing the inpainting loss (Eq. 7) and mask-aware multi-view refinement (Eq. 8). We will definitely shorten the preliminary for diffusion models and present more interesting qualitative results in the main paper!\n\n\n---\n\n**Q1: Initial 3D model generation**\n\nThanks for pointing out! In our current implementation, we use LGM for lifting the image from 2D to 3D. Actually, the method for lifting from 2D to 3D can be arbitrarily chosen, because our tracking and inpainting infill the unobserved regions with only 2D models. Therefore, for any chosen method to lift the image from 2D to 3D, we can always render multi-view observations from whatever 3D representation is used, and apply our algorithm afterwards.\n\n---\n\n**Q2: Prompts used during progressive inpainting**\n\nWe thank the reviewer for the valuable question! The inpainting process can actually work well without the class name of the subject! The justification for us to choose this implementation is to follow the solution in RealFill [b], which demonstrates that the inpainting process is still feasible without having the object name or the scene description in the text prompts. We keep this setting in our work, so it serves as a more general and simpler solution for inpainting the scene. The reason for the model being able to inpaint without the guidance of the text prompt is that the special token $\\hat{V}$ (or *sks*) is supposed to be optimized for capturing the content of the scene, instead of only representing the identity of the object. This is because the only task the model needs to focus on is to inpaint this specific scene, instead of generating the object in novel text prompts like the traditional setting mentioned in your previous comment of ***W1: More clarification on the task setting***. Therefore, the model still works well without mentioning the class name or concrete contents of the scene.\n\nNevertheless, your comment is indeed very inspiring, as including the scene content in the text prompt would definitely have the potential to further improve the inpainting performance!\n\n[b] Tang et al. RealFill: Reference-Driven Generation for Authentic Image Completion. ACM TOG 2024.\n\n---\n\n**Q3: Quantitative evaluation**\n\nThe original quantitative evaluation in the paper is to showcase the limitation of the existing evaluation metrics in subject-driven generation. Now, with your valuable suggestion, we are delighted to present a more reliable and sound metric as mentioned in the response to your question ***W4-1: Evaluation on identity preservation***, and it shows the superiority of our method on identity preservation compared with current state-of-the-arts.\n\n\n---\n\nWe sincerely thank you once again for your valuable comments, which have greatly helped improve our work. We hope that the explanation above can well address your concerns!"}, {"review_id": "Da3OIsKhT4", "reviewer": "Reviewer_AB6n", "summary": "TIRE (Track, Inpaint, REsplat)  method for 3D/4D generation with semantic consistency across views.\nThis method maintains the consistency of the target through three stages of processing. The tracking strategy adopts a backward tracking optimization scheme that supports any 3D representation form to improve tracking accuracy. The repair strategy uses a progressive repair strategy to solve the consistency problem when the target perspective and the source perspective differ too much. The final reconstruction stage uses multi-perspective consistency optimization to improve the visual consistency of the final result.", "questions": "- For tracking, which 3D expression works best and which works worst? Why? Please add some ablation experiments\n- For tracking, backward tracking. What are the disadvantages of backward tracking, and what are the common situations where backward tracking fails and normal tracking succeeds?\n- For the inpaint phase, it is necessary to explain the correlation between the current progressive design and loss design, and add experiments with different degrees of progressiveness.\n- Please optimize the description of Figure 1 so that it does not appear to be an automatic annotation or determination method based on LLM.\n-", "limitations": "- The multi-stage solution will accumulate errors. At the same time, there is no overall optimization loss or optimization goal between the stages, or in the end, resulting in the three stages being equivalent to three independent algorithms.\n- The algorithm based on progressive rotation restoration is prone to misalignment in details and will have obvious segmentation lines. You can try to give some experimental results of leopards with spots or giraffes with more patterns.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The three-stage approach solves several core contradictions in visual consistency, and creatively uses backward tracking to improve the effect.\n- From the experimental results, compared with the existing SOTA 3D solutions (including closed-source solutions), it is indeed better in terms of visual consistency.\n- In theory, this method can handle a variety of different 3D representations well.\n\nWeaknesses:\n- The experimental phase did not experiment with different 3D representations, so it is not possible to judge which representation the method is most suitable for and which representation it will have problems with.\n- The method part does not explain much about backward tracking, and does not give the specific implementation of backward tracking and the problems that may arise (why a solution like cotracker can use backward tracking, and why backward tracking is used in the field of 3D generation)\n- Figure 1 is somewhat misleading. In theory, the whole process can be automatically run in stages, but Figure 1 looks like it requires more human thinking and intervention.", "was_revised": true, "final_justification": "The authors have provided a strong rebuttal along with additional experiments that will be included, so I am upgrading my rating to Borderline Accept.", "rebuttal": "We sincerely appreciate the detailed feedback you provide for our work! Below, we provide the following clarifications to address your concerns:\n\n---\n\n**W1 & Q1: Which 3D representations are more suitable?**\n\nThanks for the valuable comment! Since our tracking is performed on rendered 2D images from the 3D assets, the performance relies on the quality of the rendered images, rather than solely depending on the type of 3D representations (e.g., NeRFs, 3D Gaussians, 3D mesh). Basically, better cross-view consistency of the generated 3D assets would lead to better performance in tracking, because they have less floating artifacts that can distract the tracking model. \n\nFor the performance of the whole pipeline, it is better to adopt the 3D generation methods that can generate decent geometric shapes of the subject. The reason is that our pipeline is designed mainly to fix the unnatural appearance issue to make the generated assets match the identity of the given source view, with little modification on the geometry and shape of the subject (there are geometric fixes from our pipeline, as demonstrated in Fig. 6 in the main paper and Fig. I in the appendix, but they are not major changes to the geometry). Therefore, starting from a 3D method that can produce decent geometry and shape is more favorable for our method to achieve optimal performance.\n\nWe have prepared interesting visual comparisons for ablations on different methods with different 3D representations. However, due to the policy of the rebuttal not allowing for PDF or other types of media, we choose to describe the qualitative comparisons in detail to the reviewer. *The qualitative comparisons requested for the following questions are addressed in the same way.*\n\n> ***Description of Visual Comparisons***\n\n> If solely focusing on the tracking performance, methods like Wonder3D with 3D mesh as representation produces slightly better tracking results than methods like LGM with 3D Gaussians and SV3D with NeRFs, because 3D mesh generally has fewer floating artifacts than NeRFs and 3D Gaussians, which makes it easier for tracking. \n\n> However, for producing a better final generated 3D asset, we empirically find that LGM has an overall better capability of producing realistic shapes and geometry of the subjects, better handling the depth of the subject along with the spatial relationship between the objects. Although the tracking performance on LGM may not be as accurate as Wonder3D (but not far away from Wonder3D), we observe that our method can offset the effect in the following *Resplat* stage when rectifying the multi-view geometry, generally being robust towards the minor inaccuracies from the tracking stage.\n\nTherefore, using LGM helps us yield the most pleasant results for the final generated 3D assets, which is the reason for our current implementation to use LGM as the initial 3D generation method in the pipeline.\n\n\n---\n\n**W2 & Q2: Normal tracking v.s. backward tracking**\n\nThanks for the question! In our method, backward tracking is to stack the frames from the target view to the source view to form a video, and then use the video tracking tool CoTracker to track the pixels in the target view. If the tracking result of the pixel gets lost or falls outside the valid regions of the source view, it means the pixel needs to be inpainted in the following *Inpaint* stage. The only difference between the videos for forward tracking and backward tracking is the camera movement direction. Therefore, the video tracking tool CoTracker can well handle the videos with reversed-order frames as normal.\n\nFor comparing normal tracking (forward tracking) and backward tracking, backward tracking is almost always a better choice than normal tracking, both from the design principle and the experimental experience. For the underlying design principle of backward tracking, we can make an analogy to *image warping* to better understand this. There are two warping solutions: **(1)** Forward mapping, and **(2)** backward mapping (reverse mapping). In forward mapping, the algorithm iterates all the pixels in the source image $I_1$ to map their values to the target image $I_2$. However, this can lead to \"holes\" in the target image because many pixels in $I_2$ may not get mapped from a certain pixel in $I_1$. In comparison, when performing backward mapping, any pixel in $I_2$ that should be mapped to a certain pixel in $I_1$ can successfully find its corresponding pixel in $I_1$. The problem of backward mapping, in the image warping context, is that pixels in $I_2$ which are occluded or do not exist in $I_1$, cannot get its value during the mapping process. However, in our context of backward tracking, ***it is exactly what we are searching for!*** The goal of the *Track* stage is exactly to identify the pixels in the target view that are occluded or missing in the source view. Therefore, our backward tracking design perfectly matches our need for figuring out which parts in the target views need to be inpainted. Additionally, from our experimental experience (Fig. 3 as an example), backward tracking is indeed a more suitable solution for our pipeline, which matches the design principle of our backward tracking algorithm.\n\n---\n\n**W3 & Q4: Figure 1 design**\n\nThanks for the suggestion! We will revise the figure to remove the icon of the thinking person, and reorganize the components of the figure to better convey the broad picture of our method. \n\n---\n\n**Q3: The correlation between the progressive design and loss design, and ablations on degrees of progressiveness**\n\nThanks for the comment! If we understand correctly, the reviewer would like to know about the correlation between our adopted inpainting loss (Eq. 7) and the general diffusion-based inpainting loss in works like RealFill [a]. The loss calculation steps between these two are the same, and the difference is that we are performing an innovative ***progressive*** inpainting strategy. This is because our task poses additional challenges compared to the original RealFill setting, which requires the model to infill the viewpoints that have significant variance from the source viewpoint. If we only train the inpainting model with the source view, the model will not be competent enough to fill in the textures for the side and back views that have large variations. \n\nFor the ablations on the degrees of progressiveness, we have also prepared informative visual comparisons for the reviewer, and the description of the qualitative comparisons is shown below: \n\n> ***Description of Visual Comparisons***\n\n> Basically, for the degrees of progressiveness smaller than the current choice (current choice is $20^\\circ$, and we choose $10^\\circ$ for ablation), the visual results are very similar to the current results, but we need additional steps to calculate the inpainting masks for the increased number of training stages. The total training time can roughly stay the same, because we can half the iterations needed for each stage and still get comparable results. For the degrees of progressiveness larger than the current choice (we choose $30^\\circ$), we could observe that the regions (especially for those that are far away from the reference view, e.g., the regions that are close to the back view for the target views of $\\pm 90^\\circ$) are inpainted worse than the current implementation. Our current choice of $20^\\circ$ of progressiveness balances between the number of frames that need for tracking, and the difficulty of inpainting the unseen regions that are far away from the source view.\n\nIf the reviewer still has questions regarding the correlation between the current progressive design and loss design, please feel free to further clarify this question and we are happy to answer!\n\n[a] Tang et al. RealFill: Reference-Driven Generation for Authentic Image Completion. ACM TOG 2024.\n\n---\n\n**L1: Error accumulation with the multi-stage solution**\n\nThanks for the comment! It is true that multi-stage solutions have the disadvantage of error accumulation. However, our three stages are well orchestrated, with the earlier stage striving to make the task of the later stage easier. At the same time, the later stage is designed to make up for the errors which are caused by the inaccuracies from the earlier stages. For example, one of the purposes to use backward tracking is to produce inpainting masks that are more suitable for the following *Inpaint* stage. The *Resplat* stage refines the multi-view consistency with mask-aware latent updates to mitigate the error coming from the prior *Track* and *Inpaint* stages. Therefore, the three-stage solution is cohesively integrated with each other, instead of just functioning as three independent algorithms.\n\n---\n\n**L2: Limitation on progressive rotation restoration**\n\nThanks for the insightful comment! Subjects with complex patterns are indeed challenging for our current model, which are also the hard cases for other 3D generation models. However, we believe that our work can provide inspiration on how to utilize the powerful 2D tools like tracking and inpainting for solving this challenge. For the segmentation lines, it can be largely suppressed with using soft masks during inpainting. Basically, at the mask boundaries we apply feathering to blur the mask, then use the blurred soft mask for integrating the original image and the inpainted image. With this simple operation, the segmentation line issue can be significantly addressed.\n\n\n---\n\n\nWe sincerely thank you once again for your valuable comments, which have greatly helped improve our work. We hope that the explanation above can well address your concerns."}]

Author rebuttal:

Meta-review: This paper introduces a pipeline for 3D/4D generation with improved subject identity preservation and consistency across views. The approach leverages a tracker to detect invisible regions, a progressive inpainting module for filling in missing details, and a resplatting stage for final 3D/4D Gaussian reconstruction. The method is modular, lightweight, and can be applied on top of existing image-to-multiview and multiview-to-3D pipelines.

Strengths include the clear and lightweight pipeline and practical improvements in handling view-inconsistency. The problem setting is timely and relevant  to the broader community.

Weaknesses initially concerned the rigor of the experimental evaluation and quality of the results. However, those concerns were largely addressed in the authors' rebuttal, resulting in an overall positive consensus.

The technical contribution is not strong enough for a spotlight/oral but provided that the reviewers' feedback is carefully incorporated in the final version, I recommend accept as a poster.
Decision: Accept (poster)