SyncHuman: Synchronizing 2D and 3D Generative Models for Single-view Human Reconstruction
Authors: Wenyue Chen
Peng Li
Wangguandong Zheng
Chengfeng Zhao
Mengfei Li
Yaolong Zhu
Zhiyang Dou
Ronggang Wang
Yuan Liu
Venue: NeurIPS 2025 poster ()
Keywords: 3D Human Reconstruction
Diffusion Model
Generation Model
Primary area: deep_learning
TL;DR: We introduce SyncHuman, a full-body human reconstruction model using synchronized 2D and 3D generative model. Given a single image of a clothed person, our method generates detailed geometry and lifelike 3D human appearances across diverse poses.

Abstract: Photorealistic 3D full-body human reconstruction from a single image is a critical yet challenging task for applications in films and video games due to inherent ambiguities and severe self-occlusions. While recent approaches leverage SMPL estimation and SMPL-conditioned image generative models to hallucinate novel views, they suffer from inaccurate 3D priors estimated from SMPL meshes and have difficulty in handling difficult human poses and reconstructing fine details.In this paper, we propose SyncHuman, a novel framework that combines 2D multiview generative model and 3D native generative model for the first time, enabling high-quality clothed human mesh reconstruction from single-view images even under challenging human poses.Multiview generative model excels at capturing fine 2D details but struggles with structural consistency, whereas 3D native generative model generates coarse yet structurally consistent 3D shapes. By integrating the complementary strengths of these two approaches, we develop a more effective generation framework. Specifically, we first jointly fine-tune the multiview generative model and the 3D native generative model with proposed pixel-aligned 2D-3D synchronization attention to produce geometrically aligned 3D shapes and 2D multiview images. To further improve details, we introduce a feature injection mechanism that lifts fine details from 2D multiview images onto the aligned 3D shapes, enabling accurate and high-fidelity reconstruction.Extensive experiments demonstrate that SyncHuman achieves robust and photorealistic 3D human reconstruction, even for images with challenging poses. Our method outperforms baseline methods in geometric accuracy and visual fidelity, demonstrating a promising direction for future 3D generation models.

SyncHuman: Synchronizing 2D and 3D Generative Models for Single-view Human Reconstruction
Wenyue Chen', Peng Li Wangguandong Zheng', Chengfeng Zhao? Mengfei Li", Yaolong Zhu', Zhiyang Dou", Ronggang Wang', Yuan Liu
PKU, PHKUST, SEU, *MIT
https://xishuxisih wtt.aaii.......i.otiiiiioo.ee hwt.:ecctuuaanniggtiiiii.. github. io
Figure 1: We introduce SyncHuman, full-body human reconstruction model using synchronized 2D and 3D diffusion model. Given a single image of clothed person, our method generates detailed geometry and lifelike 3D human appearances across diverse poses.
Abstract
Photorealistic 3D full-body human reconstruction from a single image is critical yet challenging task for applications in films and video games due inherent ambiguities and severe self-occlusions. While recent approaches leverage SMPL estimation and SMPL-conditioned image generative models hallucinate novel views, they suffer from inaccurate 3D priors estimated from SMPL meshes and have difficulty in handling difficult human poses and reconstructing fine details. In this paper, we propose SyncHuman, a novel framework that combines 2D multiview generative model and 3D native generative model for the first time, enabling high-quality clothed human mesh reconstruction from single-view im- ages even under challenging human poses. Multiview generative model excels capturing fine 2D details but struggles with structural consistency, whereas 3D native generative model generates coarse yet structurally consistent 3D shapes. By
"Coresponndig authors
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

integrating the complementary strengths of these two approaches, we develop a more effective generation framework. Specifically, we first jointl fine-tune the multiview generative model and the 3D native generative model with proposed pixel-aligned 2D-3D synchronization attention to produce geometrically aligned 3D shapes and 21) multiview images. To further improve details, we introduce a feature injection mechanism that lifts fine details from 2D multiview images onto the aligned 3D shapes, enabling accurate and high-fidelity reconstruction. Extensive experiments demonstrate that SyncHuman achieves robust and photo- realistic 3D human reconstruction, even for images with challenging poses. Our method outperforms baseline methods in geometric accuracy and visual fidelity, demonstrating a promising direction for future 3D generation models.
Introduction
Reconstructing 3D clothed humans from single RGB image a fundamental yet challenging task.
It has broad applications in AR/VR, virtual try-on, gaming, and film production [33 36.. Compared to parametric body reconstruction [54), clothed human reconstruction (83) requires capturing not only the underlying body shape but also the diverse topology, geometry, and dynamics of garments.
Recent progress in implicit representations and generative models has led to significant advances in this area. PIFu 44 pioneered this direction with predicted neural implicit field, followed by methods such ICON 633 ECON [22. and PaMIR [82], which introduced improvements in SMPL priors, normal estimation, and feature representation. respectively. With the advancement of generative models techniques 30 31 recent works I3 78 25] have introduced multiview generative model for novel-view human image prediction, enhancing 3D reconstruction fidelity, detail preservation, and robustness.
However, accurately reconstructing 3D clothed humans from a single 2D image is still challenging,
especially for images with challenging poses. The reason is that most methods [K] 25 strongly rely
on human shape priors, i.e., SMPL estimation, to provide structural information to generate multiview images. Unfortunately, existing single-view human pose estimation methods 7
3 often lack sufficient accuracy, especially when dealing with occlusions or challenging poses. as shown in Fig. 2 (a). Moreover, the estimated SMPL meshes represent only naked human bodies and fail to accurately model loose clothing. Thus, conditioned on inaccurate SMPL meshes, the multiview generative models often generate images with incorrect body topologies and mismatched details, leading to reconstruction artifacts, as shown in Fig. (b).
72
I 39
An alternative approach employs native 3D gen- erative models 773 61 27 [79 to generate the 3D human shapes directly. However, these meth- ods often produce results lacking in detail and fi-
delity. By training on large-scale 3D datasets, re- cent 3D native generation methods [6]] 79, 27]
demonstrate improved capability for construct- ing 3D human meshes from single-view images, even for challenging human poses, as shown in Fig. 2 (c). However, these 3D native gener- ation methods typically generate only coarse, low-fidelity shapes that poorly match the input image characteristics. Enhancing both the geo- metric detail and input-consistency of 3D-native generation outputs remains an open challenge.
Input
SMPL (b) PSIlmmnn (c) Trellis
(d) Ours
Figure 2: Geometric comparison between SMPL estimation [39], 2D multiview generative model (MVD) PSHuman 25), native 3D generative model Trellis 66 and our method. 2D MVD produces high-quality details but has geometry artifacts when conditioned on inaccurate SMPL. meshes. Native 3D generative model produces correct coarse structure but loses fine details and fi- delity. Our method combines the strengths of both 2D and 3D generative models to produce detailed 3D human meshes with high fidelity.
In this paper, we propose SyncHuman, novel framework that combines 2D multiview gener- ative model and native 3D generative model for the first time, leveraging their complementary strengths to address these challenges, as shown in Fig. (d). Instead of simply relying on the SMPL estimation, utilize the more accurate

---

3D shapes generated by the native 3D generative model to guide the generation of 2D multiview images, greatly improving the multiview consistency. At the same time, the detailed multiview images also guide the 3D generative model to carve the 3D shapes with detail and high fidelity.
In implementation, SyncHuman consists of two main components. First, design unified 2D-3D cross-space generative model with two branches, i.c., 2D multíview generation branch and a 3D sparse structure generation branch, which interact via 2D-3D synchronization attention layers. The 2D-3D attention layers align the multiview images with the generated 3D shapes, which simultaneously utilize the 3D shapes improve the cross-view consistency and employ the multiview images to enhance the fidelity of the generated 3D shapes. Next, to obtain high- -ualitt 3D meshes, information of generated
we design multiview guided decoder to incorporate the multiview images into the 3D generation during the decoding process. which not only carves fine geometric detail but also greatly improves the texture fidelity.
We conduct extensive experiments on multiple datasets to evaluate the effectiveness of SyncHu- man. The results demonstrate that the proposed method outperforms previous single-view human reconstruction methods [62 13 25 while even achieving higher fidelity and texture quality than the large-scale 3D generative models [6]] trained with datasets hundreds of times larger than ours. SyncHuman unifies 2D multiview generative model and native 3D generative model within a unified framework, enabling higher-quality image-to-33 generation with improved fidelity. This demon- strates significant potential for future 3D generation model development.
2 Related works
Single image human reconstruction. Prior the advent of generative approaches, single- image human reconstruction primarily followed either explicit or implicit representation paradigms. Explicit methods, including voxel-based techniques [55]
83 visual hull approaches [44, and
depth/normal 88 50 I0 prediction frameworks, offer computational efficiency but often sacrifice local geometric details. The explicit normal integration in ECON 62 made a significant advancement in reconstruction robustness for the explicit paradigm. In contrast, implicit methods emerged as the dominant approach due to their continuous representation capabilities. The field was revolutionized by PIFu 45 6 67]. which established pixel-aligned implicit functions for detailed geometry recov- cry from single images. Subsequent approaches enhance the robustness [I 63
82, 688 through
parametric body model integration and additional supervision from surface normals [46) and depth
41 incorporate transformer architec- ture and utilize large-scale human datasets to reduce inductive bias, enhancing the generalization capability. While these methods demonstrate exceptional performance in handling complex clothing and topological variations, they remain inherently constrained by their reliance on the input image, struggling with photorealistic appearance and detail recovery.
information [70 8T. Most recent works [77 78
18 69
3D Generation. 3D generation has been significantly advanced by generative models, which can be roughly categorized into multiview generation approaches and native 3D generation methods.
Multiview generation techniques [49 31 30
typically employ two-stage pipeline: first generating consistent multiview images, followed by either
24
25
17
56
85
52
65
64
57
59
53
23 The multiview gen- eration stages involve fine-tuning an image generative model (43] or video generative models [2] by incorporating view-aware attention layers to ensure cross-view consistency. Native 3D gener- ative models [80 61
optimization-based reconstruction 377 or feed-forward generation 22
26 operate directly in 3D representation spaces (e.g., 3D Volume [6 or Signed Distance Field 38 ). typically comprising large variational autoencoder combined with a latent diffusion transformer (DiT) [40]. Trained on extensive 3D datasets, these models demonstrate exceptional geometric quality and strong generalization capabilities. Building upon these foundations, our work adapts and fine-tunes such a native 3D generator specifically for human body shape while preserving its generalization capacity.
27 79
60
Generative Human Reconstruction Generative models, such Stable Diffusion, have emerged as a powerful tool for 3D human reconstruction. Pioneering works 28 16 15 employ score distillation sample (SDS) to optimize textured human mesh per case, which is time-consuming and typically only cext-constrained. Feed-forward methods [
leverage pose-guided ControlNet [74] to predict plausible back views for neural reconstruction or Gaussian splatting, but their robustness suffers from
3

---

2D-3D Cross-Space Generative Model
Multiview Guided Decoder(MVVG)
*
Hush
Multiview Generative
Dinov2
Enc
Dec
Model
UVV
M
pixel pixel-wise -wise
injaction
2D-3D synchronization
attention
Ivv
Do
*
Structure
Generative
Model
Structured Labents Generative Madel
Condition
gadd
Prsse
Gistent
Latents
Trainable Module
Freezed Module
Noise Latent
Figure 3: Overview. Given single human image, SyncHuman first generates multiview color and normal maps, along with an aligned sparse voxel grid, which is further transformed into a set of structured latents. Then, we propose inject the high-quality images into the 3D latents via Multiview Guided Decoder and output the detailed high-fidelity textured human mesh.
address this problem by fine-tuning 2D generative models produce sparse multiview human generations. Despite improved performance, these models struggle with cross-view consistency, leading to inevitable appearance artifacts. Human3 [66] attempts to enhance multiview coherence by integrating 3D representations as intermediate constraints during the denoising process. However, reliance on 2D denoising generative models often leads to anatomically implausible human structures due to the absence of body prior. Unlike prior work, this study aims to align the pretrained 2D multiview and 3D native generative models, enabling producing geometrically consistent and robust 3D human models without reliance on any human prior.
limited multiview cues. Other approaches 20 25
Method
Overview. SyncHuman aims to reconstruct a 3D clothed human mesh from a single color image. As shown in Fig. given full-body human image, we first propose 2D-3D Cross-Space generative model (Section 3.T to synthesize multiview color and normal maps, along with an aligned sparse 3D voxel grid, which is further transformed to aligned structured latent through pretrained flow transformer. Then, a Multiview Guided Decoder (Section 3.2) h is introduced to decode the structured latents into high-quality detailed, textured mesh with the help of generated multiview images.
3.1 2D-3D Cross-Space Generative Model
Multiview generative models have shown powerful novel-viee generation and generalization capabil- ity. Given human image as input, they could hallueinate multiple views with high-resolution details such identity, skin texture, and clothing wrinkles, but often struggle with cross-view consistency. In contrast, native 3D generative models naturally maintain 3D structural consistency, yet typically lack fidelity. In this section, introduce 2D-3D Cross- Space Generative Model, which combines the strengths of 2D multiview generative models and native 3D generative models.
Multiview Generative Model. Taking the input image / as the front view, we use the network structure from PSHuman 25 to generate color and normal maps on four predefined orthogonal viewpoints, front, back, left, and right, which employs an efficient row-wise multiview attention to enhance cross view consistency. This module could be formulated as
JuV GMv (T):
where JM is the generated multiview images and normal maps. Previous methods usually use the estimated 3D SMPL meshes to improve the multiview consistency in LMv. but often

---

2D to 3D attention
3D to 2D attention
q
k,v
k,v
Figure 4: 2D-3D synchronization attention. 2D to 3D attention: each 3D voxel feature is orthogonally projected onto front, back, left, and right view planes retrieve corresponding 2D features, and refines the voxel feature with cross-attention. 3D to 2D attention: each 2D multiview feature is projected into space to attend to a column of voxel features, enhancing the 2D features. This mutual refinement ensures that 2D generative model and 3D generative model align with each other in a shared 3D space.
suffer from inaccurate SMPL estimation. Thus, we introduce the native 3D generative model to provide 3D structural guidance for the multiview generation in the following.
3D structure Generative Model. Our native 3D generative model follows Trellis 61 A 3D noise grid is first used to produce sparse structure latent through DiT-based flow transformer Ggad- The sparse structure latent is subsequently decoded into an occupied voxel grid Verid via Conv-based decoder D vaxel.
Vrri Vooxel (ggria (1)),
where the input image 7 is fed into the generative model layer by cross-attention layers. The sparse structure generated by Trellis [61] produces reasonable 3D shapes but loses fidelity and details. We add novel 2D-3D synchronization attention to improve the fidelity and retrieve more details from multiview images when transforming the 3D structure to textured meshes.
2D-3D synchronization attention. We introduce 2D-3D synchronization attention mechanism between the 2D multiview generative model and the 3D generative model to let them benefit each other in the generation. This consists of 2D to 3D attention and 3D to 2D attention layers as follows.
(1) 2D to 3D attention. As shown in Fig. 4 for each 3D voxel feature, corresponding 2D features on generated four normal maps. Then, the 3D voxel feature is used as the query token, and the concatenated 2D features from four views serve as keys and values for cross-altention. The cross-attended features are processed by an output MLP with zero initialization, and the resulting features are added to the original 3D voxel feature for refinement.
first sample the
(2) 3D to 2D attention. Then, for each 2D feature on multiview images, we query the corresponding 3D voxel columns as in Fig. [ Then, the 2D feature is used the query while the 3D voxel feature serves as keys and values for cross- aoosno. The cross-attended features are processed by an output layer with zero initialization, and the results added the 2D features.
Discussion. Our method establishes an explicit correspondence between the 2D and 3D generative models, which benefits each branch. Through this synchronized attention, 3D generative model provides 3D structural guidance for the 2D generative model to improve the multiview consistency while the 2D generative model regularizes the 3D generative model to generate shapes that are more aligned with the input image with better fidelity. This integration enables our model to combine the advantages of both approaches: the 2D generative model provides detailed, high-fidelity results, while the 3D generative model ensures structural integrity and robust handling of complex human poses.
2D-3D joint training, employ the flow matching objective to train our 2D-3D cross-space generative model with the training loss defined by
L 1 (x
(xx -e2
(x, (xx) e end
(3)
5

---

where edd and edd is the 2D noise maps and 3D noise grid, x² and x² is the latent features at timestep I and vg and vg are the corresponding predicted velocity during denoising process, respectively. Note that the multiview generative model is based on the Stable Diffusion 2.1 [44] and we retarget to the same flow matching model as Trellis for jointly training.
3.2 Multiview Guided Decoder (MVGD)
This section utilizes the generated multiview images and sparse voxels to recover textured 3D meshes.
Structured latent generation. We first apply another DiT-based generative model Gtatent in Trel- lis [1,, which is named as Structured Latents Generative Model in Fig. to generate a set of structured latents Viaten-- Each of them is attached to previously generated 3D voxel. These struc- tured latents can processed by either a mesh decoder D,, or a 3D Gaussian Splatting [9] (3DGS) decoder Dgs to generate mesh or 3DGS representation. For simplicity, we unify these decoders as Do- However, directly decoding these latent to mesh or 3DGS leads to a lack of reconstruction details, particularly noticeable in areas such the face and clothing wrinkles, as demonstrated in Fig. g To address this, we propose a multiview feature injection mechanism to incorporate the generated high-resolution multiview images into the original decoder.
35 features of generated multiview images, and process them with several trainable MLP layers. For each 3D voxel, we query the corresponding four-view image features and concatenate them with the generated structure latent. The concatenated features are first passed through a MIP, and the resulting representations are subse- quently fed into the original decoder D, to produce a high-quality mesh and 3DGS representation. This simple but efficient feature injection allows for preserving the geometry fidelity and appearance realism to a great extent, as shown in Fig. [] We render images from 3DGS and then bake onto the mesh to obtain the final textured human mesh M. The overall decoding process can be formulated as
Multiview feature injection. Specifically, we extract DINOV2
M Do Do (Grsteu (I, Vgria), IMv).
(4)
Training Loss. We train the multiview guided decoder for the 3DGS branch and the mesh branch separately. For the 3DGS branch, we use LI loss, Structural Similarity Index (SSIM), Learned Perceptual Image Patch Similarity (LPIPS) loss between renderings and ground-truth images, and a regularization loss to avoid extremely large or small opacity. For the mesh branch, we render the foreground mask, depth maps, and normal maps from the generated 3D meshes. Then, compute the LI or Huber loss between the ground truth and the renderings to train the decoder. More architectural design and training details are given in the supplementary material.
Experiments
4.1 Experiment Setup
Dataset. Our models are trained on several widely used 3D human scanning datasets, including 1 70,, CustomHumans [2], THuman3.0 [5I). and 2K2K [00. To construct training images, we render 8 ground-ttrth images using orthographic cameras with evenly distributed azimuth angles and fixed (' elevation with a resolution of 768 768. For quantitative evaluation, we utilize 100 scans from X-Humans [7 and 150 scans from CAPE (32). X-Humans contains 233 sequences of high-quality textured scans from 20 participants. We randomly selected textured scans from each of the 20 participants in the X-Humans dataset, resulting in 100 test samples. Following ICON's partitioning criteria, we subdivide CAPE into "CAPE-FP" (50 samples) and "CAPE-NFP" (100 samples) to test the generalization ability in real-world examples. We conduct comparison with the baseline methods on the aforementioned X-Humans subset and CAPE subset, and perform ablation experiments on the same X-Humans subset.
Metric. To evaluate reconstruction capability. we employ three primary metries: 1-directional pourf-to-surface (P2S), L2 Chamfer Distance (CD), and Normal Consistency (NC). For geometry evaluation, we align the centers of the reconstructed mesh and the ground truth mesh and then scale them so that the coordinate range of the longest axis is 1. For appearance evaluation, we render front, back, left, and right views and compute PSNR (58), structural similarity index (SSIM) (75). and perceptual image patch similarity (LPIPS) (76).

---

ECON
Human3Diff
SIFU
PSHuman
Ours
Input
Figure 5: Geometry comparisons between BCON [81] Human3Difi (66), SIFU 78]. PSHuman
and ours. Our method could reconstruct 3D shapes with complete body structure and rich details.
GTA
Human
SIFU
PSHuman
Ours
Input
Figure 6: Appearance qualitative comparisons between GTA [81) Human3 (66), PSHuman [5] and our method.
SIFU (78)

---

Table I: Quantitative comparison of geometry and appearance on the CAPE-NFP 32). X- K-Humans
CAPE-FP 32
B2
datasets. Our method achieves superior performance on all metrics than baseline
methods.
Method
CAPE-NEP
CAPE-FP
HuHanns
Cham.
P28
NC1
Cham
P2S
NC1
Cham
P2S
NC1
PSNRT
SSIM
LPIPS
0.7974
1.2018
ECON
62
18335
1.5391
0.7731
1.3129
1.2962
0.8225
1.6425
1.43%8
0.5054
GTA
1,6111
0.7890
0.8277
0.8044
20.0084
0.8502
0.1129
SIFU
28
1,6573
1.5130
0.7895
1,2759
1.22255
0.8289
1.591
1.4331
0,8093
20.6747
0.8455
0.1104
SITH
13
1,6661
1.2043
0.7914
1,0377
0.9767
0.8516
1.5104
1.4345
0.7972
19.8245
0.8204
0.1182
Human3Diff
1.599
1.2016
0.7427
0.7914
1.5034
1.4219
0.7468
19.7181
0.8065
0.1334
PSHuman
1.3726
0.9863
0.8276
0.7764
0.6527
0.8850
4337
1.1385
0,8393
20,8405
0.852.
0.0980
TRELLIS
OURSS
0.9127
0.8113
0.8483
0.6409
0.5962
0.8958
0.8353
0.7593
0,8872
21,8385
0.8741
0.0786
Trellis
Trellis(tuned)
PSHuman
Ours
Figure 7: Ablation study of the 2D-3D synchronization attention for joint 2D-3D modeling between PSHuman (25), fine-tuned Trellis [61], and our model.
4.2 Comparison with baseline methods
Baselines. We conducted comprehensive comparison of our method against state-of-the-art single- view human reconstruction approaches, including classic implicit function based methods (ICON (63] GTA 77). SiFU (78). explicit work (ECON (62),, and other baselines with generative priors SiTH 13 Hufan3Diff [66] and PSHuman (25]). All evaluations are conducted with the official open-source codes, applying a unified evaluation method. More comparisons and visual results are provided in the Appendix.
Comparison of geometry quality. Combining the advantages of accurate 3D coarse structure from the native 3D generative model and the rich details of the multiview generative model, our method outperforms existing approaches in geometry quality as shown in Tab. [] The qualitative comparison in Fig. 5 highlights that our method also handles complex human poses correctly, demonstrating significant improvements in structural integrity, correctness, and detail richness over baseline methods.
Comparison of appearance quality. We render four views with resolution of 768 for each sample and evaluate the appearance quality by reporting average PSNR, SSIM, and LPIPS. The results presented in Tab. demonstrate that our method significantly outperforms existing approaches on all metrics. As illustrated by the qualitative results in Fig. 6 our method generates high-quality appearances on novel viewpoints, delivering natural and photorealistic reconstruction quality. In contrast, existing methods exhibit notable limitations in both unseen views and occluded regions, including blurred colors and artifacts.
original decoder
original decoder(tuned)
MVGD
Ours
Figure 8: Ablation of different decoder settings. "original decoder" means the pretrained Trellis 61 decoder, while "original decoder (tuned)" and "multiview guided" are trained on the same human scans.All models use the same structured latents but decode them with different decoders.

---

Table 2: Ablation study 2D-3D cross-space generative model on X-Humans
47
subset.
Method
PSNR 1
SSIM f
LPIPS
Cham. Dist +
P2S
NC 1
Trellis 66
17.079
0.724
0.153
2.004
1.505
0.772
Trellis 61 (tuned)
20.344
0.844
0.101
1.135
1.041
0.848
PSHuman 25
20.840
0.852
0.098
1.438
1.138
0,839
Ours
21.838
0.874
0.0786
0.835
0.759
0.887
Table 3: Ablation study of our multiview guided decoder (MVGD) on X-Humans models employ the same structured latents but different decoders.
47
subset. All
Method
PSNR f
SSIM t
LPIPS
Cham. Dist
P2S
NC t
original decoder
21.083
0.862
0.092
0.895
0.820
0.875
original decoder (tuned)
21.362
0.866
0.090
0.887
0.810
0.877
MVGD
21.838
0.874
0.0786
0.835
0.759
0.887
4.3
Ablation Study
2D-3D cross-space generative model. We ablate the effectiveness of 2D-3D cross-space generative model on X-Humans subset by removing the 2D-3D synchronization attention. For fair compar- ison, we fine-tuned all the models on the same dataset. Compared with PSHuman (2D multiview generative model + remeshing) and Trellis native 3D generation model), this cross-space attention significantly enhances geometric accuracy and texture fidelity, as shown in Tab. 2 and Fig. 7
Multiview guided decoder (MVGD). To evaluate the effect of MVGD, of structured latent decoders: (1) the original Trellis decoder, (2) the Trellis decoder fine-tuned on human scans, and (3) the decoder guided with multiview images (our MVGD).
compare three types
We conduct the comparison on the same X-Humans subset, evaluating both geometry and appearance. We apply mesh normalization and ICP registration to align the output meshes with the round truth scans to ensure a fair comparison. For each mesh, we render four views with a 768 resolution and report the average PSNR, SSIM, and LPIPS. The results in Tab. 3 show that multiview guided decoding significantly enhances geometric accuracy and texture quality. Fig. S also clearly illustrates that incorporating multiview image information improves the details and fidelity.
Comparison between our structure and SMPL estimation. To demonstrate that our structure handles some complex human poses better than the SMPL estimation, Fig. shows the SMPL estimation from 4D-Humans g and the 3D structure generated by our method given the same input image. The SMPL. estimation has obvious errors like self- eeff-intersection, while the structure generated by our method aligns better with the inputs without artifacts.
TAAS
Input
SMPL
Our Structure
Figure Robustness Analysis of the Generated Structure. The results demonstrate the robust reconstruction capabilities of our approach.

---

Limitation and Conclusion
In this work, we propose SyncHu- man, a novel framework for robust 3D human generation from single image. By introducing 2D-3D cross- space generative model, we gener- ate high-fidelity 3D structures and cross-view consistent multiview im- ages. Then, we employ multiview guided decoder to obtain detailed and structurally completed 3D human tex- tured meshes. Extensive experiments demonstrate that SyncHuman can generate 3D humans with intricate geometric details and lifelike appearances, outperforming existing methods.
Input
Generated
Figure 10: Unnatural textures under non-uniform lighting.
Limitations. Our method inherits certain constraints from the training data. First, since our training dataset rendered with uniform light source, reconstructed textures may exhibit artifacts under extreme lighting conditions (e.g., localized overexposure or shadows, as shown in Fig. TTp Moreover, our multiview generation model fine-tuned from SD 2.1 using only ~5,000 human scans, so its generation quality is still constrained. It will be promising to scale up our model using video generative models or large-scale multiview human datasets in future work.
References
[1] Fabien Baradel", Matthicu Armando, Salma Galaaoui, Romain Brégier, Philippe Weinzaepfel, Grégory Rogez, and Thomas Lucas". Multi-hmr: Multi Multi-person whole-body human mesh recovery in single shot. In ECCV, 2024.
[2] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent vidco diffusion models to large datasets. arXiv preprint arXiv:231 11Xx:22311.11127, 2023.
Zhongang Cai, Wanqi Yin, Ailing Zeng, Chen Wei, Qingping Sun, Wang Yanjun, Hui En Pang, Haiyi Mei, Mingyuan Zhang, Lei Zhang, et al. Smpler-x: Scaling up expressive human pose and shape estimation. Advances in Neural Information Processing Systems, 36:1 1454-11465 2023.
Jinnan Chen, Chen Li, Jianfeng Zhang, Lingting Zhu, Buzhen Huang, Hanlin Chen, and Gim Hec Lee. Generalizable human gaussians from single-view image. arXiv preprint ar arXiv22406.06050, 2024.
[5] Rui Chen, Jianfeng Zhang, Yixun Liang, Guan Luo, Weiyu Li, Jiaru Liu, Xiu Li, Xiaoxiao Long, Jiashi Feng, and Ping Tan. Dora: Sampling and benchmarking for 3d shape variational auto-encoders. arXiv preprint Aix::2242..1 17808, 2024.
[6] Julian Chibane, Thiemo Alldieck, and Gerard Pons-Moll. Implicit functions in feature space for 3d shape reconstruction and completion. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 6970 6981, 2020.
[7] Yao Feng, Vasileios Choutas, Timo Bolkart, Dimitrios Tzionas, and Michael J. Black. Collaborative regression expressive bodies using moderation. In International Conference on 3D Vision (3DV), 2021.
[8] Valentin Gabeur. tean-Sebastien Franco, Xavier Martin, Cordelia Schmid, and Gregory Rogez. Moulding humans: Non- Non-parametric 3d human shape estimation from single images. In Proceedings of the IEBE/CVF International Conference Computer Vision (ICCV), October 2019.
[9] Shubham Goel, Georgios Pavlakos, Jathushan Rajasegaran, Angjoo Kanazawa. and Jitendra Malik. Hu- mans in 4d: Reconstructing and tracking humans with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14783-14794, 2023.
[10] Sang-Hup Hun Han, Min-Gyu Park, Ju Hong Yoon, Ju-Mi Kang. Young-Jac Jae Park, and Hae-Gon Jeon. High- fidelity 3d human digitization from single 2k resolution images. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 12869-1222 12879, June 2023.
[11] Tong He, John Collomosse, Hailin Jin, and Stefano Soatto. Geo-pifu: Geometry and pixel aligned implicit functions for single-view view human reconstruction. Advances in Neural Information Processing Systems. 33:9276-999 92009927772222 2020.
10

---

[12] Hsuan-I Ho. Lixin Xue. Jie Song, and Otmar Hilliges. Learning locally editable virtual humans. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 21024-21035, June 2023.
[13] Ho, Jie Song, Otmar Hilliges, et al. Sith: Single-view textured human reconstruction with image- conditioned diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattem Recognition, pages 538-549, 2024.
[14] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12599 2022.
[15] Xin Huang, Ruizhi Shao, Zhang. Hongwen Zhang, Ying Feng, Yehin Liu, and Qing Wang. Humannorm: Learning normal diffusion model for high-quality and realistic 3d human generation. In Proceedings of the LEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4568 4577, 2024.
[16] Yangyi Huang, Hongwei Yi, Yuliang Xiu, Tingting Liao, Jiaxiang Tang, Deng Cai, and Justus Thics. Tech: Text-guided reconstruction of lifelike clothed humans. In 2024 International Conference on 3D Vision (3DV), pages 1531-1542. IEEE, 2024.
[17] Zehuan Huang, Yuan-Chen Guo, Haoran Wang, Ran Yi, Lizhuang Ma, Yan-Pei Cao, and Lu Sheng. Mv-adapter: Multi-view consistent image generation made easy. arViv preprint arXiv:2412.03632 2024.
[18] Yash Kant, Ethan Weber, Jin Kyu Kim, Rawal Khirodkar, Su Zhaoen, Julieta Martinez, Igor Gilitschenski, Shunsuke Saito, and Timur Bagautdinov. Pippo: High-resolution multi-view humans from a single image. arXiv preprint arXiv:2502.07785. 2025.
[19] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ToG, 42(4):1-14, 2023.
[20] Byungjun Kim, Patrick Kwon, Kwangho Lec, Myunggi Lee, Sookwan Han, Daesik Kim, and Hanbyul Joo. Chupa: Carving 3d clothed humans from skinned shape priors using 2d diffusion probabilistic models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 15965-15976, October 2023.
[21] Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. Modular primitives for high-performance differentiable rendering. ACM Transactions on Graphics (ToG), 39(6):1- 14, 2020.
[22] Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. arXiv preprint 1xXiv:2311.06214, 2023.
[23] Mengfei Li, Xiaoxiao Long, Yixun Liang, Weiyu Li, Yuan Liu, Peng Li, Xiaowei Chi, Xingqun Qi, Wei Xue, Wenhan Luo, et al. M-Im: Multi-view large reconstruction model. arXiv preprint arX1v:2406.07668, 2024.
[24] Peng Li, Yuan Liu, Xiaoxiao Long, Feihu Zhang, Cheng Lin, Mengfei Li, Xingqun Qi, Shanghang Zhang, Wenhan Luo, Ping Tan, et al. Era3d: High nigh-resolution multiview diffusion using efficient row-wisc attention. arXiv preprint arXiv:240S. 1616, 2024.
[25] Peng Li, Wangguandong Zheng, Yuan Liu, Tao Yu, Yangguang Li, Xingqun Qi, Mengfei Li, Xiaowei Chi, Siyu Xia, Wei Xue, et al. Pshuman: Photorealistic single-view human reconstruction using cross-scale diffusion. arXiv preprint arXiv:2409. 2024.
[26] Weiyu Li, Jiarui Liu, Rui Chen, Yixun Liang, Xuclin Chen, Ping Tan, and Xiaoxiao Long. Craftsman: High-fideliny mesh generation with 3d native generation and interactive geometry refiner. arXiv preprint #rXiv:2405.14977 2024.
[27] Yangguang L.,, Zi-Xin Zou, Zexiang Liu, Dehu Wang, Yuan I. Liang. Zhipeng Yu, Xingchao Liu, Yuan-Chen Guo, Ding Liang, Wanli Ouyang, al. Triposg: High-fidelity 3d shape synthesis using large-scale rectified flow models. arXiv preprint arXiv:2502. 06608, 2025.
[28] Tingting Liao, Hongwei Yi, Yuliang Xiu, Jiaxaing Tang, Yangyi Huang, Justus Thics, and Michael J Black. Tada! text to animatable digital avatars. arXiv preprint arXiv:2308. 10899, 2023.
[29] Yaron Lipman. Ricky TQ Chen, Heli Ben-Hamu Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.

---

[30] Yuan Liu, Cheng Lin, Zijiao Zeng. Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Generating multiview-consistent images from single-view image. arXiv preprint arXiv:2309.03453, 2023.
[31] Xiaoxiao Long. Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song- Hai Zhang, Mare Habermann, Christian Theobalt, and Wenping Wang. Wonder3d: Single image to 3d using cross-domain diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogmition (CVPR), pages 9970-9980, June 2024.
[32] Qianli Ma, Jinlong Yang, Anurag Ranjan, Sergi Pujades, Gerard Pons-Moll, Siyu Tang, and Michael J. Black. Learning to dress 3d people in generative clothing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[33] Shugao Ma, Tomas Simon, Jason Saragih, Dawei Wang, Yuecheng Li, Fernando De La Torre, and Yaser Sheikh, Pixel codec avatars. Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 64-73, 2021.
[34] Ryota Natsume, Shunsuke Saito, Zeng Huang, Weikai Chen, Chongyang Ma, Hao Li, and Shigeo Mor- ishima. Siclope: Silhouette based clothed people. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[35] Maxime Oquab, Timothée Dareet, Théo Moutakanni, Huy Vo, Mare Szafranice, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaacldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.0719. 2023.
[36] Sergio Orts-Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip I. Davidson, Sameh Khamis, Mingsong Dou, et al. Holoportation: Virtual 3d teleporta- tion in real-time In Proceedings of the 29th annual symposium on user interface software and technology. pages 741-754, 2016.
[37] Werner Palfinger. Continuous remeshing for inverse rendering. Computer Animation and Virtual Worlds, 33(5);e2101, 2022.
[38] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2019.
[39] Priyanka Patel and Michael J Black. Camerahmr: Aligning people with perspective. arXiv preprint arXiv:2411.08 2024.
[40] William Peebles and Saining Xic. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference computer vision, pages 4195-4205, 2023.
[41] Lingteng Qiu, Xiaodong Gu, Peihao Li, Qi Zuo, Weichao Shen, Junfei Zhang, Kejie Qiu, Weihao Yuan, Guanying Chen, Zilong Dong, et al. Lhm: Large animatahle human reconstruction model from a single image in seconds. arXiv preprint arXiv:2503.
2025.
[42] Lingteng Qiu, Shenhao Zhu, Qi Zuo, Xiaodong Gu. Yuan Dong. Junfei Zhang, Chao Xu, Zhe Li, Weihao Yuan, Liefeng Bo, al. Anigs: Animatable gaussian avatar from single image with inconsistent gaussian reconstruction. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 21 141--8-22 1114488-115 21152, 2025.
[43] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, pages 10684-10695, 2022.
[44] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. Proceedings the TEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684-10695, June 2022.
[45] Shunsuke Saito, Zeng Huang, Ryota Natsume. Shigeo Morishima, Angjoo Kanazawa, and Hao Li. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2304 2314, 2019.
[46] Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. Pifuhd: Multi-level pixel-aligned implicit function for high- nigh-resolution 3d human digitization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 84 93, 2020.
12

---

[47] Kaiyue Shen. Chen Guo, Manuel Kaufmann. Juan Jose Zarate. Julien Valentin, Jie Song, and Otmar Hilliges. X-avatar: Expressive human avatars. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 16911-1692), 2023.
[48] Tianchang Shen, Jacoh Munkherg, Jon Hasselgren, Kangxue Yin, Zian Wang, Wenzheng Chen, Zan Gojeie, Sanja Fidler, Nicholas Sharp, and Jun Gao. Flexible isosurface extraction for gradient-based mesh optimization. ACM Transactions on Graphics (TOG), 42(4))1-16, 2023.
[49] Yichun Shi, Peng Wang. Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for generation. arXiv preprint arXiv:2308. 16512, 2023.
[50] David Smith, Matthew Loper, Xiaochen Hu, Paris Mavroidis, and Javier Romero. Faesimile: Fast and accurate scans from an image less than a second. In Procoedines of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
[51] Zhaoqi Su, Tao Yu, Yangang Wang, and Yebin Liu. Deepeloth: Neural garment representation for shape and style editing, IEEE Transactions on Pattern Analysis and Machine Intelligence. 45(2):1581-1593, 2022.
[52] Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng. and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. European Conference on Computer Vision, pages 1-18. Springer, 2024.
[53] Shitao Tang, Jiacheng Chen, Dilin Wang, Chengzhou Tang, Fuyang Zhang. Yuchen Fan, Vikas Chandra, Yasutaka Furukawa, and Rakesh Ranjan. Mvdiffusion+++: A dense high-resolution multi-view diffusion model for single or sper 3d object reconstruction. In European Conference on Computer Vision, pages 191-191. Springer, 2024.
[54] Alexander Toshev and Christian Szegedy. Deeppose: Human pose estimation via deep neural networks. In Proceedings of the IEEE conferenec on computer vision and pattern recognition, pages 1653-1660, 2014.
[55] Gul Varol, Duygu Ceylan, Bryan Russell, Jimei Yang, Ersin Yumer, Ivan Laptev, and Cordelia Schmid. Bodynet: Volumetric inference of 3d human body shapes. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
[56] Vikram Voleti, Chun- Han Yao, Mark Boss, Adam Letts. David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view synthesis and 3d generation from single image using latent video diffusion. arXiv preprint arXiv:2403. 120000 2024.
[57] Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xiang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, and Jun Zhu. Crm: Single image to 3d textured mesh with convolutional reconstruction model. In European Conference on Computer Vision, pages 57 74, Springer, 2024.
[58] Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Ecro P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4):600-6 612000661,2220 2004.
[59] Kailu Wu, Fangfu Liu, Zhihan Cai, Runjie Yan, Hanyang Wang. Yating Hu, Yueqi Duan, and Kaisheng Ma. Unique3d: High-quality and efficient 3d mesh generation from single image. In The Thirty-cighth Annual Conference on Neural Information Processing Systems, 2024.
[60] Shuang Wu, Youtian Lin, Feihu Zhang. Yifei Zeng, Jingxi Xu, Philip Torr, Xun Cao, and Yao Yao. Direct3d: Scalable image-to-30 generation via latent diffusion transformer. arXiv preprint arXiv:2405. 14832, 2024.
[61] Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation. arXiv preprint arXiv:2412.01506, 2024.
[62] Yuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas, and Michael Black. Econ: humans optimized via normal integration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 312-523, 2023.
Explicit clothed
[63] Yuliang Xiu, Jinlong Vang, Dimitrios Tzionas, and Michael Black. Icon: Implicit clothed humans obtained from normals. In 2022 IERE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13286-13296. IEBE, 2022.
[64] Jiale Xu, Weihao Cheng. Yiming Gao, Xintao Wang. Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from single image with sparse-view large reconstruction models. arXiv preprint MxXiv22404.07199 2024.
13

---

[65] Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng. Yujun Shen, and Gordon Wetzstein. Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation. In European Conference on Computer Vision, pages 1-20. Springer, 2024.
[66] Yuxuan Xue, Xianghui Xie, Riccardo Marin, and Gerard Pons Pons-Moll Moll. Human 3diffusion: Realistic avatar creation via explicit 3d consistent diffusion models. arXiv preprint arXiv:2406.08475 2024.
[67] Xueting Yang. Yihao Luo, Yuliang Xiu, Wei Wang. Hao Xu, and Zhaoxin Fan. D-if: Uncertainty- aware human digitization via implicit distribution field. In Proceedings of the IERF/CVF International Conference on Computer Vision (ICCV), pages 9122-9132, October 2023.
[68] Yifan Yang, Dong Liu, Shuhai Zhang, Zeshuai Deng, Zixiong Huang, and Mingkui Tan, Hilo: Detailed and robust 3d clothed human reconstruction with high-and low-frequency information of parametric models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattemn Recognition, pages 1067 1- 10681, 2024.
[69] Yuhang Yang. Fengqi Liu, Yixing Lu, Qin Zhao, Pingyu Wu, Wei Zhai, Ran Yi, Yang Cao, Lizhuang Ma. Zheng- Jun Zha, et al. Sigman: Scaling 3d human gaussian generation with millions of assets. arXiv preprint wxXiv:25040069
2025.
[70] Tao Yu, Zerong Zheng, Kaiwen Guo, Pengpeng Liu, Qionghai Dai, and Yebin Liu. Function4d: Real-time human volumetric capture from very sparse consumer rgbd sensors. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5746-5756, June 2021.
[71] Biao Zhang. Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neural fields and generative diffusion models. arXiv preprint arXiv:2301. 11445, 2023.
[72] Hongwen Zhang, Yating Tian, Yuxiang Zhang, Mengcheng Li, I Liang An, Zhenan Sun, and Yehin Liu. Pymaf-x: Towards well-aligned full-body model regression from monocular images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10):12287-1230
2023.
[73] Longwen Zhang, Ziyu Wang. Qixuan Zhang, Qiwei Qiu, Angi Pang, Haoran Jiang, Wei Yang. Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creating high-quality assets. ACM Transactions on Graphics (TOG), 43(4): 20, 2024.
[74] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836-3847, 2023.
[75] Richard Zhang, Phillip Isola, Alexii A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effec- tiveness of deep features as perceptual metric. In CVPR, pages 3965595.00 Computer Vision Foundation / IEEE Computer Society, 2018
[76] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effec- tiveness of deep features as perceptual metric. In CVPR, pages 586-595. Computer Vision Foundation! IEEE Computer Society, 2018
[77] Zechuan Zhang, Li Sun. Zongxin Yang. Ling Chen, and Yi Yang. Global-corrlaatd 3d-decoupling transformer for clothed avatar reconstruction. Advances in Neural Information Processing Systems. 36, 2024.
[78] Zechuan Zhang, Zongxin Yang, and Yi Yang. Sifu: Side-view conditioned implicit function for real-world usable clothed human reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9936-9947, 2024.
[79] Ziho Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang. Yifei Feng. Mingxin Yang, Sheng Zhang, Xianghui Yang, et al. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation. arXiv preprint arXiv:2501.12202. 2025.
[80] Zibo Zhao, Wen Liu, Xin Chen, Xiansang Zeng. Rui Wang, Pei Cheng, Bin Fu, Tao Chen, Gang Yu, and Shenghua Gao. Michelangelo: Conditional 3d shape generation based on shape-image-tex aligned latent representation. Advances in neural information processing systems, 36:73969 73982, 2023.
[81] Ruichen Zheng, Peng Li. Haoqian Wang, and Tao Yu. Learning visibility field for detailed 3d human reconstruction and relighting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter Recognition (CVPR), pages 216-226. June 2023.
14

---

[82] Zerong Zheng, Tao Yu, Yebin Liu, and Qionghai Dai. Pamir: Parametric model-conditioned implicit representation for image-based human reconstruction. IEEE transactions pattern analysis and machine intelligence, (466)3170-31833 2021.
[83] Zerong Zheng. Tao Yu, Yixuan Wei, Qionghai Dai, and Yebin Liu. Deephuman: 3d human reconstruction from a single image. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
[84] Yiyu Zhuang. Jiaxi Lv, Hao Wen, Qing Shuai, Ailing Zeng. Hao Zhu, Shifeng Chen, Yujiu Yang, Xun Cao, and Wei Liu. Idol: Instant photorealistic 3d human creation from a single image. arXiv preprint MxXv:2412.14963. 2024.
[85] Qi Zuo, Xiaodong Gu, Lingteng Qiu, Yuan Dong, Weihao Yuan, Rui Peng. Siyu Zhu, Liefeng Bo, Zilong Dong, Qixing Huang, et al. Vidcomv: Consistent multi-view generation based on large video generative model. 2024.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims presented the abstract and introduction accurately reflect core contribu- tions and scope of the paper. They are well-supported by experimental results, demonstrating both validity and credibility.
Guidelines:
The answer NA mcans that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be pereeived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine include aspirational goals motivation long it is clear that these goals attained by the paper.
2.
Limitations
Question:
Does paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Our method inherits certain constraints from training data. First, since our training datasct is rendered with uniform light source, reconstructed textures may exhibit artifacts under extreme lighting conditions Moreover, our multiview generation model is fine-tuned from SD 2.1 using only ~5,000 human scans, so its generation quality is still constrained.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well- specilsccciiiainnn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
authors should reflect on scope the claims made, e.g.. the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to text system might not be used reliably to provide closed captions
online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: This work do not contain theoretical result.
Guidelines:
16

---

The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should he numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We report the experiment details and we will release related codes.
Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution novel architecture, describing the architecture fully might suffice, if the contribution is specific model and empirical evaluation, it may be necessary to either make possible others to replicate the model with same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results. access to hosted model (e.g.. in the of large language model), releasing of model checkpoint, or other means that are appropriate to research performed.
While NeurIPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily architecture clearly and fully.
model architecture, the paper should describe the
(c) If the contribution is a new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results way to reproduce the model (e.g.. with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access the model is limited in way (e.g.. registered users), hut should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code. with sufficient instructions to faithfully reproduce the main experimental results, as descrihed in supplemental material?
Answer: [No]
Justification: Our evaluation uses the public datasets. We do not provide code in Supplementary Material. But they will be made publicly available once they have been fully prepared.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines htttpe://nips..ccpubbb
guides/ CodeSooodeeeaa ietbbmmsssionPooo Stt.uuuaissiioooooooo.oooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
17

---

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines htttps://nips.cc/putte guidess/ooeeuubiisssoooooo..
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for new proposed method and baselines. If only a subset experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should relcase anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental sefting/details
Question: Does the paper specify all training and test details (e-g.. data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We discuss the training and test details in the section on experiments and supplementary
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that necessary appreciate the results and make sense them.
The full details can be provided either with the code, in appendix, or supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments?
Answer: [Yes]
Justification:
The reported quantitative evaluations are listed in Tab. 1 Tab. and Tab. B
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It OK report I-sigma error hars, but one should state it. The authors should preferably report a 2-sigma error har than state that they have 96% CI, if the hypothesis of Normality errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures
symmetric eиror bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Answer: [Yes]
Justification:
We train SyncHuman with 8 H800 GPUs.
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster. or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethies httts:://eerrip: http://errrppp.cccppuil/i//aaaaa wc/ppublic/Bbbiccoouddee......
Answer: Yes]
Justification:
This work conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure preserve anonymity (e.g., if there is special considcration duc laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts the work performed?
Answer: [Yes]
The paper includes the Broader Impacts statement in subsection Ethics Statement in Supp.
Justification:
Guidelines:
The answer NA means that there is societal impact of the work performed. If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance). fairness considerations (e.g., deploy- ment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms monitor- ing misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: To mitigate the risks associated with our technology, we are implementing the following dreguurrdsImplementtee
email checking system that requires users to acknowledge and agree to guidelines before accessing our model.Integratinn pre-trained facial recognition and human body detection models to identify and flag potentially sensitive content.Incorponatt Stable Diffusion to filter sensitive and harmful input images, including Content Filtering, NSFW (Not Safe For Work) Detection, Watermarking and Tracing. Our code and pre-trained model will be released under strict licenses (like Ethical Source License) that explicitly prohibit illegal or unethical use.
safeguard strategies in
19

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk misuse or dual-use should he released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: This paper cites the related datasets and codes used in work.
Guidelines:
The answer means that the paper does not use existing assets.
authors should cite the original paper that produced the code package or dataset.
authors should state which version the asset is used and, possible, include a URL.
name of license (e.g.. CC-BY 4.0) should be included each asset. scraped data from particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paperswi wwwerdewthcoee..... com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release assets.
Guidelines:
The answer means that the paper does not release new assets.
Researchers should communicate details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training. license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Although the method involves 3D human models, rely on datasets collected before this work; we refer to them for their specifics{ 4,5,6, [1] Function4D: Real-time Human Volumetric
Capture from Very Sparse RGBD Sensors [2] High-fidelity 3D Human Digitization from Single 2K
Resolution Images [3] Learning Locally Editable Virtual Humans [4]DeepCloth: Neural Garment Representation for Shape and Style Editing [S]Learin Dress 3D People in Generative Clothing
[6]X- -vatar:: Expressive Human Avatars
Guidelines:
20

---

The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then much detail as possible should he included in the main paper.
According to the NeurlPS Code of Ethics, workers involved data collection, curation, other lahor should be paid at least. minimum wage in country data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing rescarch with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this the paper.
recognize that procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such as the institution conducting the review.
Declaration LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original. or non-standard methods in this research? Note that if the LLM is used only for writing, editing. or formatting purposes does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
component of the
Answer: [NA]
Justification: Our method does not involve LLMs as any important, original, or non-stame non-standard compo- nents.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy http:://eeaarrpeeecc/////eeeeeee for what should or should not be deseribed.
21

---

A Details
A.1 Training Details
2D-3D Cross-Space Generative Model. Our 2D-3D Cross-Space generative model was trained on 8 NVIDIA H1800 GPUs. For the multiview generative model branch, we adopt the architecture of PSHuman (25] retrain it using flow matching from the open-source pre-trained text-to-image generation model, SD2. lunclli 4% We train the multiview generation branch separately with a batch size of 32 for a total of 30,000 iterations. We adopt an adaptive learning rate schedule, initializing the learning rate 1c-4 and decreasing it 5e-5 after 2,000 steps. For 2D-3D Cross Cross-Space generative model, we initialize the network weights using: the fine-tuned weights from our multiview generation branch (as described above). a tre-trained image-to-3D model (Trellis [61].. Additionally, we perform zero-initialization on the output layer of the 2D-3D synchronization attention module. We train the 2D-3D Cross-Spas Pross-Space generative model with batch size of 32 for total of 50,000 iterations. We adopt an adaptive learning rate schedule, initializing the learning rate at 2.5c-5 and decreasing to 1.25e-5 after 2,000 steps. enable class-free free guidance (CFG) during inference, we randomly omit the image condition at a rate of 0.05 during training.
Multiview Guided Decoder. Our Multiview Guided Decoder was trained on NVIDIA H800 GPU. We train the decoder with a batch size of 4 for total of 14,000 iterations, using a learning rate of 1e-4.
The loss design largely adheres to Trellis [61]ss 's setup. For the GS decoder, the loss includes reconstruction loss and regularization loss, with regularizations employed for the volume and opacity of the Gaussians to prevent their degeneration, specifically to avoid them becoming excessively large or transparent. Lreca is composed of Li (L1 loss), Structural Similarity Index (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS). The full training objective is defined as follows:
Lus Lrecon + Lvot + La
(5)
where:
Lreeen
L1
+ 0.2001111111
SSIM) + -PIPS,
L K
1
Lut
LK
(6)
K
La = T.K
-E ky2
where I. is the total number of active voxels.For each active voxel, K Gaussians are predicted,s and a are the scale and opacity of Gaussian, respectively.
For the mesh decoder; we utilize Nydiffrast 21 to render the extracted mesh along with its attrihutes, producing a foreground mask M, depth map D, a normal map Nm. directly derived from the mesh an RGB image C, and a normal map N from the predicted normals, a normal map N directly derived from the mesh from the front view, an RGB image froit from the front view. The training objective is then defined as follows:
Lm Leoo + 0.4Lcolor + Leegs
where Lyoo and feoter are written as:
Egeo (M( + 10 (D) + freeon (N m) + 0. 0.1 (innt Leotor = Liecou (C) + Lrecon (N) 0.1 Creoon (C
(8)
(9)
Here, Lieson is defined identically to Eq. (7), Finally, Luep consists of three terms:
Lreg = Leonsiet + Laev + 0.01 Lisat,
(10)
where Lommia penalizes the variance of attributes associated with the same voxel vertex, Caw is a regularization.
A.2 Detailed Network Structure
2D-3D synchronization attention. In the 3D branch, we inserted two 2D-to-3D attention blocks after the 8th and 16th transformer blocks respectively. Similarly. for the 2D branch, we added two 3D-to-2D attention blocks following the 3rd CrossAttDDowBBlockMWt
and the UpBlock2D modules.
(1)2D-to-3D attention. Each 3D voxel feature projected onto four view normal map planes (front, back, left, right) to obtain corresponding 2D pixel features:
Rd with coordinates (x.,, V,, x,) is orthographically
p = T. (n:), V E front, back, left, right
(1))
where P E de is the projected 2D features, and (.(+) is the orthogonal projection function.
22

---

The 3D voxel feature and 2D pixel feature pK are respectively passed through the MLP transformatton:
4 MLP, (u,)
kj MLPP&(P.) v." MLP.((p.)
(12)
Using 3D voxel feature as queries and concatenating four 2DD pixel features along the sequence dimension as keys and values, compute cross-attcntion:
K. Concat
(k!mmi front kiack k
V. Concat
vink ven
(13)
K.K. Vd
u u. + MLP
Softmax
V:
Here, u' represents the updated 3D voxel feature.
(2)3D-to-2D attention, Let the input consist of 2D pixel features p, E IR from color map normal map, with a corresponding 3D voxel space represented by U e R Xxxxxxxd,, where d, and du denote the feature dimensions 2D pixels and 3D voxels, respectively.
Each 2D pixel feature p. corresponds to ray in 3D space. Sampling H 3D voxel features along this ray forms a 3D voxel feature sequence:
U = ( u., E R
(14)
Where ú is the 3D voxel feature at the j-th depth position along the projection ray of 2D pixel feature p.. H is the length of the 3D voxel feature sequence.
The 2D pixel feature pt mapped to query vector, while each 3D voxel feature u., is mapped key and value vector:
q = (p.,(p.) k., = MLPx(u.,)) VL, = MLP,(u.,)
(15)
By concatenating all key vectors and value vectors across the sequence of 3D voxel features, we construct the complete key and value matrices
1 k. k..,] E "xd
V.
[ve1, Va2...,, vs,a] E Rxd
(16)
Compute attention output with 2D pixel feature qi as the query, and the 3D voxel feature K; and V; as the key and value:
q. Vd
P + MLP
(Softmax
V,
(17)
Here, P represents updated pixel feature.
Multiview Guided Decoder (MVGD). The sparse AruutureVgri first processed by the Structured Latents generative model, which denoises it into structured latent z. For multiview color and normal images, we first upsample the images, then we extract multilevel local patch features using the DINOv2 backbone from layers 1 E (4, 11, 17, 23} per image:
F (1
R VxVxd
(18)
The features from different layers are concatenated and then processed through a MLP form the final representation:
F. MMLPaCon (F("),((1)) (11), F(17) F FF?3:) E
Rxvxd
(19)
Given voxel position p (ac, y, z). its projection onto i-th view yields the corresponding pixel coordinates:
n (p) (us, v.), where E 0, 0. V 1.
(20)
The corresponding image features then retrieved via direct indexing:
f. (p) = F [u, vi] E IRd.
(21)
The injection feature constructed by concatenating structured latent at position p (Zp e =ed=) with multiview pixel-aligned features obtained from the color and normal maps of 4 views (8 feature vectors in total).
Formally,
Zan Concat Zp: f, (p)....
(p)) E ddd+
(22)
This injection feature is then processed by an MLP to refine the structured latent representation:
zpp = Zp + MI.((zin) dRd.
(23)
We insert a multiview injection module after each self- attention in the decoder. We apply the same multiview feature injection mechanism to both the mesh decoder and the GS decoder, resulting in refined mesh and GS representations.
23

---

Input
Hunyuan3 2.5 77
IDOL 84
LHM
41
Ours
Figure 11: Qualitative comparison of SyncHuman with Gaussians- methods (LHM, IDOL) and a native 3D model (Hunyuan3D 2.5). SyncHuman achieves visually high-fidelit results.
B
More Experiment
B.1 More Results
Comparison with Gaussians-bassed Methods and Native 3D generative model. To further evaluate the effectiveness of our method, we conduct qualitative comparison between SyncHuman and two Gaussians-basso methods (LHM (41) and IDOL
84 ), as well as more advanced native 3D model, Hunyuan3D 2.5 (79], as shown in Fig. TI All these methods are capable of producing structurally plausible and visually reasonable results. Since LHM and IDOL are based on Gaussians, they can only produce RGB images through rendering. For comparison,
render RGB images from the front view. Both LHHM and IDOL rely SMPL, and when SMPL estimation is inaccurate or fails, the resulting structure correspondingly erroneous. Furthermore, as illustrated in Fig. [|| IDOL and LHM still exhibit limited fidelity. Hunyuan3D 2.5, trained on a large-scale dataset, is a native 3D model that can also produce reasonable human structures with details. However, as observed in Fig. Hunyuan3D 2.5 produces human meshes with less fidelity.

---

Figure 12: After alignment using 2D-3D attention, the multi-view projections of the branches can almost completely overlap.
Input
Ours
Without 2D-3D attention
Figure 13: Comparison of the quality of intermediate multi-view generation with and without 2D-3D attention.
B.2 Ablation of the quality of intermediate multi-view generation and 3D structure generation
We additionally report quality of intermediate multi-view generation and structure generation on a small human scan subset. IOU of the 3D structure generation: 0.5907 (with 2D-3D synchronization attention) vs. 0.4813(wiithn) 2D-3D synchronization attention). Color and normal image quality improvements by 3D-2D attention:
Table 4: Comparison of different methods.
Method
PSNR1
SSIM
LPIPS.
w/o att (color)
23.328
0.877
0.078
ours (color)
24.027
0.894
0.070
w/o att (normal)
22.851
0.866
0.097
ours (normal)
23.439
0.882
0.087
Because generating 3D structures or multiview images from single-view inputs has ambiguity, the generation results exactly the more similar to GT. This demonstrates that our 2D-3D synchronization attention could benefit both branches to improve the multiview generation quality and 3D structure quality. As Fig, [2] after alignment using 2D-3D attention, the multi-view projections of the two branches can almost completely overlap. And as in Fig. [3 when with 2D-3D attention multiview images have
the ground-truth. However, our 2D-3D attention could produce results
reasonable human body structure.
25

---

Figure 14: Visualization of the unconditional generation task.
Figure 15: In some cases, the decoded mesh may contain some holes on the surface.
B.3
Inference time.
On single H800, the inference time follows: 38.57s Trellis 15.68s PSHuman 52.98s Our method faster than PSHuman directly decodes shape without requiring additional differentiable rendering optimization. The slower speed compared to Trellis is due to our use of the 2D multi-view generation.
B.4 Unconditional Generation.
We tested our model on the unconditional generation task as in Fig. conditional generation from single-view image.
14 The generation quality is worse than the
C
discussion
C.1
Limitations about containing holes in the generation
Our method is based on Trellis 61). which FlexiCube 48 in the trellis mesh decoder branch does put
water-tight tight constraint on the surfaces. Thus, holes may appear on surface in some cases, as shown Fig. [5]
A possible way to make the generated meshes water- tight is to adopt another SDF fitting on the generated mesh. Alternatively, we may adopt other 3D native generative models using SDFs targets, like Hunyuan3D 79 or TripoSG [27). avoid this problem. We leave this for future work.
26

---

C.2 Differences from SyncDreamer.
Our method fundamentally differs from SyncDremmer in the following two aspects. First, the synchronization subjects totally different. SyncDreamer synchronizes the generation of multiview 2D images, whereas method synchronizes the 2D generative model and simultaneously generating multiview images and 3D representations benefits each other and greatly improves the 3D generation quality. We inject information in both directions, 2D to 3D and 3D to 2D. Second, the functionality and design of the volume in our method are fundamentally different from those in SyncDreamer. SyncDreamer constructs feature volume share information among different views. In contrast, in our method, the volume is meaningful 3D representation generated from noise.
native generative model.
method demonstrates that
C.3 Ethics Statement
The objective of SyncHuman is to equip with powerful tool for creating realistic clothed 3D human models. By enabling 3D human generation from a single image, our method supports diverse ethnicities and populations, promoting equitable cultural representation. Our model was trained on the public datasets THuman2.1 770.. CustomHumans [12. THuman3.0 [5]], and 2K2K (10) and tested on X-Humans 477 and
CAPE [32] However, there is potential risk that these generated models could be misused to deceive viewers (e.g., adult content, political manipulation, exploitation of artists via digital replicas.). It noted that this issue not unique to our methodology but prevalent other human generative methodologies. Therefore, absolutely essential for current and future research in the field of 3D human generative modeling to address and reassess these considerations consistently.
27


Reviews:
[{"review_id": "CSV81AfVHY", "reviewer": "Reviewer_Pinc", "summary": "This paper proposes SyncHuman, a novel approach for single-view 3D clothed human generation. The method effectively combines the strengths of 2D multiview diffusion models and native 3D diffusion models through pixel-aligned 2D-3D synchronization attention and a multiview feature injection strategy. After fine-tuning on a collected dataset, the proposed method achieves impressive photorealistic 3D full-body human reconstruction from a single image.", "questions": "Specifically, I have a few questions regarding the paper and would appreciate it if the authors could provide clarification:\n1. During the 2D to 3D attention process, how are the resulting features added into the original 3D voxel features? Are the features averaged, concatenated, or combined in another way? Is this cross-attention mechanism applied at every diffusion timestep, or only at selected timesteps? Similarly, how are features incorporated during the 3D to 2D attention process? More detailed explanations of these mechanisms would be helpful.\n\n2. During joint training of the 2D and 3D models, are all models trained together, or are some model weights frozen? Is MVGD involved in the joint training process? Please provide more details on the training procedure, such as the specific process of the training, the amount of data used, the number of training epochs, and relevant hyperparameters.\n\n3. In the ablation experiments for the effectiveness of 2D-3D cross-space attention, could the authors compare the generation results of the 2D diffusion model before and after applying cross-attention? Theoretically, the 2D diffusion model with cross-attention should produce more cross-view consistent multi-view results compared to the original model. Similarly, does the 3D diffusion model show corresponding improvements after cross-attention is applied? Additional experiments to support these claims would strengthen the paper.\n\n4. It appears that two decoder branches were trained: 3DGS and mesh. Which branch’s results are shown in the experimental section? How significant are the differences between the two branches? It would be helpful to present results from both branches for comparison.\n\n5. In Fig. 6, although the overall pose and texture of the generated results are consistent with the input image, there are still noticeable inconsistencies in many details, such as the face shape in the first and third examples, and the skin tone in the last example. Additionally, from the video results, the generated humans still suffer from blurriness and white artifacts.\n\n6. The paper is missing several relevant references, including:\n1)\tConTex-Human: Free-View Rendering of Human from a Single Image with Texture-Consistent Synthesis\n2)\tHumanRef: Single Image to 3D Human Generation via Reference-Guided Diffusion\n3)\tDINAR: Diffusion Inpainting of Neural Textures for One-Shot Human Avatars\n4)\tZeroAvatar: Zero-shot 3D Avatar Generation from a Single Image", "limitations": "No, the paper does not provide an explanation for the artifacts present in the results. Perhaps a related discussion could be added to the Limitations section.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe approach successfully integrates the advantages of both 2D and 3D diffusion models. The experimental results convincingly demonstrate the effectiveness of the proposed design.\n\nWeaknesses:\nThe paper is missing relevant citations and lacks technical details and necessary discussion and analysis.", "was_revised": true, "final_justification": "The authors’ response has largely addressed my concerns. I agree to accept the paper on the condition that the supplementary experimental results, especially the visualizations for Q3, are included in the camera-ready version.", "rebuttal": "We sincerely thank the reviewer for these insightful comments and positive words like \"successful combination of 2D-3D diffusion models\" and \"convincingly demonstrate the effectiveness\". Hope the following responses address all your concerns.\n\n### Q1: Details of 2D-3D Cross-Space Diffusion\nThe synchronization attention is used at all denosing steps to maintain the consistency and improve the quality of the two branches. Both the 2D-to-3D and 3D-to-2D attention are implemented via cross-attention (Fig.4). In 2D-to-3D attention,  each voxel serves as a query, and the concatenation of corresponding orthogonal sample embeddings on multiview images is used as keys and values. In 3D-to-2D attention, each pixel-level embedding serves as a query, and the corresponding 3D voxel embedding in projected 3D space is the key and value. The resulting features are processed with a zero-initialized MLP and are added back to the denoising branch. The detailed implementation is elaborated on lines 35-62 of the supplementary material, and we will add more to make it clearer in the revision. \n\n### Q2: Add more training details \nThe 2D multiview generative model is initialized with SD2.1, the 3D structure generative model, and MVGD are initialized from Trellis. The training process includes two stages. At the first stage, the 2D multiview generative model ,the 3D structure generative model and 2D-3D synchronization attention are jointly tuned by tuning all parameters. At the second stage, we train MVGD. The training dataset is composed of publicly used 3d human scans, including THuman2.1, CustomHumans, THuman3.0, and 2K2K. We have included implementation details in lines 1-34 of the supplementary materials and will add more in the revision. We will release all code for the reproducibility of our method.\n\n### Q3: Could the 2D-3D synchronization attention improve the multiview generation quality or the 3D structure quality? \nIn our ablation study, we have reported the rendering quality and the geometry accuracy of the final reconstructed 3D meshes with or without 2D-3D synchronization attention. Following your suggestions, we additionally report the quality of intermediate multi-view generation and 3D structure generation. \nIOU of the 3D structure generation:  0.5907 (with 2D-3D synchronization attention) vs. 0.4813(without 2D-3D synchronization attention).\nColor and normal image quality improvements by 3D-2D attention:\n| Method          | PSNR↑   | SSIM↑  | LPIPS↓ |\n|-----------------|--------|--------|--------|\n|w/o att (color)  | 23.328 | 0.877  | 0.078  |\n|ours (color)    | **24.027** |**0.894** | **0.070** |\n|w/o att (normal) | 22.851 | 0.866  | 0.097  |\n| ours (normal)   | **23.439** | **0.882** |**0.087** |\nBecause generating 3D structures or multiview images from single-view inputs has ambiguity, the generation results are not exactly the same as the ground-truth. However, our 2D-3D attention could produce results more similar to GT. This demonstrates that our 2D-3D synchronization attention could benefit both branches to improve the multiview generation quality and 3D structure quality. We will add this result to the revision following your suggestions.\n\n### Q4: How to combine the 3DGS and mesh decoding results\nWe follow Trellis to bake the GS renderings onto the generated mesh because 3DGS has better rendering quality, while meshes have better surface geometry. We have included this in L198-199 and will explain it more to make it clearer.\n\n### Q5: Face inconsistency in Fig. 6 and highlight artifacts in videos.\nIn Fig.6, we have rotated the 3D models a little bit to show they are in 3D space, which leads to inconsistency with the input image due to the viewpoint changes of the faces. We will clarify this in the revision.\nWe agree that our method is currently not robust to highlights, and we have included this in our limitations discussion. When a white highlight appears at the edges of the human in the input image, our method may generate some white patterns in the back view. One possible solution may be to add some augmentations to improve the robustness, which requires scarce relightable 3D avatars to render under different illumination conditions.\n\n### Q6: Missing references.\nThanks for pointing this out. ConTex-Human, HumanRef, and ZeroAvatar are SDS-based methods that rely on time-consuming per-instance optimization. DINAR heavily depends on SMPL estimation and UV-based texture generation, which restricts its ability to handle loose clothing. In contrast, our approach reconstructs 3D humans using a generative model in a SMPL-free manner, significantly improving both efficiency and robustness. We will add these citations and discuss them in the related works."}, {"review_id": "5rFxex2ffS", "reviewer": "Reviewer_6GgC", "summary": "This paper presents a method to reconstruct detailed 3D human from single RGB image. The idea is to combine the texture details from multi-view diffusion model with structure priors from TRELLIS. They introduce cross attention module to synchronize the 2D features from multi-view diffusion models and 3D features from the sparse grid of TRELLIS. The synchronization is done via pixel aligned orthographic projection. They further inject the multi-view features into sparse voxels before final decoding to enhance the details. Experiment resutls on two datasets demonstrate the effectiveness of the proposed pipeline. Ablations also show the importance of the proposed modules.", "questions": "- How would this model compare to Hunyuan3D 2.0? As a more advanced version of TRELLIS, HY3D shows impressive details and on par texture details. It would be interesting to see if an off-the-shelf general object reconstruction model can already work well for the specific human reconstruction task. If not, what is the missing gap?\n\n- Strangely large errors (P2S, CD) from TRELLIS while TRELLISS (fine tuned) is much better, table 1-3. Did you do proper alignment (e.g. SiTH [1]) between reconstruction and GT mesh before reporting the numbers? Or please explain the gap between trellis and trellis-fine tuned. \n\n[1]. SiTH alignment for evaluation: https://github.com/SiTH-Diffusion/SiTH/blob/main/EVALUATE.md#1-single-view-human-3d-reconstruction-benchmark", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strength**\n- The overall writing flow is clear. \n- The proposed method works well on the evaluated benchmark datasets. \n- Ablation studies prove the effectiveness of the proposed modules. \n\n**Weakness**\n- Originality: the method overall is not very novel. The synchronized attention is pretty much the same as SyncDreamer, even the name is the same pattern -- SyncHuman. Even the MVGD module is also similar to SyncDreamer where they inject 3D volume feature to 2D multi-view but here it is in the opposite direction: 2D to 3D. \n- Clarity: Some method details are not clear. Is the 2D-3D synchronized attention done at each reverse sampling step? Eq. 3: flow matching loss is used here but in all previous text the term 'diffusion models' are used. Although mathematically they are very similar, but it is important to clarify them to be precise.\n- Clarification of the experiment setup: L240-'subset of X-humans', is it the same test set used in table 1? Also in table 2 and table 3, image metrics like PSNR/SSIM are reported, are they the generated 2D multi-view images or re-rendering of the reconstructed 3D? Although the end goal is to have better 3D, but it would be interesting to see if this cross attention mechanism would also improve the 2D multi-view diffusion model. Would this also improve the predicted multi-view normal?", "was_revised": true, "final_justification": "The author response has addressed my concerns. I will keep my original rating as weak accept.", "rebuttal": "We are grateful for your valuable feedback. We feel encouraged by your positive comments like \"works well on the evaluated benchmark datasets\". We hope the following responses address all your concerns..\n\n### Q1: Differences from SyncDreamer\nOur method fundamentally differs from SyncDreamer in the following two aspects. \nFirst, the synchronization subjects are totally different. SyncDreamer synchronizes the generation of multiview 2D images, whereas our method synchronizes the 2D generative model and the 3D native generative model. Our method demonstrates that simultaneously generating multiview images and 3D representations benefits each other and greatly improves the 3D generation quality. We inject information in both directions, 2D to 3D and 3D to 2D.\nSecond, the functionality and design of the volume in our method are fundamentally different from those in SyncDreamer. SyncDreamer constructs a feature volume to share information among different views. In contrast, in our method, the volume is a meaningful 3D representation generated from noise.\nWe have discussed SyncDreamer and related multiview diffusion methods in the paper, and will further clarify this according to your suggestion.\n\n### Q2: Is the 2D-3D synchronization attention done at each reverse sampling step? \nYes. The 2D-3D synchronization attention is used at all denosing steps to maintain consistency between 2D and 3D and improve the quality of both.\n\n### Q3: Be more careful about the terms flow matching and diffusion.\nOur model adopts flow matching to train both 2D and 3D generative models. Thanks for pointing this out, and we will rectify this in the revision.\n\n### Q4: L240-'subset of X-humans', is it the same test set used in Table 1?\nYes, the ablation study uses the same X-Humans subset as Table 1.\n\n### Q5: How do you evaluate image quality?\nThe metrics for image quality in Tables 2 and 3 are computed from the renderings of 3D meshes for all methods. For each mesh, we render four views with a 768 resolution and report the average PSNR, SSIM, and LPIPS. \n\n### Q6: Would the 2D-3D attention improve multiview generation quality?\nYes. Following your suggestion, we additionally report the quality of intermediate multi-view generation.\n| Method          | PSNR↑   | SSIM↑  | LPIPS↓ |\n|-----------------|--------|--------|--------|\n|w/o att (color)  | 23.328 | 0.877  | 0.078  |\n|ours (color)    | **24.027** | **0.894** | **0.070** |\n|w/o att (normal) | 22.851 | 0.866  | 0.097  |\n| ours (normal)   | **23.439** | **0.882** | **0.087** |\nThrough our 2D-3D attention, the 3D representations effectively associate multiview images for better quality and multiview consistency. We will add this in the revision.\n\n### Q7: Comparison with Hunyuan3D\nIn Fig.1 of the supplementary material, we conduct a qualitative comparison with the commercial model Hunyuan3D-2.5 (not open-sourced yet). Although Hunyuan3D-2.5 could produce plausible structures, it struggles with details and appearance fidelity, which is a common issue for existing native 3D generative models. \nThis gap arises because Hunyuan3D only applies cross-attention layers to associate the input image and 3D shape, lacking pixel-aligned information. In contrast, our 2D-3D synchronized attention and MVGD incorporate pixel-aligned features, significantly improving structural and visual fidelity.\n\n### Q8: Why is there a performance gap between Trellis and tuned Trellis? Are the generated meshes aligned with GT?\nFor both tuned and pretrained Trellis models, we apply mesh normalization and ICP registration to align the output meshes with the gt meshes to ensure a fair comparison. The performance gap between the original and tuned Trellis stems from two factors. First, while the original Trellis is trained on diverse datasets containing only limited human meshes, our tuned version is specifically optimized using high-quality 3D human scans. Second, the original model generates meshes in a canonical space without aligning with the input images. In contrast, our fine-tuned Trellis model generates meshes in a coordinate system aligned with the input views. This alignment correlates the 3D shape with the input image strongly and thus significantly improves both accuracy and spatial consistency. We will add this discussion to the revision following your suggestion."}, {"review_id": "hUqNhFMPFi", "reviewer": "Reviewer_dgjc", "summary": "This paper introduces SyncHuman, a method for reconstructing 3D textured human meshes from a single image by using 2D multiview diffusion and native 3D diffusion models. The framework uses a synchronization attention mechanism to bridge the two models, then uses a multiview-guided decoder to further inject fine details from generated 2D images into the 3D latent space. Experiments on CAPE and X-Humans show that SyncHuman achieves good performance and quality.", "questions": "See weaknesses.\nThe authors need to justify the numbers reported in the paper. Explain why it is worse than the reported numbers by a significant margin (larger than the performance improvements). Is it an implementation error by the authors or evaluation bugs? In general, I have completely lost trust in the numbers I see in the paper, and the qualitative results of the previous method could be downplayed as well (compared to the qualitative results reported in their own paper).", "limitations": "As disclosed in the paper, it is known that multi-view diffusion models are trained on idealized shadowless objects, so they cannot handle strong lighting. This limits its real-world applications. As mentioned in the weaknesses, the method may suffer from overly smoothened textures, but the authors did not mention relevant weaknesses.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The qualitative results are somewhat intriguing.\n2. The method avoids using SMPL, which has several known issues listed in the paper.\n\nWeaknesses:\n1. While the synchronized attention implementation is somewhat novel, the idea of enhancing geometry using a multi-view diffusion model is a widely adopted idea. The approach is primarily a well-engineered system combining two pretrained diffusion models.\n2. The numbers reported in Table 1 are different from the original numbers reported in their own paper (e.g., PSHuman). These significant downplaying remarks the potential and intentional unfavorable calibration to the evaluation, questioning the trustworthiness of certain baselines that were not evaluated on the benchmark (e.g., TRELLIS). The only interesting part of the paper is its outstanding performance shown in Table 1, but the numbers are not really reported in a responsible way. The Chamfer distances reported in the results are all unreasonably bad, which questions the generalization of the SMPL failures cases reported by the authors.\n3. The failure cases of SIFU and PSHuman in Figure 6 are significantly worse than it supposed to be. Certain distortions and limb discontinuities are so severe that iit s unbelievable.\n4. The ablation study lacks depth, and the improvements are incremental.\n5. 2D diffusion models are not very good at generating thin geometry (such as limbs and fingers), or maintain the semantic correctness (such as generating three legs, especially common in legacy StableDiffusion models). Do the authors observe similar problems? How is it addressed?\n6. The cross-view diffusion models are not robust in maintaining the geometric consistency of fine textures, such as hairs and wrinkles. Do the authors observe overly smoothened textures?\n7. The authors are extremely reckless in completing the checklist. I don't think the authors qualify to respond yes to \"7. Experiment statistical significance.\" The response to \"8. Experiments compute resources\" clearly does not read the guidelines. The answer to 10, 11, 15, simply disregards the human subject data used during the model training. Some of the training datasets do not have subject's consent. I typically don't pay too much attention to the checklist as people should have known what they are doing, but it is apparently not the case in the submission.", "was_revised": true, "final_justification": "The authors' rebuttal has addressed most of my questions. I raise my final rating to borderline reject. The concerns remain about the marginal improvements from the proposed components. The provided visual ablation figures lack clear failure modes or patterns corresponding to the main novelty of the paper. The paper lacks experimental analysis about why these new designs lead to consistent improvements and what practical problems have been \"fundamentally\" solved.\n\nBesides, I am not satisfied with the authors' original reckless attitude toward the ethical checklist. Acting ignorantly when working in a highly sensitive research domain is unacceptable. I am not very happy about this situation. Such a course of action can be waived with a simple apology. The acceptance of this paper introduces another group of reckless people getting into the community, encouraging more similar misbehavior. \n\nHowever, considering my role is not ethics reviewer, I will not fight for rejection on this.", "rebuttal": "Thank you for your efforts in reviewing our paper and providing insightful comments. We feel encouraged by your positive comments like \"intriguing qualitative results\". Hope the following responses address all your concerns.\n\n### Q1: The novelty is limited because some papers already apply multiview images to improve 3D generation.\nOur novelty lies in the effective synchronized 2D-3D generation, which shows great improvements in single-view 3D human reconstruction. While some works employ multiview generation to enhance 3D geometry, they typically adopt a sequential scheme: generating multi-view 2D images to feed into a 3D generative model, or generating a 3D model to guide the multiview generation for refinement. Such sequential methods inherently suffer from error accumulation; inaccuracies in the initial stage, like inconsistent multiview images, often limit the effectiveness of subsequent 3D generation or refinement. In contrast, our approach generates multiview images and 3D models simultaneously in a synchronized manner, enabling mutual reinforcement. Therefore, SyncHuman improves both the geometric fidelity of 3D outputs and the cross-view consistency of generated images.\n\n### Q2: The reported performances are different from those in baseline papers (e.g. PSHuman).\nThe performances are different from their papers because the single-view reconstruction has a scale ambiguity. We adopt the following scale normalization strategy for evaluation, which may differ from previous methods. We align the centers of the reconstructed mesh and the ground truth mesh and then scale them so that the coordinate range of the longest axis is 1. All evaluations in our paper were obtained by re-running the inference using the open-source codes from the relevant papers, applying a unified evaluation method. We will open-source the related evaluation code to ensure the results are reproducible.\n\n\n### Q3: The results for PSHuman and SIFU are unreasonably bad.\nWe compare against PSHuman and SIFU fairly with the same evaluation setting. \nOur reported metric numbers for PSHuman are better than the original paper. Note that in Table 1 of the PSHuman paper, the setting \"w/ SMPL-X prior\" means using ground-truth SMPL-X parameters, while \"w/o SMPL-X prior\" means using estimated SMPL-X parameters. Our method does not rely on GT SMPL, so we are comparing against the upper part (\"w/o SMPL-X prior\") of Table 1 in the PSHuman paper. \nPSHuman and SIFU heavily rely on SMPL prediction, and the SMPL errors will lead to severe reconstruction artifacts, especially for humans with complex poses in Fig.6. In contrast, our 3D generative models generate accurate 3D structures on such poses. Note that Human3Diff in Fig.6 does not utilize inaccurate SMPL estimation either and thus also shows a better structure.\n\n### Q4: Insufficient ablation, improvements are incremental.\nWe have conducted ablations to validate the effectiveness of our designs in Fig.7, Fig.8, Tab.2, and Tab.3. The results show that our 2D-3D attention effectively synchronizes 2D and 3D generation with a better quality, while our MVGD significantly enhances the geometry detail and fidelity.\n\n### Q5: Can the proposed method reconstruct thin geometry like fingers?\nYes. As shown in Rows 1-3 of Fig. 1 in the supplementary material, our method succeeds in reconstructing fingers. Though fingers are small, they can still be reconstructed from multiview generation. We agree that more accurately reconstructing fingers is still an open problem and an interesting future topic, which can be combined with parameteric hand shapes like MANO.\n\n### Q6: Will the multiview generation produce semantically incorrect results like 3 legs?\nThe vanilla multiview generation may occasionally produce such artifacts. However, by integrating 3D generation, which effectively learns human structural priors, our method significantly reduces these errors and maintains semantic correctness in the generated outputs.\n\n### Q7: Can the proposed method reconstruct fine detailed textures such as hairs and wrinkles?\nYes. As shown in Rows 1-3 of Fig.5 in the main paper, our method succeeds in reconstructing the wrinkles on the clothes. For the hair textures, as shown in Rows 2 and 3 of Fig.6 in the main paper, our method can get reasonable results for the detailed hair textures. We agree that accurately reconstructing all the textures and wrinkles still remains a challenging task, especially within our ill-posed single-view reconstruction setting, and our method still shows some blurry textures in reconstructing the exact strands of hair. Our method already shows significant improvements over previous methods by a large margin in terms of details and fidelity. We will add the oversmooth problem of hair textures in the revision.\n\n### Q8: The checklist is treated recklessly.\nThank you for pointing out that some checkpoints about ethical issues may be answered improperly. We have carefully checked the list, but we had a misunderstanding that if we all use existing public human datasets, then there would not be an ethical issue. We will revise this part according to your suggestion and discuss more about the ethical issues in the revision.\n\n### Q9: The method cannot handle strong lighting, which limits its real-world applications.\nIn the supplementary video, we provide high-quality results on real-world in-the-wild single-view images, and both our evaluation datasets, CAPE and X-Humans, are real-world datasets, which all demonstrate the potential of our method in real-world applications. \nWe have discussed the limitations that our method could produce some white artifacts in textures when strong highlights exist, but still have a reasonable geometry reconstruction. Handling such highlights may require rendering relightable 3D avatars under different lighting conditions, but such data is still scarce, and almost none of the existing single-view avatar reconstruction methods could effectively handle this problem. This could be a promising future research topic."}, {"review_id": "KUWRzbTEob", "reviewer": "Reviewer_Kbwv", "summary": "This paper proposed a framework that combines 2D multiview diffusion and 3D native diffusion, enabling high-quality clothed human mesh reconstruction from single-view images.\n\nGiving a single-view image, the method first generates 3D structures and multiview images. Then their use multi-view reconstructor to obtain 3D human textured meshes.", "questions": "Why did the author inject DINOv2 into MVGD? It requires an ablation study to demonstrate its effectiveness.     \n\nThe ablation on 2D-3D synchronization attention may unfair. The author fine-tuned the compared methods on X-Humans dataset, which only contains 100 scans from less than 20 participants. However, the propose method is trained on Thuman, 2K2K and CustomHumans which contains about more than 2,500 scans. They should fine-tuned the compared methods on all training data.\n\nThe proposed method is incremental. The author prefers to combine PShuman, Trellis, and DINOv2. The 2D-3D cross spatial diffusion simply mixes the features of two branches together", "limitations": "yes", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nThe process of 2D-3D Cross-Space Diffusion and Multiview Guided Decoder improve the final results.\n\nFrom the experimental results, it can be seen that the method proposed by the author outperforms the results of the comparative methods.\n\n\nWeaknesses\n\nWhy did the author inject DINOv2 into MVGD? It requires an ablation study to demonstrate its effectiveness.     \n\nThe ablation on 2D-3D synchronization attention may unfair. The author fine-tuned the compared methods on X-Humans dataset, which only contains 100 scans from less than 20 participants. However, the propose method is trained on Thuman, 2K2K and CustomHumans which contains about more than 2,500 scans. They should fine-tuned the compared methods on all training data.\n\nThe proposed method is incremental. The author prefers to combine PShuman, Trellis, and DINOv2. The 2D-3D cross spatial diffusion simply mixes the features of two branches together.\n\nRepeat References. The [40&41], [71&72] references are same.\n\nMissing references \nDIFu: Depth-Guided Implicit Function for Clothed Human Reconstruction, CVPR 2023.\n\nIn L-269, a period is missing.", "was_revised": true, "final_justification": "Thanks to the author for providing a detailed answer to my question in the rebuttal. I have also read the opinions of other reviewers. I still think that the main contribution of this article is not solid enough, and there may be unfairness in the comparison of SOTA methods. Therefore, I will maintain my score.", "rebuttal": "We sincerely thank the reviewer for your valuable comments. We feel encouraged by your positive comments like \"improved performance\". Hope the following responses address all your concerns.\n\n### Q1: What is the role of DINOv2 in MVGD?\nDINOv2 serves as a feature extractor for multiview images to inject multiview image features in the decoding process, which enhances the fidelity of details and structure of the 3D generation. We have conducted an ablation study by removing the multiview injection in Fig.8 and Tab.3. The \"original decoder (tuned)\"  without multiview image feature injection produces much more blurry results, while our injection module improves the fidelity and quality.\n\n### Q2: Is the ablation of the 2D-3D synchronization attention unfair with the same training data?\nYes. We fairly retrain all the baseline PSHuman and Trellis on the same training datasets in the ablation study and only conduct evaluation on the same unseen X-Humans dataset.\n\n### Q3: The proposed method is incremental. The authors combine PShuman, Trellis, and DINOv2.\nTo our knowledge, Synchuman is the first work that simultaneously generates both 2D and 3D representations for single-view 3D human reconstruction, achieving SOTA performance in terms of reconstruction precision and pose robustness. At its core, we integrate the strong prior of 3D native generative models and rich image-level details of 2D generative models, which benefit each other through effective 2D-3D attention. We further enhance the geometric details by injecting image features into the shape decoder. We believe this framework holds significant potential for advancing future 3D generation solutions, which enjoy the strengths of both well-engineered 2D generative models and emerging 3D native generative models.\n\n### Q4: Repeat and missing references\nThanks for pointing this out. DIFu improves occupancy prediction using depth priors, but it is limited in reconstructing plausible appearance. In contrast, our method leverages a pretrained 2d generative model to generate high-resolution multi-view color and normal images, achieving both detailed geometry and photorealistic appearance.  We will add this citation and discuss it in the related works."}, {"review_id": "vopEPnjrkk", "reviewer": "Reviewer_NEiA", "summary": "This paper introduces method for reconstructing photorealistic 3D full-body humans. The key contributions are:\n\n- Synchronized 2D multiview diffusion and 3D native diffusion, without SMPL priors, reconstruct high-quality clothed human mesh even under challenging human poses.\n- Their unified decoder for 3DGS and mesh captures detail and fidelity from multiview features, and structural consistency from structured latents.\n\nSyncHuman is evaluated on the CAPE and X-Humans dataset, demonstrating state-of-the-art performance in reconstructing clothed 3D human in both geometric and appearance quality.", "questions": "1. How long does SyncHuman take to reconstruct 3D human compared to baseline methods? (Weaknesses 1.)\n2. The generative diffusion models can inpaint the missing regions under human-human or human-object interactions in MultiHuman or Hi4D dataset? (Weaknesses 2.)\n3. The numerics of baseline methods in Tab. 1 are different from the ones reported in each paper. Additionally, SyncHuman is trained on larger training datasets: THuman 2.1, CustomHumans, THuman3.0, and 2K2K. The baseline methods are re-trained in same training setting or not?\n4. SyncHuman is also trained without image condition to enable CFG during inference. How about the qualitative performance of unconditional generation?", "limitations": "Computational costs might be one of the limitations.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n1. The paper is well written and easy to understand. \n2. They propose a novel framework that combines 2D multiview diffusion and 3D native diffusion for the first time.\n3. SyncHuman achieves state-of-the-art performance in both geometric and appearance quality.\n4. 2D-3D synchronization attention mechanism and MVGD are intuitive and effective for the performance, as shown in Tab. 2 and Tab. 3.\n\n### Weaknesses\n\n1. SyncHuman consists of three different diffusion: Multiview Diffusion, Structure Diffusion, and Structured Latents Diffusion. They do not report the computational costs, but the expensive computation might be one of the limitations.\n2. The qualitative results demonstrate that SyncHuman is robust to self-occlusions. However, the input images are full-body images without inter-occlusions due to human-human or human-object interactions. It might be challenging to inpaint and generate missing regions without SMPL priors.", "was_revised": true, "final_justification": "The authors have addressed my concerns. I keep my original positive rating.", "rebuttal": "We sincerely appreciate your efforts in reviewing our paper and providing insightful suggestions to help us improve our paper. We feel quite encouraged with your positive comments like \"a novel framework\", \"state-of-the-art performance\", and \"effective designs\". Hope the following responses address all your concerns.\n\n### Q1: Inference time.\nOn a single H800, the inference time is as follows: ours 38.57s vs Trellis  15.68s vs PSHuman 52.98s\nOur method is faster than PSHuman as it directly decodes a 3D shape without requiring additional differentiable rendering optimization. The slower speed compared to Trellis is due to our use of the 2D multi-view generation. \n\n### Q2: Can the proposed method handle occlusions in human-human interaction (HHI) or human-object interaction (HOI)?\nWe agree that extending our framework to HHI or HOI reconstruction is an interesting and important direction. However, our model is only trained on single-person data without HOI or HHI occlusions, so the model cannot handle such occlusions currently. This could be partially solved by introducing such occlusions during training. Meanwhile, extending to HHI or HOI reconstruction requires the model not only to guess unseen parts but also to ensure the guessed parts are compatible with the interacted subjects, which is out of the scope of our method. We think this could be an interesting future work, and thank you for pointing out such a promising future topic.\n\n### Q3: Explain more about the experimental setting.\nFor all baseline methods, we adopt their official model and codes for evaluation. We do not retrain all the baseline methods because retraining all baselines is too costly and unaffordable for us. Instead, we retrain two key baseline methods, PSHuman and Trellis, on the same training data as our method in experiments to demonstrate the effectiveness of our 2D-3D synchronization. \nThe performances are different from their papers because the single-view reconstruction has a scale ambiguity. We adopt the following scale normalization strategy for evaluation, which may differ from previous methods. First, we align the centers of the reconstructed mesh and the ground truth mesh, then scale them so that the coordinate range of the longest axis is 1. All evaluations in our paper were obtained by re-running the inference using the open-source codes from the relevant papers, applying a unified evaluation method. We will open-source the related evaluation code to ensure the results are reproducible.\n\n### Q4: Can our model do the unconditional generation?\nYes. Following your suggestions, we have tested our model on the unconditional generation task. We find that our method produces different textured human meshes from different seeds with plausible textures and structures. However, we find that the generation quality is worse than the conditional generation from a single-view image. The reason may be that the conditional input images provide a precise denoising direction for the model, while the unconditional generation does not have a clear denoising direction, and thus the generations have worse structures and textures. We also tried this on the pretrained PSHuman model and the Trellis model. We observe similar quality degeneration from conditional generation to unconditional generation."}]

Author rebuttal:

Meta-review: The paper tackles the problem of 3D human generation from a single image. SyncHuman relies on a combination of multi-view and 3D diffusion; it introduces a 2D-3D synchronized attention mechanism to make the two branches informed of each other, and a multiview feature injection, which propagates multi-view features in the 3D voxels, to enhance the details.

At the beginning of the discussion phase, the opinions on the paper were mixed. On the bright side, there have been experimental results and comparisons with the state of the art, but there were also concerns about the methodological contribution, doubts in some comparisons with the values reported in the original paper, and unclear aspects in the experimental setting. The discussion has been extensive, but positions remained mostly unchanged, with a little upgrade of a rejection to borderline (dgjc). A reviewer raised concerns on the ethical side, and two ethical reviewers engaged, assessing that such weaknesses can be fairly easily addressed in the final version of the manuscript.

After the discussion, the score situation remained uncertain, with three reviewers (NEiA, Pinc, and 6GgC) leaning toward acceptance, while two reviewers (dgjc and Kbwv) argued for rejection. The remaining concerns are:

1) Significance of the contribution and novelty (dgjc, Kbwv): Reviewers believe the work follows a road already paved by previous works, which combine multi-view generations and 3D. The Authors see in the idea of synchronizing the 2D and 3D generation the key novelty of their approach. While previous works in general generate the two in sequences (e.g., obtaining the 3D as reconstruction from the multi-views, or updating the multi-views using a 3D reconstruction), they propose to learn them jointly.

2) Motivation behind improvements of the contribution (dgjc): The Reviewer believes that further motivation behind the improvements should be provided. The Authors argue that recent methods lack a clear alignment between the images and the 3D, while the proposed approach proposes to align the 2D multi-views to the volume and orthogonally project the feature to obtain a 3D representation that is actually aligned with the views.

3) fairness w.r.t. SoTA comparison (dgjc, Kbwv): The Reviewers have doubts about fairness in baseline training, and comparison with other methods, especially due to reported qualitative results. Authors argue that they train all the baselines on the same data, and results achieved by replicating SIFU and PSHuman do not use ground-truth SMPL, and numbers are hence in line with the expectation.

The AC recognizes that the paper does not introduce any dramatic paradigm change. However, the proposed approach explores a simple yet effective idea that could be of inspiration beyond the human case (e.g., 3D/4D generation of generic objects). The AC finds the reason behind error accumulation in sequential steps reasonable. Also, implementing such an idea required introducing components to connect 2D and 3D generation, which could be useful for future work. Finally, the paper shows that pose estimation of SMPL might be inaccurate; although further details would be useful to fully assess the method quality (e.g., reporting also the case of baselines using GT SMPL, as an "unfair" upper-bound of their performance; reporting clearly if all the baselines rely on the same pose estimators and which they are), the provided evidence is convincing in demonstrating SyncHuman superiority. 

Due to these reasons, the AC leans toward acceptance, while encouraging the authors to:
1) Better state the motivation and the intuition behind the work
2) Clarify all the experimental and ablation details that have been pointed out by the reviewers
3) Take into serious consideration the ethical concerns, and integrate a sincere discussion in the paper.
Decision: Accept (poster)