GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Data
Authors: Wentao Wang
Hang Ye
Fangzhou Hong
Xue Yang
Jianfu Zhang
Yizhou Wang
Ziwei Liu
Liang Pan
Venue: NeurIPS 2025 poster ()
Keywords: 3D Human Reconstruction;  Multi-source Human Dataset; Single Image-3D Reconstruction
Primary area: applications

Abstract: Given a single in-the-wild human photo, it remains a challenging task to reconstruct a high-fidelity 3D human model. Existing methods face difficulties including a) the varying body proportions captured by in-the-wild human images; b) diverse personal belongings within the shot; and c) ambiguities in human postures and inconsistency in human textures. In addition, the scarcity of high-quality human data intensifies the challenge. To address these problems, we propose a Generalizable image-to-3D huMAN reconstruction framework, dubbed GeneMAN, building upon a comprehensive multi-source collection of high-quality human data, including 3D scans, multi-view videos, single photos, and our generated synthetic human data. GeneMAN encompasses three key modules. 1) Without relying on parametric human models (e.g., SMPL), GeneMAN first trains a human-specific text-to-image diffusion model and a view-conditioned diffusion model, serving as GeneMAN 2D human prior and 3D human prior for reconstruction, respectively. 2) With the help of the pretrained human prior models, the Geometry Initialization-&-Sculpting pipeline is leveraged to recover high-quality 3D human geometry given a single image. 3) To achieve high-fidelity 3D human textures, GeneMAN employs the Multi-Space Texture Refinement pipeline, consecutively refining textures in the latent and the pixel spaces. Extensive experimental results demonstrate that GeneMAN could generate high-quality 3D human models from a single image input, outperforming prior state-of-the-art methods. Notably, GeneMAN could reveal much better generalizability in dealing with in-the-wild images, often yielding high-quality 3D human models in natural poses with common items, regardless of the body proportions in the input images.

GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Data
Wentao Wang', Hang Ye Fangzhou Hong', Xue Yang", Jianfu Zhang*,
Yizhou Wang", Ziwei Liu', Liang Pan
Shannhaai AI Laboratory "Peking University
Nanyang Technological University "SAIS & SCS, Shanghai Jiao Tong University
Head Shot
(b) Human with Objects
Half-body body Shot
holl-bodySo
(c) Human Reconstruction with Natural
(a) Human with Varying Body Proportions
Poses and Textures
Figure 1: GeneMAN is generalizable framework for single-view-to-3D human reconstruction, built on a collection of multi-source human data. Given a single in-the-wild image of a person, GeneMAN could reconstruct high-quality 3D human model, regardless of its clothing, pose, or body proportions (e.g., s bolyy half-body, close-up shot) in the given image. The anonymous project page of GeneMAN is: https://roooooz.gitbbbbb Matiio/o
Abstract
Given a single in-the-wild human photo, it remains a challenging task to reconstruct a high-fidelity 3D human model. Existing methods face difficulties including a) the varying body proportions captured by in-the-wild human images; b) diverse personal belongings within the shot; and c) ambiguities in human postures and in- consistency in human textures. In addition, the scarcity of high-quality human data intensifies the challenge. To address these problems, we propose a Generalizable image-to-3D huMAN reconstruction framework, dubbed GeneMAN, building upon a comprehensive multi-source collection of high-quality human data, includ- ing 3D scans, multi-view videos, single photos, and our generated synthetic human data. GeneMAN encompasses three key modules. 1) Without relying on parametric human models (e.g. SMPL), GeneMAN first trains human- -seenafc- rext-to-image diffusion model and a view-conditioned diffusion model, serving as GeneMAN 2D human prior and 3D human prior for reconstruction, respectively. With the help of the pretrained human prior models, the Geometry Initialization-S -ee--&SSuullpl C-Suulpting pipeline leveraged to recover high-quality 3D human geometry given single image. 3) To achieve high-fidelity 3D human textures, GeneMAN employs the
'Equal contributions.
Corresponningg author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Multi-Space Texture Refinement pipeline, consecutively refining textures in the latent and the pixel spaces. Extensive experimental results demonstrate that Gene- MAN could generate high-quality 3D human models from single image input, outperforming prior state-of-the-art methods. Notably, GeneMAN could reveal much better generalizability in dealing with in-the-wild images, often yielding high-quality 3D human models in natural poses with common items, regardless the body proportions in the input images.
Introduction
Creating high-quality 3D human models is crucial in various real applications, including VR/AR, telepresence, digital human interfaces, film, and 3D game production. Traditional methods [5] 26 65 usually utilize a dense camera array to capture synchronized posed multi-view images for human reconstruction, which typically involves complicated and time-consuming processes. Towards efficient 3D human reconstruction, many approaches [49] 58 [50 19 64] 76,, [4] delve into the
challenge of reconstructing 3D human models from single image, which, however, remains an ill-posed problem due to the absence of comprehensive 3D human observation data.
To facilitate 3D human reconstruction, previous methods 778 68 67
76 14] often employ parametric human models, e.g., SMPL [32] or SMPL-X [40] as 3D human geometry prior. Nev- ertheless, these parametric models fail to capture 3D clothing details, which impedes the accurate reconstruction of 3D human figures, particularly when individuals are adorned in loose garments. Recently, image-to-30 human reconstruction methods [14 19 76 integrate pretrained text-to-image diffusion models [46] 47 to incorporate 2D priors into the reconstruction process. Despite advances in generalizability, single-view 3D human reconstruction has not been fully resolved, especially when dealing with in-the-wild images. The primary challenges are shown in Fig. [ (a) Varying Body Proportions: many portraits are captured with varying body proportions, such full-body, half-body, and headshots, while existing methods primarily focus on full-body reconstructions; (b) Human with Objects: in everyday photography, it is common capture people holding objects, standing on items, or wearing various accessories, which could greatly impact the reconstruction quality. (c) Human Reconstruction with Natural Pose and Textures: due to the absence of broadly applicable human- specific geometry and texture model, existing methods struggle to reconstruct credible geometry and consistent texture from real-world images. Additionally, the scarcity of high-quality human body data exacerbates the difficulty of tackling the problem.
In this paper, we propose MeneMAN, Generalizable image-to-3D huMAN reconstruction frame- work for high-fidelity 3D human reconstruction from a single image. To enhance generalizability, we first collect comprehensive, multi-source training dataset of high-quality, multi-modal human data, including 3D scans, multi-view videos, single images, and our augmented human data. Based on the multi-source human data collection, human-speeifee prior models, including a text-to-image diffusion model as the 2D prior and a view-conditioned diffuusion model as the 3D prior, have been trained, which could provide more generalizable human priors compared to traditional human parametric models. Leveraging the pretrained human prior models, GeneMAN generates 3D human models through two primary stages. 1) 3D Geometry Revonstruction. To create precise and intricate 3D human geometry. GeneMAN employs an Initialization-&& SSullptin strategy, which initially predicts a coarse human geometry using NeRF (38), followed by sculpting process for adding geometry details. 2) 3D Texture Generation. Utilizing the refined human geometry, GeneMAN implements a multi-space texture refinement pipeline to produce high-fidelity, consistent 3D textures. Initially, coarse textures are created through multi-view texturing and iteratively refined in the latent space. Subsequently, detailed 3D textures are achieved via pixel space texture refinement, optimizing the UV maps with 2D baser- ControlNet (72). Extensive experiments demonstrate that GeneMAN surpasses existing state-of-the-an (SoTA) methods, showcasing strong generalization capability and high generation quality. We would like to highlight that GeneMAN could faithfully reconstruct 3D human models with diverse clothing, complex poses, and different personal belongings, given a single in-the-will image with varying body proportions. Our contributions are summarized as follows:
We introduce GeneMAN, generalizable 3D human reconstruction framework built on human- specific prior models trained on collected multi-source data. Our framework enables high-quality 3D human reconstruction regardless of its clothing, pose, or body proportions.

---

A few effective 3D human reconstruction modules, such as Geometry Initialization & Sculpting. and Multi-Space Texture Refinement, have been proposed, facilitating template-free 3D human geometry modeling and view-consistency texturing.
According to experimental results, GeneMAN outperforms previous SoTA methods in single-view 3D human reconstruction, which could also effectively reconstruct high-quality 3D humans for real-world human images.
2
Related Work
Text-to-3D Generation The rapid advancements in 2D image generation have also greatly accelcrated
progress in text-to-3D generation. DreamFusion [43 and Magic3D 277 have demonstrated that utilizing pretrained 2D text-to-image diffusion models 48 46
guidance can greatly enhance the optimization 3D representations through Score Distillation Sampling (SDS). Trained on large-scale 2D datasets, these text-to-ing text-to-image models possess excellent capabilities in providing comprehensive 2D knowledge and hallucinating unseen scenes based on specific prompts. Nevertheless, as these models are limited to 2D knowledge, the aforementioned text-to-3D methods are susceptible to multi-view consistency issues. MVDream (55) improves 3D consistency by training a multi-view diffusion model that offers a robust multi-view prior. However, these methods are not well-suited for direct image-to-3D reconstruction tasks because images cannot be accurately captured through textual descriptions, leading to inconsistencies in color and texture across the generated assets.
Singe Image-to-3D Reconstruction Single image-to-33 reconstruction has greatly benefited from the thriving advancement of diffusion models. Zero-1-to-3 3 [30] trains view-dependent diffusion model on Objaverse [8] by explicitly incorporating the camera parameters into the diffusion process. It enables zero-shot image-conee conge-codd novel view synthesis and facilitates general image-to-3D
reconstruction with consistent geometric view. M2gic123 44 and DreamCraft3D (6] integrate
the 3D prior from Zero-1-to-3 (30] with 2D prior from text-to-image diffusion model, employing
coarse-to-fine two-stage optimization achieve high-quality 3D reconstruction. LRM [17 leverages
both synthetic data from Objaverse [8] and multi-view data from MVImageNet
7 to train
transformer-basee 3D architecture, enhancing its generalizability for 3D reconstruction. However, these general image-to-3D approaches often yield poor results in human reconstruction, resulting in inaccurate geometry of human bodies and the neglect of intricate details, such as facial features and clothing. This limitation arises from the lack of human-speciin priors.
Template-based 3D Human Reconstruction. Template- based approaches rely on human parametric 20
models 32 41 69 to guide 3D reconstruction 778 68 67
12
74
76 [4]. PaMIR [78 and
ICON leverage SMPI-derived features facilitate implicit surface regression, while ECON 67
employs explicit representations, predicting normals and depths to generate 2.51) surfaces. GTA
77
utilizes transformers with learnable embeddings to translate image features into 3D triplane features. Several methods [19 [4 76) exploit the generative capabilities of pretrained diffusion models (46) for 3D human reconstruction. SiTH 14 finetunes an image- image-conditioned diffusion model to hallucinate back views, while SIFU (76) and TeCH [19] utilize diffusion based geometry and texture refinement to improve reconstruction quality. Recently, IDOL [79] predicts Gaussian based representation [22] of 3D human enable fast reconstruction. However, template-ber based methods are highly reliant on precise human parametric models, which are frequently unattainable for in-the wild human images.
Template-Free 3D Human Reconstruction. Without parameterized human models, PIFu (49
introduces novel approach by extracting pixel-aligned features build neural fields. PIFuHD (50 improves this with high-resolution normal guidance, while PHOHRUM [4] jointly predicts 3D
geometry, surface albedo, and shading. Recently, HumanSGD 3 leverages pretrained 2D diffusion
models [46] as a human appearance prior. HumanL [44 trains ceffussiongguide ferd-forward model to predict the implicit field of a human without template priors. Nonetheless, they still encounter challenges, such as unrealistic and inconsistent textures, and inferior geometric details, largely due to insufficient priors and overly simplistic model designs.
3 Preliminaries
Diffusion Model. Diffusion models 15 57 59 learn to transform samples from a tractable noise distribution towards data distribution. They comprise forward process {qi (geooe...)) which pro-
3

---

3D Scans
Video
Single Image
Augmentation
Geametry Sealpting
Inpurt
Model
Obyaverne Date
ConpoPPeet Augmertation
Image
Cropping
Malti-Space
Model
Solti-Source Human Data Collection
Training
GeneMAN Pipeline
Figure 2: Overview of the Multi-Source Human Dataset and Our GeneMAN Pipeline. We have constructed a multi-source human dataset comprising 3D scans, videos, 2D images, and synthetic data. This dataset is utilized train human-specifi 2D and 3D prior models, which provide generalizable geometric and texture priors for our GeneMAN framework. Through geometry initialization, sculpting, and multi-space texture refinement in GeneMAN, we achieve high-fidelity 3D human body reconstruction from single in-the-wild images.
gressively adds random noise to a data point 2g (x))) and a reverse process [.1]] which gradually recovers clean data from noise. The forward process is defined by conditional Gaussian distribution: gr (21/<z0) ;= .N (a, of I), where 0te or > 0 are time-dependent coefficients. The re- verse process is defined by denoising from pi (21) (0, I) with a parameterized noise prediction network ( t) to predict the noise added to a clean data To. which is trained by minimizing
L = Emo -00---00mm))--000000....-.-----0
(u(t)lle((azoo + t,e, <|12]
where (1) is a time-dependent weighting function. After training, we approximate the real data distribution with pt qt. and thus we can generate samples from po qo-
Score Distillation Sampling, Score Distillation Sampling (SDS) is proposed to supervise synthesized novel views by distilling pretrained text-to-image diffusion model [4]] for text-to-3D generation [3]
As the diffusion model is trained on 2D datasets, the SDS loss referred as L and its gradient V SDL is formulated as:
aI 00
2D (0, I) Fr,, (T)((
<)
where (() is weighting function, (0) is the rendered image random viewpoints, I, is the noisy image from adding noise ~ (0, I) to the rendered image Io at timestep t, and y denotes the conditioned text prompt. The 3D representation parameterized by and the diffusion model
predicts the sampled noise E (Li t)
A pivotal work, Zero-1-to-3 30 finetunes variant of Stable Diffusion 46 model the Obja- verse 8 dataset, resulting in a cindicondi diffusion model é, which could synthesize novel view images at any arbitrary viewpoint given a reference image Í. Hence, á offers a strong 3D prior, based which could derive 3D-aware SDS loss L3D Its gradient V C3D is formulated as:
01 ag)
CDD (1. I) = R., (1 V 0 S
4 Method
In this section, we propose GeneMAN, generalizable image-to-3D human pipeline aimed for recon- structing high-quality 3D humans from in-the-wild images. By leveraging GeneMAN prior models, which provide human-specton reconstruction with varying body proportions, diverse poses, clothing, and personal belongings. The success of GeneMAN prior models is closely tied to the multi-source human dataset we constructed. We first introduce the construction of multi-source human dataset and GeneMAN prior models in Sec. 4.1 and Sec. S.c. respectively. Then, we illustrate the details of our GeneMAN framework, including the Geometry Initialization & Sculpting (Sec. 4.3 as well as the Multi-Space Texture
geometric and texture priors, we achieve high-fidelit 3D human
Refinement (Sec. 4.4 The overview of GeneMAN framework is shown in Fig. 2

---

4.1 Multi-Source Human Dataset
The success of LRM 17 has demonstrated that training on multi-source datasets, including the 3D synthetic dataset Objaverse [& and the video dataset MVImgNet [71), significantly enhances the generalization ability and reconstruction quality of models. However, existing 3D human reconstruction methods typically rely scarce 3D scanned human data [ or only video data [77] 7 for training, which restricts their ability to generalize to in-the-wild images, particularly human portraits with diverse poses, clothing, and various body proportions. To address this challenge, we construct a large-scale, multi-source human dataset by collecting diverse human data from 3D scans, multi-view videos, single photos, and synthetically generated human data, as illustrated in Fig. 2
Our 3D scanned human data is aggregated from the commercial dataset RenderPeople M. alongside several open-source datasets: CustomHumans [3] HuMMan [6), THuman2.0 (7)), THuman3.0 (6) and X-Humans [53]. Additionally, we enrich the dataset by integrating human-specific data filtered from Objaverse [8. For multi-view human videos, we leverage datasets such as DNA-Renderin 77].
ZJU-Mocap 42). AIST++ [24]. Neural Actor (29] and Actors-Ho -H [21]. In terms of 2D human imagery, we select data from DeepFashion [3 and LAION-SB [51] to ensure comprehensive coverage of diverse human appearances. Furthermore, we employ data augmentation strategies to synthesize additional data. Specifically, utilize ControlNet-based [72] image synthesis to generate multi-view human data with diverse clothing options through various prompts, enabling the creation of human images featuring different garments. To account for varying body proportions, we apply image cropping to preprocessed multi-view human renderings, expanding our dataset to include instances with diverse body proportions. In total, our dataset comprises over 50K multi-view instances. Further details of the dataset are provided in the supplementary materials.
4.2 GeneMAN Prior Models
highlight the complementary benefits of hybrid 2D and 3D priors for 3D reconstruction, where 2D priors provide detailed geometry and texture, and 3D priors ensure multi-view consistency. Inspired by this, we finetune a text-to-image diffusion model [46] and a view-conditioned diffusion model [30] on multi-source human dataset, serving GeneMAN 2D and 3D priors for modeling human -seanfc texture and geometry. For the GeneMAN 3D prior, finetune the pretrained Zero-1-to-3 (30) model leveraging collected 3D scans, multi-view videos, synthetic data and 2D images from DeepFashion 31. Additionally, we incorporate an extra 20% of data curated from the Objaverse [8] dataset. Training this extensive dataset enables the view-conditioned diffusion model to acquire a generalizable prior for human geometry. Notably, the base --to- model, originally trained Objaverse, demonstrates a robust capability for accurately reconstructing objects. This facilitates our finetuned model to handle humans with complex clothing and personal belongings, as discussed earlier. The fine-tuning process is conducted using AdamW [33] optimizer with learning rate of 10 on eight NVIDIA 1000 GPUs for one week. For the GeneMAN 2D prior, we finetune Stable Diffusion V1.5 [46] with our entire molti-source human dataset. An equivalent amount of images from LAION-SB [1 are included to preserve the model's 's capabilities. Finetuning is performed using Adam four NVIDIA A100 GPUs for five days.
Previous studies 27 61
3 with a learning rate of 10-5 on
4.3 Geometry Initialization & Sculpting
As illustrated in Fig. 3
we adopt hybrid representations, incorporating NeRF (8 and DMTet 54 to
reconstruct detailed human geometry from reference image. Instead of relying on SMPL 332 for initialization, we first train the NeRF network (38) using GeneMAN 2D and 3D prior models to craft template-free initialized geometry. We leverage Instant-NGP
39 our NeRF implementation
due to its fast inference and ability to recover complex geometry. To supervise the reference view
reconstruction, we maximize the similarity between the rendered image and the reference image using the following loss: Leef = I (f-9((;; ()); + I|m- 9m (0; E)1|2
(4)
where g(0; é) represents the rendered image at the reference viewpoint è, 0 denotes the NeRF parameters and @ indicates the Hadamard product. The foreground mask is denoted as m, and Im (0; é) renders the silhouette. In addition to supervising the RGB image and mask, we utilize
5

---

Geometry Initialization & Sculpting
Reference View
faf
brref
Reference View
NeRF
DMTet
Input Image
Noise
3D Prior
GeneMAN Prior
gutd
Novel View
Geometry
Sculpted Geometry
Eguid
Figure 3: Geometry Initialization & Sculpting. During the geometry reconstruction stage, we initialize a template-free geometry using NeRF [38. incorporating GeneMAN 2D and 3D priors with SDS losses. Alongside diffusion-bante based guidance, a reference loss ensures alignment with the input image. We then convert NeRF into DM'eet (54) for high-resolution refinement, guided by pretrained human-specific normal- and
diffusion models [8]
the geometry prior inferred from the reference image, specifically depth and normal information. The reference depth fa and normal f, are derived using the human foundation model Sapiens [23] To enforce consistency between the rendered and reference values, we impose depth and normal losses, Ldepth and Lnoral The normal loss is calculated the mean squared error (MSE) between
the reference normal I, and In. i.e., Laormll normalized negative Pearson correlation:
|| Inll2. For the depth loss, we employ the
cov m 2 Ia. m ④ fa)
Cdepth
am O Ia)a(m @ Id)
where cov(-) and o.)) represent the covariance and variance operators, respectively.
For novel view guidance, we leverage the GeneMAN 2D and 3D prior models, which distill human- specific priors from our multi-source human dataset for hybrid supervision. The GeneMAN 2D prior model Φb provides rich details of human geometry and texture, while the GeneMAN 3D prior model daa encodes a pluralistic 3D human prior, ensuring multi-view consistency both geometry and texture. The hybrid guidance loss Lyaid is denoted as:
Lguid = LnD g()) + LDD (Ds( (()).
(6)
To further refine the geometric details, adopt DMTet [54], hybrid SDF-Mesh representation that enables memory-efficient high-resolution 3D shape reconstruction. The trained NeRF [38 converted into mesh, which serves the geometric initialization for DMTet optimization. During optimization, we compute the reconstruction loss by applying both the MSE loss and the perceptual loss (56] between the rendered normal from the reference view and the reference normal Ín Inspired by HumanNorm [8] leverage its pretrained normal-adapted and depth-adapted diffusion models for novel view guidance to craft high-quality geometry. Furthermore, the SDF loss Lat is incorporated regularize the geometry, preventing significant deviation from the initial mesh and avoiding improper shapes. Through geometry initialization and sculpting, we achieve high-fidelity 3D human geometry with plausible poses and intricate geometric details.
4.4 Multi-Space Texture Refinement
With the detailed geometry in place, we implement multi-space texture refinement in both latent and pixel spaces, ensuring that the textures remain consistent and realistic. First, we perform latent space optimization using the hybrid SDS loss (43 30), leveraging our GeneMAN 2D and 3D prior models create an initial coarse texture. The GeneMAN 2D prior model guarantees texture realism, while the GeneMAN 3D prior model excels at maintaining view-consistent textures. We also supervise the reference view by applying MSE loss between the reference image and the rendered image ge ē)) viewpoint é. The overall loss function for latent space optimization formulated as:
Leaars Nicr (1 (o; 2) + Iim - (o: 12) + X Se
Auiia guid-

---

Multi-Space Texture Refinement
Jreef
UV Map
Reference View
Optimize
Multi-wiew Texturing
Condition
GeneMAN Prior
2D Prior-based ControlNet
SDS
GeneMAN 2D Prior
Novel View
Sculpted Geometry
Latent Space Refinement
guid
Pixel Space Refinement
Figure 4: Multi-Space Texture Refinement. In the texture generation stage, propose multi- space texture refinement to optimize texture in both latent space and pixel space. First, we generate the coarse textures using multi-view texturing, which are then iteratively refined in latent space. Subsequently, detailed textures are obtained by optimizing the UV map in pixel space with 2D prior-based ControlNet.
where X denotes the weighting parameter of the reference loss, and Liguaa represents the guidance loss in the coarse texture stage, as defined in Eg. 6
Multi- View Training. Multi-view training strategies effectively enhance consistency across views (30 55), but they typically require retraining diffusion models. In contrast, we employ a training- free approach by adding identical Gaussian noise batch of images rendered from differ- ent views. These images are then concatenated into a single image for inference, ensuring that the resulting multi-view images exhibit consistent textures and colors.
The coarse texture stage prioritizes learning consistent and plausible texture but still lacks realism. To enhance texture realism, we employ pixel-space optimization by minimizing the distance between the rendered multi-view images and their refined counterparts, thereby improving the UV texture map. Akin to DreamGaussian (62]. we adopt the image-to- -o-imaae synthesis framework of SDEdit 377 to generate the refined images. Specifically, we render the coarse image Ioonse from an arbitrary camera view c, perturb it with random Gaussian noise, and apply multi-step denoising process using the GeneMAN 2D prior based ControlNet 772 Φ2 () to obtain the refined image T Tine:
Iine = d2a coarse E(fstart; startt e),
where <(tsam)) represents random noise timestep taatt, and e denotes the conditioned text embeddings. We employ MSE loss for pixel-wise reconstruction and LPIPS [73] loss to enhance texture details. Let ALP denote the weight of the LPIPS loss. The total loss for pixel space optimization Line is:
Lime
I
Tine
Tewasel
+ AL.LPIIPS(Time T
5 Experiments
5.1 Implementation Details
[10] During the geometry stage, we progressively increase the resolution of NeRF 38 from 256 384 over 5, 000 steps. We then convert it to explicit mesh, which serves as the geometry initialization for DMTet (54) at resolution of 512. We subsequently optimize DMTet for 3, 000 steps to sculpt fine- grained geometric details. In the texture stage, we perform an initial coarse texture optimization over 10, 000 steps, followed by a refinement of the texture UV map for 1, 000 steps. The full optimization process takes approximately 1.4 hours single NVIDIA A100 80G GPU. Additional details are provided in the supplementary materials.
Training Details. Our framework is built upon the open-source project ThreeStudio
Testing Details. For both qualitative and quantitative evaluation, we randomly select 50 samples from the Internet and CAPE [35]. The samples sourced from the Internet include challenging scenarios, such as humans with varying body proportions (head shots, half-body shots, full full-body shots), as well as humans in diverse poses, clothing, and with personal belongings. We compare our model with state-of-the-art image-to-3D human reconstruction methods, including PIFu 49 GTA (75). TeCH [19] and SiTH [14. For each method, we render 120 viewpoints across 360 degrees of the reconstructed results for evaluation. We compute PSNR and LPIPS 73 between the reference view
7

---

Table 1: Quantitative Comparison with State-of-the-art Methods. Evaluation images are sourced
from in-the-wild and the CAPE dataset 335
Best results in bold, second-best are underlined.
in-the-wild
CAPE
Method
PSNR1
LPIPS
CLIP-Simt
PSNR1
LPIPS
CLIP-Simt
PIFu 49
26.966
0.035
0.594
26.912
0.028
0.764
GTA 774
25.060
0.064
0.568
30.376
0.019
0.785
TeCH 19
25.740
0.053
0.713
27.601
0.025
0.826
SiTH 14
20.412
0.129
0.608
21.992
0.048
0.815
GeneMAN (Ours)
32.238
0.013
0.730
28,490
0.015
0.838
and the rendered front view to evaluate reconstruction quality. Additionally, CLIP similarity [45] between the reference view and 119 novel views is measured to assess multi-view consistency.
5.2 Quantitative Comparison
Tab. D presents quantitative metrics comparing GeneMAN with baseline methods on in-the-wild images and the CAPE dataset [35]. For in-the-wild images, GeneMAN outperforms all competing approaches, showcasing its superiority in generalizable human reconstruction, with the highest CLIP-Similarity (45) indicating better multi-view consistency. It also leads in PSNR and LPIPS, highlighting its improved reconstruction quality. On CAPE dataset, GeneMAN continues to excel, achieving the best results LPIPS and CLIP-Similarity while maintaining competitive performance in PSNR, proving its effectiveness on laboratory dataset as well. Additional geometric results are placed in the supplementary materials.
5.3 Qualitative Comparison
We present qualitative results on testing samples from in- the-wild images and the CAPE (35] dataset, showcased in Fig. and Fig. [ respectively. To better illustrate the reconstruction quality, we visualize the multi-view surface normals and color renderings. PIFu [49] fails to recover realistic geometry and appearance, particularly in side views. Template-based approaches, including GTA (74), TeCH [19] and SiTH [4 heavily rely on precise human pose and shape estimation (HPS). However, HPS meth- ods [9] often produce artifacts such as "bent legs" and struggle with the shapes of children, as seen in both Fig. 6 and Fig- 7 These limitations lead to cumulative errors in geometry reconstruction that are difficult to eliminate, even with refinement. While TeCH captures more lifelike details with optimization-bant based refinement, the reconstructed surface appears excessively noisy, and the textures are inconsistent, especially in the back views of garments. In contrast, GeneMAN demonstrates exceptional generaliz- ability to in-the-wild images featuring diverse clothing styles, poses and varying body proportions. Its template-fres design enables superior modeling personal belongings and loose clothing, such as basketball, skirts and dresses, while maintaining detailed, realistic geometry and natural body shapes. GeneMAN also produces sigh-iiielity, multi-view consistent textures for clothing and hairstyles. In present additional results to highlight the robust reconstruction capabilities of GeneMAN under complex poses. More results on complex poses are provided in the supplementary.
Figure 5: Qualitative Results of Gene- MANwith Complex Poses.
Fig. 5
5.4 User Study
We conduct a user study involving 40 participants to assess the reconstruction quality of GeneMAN compared to each baseline method across 30 test cases, evaluating both geometry and texture quality. Participants are provided with free-view rendering video for each method, and asked to select the most preferred 3D model from five randomly shuffled options. A total of 40 x 30 1200 comparisons are conducted. As shown Fig. TO, 73.08% of participants favor our method, demonstrating a significant improvement over the basclines. This indicates that our model generates more plausible and detailed
8

---

Reference
TeCH
SITH
GeneMAN (Ours)
Figure 6: Qualitative Comparison with State-of-the-art Methods on in-the-wild Images. To validate the generalizablity of each method, select diverse set of images for demonstration, including human with complex poses, children, varying body proportions and human with personal belongings. GeneMAN shows superiority over compared methods in image-to-3D human reconstruc- tion, achieving both plausible geometry and realistic, consistent texture.
Reference
PIFu
GTA
TeCH
SiTH
GeneMAN (Ours)
Figure Qualitative Comparison with State-of-the-art Methods CAPE (35]. Without accurate HPS results, template-basee based methods [74] 14 19 suffer from artifacts like the "bent-leg" effect. In contrast, GeneMAN reconstructs humans with natural poses, detailed geometry, and realistic textures.
geometry, along with realistic and consistent textures, confirming GeneMAN's effectiveness in generating high-quality reconstructions.
5.5 Ablation Study
Effectiveness of Geometry and Texture Reconstruction. Fig. S anaalyee the effectiveness of each stage proposed in our framework. Compared with geometry initialization (a), geometry sculpting in (b) smooths the excessively noisy surface and recovers figh-frequency details, such clothing wrinkles and facial features. In (c), the latent space texturing yields reasonably satisfactory result; however, the back view remains inconsistent, and the texture appears slightly blurred. To address this, (d) enhance the coarse texture into a final refined texture through pixel space texturing.
Effectiveness of GeneMAN 2D Prior Model. To assess the efficacy of GeneMAN 2D prior model, we compare the mesh results extracted from the NeRF (38 stage utilizing the pretrained DeepFloyd- IF [47] model ("original 2D prior") versus GeneMAN 2D prior model ("GeneMAN 2D prior"). Note that we select the NeRF stage output for visualization to clearly demonstrate the impact of the diffusion prior, whereas subsequent geometry and texture disentanglement may obscure this effect. As shown in Fig. (a) and (b), the red-boxed regions reveal that the original 2D prior results in an uneven shirt hem, with the back extending lower than the front. In contrast, GeneMAN 2D prior model ensures multi view consistency in both geometry and texture, enhancing overall reconstruction.

---

1.33%
3.50%
16.50%
5.58%
Geonuttyy Initialization Geometry Sculpting Space Testuring (4) Space Testuring
73.09%
Figure 8: Ablation on Geometry and Texture Reconstruction. (a) shows the geometry initialization, (b) recovers finer details through geometry sculpting, (c) applies latent-space texturing, and (d) achieves the final reconstruction with pixel-space optimization.
PIFu
GTA
TeCH
SITH
GeneMAN (Ours)
User Study. 73.09% of subjects prefer our method over the base- lines terms of geometry and texture, highlighting the significant superiority our approach in recovering ac- curate geometry and high- fidelity appearances.
Figure 10:
Original
(b)
Prior
Figure 9: Ablation on GeneMAN 2D and 3D Prior Models. (a) and (b) demonstrate the 2D prior model improves multi-view sistency and texture details. (c) and (d) illustrate the enhanced pose naturalness and geometric accuracy from the 3D prior model.
Effectiveness of GeneMAN 3D Prior Model. To assess how GeneMAN 3D prior model ("Gene-
MAN 3D prior") make influence reconstruction results, we conduct comparison by substituting it with the original Zero-I-toe 1-to-3 (30] model ("original 3D prior") in our framework. As indicated in Fig. 9 (c) reveals unnatural human poses, with the neck and head leaning forward, misaligned with the input image. In contrast, our GeneMAN 3D prior model, finetuned on a large-scale multi-source human dataset, captures more natural pose and shape prior, as shown (d). This comparison high- lights that our GeneMAN 3D prior offers more reliable and coherent guidance, leading to improved geometric accuracy and more realistic human structure.
6 Conclusion
In this paper, we present GeneMAN, generalizable framework for single-image 3D human recon- struction. By leveraging the GeneMAN 2D and 3D prior models, trained on a large-scale, multi-source human dataset, GeneMAN is capable of reconstructing high-fideliyy 3D human models from in-the- wild images, accommodating varying body proportions, diverse clothing, personal belongings, and complex poses. Extensive experiments validate the effectiveness our approach, demonstrating its superiority over state-of-the-art methods.
Acknowledgments
This work was partly supported by National Natural Science Foundation of China (62506229, 62302295), and Natural Science Foundation Shanghai (252R1402266 2021SH2DZZ0102), We sincerely thank Wayne Wu, Shikai Li, and Honglin He for their insightful suggestions and feedback on manuscript.
References
[1] Renderpeople. https://renderpeopple 2018.
[2] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Tlge Akkaya, Florencia Leoni Aleman, Diogo Almeida. Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303.0877 2023.
[3] Badour AlBahar, Shunsuke Saito, Hung-Yt Yu Tseng, Changil Kim. Johannes Kopf, and Jia-Bin Huang. Single-image 3d human digitization with shape-guided diffusion. In SIGGRAPH Asia 2023 Conference Papers, pages 1-11, 2023. 2023.
10

---

[4] Thiemo Alldieck, Mihai Zanfir, and Cristian Sminchisescu. Photorealistic monocular 3d reconstruction of humans wearing clothing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1506-1515, 2022.
[5] Alexandru O Balan, Leonid Sigal, Michael J Black, James E. Davis, and Horst W Haussecker. Detailed human shape and pose from images. 2007 JEEE Conference on Computer Vision and Pattern Recognition. pages 1-8, IEEE, 2007.
[6] Zhongang Cai, Daxuan Ren, Ailing Zeng. Zhengyu Lin, Tao Yu, Wenjia Wang, Xiangyu Fan, Yang Gao. Yifan Yu, Liang Pan, et al. Humman: Multi-modal 4d human dataset for versatile sensing and modeling. In European Conference Computer Vision, pages 557-577. Springer, 2022.
[7] Wei Cheng, Ruixiang Chen, Siming Fan, Wanqi Yin, Keyu Chen, Zhongang Cai, Jingho Wang, Yang Gao, Zhengming Yu, Zhengyu Lin, al. Dna-rendering: A diverse neural actor repository for high-fidelity human-centric rendering. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 19982-19993, 2023.
[8] Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsami, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe annotated 3d objects. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 13142-
13153, 2023.
[9] Yao Feng, Vasileios Choutas, Timo Bolkart, Dimitrios Tzionas, and Michael J Black. Collaborative regression of expressive bodies using moderation. In 2021 International Conference on 3D Vision (3DV), pages 792-804. IEEE, 2021.
[10] Yuan-Chen Guo, Ying-Tian Tian Liu, Ruizhi Shao, Christian Laforte, Vikram Voleti, Guan Luo, Chia-Hao Chen, Zi-Xin Zou, Chen Wang, Yan-Pei Cao, and Song- Hai Zhang. threestudio: A unified framework for 3d content generation. httts:///ithan
2023.
[11] Sang-Hun Hun Han, Min-Gyu Park, Hong Yoon, Ju-Mi Kang, Young-Jae Park, and Hae-Gon Jeon. High- fidelity 3d human digitization from single 2k resolution images. In Proceedings of the IFEF/CVF Conference on Computer Vision and Pattern Recognition, pages 12869-12879, 2023.
[12] Tong He, Yuanlu Xu, Shunsuke Saito, Stefano Soatto, and Tony Tung. Arch++: Animation-ready clothed human reconstruction revisited. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11046-11000
2021.
[13] Hsuan-I Ho. Lixin Xue. Jie Song, and Otmar Hilliges. Learning locally editable virtual humans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21024- 21035, 2023.
[14] Ho. Jie Song, Otmar Hilliges, al. Sith: Single-view textured human reconstruction with image- conditioned diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 538-549, 2024.
[15] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:684006851, 2020.
[16] Fangzhou Hong, Jiaxiang Tang, Ziang Cao, Min Shi, Tong Wu, Zhaoxi Chen, Tengfei Wang, Liang Pan, Dahua Lin, and Ziwei Liu. 3dtopia: Large text-to-3d generation model with hybrid diffusion priors. arXiv preprint arX:v:2403.00000 02234, 2024.
[17] Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. L.m: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.0444 1.0400,, 2023.
[18] Xin Huang, Ruizhi Shao, Qi Zhang, Hongwen Zhang, Ying Feng. Yebin Liu, and Qing Wang. Humannorm: Learning normal diffusion model for high-quality and realistic 3d human generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4568-4577, 2024.
[19] Yangyi Huang, Hongwei Yi. Yuliang Xiu. Tingting Liao, Jiaxiang Tang. Deng Cai, and Justus Thies, Tech: Text-guided reconstruction of lifelike clothed humans. In 2024 International Conference on 3D Vision (3DV), pages 1531-1548 IEEE, 2024.
[20] Zeng Huang. Yuanlu Xu, Christoph Lassner, Hao Li, and Tony Tung. Arch: Animatable reconstruction of clothed humans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 2020.

---

[21] Mustafa Işık, Martin Rünz, Markos Georgopoulos, Taras Khakhulin, Jonathan Starck, Lourdes Agapito, and Matthias Nießner. Humanrf: high-fidelity neural radiance fields for humans in motion. ACM Transactions on Graphics (TOG), 42(4):1-12, 2023.
[22] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139-1 2023.
[23] Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stu- art Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision models. arXiv preprint arXiv:2408. 12569, 2024.
[24] Ruilong Li, Shan Yang. David A Ross, and Angjoo Kanazawa. Ai choreographer: Music conditioned 3d dance generation with aist++. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13401 13412, 2021.
[25] Shikai Li, Jianglin Fu, Kaiyuan Liu, Wentao Wang, Kwan-Yee Lin, and Wayne Wu. Cosmicman: A text-to-image foundation model for humans. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 6955-6965, 2024.
[26] Junbang Liang and Ming C Lin. Shape-aware human pose and shape reconstruction using multi-view images. In Proceedings of the IEFF/CVF international conference on computer vision, pages 4352-4362, 2019.
[27] Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng. Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Y Yu Liu, and Tsung- Yi Lin. Magic3d: High-resolution text-to-3d 3d content creation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 300-309, 2023.
[28] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
[29] Lingjie Liu, Mare Habermann, Viktor Rudnev, Kripasindhu Sarkar, Jiatao Gu, and Christian Thcobalt. Neural actor: Neural free-view synthesis of human actors with pose control. ACM transactions graphics (TOG), 40(6):1-16, 2021.
[30] Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey 7akharov, and Carl Vondrick. Zero- 1-to-3: Zero-shot one image to 3d object. Proceedings of the IEEF/CVE international conference on computer vision, pages 9298-9309. 2023.
[31] Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, and Xiaoou Tang. Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1096-1 104, 2016.
[32] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael Black. Smpl: skinned multi-person linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 851-866. 2023.
[33] Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.051001 2017.
[34] Tiange Luo, Chris Rockwell, Honglak Lee, and Justin Johnson. Scalable 3d captioning with pretrained models. Advances in Neural Information Processing Systems, 36, 2024.
[35] Qianli Ma. Jinlong Yang. Anurag Ranjan, Sergi Pujades. Gerard Pons-Moll, Siyu Tang, and Michael J Black. Learning to dress 3d people in generative elothing. In Proceedings of the IÉEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6469-644 64788, 2020.
[36] Yifang Men, Biwen Lei, Yuan Yao, Miaomiao Cui, Zhouhui Lian, and Xuansong Xie. En3d: An enhanced generative model for sculpting 3d humans from 2d synthetic data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9981-9991, 2024.
[37] Chenlin Meng. Yutong He, Yang Song. Jiaming Song. Jiajun Wu, Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073 2021.
[38] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1):99 106, 2021.
12

---

[39] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with multiresolution hash encoding, ACM trunsactions on graphics (TOG), 41(4):1-15, 2022.
[40] Georgios Paylakos, Vasileios Choutas, Nima Ghorhani, Timo Bolkart, Ahmed AA Osman, Dimitrios Tzionas, and Michael J Black. Expressive body capture: 3d hands, face, and body from single image. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pages 10975-10985, 2019.
[41] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed AA Osman, Dimitrios Tzionas, and Michael Black. Expressive body capture: hands, face, and body from single image. In Proceedings of the IEFF//VV conference on computer vision and pattern recognition, pages 10975-10985, 2019.
[42] Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang. Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dy- namic humans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9054-9063, 2021.
[43] Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall, Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint ar Xiv: 14988, 2022.
[44] Guocheng Qian, Jinjie Mai, Abdullah Hamdi, Jian Ren, Aliaksandr Siarohin, Bing Li, Hsin- Hsin-Ying Lee, Ivan Skorokhodov, Peter Wonka, Sergey Tulyakov, et al. Magic123; One image to high-quality 3d object generation using both 2d and 3d diffusion priors, arXiv preprint arXiv:2306. 17843, 2023.
[45] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry. Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
[46] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[47] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, al. Photorealistic text-to- image diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479 364999, 2022.
[48] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to- image diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479-36494, 2022.
[49] Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2304-2314 2019,
[50] Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization. In Proceedings of the IEEE/CVF conference on computer vision and pallem recognition. pages 84-93, 2020.
[51] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-Sh: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 11-25278-252944 2022.
[52]
Nikita Selin. CatveKit. gethbbbom/OOHHoortr/////oeoeeeee bacp:rangoe-baoeguuuuuuueereeeee romoooooooooroooootoooooooooooo 2023.
[53] Kaiyue Shen, Chen Guo, Manuel Kaufmann, Juan Jose Zarate. Julien Valentin, Jie Song, and Otmar Hilliges. X-avatar: Expressive human avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16911 16911-16921, 2023.
[54] Tianchang Shen, Jun Gao, Kangxue Yin, Ming- Yu Liu, and Sanja Fidler, Deep marching tetrahcdra: a hybrid representation for high-resolution 3d shape synthesis. Advances in Neural Information Processing Systems, 34:6087-6101, 2021.
[55] Yichun Shi, Peng Wang, Jianglong Ye. Mai Long, Kejie Li, and Xiao Yang. Mydream: Multi-view diffusion for 3d generation, arXiv preprint arXiv: 16512, 2023.
13

---

[56] Karen Simonyan. Very deep convolutional networks for large-scale image recognition. arXiv preprint wiXiv:140..15555 2014.
[57] Jascha Sohl-Dickste Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256-2265. PMI.R, 2015.
[58] Dae-Young Young Song, HeeKyung Lee, Jeongil Seo, and Donghyeon Cho. Difu: Depth-guided implicit function for clothed human reconstruction. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 8738 8747, 2023.
[59] Yang Song, Jascha Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint @xiv:2011..3456 2020,
Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben
[60] Zhaoqi Su, Tao Yu, Yangang Wang, and Yebin Liu. Deepeloth: Neural garment representation for shape and style editing. IEEE Transactions on Puttern Analysis and Machine Intelligence. 45(2):1581-1593, 2022.
[61] Jingxiang Sun, Bo Zhang, Ruizhi Shao, Lizhen Wang, Wen Liu, Zhenda Xie, and Yebin Liu. Dreameraftod: Hierarchical 3d generation with bootstrapped diffusion prior, arXiv preprint arXiv: 10,1618, 2023.
[62] Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. arXiv preprint arXiv:2309.1665 2023.
[63] Chien-Yat Yao Wang. Alexey Bochkovskiy, and Hong- Yuan Mark Liao. Yolov7: Trainable bag-of-freebies sets new state-of ofte-oof for real-time object detectors. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7464,7475, 2023.
freebies
[64] Zhenzhen Weng, Jingyuan Liu, Hao Tan, Zhan Xu, Yang Zhou, Serena Yeung-Levy, and Jimei Yang. Single-view 3d human digitalization with large reconstruction models, arXiv preprint arXiv:2401.12177 2024.
[65] Minye Wu, Yuehao Wang. Qiang Hu, and Jingyi Yu. Multi view neural human rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2020.
[66] Zhangyang Xiong, Chenghong Li, Kenkun Liu, Hongjic Liao, Jianqiao Hu, Junyi Zhu, Shuliang Ning, Lingteng Qiu, Chongjic Wang. Shijie Wang, et al. Mvhumannct: A large-scale dataset of multi-vien view daily dressing human captures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19801-19811, 1, 2024.
[67] Yuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas, and Michael Black. Econ: Explicit clothed humans optimized via normal integration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 312-523, 2023.
[68] Yuliang Xiu, Jinlong Yang, Dimitrios Tzionas, and Michael J Black. Tcon: obtained from normals. In 2022 TEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13286-1 1 13296. IEEE, 2022.
Implicit clothed humans
[69] Hongyi Xu, Eduard Gabriel Bazavan, Andrei Zanfir, William T Freeman, Rahul Sukthankar, and Cristian Sminchisescu. Ghum & ghuml: Generative 3d human shape and articulated pose models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6184 6193, 2020.
[70] Tao Yu, Zerong Zheng, Kaiwen Guo, Pengpeng Liu, Qionghai Dai, and Yebin Liu. Function4d: Real-time human volumetric capture from very sparse consumer rghd sensors. In innPree Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5746-5756, 2021.
[71] Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Chenming Zhu. Zhangyang Xiong. Tianyou Liang, et al. Mvimgnet: large-scale dataset of multi-view images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9150-9161, 2023.
[72] Lymin Zhang, Anyi Rao, and Mancesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836-3847, 2023.
[73] Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
14

---

[74] Zechuan Zhang, Li Sun. Zongxin Yang. Ling Chen, and Yi Yang. Global-correlated 3d-decoupling transformer for clothed avatar reconstruction. Advances in Neural Information Processing Systems, 36, 2024.
[75] Zechuan Zhang, Li Sun, Zongxin Yang. Ling Chen, and Yi Yang. Global-corelated 3d-decoupling transformer for clothed avatar reconstruction. Advances in Neural Information Processing Systems. 36, 2024.
[76] Zechuan Zhang, Zongxin Yang. and Yi Yang. Sifu: Side-view conditioned implicit function for real-workd usable clothed human reconstruction. In Proceedings of the IEEE/CVE Conference on Computer Vision and Patter Recognition, pages 9936 9947, 2024.
[77] Xiaoyun Zheng, Liwei Liao, Xufeng Li, Jianbo Jiao, Rongjee Wang. Feng Gao, Shiqi Wang. and Ronggang Wang. Pku-dymvhumans: A multi-view video benchmark for high-fidelity dynamic human modeling. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22530- 22540, 2024.
[78] Zerong Zheng, Tao Yu, Yehin Liu, and Qionghai Dai. Pamir: Parametric model-zonditioned implicit representation for image-based based human reconstruction. IEEE transactions pattern analysis and machine intelligence, 44(6):3 (000:3370-388- 2021.
[79] Yiyu Zhuang, Jiaxi Lv, Hao Wen, Qing Shuai, Ailing Zeng Hao Zhu, Shifeng Chen, Yujiu Yang, Xun Cao, and Wei Liu. Idol: Instant photorealistic 3d human creation from a single image. arXiv preprint arXiv:2412. 14963, 2024.
15

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
The main claims made in the abstract and introduction can accurately reflect the paper's contributions and scope.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
16

---

The paper discusses the limitations of the work in Section E ("Limitations") of the supplementary materials.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper focuses on applications rather than presenting new theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides all necessary details regarding the training pipeline and dataset settings, which are essential for reproducing the main experimental results of the paper that support its key claims.
17

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The paper utilizes a combination of commercial and open-source data for training. The open-source data is publicly accessible, whereas the commercial data requires acquisition through purchase from relevant websites. The code will be open-sourced after the review process.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c ec/ boblic/guidess http:///uuiiee//eeuuubiiiiseee
onnooocyPyyy) for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips cc/ wwwpcc/pplcc....c httpal///guudess//eeeeubliiieeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
18

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper presents the training and test details in the Section 5.1 "Implementa- tion Details" in the main text and Section "Implementation Details" in the supplementary materials.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to a level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: The paper follows similar setups in the community to conduct the experiments, but does not include errors bars or other statistical significance measures the experimental results.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
19

---

Answer: [Yes]
Justification: The paper provides sufficient information on the computer resources in Section 5.1 "Implementation Details" in the main text.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e-g-. preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper adheres to all stipulations presented in the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses both potential positive and negative societal impacts in Section F "Broader Impacts and Safeguards" of the supplementary materials.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
20

---

11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: The paper discusses safeguards in Section F "Broader Impacts and Safeguards" the supplementary materials.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The paper properly cites all external resources used in our experiments, including datasets and codebases. The corresponding licenses and terms of use are also clearly stated in the paper.
Justification:
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com/ datasete has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper provides appropriate documentation for the new assets introduced in the main text and supplementary materials.
Guidelines:
The answer NA means that the paper does not release new assets.
21

---

Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
The paper only uses the LLM for writing, editing, or formatting purposes.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https : //tp:://n.uu...p htt////rppscc//oofeeeeeeeeeeeeee cccc/ooffeeencce/22255555 for what should or should not be described.
22

---

This appendix serves to further enrich the discourse established in our main paper. Sec. A presents the implementation details of GeneMAN. In Sec. B we elaborate on the details of our multi- human dataset. Sec. provides supplementary experimental results, including more visualizations for the qualitative comparison, additional results of GeneMAN on in-the-wild images and CAPE [55) additional quantitative comparisons across different body proportions, discussion on geometric quantitative evaluation, and ablation studies conducted on our multi-source human dataset. In Sec. [ we discuss the limitations of our approach, as well as the broader impacts and safeguards.
and D
A
Implementation Details
A.1 Camera Sampling
For novel view guidance, we sample the camera distance from the range 4(300, 3.5). The eleva- tion angle Φi is drawn from (4(-100,,45 and the azimuth angle 0 is uniformly sampled from [-180", ). Additionally, the field of view (FOV) is constrained to the range [20", ,29), aligning with the fixed camera intrinsics optimized for our finetuned human diffusion models. Building upon HumanNorm [18], we segment the human body into four regions: the head, upper body, lower body, and full body. To enhance part-aware reconstruction with high-fidelity detail, we allocate sampling probability of 0.7 to the full body, and 0.1 to each of the head, upper body, and lower body. We also manually adjust the camera distance for zooming in specific parts. For instance, when refining the facial region, the camera distance is reduced to (0.8. 1.0). For half-body refinement (either upper or lower body), the camera distance is adjusted to (1155, 2.0). Additionally, the camera center is shifted to ensure that the relevant keypoint aligns with the center of the rendered images. To achieve this, use off-the-shelf tool identify 2D keypoints, which are then used during rasterization to back-pu proj-ptoject the coordinates into 3D space. Note that if a keypoint lies outside the image boundaries, assign sampling probability of zero that point and renormalize the distribution accordingly.
A.2 Details of Each Stage
Geometry Initialization & Sculpting. In the phase of Geometry Initialization, we optimize Instant- NGP [39] from resolution of 128 to 384 over the course of 5, 000 steps. The loss weights for this stage are set as follows: X = x 10², Am = 100, A = 0.05, A = 1, Azo = 0.1, ASD = 0.1. Subsequently, we extract the resulting mesh from Instant-NGP with isosurface resolution of 256 as the geometry initialization. During the geometry sculpting stage, we refine the geometry adopting DMTer [54] at resolution of 512 for 3, 000 steps, enabling the capture of intricate details of humans. Inspired by HumanNorm 18], we incorporate a progressive positional encoding technique, where the mask on the position encoding for DMTet's SDF features is gradually lifted to introduce higher-frequency components as training progresses. After 2, 000 iterations, the mask fully reveals all positions, allowing the encoding capture both low- and high-frequency details. Empirically, we observe that omitting progressive positional encoding results in noisy surface reconstructions. The loss weights are set as follows: A, = 5 x 103, Aegs = x 10 Avdl/ 1.5 10 and Aede = 1.0. To ensure consistency between the rendered front view and the imput image. we apply relatively small guidance scale of 20 for novel view generation using the normal- and depth-adapted diffusion models [8]. The noise timestep / is sampled from (0.002, 0.8). Besides, we adopt the Adam 3 optimizer with a base learning rate of 0.01 during Geometry Initialization and 2 x 10-5 during Geometry Sculpting.
Multi-Space Texture Refinement. For latent space texture refinement, we employ SDS op- timization 43 the latent space to optimize the coarse texture at an image resolution of 1024 x 1024 for 10, 000 steps. The loss weights for the coarse texture stage are configured as follows: Aesier
x 10², > = 100, Agol = 0.1, Agolar = 0.1. Following this, we perform pixel-space texture refinement by optimizing the UV texture map for an additional 1,000 steps to achieve finer texture details. To ensure that the added noise does not corrupt the original content while retaining the capacity to enhance image details, we empirically set the starting timestep laan to 0.05 in Eq.8 in the main text. The weight for the VGG loss (56) in the pixel space texture refinement set to ALP = 0.01. We adopt the Adam W [3 optimizer with base learning rate of 0.01 in the coarse texture stage and 0.02 in the fine texture stage.

---

Table 2: The Statistics of Multi-Source Human Dataset. Our multi-source human dataset encom- passes a diverse range of human data, categorized into 3D part data: 3D scans, multi-view videos, synthetic data, and 2D data: single images. The number of 2D images and 3D instances for each dataset is summarized in the table below.
Multi-Source Human Dataset
3D Scanned and Synthetic Human Data
RenderPeople 1
Thuman2.0
70
Thuman3.0 .0 (60
HuMMan 6
1853
526
458
9072
CostomHuman
X Humans
53
Objaverse Human
3D Part Data
647
3384
3388
Video Human Data
DNA Rendering
ZJU-Mocap
Neural Actor
22
8780
2646
4800
+IST++ 24
Actors-HQ
21
3853
3600
Our Synthetic Human Data
ControlNet- Synthetic
Image Cropping Synthetic
5642
3706
DeepFashion
LAION-5B
51
2D Part Data
20K
80K
(a) Three-quarters Body Cropping
(b) Half Body Cropping
Figure 11: Image Cropping Synthesis. We apply half-body and three-quarters-lennth cropping the multi-view renderings of 3D scans to generate synthetic cropped images.
B Details of Multi-Source Human Dataset
To facilitate the training of multi-source human diffusion models, we construct a comprehensive multi-source human dataset containing 100K 2D human images and 52, 345 multi-view 3D human instances in total. Detailed statistics of our multi-source human dataset are provided in Tab. 2 with its composition described follows: (1) The 2D human data consists of 20K human images from DeepFashion (31] and 80K human images filtered from LAION-5B [5T]. The filtering process is conducted using YOLOv7 [63], climinating images without human subjects or containing multiple individuals. Additionally, images are excluded if they have an aesthetics score below 4.5, if the largest detected face measures less than 224 224 pixels, the overall resolution falls below 640 1280. To enhance semantic richness, each image is captioned using finetuned BLIP model 25 (2) The 3D human data is collected from 3D scanned human data, synthetic human data, video human data, and our synthetic human data. 3D Scanned Human Data contains human models sourced from commercial dataset RenderPeople [ and open-source datasets including CustomHumans [3] HuMMan 6 THuman2.0 700, THuman3.0 [60] and X-Humans 53 3D Synthetic human data contains human haman-category objects filtered from Objaverse [8]. For both scanned and synthetic data, we adopt the dataset creation protocol of Zero-llto-3 [30], with the exception that uniformly select 48

---

(a) Templated Pose Synthetic
Template-free Pose Synthetic
Figure 12: Control Synthesis. We perform ControlNet-base synthesis for both templated
poses and template-free free poses.
viewpoints across 360 degrees azimuth and the resolution of the rendered image to 1024 1024. Video Human Data contains five open-source datasets: DNA-Rendering (7). ZJU-MoCap [42), Neural Actor [29], +IST++
24 and Actors-HQ 21. For datasets containing background imagery, we apply CarveKit [52] to extract human silhouettes and produce corresponding RGBA images. When datasets feature multiple groups surrounding cameras, we prioritize the group capturing the human subject at the image center. Note that we filter out cases of matting failures in these video datasets. As for DNA-Rendering (7) only Parts and 2 of the released data are utilized. Our Synthetic Human Data contains human data synthesized by two augmentation techniques: 72 image synthesis and image cropping synthesis. To further enhance the diversity of buman identities and outfits in the dataset, inspired by En3D [36], use ControlNet to synthesize a batch of multi-view human data based on diverse prompts describing outfits, genders, ages, and more, all generated by ChatGPT 22. Unlike En3D, we extract templated or template-free poses and depth images from multi-view renderings of scanned human data (RenderPeople and XHumans (53)) conditions fed into ControlNet and synthesize multi-view human images using diverse prompts. As shown in Fig. 11 and Fig. [2] examples of Image Cropping Synthesis and ControlNct- Synthesis are presented, respectively. To ensure the quality of synthetic images, concatenate pose and depth images horizontally across all views, generating multi-view humans a single inference. This concatenation strategy ensures the texture consistency of the synthetic multi-view images. To handle the reconstruction of in-the-win wild images with arbitrary human proportions, we crop the above fully body-length multi-view 3D human data into half-body length or three-quarters-body length images to create the augmented images. As for 3D data captioning, inspired by 3DTopia [16] multi-modal large language model LLaVA 28 generate captions for
ControlNet-
and Cap3D RA], 3D objects by aggregating the descriptions from multiple views. The success of our GeneMAN framework demonstrates the effectiveness of our dataset in providing generalized priors for diverse human geometry and textures.
C Additional Experiment Results
C.1
More Qualitative Results
For a more comprehensive geometric evaluation, we incorporated six additional SOTA human geometry reconstruction methods for comparison: PIFuHD [50], PaMIR (78). ICON (88] ECON 67 PHORHUM 4, and SIFU (76). The geometric comparison results are presented in Fig. and Fig. 18 8 It can be seen that our model effectively recovers natural human poses, accommodates loose clothing, and excels at reconstructing images with diverse body ratios. addition, we provide more qualitative comparisons with state-of-the-art methods: PIFu [49] GTA 74,, TeCH [19, SiTH [14 as shown in Fig. 19 to Fig. 25 Our method surpasses the compared methods in generating consistent, highly realistic textures with exceptional fidelity.

---

Besides, in Fig. 13 manLRM 66 and IDOL 779. both of which are feed-forward models trained on large-scale human datasets. As the code of Human-L.RM is unavailable, we use the examples provided in its paper. Our model authentically reconstructs 3D humans that elosely align with the front view, while IDOL fails to preserve facial details. Our model also produces natural poses with reasonable geometry (see the legs of the first and the third subjects in Fig. 13p. Moreover, it achieves high-fidelity appearance with view-consistent details, significantly outperforming HumanLRM.
we conduct additional comparisons with two generalizable methods: Hu-
We provide additional visualization results. with particular focus on complex poses, in Fig. L5 (in-the-wild images) and Fig. [66 (CAPE (35). These results highlight the strong generalization ability of GeneMAN challenging poses.
Reference
HumanLRM
IDOL
GeneMAN (Ours)
Figure 13: Additional Comparison on in-the-wild Cases with HumanLRM [64] and IDOL [79]. conduct experiments on the examples reported by HumanLRM, Best view with zoomed in.
C.2 Additional Quantitative Comparison Different Across Body Proportions
To assess the robustness of our method under varying human proportions, additionally evaluate on in-the-wild images that include head-shot, half-body, and full-body inputs. For comparison, only consider the cemplate-free method PIFu, since template-basee approaches generally fail to reconstruct head-shot and half-body inputs reliably. As shown in Table GeneMAN consistently outperforms PIFu across all proportions setting, demonstrating its strong generalization capability.
Table 3: Quantitative Comparison Across Different Human Proportions. The best results are highlighted in bold.
Full- bull-body
Half-body
Head shot
Method
PSNR
I.PPS
CLIP--im
PSNR
LPIPS J
CLIP--im
PSNR
I.PISS
CLIP-Sim
PIFu
28.039
0.029
0.622
25.383
0.052
0.477
25.047
0.075
0.522
GeneMAN (Ours)
32.529
0.012
0.751
28.245
0.016
0.681
26.374
0.022
0.661
C.3
Quantitative Geometric Evaluation
Following prior work, we perform quantitative geometric comparison with baseline methods 49
78 68 67
74
19
14 76 on the CAPE 335 dataset to assess geometry reconstruction quality.
Specifically, we report two commonly used metrics: Chamfer Distance (CD) and Point-to-Surfar distance (P2S), both measured in centimeters. between the ground truth scans and the reconstructed meshes. Additionally, to assess the fidelity of reconstructed local details, we calculate the Le error between normal images rendered from the reconstructed and GT surfaces, referred to as Normal Consistency (NC). These renderings are obtained by rotating the camera around the surfaces angles of (09, 120" 240%) relative to the frontal view.

---

However, it is crucial to highlight that previous template-based methods [78 68 67 19 14 76 directly utilize ground truth SMPL [32 41 provided in CAPE for evaluation. However, estimating body shape and pose parameters from single image is an ill-posed problem due to ambiguity, leading to multiple possible solutions, illustrated in Fig. 14 This presents an unfair advantage over our template-free approach. Moreover, our work focuses on reconstructing 3D humans with high fidelity from in-the-wild images, where accurate SMPL estimates are unavailable. To provide fair comparison, we re-evaluate the baselines the CAPE dataset under inference mode, where GT body poses are not provided as input. The lack of access to such ground truth estimations leads to notable performance drop for template-ber based methods, which deviates from the results presented in the original papers. The quantitative results are summarized in Tab. where our method outperforms the compared approaches across all geometric metries, demonstrating the superior reconstruction quality our method.
Ours
GT
Input Image
Figure 14: Visualization Results of the Reconstructed Geometry on the CAPE (35] dataset. While our template-fre free method successfully recovers plausible shapes, the orientation of the arms does not perfectly align with the ground truth, as highlighted by the red bounding box. This misalignment is difficult to avoid due the inherent ambiguity, which can result large 3D errors. Consequently, it is unfair to compare our approach with SMPL-based methods that utilize GT body parameters directly.
Table 4: Geometric Comparison with State-of-the-art Methods on the CAPE 35 dataset. The best results highlighted in bold. The second-place results are underlined.
Methods
CD 1
P2S +
NC +
PIFu
49
2,5580
2.5770
0.0925
PaMIR
78
2.5502
2.5920
0.0925
ICON 68
2.4147
2.4581
0.0872
BCON
2.0782
2.0296
0.0798
GTA 74
2,6785
2.7760
0.0914
TeCH 19
2.3217
2.4163
0.0935
SiTH 14
1.9182
2.0427
0.0726
SIFU
2.5226
2.5692
0.0875
GeneMAN (Ours)
188622
1.9724
0.0712
C.4 Ablation of Multi-Source Human Dataset
To validate the effectiveness of our constructed multi-source human dataset, we perform an ablation study each component: 3D scan human data, video human data, augmented human data, and 2D
5

---

CLIP-Sim human data. Specifically, we compare the reconstruction results using prior models trained with various combinations of these data components. The following experimental settings are evaluated: (a) "Baseline", which replaces 2D) and 3D prior models in GeneMAN with their original counterparts (Stable Diffusion V1.5 46 and Zero-I-to-3 [30]; (b) "Baseline + 3D", which uses prior models trained solely on 3D scanned human data; (c) "Baseline + 3D + Video", which employs prior models trained on both 3D scanned human data and video human data; (d) "Baseline + 3D + Video + AUG" which incorporates prior models trained on 3D scan human data, video human data and augmented human data; (e) "Ours", which utilizes prior models trained on the complete dataset. As detailed in Sec. 5.2 of the main text, evaluate the performance of each setting using PSNR. LPIPS, and scans significantly improves the model's multi-view consistency, resulting in a 0.046 increase in CLIP-similarity. Furthermore, adding video data and augmented data enhances both texture quality and consistency. By utilizing the full dataset, our full-fledged method achieves the best performance in both texture quality and consistency.
Similarity across a set of 30 test cases. As shown in Tab. | incorporating 3D
Table 5: The Effectiveness of Multi-source Human Dataset. The best results are highlighted in bold.
Methods
PSNR f
LPIPS +
Baseline
31,503
0.018
0.662
Baseline
30.873
0.017
0.708
Baseline + 3D + Vidco
31.326
0.015
0,713
Baseline 3D + Video + AUG
31.205
0.015
0.722
GeneMAN (Ours)
32.238
0.013
0.730
Table 6: Model Efficiency. Comparison of inference times between our model and the basclines. Note that PIFu [4)], GTA 174), and SiTH [14] are feed-forward methods, whereas TeCH [19] and GeneMAN (ours) are optimization based approaches that require per-subject optimization. Our GeneMAN model requires total of 1.42 hours to generate 3D human asset, with the following time breakdown: Geometry Initialization (22 min), Geometry Sculpting (15 min), Latent Space Texture Refinement (37 min), and Pixel Space Texture Refinement (11 min). All methods tested
single NVIDIA A 110 80GB GPU.
PIFu 49
GTA 774
TeCH 19
SITH [14
GeneMAN(OUrs)
Inference Time
4.6s
24s
3.5h
2min
1.42h
D Limitations
Although our method achieves superior reconstruction performance in-the-wild images compared to state-of-the-art approaches, requires a longer optimization time to generate each 3D asset compared to feed-foree reed-forwrrd methods such as PIFu 49] GTA (74), and SiTH [14), which may limit its practical applicability. Nevertheless, our approach remains faster than TeCH [19], offering 59.4% increase in efficiency while delivering better quality. The model efficiency of both baseline methods and ours are reported in Tab. Regarding reconstruction quality, our method still struggles to achieve fine-grained modeling for certain parts, such as the hands of full-body humans. Additionally, lacks specific designs for handling occluded individuals. For human-with with-objects reconstruction, our approach primarily focuses on reconstructing personal belongings, but it performs poorly with particularly large or complex objects, such bicycles. Research on human-with-objed reconstrrctioon will a key focus of our future work.
E Broader Impacts and Safeguards
GeneMAN introduces generalizable framework reconstructing high-quality 3D human models from single in-the-wild image. This method is capable of handling various body proportions,

---

poses, clothing, and personal belongings, offering a wide range of applications, including virtual reality (VR), augmented reality (AR), telepresence, digital human interfaces, film production, and 3D game development. However, the widespread application of GeneMAN may also pose risks. For instance, 3D human reconstruction technology could be misused to generate synthetic content, which may raise ethical and legal concerns. To ensure that the application of GeneMAN is positive and sustainable, we have implemented the following safeguards: 1) Ensure that the technological outcomes of GeneMAN are equitably accessible to diverse social groups, including marginalized communities. 2) Actively collaborate with communities and stakeholders to ensure that technological applications meet societal needs and expectations. 3) Ensure that the development and application of GeneMAN comply with all relevant laws and regulations, including data protection laws and intellectual property rights. 4) Respect all intellectual property rights and ensure that all used data and methods have been appropriately authorized.

---

Reference
Figure 15: More MeN Results with Complex Poses on in-the-wild Images. Best view with zoomed in.
8

---

Reference
Figure 16: More GeneMAN Results with Complex Poses on CAPE 35 Best view with zoomed
in.

---

Input Image
PIFu
PIFuHD
PaMIR
ICON
ECON
PHORHUM
TeCH
SITH
SIFU
Ours
Input Image PIFu
PiFuHD
PaMIR
ICON
ECON
PHORHUM
GTA
TeCH
SITH
SIFU
Ours
Figure 17: Geometric Comparison in-the-wild Images. Best view with zoomed in.
10

---

Input Image
PiFu
PIFuHD
PaMiR
ICON
ECON
PHORHUM
GTA
TeCH
SITH
SIFU
Ours
Input Image
PaMIR
PHORHUM
Figure 18: Geometric Comparison on in-the-wild Images. Best view with zoomed in.

---

PIFu
GTA
TeCH
SiTH
GeneMAN (Ours)
Figure 19: Qualitative Comparison on in-the-wild Image, Best view with zoomed in.
12

---

PIFu
GTA
TeCH
SiTH
GeneMAN (Ours)
Figure 20: Qualitative Comparison on in-the-wild Image, Best view with zoomed in.
13

---

PIFu
GTA
TeCH
SiTH
GeneMAN (Ours)
Figure 21: Qualitative Comparison on in-the-wild Image, Best view with zoomed in.
14

---

PIFu
11
GTA
11
TeCH
SiTH
GeneMAN (Ours)
Figure 22: Qualitative Comparison on in-the-wild Image, Best view with zoomed in.
15

---

PIFu
GTA
TeCH
SiTH
GeneMAN (Ours)
Figure 23: Qualitative Comparison on in-the-wild Image, Best view with zoomed in.
16

---

PIFu
GTA
TeCH
SiTH
GeneMAN (Ours)
Figure 24: Qualitative Comparison on in-the-wild Image, Best view with zoomed in.
17

---

PIFu
GTA
TeCH
SiTH
GeneMAN (Ours)
Figure 25: Qualitative Comparison on in-the-wild Image, Best view with zoomed in.
18


Reviews:
[{"review_id": "cwohkRm1cL", "reviewer": "Reviewer_JD8y", "summary": "This paper focuses on the classic single-view human reconstruction task and proposes a well-designed framework, named GeneMAN, to address the challenges of varying body proportions, diverse personal belongings, and inconsistencies in human texture and posture. The proposed GeneMAN leverages a comprehensive collection of multi-source human data and consists of three main modules: human prior learning using diffusion models, geometry initialization and sculpting, and multi-space texture refinement. Experiments demonstrate that GeneMAN outperforms state-of-the-art methods in generating high-quality 3D human models from single images, exhibiting better generalizability and producing more plausible and detailed geometry with realistic textures.", "questions": "Considering the strengths and weaknesses of this paper, I think this is a borderline work. Please refer to the three questions listed in the weaknesses section.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. This paper introduces a well-designed method that utilizes various knowledge and skills referred from existing research, including using a comprehensive multi-source collection of high-quality human data, 2D and 3D human priors, and coarse-to-fine strategies for geometry and texture reconstruction, to improve the quality and generalization of single-view 3D human reconstruction.\n2. This paper provides sufficient experiments, including quantitative and qualitative results, to demonstrate the effectiveness of the proposed method.\n3. This paper is well-organized and easy to read.\n\nWeaknesses:\n1. The well-designed method seems somewhat incremental. The novelty of the technical contribution is limited. I think all the modules and training skills can be found in existing work. Is there anything new for this method?\n2. Although the proposed method achieves better 3D human reconstruction, it also consumes a longer time (i.e., approximately 1.4 hours on a single A100 GPU) to optimize the results. I think it is a trade-off.\n3. Are there any quantitative comparison results for the ablation study? I think it could be better to provide these results.", "was_revised": true, "final_justification": "I appreciate the authors for answering my questions in detail in the rebuttal. Considering the contributions and shortcomings of this paper, I am still inclined to give a score of borderline accept. I suggest that the authors carefully revise the paper if it is accepted.", "rebuttal": "**Q1: The well-designed method seems somewhat incremental. The novelty of the technical contribution is limited. I think all the modules and training skills can be found in existing work. Is there anything new for this method?**\n\n**A1:** Our main contributions lie in multiple aspects, including the curation of a multi-source data pipeline, the design of various training strategies, the promising pretrained 2D and 3D prior models, and strong experimental results. Specifically:\n\n1. _**Novel Data Pipeline**_: Our novel data pipeline integrates two data augmentation strategies and is the **first to leverage multi-source human data**—including 3D scans, videos, synthetic datasets, and 2D images—for training. This pioneering use of diverse data sources significantly enhances the effectiveness, scalability, and generalization ability of our method. In contrast, most existing human reconstruction approaches are limited to training on limited amounts of high-quality scans, which restricts both their generalizability and the realism of their results. Even more generalizable methods such as LRM still rely solely on increasing the quantity of scans and synthetic human datasets, without leveraging other modalities like videos or 2D data, thereby limiting their scalability and generalization capacity compared to ours.\n\n2. _**GeneMAN Prior Models**_: We provide both 2D and 3D prior models trained on our curated multi-source human data, which can be applied to various downstream tasks.\n\n3. _**Generalization and Results**_: Our method demonstrates strong generalization capabilities and produces consistently high-quality reconstructions, outperforming existing SOTA human reconstruction methods. Moreover, our method shows promising performance on challenging scenarios involving human with objects.\n\nFinally, we are organizing and preparing our dataset for public release. We also plan to open-source our well-performing models as soon as possible to benefit and contribute to the research community.\n\n**Q2: Although the proposed method achieves better 3D human reconstruction, it also consumes a longer time (i.e., approximately 1.4 hours on a single A100 GPU) to optimize the results. I think it is a trade-off.**\n\n**A2:** Thank you for your comment. However, we believe this is not simply a trade-off between optimization time and reconstruction quality. A longer optimization time does not necessarily lead to better results. For example, although both our method and TECH are based on SDS optimization, TECH requires 3.5 hours while our method only takes 1.4 hours. Despite the shorter runtime, our method outperforms TECH by a large margin across all evaluation metrics. This improvement results from the dedicated design of our network architecture, the integration of multi-source data and training strategies, which form the core contributions of our method and enable both efficient and high-quality reconstruction.\n\n**Q3: Are there any quantitative comparison results for the ablation study? I think it could be better to provide these results.**\n\n**A3:** Thank you for your suggestion. We have conducted an additional quantitative comparison for the ablation study. As shown in Table 1, using the original 2D/3D prior model results in the lowest performance. Incorporating the 2D prior model effectively improves texture quality, as reflected by metrics such as PSNR and LPIPS. Meanwhile, employing the 3D prior model significantly enhances geometric consistency, measured by CLIP similarity. By utilizing both our 2D and 3D prior models, we achieve the best performance across all metrics.\n\n\n**Table 1: Quantitative comparison results for the ablation study**\n\n| Model            | PSNR   | LPIPS  | CLIP-Similarity |\n|------------------|--------|--------|-----------------|\n| Ori 2D / Ori 3D  | 30.157 | 0.033  | 0.642           |\n| Ours 2D / Ori 3D | 31.856 | 0.018  | 0.696           |\n| Ori 2D / Ours 3D | 31.021 | 0.025  | 0.715           |\n| GeneMAN          | 32.238 | 0.013  | 0.730           |"}, {"review_id": "nIRkOYwjJb", "reviewer": "Reviewer_rceA", "summary": "This paper proposed a generalizable framework for single-image 3D human reconstruction names GeneMAN.\n\nGiving a single-view image, the method first obtains hybrid representations 3D human with NeRF and DMTet by using 2D and 3D prior models. Then their use geometry sculpting and texture refinement to enhance the results.", "questions": "The process of Fig. 3 is not clear, and the relationships between different parts are chaotic. For example, the author uses at least 6 different arrows, what does each arrow represent? Especially the various bidirectional arrows are even more confusing.\n\nThe comparison seems unfair with GTA, TeCH and SiTH. All three methods apply SMPL(X) as an intermediate variable which may infuse pose-error in final results. The author only shows side-view results in Figure 6. I am wondering the input-view and opposite view comparison results. Furthermore, I hope to see the comparison results with SIFU(CVPR 2024).  \n\nIn 160-161, the author claims “train on Objaverse improves human geometry results”. The author should use an ablation study to support this point.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nThe process of Geometry Sculpting, and Texture Refinement improve the final results.\n\nFrom the experimental results, it can be seen that the method proposed by the author outperforms the results of the comparative methods.\n\nWeaknesses\n\nThe process of Fig. 3 is not clear, and the relationships between different parts are chaotic. For example, the author uses at least 6 different arrows, what does each arrow represent? Especially the various bidirectional arrows are even more confusing.\n\nThe comparison seems unfair with GTA, TeCH and SiTH. All three methods apply SMPL(X) as an intermediate variable which may infuse pose-error in final results. The author only shows side-view results in Figure 6. I am wondering the input-view and opposite view comparison results. Furthermore, I hope to see the comparison results with SIFU(CVPR 2024).  \n\nIn 160-161, the author claims “train on Objaverse improves human geometry results”. The author should use an ablation study to support this point.\n\nThe texture results shown in Figure 5 are not quite consistent. The texture on the input-view to be a direct back projection of the reference image onto the mesh and contains high frequency details, while the texture on the opposite-view to have generated more distinct artifacts.\n\nAs the geometry sculpting and texture refinement directly affect by the 3D prior models.  How to ensure viewpoint consistency of 3D prior models ? \n\nIn my opinion, the section of 4.1 Multi-Source Human Dataset seems more suitable as part of the related work instead of methods.\n\nFrom the essence of the model (diffusion model of images), the 2D and 3D prior models proposed by the author seem to have little difference.\n\nMissing references \n\nDIFu: Depth-Guided Implicit Function for Clothed Human Reconstruction, CVPR 2023.\n\nHigh-fidelity 3D Human Digitization from Single 2K Resolution Images, CVPR 2023.\n\nMVHumanNet: A Large-scale Dataset of Multi-view Daily Dressing Human Captures, CVPR 2024.\n\nPKU-DyMVHumans: A Multi-View Vide", "was_revised": true, "final_justification": "The authors’ response has addressed most of my concerns. I will raise my score to borderline accept.", "rebuttal": "We sincerely thank our reviewers for devoting time to this review and offering insightful feedback. The detailed responses are attached as follows.\n\n### **Weaknesses**\n\n**Q1: The process of Fig. 3 is not clear, and the relationships between different parts are chaotic. For example, the author uses at least 6 different arrows, what does each arrow represent? Especially the various bidirectional arrows are even more confusing.**\n\n**A1:** Thanks for the comment. We have already revised Fig. 3 **in the current version** to improve clarity. Specifically, we have removed all bidirectional arrows. There are now only three distinct types of arrows: the blue arrow indicates the conversion between NeRF and DMTet, while the black arrows represent the steps in the pipeline. Additionally, the two curved black arrows are included solely for typesetting purposes. We hope these changes help to clarify the relationships between the different components of the pipeline. Please refer to the PDF for double checking.\n\n**Q2: The comparison seems unfair with GTA, TeCH and SiTH. All three methods apply SMPL(X) as an intermediate variable which may infuse pose-error in final results. The author only shows side-view results in Figure 6. I am wondering the input-view and opposite view comparison results. Furthermore, I hope to see the comparison results with SIFU(CVPR 2024).**\n\n**A2:** (1) Template-based methods inevitably suffer from errors introduced during SMPL(X) estimation. In contrast, our method does not rely on such templates, offering a more robust approach. Additionally, during our evaluation with in-the-wild images, we ensured that the input to the model remained consistent across all comparisons, maintaining fairness in the evaluation process. (2) Regarding the side view in Fig. 6, we intentionally chose it to highlight the geometry quality. However, to provide a more comprehensive analysis, we have included 360-degree rotating videos in the supplementary materials, which cover the input, side, and opposite views. (3) For a detailed comparison with SIFU, please refer to Tab. 2 and Fig. 7, where we provide both quantitative and qualitative comparisons of geometry performance. It can be seen that our method produces more plausible geometry and finer textures compared to SIFU.\n\n**Q3: In 160-161, the author claims “train on Objaverse improves human geometry results”. The author should use an ablation study to support this point.**\n\n**A3:** We would like to clarify that the statement in lines 160-161 does not suggest that training with 20% Objaverse data improves human geometry reconstruction. The Zero123 model is trained on Objaverse, which provides it with the foundational capability to reconstruct 3D objects. This capability enables our fine-tuned GeneMAN model to handle HOI reconstruction. To ensure training stability and preserve consistency with the data distribution, we fine-tune the Zero123 model by mixing Objaverse data with other sources. However, the primary contribution to human geometry reconstruction comes from training on our curated multi-source human dataset. We apologize for the confusion and will revise our statement in the revision.\n\n**Q4: The texture results shown in Figure 5 are not quite consistent. The texture on the input-view to be a direct back projection of the reference image onto the mesh and contains high frequency details, while the texture on the opposite-view to have generated more distinct artifacts.**\n\n**A4:** The qualitative results in Fig. 5 highlight the advantages of our method over state-of-the-art approaches, particularly when handling complex poses such as running and hopping. GeneMAN demonstrates superior performance in recovering accurate geometry for natural poses, though there is still room for improvement in texture consistency.  Due to the rebuttal format, we cannot include additional visualizations for detailed comparisons, but these will be available in the revised paper. From the results, it is clear that our method consistently avoids the unrealistic limb distortions and noisy surfaces seen in TeCH, SiTH, and GTA. While multi-view texture consistency is inherently a challenging problem in single-image 3D reconstruction, our approach either demonstrates a slight improvement or performs on par with the baselines in this regard.\n\n**Q5: As the geometry sculpting and texture refinement directly affect by the 3D prior models. How to ensure viewpoint consistency of 3D prior models?**\n\n**A5:**  As demonstrated by Zero123, integrating camera parameters into diffusion models provides a scalable solution for achieving view-consistent 3D reconstruction. While there are no explicit 3D representations, the use of implicit controls has proven highly effective when trained on a large amount of multi-view images. Note that our curated dataset includes multi-view human data. Our ablation study in Fig. 9 (c) and (d) demonstrate the model's effectiveness in preserving 3D consistency for reconstructing humans.\n\n**Q6: In my opinion, the section of 4.1 Multi-Source Human Dataset seems more suitable as part of the related work instead of methods.**\n\n**A6:** Thank you for your suggestion. We will move a portion of the \"Multi-Source Human Dataset\" to the \"Related Work\" section. Additionally, we will retain a more concise discussion of the dataset within the \"Methods\" section to better highlight its relevance to our contributions.\n\n**Q7: From the essence of the model (diffusion model of images), the 2D and 3D prior models proposed by the author seem to have little difference.**\n\n**A7:** The architectures of the 2D and 3D prior models differ fundamentally. Specifically, the 3D prior model incorporates camera embeddings as input, enabling it to capture 3D-awareness. For a more in-depth discussion on the necessity and benefits of hybrid priors, you can refer to works such as *Magic123* (Qian et al.) and *DreamCraft3D* (Sun et al.). In brief, 3D priors contribute to enhanced multi-view texture consistency and more accurate geometric representation, while 2D priors, trained on large datasets, excel in generalization and generating high-fidelity details but may lack consistent 3D coherence. Our ablation study, as shown in Fig. 9, further illustrates the complementary effects of both 2D and 3D priors."}, {"review_id": "esB4ZuFktj", "reviewer": "Reviewer_itRn", "summary": "In this paper, the authors introduce GeneMAN, a method to reconstruct a 3D model of a human given a single image. The method consists of 3D geometry initialization and 3D texture generation steps, with the former step focusing on constructing a geometrically sculpted NeRF representation and the latter on optimizing and refining the texture of the resulting human. A 3D scanned human dataset is used to train 3D and 2D priors to assist the method, which performs decently well quantitatively and qualitatively. The authors also perform a user study to show that their work is preferred amongst their baselines.", "questions": "1. The approach is based on NeRFs instead of Gaussian splatting. While a single GS method is mentioned in the introduction, the reason for selecting NeRFs for this method is unclear. Additionally, the reported computation time of 1.4 hours on an A100 seems to impact the method's practicality —could the authors touch on this point in their introduction and related work?\n2. The authors state that templated approaches require precise human parametric models that are “frequently unattainable for in-the-wild human images”. However, the dataset examples do not appear particularly difficult for SMPL fitting, and this claim overlooks a notable advantage of templated methods—their ability to be transformed and animated to new poses. Would the authors consider revisiting or refining this argument to better reflect the strengths and limitations of both approaches?\n3. The authors claim that the method operates given an in-the-wild image. Can these images include object occlusions, and how does the method perform when faced with objects blocking the human body? In addition, does this \"in-the-wild\" term include images from side, back, and top-down views?\n4. ControlNet is used to augment data in MoVid. Wouldn’t this method introduce synthetic appearances, negatively effecting the realism of the priors?\n5. The method claims to work on numerous body proportions and poses. I recommend conducting a more extensive study of this to validate this point.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 1, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The overall approach with geometry construction and generation steps makes sense for creating a good 3D representation of a human from 2D.\n2. The figures are very clear and make the method and results very easy to understand.\n3. The paper itself is clear and well written.\n\nWeaknesses:\n\n1. The visual results of the method are poor. Facial side profiles tend to look flat (see Figure 6), and texture detail in clothing is smoothed out in unseen angles.\n2. The approach has limited novelty. The idea of using 2D and 3D priors for single-view reconstruction with SDS has been used by numerous works in the literature. Ablation studies in Figure 9 do not demonstrate very significant improvement based on these priors, suggesting the majority of the successful reconstruction is a result of the base priors and model.\n3. The evaluations are insufficient and the results appear to be visually poorer and more computationally expensive than current work. Notably, no Gaussian splatting-based methods are evaluated against. Comparisons should be conducted against DreamGaussian (Tang, 2023), as well as HGM (Chen, 2025), and LHM (Qiu, 2025).\n\nDue to the weaknesses of this work in comparison to the current literature – especially with regards to significance and originality – I recommend borderline reject.", "was_revised": true, "final_justification": "The authors addressed many of my points in the rebuttal. I have also reviewed the other reviews, and can see that many of the other reviewers share my sentiments of the method's limited novelty and quality. I am still not convinced of the method's robustness when it comes to object occlusions, and disagree with the authors' claim in the rebuttal that flat faces are \"infrequent\", as almost every side profile result in Figure 6 has this issue.\n\nWith all of this in mind, however, the authors' rebuttal did clarify most of my other questions and issues. I have decided to raise my rating to a borderline accept.", "rebuttal": "**Q1: The visual results of the method are poor. Facial side profiles tend to look flat (see Figure 6), and texture detail in clothing is smoothed out in unseen angles.**\n\n**A1:**  1. **Flat Side Face:** Thanks for your comment. We acknowledge that, due to the absence of strong priors such as the SMPL parametric model, some reconstructions may exhibit a relatively flat appearance in the facial region. However, such cases are infrequent. Moreover, this limitation could be mitigated by incorporating parametric models into our framework. Despite these imperfections, our method still outperforms state-of-the-art approaches in terms of overall reconstruction quality and generalization capability. (More comparisons are presented in the supplementary, including images and videos.)\n2. **Smoothed Detail:** Thanks for your advice. The details of the normal map can be adjusted via hyper-parameters, allowing for a trade-off between preserving more fine-grained details and achieving smoother surfaces.  In our current experimental setup, we have adopted a relatively strong smoothness regularization as the default setting. In the revised version, we will include additional results showcasing different levels of detail versus smoothing.\n\n\n**Q2: The approach has limited novelty.**\n\n**A2:** **Novelty:** Thanks for your comments. The use of 2D and 3D priors in our method is primarily intended to enhance performance, rather than being positioned as a core contribution of our approach. Our main contributions span multiple aspects, including the dataset, model design and training strategies, as well as comprehensive experimental results. The detailed explanation can be found in our reply to Reviewer JD8y’s Q1.\n\n**Ablation Studies:** The ablation studies presented in Figure 9 are intended to visually demonstrate the contributions of each stage and the two prior models. To further illustrate the improvements brought by the two prior models, we have included additional quantitative experiments.\n\n\n\n**Q3: The evaluations are insufficient and the results appear to be visually poorer and more computationally expensive than current work. Notably, no Gaussian splatting-based methods are evaluated against. Comparisons should be conducted against DreamGaussian (Tang, 2023), as well as HGM (Chen, 2025), and LHM (Qiu, 2025).**\n\n**A3:**  **Reconstruction Quality:** Current human reconstruction methods are still unable to achieve complete and reliable reconstruction on arbitrary examples, especially in complex in-the-wild scenarios. Despite some imperfections, both the main paper and supplementary clearly demonstrate that our approach outperforms existing state-of-the-art methods across several key aspects emphasized in this work: generalizability, geometric and texture consistency, the ability to handle loose clothing and complex poses, adaptation to arbitrary human body proportions, and geometric rationality with respect to the input image.\n\n**Insufficient Experiments:** Actually, we have made an effort to conduct comprehensive experiments to evaluate our method from multiple perspectives. In both the main paper and supplementary, we present quantitative and qualitative results on two datasets (CAPE and in-the-wild images), covering both geometric accuracy and texture quality. We also include evaluations under complex poses to further assess the robustness of our method. For geometry, we compare our method with 10 existing approaches; for texture, we include comparisons with 6 representative methods.\nIn selecting the baselines, we aim to cover a diverse set of paradigms. Specifically, we include traditional reconstruction methods such as PIFu, PIFuHD, PaMIR, ICON, ECON, and PHORHUM; optimization-based SDS methods like TECH and SiFu; feed-forward approaches such as HumanLRM, SiTH, and GTA-Human; as well as Gaussian-based methods like IDOL in the supplementary.\n\nThank you for your suggestion. We will include additional comparisons with more recent Gaussian-based methods, such as LHM, in our experiments. The pretrained models of HGM have not been released so far.   As shown in following Table, while LHM benefits from fast processing speed, its reconstruction quality is significantly inferior, particularly for side and rear views. Additionally, it struggles with complex garments—skirts, for instance, are often incorrectly reconstructed as legs.\n\n| Metric   | PSNR  | LPIPS | CLIP-SIMI |\n|----------|-------|-------|-----------|\n| LHM      | 24.067| 0.019 | 0.726     |\n| Ours     | 32.529| 0.012 | 0.751     |\n\n\n**Q4: While a single GS method is mentioned in the introduction, the reason for selecting NeRFs for this method is unclear.**\n\n**A4:**  NeRF is used only for initialization; our final representation is a mesh. The Gaussian-based method produces better textures but suffers from inferior geometry quality. Although increasing the number of Gaussian kernels can improve its quality, this comes at the cost of reduced speed. Nevertheless, even with such improvements, the geometric quality of Gaussian methods cannot match that of mesh-based approaches.\n\nUnlike Gaussian-based methods, which lack explicit geometry, the mesh representation supports graphics engines and enables a wider range of downstream applications. One limitation of Gaussian Splatting (GS) is that it heavily depends on the initialization and struggles to adapt to complex topologies. We also experimented with using Gaussian splatting as the primary representation stage but found the results unsatisfactory. In our comparisons with Gaussian-based methods, NeRF-based approaches achieve better geometric accuracy.\n\n**Inference time:**  Although our method is slower than Gaussian-based approaches, it achieves superior geometry and texture quality as well as better generalization capability. Moreover, compared to the state-of-the-art SDS method TECH, which requires 3.5 hours, our method only takes 1.4 hours. Despite the shorter runtime, our approach significantly outperforms TECH across all evaluation metrics. This improvement results from the dedicated design of our network architecture, the integration of multi-source data and training strategies, which form the core contributions of our method and enable both efficient and high-quality reconstruction.\n\n**Q5: The authors state that templated approaches require precise human parametric models that are “frequently unattainable for in-the-wild human images.” However, the dataset examples do not appear particularly difficult for SMPL fitting, and this claim overlooks a notable advantage of templated methods—their ability to be transformed and animated to new poses.**\n\n**A5:**  Thank you for your comment. We agree that SMPL-based methods can produce reasonable estimations for standard images, particularly those with tight clothing and neutral poses. However, in more complex real-world scenarios—such as subjects wearing loose garments, children, elderly individuals, upper-body-only images, or headshots—the template-based paradigm often introduces significant geometric bias. These errors can further propagate during subsequent optimization. For example, in our experiments, we observed that LHM often reconstructs skirts as human legs.\n\nTo address this problem, our work adopts a generation-centric, template-free approach, focusing on improving generalizability across diverse and challenging cases. This demonstrates the significance of our method in tackling such reconstruction scenarios and also constitutes one of our key contributions.\n\nWhile we acknowledge that pose-driven animation and transformation are appealing and important objectives, driving reconstructed avatars across poses remains a long-term challenge. This involves not only modeling human articulation but also handling the dynamic behavior of clothing, which remains far from solved. Template-based methods indeed offer one viable path toward animation, but we believe enhancing the generalization ability of static reconstruction is an equally important and under-explored direction in the field of human modeling.\nIn particular, we believe that the generalizable static representations produced by our method provide a strong foundation for future work on animation and 4D reconstruction. We will add a discussion on this aspect in the Future Work section of the revised manuscript.\n\n**Q6:Can these images include object occlusions, and how does the method perform when faced with objects blocking the human body? In addition, does this \"in-the-wild\" term include images from side, back, and top-down views?**\n\n**A6:** Thanks for your comment. We include challenging cases involving humans with objects and side views in Figure 1, Figure 5, Figure 6 of the main paper, and Figure 15 in the supplementary. As for back and top-down views, they generally lack identifiable features of the person, making the reconstructed identity unreliable and less practical for most real-world applications. Therefore, like most existing methods, our approach primarily focuses on front-facing humans, potentially with some degree of rotation. In our work, generalizability refers more to robustness across diverse clothing styles, age groups , complex poses, human-object interactions, and varying body proportions.\n\n**Q7: ControlNet is used to augment data in MoVid. Wouldn’t this method introduce synthetic appearances, negatively affecting the realism of the priors?**\n\n**A7:** Overusing ControlNet-based synthetic data can indeed negatively impact generation quality. In our experiments, we carefully controlled the proportion of synthetic data to ensure it contributes positively to model performance.\n\n**Q8: The method claims to work on numerous body proportions and poses. I recommend conducting a more extensive study of this to validate this point.**\n\n**A8:** Thank you for your suggestion. We have conducted an additional quantitative comparison for the ablation study, shown in the following table."}, {"review_id": "MXYXECacHX", "reviewer": "Reviewer_VcUE", "summary": "This paper introduces a multi-source human dataset aimed at facilitating 3D human reconstruction in the wild. The dataset combines 3D scans, multi-view videos, images, and ControlNet-synthesized data. The authors utilize this dataset to train human-specific 2D and 3D prior models. These priors are used to guide the 3D human reconstruction process through score distillation sampling (SDS) losses. Thanks to the generalizability of the learned priors, the proposed method demonstrates robustness to varying body proportions (e.g., full-body shot, head shot), diverse human belongings, and postures.", "questions": "(i) Existing reconstruction methods are typically trained with full-body images, and thus might be biased towards such inputs. Can the authors provide quantitative evaluation results exclusively for head shots, half-body, or full-body inputs? It would be beneficial to understand the advantage of the proposed approach. According to Figure 11 of the supplementary material, it substantially outperforms other SoTA methods on head shots.\n\n(ii) As the proposed method is template-free, it enables joint human-object reconstruction. Figure 1(b) seems to be interesting. How much does the proposed method outperform other template-free methods (e.g., HumanLRM) in those scenarios?\n\n--- \n\nI am currently leaning toward `Borderline Reject`, but open to raising the score if the authors clearly show the method's advantages over existing approaches.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "< Strengths > \n\n(i) The multi-source human dataset is comprehensive, capturing a wide variety of human conditions and appearances. This diverse dataset can facilitate in-the-wild 3D human reconstruction. The authors utilize the dataset to build generalizable human-specific prior models, which would benefit the research community.\n\n(ii) The proposed geometry reconstruction method is technically sounding. The SDS losses using the generalizable prior models would be effective to handle diverse scenarios. The authors also utilize depth and normal maps estimated from the human foundation model, Sapiens [A], which is also reasonable.\n\n(iii) The multi-space texture refinement seems to be effective for alleviating over-smoothing or color deviation issues commonly caused by SDS losses. The pixel space refinement is technically sounding.\n\n(iv) The proposed method outperforms existing state-of-the-art reconstruction methods (e.g., TeCH [B], SiTH[C]), as shown in Table 1 and Figures 5-7. \n\n[A] Khirodkar et al., “Sapiens: Foundation for Human Vision Models”, ECCV, 2024.\n\n[B] Huang et al., “TeCH: Text-guided Reconstruction of Lifelike Clothed Humans”, 3DV, 2024.\n\n[C] Ho et al., “SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion”, CVPR, 2024.\n\n---\n\n< Weaknesses >\n\n(i) While the paper highlights the multi-source human dataset as its main contribution, a significant portion of the data is sourced from existing public datasets, as shown in Table 1 of the supplementary material. Furthermore, existing datasets already offer a broader scale and diversity (e.g., 100K diverse high-fidelity humans in HuGe100K [D]), surpassing the 50K instances in the proposed dataset.\n\n(ii) The proposed reconstruction method appears to be incrementally novel. The geometry initialization is similarly done in Magic123 [E], the geometry sculpting follows a similar approach to HumanNorm [F], and the multi-space texture refinement resembles techniques commonly used in image-to-3D object reconstruction methods. Also, the high-level idea is similar to HumanLRM [G], which also leverages a pre-trained diffusion model for high-quality template-free reconstruction; although it does not use SDS losses.\n\n(iii) As shown in Table 4 of the supplementary material, the proposed method is significantly slower than feed-forward approaches. This raises concerns about its practical applicability. It would be interesting to see the trade-off between reconstruction quality and computational efficiency, particularly in comparison to SoTA feed-forward methods like HumanLRM [G]. As shown in Figure 3 of the supplementary material, HumanLRM appears to achieve competitive reconstruction results at much faster inference speeds.\n\n[D] Zhuang  et al., “IDOL: Instant Photorealistic 3D Human Creation from a Single Image”, CVPR, 2025.\n\n[E] Qian et al., “Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors\", ICLR, 2024.\n\n[F] Huang et al., “HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generation”, CVPR, 2024.\n\n[G] Weng et al., “Template-Free Single-View 3D Human Digitalization with Diffusion-Guided LRM”, arXiv:2401.12175, 2024.", "was_revised": true, "final_justification": "The authors have addressed most concerns and included evaluation results demonstrating the proposed method's generalization capability. While HumanLRM [A] appears competitive and faster, it is not yet open-sourced. As the authors will release the GeneMAN model and the multi-source human dataset, this work can provide meaningful value to the research community. I therefore raise my rating to `Borderline Accept`.\n\n- [A] Weng et al., “Template-Free Single-View 3D Human Digitalization with Diffusion-Guided LRM”, arXiv:2401.12175, 2024.", "rebuttal": "**Q1: While the paper highlights the multi-source human dataset as its main contribution, a significant portion of the data is sourced from existing public datasets, as shown in Table 1 of the supplementary material. Furthermore, existing datasets already offer a broader scale and diversity (e.g., 100K diverse high-fidelity humans in HuGe100K [D]), surpassing the 50K instances in the proposed dataset.**\n\n**A1:** It is important to note that the HuGe100K dataset proposed in IDOL is synthesized using a video generative model. In contrast, our multi-source dataset primarily consists of real-world data (except the augmented portion), which contributes to more natural geometry and appearance reconstruction with high fidelity. Furthermore, HuGe100K is limited to full-body reconstructions, whereas GeneMAN can robustly handle images with varying body proportions. The comparison with IDOL in Figure 13 clearly demonstrates that our method, trained on a diverse multi-source dataset, yields superior results compared to training on synthetic data alone. Besides, IDOL is a concurrent work to ours, and our dataset was developed around the same time as theirs.\n\n**Q2: The proposed reconstruction method appears to be incrementally novel. The geometry initialization is similarly done in Magic123 [E], the geometry sculpting follows a similar approach to HumanNorm [F], and the multi-space texture refinement resembles techniques commonly used in image-to-3D object reconstruction methods. Also, the high-level idea is similar to HumanLRM [G], which also leverages a pre-trained diffusion model for high-quality template-free reconstruction; although it does not use SDS losses.**\n\n**A2:** Thanks for your comments. We leverage certain existing techniques with the intent of enhancing performance, rather than positioning them as core contributions of our approach. Our main contributions span multiple aspects, including the dataset, model design and training strategies, as well as comprehensive experimental results. Specifically:\n\n1. _**Novel Data Pipeline**_: Our novel data pipeline integrates two data augmentation strategies and is the **first to leverage multi-source human data**—including 3D scans, videos, synthetic datasets, and 2D images—for training. This pioneering use of diverse data sources significantly enhances the effectiveness, scalability, and generalization ability of our method. In contrast, most existing human reconstruction approaches are limited to training on limited amounts of high-quality scans, which restricts both their generalizability and the realism of their results. Even more generalizable methods such as LRM still rely solely on increasing the quantity of scans and synthetic human datasets, without leveraging other modalities like videos or 2D data, thereby limiting their scalability and generalization capacity compared to ours.\n\n2. _**GeneMAN Prior Models**_: We provide both 2D and 3D prior models trained on our curated multi-source human data, which can be applied to various downstream tasks.\n\n3. _**Generalization and Results**_: Our method demonstrates strong generalization capabilities and produces consistently high-quality reconstructions, outperforming existing SOTA human reconstruction methods. Moreover, our method shows promising performance on challenging scenarios involving humans with objects.\n\nFinally, we are organizing and preparing our dataset for public release. We also plan to open-source our well-performing models as soon as possible to benefit and contribute to the research community.\n\n**Q3: As shown in Table 4 of the supplementary material, the proposed method is significantly slower than feed-forward approaches. This raises concerns about its practical applicability. It would be interesting to see the trade-off between reconstruction quality and computational efficiency, particularly in comparison to SoTA feed-forward methods like HumanLRM [G]. As shown in Figure 3 of the supplementary material, HumanLRM appears to achieve competitive reconstruction results at much faster inference speeds.**\n\n**A3:** HumanLRM is not open-source, so our comparison is based on results from their official website, which may involve selectively chosen cases. This means we cannot fully evaluate its performance on a broader range of in-the-wild cases. In contrast, GeneMAN is designed to excel in fine-grained geometry and generalization for challenging cases, which can lead to slower inference speeds but offers superior reconstruction quality in complex situations. As shown in Supp. Figure 3, HumanLRM exhibits artifacts, such as unrealistic deformations in the head and beanie, and issues with the backpack and hair, while our method produces more accurate and detailed results. As HumanLRM is not intended to be open-sourced, we will release our method to fill this gap, offering the community a high-quality and generalizable approach for human reconstruction.\n\n**Q4: Existing reconstruction methods are typically trained with full-body images, and thus might be biased towards such inputs. Can the authors provide quantitative evaluation results exclusively for head shots, half-body, or full-body inputs? It would be beneficial to understand the advantage of the proposed approach. According to Figure 11 of the supplementary material, it substantially outperforms other SoTA methods on head shots.**\n\n**A4:** Thank you for the valuable suggestion. We have conducted additional experiments to investigate the performance over in-the-wild images of different body proportions, i.e., head shots, half-body, or full-body inputs. The quantitative metrics are reported in the following tables, showing that our method outperforms PIFu, which is known as one of the most generalizable methods, across all settings. We will include this comparison in the revised version to better highlight our advantages.\n\n**Full-body**\n\n| Method | PSNR   | LPIPS  | CLIP-Similarity |\n|--------|--------|--------|-----------------|\n| PIFu   | 28.039 | 0.029  | 0.622           |\n| Ours   | 32.529 | 0.012  | 0.751           |\n\n**Half-body**\n\n| Method | PSNR   | LPIPS  | CLIP-Similarity |\n|--------|--------|--------|-----------------|\n| PIFu   | 25.383 | 0.052  | 0.477           |\n| Ours   | 28.245 | 0.016  | 0.681           |\n\n**Head-shot**\n\n| Method | PSNR   | LPIPS  | CLIP-Similarity |\n|--------|--------|-----------------|----------------|\n| PIFu   | 25.047 | 0.075 | 0.522           |\n| Ours   | 26.374|0.022|0.661          |\n\n\n**Q5: As the proposed method is template-free, it enables joint human-object reconstruction. Figure 1(b) seems to be interesting. How much does the proposed method outperform other template-free methods (e.g., HumanLRM) in those scenarios?**\n\n**A5:** As noted in A3, HumanLRM is not open-sourced, and despite our attempts to obtain the code by reaching out to the authors, we did not receive a response. Therefore, we can only compare against the results provided in their paper or official website. In addition, we conduct comparisons with other template-free methods, such as PIFu and PIFuHD. It is noteworthy that the majority of recent methods are template-based. Furthermore, we present HOI examples in Figure 1, Figure 5, and Figure 6 in the main text. In the revision, we will include additional HOI examples to further showcase the advantages of GeneMAN in such scenarios."}]

Author rebuttal:

Meta-review: This paper presents an approach to reconstruct the 3D geometry and appearance of a person from a single image. The method leverages multiple sources of data (e.g., images, videos, and 3D scans) to learn human-based priors and use them during reconstruction. Initially, the paper received one Borderline Accept and three Borderline Reject ratings. Reviewers generally appreciated the quantitative results and the dataset contribution, but expressed concerns that the technical approach was incremental and slow, with questions about its generalization ability and the fairness of the evaluation. The rebuttal initiated discussion between the authors and the reviewers, and it helped clarify several critical concerns. While some issues remain (e.g., limited novelty, uncertainty regarding robustness), the clarifications and additional results provided in the rebuttal addressed the most significant reservations. As a result, the reviewers updated their ratings to four Borderline Accepts. Given the unanimous acceptance recommendation from four knowledgeable reviewers, there is no basis to overturn these reviews. The AC recommends acceptance. Authors should still consider any additional comments or feedback from the reviewers while preparing their final version and of course update the manuscript to include any additional promised changes, analysis, results and/or discussion they provided in their rebuttal.
Decision: Accept (poster)