Title: GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Data
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D Human Reconstruction;  Multi-source Human Dataset; Single Image-3D Reconstruction
Abstract: Given a single in-the-wild human photo, it remains a challenging task to reconstruct a high-fidelity 3D human model. Existing methods face difficulties including a) the varying body proportions captured by in-the-wild human images; b) diverse personal belongings within the shot; and c) ambiguities in human postures and inconsistency in human textures. In addition, the scarcity of high-quality human data intensifies the challenge. To address these problems, we propose a Generalizable image-to-3D huMAN reconstruction framework, dubbed GeneMAN, building upon a comprehensive multi-source collection of high-quality human data, including 3D scans, multi-view videos, single photos, and our generated synthetic human data. GeneMAN encompasses three key modules. 1) Without relying on parametric human models (e.g., SMPL), GeneMAN first trains a human-specific text-to-image diffusion model and a view-conditioned diffusion model, serving as GeneMAN 2D human prior and 3D human prior for reconstruction, respectively. 2) With the help of the pretrained human prior models, the Geometry Initialization-&-Sculpting pipeline is leveraged to recover high-quality 3D human geometry given a single image. 3) To achieve high-fidelity 3D human textures, GeneMAN employs the Multi-Space Texture Refinement pipeline, consecutively refining textures in the latent and the pixel spaces. Extensive experimental results demonstrate that GeneMAN could generate high-quality 3D human models from a single image input, outperforming prior state-of-the-art methods. Notably, GeneMAN could reveal much better generalizability in dealing with in-the-wild images, often yielding high-quality 3D human models in natural poses with common items, regardless of the body proportions in the input images.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a multi-source human dataset aimed at facilitating 3D human reconstruction in the wild. The dataset combines 3D scans, multi-view videos, images, and ControlNet-synthesized data. The authors utilize this dataset to train human-specific 2D and 3D prior models. These priors are used to guide the 3D human reconstruction process through score distillation sampling (SDS) losses. Thanks to the generalizability of the learned priors, the proposed method demonstrates robustness to varying body proportions (e.g., full-body shot, head shot), diverse human belongings, and postures.\n\nSTRENGTHS AND WEAKNESSES:\n< Strengths > \n\n(i) The multi-source human dataset is comprehensive, capturing a wide variety of human conditions and appearances. This diverse dataset can facilitate in-the-wild 3D human reconstruction. The authors utilize the dataset to build generalizable human-specific prior models, which would benefit the research community.\n\n(ii) The proposed geometry reconstruction method is technically sounding. The SDS losses using the generalizable prior models would be effective to handle diverse scenarios. The authors also utilize depth and normal maps estimated from the human foundation model, Sapiens [A], which is also reasonable.\n\n(iii) The multi-space texture refinement seems to be effective for alleviating over-smoothing or color deviation issues commonly caused by SDS losses. The pixel space refinement is technically sounding.\n\n(iv) The proposed method outperforms existing state-of-the-art reconstruction methods (e.g., TeCH [B], SiTH[C]), as shown in Table 1 and Figures 5-7. \n\n[A] Khirodkar et al., “Sapiens: Foundation for Human Vision Models”, ECCV, 2024.\n\n[B] Huang et al., “TeCH: Text-guided Reconstruction of Lifelike Clothed Humans”, 3DV, 2024.\n\n[C] Ho et al., “SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion”, CVPR, 2024.\n\n---\n\n< Weaknesses >\n\n(i) While the paper highlights the multi-source human dataset as its main contribution, a significant portion of the data is sourced from existing public datasets, as shown in Table 1 of the supplementary material. Furthermore, existing datasets already offer a broader scale and diversity (e.g., 100K diverse high-fidelity humans in HuGe100K [D]), surpassing the 50K instances in the proposed dataset.\n\n(ii) The proposed reconstruction method appears to be incrementally novel. The geometry initialization is similarly done in Magic123 [E], the geometry sculpting follows a similar approach to HumanNorm [F], and the multi-space texture refinement resembles techniques commonly used in image-to-3D object reconstruction methods. Also, the high-level idea is similar to HumanLRM [G], which also leverages a pre-trained diffusion model for high-quality template-free reconstruction; although it does not use SDS losses.\n\n(iii) As shown in Table 4 of the supplementary material, the proposed method is significantly slower than feed-forward approaches. This raises concerns about its practical applicability. It would be interesting to see the trade-off between reconstruction quality and computational efficiency, particularly in comparison to SoTA feed-forward methods like HumanLRM [G]. As shown in Figure 3 of the supplementary material, HumanLRM appears to achieve competitive reconstruction results at much faster inference speeds.\n\n[D] Zhuang  et al., “IDOL: Instant Photorealistic 3D Human Creation from a Single Image”, CVPR, 2025.\n\n[E] Qian et al., “Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors\", ICLR, 2024.\n\n[F] Huang et al., “HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generation”, CVPR, 2024.\n\n[G] Weng et al., “Template-Free Single-View 3D Human Digitalization with Diffusion-Guided LRM”, arXiv:2401.12175, 2024.\n\nQUESTIONS:\n(i) Existing reconstruction methods are typically trained with full-body images, and thus might be biased towards such inputs. Can the authors provide quantitative evaluation results exclusively for head shots, half-body, or full-body inputs? It would be beneficial to understand the advantage of the proposed approach. According to Figure 11 of the supplementary material, it substantially outperforms other SoTA methods on head shots.\n\n(ii) As the proposed method is template-free, it enables joint human-object reconstruction. Figure 1(b) seems to be interesting. How much does the proposed method outperform other template-free methods (e.g., HumanLRM) in those scenarios?\n\n--- \n\nI am currently leaning toward `Borderline Reject`, but open to raising the score if the authors clearly show the method's advantages over existing approaches.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors introduce GeneMAN, a method to reconstruct a 3D model of a human given a single image. The method consists of 3D geometry initialization and 3D texture generation steps, with the former step focusing on constructing a geometrically sculpted NeRF representation and the latter on optimizing and refining the texture of the resulting human. A 3D scanned human dataset is used to train 3D and 2D priors to assist the method, which performs decently well quantitatively and qualitatively. The authors also perform a user study to show that their work is preferred amongst their baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The overall approach with geometry construction and generation steps makes sense for creating a good 3D representation of a human from 2D.\n2. The figures are very clear and make the method and results very easy to understand.\n3. The paper itself is clear and well written.\n\nWeaknesses:\n\n1. The visual results of the method are poor. Facial side profiles tend to look flat (see Figure 6), and texture detail in clothing is smoothed out in unseen angles.\n2. The approach has limited novelty. The idea of using 2D and 3D priors for single-view reconstruction with SDS has been used by numerous works in the literature. Ablation studies in Figure 9 do not demonstrate very significant improvement based on these priors, suggesting the majority of the successful reconstruction is a result of the base priors and model.\n3. The evaluations are insufficient and the results appear to be visually poorer and more computationally expensive than current work. Notably, no Gaussian splatting-based methods are evaluated against. Comparisons should be conducted against DreamGaussian (Tang, 2023), as well as HGM (Chen, 2025), and LHM (Qiu, 2025).\n\nDue to the weaknesses of this work in comparison to the current literature – especially with regards to significance and originality – I recommend borderline reject.\n\nQUESTIONS:\n1. The approach is based on NeRFs instead of Gaussian splatting. While a single GS method is mentioned in the introduction, the reason for selecting NeRFs for this method is unclear. Additionally, the reported computation time of 1.4 hours on an A100 seems to impact the method's practicality —could the authors touch on this point in their introduction and related work?\n2. The authors state that templated approaches require precise human parametric models that are “frequently unattainable for in-the-wild human images”. However, the dataset examples do not appear particularly difficult for SMPL fitting, and this claim overlooks a notable advantage of templated methods—their ability to be transformed and animated to new poses. Would the authors consider revisiting or refining this argument to better reflect the strengths and limitations of both approaches?\n3. The authors claim that the method operates given an in-the-wild image. Can these images include object occlusions, and how does the method perform when faced with objects blocking the human body? In addition, does this \"in-the-wild\" term include images from side, back, and top-down views?\n4. ControlNet is used to augment data in MoVid. Wouldn’t this method introduce synthetic appearances, negatively effecting the realism of the priors?\n5. The method claims to work on numerous body proportions and poses. I recommend conducting a more extensive study of this to validate this point.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 1, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a generalizable framework for single-image 3D human reconstruction names GeneMAN.\n\nGiving a single-view image, the method first obtains hybrid representations 3D human with NeRF and DMTet by using 2D and 3D prior models. Then their use geometry sculpting and texture refinement to enhance the results.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nThe process of Geometry Sculpting, and Texture Refinement improve the final results.\n\nFrom the experimental results, it can be seen that the method proposed by the author outperforms the results of the comparative methods.\n\nWeaknesses\n\nThe process of Fig. 3 is not clear, and the relationships between different parts are chaotic. For example, the author uses at least 6 different arrows, what does each arrow represent? Especially the various bidirectional arrows are even more confusing.\n\nThe comparison seems unfair with GTA, TeCH and SiTH. All three methods apply SMPL(X) as an intermediate variable which may infuse pose-error in final results. The author only shows side-view results in Figure 6. I am wondering the input-view and opposite view comparison results. Furthermore, I hope to see the comparison results with SIFU(CVPR 2024).  \n\nIn 160-161, the author claims “train on Objaverse improves human geometry results”. The author should use an ablation study to support this point.\n\nThe texture results shown in Figure 5 are not quite consistent. The texture on the input-view to be a direct back projection of the reference image onto the mesh and contains high frequency details, while the texture on the opposite-view to have generated more distinct artifacts.\n\nAs the geometry sculpting and texture refinement directly affect by the 3D prior models.  How to ensure viewpoint consistency of 3D prior models ? \n\nIn my opinion, the section of 4.1 Multi-Source Human Dataset seems more suitable as part of the related work instead of methods.\n\nFrom the essence of the model (diffusion model of images), the 2D and 3D prior models proposed by the author seem to have little difference.\n\nMissing references \n\nDIFu: Depth-Guided Implicit Function for Clothed Human Reconstruction, CVPR 2023.\n\nHigh-fidelity 3D Human Digitization from Single 2K Resolution Images, CVPR 2023.\n\nMVHumanNet: A Large-scale Dataset of Multi-view Daily Dressing Human Captures, CVPR 2024.\n\nPKU-DyMVHumans: A Multi-View Vide\n\nQUESTIONS:\nThe process of Fig. 3 is not clear, and the relationships between different parts are chaotic. For example, the author uses at least 6 different arrows, what does each arrow represent? Especially the various bidirectional arrows are even more confusing.\n\nThe comparison seems unfair with GTA, TeCH and SiTH. All three methods apply SMPL(X) as an intermediate variable which may infuse pose-error in final results. The author only shows side-view results in Figure 6. I am wondering the input-view and opposite view comparison results. Furthermore, I hope to see the comparison results with SIFU(CVPR 2024).  \n\nIn 160-161, the author claims “train on Objaverse improves human geometry results”. The author should use an ablation study to support this point.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the classic single-view human reconstruction task and proposes a well-designed framework, named GeneMAN, to address the challenges of varying body proportions, diverse personal belongings, and inconsistencies in human texture and posture. The proposed GeneMAN leverages a comprehensive collection of multi-source human data and consists of three main modules: human prior learning using diffusion models, geometry initialization and sculpting, and multi-space texture refinement. Experiments demonstrate that GeneMAN outperforms state-of-the-art methods in generating high-quality 3D human models from single images, exhibiting better generalizability and producing more plausible and detailed geometry with realistic textures.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper introduces a well-designed method that utilizes various knowledge and skills referred from existing research, including using a comprehensive multi-source collection of high-quality human data, 2D and 3D human priors, and coarse-to-fine strategies for geometry and texture reconstruction, to improve the quality and generalization of single-view 3D human reconstruction.\n2. This paper provides sufficient experiments, including quantitative and qualitative results, to demonstrate the effectiveness of the proposed method.\n3. This paper is well-organized and easy to read.\n\nWeaknesses:\n1. The well-designed method seems somewhat incremental. The novelty of the technical contribution is limited. I think all the modules and training skills can be found in existing work. Is there anything new for this method?\n2. Although the proposed method achieves better 3D human reconstruction, it also consumes a longer time (i.e., approximately 1.4 hours on a single A100 GPU) to optimize the results. I think it is a trade-off.\n3. Are there any quantitative comparison results for the ablation study? I think it could be better to provide these results.\n\nQUESTIONS:\nConsidering the strengths and weaknesses of this paper, I think this is a borderline work. Please refer to the three questions listed in the weaknesses section.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}