Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models
Authors: Yuxuan Xue
Xianghui Xie
Riccardo Marin
Gerard Pons-Moll
Venue: NeurIPS 2024 poster (2024)
Keywords: 3D Reconstruction
3D Human Reconstruction
Diffusion Models
3D Generative Models
2D Foundation Models
Primary area: machine_vision
TL;DR: A novel 3D consistent diffusion framework that utilize pretrained 2D diffusion prior for 3D reconstruction and use reconstructed 3D to guide 2D sampling process

Abstract: Creating realistic avatars from a single RGB image is an attractive yet challenging problem. To deal with challenging loose clothing or occlusion by interaction objects, we leverage powerful shape prior from 2D diffusion models pretrained on large datasets. Although 2D diffusion models demonstrate strong generalization capability, they cannot provide multi-view shape priors with guaranteed 3D consistency. We propose Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion. Our key insight is that 2D multi-view diffusion and 3D reconstruction models provide complementary information for each other. By coupling them in a tight manner, we can fully leverage the potential of both models. We introduce a novel image-conditioned generative 3D Gaussian Splats reconstruction model that leverages the prior from 2D multi-view diffusion models, and provides an explicit 3D representation, which further guides the 2D reverse sampling process
to have better 3D consistency. Experiments show that our proposed framework outperforms state-of-the-art methods and enables the creation of realistic avatars from a single RGB image, achieving high-fidelity in both geometry and appearance. Extensive ablations also validate the efficacy of our design, (1) multi-view 2D priors conditioning in generative 3D reconstruction and (2) consistency refinement of sampling trajectory via the explicit 3D representation. Our code and models are released at https://yuxuan-xue.com/human-3diffusion/.

Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models
Yuxuan Xue Xianghui X2,3 Riccardo Marin 1.2
Gerard Pons-Moll 4.2.3
1 University of Tübingen 2 Tübingen Al Center
Max Planck Institute for Informaties, Saarland Informatics Campus
https: /yyple///uuuuuuuee -uee httm/hummn-diiie
Figure 1: Given single image of person (top), our method Human-3Diffusin creates 3D Gaussian Splats realistic avatars with cloth and interacting objects with high-fideliyy geometry and texture.
Abstract
Creating realistic avatars from a single RGB image is an attractive yet challenging problem. To deal with challenging loose clothing occlusion by interaction ob- jects, we leverage powerful shape prior from 2D diffusion models pretrained on large datasets. Although 2D diffusion models demonstrate strong generalization capability, they cannot provide multi-view shape priors with guaranteed 3D consis- tency. We propose Hluman-DDifussii Consistent Diffusion. Our key insight is that 2D multi-view diffusion and 3D recon- struction models provide complementary information for each other. By coupling them in tight manner, we can fully leverage the potential of both models. We introduce novel image-conditioned generative 3D Gaussian Splats reconstruction model that leverages the prior from 2D multi-view diffusion models, and provides an explicit 3D representation, which further guides the 2D reverse sampling process to have better 3D consistency. Experiments show that our proposed framework outperforms state-of-the-art methods and enables the creation realistic avatars from single RGB image, achieving high-fidelity in both geometry and appearance. Extensive ablations also validate the efficacy of our design,
Realistic Avatar Creation via Explicit 3D
(I) multi-view 2D priors conditioning in generative 3D reconstruction and (2) consistency refinement
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

of sampling trajectory via the explicit 3D representation. Our code and models will be released here.
Introduction
Realistic human avatar creation is crucial for various applications such as AR/VR, as well as the movie and gaming industry. Methods for creating a 3D avatar from a single RGB image are especially important to scale up avatar creation and make it more consumer-friendy. compared to traditional studio-based capture methods. This task is, however, very challenging due to the vast diversity of human bodies and poses, further complicated by the wide variety of clothing and accessories. These challenges are exacerbated by the lack of large-scale 3D human data and ambiguities inherent in a monocular 2D view setting.
Recent image-to-3D approaches can be categorized into reconstruction-basee and multi-view diffusion- methods. Reconstruction--asse approaches directly predict a 3D representation that can be rendered from any viewpoint. Due the explicit 3D representation, these methods produce an arbitrary number of consistent viewpoint renderings. They obtain the 3D reconstruction either based on common template [23, 87, 88, 108] which utilize the SMPL [46] body model as the shape prior, or flexible implicit function to represent loose clothing [5, 56, 57]. These methods, either template-based [23, 87, 88, 108] or template-free [5, 26, 56, 57, 74, 110], are typically deter- ministic which produce blurry textures and geometry in the occluded regions. More importantly, they are trained on small-scall scale datasets due to the limited amount of high-quality 3D data, which further restricts their ability to generalize to diverse shapes and textures.
Multi-view diffusion methods [42, 60, 77] distill the inherent 3D structure present in 2D diffusion models [55]. Typically, they fine-tune large-scale 2D foundation model [25, 64] on large 3D dataset of objects [13, 82, 99], to produce a fixed number of viewpoints. However, since these models diffuse images purely in 2D without explicit 3D constraints or representation, the resulting multi-views often lack 3D consistency [54, 41J, which restricts downstream applications [67].
To address these challenges, we propose :ifffan-Diffusion realistic avatar creation via 3D consis- tent Diffusion models. We design method based two key insights: 1) 2D multi-view diffusion models provide large-scale shape priors that can help 3D reconstruction: 2) A reconstructed 3D representation ensures 3D consistency across multi-views 2D diffusion. Specifically, we propose a novel diffusion method, which bridges 3D Gaussian Splatting (3D-GS) [36] generation with a 2D multi-view diffusion model. At every iteration, multi-view images are denoised and reconstructed to 3D-GS to be re-rendered to continue the diffusion process. This 3D lifting during iterative sampling ensures the 3D consistency of the 2D diffusion model while leveraging large-scatt foundation model trained on billions of images. Our framework elegantly combines reconstruction methods with multi-view diffusion models. In summary, our contributions are:
We propose a novel mage-condiitonnel 3D-GS generation model for 3D reconstruction that bridges large-scale priors from 2D multi-view diffusion models and the efficient and explicit 3D-GS representation.
A sophisticated diffusion process that incorporates reconstructed 3D-GS to improve the 3D consistency of 2D diffusion models by refining the reverse sampling trajectory.
Our proposed formulation enables us to jointly train 2D diffusion and our 3D model 6000 high-quality human scans and our method shows superior performance and generalization capability than prior works. Our code and pretrained models will be publicly released on project page.
2
Related Work
Image 3D. Creating realistic human avatar from consumer grade sensors [33, 96, 73, 92-94]
is essential for downstream tasks such as human behaviour understanding [9, 51, 84, 86, 85] and gaming application [38, 44, 20, 105, 106). Researchers have explored avatar creation from monocular RGB [31, 80), Depth [16, 93] video single image [56, 57, 59, 87, 88]. Avatar from single image is particularly interesting and existing methods can be roughly categorized as template- based [23, 87, 88, 108] and template-free [56, 57, 59, 95]. Despite the impressive performance,

---

template-basee approaches rely on the naked body model [46, 50] and fail to reason extremely loose clothing, while template-fred free methods produce blurry back side textures. Instead of SMPL shape prior, our method is template-free and leverages strong 2D) image priors to create high-quality avatars. Orthogonal to humans, object reconstruction methods typically adopt template-free paradigms and early works [6, 71, 81, 83, 107] focus mainly on geometry. With the advance of 21D diffusion models [55] and efficient 3D representation [12], recent works can reconstruct 3D objects with detailed textures [26, 41, 45, 60, 67, 74, 89, 90, 110]. One popular paradigm is first using strong 2D models [42, 61, 77] to produce multi-view images and then train another model to reconstruct 3D from -iew images [41, 40, 45, 67, 91]. In practice, their performance is limited by the accuracy of the multi-view images generated by 2D diffusion modes. Our method tightly couples 2D and 3D models and yields better performance by guiding 2D sampling with 3D reconstruction.
Shape Prior from 2D Diffusion Model. Being trained on billions of images [58]. 2D image diffusion models [55] have been shown to have 3D awareness and some works tried to use score distillation sampling [53, 79] to distil 3D knowledge of 2D models [39, 48, 109]. Other works propose to further enhance the 3D reasoning ability by fine-tuning the model on large-scale datasets [13, 82, 99] to generate multi-view images [34, 37, 42, 43, 60, 61, 68, 76, 77]. Dense self-attention [75, 77], depth-aware attention [28] or epipolar attention [30, 66] are introduced to enhance the 3D consistency of multi-views. However, these methods do not have explicit 3D while our method incorporates explicit 3D consistency into the reverse sampling process and obtains better results.
3 Preliminaries
Denoising Diffusion Probabilistic Models. DDPM [25] is generative model which learns data distribution by iteratively adding (forward process) and removing (reverse process) the noise. Formally, the forward process iteratively adds noise a sample xo drawn from distribution Paata Paaa(x):
X, N(x; Va(xi-1, (1 aI) vāpa vI - Vāāc where
(O,
(1)
where at a. schedules the amount of noise added at each step & [25]. To sample data from the learned distribution, the reverse process starts from XT ~ NO, I) and iteratively denoises it until t 0:
X (x (x,, t), II), where _ 1 - -
az)
(2)
1 0tt
network parametrized by is trained to estimate the posterior mean (s0 at each step L. One can also model conditional distribution with DDPM by adding the condition to the network input [15, 24].
2D Multi-View Diffusion Models. Many recent works [42, 43, 45, 60, 68, 77] propose to leverage strong 2D image diffusion prior [55] pre-trained on billions images [58] to generate multi-view images from a single image. Among them, ImageDream [77] demonstrated a superior generalization capability to unseen objects [67]. Given a single condition image x and an optional text description y, ImageDream generate 4 orthogonal target views x with model eg. which is trained to estimate the noise added at each step l. With the estimated noise eg, one can compute the "clear" target views x)e with close-form solution in Eq. (1):
1
(x.
V1 ox xf, v. ())-
Vaa
!gt This one-step estimation of x can be noisy, especially when t is large and x let is extremely noisy. Thus, the iterative sampling of xX" is required until t 0. To sample next step xX"", standard DDPM [25] computes the posterior mean 140 from current x!" and estimated x/et at step with:
Vor (1 āx-1) gst x
v 1 ae xx where 8
(x(x) 1gt .) = pt- tet 1st
0t-
1
Afterwards, XX_ can be sampled from Gaussian distribution with mean 14-1 and variance 3 1 (Eq. (2)) and used as the input for the next iteration. The reverse sampling is repeated until t = 0 where 4 clear target views are generated.
Although multi-view diffusion models [43, 60, 77] generate multiple views together, the 3D consis- tency across these views is not guaranteed due to the lack of explicit 3D representation. Thus, we propose a novel 3D consistent diffusion model, which ensures the multi-view consistency at each step of the reverse process by diffusing 2D images using reconstructed 3D Gaussian Splats [36].
3

---

(B) generation guided by prior
(C) 2D sampling trajectory
refinement with 3D representation
313 Gaussian
Photoreddioiic
Splats
2D Mulri-View Diffusion Model Fi
312 Generation Model Be
(A) Ingut
introducos:
Estimated %
Shape prior
Noisy multi-viow x
consistent renderings %
3D inconsistency
Reverse noise sumpling (D) 3D Gaussian Splats 5
3D Consistent Reverse Sumpling Step
Reverse sampling
Forward timing
X
Figure 2: Method Overview. Given a single RGB image (A), we sample a realistic 3D avatar represented 3D Gaussian Splats (D). At each reverse step, 3D generation model go leverages 2D mieti-view diffusion prior from 69 which provides a strong shape prior but is not 3D consistent (B, cf. Sec. 4.1). We then refine the 2D reverse sampling trajectory with generated 3D renderings that are guaranteed to be 3D consistent (C, cf. Sec, 4.2). Our tight coupling ensures 3D consistency at each sampling step and obtains high-quality 3D avatar (D).
4 3Diffusion
Overview. Given single RGB image, aim create realistic 3D avatar consistent with the input. We adopt an image-conditioned 3D generation paradigm due to inherent ambiguities in the monocular view. We introduce novel 3D Gaussian Splatting (3D-GS [36]) generative model that combines shape priors from 2D multi-view diffusion models with the explicit 3D-GS representation. This allows us jointly train our 3D generative model and 2D multi-view diffusion model end-to-end and improves the 3D consistency of 2D multi-view generation at inference time.
In this section, we first introduce our novel generative 3D-GS reconstruction model in Sec. 4.1. We then describe how we leverage the 3D reconstruction to generate consistent multi-view results by refining the reverse sampling trajectory (Sec. 4.2). An overview of our method can be found in Fig. 2.
4.1 Generative 3D-GS Reconstruction with Diffusson Priors
Given context image x we use conditional diffusion model to learn and sample from plausible 3D distribution. Previous works demonstrated that 3D generation can be done implicitly via diffusing rendered images of differentiable 3D represetation [7, 35, 70] such as NeRF [49, 97]. In this work, we introduce a novel generative model for 3D Gaussian Splatings [36], which diffuses rendered images of 3D-GS and enables sampling of 3D-GS inference time. Single image 3D generation is however very challenging, we hence propose to leverage 2D multi-view diffusion models in a tightly coupled manner which allows us train it end-to-end with novel 3D generative model.
Generative 3D-GS Reconstruction. In this work, propose 3D-GS generative model go- which is conditioned on input context image x to perform reconstruction of 3D Gaussian Splats G. Diffussng directly in the space of G parameters requires pre-computing Gaussian Splats from scans, which is exorbitant. Instead, we diffuse the multi-view renderings of G using a differentiable rendering function renderer.
We denote x) the ground truth images at target views to be diffused and x novel the additional

---

novel views for supervision. At training time, we uniformly sample timestep t ~ 4(0, T) and
add noise to x uu using Eq. (1) to obtain noisy target views x Our generative model Jø takes x," diffusion timestep and the conditional image x" as input, and estimates 3D Gaussians G:
go g t x,x",, where x - v 1st
v1 OeE, and I)(11)
(5)
We adopt an asymmetric U-Net Transformer proposed by [67] for to directly predict 3D-GS parameters from per-pixel features of the last U-Net layer. To supervise the generative model g we use differentiable rendering function renderer {G, ) + x" to render images at target views x and additional novel views π -_=vel_ Denoting Xo := {xo tgt xx as ground truth and xy = x xxmd) as rendered images, we compute the loss on images and generated 3D-GS:
,x0
Xx
gg X1 LMSE (xo: Xo) + X2 Lpercop (xo: xo) + A3 Leeg (Bx x), where Ro {x00 XD > renderer (x ,, x") {n , m)ve)))
(6)
here LSSE denotes the Mean Square Error (MSE) and Peeeep is the perceptual loss based on VGG- 19 [62]. We also apply Crgg a geometry regularizer [29, 100] to stabilize the generation of S.
With this, we can train a generative model that diffuses 3D-GS implicitly by diffusing 2D images x At inference time, we can generate 3D-GS given the imput image by denoising 2D multi-views sampled from Gaussian distribution. We initialize x from No,,,), and iteratively denoise the rendered images of predicted & from our model Jø- At each reverse step, our model do estimates a clean state 9 and render target images x We then calculate target images xX__ for the next step via Eq. (4) and repeat the process until 1 = 0. For more details, please refer to Appendix A.3
Our generative 3D-GS reconstruction model archives superior performance on in-distribution human reconstruction yet generalizes poorly to unseen categories such as general objects (Sec. 5.3 Fig. 5). Our key insight for better generalization is leveraging strong priors from pretrained 2D multi-view diffusion models for 3D-GS generation.
3D-GS Generation with 2D Multi-view Diffusiin. Pretrained 2D multi-view diffusion models (MVD) [43, 61, 77] have seen billions of real images [58] and millions of 3D data [13], which provide strong prior information and can generalize to unseen objects [67, 89]. Here, we propose a simple yet elegant idea for incorporating this multi-via -iew prior into our generative 3D-GS model ga. We can also leverage generated 3D-GS to guide 2D MVD sampling process which we discuss in Sec. 4.2.
Our key observation that both 2D MVD and our proposed 3D-GS generative model are diffusion- based and share the same sampling state x at timestep Thus, they are synchronized. This enables us to couple and facilitate information exchange between 2D MVD Eg and 3D-GS generative model Is at the same diffusion timestep f. To inject the 2D diffusion priors into 3D generation, we first compute one-step estimation of x (Eq. (3)) using 2D MVD cg, and condition our 3D-GS generative mode Io additionally on it. Formally, our 3D-GS generative model enhanced with 2D multi-view diffusion priors is written as:
ف = (x 12t l, 1,x", (,x%, ) where x
(x VI CrEa ,))
.gg The visualization of X along the whole sampling trajectory in Fig. 7 shows that the pretrained 2D diffusion model 6g can already provide useful multi-view shape prior even in large timestep / = 1000. This is further validated in our experiments where the additional 2D diffusion prior x leads to better avatar reconstruction (Tab. 4) as well as more robust generalization to general objects (Fig. 5). By utilizing the timewise iterative manner of 2D and 3D diffusion models, we can not only leverage 21) priors for 3D-GS generation but also train both models jointly end to end, which we discuss next.
Joint Training with 2D Model. We adopt pretrained ImageDream [77] as our 2D multi-view diffusion model and jointly train it with our 3D-GS generative model go- We observe that our joint training is important for coherent 3D generation, as opposed to prior works that frozen pretrained 2D multi-view models [67, 74). We summarize our training algorithm in Algorithm 1. We combine the loss of 2D diffusion and our 3D-GS generation loss c Eq. (6)):
Liotal LMSE(E, Ea) + Cgs
(8)
5

---

Algorithm Training
Algorithm 2 3D Consistent Sampling
Input: Dataset of posed multi-view images X. XD
Input: A context image x" and text y: Converged 2D diffusion model eu and 3D generative model go Output: A 3D Gaussian Avatar G of the 2D image x"
Jowel x neneel a context image x", text description y
Output: Optimized 2D multi-view diffusion model eg
and 3D-GS generative model Go
1: xf (,
1: repeat
2: for [ = T T..... do
{x] Xxfovel, xx,x) / ~ q( (x5 x xB x x.,))
X vire (xg
-
v1 (x M. 1))
3: 1~ Uniform(
- NO.1))
4: & go (x." x,x xx)))
4: x!"' vaixb 18t + vI are
5: X = renderer (9,
5: x = Vas (x] -I- 6x x", U, t))
6: I = Do (xt" xt,x, xx") // Enhance conditional
6: px-1 ,00 = - (011) x
T-oy x //
I-as
3D generation with 2D diffusion prior X from ca
Guide 2D sampling with 3D consistent renderings
7: {x0" x0) = renderer (g. (n x =)))))
7: x ~ (xm" A (x4" x B1)
8: Compute loss Liotal ( (Eq. (8))
8: end for
9: Gradient step to update ca, 9.
10: until converged
9: return
T 9 (xD X x =0)
Once trained, one sample a plausible 3D-GS avatar & conditioned the input image from the learned 3D distributions. However, we observe that the multi-view diffusion model co can still output inconsistent multi-views along the sampling trajectory (see Fig. 2). On the other hand, our 3D generator produces explicit 3D-GS which can be rendered as 3D consistent multi-views. Our second key idea is to use the 3D consistent renderings guide 2D sampling process for more 3D consistent mulit-view generation. We discuss this in Sec. 4.2.
4.2 Guide 2D Multi-view Sampling with Reconstructed 3D-GS
With the shared and synchronized sampling state x²" of 2D multi-view diffusion model cy and 3D-GS reconstruction model gø, we couple both models at arbitrary t during training. Similarly, they are also connected by both using estimated clean multi-views 1gt potential of both models, we carefully design a joint sampling process that utilizes the reconstructed 3D-GS & each timestep to guide 2D multi-view sampling, which is summarized in Algorithm 2. We observe that the key difference between the clean multi-views estimated x5 from 2D diffusion model and our 3D-GS generation lies in 3D consistency: 2D MVD computes multi-view X from 2D network prediction which can be 3D inconsistent while our X are rendered from explicit 3D-GS representation which are guaranteed to be 3D consistent. Our idea is to guide the 2D multi-view reverse sampling process with our 3D consistent renderings x such that the 2D sampling trajectory is more 3D consistent. Specifically, we leverage 3D consistent multi-view renderings to refine the posterior mean μa (x)" t) at each reverse step:
at sampling time. To leverage the full
Original:
po tot !) == 1x-1(xx) tgt x
+
Ours: (x(x 1) == p4 1 1st 1et
Var, (1 0z-1) 1) x, + /a 1gt
(9)
where tst = rendererrr " and (x 1ot
18t
ac
With this refinement, we guarantee the 3D consistency at each reverse step and avoid 3D incon- sistency accumulation in original multi-view sampling [77]. In Fig. 7, we visualize the evolution of originally generated multi-views x and milti-views rendering x from generated 3D-GS ς along the whole reverse sampling process. It intuitively shows how effective the sampling trajectory refinement is. We perform extensive ablation in Sec. 5.3 showing the importance of the consistent refinement for sampling trajectory.
5 Experiments
In this section, we first compare against baseline methods for human reconstruction Sec. 5.2 and then ablate our design choices in Sec. 5.3.

---

Input
Ours
SIFU
Ours
ICON
RCON
Figure 3: Qualitative comparison with baselines. Recent avatar reconstruction works ICON [87], ECON [88], SiTH [23] and SIFU (108]) cannot reconstruct loose clothing coherently. Additionally, SiTH and SIFU generate blurry texture in unseen regions due their deterministic formulation of regressing 3D avatar directly from single RGB imagse. In contract, our method is able to reconstruct avatars with realistic textures and plausible 3D geometry in both seen and unseen region.
5.1 Experimental Setup
Datasets. We train our model on a combined 3D human dataset [1, 3, 4, 2, 21, 27, 65, 98] compromis-
ing ~ 6000 high quality scans. We evaluate qualitatively and quantitatively on CAPE [47, 52, 103), Sizer [8. 72] and IIIT [32] dataset. Please refer Appendix D.1 and Appendix D.2 for more details.
Implementation Details. We trained our model on 8 NVIDIA A100 GPUs over approximately 5 days. Each GPU was configured with batch size 2 and gradient accumulations of 16 steps to achieve an effective batch size of 256. For more training details regarding hyperparameters, diffusion schedulers, etc., please refer to Appendix A.1 for more details. Our model creates 3D Avatar from single images in 22.6 seconds on A100 GPU and only consumes 11.7 GB VRAM, which allows the efficient large-scale avatar generation.
Evaluation Metries. We evaluate the geometry quality using Chamfer Distance (CD in em), Point- to-Surface Distance (P2S in cm), F-score [69] (w/ threshold of 0.01m), and Normal Consistency (NC) between the extracted mesh (Appendix A.4) and the groundtruth scan. Appearance quality is assessed by rendering the reconstructed avatar from 32 novel views with uniform azimuth and elevation angle. The metries for appearance reported include multi-scale Structure Similarity (SSIM) [78], Learned Perceptual Image Patch Similarity (LPIPS) [104], and Peak Signal to Noise Ratio (PSNR) between rendered and ground-trui truth views. Moreover, we report the Fréchet inception distance (FID) [22] which reflects the quality and realism of the unseen regions.
5.2 Realistic Avatar from Image
We compare our approach against prior methods for image-to-avee avaatar reconstruction, including template-based [18, 23, 87, 88, 56, 108], template-free [56] human reconstruction methods, as well as general image-to-3D methods [67, 74, 89]. To further assess performance, we also fine-tuned the state- of-the-art object reconstruction method L.GM [67] and deployed multi-view diffusion model [77] on our training data, denoted as LGM Quantitative evaluations reported in Tab. I demonstrate
7

---

Method
CD
P2S 4
F-score T
NC
SSIM T
LPIPS
PSNR 1
FID 4
PIFu [56]
2.75
2.68
0.359
0.778
0.909
0.077
21.06
29.57
SiTH [23]
4.00
4.00
0.257
0.749
0.907
0.073
20.00
22.33
SIFU [108]
3.50
3.50
0.273
0.760
0.0.900
0.081
20.73
40.75
haman
3.44
3.44
0.272
0.560
0.893
0.088
20.56
14.22
FoF [18]
5.43
5.34
0.183
0.683
ICON [87]
3.88
3.94
0.244
0.749
ECON [88]
2.83
3.61
0.291
0.767
Ours
1.41
1.37
0.557
0.791
0.916
0.058
21.61
8.45
Table 1: Quantitative evaluation on CAPE [47], SIZER [72], and IIIT [32] dataset. Our method can perform better reconstruction in terms of more accurate geometry (CD, P2S, F-score, NC) and realistic textures (SSIM, LPIPS, PSNR, FID).
Input
Ours
MVD
MVD
Figure 4: 3D reconstruction conditioned on different multi-view priors. Without our 3D-consistent sampling, the 2D diffusion model cannot generate 3D consistent multi-views (MVD, MVD,), leading to artifacts like floating 3D Gaussians splats.
that our proposed method excels in reconstructing realistic avatars with accurate geometry (CD, NC, F-score) and realistic texture (SSIM, LPIPS, PSNR, FID) from a single RGB image.
We present qualitative comparison examples in Fig. and Appendix B. highlighting the strengths and weaknesses of competing methods. Template-basee methods such SiTH [23] and SIFU [108] struggle to accurately reconstruct the geometry of loose clothing (as shown in row 4) due to their reliance the naked SMPL body shape. In contrast, template-free methods like PIFu [56] and TripoSR [74] offer greater flexibility and better performance on loose clothing. However, they are not generative models and their deterministic formulations lead blurry textures in unseen regions, as they tend to produce average textures rather than distinct details. Similar to our approach, LGM [67] and InstantMesh [89] utilize 2D diffusion models generate multi-view images and perform sparse- view 3D reconstruction. Nonetheless, their separation of 2D and 3D models cannot correct the 3D inconsistencies that may arise from the 2D models. Even further fine-tuning of LGM on human seans (Fig. 4) does not adequately address these challenges due to the complex and sensitive nature of human geometry and textures. In contrast, our conditional generative formulation and inherent 3D consistency by tightly coupling 2D-3D models allow us to obtain accurate reconstruction in front view and realistic generation in unseen regions. We also show the generative power our method in Appendix C.5: by sampling with different seed, we obtain diverse yet plausible reconstruction.
Please also refer Fig. 6, Appendix and our project page for additional reconstruction results on challenging subjects not previously observed, encompassing diverse range of appearances such as loose skirts and custom suits, as well as accessories like bags and gloves.
5.3 Ablation Studies
Importance of Trajectory Refinement. One of our key ideas is leveraging our explicit 3D model to refine
Method
LPIPS 1
SSIM
PSNR 1
MVD
0.078
0.911
22.32
the 2D multi-view reverse sampling trajectory, ensur- ing 3D consistency in Multi- View Diffusion (MVD) generation (see Sec, 4.2 and Eq. (9)). To evaluate this, we compare the multi-view images generated by pretrained MVD, fine-tuned MVD on our data (MVD;) and MVD with our 3D consistent sampling
MVD,
0.061
0.926
24.14
Ours
0.048
0.934
24.69
Table Evaluating trajectory refinement for 2D multi-view diffuusion. Our proposed refinement improves multi-view image quality.
8

---

(ours), as shown in Tab. 2. The results demonstrate that our proposed method effectively enhances the quality of generated multi-view images by leveraging the explicit 3D model to refine sampling trajectory. Additionally. we analyze the 3D reconstruction results with the multi-view images gener- ated by these models in Fig. MVD and MVD, produce inconsistent multi-view images, which typically lead to floating Gaussian and hence blurry boundaries. In contrast, our method can generate more consistent multi-views, result better 3D Gaussians Splats and sharper renderings.
We further quan- titatively evaluate the impact of our proposed sam- pling trajectory refinement on final 3D reconstruction in Tab. 3. We compare the reconstruction results of methods with and without our trajectory refinement while using the same 2D MVD and 3D reconstruction models. It can be clearly seen that our trajectory refinement improves the quality of 3D reconstruction.
Method
F-scoret
NC
LPIPS
SSIMT
PSNRT 21.09
Our w/o Traj. Ref. Oury
1.57
0.498
0.794
0.064
0.908
1.35
0,550
0.798
0.060
0.918
21.49
Table 3: Evaluating trajectory refinement for final 3D reconstruction. Our sampling trajectory refinement ensures multi viitw- consistency and hence yields better 3D results.
Importance of 2D Multi-view Prior. Another key idea of our work is the use of multi-view
priors x from 2D diffusion model pretrained on massive data [13, 55, 58] to enhance our
3D generative model. This additional prior information is pivotal for ensuring accurate recon-
struction of both in-distribution human dataset and generalizing to out-of-distribution objects.
We evaluate the performance of our 3D model P6 by comparing
Method
PSNR f
generation results with and without the 2D diffusion prior x (refer
Ours w/o X
20.98
to Eq. (7) and Eq. (5)). Notably, without the 2D multi-view condi- tioning, the alignment of the generated 3D model in the front view is not guaranteed due to the relative camera pose settings in our 3D generative model ge- Therefore, we evaluate the overall quality solely through the Fréchet Inception Distance (FID).
Our full model
21.49
Table
2D multi-view priors x improve human reconstruc- tion quality.
For avatars reconstruction, our powerful 3D reconstruction model can already achieve state-of-the-art performance. Morcover, full model with multi-view prior x. generates avatars with higher quality as demonstrated in Tab. 4. We further evaluate it on the GSO [17] dataset which consists of unseen general objects to model. The improvements are highlighting the challenges of generating coherent 3D structures from a single 2D image, particularly with unseen objects. For additional examples, please see Fig. 15 in Supp..
more pronounced in this setting,
Method
PSNR f
w/o X
14.45
Ours
16.12
Input
Ours
pure 3D generative wio 2D prior
Figure 5: 2D multi- viet priors x enhances generalization general objects in GSO [17] dataset.
6 Limitations and Future Work
Currently, method is constrained by the 256 x 256 resolution of the multi-view diffusion model, which restricts the sharpness of texture details (see Appendix E). Upgrading to more powerful high-resolution (512 512) multi-view diffusion models [19, 68] could potentially resolve these issues. Moreover, our approach may struggle in reconstructing subjects with challenging poses, as we further discussed in Appendix E. Synthesizing training data with challenging poses [10, 86] could be a potential solution.
Our method is general framework for image-to-3D reconstruction, which is applicable to various objects and compositional shapes like human-object interactions. We leave these to future works.
7 Conclusion
In this paper, we introduce Human-3Diffusion, 3D consistent diffusion model for creating realistic avatars from single RGB images. Our key ideas are two folds: 1) Leveraging strong multi-view priors

---

Iaput
Mesh
3D-GS
Input
Mesh
3D-G2 Rendering
(D)
(8)
(F)
Figure 6: Visualization of reconstructed mesh and synthesized novel view of generated 3D-GS on subjects from Sizer [72], RenderPeople [2], Twindom [4], UBC Fashion [101], GSO [17] and online image. More results are presented in Appendix C and our project page.
from pretrained 2D diffusion models to generate 3D Gaussian Splats, and 2) Using the reconstructed explicit 3D Gaussian Splats to refine the sampling trajectory of the 2D diffusion model which enhances 3D consistency. We carefully designed diffusion process that synergistically combines the strengths of both 2D and 3D models. Our experiments show that approach outperforms all previous reconstruction works in both appearance and geometry. We also extensively ablate our method which proves the effectiveness of our proposed ideas. Our code and pretrained models will be released on our Project Page to foster future research.
Acknowledgements We appreciate G. G.Tiwari, Y.He, Y. Xiu, Z.Liu, Z.Qiu, S.Li and others for their feedback to improve the work. This work is made possible by funding from the Carl Zeiss Foundation. This work is also funded by the Deutsche Forschungsgemeinschaff (DFG, German Research Foundation) 409792 180 (EmmyNoether Programme, project: Real Virtual Humans) and the German Federal Ministry of Education and Research (BMBF): Tübingen AI Center, FKZ: 01IS1 A. The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Y.Xue. R. Marin has been supported by innovation program under the Marie Sklodowska- grant agreement No 101 109330. G. Pons-Moll is member of the Machine Learning Cluster of Excellence, EXC number 2064/1 Project number 390727645.
10

---

References
[1] Axyz, Nov 2023. URL UK..atpps///eccnee..yy..e.e..p.. 7,34,
[2] Renderpeople, Nov 2023. URI. wtt.tttpss/reedddepppel...c 7, 10,
[3] Treedy, Nov 2023. URL https: 7.34
[4] Twindom, Nov 2023. URL https: www.wewm....iidmm...... com/ 7. 10, 34, 36
[5] Thiemo Alldieck, Mihai Zanfir, and Cristian Sminchisescu. Photorealistic Monocular 3D Reconstruction of Humans Wearing Clothing. In 2022 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 1496-1505, New Orleans, LA, USA, June 2022. IEEE. ISBN 978- 1-66546-946-3, doi: 10. (LL.L//CVPRS228..222200000 // //ieeexplore.c leee. org/ wttp///ccmmnttt88785/5e00
URL https:
[6] Kalyan Vasudev Alwala, Abhinav Gupta, and Shubham Tulsiani. Pre-train, self-train, distill: A simple recipe for supersizing 3d reconstruction. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 3773-3782, June 2022.
[7] Titas Anciukevicius, Zexiang Xu, Matthew Fisher, Paul Henderson, Hakan Bilen, Niloy J. Mitra, and Paul Guerrero. Renderdiffusion: Image diffusion for 3d reconstruction, inpainting and generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 12608-1200 12618. IEEE, 2023. doi: 10.1 N..119//CVPR327922223.0222 URL https:///////. //doi org .tt//10.11000 wtt.:1119/////111 2023.
01213. 21
[8] Dimitrije Antic, Garvita Tiwari, Batuhan Ozcomlekci, Riccardo Marin, and Gerard Pons-Moll. Close: A 3d clothing segmentation dataset and model. In International Conference on 3D Vision, 3DV 2024, Davos, Switzerland, March 18-21, 2024, pages 591-601. IEEE, 2024. doi: 10. 10.1100/3DV62453.2000 2022.0022222
URL wwt.h:tt///dite //tp:i///ooo.ee org/10../0 1199/3DV224233 2024.
00020. 7
[9] Bharat Lal Bhatnagar, Xianghui Xie, Ilya Petrov, Cristian Sminchisescu, Christian Theobalt, and Gerard Pons-Moll. Behave: Dataset and method for tracking human object interactions. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, jun 2022.
[10] Michael Black, Priyanka Patel, Joachim Tesch, and Jinlong Yang. BEDLAM: A synthetic dataset of bodies exhibiting detailed lifelike animated motion. Proceedings IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pages 8726-8737, June 2023. 9
[11] Xu Cao, Hiroaki Santo, Boxin Shi, Fumio Okura, and Yasuyuki Matsushita. Bilateral normal integration. 2022. 23
[12] Eric R. Chan, Connor 7. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J. Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, and Gordon Wetzstein. Efficient geometry-aware 3d generative adversarial networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24 2022, pages 16102-1611 12. IEEE, 2022. doi: 10. D0.100//CVP522288.2200 2T..522688..22.0156666 2022. 01565. 3
URL https: / /oi..ii.oo// http///ioooo/o//////o/...oooo 10.
[13] Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Osear Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani. Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC. Canada, June 17-24, 2023, pages 13142-13153. IEEE, 2023. doi: 10. R1110//CVPP2222..00 03.6..2........ URL https: /doi wwddoi..ogg111 10. 11111//////22222 2023.
01263. 2, 5,9,
[14] Maximilian Denninger, Dominik Winkelbauer, Martin Sundermeyer, Wout Boerdijk, Markus Knauer, Klaus H. Strobl, Matthias Humt, and Rudolph Triebel. Blenderproc2: A procedural pipeline for photorealistic rendering. Journal of Open Source Software, 8(82),401, 2023. doi: 10.2 10.21105/joss.0e 14901. URL pettps://oot org/ 10. 21105/ joss. 04901. 34

---

[15] Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In A. Beygelzimer, Y. Dauphin, P. Liang, and Wortman Vaughan, editors. Advances in Neural Information Processing Systems, 2021. URL. htttps://openreviee net/forum? htt./foruum?d-AAAWOOOVaZ...
[16] Zijian Dong, Chen Guo, Jie Song. Xu Chen, Andreas Geiger, and Otmar Hilliges. Pina: Learning a personalized implicit neural avatar from a single rgb-d video sequence. arXiv, 2022.
2
[17] Laura Downs, Anthony Francis, Nate Koenig, Brandon Kinman, Ryan Hickman, Krista Reymann, Thomas Barlow McHugh, and Vincent Vanhoucke. Google scanned objects: A high-quality dataset 3d scanned household items. In 2022 International Conference on Robotics and Automation, ICRA 2022, Philadelphia, PA, USA, May 23-27, 2022, pages 2553- 2560. IEEE, 2022. doi: 10.1 199/CCA44639, 20.......2222222 1809. URL https: /doi. /ttp::/:///oo..//11.... 1.09/ICRA466330 2022. 9811809. 10, 26, 31
[18] Qiao Feng, Yebin Liu, Yu-Kun Lai, ingyu Yang, and Kun Li. Fof: Learning fourier occupancy field for monocular real-time human reconstruction. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022. New Orleans. LA, USA, November 28- December 9, 2022, 2022. 7,8
[19] Ruiqi Gao Aleksander Holynski*, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualle Pratul Srinivasan, Jonathan T. Barron, and Ben Poole*. Cat3d: Create anything 3d with multi-view diffusion models. arXiv, 2024. 9,38
[20] Vladimir Guzov, Aymen Mir, Torsten Sattler, and Gerard Pons-Moll. Human poseitioning system (HPS): 3d human pose estimation and lelf-localization in large scenes from body- mounted sensors. CoRR, abs/2103.17265, 2021. URL https: ptttps://arxit /arxivv.... wariii.orrg/aae org/abs/2103. 17265. 2
[21] Sang-Hun Han, Min-Gyu Park, Ju Hong Yoon, Ju-Mi Kang, Young-Jae Park, and Hae- Gon Jeon. High-fidelity 3d human digitization from single 2k resolution images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR2023), June 2023. 7, 34, 36
[22] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two scale update rule converge to a local nash equilibrium. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vish- wanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 6626-6637, 2017. URL httpthhtt////eeeeee /proceeddgs///oo//s/g/ wtwpoceediinnsnnuuirp...... neurips. cc/paper http://////ae1177777700aaaa21110
htwp:/92-dd-----a-baeeeee Abstract html.
[23] Hsuan-I Ho, Jie Song, and Otmar Hilliges. Sith: Single-view textured human reconstruction with image-conditioned diffusion. CoRR, abs/2311.15855, 2023, doi: 10.48550/ARXIV.3111 15855. URL attps://don org/ 10. 48550/arXiv 2311. 15855. 2, 7, 8, 34,
[24] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. NeurIPS 202/ Workshop Deep Generative Models and Downstream Applications, 2021. URL https:
/ //openreview ntt/ffrumm iuu.ruuniIduuu
[25] Jonathan Ho, Ajay Jain, and Picter Abbeel. Denoising diffusion probabilistic models. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan- Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Con- ference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6- 12, 2020, virtual, 2020. URL https: :ttpps://proccodinnggs http://ppooooddinngnorropppppppp neurips. wttpeiiss.cccpppeee httppppppp////aa////p AG5oofee8B5441100001111110010110
Abstract html. 3,
[26] Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. LRM: large reconstruction model for single image to 3d. CORR, abs/2311 1.004/00, 2023. doi: 10.48550//RRIV.2211 1.04400. URI. https:
//doi. org...rg//1/ 48550/ arrii.iivvv..vi. 2311. 04400. 2.
12

---

[27] Ho Hsuan-I, Xue Lixin, Song Jie, and Hilliges Otmar. Learning locally editable virtual humans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 7,34,35 36
[28] Hanzhe Hu, Zhizhuo Zhou, Varun Jampani, and Shubham Tulsiani. MVD-Fusion: Single-view
3D via Depth Depth-consistent Multi-view Generation, April 2024. 3
[29] Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. CoRR. fbs//2033.17888, 2024. doi: 10.48550/ARXIV224003 17888. URL
48550/ arxiv 2403. 17888. 5
[30] Zehuan Huang. Hao Wen, Junting Dong, Yaohui Wang, Yangguang Li, Xinyuan Chen, Yan-Pei Cao, Ding Liang, Yu Qiao, Bo Dai, and Lu Sheng. EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constranl Constrained Diffusion, April 2024.
[31] Tianjian Jiang, Xu Chen, Jie Song, and Otmar Hilliges. Instantavatar: Learning avatars from monocular video in 60 seconds. arXiv, 2022. 2
[32] Sai Sagar Jinka, Astitva Srivastava, Chandradeep Pokhariya, Avinash Sharma, and P. J. Narayanan. SHARP: shape-aware reconstruction of people in loose clothing. Int. J. Comput. Vix., 131(4):918-937 2023. doi: 10.1007/S I 1263-022-01736-2. URL https:
//doi.org// 10. 100000000/116 011-111226-22221777110 022-01736- 7, 8, 24, 25, 34, 35, 37
[33] Berna Kabadayi, Wojciech Zielonka, Bharat Lal Bhatnagar, Gerard Pons-Moll, and Justus Thies. Gan-avatar: Controllable personalized gan-based human head avatar. In International Conference on 3D Vision (3DV), March 2024. 2
[34] Yash Kant, Ziyi Wu, Michael Vasilkovsky, Guocheng Qian, Jian Ren, Riza Alp Güler, Bernard Ghanem, Sergey Tulyakov, Igor Gilitschenski, and Aliaksandr Siarohin. SPAD Spatially aware multiview diffusers. CoRR, abs/2402.05235, 2024. doi: 1048550ARXII22402.2222 URL https://doi. org/10 /0//110.400 400.48550/ariiv 2402. 00235.. 3
[35] Animesh Karnewar, Andrea Vedaldi, David Novotny, and Niloy Mitra. Holodiffusion: Training 3D diffusion model using 2D images. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, 2023. 4, 21
[36] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühllr, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering, ACM Trans. Graph. 42(4): 139:1119:111 2023. doi: 10.1145/3592433, URL http::::///ooooo///////s
1111//111111///5233333 2, 3, 4, 21
[37] Xin Kong, Shikun Liu, Xiaoyang Lyu, Marwan Taher, Xiaojuan Qi, and Andrew J. Davison. Eschernet: generative model for scalable view synthesis. CORR, abs/2402.03908 2024. doi: (D44850/ARXXV..240000000
URL https: //doi. org/10. 48550/ 2402. 03908. 3
[38] Yifei Li, Hsiao-yu Chen, Egor Larionov, Nikolaos Sarafianos, Wojciech Matusik, and Tuur Stuyck. DiffAvatar: Simulation-ready garment optimization with differentiable simulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2024.7
[39] Tingting Liao, Hongwei Yi, Yuliang Xiu, Jiaxiang Tang. Yangyi Huang, Justus Thies, and Michael J. Black. Tada! text to animatable digital avatars. CoRR, abs/2308. 10899, 2023. doi: 10.4 (0.48550/ARXIV.2300 10899. URL //doi. org/10. arg/10.48550//an arriv 2308 10899. 3
[40] Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Jiayuan Gu, and Hao Su. One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion. CoRR, abs/231 1. 107885, 2023. doi: 10.48550/ARXIV.2311 117888851188.... URL https: //doi. org/ 10. 48550/ aarvvvv.v.vv.v. 2311. 1. 07885.
[41] Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimization. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS' 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL. http: http::/pppps:ss pwwp/sppprrre.eee wwappperrr.inis..eeeeeeeeeeeeeee nips cwwpapprrppppe..ecc. ppaper_filie wwwes/ppppppee hawpr//023//a// /haah/ 46630e66laa325555011111100101111 Abstraatbbbbtttttaaaaa Coneee-ooooeeeeeee wenfereene..o html. 2, 3
13

---

[42] Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov. Sergey Zakharov, and Carl Vondrick. Zero- Zero-I-to-3: Zero-shot one image to 3d object. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pages 9264- 9275. IEEE, 2023. doi: 10. N0.1109/CCCV5100702220.0099 1109/ 1109/ICCV51070 2023. 00853. 2,3
URL. https: /ttp://ooo.ooo....////
[43] Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang- Synedreamer: Generating multiview-consister images from a single-view image. CoRR, abs/2309.03453, 2023. doi: 10.4850/ARXIV.2300..65 033453. URL attps://dol org/
10. 48550/arXiv. 2309. 03453. 3,5
[44] Zhen Liu, Yao Feng, Yuliang Xiu, Weiyang Liu, Liam Paull, Michael J. Black, and Bernhard Schölkopf. Ghost on the shell: An expressive representation of general 3d shapes. arXiv preprint arXiv:2310.. 168 2023.
[45] Xiaoxiao Long, Yuan-Cher Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma. Song-Hai Zhang, Marc Habermann, Christian Theobalt, and Wenping Wang. Wonder3d: Single image 3d using cross-domain diffusion. CoRR, abs/2310. 15008. 2023, doi: 10. 48550/ARXIV.2310.155
URL https: //tp//doii..oo wtt.://..rr/....////////////// 48550/ arxiv. 2310 2310.15008. 3
[46] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black.
SMPL: skinned multi-person linear model. ACM Trans. Graph., 34(6):248:1-248) 16, 2015. doi: 10.1145/2816795.. 0777779992281111. URL https://doi. org/10 1145/2816796.28881111. 3
2818013. 2,
[47] Qianli Ma, Siyu Tang, Sergi Pujades, Gerard Pons-Moll, Anurag Ranjan, and Michael J. Black. Dressing 3d humans using a conditional mesh-vae-gan, CoRR, abs//1907.136/1 2019. URL http://arxiv. org/ abs/1907 13615. 8, 34, 35, 37
[48] Luke Melas-Kyriane Christian Rupprecht, Iro Laina, and Andrea Vedaldi. Realfusion: 360 reconstruction of any object from single image. In Arxiv, 2023.
[49] Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: representing scenes as neural radiance fields for view synthesis. Commun. ACM, 65(1):99-106 2022. doi: 10. 10.1145/3503250. URL https://don org/10.11111 3503250. 4, 21
[50] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3d hands, face, and body from a single image. CoRR, htss/1904.58666 2019. URL http:/ /ttp:://arxa /arxxivvv..v org/ 05866. 3
org/abs/////eeeoooeo..oootaaeere abs//b/////// abs/1904.
[51] Ilya A. Petrov, Riccardo Marin, Julian Chibane, and Gerard Pons-Moll. Object pop-up: Can infer 3d objects and their poses from human interactions alone? In /EEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 4726-4736. IEEE, 2023. doi: 10. 1AL11//VVPP279922220000222 2wt.09//VV22222999 2023. 00458. 2
URL
https: http://oo..orrrogg
[52] Gerard Pons-Moll, Sergi Pujades, Sonny Hu, and Michael Black. Clothcap: seamless
4d clothing capture and retargeting. ACM Trans. Graph., 36(4))33:--3:15, 2017. doi: 10. 00.14457772959.07777711 URL https://dool //to:i////oiiee/ wtg//0.0///0..... 1111111111///111955/ w0w.777299999777111...1 3073711. 7,34
[53] Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv, 2022.
[54] Guocheng Qian, Jinjie Mai, Abdullah Hamdi, Jian Ren, Aliaksandr Siarohin, Bing Li, Hsin- Ying Lee, Ivan Skorokhodov, Peter Wonka, Sergey Tulyakov, and Bernard Ghanem. Magic123 One image to high-quality 3d object generation using both 2d and 3d diffusion priors. CoRR, abs/2306. 17843, 2023. doi: 10.48550/ARXIV.23066 17843. URL. petts:://dot https:///ooooo.......//// 48550/ arXiv. 2306. 17843. 2
14

---

[55] Robin Rombach, Andreas Blattmann. Dominik Lorenz, Patrick Esser, and Björ Ommer. High-resolution image synthesis with latent diffusion models. IEEE/CVF Conference on Computer Vision and Puttern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 10674-10685. IEEE, 2022. doi: 10. https: http::::///oo.rg////////oooo orgg 10. . / COO.9//222228..... 2022. 01042. 3,
20..22268.002.01042... URL.
[56] Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Hao Li, and Angjoo Kanazawa. Pifu: Pixel-aligned implicit function for high-resolution elothed human digitization. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 November 2. 2019, pages 2314- IEEE, 2019. doi: 10.1 109/CC. 2019.00239. URL htt.thtts://deee org/10 10. 1109/ICCV 2019. 2019.00239. 2, 7, 8, 33
[57] Shunsuke Saito, Tomas Simon, Jason M. Saragih, and Hanbyul Joo. Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitiza-
tion. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recogni-
tion, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 81-90. Computer
Vision Foundation / IEEE, 2020. doi: (0.1109/VPP4226..220.0000 ccontent_vPP.....c http///222/hhhmeaaee
URL
http//htll//tt htttt//ttlta/at/e////// _arratttoPPFuHD... https://openaccess. thecvf auplicit_Functionn corrictiooooo__ooo Batr__Hghheet gigh-Resolution_33 Inlti--Leee_PPxeel- Human_Digitizations CVPR_ 2020_paperr ptpp.p..te...................... html. 2, 23, 33
[58) Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wight- man, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. LAION-5B: an open large-scale dataset for training next generation image-text models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Sys- tems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022. URL http://papers mips. pocppaperffiles fileffiee///eee wiiles/paapei wtt.:/ffeesaeeeeeeeeeeeee ppppppppp///maaapappppppppppppap 115500011b690004455440bbeebbe
Abstract-Dataseeeee and
Benchmarks BaeeettsaandBBechaaareeeeee
html.
3,5,9 9
[59] Akash Sengupta, Thiemo Alldieck, Nikos Kolotouros, Enric Corona, Andrei Zanfir, and Cristian Sminchisescu. DiffHHuma: Probabilistic Photorealistic 3D Reconstruction of Humans, March 2024. URL http::tttt:/aaae http::::/aaxvvv.oooeeeme abs//4/////444/ 00485, arXiv:2404.00485 [cs]. 2
[60] Ruoxi Shi. Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zerol23++: a single image to consistent multi-view diffusion base model. CoRR, abx/2310.15110, 2023. doi: 10.48550/ARXIV.23100555 10. URL https:
//doi.org/11 10. 48550/arXiv 2310. 15110. 3,
[61] Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi- view diffusion for 3d generation. CoRR, abs/2308. 16512, 2023. doi: 10.48555/AXXV.2330.. 16512. URL https: / //w.i org/10. 48550/arXiv 2308. 16512. 3,5
[62] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http::////// /tap:p////a/vv..aaaeeeaaeaee.aaa org/ abs/ 1409 1556.
[63] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In 9th International Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL id=St1giarCHLP. 21
http://ppnrrviii.nttteeeee
[64] Yang Song. Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equations, February 2021. http: htttt://ra/iiii org/ ab//aaas//111 2011. 13456. arXiv:201 1.2011...44. 13456 [cs, stat].
[65] Zhaoqi Su, Tao Yu, Yangang Wang, and Yebin Liu. Deepcloth: Neural garment representation for shape and style editing. IEEE Transactions on Puttern Analysis and Machine Intelligence, 45(2):1581-1593, 2023. doi: 10. 20.10099/AAMM.222233555555 7, 34, 36
15

---

[66] Mohammed Suhail, Carlos Esteves. Leonid Sigal, and Ameesh Makadia. Generalizable patch- based neural rendering. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXII, volume I 13692 of Lecture Notes in Computer Science, pages 156-174, Springer, 2022. doi: URL httpp::/dti..... org/ 10. 1970/977-7100 3 1-----31--9-----1 1_000-31112
[67] Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. LGM: large multi-view gaussian model for high-resolution 3d content creation. CoRR, 0as/2402.05054, 2024. doi: 10.48550/RRXI..442.66555
URL https://doi.on.
48550/arXiv 05054. 2, 3, 7, 8,
[68] Shitao Tang, Jiacheng Chen, Dilin Wang. Chengzhou Tang, Fuyang Zhang, Yuchen Fan, Vikas Chandra, Yasutaka Furukawa, and Rakesh Ranjan. Mydiffusion+++: A dense high- resolution multi-view diffusion model for single sparse-view 3d object reconstruction. CoRR, abs/2402. 2024. doi: 1044550/AXXX:2442227777 URL https://doi. org/ 10. 48555000riii..... 2272. 3, 38
[69] Maxim Tatarchenko, Stephan R. Richter, René Ranftl, Zhuwen Li, Vladlen Koltun, and Thomas Brox. What do single-view 3d reconstruction networks learn? In IEEE Conference on Computer Vision and Puttern Recognition, CVPR 2019, Long Beach, CA. USA, June 16-20, 2019, pages 3405-3414. Com- puter Vision Foundation / IEEE, 2019. doi: 10.11009CVPR.2209.0000 http:///penaccees..u thecvf. com/ Do Do_Single-Viee Vieggle--iee___a 3D
URL _hhachhhhhhhhhhh_____h_h html. 7
[70] Ayush Tewari, Tianwei Yin, George Cazenavette, Semon Rezchikov, Josh Tenenbaum, Frédo Durand, Bill Freeman, and Vincent Sitzmann. Diffusion with forward models: Solving stochastic inverse problems without direct supervision. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Ad- vances in Neural Information Processing Systems 36: Annual Conference on Neural In- formation Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL http::ttt:/ppppeee http:://pppppr.........pp wwtpccpppppppeeeeeeeeeee silessppppet (eopprr/223////e 2023/hash/ - Abstract-tt---eeee Conference Abttraactt----aeeeeceeeee html. 4, 21
[71] Anh Thai, Stefan Stojanov, Vijay Upadhya, and James M. Rehg. 3d reconstruction of novel object shapes from single images, 2020. 3
[72] Garvita Tiwari, Bharat Lal Bhatnagar, Tony Tung, and Gerard Pons-Moll. SIZER: A dataset and model for parsing 3d clothing and learning size sensitive 3d clothing. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer Vision ECCV 2020 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III, volume 12348 of Lecture Notes in Computer Science, pages 1-18. Springer, 2020. doi: 1007/ URL htt.h::t///oo. org/ / 10. 10070007/9900 0-1-8----300-88-0---------- 7, 8, 10, 24, 25, 34, 35,
3--030-58580---00
[73] Garvita Tiwari, Nikolaos Sarafianos, Tony Tung, and Gerard Pons-Moll. Neural-gif: Neu- ral generalized implicit functions for animating people in clothing. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 1 1688-11698. IEEF, 2021, doi: 10. 109//CCV48922.2001 1.0000......... URL https://doi.orge
1w1111100///19220 2021. 01150. 2
[74] Dmitry Tochilkin, David Pankratz, ZeXiang Liu, Zixuan Huang, Adam Letts, Yangguang
Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image. CoRR, abs/2403.0211 2024. doi: 10.48550/ARXII. 2403.02151. URL https::/ //ooi org/10. 10..4.0. 48550/ arXiv arx.v.2403.......... 02151. 2, 3, 5, 7, 8, 33
[75] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V.N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA,
16

---

pages 2017. URL https::/prooeeddingg www.eeeiddinnnn.nni....... 3ttee244776631110051448888a----3 httpa//cccceaaaaaaaaaatttttettaa html.
[76] Vikram Voleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion, March 2024. 3
[77] Peng Wang and Yichun Shi. Imagedream: Image-prompt multi-view diffusion for 3d generation. CoRR, abs/2312.02201,, 2023. doi: 10.48550AARXXV233222022
URL
48550/85555. 02201. 2, 3, 5, 6, 38
[78] Z. Wang, E.P. Simoncelli, and A.C. Bovik. Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, volume 2. pages 1398-1402 Vol.2, 2003. doi: 10.1 ALL.://ACSSC.200.122222222
7
[79] Zhengyi Wang, Cheng Lu, Yikai Wang. Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lificdreamer: High-fidele fidelity and diverse text-to-3d generation with variational score distillation. arXiv preprint arXivv2335. 2023. 3
[80] Chung-Yi Yi Weng, Brian Curless, Pratul P. Srinivasan, Jonathan T. Barron, and Ira Kemelmacher- Shlizerman. HumanNeRF: Free-viewpoint rendering of moving people from monocular video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16210-16220, June 2022. 2
[81] Jiajun Wu, Chengkai Zhang, Xiuming Zhang. Zhoutong Zhang, William T Freeman, and Joshua B Tenenbaum. Learning 3D Shape Priors for Shape Completion and Reconstruction. In European Conference on Computer Vision (ECCV), 2018.
[82] Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang. Jiawei Ren, Liang Pan. Wayne Wu, Lei Yang. Jiaqi Wang, Chen Qian, Dahua Lin, and Ziwei Liu. Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 803-814 IEEE, 2023. doi: 10.1 2TCCV/PR27722222.00000 https: //tp::///do...o http://///io.o//1..... orw..0.0...00 10w./119///CC /1L.9/////222999... 2020.00084. 2, 3,
URL
[83] Yongqin Xian, Julian Chibane, Bharat Lal Bhatnagar, Bernt Schiele, Zeynep Akata, and Gerard Pons-Moll. Any-shot gin: Generalizing implicit networks for reconstructing novel classes, In 2022 International Conference on 3D Vision (3DV). IEBE, 2022.
[84] Xianghui Xie, Bharat Lal Bhatnagar, and Gerard Pons-Moll. CHORE: contact, human and object reconstruction from a single RGB image. CoRR, abs/2204.02445,, 2022. doi: 10. 10.48550/ARXII.2204444 02445. URL https://o//i. htt:///oooorgg//////// 11/11...8/5//aiiii 2204. 02445.
[85] Xianghui Xie, Bharat Lal Bhatnagar, and Gerard Pons-Moll. Visibility aware human-object interaction tracking from single RGB camera. In IEFF/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC. Canada, June 17-24, 2023, pages 4757-4768. IEEE, 2023. doi: 10. (19////P55229..223..000 10. 10.1109/CVPR5271 2023.00461. 2
URL patttps://olt org/
[86] Xianghui Xie, Bharat Lal Bhatnagar, Jan Eric Lenssen, and Gerard Pons-Moll. Template free reconstruction of human-object interaction with procedural interaction generation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2024. 2, 9, 26, 32
[87] Yuliang Xiu, Jinlong Yang, Dimitrios Tzionas, and Michael J. Black. ICON: implicit clothed humans obtained from normals. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 1228-133200 2022000222222 URL https: /doi org/ /0/ 1109/
IEEE, 2022. doi: 10. N0.1109/CVPR522888.... CVPR52688 2022 01294. 2, 7, 8, 34, 37,
[88] Yuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas, and Michael J. Black. ECON: explicit clothed humans optimized via normal integration. In IEEE/CVE Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 512-523. IEEE, 2023. doi: 10.1 19.1CS/22222 20212222292222.....22222222 0.0500 URL. https: //doi org/ 10. (199CCPP52729 2023. 00057. 2, 7, 8, 34, 35
17

---

[89] Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. In- stantmesh: Efficient 3d mesh generation from a single image with sparse-view large recon- struction models. arXiv preprini arXiv:240..07191, 2024. 3, 5, 7,8
[90] Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang. Jiahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, and Kai Zhang. DMV3D: denoising multi-view diffusion using 3d large reconstruction model. CoRR, abs/2311.09217,, 2023. doi: 10.48550/ARXIV. 2311 1.111.9 URL pattps://don
org/10 10. 48550/ arXiv 2311 .0w.99921... 3,
[91] Yinghao Xu, Zifan Shi, Wang Yifan, Sida Peng, Ceyuan Yang, Yujun Shen, and Wetzstein Gordon. Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation. arxiv: 2403. 14621, 2024. 3
[92] Yuxuan Xue, Haolong Li, Stefan Leutenegger, and Joerg Stueckler. Event-based non-rigid reconstruction from contours. In 33rd British Machine Vision Conference 2022, BMVC 2022, London, UK, November 21-24, 2022, page 78. BMVA Press, 2022. URL https:
/ //bpvc2022...mm mpi- inf mpg degg.de/7//22
[93] Yuxuan Xue, Bharat Lal Bhatnagar, Riccardo Marin, Nikolaos Sarafianos, Yuanlu Xu, Gerard Pons-Moll, and Tony Tung. NSF: neural surface fields for human modeling from monocular depth. In IEEE/CVE International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pages 55004-15014 IEEE, 2023. doi: 10. 109//CCV51000..223..111 URL. htRps:/::://ee /tttpp::://oi... .rg/ 10. 11111110///11//// 2023 01382.
[94] Yuxuan Xue, Haolong Li, Stefan Leutenegger, and Jörg Stückler. Event-based non-rigid reconstruction of low-rank parametrized deformations from contours. In International Journal of Computer Vision (ICCV). Springer Science and Business Media LLC, February 2024. doi: 10. 1007/ss1123-002-0220 1-z. URL http:///eet /tpp//xxdoot//////// doi. org/10. 10. 1007 122111223----2000 2
[95] Xueting Yang, Yihao Luo, Yuliang Xiu, Wei Wang, Hao Xu, and Zhaoxin Fan. D-IF: Uncertainty-aware Human Digitization via Implicit Distribution Field. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9088-9098, Paris, France, Oc- tober 2023. IEEE, ISBN 9798350307184, https: http://eeeeeeepeeeeee ieee. /tt.//enttt1777794/////////
doi: 10.1 10011000CCV11000022..00333 URL
[96] Kim Youwang, Tae-Hyun Oh, and Gerard Pons-Moll. Paint-it: Text-to-texture synthesis via deep convolutional texture map optimization and physically-based rendering, In IEEE Conference on Computer Vision and Puttern Recognition (CVPR), 2024. 2
[97] Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one or few images. In IEEE Conference on Computer Vision and Patter Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 4578-4587. P0B.CCCVRR64432220..0000
Computer Vision Foundation IEEE, 2021. doi: URL https::/openaccees.. theeeessssstteeeeeeeeeeeeee thecvf com/ htt///onnen////P2222222 Neural_Radiance, 21
htt//////////////aa /au_//u_p___e pi_eeeeeeppeeFF_ One or Fer__Fe__rree Iww._emmeee__________e 2aiia__21___p_____e 2021_paaeer__p__e1e html. 4,
[98] Tao Yu, Zerong Zheng, Kaiwen Guo, Pengpeng Liu, Qionghai Dai, and Yebin Liu. Function4d: Real-time human volumetric capture from very sparse consumer rgbd sensors. In IEEE Conference on Computer Vision and Puttern Recognition (CVPR202J), June 2021. 34, 36
[99] Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Chenming Zhu, Zhangyang Xiong, Tianyou Liang, Guanying Chen, Shuguang Cui, and Xiaoguang Han. Mvimgnet: A large-scale dataset of multi-view images. In IEEÉ/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 9150-9161. IEEE, 2023. doi: ML.110//CVPPS222..22222 URI. https::tt/// / doi. org...rg//o. 12/g/10.00 10. 1109 CVVR0222VPP 2023. 00883. 2, 2, 3,3.4 34
[100] Zehao Yu, Torsten Sattler, and Andreas Geiger. Gaussian opacity fields: Efficient high-quality compact surface reconstruction in unbounded scenes. arX1v:2404.1077,, 2024. 5, 21, 23
18

---

[101] Polina Zablotskaia, Aliaksandr Siarohin, Bo Zhao, and Leonid Sigal. Dwnet: Dense warp- based network for pose-guided human video generation. In 30th British Machine Vision Conference 2019, BMVC 2019, Cardiff. UK, September 9-12, 2019, page 51. BMVA Press, 2019. URL. Rol.hhttss:/bmvc22000 http://w.............eaaae wttp/wp-content//plloooe hwwpppppaddppppp....ppp papers/ hapers/10399 1039- 1 pdf. 10, 26, 30
[102] Andy Zeng, Shuran Song. Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser. 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In CVPR, 2017. 21,
[103] Chao Zhang. Sergi Pujades, Michael J. Black, and Gerard Pons-Moll. Detailed, accurate, human shape estimation from clothed 3d scan sequences. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 5484-5493. IEEE Computer Society, 2017. doi: 10.1109/CVPR.2017...22 URL 34
https://doi. http://ooo.or/////// org/10. 10. 1ttp//10.111///0011 1109/CVPR. 2017 582.
[104] Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as perceptual metric. In 2018 IEEE Conference on Computer
Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 586-595. Computer Vision Foundation / IEER Computer Society, 2018. doi: URL http::/ thecvf. com/ content_cvpr papee html. 7
10.1 2ttp://CCCR..28...000000 2018/htm1 httph/ttmm////ag.///// The swrreaannaleeBfffettiveeessssss
[105] Xiaohan Zhang, Bharat Lal Bhatnagar, Sebastian Starke, Vladimir Guzov, and Gerard Pons- Moll. COUCH: towards controllable human-chair interactions. In Shai Avidan, Gabriel J. Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision ECCV 2022 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part V. volume 13665 of Lecture Notes Computer Science, pages 518-535, URL https:: //tp:///iii... org/ 10. 1007/
Springer, 2022. doi: 10. 00.1007/978-3-01--2006----- 001-3-331-2006653 30. 2
[106] Xiaohan Zhang, Bharat Lal Bhatnagar, Sebastian Starke, Ilya Petrov, Vladimir Guzov, Helisa Dhamo, Eduardo Pérez-Pellitero, and Gerard Pons-Moll. FORCE: dataset and method for intuitive physics guided human-object interaction. CoRR, abs/2403. 11237,, 11237, 2024. doi: 10. 48550/ARXIV:2403. 1237. URL https: //t////oii/i//// 100/10.0//// 48550/arXiv. 2403 11237. 2
[107] Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Joshua B Tenenbaum, William T Freeman, and Jiajun Wu. Learning to Reconstruct Shapes From Unseen Classes. In Advances Neural Information Processing Systems (NeurIPS), 2018.
[108) Zechuan Zhang, Zongxin Yang, and Yi Yang. SIFU: side-view conditioned implicit function for real-world usable clothed human reconstruction. CORR, abs/2312.06704 2023. doi: 00.48550/ARXI.2312222
URL https://doi.
arrvv.....v.v. 2312. 2312.06704.
2, 7, 8, 33, 34, 35
[109] Zhizhuo Zhou and Shubham Tulsiani. Sparsefusion: Distilling view-conditioned diffusion for 3d reconstruction. In CVPR, 2023.
[110] Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, and Song-Hai Hai Zhang. Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers. CORR, abs/2312.09147, 2023, doi: 1044855/ARRXIV2312.. 09147. URL https: //ooi//oi///// http:///dooooo....eee/ 41r5/10.48550/400 http 09147. 2,
19

---

Appendix
Table of Contents
A Implementation Details
21
A.1 Training Details
21
A.2 Joint Framework
21
A.3 Generative 3D-GS Reconstruction Model
A.4
Textured Mesh Extraction
23
B
Comparison
24
B.I Qualitative Comparison
24
C More Qualitative Results
26
C.I In-the-wild Data
................................
26
C.2 UBC Fashion Dataset
................................
30
C.3 Google Scan Objects (GSO)
31
C.4 Human Object Interaction
32
C.5
Generative Power in Reconstruction
33
D Datasct Overview
34
D.I Training Dataset
34
D.2 Evaluation Dataset
Failure Cases
F Broader Impacts
38
20

---

A Implementation Details
A.1 Training Details
As described in Sec. 5.1, we use an effective batch size 256. Each batch involved sampling orthogonal images with zero elevation angle as target views x and 12 additional images as novel views xf to supervise the 3D generative model Eq. (8). The hyperparameters for training Eq. (8)
were set as follows: X = 1.0, 22 1.0, and X = 100.0.
During training, we employed the standard DDPM scheduler [25] to construct noisy target images x²" The maximum diffusion step T is set to 1000. At inference time, we use DDIM scheduler [63] to perform faster reverse sampling. The reverse steps set to in following experiments. The text prompt y used in our multi-view diffusion model(Fq. (3)) is set to "Photorealistic 3D human" for both training and inference across subjects.
A.2 Joint Framework
Implementation. Our 2D multi-view diffusion model 69 is latent diffusion model [77]. Thus, we use the frozen VAE in [55] obtain input x in image space for the 3D generative model go and encode refined xe1 back latent space for Eg. We extract triangle mesh from predicted Gaussian splats using Gaussian Opacity field [100] and TSDF [102]. Please refer to Appendix A.4 for more details.
A.3 Generative 3D-GS Reconstruction Model
In this section, we provide details about our 3D generative model gg in Eq. (5) and Eq. (7) as well as the renderer(o). Following [7, 35, 70], we learn the 3D generative model by adding and removing noise on the rendered 2D images from a 3D representation. A pseudo algorithm of the training and sampling process of our 3D generative model can be found in Algorithm 3.
Since we integrate both function into the reverse sampling process iteratively (eq. 2), we expect them be efficient and fast to execute. Tewari et al. [70] base their model on pixelNeRF [97]. which is generalizable NeRF [49] conditioned on a context view image. We adopt 3D Gaussian Splats [36] 3D state representation due to its efficiency and simplicity. Our renderer(o) is the differentiable rasteraizer accelerated and implemented in CUDA, which achieves around 2700 times faster rendering than volume-rendering- based renderer(o) in [49, 70, 97].
For sampling the 3D State from x x.)' x", and (eq. 7), we adopt the condiiondiiooned UNet- Transformer architecture [55] due to the efficiency of convolutional layers and the scalability of transformers. For enabling the awareness of camera poses in the encoding process, we concatenate the Plücker Camera Ray Embedding (o, x d,, ] [67, 90] with the image x!_ and x To enhance the control ability of context view in the 3D generation process, we additionally concatenate the clear context view x" with target images x! following [77]. This operation enables 3D dense self-attention process between the input multi-view target images and the clear context view image, provides pixel-level local conditional signal. Since the camera pose of context view x" is unknown, we use the O-vector as its embedding.
Algorithm 3 Learn 3D distribution
Algorithm 4 Sample from 3D distribution
Input: Dataset of posed multi-view images x 18t
Input: A context image x"; Converged 3D diffusion
xb novel nt nowe a context image x"
model Bo
Output: Optimized 3D State diffusion network de
Output: A 3T Gaussian Avatar of the 2D image x"
I: repeat
1: x ~ ~N(00
{xe]" xu novel x.y3 ~ q( q((xx) x govel x² x²,)))
2: for T...., 1 do
3: Uniform({1. (tl.., "il...,,,")eee. jiee TT)eeeN((...
3: g = 9o (x, x
4: x!" = V +V1 ā,E
4: * renderer (G,
5: Ć= xx" .t)
xxx
5: (x-1(x in 2l0))
(ar -Br1)
6: {x1 x = renderer (G. (or ( )))eel)))
x
I-
x N (x"= fia (x." xxx5 BB)
7: Compute loss Zy- Eq. (6))
8: Gradient step to update ge
7: end for
until converged
8: return & = (x x" x
21

---

Figure 7: Visualization of intermediate sampling steps from Gaussian Noise (t 1000) to the last denoising step (l 0). From top to bottom: current state x lel estimated clear view by 2D diffusion models 0 and corrected clear view by generated 3D Gaussian Splatting x Our 2D diffusion model (o) already provides strong multi-vier prior at am early stage with large t. Our 3D
reconstruction model go (o) can correct the inconsistency in X illustrated red circle.
+
timestep
22

---

A.4 Textured Mesh Extraction
Gaussian Opacity Fields [100] enables extraction triangle meshes from an existing 3D Gaussian Splatting. However, because the location of 3D- Gaussian Splats is not necessary to be on the real surface, we observe that the extracted meshes as well as the rendered depth maps are noisy. Since our method generate realistic RGB images, we use PiFU-HI [57] to estimate the normals and use Bilateral Normal Integration (BiNT) [11] to refine the noisy rendered depth with the estimated normal. As we only want the estimated normal to denoise the rendered depth map instead of modifying geometry, we set up the hyperparameter in BiNI with
x 10 Such large number ensures that the normal map not used to modify the geometry just regularize the depth map.
Assuming we have generated 3D-Gaussian Splats from go()) and n. camera views π ,²² ....T", we obtain paris of posed RGB-D images by Gaussian Splatring, Normal Estimation, and Bilateral Normal Integration. Finally, we perform volumetric TSDF fusion [102] to obtain high quality textured mesh from pairs RGB-D images. Given generated 3D-GS G. we set up 36 views to obtain the refined RGB-D image pairs. The rendering view of each camera can calculated as:
1
elevation;
+
(10)
4
36'
azimuth; = 0 + 3x *
(11)
36
23

---

B Comparison
B.1
Qualitative Comparison
Input
Ours
SIFU
TripoSR
LGM
InstantMesh
Figure Qualitative comparison on Sizer [72] and IIIT [32].
24

---

Input
Ours
SITH
PIFu
SIFU
TripoSR
LGM
InstantMesh
Figure Qualitative comparison on Sizer [72] and IIIT [32].
25

---

C More Qualitative Results
In this section, we show more qualitative results in-the-wild data, UBC fashion dataset [101],
GSO dataset [17], and human-object interaction data [86].
C.1 In-the-wild Data
Figure 10: Qualitative results on unseen data during training. Input image in left column. Our method successfully reconstructs different degree of loose clothing.
26

---

Figure 11: Qualitative results on more unseen data during training. Input image is in left column. Our method successfully reconstructs different types of clothing, including casual, sport, suits, custom, etc., in both appearance and geometry.
27

---

VEBITRO
Figure 12: Qualitative results on more unseen data during training. Input image is in left column. Our method successfully reconstructs clothing and interacting objects (racket and bag here) both appearance and geometry.
28

---

Figure 13: Qualitative results on more unseen data during training. Input image in left column.
Our method successfully reconstructs rarely seen suits and objects, in both appearance and geometry.
29

---

C.2 UBC Fashion Dataset
In this section, we show qualitative result our model on UBC fashion [101] dataset. The input images are the first frame extracted from each video in the dataset.
Figure 14: Qualitative results on UBC fashion [101] dataset. Results demonstrate that our model generalizes well to real world images in both geometry and appearance.
30

---

C.3
Google Scan Objects (GSO)
Ours
Input:
Racoen
3D generative model wio 2D prior &
Input:
Dog
3D generative model wio 2D prior &
Ours
Nickelodiem Teemuge Muhml Ninja Turtles Raphael
3D generative model wio 2D prior 2
Inpul:
Guardians of the Galaxy Galactic Battlers Rocket Raccoon Figure
generative prior tet
Figure 15: Ablation study: benefit of 2D multi-view prior X in 3D generation. The 2D prior from 2D diffusion model is essntial for generalization on general objects dataset GSO [17].
31

---

C.4 Human Object Interaction
Figure 16: Qualitative results of Human-Object Interaction reconstruction online stock images. Results show that our model is able to generalize casual human-ohject-interacee
Figure 17: Qualitative results of Human- Human-Object Interaction reconstruction on ProciGen [86] dataset. Results demonstrate that our model can reconstruct some simple Human-Ohe Sawasn--Oheeeteeeaeeeee images with large objects.
32

---

C.5 Generative Power in Reconstruction
Our model learns conditional distribution of the 3D representation given 2D context image. Thus, by sampling from the distribution with different seed. we obtain diverse yet plausible 3D representation. As illustrated in Fig. 18, the appearance of the occluded region (back side of subject) is different with different sampling in hair styple, texture, and cloth wrinkles.
The generative power of approach is the key generate clear self-occluded regions, which is impossible by non-generative reconstruction methods [56, 57, 74, 108). As shown Fig. 3 and Fig. 8, non-generative approaches tend to generate blurry self-oecluded results because they cannot sample from distribution but only regress to mean value of the training datasets.
Input
Seed
Seed
Seed 3
Figure 18: Our model learns 3D distribution. By different sampling from the learned distribution, we obtain diverse yet plausible 3D representations. The generative power is key generate clear self- occluded region, which is impossible in non-generative reconstruction approaches [56, 57, 74, 108].
33

---

D Dataset Overview
To ensure robust performance and generalization, we train our model on combined dataset compris- ing 3520 scans from publicly available datasets [21, 27, 65, 98] and 2320 scans from commercial 3D human datasets [1, 3, 4, 2]. These datasets encompass diverse range of body shapes, genders, ages, clothing, accessories, and interacting objects. For further details and examples of our training datasets, please refer to section D.I. All 3D scans are rendered into RGB-A images using BlenderProe [14] along a spiral path as described in Eq. (13).
For evaluation, we note that the commonly-used CAPE dataset [47, 52, 103] in previous works [23, 87, 88, 108] often contains artifacts in scans, such as holes, and not all 3D scans are fully publicly available. To effectively and fairly evaluate performance,
propose using Sizer [72] and IllT-Human [32] datasets, from which we randomly sample 150 scans each for evaluation. While Sizer [72] provides scans with normal human appearance similar our training datasets, IIIT- Human [32] can be considered as out-of-distribution (o.o.d.) evaluation dataset due to its inclusion of unseen clothing types, such traditional Indian suits. For more additional examples and analysis,
please see Fig. 3 and Appendix D.2.
D.1 Training Dataset
Datasets To prevent the overfitting of our large neural network, namely the 2D multi-view diffusion much data as we can. Unlike general objects community which has massive dataset such as Objaverse (800K) and Objaverse-XX (10M) [13]) OmniObject3D (6K) [82], MVImageNet (87K) [99], we don't 't have a single 3D human dataset available at such a scale. To collect data as much as possible, we collect both following public datasets and commercial human scans.
models (o( and 3D generative models gø(o), we train
We collect several publicly available datasets inluding 2k2k (2K) [21]. CustomHuman (640) [27], Thuman2.0 (520) [98], and Thuman3.0 (360) [65]). Among them, CustomHuman, Thuman2.0, and Thuman3.0 O have more repeating subjects with different poses, which have less diverse subject appearance compared 2k2k. It is worth mentioning that 2k2k [21] is a high quality dataset which contains human with diverse clothing (such as skirt) and accessories (such as cap, hat, scarf).
We also utilize in total 2320 high quality commercial scans from AXYZ[1], Treedy [3], Twindom [4], and RenderPeople [2]. All of these scans are with casual clothing and without interaction with objects.
Rendering For each scan, we render 100 views following a spiral trajectory with each view
elevation,
4 π + - 8" 100
(12)
azimuth; = + 5 *
(13)
100"
Additionally, we render 32 views uniformly around z-axis with each view j:
elevation, = O.
(14)
j 32
azimuth; = 0 + π
(15)
To protect the privacy of subjects in the training dataset, only use the frontal view (with azimuth;
- =) 5 as the input context view during training. Thus, we expect the model will not learn faces of [- 2.22 -
subjects when it takes the back view input.
D.2 Evaluation Dataset
For quantitative evaluation, we use Sizer [72] and IIIT 3D human dataset [32]. In this section, we start with introducing the two used evaluation datasets, and explain why we omit the commonly used CAPE dataset [47] in our experiments. Finally, provide summary of the evaluation datasets.
34

---

CAPE Cape [47] is popular evaluation dataset which is widely used by previous methods [23, 87, 88, 108]. We observe that CAPE has limitation appearance variety, geometrical artifacts as well as the publicly unavailability. CAPE only contains simple clothing such as T-shirts and jeans, but no garments of loose clothing. As illustrated in Fig- 22, CAPE contains several artifacts, such as holes on the head, missing hands, and wrong mesh geometry. We manually removed the noise and artifacts sean to serve as our evaluation dataset. Moreover, CAPE doesn't tt have most original scans publicly available, but only the SMPL+D fitting. Due to this reason, cannot render the CAPE scan to RGB images at desired camera view to evaluate the appearance performance such as PSNR, SSIM, and LPIPS. For above mentioned reasons, also propose use Sizer [72] and IIIT [32] which publicly available.
Sizer Sizer [72] is high quality 3D human scan dataset which contains 100 different subjects wearing casual clothing items various sizes. We randomly sample 150 scans from Sizer [72] as one of our evaluation dataset.
IIT IIIT 3D Humans [32] is a high quality dataset from III Hyderabad in India. Different from the casual clothing setup in Sizer [72], IIIT dataset mainly focuses subjects wearing traditional India custom suits, including ethnicity, diverse color pattern and extremely loose clothing (Fig. 21). It brings the huge variety of the subject appearance which be considered o.o.d. evaluation set to our model and baselines. We randomly sample 150 scans from IIIT dataset [32] for evaluation.
Summary We observe that the high quality datasets Sizer [72] and IIT 3D Human [32] are unexplored for the community of 3D avatars reconstruction. In fact, Sizer [72] contains casual clothing which is suitable evaluate performance, and IITT [32] contains challenging texture and loose clothing which suitable to evaluate robustness. All high quality scans in [32, 72] have no severe artifacts and are fully publicly available, which are the benefits unprovided CAPE dataset [47]. By evaluating on these datasets [32, 72] and release our randomly sampled subjects which are used in experiments, we hope the 3D avatars community can discover and benefit from them.
35

---

BALL
2k2k
Custom Human
THuman 2.0
THuman 3.0
Commercial
Figure 19: Example scans in training datasets [1-4, 21, 27, 65, 98].
36

---

Figure 20: Example scans in Sizer [72] dataset. Sizer contains human in casual clothing.
Figure 21: Example scans in IIT [32] dataset. IIT contains subjects with diverse color pattern and loose garments, which rarely appear in training datasets.
Figure 22: Example artifacts in CAPE [47] dataset. Images shown here are rendered by ICON [87] due to the inaccessibility original scans.
37

---

E
Failure Cases
Limited by low resolution (256 256) of our multi-view diffusion model [77]. our model can often fail in reconstructing fine details such as text on the cloth as illustrated in Fig. 23. One potential solution to switch a recent powerful high-resolution multi view diffusion models [19, 68).
Input
Reconstruction
402
Figure 23: Failure Case: our model cannot reconstruct the numbers on the cloth.
In addition, we observe that our model can fail when reconstructing human extremely challenging poses. As shown in Fig. 24, our model cannot infer head geometry and appearance accurately due to the challenging pose in input image.
Input
Reconstruction
Figure 24: Failure Case: our model fails in infer appearance of human with challenging pose.
F
Broader Impacts
Our work shows generality across different ethnicities and humans, providing useful tool for fair representation different cultures. Having robust method to synthesize realistic 3D geometry from a single RGB image may be used in surveillance and inappropriate content generation.
38

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Contributions and scope are included in the abstract and the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please refer to Appendix E
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
39

---

Justification:
The paper does not contain theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please refer our Appendix for Implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
40

---

Answer: [Yes]
Justification:
All code and scripts are publiely available here.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.cm cc/
heellic/guies///ee souideesCooeSuumiisi iesioPPooo onooonooooolll olicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //niss... wtt.c//ppub. wwtpccccpcc//cccccccccccccccclct wwwpce/ggggee/eeee httpc/guiidee/oeeuuuiiin ssissiinn onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We include details about datasets and implementation in Sec. 5.1, Ap- pendix A.1, Appendix D. Appendix D.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We follow standard evaluation protocols reporting average errors on exten- sive test set.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
41

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer:
[Yes]
Justification: The paper includes the computational resources used in our experiments in Section 5.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: : http///eeuuippp.peeee wwwpurips.ccpuuuae httpppullic/ttiihsiidiidcoooeeee
Answer: [Yes]
Justification:
We read and adhere to the NeurIPS Code Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The paper include Broader Impacts statement in Appendix
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
42

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not see such high risk posed by our paper.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
We cite and explicitly refer to all the legitimate sources of code, data, and models.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
43

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: We do not include new assets, but we will include appropriate documentation upon code release after acceptance.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Although the method involves 3D human models, we rely on datasets collected before this work; we refer them for their specifics.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
supplemental material is fine, but if the main contribu-
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: NA
Justification; Although the method involves 3D human models, we rely on datasets collected before this work; we refer to them for their IRB approvals.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
44

---

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
45


Reviews:
[{"review_id": "XfpK89vsjU", "reviewer": "Reviewer_KYnm", "summary": "In this paper, the authors propose to create realistic avatar representations by coupling the 2D multi-view diffusion and 3D reconstruction models which complement each other. Specifically, the 3D Gaussian Splatting (3D-GS) reconstruction leverages the priors from 2D diffusion models and produces an explicit 3D representation. Meanwhile, the rendered images from 3D-GS representations further guide the reverse sampling of 2D diffusion models to improve 3D consistency. Experimental improvements on some examples are achieved to demonstrate the empirical effectiveness of the proposed method.", "questions": "+ In Table 4, why only using FID for evaluation? Could the authors provide the results with other metrics like PSNR, SSIM and LPIPS?\n+ How many camera views and how many examples are used for evaluation? Will the authors release their exact evaluation settings in the future?\n+ It seems that the results of Table 2, 3, 4 are performed under different settings. Could the authors align all ablated revisions with the settings used in Table 1? It can better help the readers to comprehend the importance of each key technical idea compared to previous baselines.\n\nI am looking forward to hearing about these points from the authors.", "limitations": "The authors address the limitation and societal impact in the appendix.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ How to use one single image to infer 3D structure is an important problem, which can affect many down-streamed applications.\n+ The **Human 3Diffusion** method improves the existing methods in some scenarios.\n+ The paper is well written with nice figures. I can follow it easily.", "weaknesses": "+ In Figure 3, as for me, the outputs from Human 3Diffusion are similar to those of SiTH in terms of image quality. Actually, some examples in Figure 8 even show that SiTH can produce better results than the proposed method, such as the right hand is more reasonable on the 4th row.\n+ Although the authors argue why they do not evaluate on the CAPE dataset, which is a standard testbed for previous methods, I think the Sizer and IIIT datasets are also not perfect. For example, as shown in Figure 7, some inputs contain the noisy ground part while the SiTH method produces cleaner background than Human 3Diffusion. I guess this also might yield worse metrics for SiTH in Table 1. So I suggest the authors additionally report the metrics on CAPE and CustomHuman datasets by following the evaluation settings of SiTH such that the readers can better perceive the empirical improvements from the proposed contribution.\n+ Some wordings are a bit too strong. For example, in L198, the authors state that \"guarantee the 3D consistency...\". I understand that the developed 2D Multi-view Sampling could improve the 3D consistency, but it is hard to say it can fully address this problem.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for acknowledging the **importance of our task** and highlighting that our paper is **well written**, and our method **improves existing methods**. We address the concerns raised below and are open to further discussion and questions.\n\n---\n\n### **Q1: Performance of Human Reonctruction similar to  SiTH**\nA1: We acknowledge SiTH as a strong baseline (CVPR’24, code released in April). While our examples may not fully highlight our method's advantages, Table 1 shows quantitative superiority over SiTH. We provide further qualitative comparisons in the **Rebuttal PDF** (Figure 1). SiTH relies on the SMPL estimation which might produce good hands or faces, but cannot represent geometry, which deviates significantly from SMPL body. Our method offers **greater flexibility** in modeling challenging scenarios like **loose clothing**, **interaction**, and **children**, as demonstrated in Supplementary PDF Figures 9-12 and **Rebuttal PDF** Figure 1.\n\nTo further evaluate reconstruction quality, we conducted a user study detailed in the general rebuttal section. The results show that **86.6%** of participants prefer our reconstructions, clearly indicating superior quality over SiTH and other SOTA baselines.\n\n---\n\n### **Q2: Evaluation on new Datasets like CAPE and CustomHuman**\nA2: Thank you for the suggestion. We've now included results for the CAPE and CustomHuman datasets under the same settings as IIIT and Sizer. Since some high-quality CustomHuman scans were used in training, we report only the results for unseen subjects (ID0636 - ID0641). The FID score is higher in this case because fewer examples were available to compute the image distribution. It can be seen that our method consistently outperforms SiTH and SiFU on both datasets. We will add these evaluations to Table 1.\n\nCAPE|PSNR|SSIM|LPIPS|FID|CD(cm)|NC|F-score\n:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:\nOur|21.5|**0.916**|**0.064**|**16.40**|**1.89**|**0.80**|**0.49**\nSiTH|**22.2**|0.908|0.082|28.46|3.76|0.78|0.27 \nSiFU|22.0|0.907|0.085|43.63|3.72|0.77|0.27\n\nCustomHuman|PSNR|SSIM|LPIPS|FID|CD(cm)|NC|F-score\n:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:\nOur|**22.3**|**0.926**|**0.048**|**28.94**|**1.03**|**0.85**|**0.66**\nSiTH|20.8|0.915|0.073|60.37|2.82|0.82|0.30\nSiFU|20.1|0.908|0.081|87.09|3.10|0.81|0.31\n\nWe hope the additional experiments address the concern of reviewer.\n\n### **Q3. Noise on the Sizer test set.**   \nA3: We agree that Sizer evaluation set is not perfect due to the noise on the ground. We remove the floor noise and redo the evaluation. Results are reported below and our method consistently outperforms baseline. We will release the evaluation dataset with rendered input images for more convenient benchmarking. \n\nSizer denoise|PSNR|SSIM|LPIPS|FID|CD(cm)|NC|F-score\n:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:\nOur|**21.3**|**0.928**|**0.047**|**10.01**|**1.06**|**1.06**|**0.63**\nSiTH|18.9|0.912|0.063|21.87|3.38|0.75| 0.28\nSiFU|18.0|0.912|0.068|36.64|2.69|0.78|0.33\n\n---\n\n### **Q4: Some wordings like 'guaranteed 3D consistency' are too strong.**\nA4: Thank you for pointing this out. We agree that we should avoid over-claiming in the paper. However, we believe this might be due to misunderstanding of L198: here we meant the renderings of the predicted 3D-GS are guaranteed to be 3D consistent. This is true because we have an explicit 3D-GS representation. To help 2D diffusion model, we add noise to the renderings as the input to next step (L7, Alg.2). These noised renderings are indeed not 3D consistent. We will clarify this better in L198-200. We are open to have further discussion and obtain feedback from the reviewer to improve the manuscript.\n\n---\n\n### **Q5: Why only FID metric in Table 4? Can author provide other metrics like PSNR, SSIM, LPIPS?**\nA5: Thank you for the suggestion. We provide the other metric below: \n\nAblation Tab.4|PSNR|SSIM|LPIPS|FID|CD(cm)|NC|F-score\n:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:\nOur w/o 2D prior|20.98|0.912|0.068|11.70|1.75|0.795|0.498\nOur|21.49|0.918|0.060|9.57|1.35|0.798|0.550\n\nIn the original submission, the baseline was trained with relative camera system, which means the camera pose of the input and 3D reconstruction are unknown. This makes it difficult to compute PSNR and SSIM that require good image alignment. We retrain the baseline with a global camera system, which makes comparison possible and also explains the difference in FID compared to original table 4. But the conclusion is the same: 2D prior helps 3D reconstruction. We will update table 4 with the new numbers. \n\n---\n\n### **Q6:. Details about evaluation setting**\nA6: We use 32 uniformly rendered views around the human with zero elevation angle (Supp. L840-844). The number of subjects evaluated in each datasets are: IIIT (155), Sizer (136), CAPE (107) and CustomHuman(8). **We will release the full evaluation setting**, including Blender rendering and metrics calculation scripts. To ensure the reproducibility, we will also **release the processed Sizer denoised dataset**.\n\n---\n\n### **Q7: Do Table 2, 3, 4 have same the setting as Table 1? What is the difference between these ablations?**\nA7: Table 1, 3, 4 all have the same evaluation setting and the numbers of ours are aligned. Table 2 ablates the influence of our 3D model on 2D multi-view diffusion outputs. Hence we evaluate only on the 4 output images from MVD instead of 32 views used on other tables. this leads to different numbers for our method. We will move table 2 after table 3 and 4, making it easier for reader to connect the information across these tables.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fEVG5UUp5o", "reviewer": "Reviewer_uByW", "summary": "The paper introduces a framework that combines 2D Multi-view Diffusion model and Gaussian Splatting to achieve the task of 3D clothed human body reconstruction from a single view. The focus of the paper is to deal with the 3D inconsistency present in 2D multi-view diffusion models.", "questions": "Are all the SOTA methods trained with the same training set (with the same human subjects) as your model?\n\nWhat is the inference time required to generate each mesh, and how does that compare against the existing SOTA methods?", "limitations": "No potential negative societal impact.\n\nI hope the authors address the aforementioned concerns during the rebuttal.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "A novel framework that has reasonable motivations. The ablation studies support the various components introduced by the authors.\n\nThe method appears to be robust to various different input data.", "weaknesses": "Point-to-surface (P2S) metric, widely used in established works like PIFu, PIFuHD, ICON, and ECON, is not used in this paper.\n\nIn terms of the resolution of the generated meshes (especially for facial features), the proposed model seems to perform poorer than what has been observed in SOTA like ECON or ICON. Can the authors provide a comparison of the proposed model with the SOTA methods but show only the geometry and not the texture?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **Q1: Point-to-surface (P2S) metric not reported in the paper**\nA1: Thanks for the question. We would like to point out that the chamfer distance (CD) reported in the paper is a bidirectional point-to-mesh distance. It measures the distance from both Point-to-Surface (P2S, reconstructed mesh to GT scan) and Surface-to-Point (S2P, GT scan to reconstructed mesh). We understand and agree with Reviewer that reporting the P2S and S2P separately helps analyze the performance. Hence we report the numbers for the reconstruction from table 1 below. We additionally report the results of ICON and ECON as suggested by R3 (RJbF). We will integrate these numbers into Table 1.\n\nAccuracy|CD(cm)|S-to-P(cm)|P-to-S(cm)\n-|:-:|:-:|:-:\nOur|**1.35**|**1.38**|**1.31**\nSiTH|3.92|4.18|3.66\nSiFU|3.60|3.50|3.70\nECON|3.52|3.54|3.49\nICON|4.06|4.03|4.08\nPiFU|2.83|2.94|2.70 \nLGM|3.29|2.83|3.775\nTripoSR|2.59|2.65|2.52\nInstantMesh|2.47|2.59|2.34\n\n---\n\n### **Q2: Geometry Reconstruction performance and comparison with SOTA methods like ICON and ECON**\nA2: We have presented the results of geometry reconstruction using ICON and ECON in our **Rebuttal PDF** (Figure 1). Our model demonstrates superior performance compared to ICON and ECON. This advantage arises from the limitations of ICON and ECON in modeling **loose clothing**, **accessories**, and **children** due to their dependency on SMPL.  In contrast, our model is not constrained by that. Additionally, we have further assessed the quality of the geometry through a user study. We use 20 subjects, randomly sampled from IIIT, Sizer, and CustomHumans, and conduct a user study involving 70 people. \nIn our user study, users chose the geometry reconstructed by our method over ICON and ECON 73.8% of the time.\n\nMoreover, previous works like ECON uses Poisson reconstruction and Laplacian Smoothing as the post processing. It also directly replaces the reconstructed face with the facial part of SMPL-X, and stitches the face part using poisson reconstruction. Our method uses 3D-GS to represent human which is **flexible to represent diverse clothing geometry** but it is still an open question to extract high-quality meshes from 3D-GS. Nevertheless, **we do not have extensive post-processing to optimize the geometry but still obtain better reconstruction**. With the rapid advancement of remeshing for 3D-GS (e.g. ref. [1]), we believe our geometry can be even better and the advantage of our 3D-GS representation will prevail more in the future. \n[1]. DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing. \n\n---\n\n### **Q3: Settings of SOTA baselines**\nA3: Thanks for raising this question. We reuse some weights from LGM which was trained on Objaverse and fine-tune on 6k human scans. We also fine-tune LGM on the same human data (LGM_human in the paper). For human reconstruction baselines, we use the official released model from each baselines to evaluate the performance. This is a standard setting in most recent papers in this area (e.g. SiTH, SiFU). It is not possible to retrain all baselines due to compute limitation. Apart from compute, it is also impossible to train baselines like **SiTH, SiFU, ICON, ECON that requires GT SMPL fits to scans**, as it is very **difficult to fit SMPL for scans with wide clothing, clutter, children or missing parts**. No method currently achieves this reliably. This remains an open challenge and poses a limitation for methods dependent on SMPL GT. **Our method does not rely on SMPL hence allows us to train on 6k scans where only 1500 scans have GT SMPL fits**. \nHowever, we fully understand that ablating the contribution of model design and data is important for future works. Thus, we trained our model on Thuman 2.0 only, which is the same training dataset of SiTH, SiFU, ICON, and ECON. We adopt pretrained 2D Multi-view diffusion (MVD) on objaverse and pretrained diffusion-based 3D-GS generator on ShapeNet. We report the performance as follows:\n\n\nAccuracy|PSNR|SSIM|LPIPS|CD(cm)|S-to-P(cm)|P-to-S(cm)\n:-:|:-:|:-:|:-:|:-:|:-:|:-:\nSiTH|20.88|0.907|0.074|3.92|4.18|3.66\nSiFU|20.39|0.896|0.085|3.60|3.50|3.70\nOur (Thuman2.0 only)|21.21|0.907|0.066|1.60|1.66|1.63\nOur|21.5|0.918|0.060|1.35|1.38|1.31\n\nThe results shows that our model trained on the same data (Thuman 2.0) still outperforms baselines and achieves SOTA performance. We would like to emphasize that the Thuman2.0 dataset is smaller (approximately 500 samples) and offers less diversity in terms of clothing, subjects, and poses compared to our full training dataset. **Despite this, our model outperforms SOTA methods and its performance is very close to that of our model trained on the full dataset**. This demonstrates the strength and effectiveness of our proposed model. We thank the reviewer for encouraging us for this experiment.\n\n---\n\n### **Q4: Inference Time of proposed approach**\nA4: In the introduction section, we have compared our model's performance with other models in terms of memory and processing time. Although our method is based on a diffusion approach, which involves iterative sampling, it only requires 50 feedforward steps (DDIM), resulting in a generation time of **approximately 22 seconds**. In contrast, baseline methods depend on SMPL estimation and test-time optimization, which significantly slows down their performance, taking up to 2 to 5 times longer.\n\nFor more details and table, please refer to the introduction section.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ds2J9oHqit", "reviewer": "Reviewer_RJbF", "summary": "This paper proposed an image-conditioned 3D-GS generation model for human 3D reconstruction. 2D diffusion models fall short in offering 3D consistency for multi-view shape priors. To address this, the authors introduce a method that combines the strengths of 2D multi-view diffusion and 3D reconstruction models to create avatars with explicit 3D consistency. They propose an image-conditioned 3D Gaussian Splats model that uses 2D diffusion priors to enhance the 3D reconstruction and guide a 2D reverse sampling process, resulting in geometry and appearance. It’s capable of joint training a 3D generative model and a 2D multi-view diffusion model end-to-end.", "questions": "1. How long is the training period? Is it possible to encounter the phenomenon of Gaussian overfitting before the diffusion model has converged? If it occurs, how is it resolved?\n2. What is the inference speed like, and does it have an advantage compared to other methods?\n3. The multi-view human generation results of your 2D Diffusion model should be displayed.", "limitations": "Limited reconstruction quality and lack of comparison results with state-of-the-art methods.", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.The motivation and insight behind this paper are reasonable. I agree that only by obtaining 2D priors with good 3D consistency can one achieve high-quality 3D Gaussian reconstruction results.\n2. An interesting idea of joint training 2D diffusion model and 3D Gaussian end-to-end.\n3. The paper is well-written and easy to follow.", "weaknesses": "1. The resolution supported by the model is too low.\n2. By observing the qualitative results, I don't think the method has superior performance in geometry or texture; it seems to have a significant gap compared to existing methods that were not compared. However, the quantitative metrics are surprisingly good, which puzzles me. \n   3. Comparative experiments lack comparisons with many other works. I suggest adding comparative experiments with the following studies:\n1.TeCH: Text-guided Reconstruction of Lifelike Clothed Humans（3DV 2024）\n2. HumanRef: Single Image to 3D Human Generation via Reference-Guided Diffusion（CVPR 2024）\n3. Human-SGD: Single-Image 3D Human Digitization with Shape-Guided Diffusion（SIGGRAPH Asia 2023）\n4. ECON: Explicit Clothed humans Optimized via Normal integration（CVPR 2023）\n5. ICON: Implicit Clothed humans Obtained from Normals（CVPR 2022）\n6. FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstruction（NIPS 2022）", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for **recognizing our insight, motivation, and the interesting idea of the proposed framework**. We notice that the reviewer has concern about the performance comparisons with SOTA methods and thus looks forward to more comparison and results. We address these concerns here and are happy to take part in further discussions.\n\n---\n\n### **Q1: The supported resolution is low**\nA1: The reviewer may refer We guess the reviewer refers to the resolution of our 2D Multi-view diffusion model. We adopt ImageDream pretrained multi-view diffusion model, where the resolution is already limited by (256x256). We denote it as a limitation (L288), and believe that it can be resolved by leveraging a more powerful high-resolution multi-view diffusion model such as SV3D (576x576).\nWe would like to emphasize that our proposed approach is not limited to the ImageDream. Our framework allows us to use any 2D multi-view diffusion models and further improve them with our proposed tight-coupling of 2D and 3D generation models. Even with low-resolution input, our method achieves sota results, highlighting the strength of our idea. \n\n---\n\n### **Q2: Qualitative results don't achieve superior performance than previous SOTA works but quantitative results are high**\nA2: Thanks for raising this concern. Our examples shown in fig2. might not well represent the superiority of our method. We add 10 more comparison with SiTH, SiFU, ECON and ICON in our rebuttal. Furthermore, we randomly select 40 subjects from the test set and conducted asked 70 users to select which method has the best reconstruction quality. **Results suggest that our method is preferred by 80.3% of the users, which is aligned with the quantitative results we reported in the paper**. \nWe would like to point out that, even though ECON showed impressive results and robustness to diverse clothing and challenging poses, it heavily relies on SMPL estimations which can be inaccurate in challenging cases. As we shown in rebuttal pdf, inaccurate SMPL estimation leads to incorrect human shape and clothing geometry. In contrast, our method does not rely on SMPL and is **more flexible to represent different clothing, accessories, and children**. Therefore, we obtain better results in the test set IIIT, Sizer (initial submission) and Cape, CustomHuman (commonly used benchmarks).\n\n---\n\n### **Q3: Comparative experiments lack of baselines, suggest adding comparison with TeCH, HumanRef, HumanSGD, ICON, ECON, FoF**\nA3: Thanks for pointing out this extensive list of comparable works. We compared with SiTH and SiFU, both are SOTA human reconstruction methods published at CVPR2024. Since they already outperform prior SoTA ECON/ICON, we omit the comparison in the initial submission. We do understand that thorough comparison with more baselines can strengthen the arguments. However, some of the listed baselines have either too long runtime (Tech, >6h per image and requires more than 48GB GPU), poorly maintained codebase (FoF, no instruction), released after NeurIPS (HumanRef released June 19, 2024), or no code release (HumanSGD). Therefore, we compare additionally only with ICON, ECON as they are the most popular baselines. The results are reported in the table below. \n\nMethod|Published at|SMPL prior|CD(cm)|NC|F-score\n:-:|:-:|:-:|:-:|:-:|:-:\nPiFU|ICCV2019|❌|2.83|0.769|0.333\nICON|CVPR2022|✅|4.06|0.728|0.230\nECON|CVPR2023|✅|3.52|0.749|0.278\nSiTH|CVPR2024|✅|3.92|0.735|0.250\nSiFU|CVPR2024|✅|3.60|0.739|0.235\nLGM|ECCV2024|❌|3.29|0.562|0.275\nTripoSR|03.2024|❌|2.59|0.771|0.360 \nInstantMesh|04.2024|❌|2.47|0.787|0.338\nOur|-|❌|**1.35**|**1.38**|**1.31**\n\nThe quantitative results show that our approach outperforms all baselines. We include additional qualitative examples in the rebuttal PDF. We also conducted a user study to evaluate the qualitative results (see R1Q1). Overall, our method is preferred over ICON, ECON, SiTH, SiFU by approximately 80% of 70 users. This clearly shows that our method outperforms baselines. \n\n---\n\n### **Q4: How long is training period? Is it possible to encounter the phenomenon of Gaussian overfitting before the diffusion model has converged?**\nA4: It takes 5 days on 8 A100@80GB to train our model (L213). Preventing large models from overfitting to small datasets is an open research question. In our setting we did not observe this problem as our model generalizes to subjects with diverse appearance and geometry(supp. fig. 9-12) and even general objects (Fig5, fig 14). We believe overfitting is mitigated due to these aspects:\n- Pretraining on a large-scale 3D dataset. We reuse some model weights from ImageDream (pretrained on LAION5B, Objaverse) and LGM (pretrained on 80k Objaverse objects). This pretraining provides strong prior to reason 3D shapes even from very noisy multi-view and a single clean input image. \n- Data augmentation for camera poses. We added small noise to the camera poses when first training the 3D-GS model alone. This helps align the 3D-GS generator to quickly adapt to 3D inconsistent multi-view images from 2D MVD model. \n- Small learning rate for fine-tuning. As is common in the literature, we employ small learning rates to fine-tune the pretrained MVD (1e-5 with Cosine Annealling) and 3D-GS generator (5e-5). \n \n\n\n---\n\n### **Q5: What is the inference speed and does it have an advantage compared to other methods**\nA5: Our generation time is **approximately 22 seconds**. In contrast, baseline methods depend on SMPL estimation and test-time optimization, which significantly slows down their performance, taking up to 2 to 5 times longers. For more details, please refer to general Rebuttal Q2.\n\n---\n\n### **Q6: The multi-view human generation results of your 2D Diffusion model should be displayed**\nA6: A6: Thanks for the suggestion. We showed one example in fig. 6 last row and 2nd column. We include more examples of 2D diffusion outputs in **Rebuttal PDF** (figure 3). We will add these to the supplementary.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "USuEoZiJK3", "reviewer": "Reviewer_iJQG", "summary": "The paper presents a pipeline for creating a 3D model of a full body avatar given a single input image. \nWhile the previous methods show a single image to 3D using a 2D diffusion model such as ImageDream, as a 2D model, they suffer from 2D inconsistencies. This paper combines the benefit of both the large-scale 2D pre-trained multiview image model as well as the 3D consistent generative model (here they used gaussian splatting as such a representation). \n\nGiven a single input image, the method first creates 4 orthogonal views of the input using 2D ImageDream. While this creates some realistic views, those views might be inconsistent. To create more consistent views, the method renoises the image and pass it to 3D generative model based on a gaussian representation to create a 3D consistent version of the 2D output. The output from the 3D generative model is 3D consistent but the texture might not be as good as the 2D one. So the model finally redecodes the 3D rendering with a 2D model to create the final 3D-aware rendering. \n\nBoth the 2D ImageDream and 3D gaussian-based generative models are trained jointly with a tightly coupled manner. To enable the tight coupling, the 3D generative model also takes a noisy image as well as a time step t, just like the 2D counterparts. The 3D generative model part is built on top of the U-Net decoder from LGM, which takes a 4-view image and produces gaussian splatting parameters. \nThe model is trained on a combined dataset of 6000 human scans, which are converted to rendering images via Blender. \n\nThe paper demonstrates convincing results on multiple challenging datasets (Sizer, IIT-Human, and GSO) with extensive numerical / qualitative comparisons.", "questions": "* How long will it take to process an image? \n* How is X^tgt_{t} exactly generated? In Fig2, it looks like the noisy images have some structured noise rather than a regular gaussian noise added. Why is that? \n* The visualization of the used views in Figs9 and 10 are inconsistent. For some subjects only front views are shown and other subjects only back views are shown rather randomly. Please make them consistent or show both.  \n* The following papers are relevant for citation: \nGenerative Novel View Synthesis with 3D-Aware Diffusion Models (ICCV2023)\nReconFusion: 3D Reconstruction with Diffusion Priors (CVPR2024)\nZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Real Image\n\nTypos\n* line225: typo, extra period after \"reconstruction\"\n* Figure 6: Visualization \"of\" intermediate sampling steps\n* line781: Follow[ing] [71]", "limitations": "The paper discusses both the broader impact and the limitations.\n\nSome more limitations could be clarified:\n* Currently, limited materials are supported (e.g., shiny green dress on the right col of Fig13).\n* Fine-scale details such as stuffed toy furs (e.g., first image of Fig14) seems to have blobby 3D details likely from the 3DGS, which seem not just the problem of the low resolution generator.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Convincing results are demonstrated on challenging datasets of Sizer, IIT-Human, UBC Fashion and GSO (Google Scanned Objects)\n* While the paper mostly focuses on humans, the method itself is fairly generic and could also work on general objects (some results are shown)\n* extensive numerical and qualitative comparisons are provided and the proposed method performs better compared to the previous work\n* While many previous methods used mostly 2D generative priors, the paper shows how to use both 2D and 3D generative models to create a better result.\n* Paper is overall easy to follow and good amount of technical details are provided (but not all).", "weaknesses": "* the paper provided most of the technical details but some key details are still missing such as the exact architecture of the noise conditioned gaussian based 3D generative model and the timing of how long it will take to process an image. \n* the overall pipeline seems fairly complicated to need to train both 2D and 3D generative models and that they rely on 3D scans, which are fairly limited for humans (only 6000). \n* the paper text has some English grammar errors and typos and could benefit from proofreading.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer for recognition of our method’s strengths, **including its generalization, potential extensions, performance, and clarity**. We maximize our effort to answer every comment seriously and hope our response can address the reviewer’s concerns. If there are any remaining questions, we are more than happy to address them.\n\n---\n\n### **Q1: The paper provided most of the technical details but some key details are still missing such as the exact architecture of diffusion-based 3D-GS generative model**\nA1: Thanks for pointing out this problem. We start with the asymmetric U-Net proposed by LGM for our 3D-GS generator. The key modifications we did are:\n- Following stable diffusion we inject time embedding into the U-Net. \n- We add additional clear context image in addition to the other 4 views.\n- We add 3D-attention across views and the context view to inject condition information during the generation process.\n\nWe hope this clarifies the details, and we will add these details to our supplementary. \n\n---\n\n### **Q2: The overall pipeline seems fairly complicated to need to train both 2D and 3D generative models, and the method only rely on limited 6000 subjects scans**\nA2: We agree that the number of human scans are small-scale compared to objects dataset such as Objaverse (800K to 10M objects). However, we empirically didn’t observe the overfitting. On opposite, we show that our approach generalizes well to unseen subjects with various type of clothing and appearance. \nIn our analysis, the pretrained Multiview diffusion model and initialization from LGM helps in preventing the overfitting.\nMoreover, as reviewer mentioned, our proposed approach is generic. Thus, we can pretrain on large-scale objects dataset and finetune on limited human scan dataset. To validate this, we conduct an toy example:\nWe test to train the model jointly with only Thuman2.0 (500 subjects, simulate limited human data) and Thuman 2.0+ShapeNet (12K objects + 500subjets, simulate large-scale objects data + small scale human data). For both experiments, we adopt pretrained Imagedream and use LGM to initialize layer weights for 3D-GS generator.\n\nWe agree that the number of human scans is small-scale compared to object datasets such as Objaverse (800K to 10M objects). However, we empirically didn’t observe the overfitting. On the contrary, we show that our approach still generalizes well to unseen subjects with various types of clothing and appearance (supp. fig. 9-12) and also objects (Fig5, fig 14). In our analysis, the pretrained mult-iview diffusion model and initialization from LGM help in preventing overfitting.\n\nMoreover, as the reviewer mentioned, our proposed approach is generic. Thus, we can pretrain on large-scale objects dataset and finetune on limited human scan dataset. To validate this, we finetune the model pretrained on Objaverse on Thuman (500 human scans). We don’t observe a significant drop in the performance, see more details in R2Q3. \n\n---\n\n### **Q3. Training requires 3D scans, which is limited for humans (only 6k).**\nA3: We would like to point out that we do not rely on full 3D to train the model. Instead, we compute losses on the multi-view images only. Therefore, in principle, we can also train our model on multi-view images with good camera poses, which is much more abundant than 3D scan data. Future works can extend our method to **train on multi-view or video data**, and we believe there is a lot of potential in this direction. \n\n---\n\n### **Q4: How long will it take to process an image?**\nA4: Thanks for the question. We also discuss this with other reviewers (KYnm, Q4) and reviewer (uByW, Q5). Please refer to Q2 in general rebuttal for the table. In summary, it takes around 22.5 seconds to process an image on one Nvidia A100.  We further evaluate each key component inside of individual diffusion steps, and report the runtime of subcomponent here:\n- each DDIM steps of 2D-3D joint diffusion: 0.46 seconds\n    - where 2D denoising step: 0.08 seconds\n    - where 3D denoising step: 0.38 seconds\n\n---\n\n### **Q5: How is X^tgt_{t} exactly generated? In Fig2, it looks like the noisy images have some structured noise rather than a regular gaussian noise added.?**\nA5: X^tgt_{t} was initialized from Gaussian noise in reverse diffusion step T. We visualize the intermediate step results in Fig2, where the MVD predictions already have some structure. Visualization of more intermediate steps can be found in supp figure 6. We will clarify this better in Fig2. \n\n---\n\n### **Q6: The visualization of the used views in Figs9 and 10 are not all aligned. For some subjects only front views are shown and other subjects only back views are shown rather randomly. Please make them consistent or show both.**\nA6: Thanks for pointing this out. We showed different views for each subject in order to showcase diverse viewing angles of our results. In our rebuttal pdf we show comparisons in consistent viewing angles. We will update Figure 9-12 with consistent viewing angles. \n\n---\n\n### **Q7: Some more imitations could be clarified**\nA7: Thanks for the suggestion. We agree that these points are indeed the limitations of our method. They are also the unsolved problems in current sota multi-view diffusion models or 3D reconstruction methods. We will add this discussion to the limitations.\n\n---\n\n### **Q8. Typos and English grammar error**\nA8: Thanks for pointing them out. We will correct them in the final manuscript. \n\n---\n\n\n### **Q9: Missing citations of relevant papers**\nA9: Thanks for the additional reference, we will add paper [1] to L125, L767, paper [2] to L125, L767 and paper [3] to L74. \n[1]. Generative Novel View Synthesis with 3D-Aware Diffusion Models (ICCV2023).\n[2]. ReconFusion: 3D Reconstruction with Diffusion Priors (CVPR2024).\n[3]. ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Real Image.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers and Area Chairs,

We sincerely thank all reviews and ACs for their time and insightful feedback. We are glad that they found our work novel and addressing an important task (R1) and appreciating the technical contribution of of integrating 3D Gaussian Splatting generation (R3, R4) within 2D diffusion. Reviewers appreciated also our experiments, since we improve over existing methods (R1, R4), validate with a comprehensive ablation study (R2), and show robustness to different input (R2, R4) as well as general objects (R4).  

The main concerns raised from the reviews are the evaluation and qualitative comparison with more baselines, and inference time cost. We address these by adding comparisons with ICON, ECON on our initial evaluation dataset and CAPE, CustomHuman. We also report the inference time and compare with baselines. Please see Q1 and Q2 below. We answer each questions in more details in the replies to each reviewer. We sincerely hope that ours replies could address all the concerns. We are also open for discussion and are happy to clarify or address any further questions. 

---

### **Q1: the Qualitative Results are not outstanding compared to SOTA baselines, why is it?**
A1: We appreciate the reviewer’s assessment. However, we maintain that our results, as shown in Figure 3 of the main paper and Figures 7 and 8 in the supplementary material, indeed surpass current SOTA methods. To further support our position, we have included additional qualitative results as suggested by reviewers and a user study in our rebuttal.

We have included additional qualitative results (see Figure 1) in the **Rebuttal PDF**, where we compare our method against SiTH (CVPR2024), SiFU (CVPR2024), ICON (CVPR2022), and ECON (CVPR2023). Our results highlight the advantages of our approach in handling challenging scenarios such as **large loose skirts**, **children**, **anime characters**, and **diverse accessories**, where prior methods often struggle and sometimes completely fail to generate anything reasonable. SOTA methods rely on the SMPL template which might produce good hands or faces, but it is limited by the naked body shape. In contrast, our method does not rely on SMPL and is more flexible. We invite reviewers to examine Figures 9-12 in the supplementary materials and **Figure 1 in the Rebuttal PDF**. This flexibility is also evident in our quantitative results, where SOTA baselines falter due to SMPL estimation inaccuracies.

Moreover, we thoroughly assessed our results through quantitative analysis and a **user study with 70 participants**. This study compared 20 textured subjects against SiTH and SiFU, and 20 geometry-only subjects against ICON and ECON, using subjects from the IIIT, Sizer (w/o floor noise), Cape, and CustomHuman test sets. Participants were asked to select the best reconstruction among three options. Details and a demo of the study are provided in the **Rebuttal PDF**.

User Study|Our|SiTH|SiFU
---|:-:|:-:|:-:
Appearance & Geometry|**86.6%**|7.6%|5.8%

User Study|Our|ICON|ECON
---|:-:|:-:|:-:
Geometry only|**73.8%**|8.0%|18.2%

In summary, our model approach is preferred by **80.3%** of participants in our user study, indicating a significant preference over the baseline models.

We trust that our additional results comprehensively address the concerns raised. We remain open to further suggestions on how to better demonstrate the advantages of our approach over the baselines.

---

### **Q2: What is the runtime to infer one image? Does Human-3Diffusion have advantage in efficiency compared to other works?**
A2: Thanks for the question. Our Human-3Diffusion is a diffusion-based feed-forward approach, avoiding the SMPL estimation and test-time optimization required by models like ICON, ECON, SiTH, and SiFU. This approach significantly boosts our model’s inference speed. We provide a runtime comparison on an Nvidia A100 GPU below, detailing the inference time from an RGB image to the final 3D representation:

Model| Time (s) | VRAM (GiB) 
:-:|:-:|:-:
SiTH | 106.2| 22.0
SiFU | 48.9 | 12.0
ICON | 60.5 | 6.3
ECON | 45.3 | **5.9**
Ours |**22.6**| 11.7

For mesh extraction, we use Gaussian Opacity Fields (11.5s, resolution=256) and TSDF-Fusion (14.8s, across 24 views, resolution=256). We will provide comprehensive details in the [updated version].

---

For other individual comments, we have addressed each within the respective sections assigned to each reviewer. We deeply appreciate all the effort and time invested by the reviewers and Area Chairs.

Best,\
Authors

Meta-review: This paper proposes a framework that utilizes 2D multi-view diffusion models and ensures 3D consistency through explicit 3D Gaussian splatting to generate 3D clothed avatars. The initial scores were mixed, with two borderline accepts (5), one weak accept (6), and one reject (3). Following the rebuttal, the reject was revised to a borderline reject (4). The main concerns raised by the reviewers focused on the quantitative and qualitative evaluations. While the reviewer RJbF still has reservations about the qualitative comparison results and input resolution, the other reviewers found the responses satisfactory and maintained a positive attitude during rebuttal. Given the innovative joint diffusion process that synchronizes 2D multi-view diffusion with 3D generation, which was acknowledged by all reviewers, I recommend accepting this paper.
Decision: Accept (poster)