LIFe-GoM: Generalizable Human Rendering with Learned Iterative Feedback Over Multi-Resolution Gaussians-on-Mesh
Authors: Jing Wen
Alex Schwing
Shenlong Wang
Venue: ICLR 2025 Poster (2025)
Keywords: Generalizable human rendering
error feedback
dual representation
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Generalizable rendering of an animatable human avatar from sparse inputs relies on data priors and inductive biases extracted from training on large data to avoid scene-specific optimization and to enable fast reconstruction. This raises two main challenges: First, unlike iterative gradient-based adjustment in scene-specific optimization, generalizable methods must reconstruct the human shape representation in a single pass at inference time.
Second, rendering is preferably computationally efficient yet of high resolution.
To address both challenges we augment the recently proposed 
dual shape representation, which combines the benefits of a mesh and Gaussian points, in two ways. 
To improve reconstruction, we propose an iterative feedback update framework, which successively improves the canonical human shape representation during reconstruction.
To achieve computationally efficient yet high-resolution rendering, we study a coupled-multi-resolution Gaussians-on-Mesh representation.
We evaluate the proposed approach on the challenging THuman2.0, XHuman and AIST++ data. Our approach reconstructs an animatable  representation from sparse inputs in less than 1s, renders views with 95.1FPS at $1024 \times 1024$, and achieves  PSNR/LPIPS*/FID of 24.65/110.82/51.27 on THuman2.0, outperforming the state-of-the-art in rendering quality.

Published as a conference paper at ICLR 2025
LIFE-GOM: GENERALIZABLE HUMAN RENDERING WITH LEARNED ITERATIVE FEEDBACK OVER MULTI-
RESOLUTION GAUSSIANS-ON-MER
Jing Wen, Alexander G. Schwing & Shenlong Wang
University of Illinois Lirana-Champae fjw116, aschwing, shenlong eilliniis edu https://weet github..ittb....... Lo/LIFe-GoM/
ABSTRACT
Generalizable rendering of an animatable human avatar from sparse inputs relies on data priors and inductive biases extracted from training on large data to avoid scene-specific optimization and to enable fast reconstruction. This raises two main challenges: First, unlike iterative gradient- adjustment in scene-specific opti- mization, generalizable methods must reconstruct the human shape representation in a single pass at inference time. Second, rendering is preferably computation- ally efficient yet of high resolution. To address both challenges we augment the recently proposed dual shape representation, which combines the benefits of a mesh and Gaussian points, in two ways. To improve reconstruction, we propose an iterative feedback update framework, which successively improves the canon- ical human shape representation during reconstruction. To achieve computation- ally efficient yet high-resolution rendering, we study a coupled-mulde multi-resolution Gaussians-on-Mess representation. We evaluate the proposed approach on the challenging T'Human.00, XHuman and AIST+ data. Our approach reconstructs an animatable representation from sparse inputs in less than Is, renders views with 95. IFPS at 1024 x 1024, and achieves PSNR/L PERR/PIIPS//FIIII of 24.65/1 10.82/51.27 on THuman2.0, O, outperforming the tfateof-the-an art rendering quality.
INTRODUCTION
Generalizable rendering of an animatable human avatar from sparse inputs, i.c. images showing human in the same clothing and environment but not necessarily the same pose, is important problem for augmented and virtual reality applications. Envision generation of an animatable avatar from few quickly taken pictures in unconstrained environment and efficient yet high-quality pose-conditioned rendering in a virtual world.
To address this application, recent approaches (Kwon al. 2024 Zheng al. 2024 Li et al. 2024
Hu et al.) 2023 Pan et al.) 2023) resort to generalizable reconstruction methods. Generalizable methods avoid scene-specific optimization at inference time but instead use just a single deep forward pass, making reconstruction efficient. During an offline training phase the deep net extracts data priors and inductive biases from reasonably large dataset. Due to the learned priors, can be applied to sparser inputs compared to scene-specific training.
For rendering, recent methods (Wen et al. 2024: Paudel al.] 2024 Guédon & Lepetit) 2024) introduce a dual shape representation, combining the advantages of mesh, i.e., regularization via the manifold neighborhood connectivity induced by the triangle mesh, with those of Gaussian splats, i..., fast and flexible rendering.
However, the use of just a single deep net forward pass during reconstruction prevents present- day methods from refining their prediction. This is concern because apparent emrors that can be detected by comparing available inputs to a corresponding rendering of the reconstruction are not utilized. Moreover, w.r.t. the dual shape representation for human rendering, GoMAvatar (Wen) cet al. 2024) and iHuman (Paudel et al. (024) employ identical resolutions for the underlying mesh and Gaussians, i.e., Gaussian for each triangle face in the mesh. This is concern because

---

Published as a conference paper at ICLR 2025
Multiview images
Novel view synthesis
OR
Multi-frame images
Novel pose Synthesis
Coupled multi-resolution
Inputs: sparse source images
Gaussianson--Mesh 908 ms
Rendering 95 FPS
Figure/ I: Overview. We tackle the problem of generalizable human rendering. Given sparse source images (multiview images or multi-frame images), we reconstruct the 3D human representation in canonical T-pose space. The canonical representation can be animated and rendered in novel views.
reasonably low-dimensional mesh representation is desirable for efficient reconstruction, while high-dimensional Gaussian splat representation desirable for high-quality rendering. GaussianA- vatar (Qian et al. 2024a)
adaptive density control based gradients to densify Gaussians on the mesh. However, generalizable human rendering reconstructs and renders subjects in a feed- forward pass and therefore gradients unavailable guide the densification.
To address the first concern not leveraging apparent errors, we propose a novel iterative feedback- based reconstruction network. The iterative update mechanism augments generalized methods via a feedback mechanism to improve results by fusing information from inputs, the current 3D recon- struction, and current rendering from input views. Importantly, the designed iterative update mech- anism is end-to-end trainable, i.e., the feedback is taken into account when training the generalized reconstruction. Note that the iterative update mechanism makes reconstruction slightly slower, yet our un-optimized version still performs the task in less than one second. Since reconstruction is one-off task, independent from pose-conditioned rendering, we think it makes sense to spend bit more effort than a simple deep network.
To address the second concern, we study ponpled-muuti-resollee Gaussians Morssiinn-o--Meeees on-Mesh represen- tation. More specifically, reconstruction is performed with low-resolution mesh while we increase the number Gaussians by attaching multiple ones to single triangle face. This is achieved sub-division-like procedure. Beneficially, reconstruction remains efficient while rendering can achieve high-quality and high-resolution results.
We illustrate our method in Fig. I and observe compelling rendering quality and speed. We assess the efficacy of the proposed method on the challenging THuman2.0, XHuman and AIST+ data. As mentioned, reconstruction needs less than second and rendering runs at 95.1 FPS on one NVIDIA A100 GPU. The rendering quality of the designed method outperforms the sfate-of-the- for GHG (Kwon)
improving PSNR/LDPIPS/FIDD 24.65/1 et al. 2024).
1.27 from
RELATED WORK
Rendering of human avatars can be broadly categorized into two main areas: 'per-scene optimized human rendering' and "generalizable human rendering'. We review both areas next before dis- cussing prior work on dual shape representations, which combine Gaussians and meshes, and itera- tive networks.
2

---

Published as a conference paper at ICLR 2025
Per-scene optimized human rendering. Human rendering from multiview monocular videos has achieved great results in recent years, benefitting from progress in neural rendering, e.g., neural radiance fields (NeRF) (Mildenhall et al. 2020) and Gaussian splatting (Kerbl et al. 2023).
NeuralBody (Peng et al. 2021) is one of the earlier works that explores NeRFs for human rendering. It regresses the colors and opacities based on the latent codes associated with the vertices of deformable mesh. HumanNeRF (Weng et al.) (022) learns subject-specific representations from a monocular video and improves over prior works by introducing non-rigid transformations. Followup NeRF-based works further improve the rendering quality (Yu et al. 2023), training speed (Geng
al. 2023 Jiang al. 2023), and rendering speed (Jiang et al. (023). Later, Gaussian splatting was
adopted by human rendering techniques due to its superior rendering speed (Lei et al.) [224] Wen
ct al. 2024
Hu et 2024 Kocabas et al., 2024 Li al. 2023 Paudel et al. 2024). Human (Li
et al. 2023) advances the training speed to ~ 100s on ZJU-MoCap and MonoCap. iHuman (Paudel
jt al.
2024) further improves the training speed to 12s on PeopleSnapshot and can be trained on
as few 6 frames. Even though the training speed improves significantly when using Gaussian splatting, real- world applications often prefer sub-second training times. Moreover, without learned priors from large-scal datasets, per-scene optimization approaches suffer from overfitting when the training views are sparse.
Differently, in this work, we adopt the dual shape representation introduced by GoMAvatar (Wen et al., 2024) and adapt it to generalizable human rendering. This permits to reconstruct the 3D representation in less than one second and further excels even if only sparse inputs are available.
Generalizable human rendering. Generalizable human rendering operates on sparse source views and benefits from learned priors and inductive biases extracted during a training phase from large- scale datasets. In addition, it has a greater potential to attain a faster speed when recovering a 3D representation from the source views. ActorsNeRF (Mu al. 2023) combines per-scene opti- mization with priors learned from large-scale datasets using a two-stage training. Diffusion-bard -ased approaches and large-reconstruucion model-based methods (Weng al.) 2024; Chen et al. 2024 Xue et al. 2024 Kolotouros et al., 2024; Pan et al. (024) denoise the multiview images or other properties. Since requires multiple steps for each denoising process, diffusion-
approaches usually take 2-10s to reconstruct the human avatar from images. Another line of works (Remelli ctt al. 2022 Hu al., 2023 Kwon et al., 2021 2023 Li et al., 2024 Pan et al., 2023 Zheng et al. 2024 build single reed-forward approach recover 3D representation. They operate source views and output a 3D representation for novel view rendering. Without evaluating the network several times, feed-forward methods are much faster compared diffusion-based methods.
Our approach falls in the feed-forward category. However, differently, we devise an end-to-end trainable iterative feedback module to improve performance. As show quantitatively and quali- tatively in Section 4 our approach achieves better rendering quality compared to prior feed-forward methods, while not being significantly slower.
Gaussians-on-Mesh dual shape representation. Though Gaussian splatting alone achieves supe- rior rendering quality and speed, it suffers from overfitting when a good position initialization is not available (Wen et al., 2024) and its underlying geometry is less accurate (Paudel et al.) 2024] Qian et al.] 2024a). Prior work (Wen al., 2024 Paudel et al.] 2024) regularizes the Gaussians and enables animation using parametric models is such as FLAME (Li et al., 2017) and SMPL (Loper jet al. (2015). We also combine Gaussian splatting with a mesh. Different from the use of one Gaus- sian per face by Wen et al. (2024) and Paudel et al. (2024). we adopt coupled-multi-resoutiin representation: a Jow-resolution mesh is deformed and Gaussians are linked to a high-resolution mesh. Different from Qian et al. (2024a), who split the Gaussians based on gradient signals, we subdivide the mesh and bind the Gaussians on the subdivided mesh since gradients are unavailable in our generalized human rendering setting which uses only foed-forward pass. SuGaR (Guédon & Lepetit 2024) works on general static scenes and attaches multiple Gaussians to each triangle based on predefined barycentric coordinates. However, the Gaussians' scales learned in the world co- ordinates, while we define Gaussian parameters in a triangle's local coordinates. This modification important for modeling dynamic scenes.
Iterative network. Our approach falls into the category of iterative feedback networks (Adler & Öktem 2017; Manhardt et al. 2018 Carreira et al. 20166] Li et al., 2018 Ma et al., 2020). The core idea is to learn to iteratively update the output through a forward process. This method works
3

---

Published as a conference paper at ICLR 2025
particularly well when feedback signals can be incorporated at each step to improve the estimation.
Previous works either unrolled standard optimizers into differentiable feedforward networks (Wang
et al. 2016 Belanger & McCallum 2016 Schwing & Urtasun 2013] Zuo & Deng 2025)
explic-
itly optimizing an energy function, or trained a generic iterative network with supervised learning without explicit energy formulation (Andrychowicz et al.] 2016 Wichrowska et al. [0077 Flynn it al.) 2019 Teed & Deng 2020). In computer vision, these methods have been used for pose estimation (Li et 2018: Carreira et al., 2016), inverse problems (Ma al.) 2020), dense recon- struction (Flynn et al. 2019). optical flow (Teed & Deng 2020), and depth estimation (Zuo & Deng.
2025]. Our work presents a novel use of this iterative framework for generalizing human avatars.
3
METHOD
In the following we first provide an overview of the proposed approach in Section 3.1 We then detail our two contributions: first the Section 3.2 and then our reconstruction approach with iterative feedback in Section 3.3 Finally provide some information on training of the proposed method in Section 3.4
Gaussiansonn-Mes representation in
3.1 OVERVIEW
Input. The proposed method operates on set of source images {I J corresponding binary source masks {M,) identifying the human, source camera extrinsics { 1 In=1 source camera intrinsics { {K,, \. n=1 and human poses (Px J Here, N is the number of source images. The human pose Pn RR2, T is represented by a collection of rotations Rt and translations TV
Output. Given this input, goal is render the target image I and its corresponding binary mask Mpred given as additional input the target camera extrinsics Egg, intrinsics Kig. and the target human pose Pig. again specified via collection of J rotation matrices and translation vectors.
Method overview. We render Ip and M by transforming learned canonical Gaussian-on- Mesh representation GoM specified in a T-pose space. For this, Gaussians and mesh (i.e., GoM") first articulated using the target pose Pig and subsequently transformed to target image space via the target camera parameters. We provide details in Section 3.2 and formally write this as
ppred ig
Mpred ReRdeeeeerreee (GoM Pig: Fug. Kig).
The canonical 3D representation GoM is extracted from the N source images. We abstract this via
GoM Reconstructor
[M (Pr) W. (K))
and provide details in Section 3.3 Unlike Gas--aas (Zheng al.. 2024). we choose re- construct the subject in the canonical T-pose instead of the poses provided as an input. Benefitting from this choice, our representation can be retargeted novel poses without any post-processing, such skeleton binding. Further, our model can operate images showing different poses. No- tably, our GoM representation uses different resolutions for the Gaussians and the mesh and the Reconstructor benefits from iterative feedback update.
3.2 COUPLED--UULT--RESOLL
GAUSSIANS-ON-MA REPRESENTATION
In this section, we describe the details the Renderer used in Eq. TT. We first define the coupled- multi-resolution Gaussians-on-Mel representation in Section 3.2.1, which refers to our canonical
T-pose shape. Next, we detail articulation and rendering Section 3.2.2 and Section 3.2.3
3.2. CANONICAL REPRESENTATION
The classic Gaussiansoon-Mesh (GoM) representation associates one Gaussian with one tri- angle face of mesh, i.e.. the number Gaussians is identical the number of tri- angle faces. Further note, in GoMAvatar (Wen et al.] 2224], the vertices of the mesh and the Gaussians' parameters in the triangle's local coordinates are optimized per scene.
4

---

Published as a conference paper at ICLR 2025
To achieve high-quality rendering, GoMAvatar subdivides the mesh to increase the number of Gaussians. However, in the generaliz- able human rendering setting, naively subdi- viding the mesh significantly increases the construction time from less than Is to ~13 since the network operates on larger of points. We therefore study the coupled-multi- resolution Gaussians-on-Me on-Mesh representation. It reduces the computational cost while simulta- neously improving the rendering quality. Con- cretely, we achieve this by deforming the ver- tices a low-resolution mesh and attaching the Gaussians coupled high-resolution mesh.
TJ. O.
Vowures veree
Subdivide
Figure 2: Multi-resolution Gaussians-on-Mest representation. We use a low-res mesh for faster animation and simpler geometry and attach Gaus- sians on a high-res mesh for better rendering.
Formally, we define the coupled-muline multi-resolutionn Gaussians-on- representation in the canoni- cal space follows:
GoM {(oft)) Ji=1, (w 1ff
Here, {u, Jim1 and {fg define the V4 vertices and F faces of the low-resolution mesh spectively. Note, f3 (14) 1 where A E (I...., v+] is the k-th vertex index of the j-th triangle in the low-resolution mesh. To articulate it to any given human pose, we utilize linear blend skinning weights w. E R corresponding to the i-th vertex oy in the low-resolution mesh.
The high-resolution mesh is specified via (uf)y= and { { which subsume the V vertices and F faces. These are obtained by subdividing the low-resolution mesh. Different from the low-resolution mesh representation, we attach Gaussians to the high- reghurreooution face fj, i.c.,
fy - (rz, Sy. Sy. Cj. oj. (==))
with j E (1....: F). Here, ry E 80(3) and 8y TR3 are the rotation and scale in the faces's local coordinate system. Moreover, e R is the RGB color, oy is the offset defined in the faces's local coordinate system, and [Aj, are the three vertex indices belonging the j-th triangle, i.e.. Aj.k E (1...., V). We illustrate the representation in Fig.
3.2.2 ARTICULATION
remains to answer 1) how we transform the defined coupled- multi--esoottiiionn Gaussians-on- Mesh representation to the target pose; and 2) how we perform rendering. To answer the first question, given a target pose Pig. we articulate the canonical coupled-multt--esouutio Gaussians-on-Meshh representation GoM to a Gaussians-on-Meshh representation GoM - (1o JeE ( (u (ا}}ー in the pose space utilizing linear blend skinning. Note that this representation is still multi-resolution because linear blend skinning is performed in the low-resolution space for efficiency reasons while high-quality rendering requires high-resolution Gaussian information. Coneretely, we transform the canonical low-resolution 3D vertex coordinates ei to posed low-resolution 3D vertex coordinates
Σj- (R) 4 + +t))
10t LBS (0,), wt, PiE)
(5)
Elnw" w.
Here, LBS refers to classic linear blend skinning. Since the high-resolution canonical space mesh (o } is obtained from the low-resolution canonical space mesh via subdivision, straightforward to transfer the vertex transformations between the posed low-resolution 3D vertex
coordinates o
e4 and its canonical counterpart 2i the high-resolution mesh and obtain {u?) V
3.2.3 RENDERING WITH GAUSSIAN SPLATTING
Given the pose space Gaussians-on-Mer Meshh representation GoM and the target camera parameters Fag and Keg, we render the target image yared and the mask MPree with Gaussian splatting.
5

---

Published as a conference paper at ICLR 2025
Human Poses
Iterttvve update at
Updated G
Mesh decoder
Itcrative feedback net
Feedback feature !
Camera Params
Articulate & Render
(Pt)
GSplat decoder
Input Images
Rendered Images
xT
Figure 3: Iterative feedback. We iteratively update in a feed-forwaid way the vertices of the low- resolution mesh and the Gaussian parameters attached to the high-resolution mesh. We repeat the update for 7 steps. Each step operates on the source images, camera parameters and human poses, as well as the last iteration's 's results including the canonical representation GoM, and the predicted source images rendered by G (the brown arrows).
Our Gaussian parameters defined in Eq. (4) are located in the triangle's local coordinates. To render the images, we first transform the local Gaussian parameters to the world coordinates. Following Go- MAvatar (Wen et al. 2024), we denote the scal-tt-worlle transformation of the j-th high-resolution face as Ay. The mean the Gaussian and its covariance are computed via
3
19
E
S + A5.
and = AR
where Rg and Sy are the matrices encoding rotation ry and scale sj. The color of the Gaussian is cy.
3.3 RECONSTRUCTION WITH ITERATIVE FEEDBACK
It remains to answer how reconstruct the canonical space coupled-muilti-resolutee Gaussians- on-Mesh representation GoM For this, our Recons reconstructeo ructor defined in Eq. ( uses sparse source images [_,) and masks (M, n=1- Note that the sparse inputs be multiview im- ages or multi-frame images sampled from a monocular video, where human poses are not neces- sarily identical across frames. We also assume that human poses {Pr 1N and camera parameters {E,, | { K, J are given which can be human- annotated or predicted from off-the-shelf tools.
We reconstruct the canonical representation in T-pose rather than the input poses, enabling animation without any post-processing and allowing the model to handle images in unaligned poses. The added difficulty due to this choice: the gap between the canonical pose and the input poses. While scene- specific methods refine the canonical representation with gradient-based optimization, generalizable approaches must predict it in feed-forward pass which leads undesired reconstruction quality. To address this challenge, we propose iterative feedback updates that successively 'refine" the canonical representation in feed-forward manner, as illustrated in Fig. 3
To compute GoM we perform T step iterative feedback update. We use GoM, to denote the output representation from the f-th step, i.c., t E {O...., T} and let
GoM A ( (w Uن (u (nnb
Here, the step-dependent face information given by
fi. (re.j: St.j, Ct.j oz, 12=)) with j (1....,,FF)
(8)
Note, GoMy, the canonical representation at /. 0, is the initialization and GoM = GoM
We emphasize that our iterative feedback updates the low-resolution mesh vertices ) V and the Gaussian parameters (r3, 8j; Cj. 0z f associated with the high-resolution faces. The vertices in

---

Published as a conference paper at ICLR 2025
the high-resolution mesh {ur V follow the low-resolution update, analogously to the articulation
update discussed in Section 3.2.2
At each step t, update the low-resolution mesh vertices and high-resolution Gaussian parameters using the following equations:
0 u/-1, + (L. Te.j Ct. Or.j MLP(cat(Fe.,, (Pott ())))))=)))
(10)
Here, Pr, E (1,..., V43 is our 'feedback" feature for the i-th vertex in the low-resolution mesh. Further, Fi.j.j E f1...., F} in Eq. (10) is a 'feedback' feature for the j-th face in the high- resolution mesh. It is acquired by first interpolating Ft. i {1...., V+} to get vertex features in the high-resolution mesh and then concatenating the 3 vertices' features belonging to the j-th face. To preserve details, also concatenate source image features {Pnn,.) which are obtained by projecting the mean of the j-th Gaussian at step to the n-th view.
To compute the 'feedback' feature Ft i E (1...., V4), we first render the source views using the canonical representation from the last iteration via
1,1-1 M M
n Renderer
En, Kn), n E
(11)
Then we extract image features from [ For each vertex o4 in the low-resolution mesh, extract pixel-aligned source image features and predicted image features. We concatenate both and feed them into an iterative feedback network. The iterative feedback network consists of multi-source fusion block that mixes the information from N sources, and Point Transformer that encodes all the vertices. Its output feature is (Ft, {Fit)e Please refer to Appendix for more details.
3.4 TRAINING
Both rendering and reconstruction using our iterative feedback network and coupled representa- tion are end-to-end differentiable. To learn the network parameters, we use training loss com- posed of LI and perceptual losses, comparing predicted and ground-truth RGB images, LI loss for masks as well as a Laplacian loss for regularization. The loss is averaged over all source and target images, well as all T iterative feedback steps. Formally, we minimize the average of Lossn,t, n (I,...,,,1t//
{I...., and
Loss,,, = (1 ( + Aper Perceeeptuull (Nn, 1 + L MAT + >mnLLap(GoM)). (12)
Here, Li(:-) the LI loss. Perceptual ()) is the perceptual loss between predictions and ground- truths, e.g., SSIM or LPIPS. Lap(-) is the Laplacian loss applied on the low-resolution mesh of the canonical GoM representation. Aper. AM and Alap are user-specified hyperparameters.
4 EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
Representation details. We initilize the low-resolution mesh (16t))) (. in GoMi with SMPL SMPL-X, depending the human pose representation used in the dataset. The high- resolution mesh is obtained by subdividing the low-resolution mesh.
Architecture details. We provide the detailed architecture in Appendix
Training details. We set Aper = 1.0, Am 5.0 and Atap 100 Eq. 12) on THuman2.0 and Aper 1.0, AM 0 and Atpp 100 in Eq. 12) AIST++ We use the SSIM loss in THuman2.0 0 and the LPIPS loss in AIST++ following the baselines. We use Adam as the optimizer. On THuman2.0, the learning rates of the image encoder and the rest of the model are 1e-4 and 5e-5 respectively On AIST++, we set the learning rate of all parameters to 5e-5. We optimize the model for 200K iterations THuman2.0 and 100K iterations AIST++
7

---

Published as a conference paper at ICLR 2025
Table 1: Comparison on THuman2.0. The proposed method improves state-of-the-art in PSNR, LPIPS" and FID. We highlight the best result in bold font. Methods marked in gray are per-scene optimized methods.
Number of source views
Method
PSNR1
LPIPS
FID
GoMAvatar Wen et al.
2024
23.05
133.98
87.51
3DGS-Avatar Qian et 2024b
21.25
160.48
157.21
iHuman Paudee
22.77
131.67
101.70
NHP <Wwon et al. 3
2021
23.32
184.69
136.56
NIA Kwon et al.
20231
23.20
181.2
127.30
GHG (owon ct al. 2024
21.90
133.41
61.67
LIFe-GoM (Ours)
24.65
110.82
51.27
GPS-Gaussian (Zheng et al.
2024)
20.39
152.34
65.90
LIFe-GoM (Ours)
25.57
105.39
38.57
Table 2: Comparison on XHuman. We evaluate on XHuman to prove the ability of cross-domain
generalization. The proposed method improves state-of-the-an in PSNR, LPIPS and FID. We highlight the best result in bold font.
Method
PSNR
LPIPS*
FID
GHG (Kwon et al. 2024 LIFe-Gole MeFe-GMMin (Ours)
23.52
112.91
50.51
25.32
99.32
42.90
4.2 EXPERIMENTAL. SETUP
We evaluate our approach in two settings: 1) Multiview source images. Our approach can take multiview images as input to produce canonical representation; 2) Multi-frame source images. Since our approach directly learns 3D representation in the canonical space instead of a posed space, method can also operate images showing various human poses, e.g., frames sampled from a monocular video. Our approach can synthesize both novel views and novel poses.
Datasets. We validate approach THuman2.0 Yu et al. 2021). XHuman (Shen et al. 2023) and AIST+ (Li et al. 2021) quantitatively. We use THuman2.0 to evaluate our approach in the setting of multiview source images. XHuman is used to validate the cross-domain generalization of our approach. In other words, we train our model on THuman2.0 and test on XHuman without fine-tuning. The AIST++ dataset is used to evaluate the multi-frame source image setting. Please see Appendix for detailed dataset setup.
Baselines. We compare with GoMAvatar (Wen et al. 2024 3DGS-Avatar Qian et al. 2024b) iHuman (Paudel et al., 2024), NHP (Kwon et al.) 2021), ,, NIA (Kwon et al. 2023), GHG K Kwon al. 2024) and GPS- Gasssaasn (Zheng et al. 2024) on THuman2.0. On AIST+ we compare with
HumanNeRF (Weng et al. (2022), GoMAvatar (Wen et al. (024)). 3DGS-Avatas (Qian et al. (024b),
iHuman (Paudel et al. 2024) and ActorsNeRF (Mu et al. 2023). Please refer to Appendix for details.
Evaluation metrics. We report PSNR, (PIPS' LPIPS 1000) and FID on THuman2.0 fol- lowing GHG (Kwon et al. 2024). We report PSNR, SSIM and LPIPS on AIST+ following Ac- torsNeRF (Mu et al. 2023).
4.3 QUANTITATIVE RESULTS
THuman2.0. We summarize our results in Tar both the three-view and the five-view setting.
In the three-view setting, our method significantly outperforms per-scene optimized methods includ- ing GoMAvatar, 3DGS-Avatee and iHuman, and generalizable approaches including NHP, NIA, and GHG in PSNR, LPIPS* and FID. Our approach achieves 24.65/1 82/51.27 in PSNR/LPIPS*/FI compared to GHG's 90//133.4/66.....
Importantly, we use 330K Gaussians for splatting, 7.5x 2.8M. resulting in faster rendering (10.52ms vs. GHG's s 20.30ms) at 1024 x 1024
fewer than GHG's s resolution on a NVIDIA A100 GPU. Our method takes 907.92ms to reconstruct the coupled-multi- resolution Gaussians-on, -n-Mssh in canonical space, significantly faster than scene-specifi. methods but slower than GHG. That said, reconstruction only needs to be done once per input subject, as the reconstructed avatar will be cached and reused for articulation and rendering, which runs at 95 FPS.
8

---

Published as a conference paper at ICLR 2025
Table 3: Comparison on AIST++ We achieve comparable quality as ActorsNeRF while requiring much less time in reconstruction optimization. We highlight the best result in bold font. Methods marked in gray are per-scene optimized methods.
Reconstruction or optimization time.
Method
PSNRT
SSIM
LPIPS*
HumanNeRF [Weng
2022
24.21
0,9760
29.66
~h
GoMAvatar (Wen et al
24.34
0,9780
25.34
~10h
3DGS-Avatar Oian et 2024b
25.14
0,9784
27.17
~min
illuman Paudel al.
25.17
0,9805
22.90
6.61s
ActorsNeKI (Mu et al.
2023
25.23
0,9809
22.45
.4h
LIFe-GoM (Ours)
25.25
0.9812
21.61
589.27ms
(a) Reference image
(b) Novel view synthesis
(a) Reference image
(b) Novel pose
synthesis
Figure 4: Cross-domain generalization on DNA-Rendering dataset w/o finetuning.
Figure Novel pose synthesis. Poses are from BEDLAM dataset.
We compare our approach to GPS-Gaussian using five images. As GPS-Gaussian relies on depth prediction between adjacent views, five images are the minimum it needs. Despite that, still fails in non-overlapping regions. Our approach significantly improves upon GPS-Gaussian in this setting.
We directly apply GHG and our approach trained on THuman2.0 in the setting of source views to the XHuman dataset without any finetuning. Our approach achieves PSDR/LPIPS$/FID of 25.32/99.32/42.99
XHuman. We summarize the cross-dataset generalization results in Table [
significantly outperforming GHG's 23.52/112.91/50551
AIST++ Table summarizes quantitative results on AIST++ Our method achieves 25.25/0.9812/21..6 in PSNR/SSIM/LPIPS* matching ActorsNeRF's 25.23/0.9809/22 passing per-scene optimized methods. Importantly, our method needs only 589 ms for 3D recon- struction, whereas iHuman, the fastest scene-specific method, requires 6.61s and other baselines take minutes to hours.
and sur-
4.4 QUALITATIVE RESULTS
Please refer Appendix E for more qualitative results, including a comparison baselines.
Cross-domain generalization. We demo our approach on cross-domain generalization in Fig. us- ing the AeA-Rendg data (Cheng al., 2023). Without fine-tuning, our approach can generalize to challenging subjects, e.g., loose clothes.
Novel pose synthesis. Instead of directly reconstructing human avatars in the pose of the source images, our approach outputs the canonical representation in T-pose via the Reconstructa. Benefitting from this choice, we can synthesize novel poses without postprocessing such as binding the skeletons. In Fig. 5 we retarget the avatar to challenging new pose sequences from the BEDLAM dataset (Black et al.) 2023). The avatar reconstructed using the model which was used to report results in the source view setting of Table T
4.5 ABLATION STUDIES
Analysis of iterative step choice. We study how the number of iterations (T) influences the recon- struction time and rendering quality. Results are summarized Table 4 and Fig. 6[a). Note that T = 1 means single feed-forward pass, i.e., iterative updates are disabled. Using more iterations improves the rendering quality at the expense of more reconstruction time (~2290ms per iteration). The PSNR improves by +0.78 and +0.91 when T and T = 3 respectively compared to T = 1. Starting with T 4, the benefit more iterations diminishes. We choose = 3 in our final model to balance rendering quality and reconstruction time.

---

Published as a conference paper at ICLR 2025
Table 4: Iterative step choice. More iterations lead to better rendering at the expense of longer reconstruction. We use iterations for the best quality-speed tradeoff, highlighted in gray.
Reconstruction time (mS)
iterations
PSNR
LPIPS
FID
23.74
124.58
64.59
328.77 79
2
24.52
112.47
52.16
618.67
24.65
110.82
51.27
907.92
24.69
110.46
51.25
1198.14 14
24.70
110.38
51.02
1563.92
(d) Ground truth
(a) Subdiviee
(b) Subdivide
(c) Subdivide
(d) Ground truth
(a) T-1
T-2
(c) T-3
x0
x1
x2
Figure 6: Ablation studies. We study the effect of iterative feedback (left). The geometry improves as the number of iterations increases, We show the importance of linking Gaussians to the high- resolution mesh (right). The high-resolution mesh is subdivided from the low-resolution counterpart. A higher resolution yields better texture details.
multi-resolluionn Gaussians-on-Mesh. Increasing the number of subdivisions improves rendering quality the cost of longer reconstruction and rendering times. We subdivide twice in our final model to ensure quality while maintaining real-time, as highlighted in gray.
Table 5:
Reconstruction time (ms)
Rendering time (ms)
subdivision
PSNR
LPIPS#
FID
0
24.76
140.60
93.44
538.02
3.20
1
24.88
118.64
58.45
607.49
3.93
2
24.65
110.82
51.27
907.92
10.52
Coupled-multi-resolttie Gaussaas-on-Meshh As mentioned in Section 3.2.1 and Section 3.3 we update the vertices of the low-resolution mesh, while the Gaussians are associated with the high-resolution mesh. Both updated jointly. This choice necessary for reasons: 1) simply updating the vertices of the high-resolution mesh increases the reconstruction time from 907.92m 12.45s, making too slow for both training and inference; 2) learning Gaussians in the high-resolution mesh guarantees good rendering quality. Note that the high-resolution mesh is obtained by subdividing the low-resolution mesh. In Table we show that the rendering improves to 1 118.6 188.6/588.45 and 1 110.82/51.27 in LPIPS* /FID when subdividing once and twice respectively from 440.60/93.44 without subdivision. The improvement can also be observed in Fig. ((b). Note that we do not observe consistent improvement in PSNR. This is because PSNR sometimes prefers blurry results. The resolution of the high- high-resolution mesh affects both the reconstruction speed and the rendering speed since render the souree images during the reconstruction stage. As the reconstruction time is still less than 1s, we choose to subdivide twice for better rendering quality.
5 CONCLUSIONS
We tackle the problem of generalizable reconstruction of an animatable human avatar from sparse inputs. We propose feed-forward network featuring iterative updates with iterative feedback and coupled-multi-resoltttoo
Mauusiaass-on-Mes representation. Our method achieves state-of-the-re rendering quality. It requires less than 1s for avatar reconstruction and renders at 95 FPS.
Acknowledgements. Work supported in part by NSF grants 2008387, 2045586, 2106825, MRI 1725729, and NIFA award 2020-67021-32799.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Jonas Adler and Ozan Öktem. Solving ill-posed inverse problems using iterative deep neural net- works. Inverse Problems, 2017.
Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. Learning to learn by gradient descent by gradient descent. NeurlPS, 2016.
David Belanger and Andrew McCallum, Structured prediction energy networks. ICML, 2016.
Michael J. Black, Priyanka Patel, Joachim Tesch, and Jinlong Yang. BEDLAM: A synthetic dataset of bodies exhibiting detailed lifelike animated motion. In CVPR, 2023.
Joao Carreira, Pulkit Agrawal, Katerina Fragkiadaki, and Jitendra Malik. Human pose estimation with iterative error feedback. In CVPR, 2016.
Jinnan Chen, Chen Li, Jianfeng Zhang, Hanlin Chen, Buzhen Huang, and Gim Hee Lee. Generaliz- able human gaussians from single-view image. arXiv, 2024.
Wei Cheng, Ruixiang Chen, Siming Fan, Wanqi Yin, Keyu Chen, Zhongang Cai, Jingbo Wang. Yang Gao, Zhengming Yu, Zhengyu Lin, et al. Dna-rendering: A diverse neural actor repository for high-fidelity human-centric rendering. In ICCV, 2023.
John Flynn, Michael Broxton, Paul Debevec, Matthew Vall,, Graham Fyffe, Ryan Overbeck, Noah Snavely, and Richard Tucker. Deepview: View synthesis with learned gradient descent. In CVPR, 2019.
Chen Geng, Sida Peng, Zhen Xu, Hujun Bao, and Xiaowei Zhou Learning neural volumetric representations of dynamic humans in minutes. In CVPR, 2023.
Antoine Guédon and Vincent Lepetit. Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. In CVPR, pp. 5354-53666 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CVPR, 2016.
Liangxiao Hu, Hongwen Zhang, Yuxiang Zhang, Boyao Zhou, Boning Liu, Shengping Zhang, and Liqiang Nie. Gaussianavatar: Towards realistic human avatar modeling from a single video via animatable 3d gaussians. In CVPR, 2024.
Shoukang Hu, Fangzhou Hong, Liang Pan, Haiyi Mei, Lei Yang, and Ziwei Liu. Sherf: Generaliz- able human nerf from a single image. In ICCV, 2023.
Tianjian Jiang, Xu Chen, Jie Song, and Otmar Hilliges. Instantavatar: Learning avatars from monoc- ular video in 60 seconds. In CVPR, 2023.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3D Gaussian Splat- ting for Real-Time Radiance Field Rendering. ACM TOG, 2023.
Muhammed Kocabas, Jen- Hao Rick Chang, James Gabriel, Oncel Tuzel, and Anurag Ranjan. Hugs: Human gaussian splats. In CVPR, 2024.
Nikos Kolotouros, Thiemo Alldieck, Enric Corona, Eduard Gabriel Bazavan, and Cristian Smin- chisescu. Instant 3d human avatar generation using image diffusion models. In ECCV, 2024.
Youngjoong Kwon, Dahun Kim, Duygu Ceylan, and Henry Fuchs. Neural human performer: Learn- ing generalizable radiance fields for human performance rendering. NIPS, 2021.
Youngjoong Kwon, Dahun Kim, Duygu Ceylan, and Henry Fuchs. Neural image-based avatars: Generalizable radiance fields for human avatar modeling. ICLR, 2023.
Youngjoong Kwon, Baole Fang. Yixing Lu, Haoye Dong, Cheng Zhang, Francisco Vicente Car- rasco, Albert Mosella-Montoro, Jianjin Xu, Shingo Takagi, Dacil Kim, al. Generalizable human gaussians for sparse view synthesis. ECCV, 2024.

---

Published as a conference paper at ICLR 2025
Jiahui Lei, Yufu Wang. Georgios Pavlakos, Lingjie Liu, and Kostas Daniilidis. Gart: Gaussian articulated template models. In CVPR, 2024.
Chen Li, Jiahao Lin, and Gim Hee Lee. Ghunerf: Generalizable human nerf from monocular
video. In 3DV, 2024.
Mingwei Li, Jiachen Tao, Zongxin Yang, and Yi Yang. Human101: Training 100+ fps human gaussians in 100s from view. arXiv, 2023.
Ruilong Li, Shan Yang, David A Ross, and Angjoo Kanazawa. Learn to dance with aist++; Music conditioned 3d dance generation. In ICCV, 2021.
Tianye Li, Timo Bolkart, Michael. Black. Hao Li, and Javier Romero. Learning model of facial shape and expression from 4D scans. SIGGRAPH Asia, 2017.
Yi Li, Gu Wang, Xiangyang Ji, Yu Xiang, and Dieter Fox. Deepim: Deep iterative matching 6d pose estimation. In Proceedings of the European Conference Computer Vision (ECCV), pp- 683-698, 2018.
Tingting Liao, Xiaomei Zhang, Yuliang Xiu, Hongwei Yi, Xudong Liu, Guo-Jun Qi, Yong Zhang, Xuan Wang, Xiangyu Zhu, and Zhen Lei. High-fidelity clothed avatar reconstruction from single image. In CVPR, 2023.
Tingting Liao, Hongwei Yi, Yuliang Xiu, Jiaxiang Tang, Yangyi Huang, Justus Thies, and Michael Black. Tada! text to animatable digital avatars. In 3DV, 2024.
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael Black. SMPL: A skinned multi-person linear model. ACM TOG, 2015.
Wei-Chiu Ma, Shenlong Wang, Jiayuan Gu, Sivabalan Manivasagam, Antonio Torralba, and Raquel Urtasun. Deep feedback inverse problem solver. In ECCV, 2020.
Fabian Manhardt, Wadim Kehl, Nassir Navab, and Federico Tombari. Deep model-based 6d pose refinement rgb. In ECCV, 2018.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV, 2020.
Jiteng Mu, Shen Sang. Nuno Vasconcelos, and Xiaolong Wang. Actorsnerf: Animatable few-shot human rendering with generalizable nerfs. In ICCV, pp. 18391-18401, 2023.
Panwang Pan, Zhuo Su, Chenguo Lin, Zhen Fan, Yongjie Zhang, Zeming Li, Tingting Shen, Yadong Mu, and Yebin Liu. Humansplat: Generalizable single-image human gaussian splatting with structure priors. arXiv, 2024.
Xiao Pan, Zongxin Yang, Jianxin Ma, Chang Zhou, and Yi Yang. Transhuman: A transformer-based human representation for generalizable neural human rendering. In CVPR, 2023.
Pramish Paudel, Anubhav Khanal, Ajad Chhatkuli, Danda Pani Paudel, and Jyoti Tandukar. ihuman: Instant animatable digital humans from monocular videos. arXiv, 2024.
Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In CVPR, 2021.
Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, and Matthias Nießner. Gaussianavatars: Photorealistic head avatars with rigged 3d gaussians. CVPR, 2024a.
Zhiyin Qian, Shaofei Wang, Marko Mihajlovic, Andreas Geiger, and Siyu Tang. 3dgs-avatar; Ani- matable avatars via deformable 3d gaussian splatting. In CVPR, 2024b.
12

---

Published as a conference paper at ICLR 2025
Edoardo Remelli, Timur Bagautdinov, Shunsuke Saito, Chenglei Wu, Tomas Simon, Shih-En Wei, Kaiwen Guo, Zhe Cao. Fabian Prada, Jason Saragih, and Yaser Sheikh. Drivable volumetric avatars using texel-aligned features. In SIGGRAPH, 2022.
Alexander G Schwing and Raquel Urtasun. Fully connected deep structured networks. arXiv, 2015.
Kaiyue Shen, Chen Guo, Manuel Kaufmann, Juan Zarate, Julien Valentin, Jie Song, and Otmar Hilliges. X-avatar: Expressive human avatars. In CVPR, 2023.
Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In ECCV, 2020.
Shenlong Wang, Sanja Fidler, and Raquel Urtasun. Proximal deep structured models. NeurlPS, 2016.
Jing Wen, Xiaoming Zhao, Zhongzheng Ren, Alex Schwing, and Shenlong Wang. GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mest In CVPR, 2024.
Chung-Yi Yi Weng, Brian Curless, Pratul P. Srinivasan, Jonathan Barron, and Kemelmacher- Shlizerman. HumanNeRF: Free-viewe vreepvieepoint Rendering of Moving People from Monocular Video. In CVPR, 2022.
Zhenzhen Weng, Jingyuan Liu, Hao Tan, Zhan Xu, Yang Zhou, Serena Yeung-Levy, and Jimei Yang. Single-view 3d human digitalization with large reconstruction models. arXivv 2024.
Olga Wichrowska, Niru Maheswaranathan, Matthew W Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Nando Freitas, and Jascha Sohl-Dickstein. Learned optimizers that scale and gen- eralize. ICML, 2017.
Yuxuan Xue, Xianghui Xie, Riccardo Marin, and Gerard Pons-Moll. Human 3diffusion: Realistic avatar creation via explicit 3d consistent diffusion models. arXiv, 2024.
Tao Yu, Zerong Zheng, Kaiwen Guo, Pengpeng Liu, Qionghai Dai, and Yebin Liu. Function4d: Real-time human volumetric capture from very sparse consumer rgbd sensors. In CVPR, 2021.
Zhengming Yu, Wei Cheng, Xian Liu, Wayne Wu, and Kwan-Yee Lin. Monohuman: Animatable human neural field from monocular video. CVPR, 2023.
Xuanmeng Zhang, Jianfeng Zhang, Rohan Chacko, Hongyi Xu, Guoxian Song, Yi Yang, and Jiashi Feng. Getavatar: Generative textured meshes for animatable human avatars. In CVPR, 2023.
Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In ICCV, 2021.
Shunyuan Zheng, Boyao Zhou, Ruizhi Shao, Boning Liu, Shengping Zhang, Liqiang Nie. and Yebin Liu. Gps-gaussian Generalizable pixel-wise 3d gaussian splatting for real-time human novel view synthesis. In CVPR, 2024.
Yiming Zuo and Jia Deng. Ogni-de: Robust depth completion with optimization-guinn guided neural iterations. In ECCV, 2025.
13

---

Published as a conference paper at ICLR 2025
APPENDIX - LIFE-GOM: GENERALIZABLE HUMAN RENDERING WITH LEARNED ITERATIVE FEEDBACK OVER CUUSIIANS-ON-MESS
This appendix is structured as follows:
Sec.
summarizes mesh representations in human modeling;
Sec.
provides the detailed architecture of the iterative feedback module;
Sec.
C
details the datasets:
Sec.
shows baseline details on the presented datasets;
Sec. provides additional results and analysis. Please visit the project webpage for more qualitative results;
Sec. F showcases failure cases in our approach.
A ADDTIONAL RELATED WORKS
Mesh representations in human modeling. Meshes as an explicit representation are easy to ani- mate and can be rendered at a fast speed. Further, meshes can be easily integrated into the classic graphics pipeline. Therefore, meshes are widely used in human modeling (Liao et al., 2024 Zhang et al.) 2023 Liao al.) 2023). However, as mentioned in GoMAvatar (Wen et al., 2024), is diffi cult to learn to deform the mesh using photometric losses and mesh rasterization. Hence, methods using meshes as the underlying representation either extract them from other types of representa- tions such
signed distance function (SDF) (Zhang et al. 2023 Liao et al. 2023).
apply
explicit supervision on the geometry, e.g., supervising surface normals (Liao et al. 2024 Zhang al. 2023 Liao al., (023). In contrast, we opt
the Gaussiansonn-Mesh representation that binds Gaussians on the mesh and uses Gaussian splatting for rendering. This enables us to overcome the difficulty in optimization. Consequently, our entire model is learned via photometric losses only. Further. Gaussians-on- on-Mesh leverages the flexibility of Gaussian Splatting, enabling more photorealistic rendering than textured meshes.
B DETAILS FOR THE ITERATIVE FEEDBACK MODULE
The detailed architecture of the iterative feedback module is provided in Fig. Given source im- ages and rendered images, first extract image features via an image encoder. Then we apply multi-source fusion which samples aligned image features for each of the low-resolution vertices {u {u--,,)e and mixes the features from N sources. After that, Point Transformer is adopted to encode all vertices. Note that the iterative feedback module operates on the low-resolution mesh.
Image encoder. We use R8 (He al. 2016) with ImageNet pretrained weights as the image encoder. The image feature is the concatenation of features from intermediate layers and therefore has dimension of 1192, i.e., C = 1192 in Fig. [ Concatenating multi-level features ensures large receptive field and is essential for iterative updates.
Multi-source fusion. Multi-source fusion first samples image features for all vertices in the low- resolution mesh. Concretely, the i-th vertex of , {1, {1,..., V4J first articulated via the available source human poses {Pn, === and then projected [Kr and extrinsics { EE J.V The aligned features are sampled at the projected points from each the N source images. Subsequently we mix the sampled features from the N source images using two Transformer encoder layers. The query matrix Q, key matrix K and value matrix V for each Transformer encoder are illustrated in Fig. [ The input, intermediate and output dimensions are C' = 1192. We use 6 heads in the attention layers. Note that in the second Transformer encoder layer, a learnable vertex embedding
images via the available camera intrinsics
Y as the query. The learnable vertex embedding is updated in iterative updates together with the low-resolution vertices.
https://went https:///eenj.gihhub. github. 10/ Fe...Fee.GoM
14

---

Published as a conference paper at ICLR 2025
Source images XW
Iterative feedback
Image Encoder
Image feature BxNxCxHxW
Multi-source Fusion
BxV'xC
Point Transformer
BxV
Rendered images Xxxx3xxx
Image Encoder
Image
Multi-source
BxV
XXNXCX XHXW
Fusion
GoME-1
Figure 7: Iterative feedback module. The iterative feedback module takes as input the represen- tation GoM,_, obtained from the previous iteration, the source images and images rendered with GoM The module is designed to compare the rendered images and source images, and to sum- marize the result in feature vector of dimension C" for each vertex in the low-resolution mesh. Here, B denotes the batch size, N refers to the number source images, and and W are height and weight of the images respectively. Further, V is the number of vertices in the low- resolution mesh, C refers to the dimension the feature vector from the image encoder, and denotes the dimension of the output feature from the Point Transformer. The entire module operates on the low-resolution mesh.
Vertex embeedinn foi-acles ExP*xxXC
Image Reature XXNXC
(RXP")XNXG
Transformer Encoder Layer
BxV
Sample
Rearrange
Encoder Layer
GoME-1
Figure 8: Multi-source fusion. Multi-source fusion first samples the vertex-aligned image features from the encoded images. Then each of the source images. In the Transformer encoder layers, the three input arrows from top to bottom represent the query matrix Q, the key matrix K, and the value matrix V of the attention layer respectively. We additionally associate a learnable vertex embedding with each vertex. Please check Appendix B for details.
two Transformer encoder layers to fuse the information from
Point Transformer. The Point Transformer (Zhao et al., 2021) is used to encode the vertices and to produce high-level features for all fow-uusolution vertices. The output dimension of each vertex is 32, i.e., C" 32 in Fig.
C DATASET DETAILS
THuman2.0 Yu et al., 2021). We use THuman2.0 to evaluate our approach in the setting of mul- tiview source images. THuman2.0 has 526 high-quality 3D human scans, texture maps and corre- sponding SMPL-X parameters. We follow the experimental setup of GHG (K (oon al. 2024) and split the dataset into 426 subjects for training and 100 subjects for evaluation. We render multiview images from the 3D scans. 3 or images used source images and the remaining ones used for evaluation.
XHuman (Shen et al. 2023). We use XHuman to validate our approach to cross-domain gen- eralization quantitatively. The dataset provides 20 subjects with high-quality scans and SMPL-X parameters. We sample three dataset the same way as THuman2.0. To validate the ability of cross-doman generalization, we only evaluate in this dataset without any finetuning.
(f00001, f00051, f00101) for each subject. We prepare the
15

---

Published as a conference paper at ICLR 2025
AIST++ (Li et al. 2021). The AIST++ dataset is used to evaluate the setting of mulli--rrame source images. The + dataset consists multiview dancing videos, camera calibration parameters, and human motions represented in SMPL poses. We adopt the training and evaluation protocol of ActorsNeRF (Mu et al. 2023). Specifically, we use subjects 1-15 and 21-30 for training and leave out subjects 16-20 for evaluation. We choose use camera for training. During evaluation, we sample source images from Camera I and use Camera 2-7 evaluate generalizable novel view and pose synthesis.
motion sequence for each subject. We only
D BASELINE DETAILS
We compare with per-scene optimized approaches including GoMAvatar (Wen et al., 2024), 3DGS- Avatar (Qian et al. 2024b and iHuman (Paudel et al. 2024). and other generalizable human render- ing approaches including NHP (Kwon et al. (202)). D. NIA (Kwon et al., 2023). GHG (Kwon et al.] 2024) and GPS-Gaussian Zheng al. 2024) on T'uuman.00. We use source images when com- paring with GoMAvatar, 3DGS-Avatar, iHuman, NHP, NIA and GHG. For the comparison with GPS-Gaussian, we adopt source images following the setting of GHG (Kwon al., 2024). since GPS-Gaussian requires the source views to overlap with each other and thus does not work well with very sparse views. We compared with the pretrained GHG (Kwon et al., 2024) on XHuman. On AIST++, we compare with HumanNeRF (Weng et al. 2022), GoMAvatar (Wen et al. 3DGS- QQian et al. 2024b), iHuman (Paudel al. 2024) and ActorsNeRF (Mu et al. HumanNeRF. GoMAvatar, 3DGS-Avatar and iHuman need to be trained per scene. ActorsNeRF adopts two-stage training: In the first stage, it learns a categorical prior from large-scale datasets. In the second stage, it adopts per-scene optimization given the source images. Now detail the training setup of each baseline.
2024).
2023
2023) and GHG (Kwon et al. 2024). We follow the same setting as reported in GHG (Kwon et al. 2024 for training and evaluation.
NHP (K won et al. 2021), NIA (Kwon et al.
GPS-Gaussian (Zheng et al. 2024). As described in GHG (Kwon et al. 2024). GPS-Gaussian can
work on as few as 5 input views. We render the THuman2.0 dataset to accommodate this setting. We use the default parameters provided in GPS-Gaussian to train the model.
GoMAvatar (Wen et al., 2024). GoMAvatar originally takes SMPL parameters as inputs. On THuman2.0, we adjust it to work with SMPL-X parameters. On AIST+ we keep the original setting and use SMPL. We train the model for 100K iterations on both datasets instead of 200K iterations stated in the paper. to avoid overfitting sparse inputs.
3DGS-Avatar (Qian et al. 2024b). Similar to Avatra, we adapt 3DGS-Avatar to take SMPL-- X parameters as inputs H'uuma2.00 and keep its original setting on AIST+ We train it for 2K iterations to avoid overfitting the very sparse inputs.
iluumn Paudel al. 2024). We adapt iHuman to work with SMPL-X parameters on the THu- man2.0 dataset. It requires subdivided SMPL or SMPL-X templates as inputs. For a fair comparison. we adopt the same subdivision strategy for the SMPL-X template as ours. in which we subdivide all faces twice. We find that the model cannot converge after 15 epochs, the number of epochs specified in the original iHuman paper. Instead, we use 150 epochs which takes a longer time for training but provides better rendering quality. On AIST++, we use the default hyperparameters for training.
HumanNeRF (Weng al. 2022) and ActorsNeRF (Mu al. 2023). We follow the same setting as reported in ActorsNeRF (Mu et al. 2023) for training and evaluation.
E ADDITIONAL ANALYSIS
For additional qualitative results refer the reader the project webpage. It contains videos for freeview rendering, cross-domain generalization and novel pose synthesis.
E ADDITIONAL COMPARISONS ON THUMAN2.0
We compare our approach against GoMAvatar (Wen et al. 2024), iHuman Paudel et al. 2024), and GHG (Kwon al. 2024) in the setting of 3 source images THuman2.0. Note that GoMAvatar
16

---

Published as a conference paper at ICLR 2025
81
(a) Ground truth Figure 9: Comparison to baselines in the setting of source images on THuman2.0,, Our method produces less noise than iHuman and GvMAvatar and more accurate geometry and sharper details than GHG.
(b) iHuman
(c) GoMAvatar
(d) GHG
(e) Ours
and iHuman are scene-specific methods while GHG is a generalizable approach, the same as ours. In Fig. 9 we showcase additional qualitative comparisons to the baselines.
In the setting of very sparse inputs, c.g., 3 views, scene-specific methods suffer from overfitting and struggle to render uncorrupted novel views. Generalizable approaches, in contrast, constrain the output space with the data priors learned from large-scale datasets, which leads to more plau- sible rendering quality. Compared to other generalizable approaches, ours outputs more accurate geometry and sharper details.
We show the qualitative comparison between ours and GPS-Gaussian in the setting of 5 views on THuman in Fig. TO GPS-Gaussian (Zheng et al. 2024) relies on stereo depth estimation to locate the Gaussians. During inference time, takes inputs two adjacent views and interpolates the novel views in between. Therefore, it requires the adjacent views to overlap with each other. As mentioned in GHG (Kwon et al. 2024), 5 views are the minimal number of input views that GPS- Gaussian can work on. Even with 5 views as inputs, we still find that it fails in the non-overlapped rendering. In contrast, ours outperforms GPS-Gaussian
regions, leaving incomplete silhouettes qualitatively and quantitatively in the setting of 5 views and can work as few as 3 views. Another key difference between UPS-Gaussian and our approach that we reconstruct the human subject in the canonical T-pose while the representation of GPS GPS-Gausee Gaussian is in the same pose as the source images. Therefore, ours can take images in unaligned poses as inputs and render novel poses without extra effort, demonstrated in Fig.
17

---

Published as a conference paper at ICLR 2025
(c) Ours Figure 10: Comparison to GPS-Caussian in the setting of source images THuman2.0. Our method produces more complete shape and sharper details.
(a) Ground truth (b) GPS-Gaussian
(n) Ground truth
(b) GPS-Gnussian
(c) Ours
Table 6: Per-scene breakdown on AIST++ We use lighter gray for scene-specific methods, while the others are generalizable methods.
PSNR 1
SSIM
LPIPS
PSNR 1
SSIM
LPIPS 1
PSNR 1
SSIM
LPIPS* 1
d16
d17
d18
HumanNeRF
24.37
0.9752
29.59
24.86
0.9762
29.39
22.77
0.9738
33.02
GoMAvatar
24.35
0.9769
24.80
25.12
0.9780
25.17
23.18
0.9771
27.57
3DGS-Avatar
25.22
0.9776
26.01
25.71
0.9787
27.70
23.75
0.9757
29.98
iHuman
25.41
0.9804
21.79
25.59
0.9805
23.69
24.25
0.9786
24.37
AclorsNeRI
25.22
0.9796
22.03
25.88
0.9808
22.85
24.50
0.9811
22.38
Ours
25.43
0.9801
21.48
25.73
0.9812
21.94
24.46
0.9810
22.21
d19
d20
Average
HumanNeRI
24.51
0.9759
28.68
24.55
0.9791
27.63
24.21
0.9760
29.66
GoMAvatar
24.36
0.9773
25.24
24.68
0.9806
23.95
24.34
25.34
3DGS-Avatar
25.32
0.9783
27.70
25.70
0.9819
24.44
25.14
0.9784
27.17
iHuman
25.11
0,9800
23.29
25.48
0.9829
21.37
25.17
0.9805
22.90
ActorsNeRF
25.24
0.9801
22.87
25.30
0.9827
21.34
25.23
0.9809
22.29
Ours
25.19
0.9805
21.42
25.43
0.9829
20.98
25.25
0.9812
21.61
E.2 ADDITIONAL COMPARISONS ON AIST++
Following ActorsNeRF, we also list the per-scene breakdown on 5 evaluation scenes on AIST+ in Table
Next, present the qualitative comparison to GoMAvatar, iHuman and ActorsNeRF in Fig. AIST++ is challenging for two reasons: 1) The subjects perform challenging and diverse poses in the videos. 2) The poses provided by the dataset off-the-shelf tools which are also less accurate. Due to the explicit nature of the Gaussians -oriiiinn------ssee representation, our method produces fewer floaters than NeRF-based ActorNeRF. Meanwhile, capture better silhouettes and produce less noise compared to iHuman and GoMAvatar, the two scene-specific methods.
less accurate and the masks predicted from
E.3 INPUT POSE SENSITIVITY
We quantitatively compare the sensitivity to input pose accuracy for approach and GHG (Kwon ctt al. 2024).
this experiment, we add Gaussian noise of increasing standard deviation (0.1. 0.3,3.3000 0.5) to the poses provided by THuman2.0. The results are summarized in Table 7 Both methods are affected by the accuracy of the input poses. However, our approach improves upon GHG in all noise levels.
To make our approach less sensitive to the accuracy of input poses, we can explore pose refinement network that jointly trained with the iterative feedback. We leave it for future work.
18

---

Published as a conference paper at ICLR 2025
TT
Ground truhh
GoMAvatar
(c) iHuman
(d) ActorsNeRF
Ours
Figure 11: Comparisons on baselines on AIST+ Our method has fewer floaters compared to ActorsNeRF and produces more complete shape than GvMAvata and iHuman. Meanwhile, ours 11x faster than iHuman in reconstruction.
Table 7: Comparison regarding inaccurate input poses. We add random Gaussian noise dif- ferent standard deviations to the poses provided by THuman2.0. Our method outperforms GHG for all noise levels.
Noise
std=0.1
std=0.3
std=0.5
Method
PSNR
LPIPS*
FID.
PSNR
LPIPS
FID,
PSNR
LPIPS*
FID.
GHG
21.25
136.87
62.03
19.66
149.73
64.15
18.53 20.43
163.48
68.57
Ours
23.96
113.80
53.15
22.02
123.15
57.22
134.86
62.84
Table 8: Importance of SMPL-X shape input. We compare our method w/ and w/o SMPL-X shape input for mesh initialization. We only observe a slight drop in performance the SMPL-X shape is not used.
Method
PSNR
LPIPS*
FID.
Ours w/o SMPL-X shape Ours w/ SMPL-X shape
24.15 24.65
112.88
52.01
110.82
51.27
E.4 INPUT SHAPE SENSITIVITY
We quantitatively evaluate the sensitivity input SMPL-X shape accuracy for our approach in Table 8. In this experiment, we initialize the canonical mesh with the average SMPI. shape by setting the beta parameter a tensor of all zeros for all subjects. Neither ground-truth nor predicted SMPL shapes are used for any subject. We call this setting "Ours w/o SMPL-X shape" in the table.
Our method attains PSNR/LPIPS*/IID of 24.15/1 12.88/52.01 w/o SMPL shapes as input. Com- pared to 24.65/1 10.82/51.27 with SMPL shapes as input, we only observe small drop, which shows the robustness shapes, our method still significantly outperforms GHG's as input. We further demonstrate the robustness in Fig. [2] Although the average shape is smaller than the ground-truth shape, method still captures the correct shape.
our method to the accuracy SMPL-X shapes. Even without SMPL-X
00/133.41/61..7 with SMPL-X shapes
19

---

Published as a conference paper at ICLR 2025
(a) Ground-truth SMPL-X shape on input images
(b) Rendered image w/ (a)
(c) Average SMPL-X shape on input images
(d) Rendered image w/ (c)
Figure 12:
Robustness to SMPL-X shape accuracy. We use the ground-truth SMPL-X shape and the average shape as initialization the canonical mesh. Although the average shape is smaller than the ground-truth shape, our method still captures the correct shape.
Reference image
Rendered image
(a)
Figure 13: Examples failure cases.
F LIMITATIONS
We present three types of failure cases in our method and discuss the possible next steps to resolve the issues.
Failure in hallucination large regions. Without explicit hallucination module, our method is unable to inpaint large invisible regions in source images, as is shown in Fig. a).. possible solution is to render the invisible parts and update our canonical representation using priors from image inpainting models.
Wrong underlying topology. Our sociates the Gaussians with the underlying mesh. Analogously to the original Gaussians- Prussians-on-Mess on-Mesh representation, since the underlying mesh is deformed from human parametric models such as SMPL. and SMPL-X, it cannot change vertex connectivities to fit the topology of clothes such as dresses and coats. Although the wrong topology will not affect the rendering, the underlying mesh for use in other downstream tasks.
Ctussians-on-Mess representation
future direction correct
Failures for unseen clothing types. We observe failures for unseen clothing such as dresses, as shown in Fig. (.)... As generalizable method, a more comprehensive training set containing different clothings and more diverse subjects is needed. We leave it for future work.
20


Reviews:
[{"review_id": "YY6w8sojRQ", "reviewer": "Reviewer_5YSm", "summary": "This paper tackles the challenge of creating an animatable human avatar from just a few images. To make reconstruction faster and more accurate, they introduce an iterative feedback approach that gradually refines the results. For high-quality, efficient rendering, they also develop a multi-resolution Gaussians-on-Mesh technique, which balances detail and speed.", "questions": "- In Figure 1, the mesh appears to be a GT-mesh rather than an SMPL model. Is the GT mesh utilized in your approach？\n- Given that Gaussian splatting has a strong tendency to overfit, how is pose generalization achieved in your method? Are the displayed novel poses primarily from the dataset, or do they include truly unseen poses?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The feedback mechanism in this generalizable method is notably innovative, refining predictions by addressing visible errors from input views, resulting in more accurate and detailed reconstructions within a second.\n- Achieving human avatar reconstruction from images in different poses is highly practical, offering greater flexibility than multi-view approaches.\n- The paper is well-written, well-structured, and clear.", "weaknesses": "- The model does not appear to have been trained on two datasets and can support both tasks simultaneously, limiting its ability to handle mixed input types effectively.\n- The method's heavy dependence on SMPL priors poses a risk; if the estimations are inaccurate, especially for loose clothing, it can lead to significant errors\n- The method shows significant performance degradation when applied across domains, particularly affecting the facial region, as in the video. Besides, the novel pose representation displays black hands in the video. \n- Minor issues: line 156 \"estimate\" to \"estimation\", line 234 translation -> scale?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8JPA4TSnMN", "reviewer": "Reviewer_vbGJ", "summary": "The paper tackles the problem of fast, sparse-view 3D articulated avatar modeling. Specifically, the paper proposes to improve Gaussian-on-Mesh method by using a dual-resolution mesh: the low-resolution mesh is used for efficient feed-forward computation, while the high-resolution mesh anchors Gaussians to represent fine details. Furthermore, the approach the sparse input images as cues to extract pixel-aligned features to iteratively improve the 3D avatar. After training, the network can reconstruct 3D avatars for a given identity in under 1-2 seconds.\n\nThe experimental results show quantitative and qualitative improvement over previous methods.\n\nAfter reviewing the rebuttal: my concerns are sufficiently addressed by the new quantitative results provided by the authors. Overall, this submission presents an effective method for generalizable 3D human modeling, and shows non-trivial improvement over the prior art. \nI therefore raise my score to 8 -- accept, good paper.", "questions": "Please address/discuss about the weakness mentioned above if they are not already in the paper.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The presented method has the following strengths:\n- The method shows fast, efficient reconstruction, and this is always a very welcome property for 3D avatar modeling.\n- The method shows promising results on reconstructing out-of-distribution avatars.\n- The method gets rid of the Gaussian split/growing heuristics, which are sometimes annoying to tune/control.\n- The qualitative and quantitative results verify the effectiveness of their proposed design choice.\n\n3D avatar modeling, the problem tackled in this paper, is an important topic in vision/graphics due to its wide impact on many fields (entertainment, sports, virtual try-on, etc). Overall, the presented approach makes sense and the results look promising,", "weaknesses": "The paper has the following weakness:\n- Missing comparisons to GoMAvatar: as mentioned in line 121-123, the presented method is derived from GoMAvatar. It therefore makes sense to compare against GoMAvatar. The iterative-feedback bears a close similarity to [1], which also utilizes pixel/texel align features to improve fidelity. It would be great if the paper could discuss about these relevant works.\n- The writing can be further improved: for example, the sentence in line 143-145: ```As a fix, to regularize the Gaussians and to ease the animation, pairing with parametric models like FLAME or SMPL helps``` may not be as straightforward as saying ```Prior work [cite] regularizes the Gaussians and enables animation using parametric models such as FLAME and SMPL```. Also, in line 180-181 ```given as additional input the target camera ...``` seems unnecessary, as line 175-179 already described the input. \n- Mesh topology limits the geometry that can be captured, as shown by the hairs in Figure 4 (also pointed out in Appendix C).\n- Missing memory profile: perhaps I have missed it, but I did not see discussions related to the memory/GPU vram consumptions for the proposed method. How much VRAM does it take to train/render the avatars? Does it work on commodity-level GPUs?\n- Potentially susceptible to body pose misalignment issues: when the body pose is imperfect/does not overlap with the correct body part, the feedback network will extract features from incorrect locations, and thus impact the reconstruction quality.\n\n[1] Drivable Volumetric Avatars using Texel-Aligned Features, SIGGRAPH 2022", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BRNqrZioB1", "reviewer": "Reviewer_3FuA", "summary": "This paper presents a method for generating high-quality, animatable 3D human avatars from sparse images. It proposes a multi-resolution Gaussians-on-Mesh representation, where low-resolution mesh is used for efficient geometry and high-resolution Gaussians for high-quality avatars. This paper also introduces an iterative feedback update mechanism to refine the model.", "questions": "Please refer to my weakness part. I also have some questions about the videos supplied.\n1. Why not show all the input images in Cross-domain generalization part? Without all input frames, it is hard to evaluate the performance. \n2. Please make the input number of images clear in Novel pose synthesis part. The novel pose performance is impressive if it is achieved by a single reference image.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The method achieves fast 3D human avatar reconstruction from sparse inputs, making it suitable for real-time applications.\n2. The multi-resolution Gaussians-on-Mesh representation balances efficiency and rendering quality.", "weaknesses": "1. The experimental comparisons are incomplete without recent works[1,2,3] that report better metrics. A comparative analysis or discussion of technical differences is needed.\n2. The cross-domain generalization shows concerning facial artifacts despite the proposed iterative feedback. What causes these failures? If it's a data bias issue, why showcase this case? Consider discussing limitations and potential solutions (e.g., face-aware regularization).\n3. The ablation study focuses only on hyperparameters rather than analyzing the effectiveness of core components. Need evaluation of:\nIterative feedback mechanism; Coupled-multi-resolution representation; Module-wise contribution analysis.\n\n\n[1] 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting, CVPR2024\n[2] GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh, CVPR2024\n[3] HUGS: Human Gaussian Splats, CVPR2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "w4zf9o5sMc", "reviewer": "Reviewer_EVBW", "summary": "The paper proposes a generalizable human rendering approach using a learned iterative feedback update mechanism. The method leverages a coupled multi-resolution Gaussian-on-mesh representation and demonstrates state-of-the-art results on benchmark datasets such as THuman 2.0 and AIST++.", "questions": "While the paper demonstrates promising results, I am curious about the method’s generalizability across diverse scenarios. In particular, on the project page, it would be insightful to evaluate the animation quality of cross-domain examples (e.g., drastically different body types, dynamic poses, or motion sequences). Addressing these aspects would provide a clearer understanding of the robustness and adaptability of the proposed approach", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe proposed iterative update net is a novel idea and is clearly motivated.\n2.\tThe writing is clear and easy to follow, with a well-structured introduction and related work section.\n3.\tExtensive experimentation on benchmark datasets shows clear performance improvements over previous methods (Table 1, Table 2, Table 3).\n4.\tThe authors provide a comprehensive ablation study, which helps justify the effectiveness of iterative step choice and subdivision.", "weaknesses": "1. In the related works section, the authors categorize human rendering into \"per-scene optimized\" and \"generalizable\" approaches. However, there is no discussion on \"large reconstruction model-based\" human rendering approaches [1,2]. It would be valuable to compare the proposed method’s performance (in terms of inference time, training time, resolution and PSNR) with such large reconstruction models.\n\n2. The proposed method employs a multi-resolution Gaussian-on-mesh representation, but a direct comparison with traditional mesh-based human representations [3, 4, 5] is missing. It would strengthen the paper if the authors could discuss the advantages or trade-offs between these representations ((in terms of rendering quality, animation flexibility, FPS).\n\n3. The method is built on SMPL/SMPLX models, which might limit its effectiveness in rendering loose clothing, such as skirts or dresses (see  examples in https://zhaofuq.github.io/NeuralAM/). Including examples of such scenarios would enhance the evaluation of the method’s generalizability.\n\n[1] Weng, Zhenzhen, et al. \"Single-view 3d human digitalization with large reconstruction models.\" arXiv preprint arXiv:2401.12175 (2024).\n\n[2] Chen, Jinnan, et al. \"Generalizable Human Gaussians from Single-View Image.\" arXiv preprint arXiv:2406.06050 (2024).\n\n[3] Liao, Tingting, et al. \"Tada! text to animatable digital avatars.\" 2024 International Conference on 3D Vision (3DV). IEEE, 2024.\n\n[4] Zhang, Xuanmeng, et al. \"Getavatar: Generative textured meshes for animatable human avatars.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n\n[5] Liao, Tingting, et al. \"High-fidelity clothed avatar reconstruction from a single image.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)