GLVD: Guided Learned Vertex Descent
Authors: Pol Caselles
Francesc Moreno-Noguer
Venue: NeurIPS 2025 poster ()
Keywords: Computer Vision
3D Face Reconstruction
Neural Fields
Few-Shot 3D Reconstruction
Primary area: applications
TL;DR: We propose GLVD, a fast and accurate hybrid method for 3D face reconstruction that combines per-vertex neural fields with dynamically predicted 3D landmarks to achieve state-of-the-art quality from just one or a few images.

Abstract: Existing 3D face modeling methods usually depend on 3D Morphable Models, which inherently constrain the representation capacity to fixed shape priors. Optimization-based approaches offer high-quality reconstructions but tend to be computationally expensive. In this work, we introduce GLVD, a hybrid method for 3D face reconstruction from few-shot images that extends Learned Vertex Descent (LVD) by integrating per-vertex neural field optimization with global structural guidance from dynamically predicted 3D keypoints. By incorporating relative spatial encoding, GLVD iteratively refines mesh vertices without requiring dense 3D supervision. This enables expressive and adaptable geometry reconstruction while maintaining computational efficiency. GLVD achieves state-of-the-art performance in single-view settings and remains highly competitive in multi-view scenarios, all while substantially reducing inference time.

GLVD: Guided Learned Vertex Descent
Pol Caselles Rico
Francesc Moreno Noguer"
Institut de Robotica Informatica Industrial, CSIC-UPC
Amazon Barcelona, Spain
Crisalix SA
Barcelona, Spain
cescmore@amazon.c es
pcaselles228gma1..com
f= 0
2
i 6
e
e
e
O
P
Figure 1: Qualitative results for two in-the-wild subjects reconstructed using GLVD
Abstract
Existing 3D face modeling methods usually depend on 3D Morphable Mod- els, which inherently constrain the representation capacity to fixed shape priors. Optimization-basee approaches offer high-quality reconstructions but tend be computationally expensive. In this work, we introduce GLVD, hybrid method for 3D face reconstruction from few-shot images that extends Learned Vertex Descent (LVD) [1 by integrating per-vertex neural field optimization with global structural guidance from dynamically predicted 3D keypoints. By incorporating relative spa- tial encoding, GLVD iteratively refines mesh vertices without requiring dense 3D supervision. This enables expressive and adaptable geometry reconstruction while maintaining computational efficiency. GLVD achieves state-of-the- performance in single-view view settings and remains highly competitive multi-view scenarios, all while substantially reducing inference time.
Introduction
High-fidelity 3D face modeling from images long-standing challenge in the computer vision community, with broad impact across applications such as Virtual Reality, Augmented Reality, healthcare, entertainment, and security. Reconstructing an accurate and coherent digital human representation from few input images is highly ill-posed task -prrticularly in uncontrolled environments- often requiring geometry-awand aware methods guided by strong prior assumptions. Adding to this challenge is the scarcity of abundant, high-quality 3D training data captured under such
"This work was conducted independently and does not relate to the author's position at Amazon
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

unconstrained conditions, which limits the generalization ability of existing models in diverse real-
world scenarios.
Statistical priors based on parametric 3D Morphable Models 2 16
54 55 62
69
17
have become the standard approach for few-shot 3D face reconstruction. By encoding facial geometry using low-dimensional set of parameters, 3DMMs provide a robust and efficient framework, particularly effective in scenarios with limited or single-view image input. However, their effectiveness is hindered by two key limitations: a bias toward the mean shape [59], and the inherently constrained expressiveness of parametric models. These models typically operate within fixed low- -iwdeneniional subspaces, making it difficult to capture fine-grained details or adapt to -f---ii-ittiiuiionn variations.
88
80
66 30
Model-free representations using voxels 26 meshes, point clouds, or Gaussian splatting 28 offer greater flexibility and high reconstruction accuracy, but they face scalability and resolution trade-offs due to memory and topology constraints. Neural fields address these challenges by encoding geometry and appearance as continuous functions via neural networks. These methods can reconstruct detailed surfaces from images without requiring 3D supervision, but they typically depend on multi-view inputs and suffer from high inference costs 38 Recent works 42 67 6 have made significant progress in reducing computational overhead. However, converting such representations into well- structured, topologically consistent meshes suitable for animation or rendering often necessitates additional post-prostt poot-proc,ssing, commonly involving template fitting.
38 58 23 22 offer faster inference at the cost of robustness and accuracy, especially under duitr--dist conditions (37). More recently, Learned Vertex Descent (LVD) [I]] introduced a hybrid strategy that uses pixel-aligned image features to guide iterative template fitting. Despite its effectiveness, LVD relies on large-scale training data with posed images and corresponding 3D geometry, and it lacks explicit global structure, predicting vertex trajectories independently and depending on the image encoder for implicit coherence.
Optimization-basee approaches produce accurate and detailed results through iterative refinement
67
6,
52 g 9]. while feed-forward methods [57
To overcome these limitations, we propose leveraging a 3D face landmark estimator derived from 2D image-based predictor to guide 3D shape refinement. We introduce GLVD, learning- based optimization approach that fuses local and global cues by combining per-vertex neural fields with dynamically predicted 3D keypoints. Each neural field predicts 3D displacements for its associated vertex based on local features sampled at its current position, while the keypoint ensemble provides global structural guidance that informs and regularizes the optimization process. Central to method is a relative encoding scheme, where each vertex is transformed based on the current keypoint estimates, allowing the network learn geometry-aware updates that are conditioned the evolving global structure.
The combination of local neural fields and global keypoint- based guidance in GLVD enables more precise control and adaptive refinement of 3D facial geometry, as shown in Figure Leveraging this fusion, we conduct a comprehensive evaluation both single-vies view and multi-view 3D face reconstruction benchmarks. Our approach achieves state-of-the-ar performance in single-image reconstruction and remains competitive with ciliiization-based methods in multi-view scenarios, demonstrating its robustness, accuracy, and broad applicability.
Related work
3D Morphable Models (3DMM). The use of 3D Morphable Models (3DMMs) has become the standard paradigm for reconstructing 3D facial geometry from images, particularly in single-view or few-shot scenarios. These statistical models [46] 32 are widely adopted and mainly focus on the facial region. In single-image settings, several methods have demonstrated effective reconstruction
performance [62
66 80 30).
Recent advancements in single-image 3D face reconstruction have explored both parametric and non-parametric strategies to improve accuracy, robustness, and detail preservation. 3DDFAv2 [2]] proposes a regression-basee approach combining a lightweight architecture with meta-joint opti- mization to achieve real-time performance while maintaining alignment accuracy. Building upon this, 3DDFAv3 (68) introduces Part Re-projection Distance Loss, which leverages dense facial part segmentation as strong geometric prior for guiding 3D reconstruction, especially under extreme

---

(A(+)
Hestapp Eatruatve
k?
sw()
gu(1)
A
(e(-)
M
9k
F
Keypointe Branch
(-)
F.
Vertex Branch
Fied Alpedd
Phad Alged
pd
Conentenation
Ak
Massting
Av
R
Deviter
Urodac
Addttio
Figure 2: Overview of GLVD, Given one or more imput images, each paired with a head mask and calibrated camera parameters, the method reconstructs 3D face mesh through two branches. (I) The 3D Keypoint Branch predicts a set of facial keypoints by extracting localized image features and estimating their 3D displacements iteratively. (2) The 3D Vertex Branch refines the full-face geometry by leveraging these keypoints to encode relative spatial information for each surface vertex. This branch extracts pixel-aligned features and predicts vertex-wit displacements in an iterative optimization process.
expressions where landmarks are unreliable. SADRNet [56] introduces a self- self-aligned sual-regression framework that disentangles pose-dependent and pose-independen features and fuses them through an occlusion-aware alignment strategy. HRN 30 proposes a hierarchical representation network that disentangles geometric components and incorporates high-frequency priors, enabling the reconstruc- tion of fine facial details, such as wrinkles and skin texture, from in-the-wild images.
Neural Fields for Face Reconstruction. Neural fields have emerged leading approach for 3D reconstruction, offering continuous and high-fidelity representations of geometry and appearance. They have been successfully applied full-head and facial modeling tasks using techniques such N 9 6 Hybrid models combine parametric approaches such as 3DMMs with neural implicit functions to increase control and expressiveness. For instance, IMFace [74] and IMFace++ [75] introduce implicit displacement fields to refine a 3DMM geometry, and NeuFace 766 proposed an approximated BRDF integration and low-rank prior for human face rendering. In 77. authors combine geometry output a signed distance field. However, these approaches tend to collapse and generate artifacts.
as volume rendering and surface priors [40 44 45
52
features with image features that
When several input images are available, line of research 20 78 79
82 19 aims to obtain
animatable full-head avatars from videos. Building on the recent success of Gaussian Splatting (28). several works [35] 15 49 6T] have integrated this representation to improve rendering efficiency and visual fidelity. Combining them with 3DMMs has been explored in recent works, with methods such as HeadGAP [77] and GPHM 71 that learns parametric head models using cauussannspllttingg -bsed models. While effective, these methods often rely on dense multi-view input. In contrast, freee--free feed-forward approaches use pixel-aligned features [57 for faster inference from sparse views.
Model-free methods leveraging pixel-aligned features have gained popularity for fast 3D reconstruc- tion, as they avoid the need for test-time optimization (57 53, (2 60
13 22 PIFu [57 introduced
pixel-aligned implicit functions, using 2D image features to predict 3D occupancy from single or
multiple views, and PHORHUM [I] extended this by employing signed distance fields for surface modeling. JIFF [7] proposed combining features from face morphable model and pixel-aligned features. In contrast, using volumetric rendering. KeypointNeRF aggregates pixel-aligned features with a relative spatial encoder. More recently, LVD [IT] emerged as learning based optimization approach that leverages pixel-aligned features to guide an serratvv-bbsed template fitting. While acquires a good trade-off between computation requirements and accuracy for human mesh re- covery, it remains unexplored for 3D face modeling. However, single feed-forward methods based on pixel-aligned features still lag behind optimization-basee approaches in terms of reconstruction quality [9].
3

---

1 view
3 views
6 views
Subject GT
1 view
3 views
views
Subjoct GT
2
2
GLVD
2
25
2
2
2
U
JIFF
2
6
2
2
SIRA+
2
2
2
2
2
2s
LND
Figure 3: Qualitative results two subjects of the H3DS dataset, for LVD
SIRA+ 9
JIFF 77, and GLVD, with an increasing number of input views.
Encoding Representations for 3D Reconstruction. Previous approaches have explored various spatial encoding strategies enhance learning. PVA (50] and PortraitNeRF [[]] utilize face- centric coordinate systems, while ARCH [25] and ARCH++ (24] adopt canonical body coordinates. KeypointNeRF 39 proposes to encode relative spatial 3D information the form of depth via sparse 3D keypoints. In [14], the authors introduced a three-step pipeline of landmark selection, low-dimensional embedding via MDS, and distance-based triangulation embed points. GLVD follows a similar idea by selecting identity-specific facial keypoints and encoding mesh vertices through their Euclidean distances to these keypoints within the learned space. this work, we conduct thorough investigation of spatial encoding and find that simple yet effective encoding based on relative distances w.r.t 3D keypoints [1] yields effective results in combination with neural fields guided by pixel-aligned features. We adopt canonical aligned space to stabilize training. In addition to achieving state-of-the-as art results on 3D face reconstruction from as few as a single input image, our approach also be used beyond face modeling.
Mesh Recovery for body. Several works 34
36 33 73 focus on full-body mesh recovery
and remain unexplored in the context of face modeling. These methods often operate in single-
image settings 34
36 33) or on video sequences [31. METRO (34] and DeFormer [72] use transformers jointly process mesh joints and vertices, differing in attention aggregation but both relying on global self-attention. METRO further conditions on a global image embedding,
72
discarding pixel-aligned spatial detail. In contrast, 3D Virtual Markers (36] prediets latent 3D markers and reconstructs the mesh as a linear combination, while One- Staget Mesh Recovery [33] directly regresses SMPL parameters via transformer. Both are constrained by the limited expressiveness of low-dimensional latent spaces. PyMAF-X [73 introduces an iterative refinement approach that samples pixel-aligned features from prior vertex predictions, but does not leverage explicit landmark or keypoint constraints for structural supervision. GART 31 leverages skeletal priors and temporal video input for avatar reconstruction, whereas GLVD is purely image-driven and effective in both monocular and few-shot settings. Finally, [53] replaces PCA-based 3D face models with convolutional mesh autoencoders learn shape priors, but focuses on latent mesh encoding rather than direct facial geometry estimation from RGB images as in GLVD
3 Method
In this section, we present our method for face reconstruction using learned keypoint guidance. We first review the Learned Vertex Descent (LVD) framework [1] and then introduce our key architectural innovations. The section concludes with details on training and inference. An overview is shown in Figure 2 with implementation details in the supplementary material.

---

MICA
HRN
FaceScape
JIPF
LVD
SIRA+
RAFaRe
3DDEA-V3 CLVD (ours)
1
2
U
2
2
2
2
4
25
2
25
25
2
2
2
25
2
25
2
Figure 4: Qualitative results on the 3DFAW dataset for single input image. Each 3D recon- structed face is accompanied by heatmap, where reddish areas indicate larger errors in mm.
3.1 Background: Learned Vertex Descent
Learned Vertex Descent (LVD) [1] is an optimization- method for 3D human shape reconstruc- tion from single-view images or scans. While it has been applied full-body and hand reconstruction, we explore its potential for 3D face modeling. The model learns transformation g(-) which takes the current 3D vertex position at iteration and the associated 2D image features f; for vertex as input, and outputs a displacement vector Av;:
g (vi, f.) + Av.
(1)
This vector represents the correction needed to align the vertex with its ground truth position v,. The updated vertex position is then given by v v + Av,. The reconstruction process involves iteratively applying this update to refine the 3D shape.
3.2 Problem Definition
Our goal is to recover a 3D face surface S from a small set of input images [I,JW Im= where each image I, paired with a head mask M,, and calibrated camera parameters T,. The surface S is represented by a fixed topology of 7,225 vertices and 14,342 faces.
We aim incorporate global 3D-aware guidance into the per-verter optimization by leveraging relative encoding based on the Euclidean distances between vertices and keypoints. As a result, propose two-stage architecture to address limited multi-view input images: (1) 3D keypoint estimation module that defines spatial keypoints on the facial surface by estimating displacements, and (2) a vertex prediction module that encodes vertices relative these keypoints estimate vertex updates. Our formulation does not rely on a predefined parametric model or fixed joint sets, making adaptable arbitrary topologies. A sparse set of surface points is conveniently selected to act as ground-truth keypoints.
3.3 Barning-Basee Keypoint Estimation
Our goal is to establish a reference space to guide vertex optimization towards the target surface. To achieve this, we estimate a fixed set of K 3D keypoints (k) \8 where kj, E R represents the j-th keypoint on the target surface, at iteration t. These keypoints are used for encoding the relative positions of query vertices V..
The 3D keypoint branch consists of two components: one responsible for extracting image features FR. and another focused on learning 3D keypoint displacements Ak, compute Fr, we first generate facial keypoint heatmaps using off-the-sher HRNet (65), which are then concatenated with vertex image features F, obtained from the first stack of the Hourglass network (4]. This combined feature map is subsequently refined using single-stack Hourglass module fa(:):
5

---

MVFNET
DENRMVS
PIFU
JIFF
LND
HoD-Net
SIRA+
GLVD (ours)
1
2
2
2
15
50
2
7
2
4
2
2
2
2
25
2
25
25
2
Figure 5: Qualitative results on the 3DFAW dataset for three input images. Each 3D reconstructed face is accompanied by a heatmap, where reddish areas indicate larger errors in mm.
fF (I, H+ Fg-
The keypoints employed to guide the vertex branch do not necessarily coincide with the facial landmarks defined by HRNet. We adopt strategy of predicting vertex displacements from local features, this approach has been demonstrated to yield more accurate geometric detail [57.5]] 58 I0 [1]. To estimate the 3D keypoints, we implement (+) (Eq. I with a 3-layer MLP that takes as input the current estimate of keypoint Kę and its local F-dimensional local features Fx,i extracted the projection of KC on the image plane, and predicts the displacement Akţ. In the first step, we begin by uniformly sampling ki within a volume of size 2 centered at the origin.
3.4 Vertex Displacement Prediction
The vertex branch consists of two modules: local feature extractor f. : (I, + F. that computes image-aligned features for each projected vertex, and a regressor (.) that predicts the final vertex displacements Av,, which also implemented
3-layer MLP.
we estimate set of predefined 3D keypoints at each iteration t. Given query vertex RS compute a keypoint-relative encoding matrix P R x3 where each row represents a displacement vector P' = k' v As shown in ablations (Sec. 4.2), this encoding outperforms alternatives such as Euclidean distances, attention, or concatenation.
extract local image features F, and F3 for each image I, and following Sec. 3.3
3.5 Displacement Learning and Optimization
The network is trained to learn the parameters of (.)....()))) 9u (n.()), and g.(.). We use a dataset of N training scenes, each with a ground truth mesh with known topology, posed RGB images, and head masks. We sample query points for each scene using hybrid strategy that combines uniform sampling with points near the mesh surface. Both model components are trained to predict displacements at each iteration f. The iteration index is not explicitly encoded during training, the model is exposed to stochastic distribution vertex states, making it inherently timestep-inde inneeeepdideeeedeent At inference time, iteration corresponds to updating each vertex by adding the displacement predicted at the previous step SSc..3.1| We pre-train the feature encoder on the 3D reconstruction task by augmenting it with signed distance function (SDF) prediction head. This improves convergence behavior and leads to higher reconstruction accuracy. Further details are provided in the supplementary material.
Keypoint Displacement Learning. We select consistent set of 3D keypoints to guide the surface displacement learning- For each mesh we choose the same subset of vertices to serve as target keypoints (k.). While their selection can be arbitrary, it must remain consistent across training

---

Table 1: 3D face reconstruction comparison. Average surface error (in mm) computed over all subjects 3DFAW and H3DS datasets. We place "-" for not applicable configurations Optimitzation- based have been included for reference.
3DFAW
H3DS 2.0
view
view
views
views
4 views 6 views
MVFNet
69
1.56
1.73
DENRMVS
1.69
1.83
DECA 17
1.71
1.99
MICA 88
1.83
2.08
Face Verse 66
1.88
2.57
FaceScape
80
1.61
1.78
HRN [30
1.60
1.73
VHAP 48
2.05
2.15
3DDFA- V3 [68
1.45
1.65
RAFaRe [22
1.68
2.54
PIFU [57
2.19
1.99
1.98
1.70
1.85
2.03
JIFF
1.48
1.47
1.85
1.80
1.79
1.79
LVD [1
1.58
1.26
1.39
1,45
1.39
1.37
GLVD (ours)
1.25
1.22
1.36
1.33
1.34
1.34
H3D-Net [22
1.70
1.37
1.44
1.41
1.21
SIRA++ 9
1.35
1.32
1.57
1.18
1.23
1.04
scenes and during test-time inference. Each query keypoint, together with the input image I, is passed the model (gh o fk)) (2),, which predicts 3D keypoint displacements Ak(.
Surface Displacement I Learning. We train (.(.) and g.((.) separately from the 3D keypoint branch. We encode vertices relative the ground truth 3D keypoints. To simulate prediction uncertainty, we perturb the sampled keypoints with noise drawn from a zero-mean multivariate Gaussian distribution with spherical covariance o², These noisy keypoints are then used for encoding. Since depth errors in camera-alle camera-aligned show higher variance, we apply noise with standard deviation 30 along the depth axis in the camera frame.
Given a ground- -oound-truthhhao truth mesh V = [f....., w and its corresponding image I. We randomly sample M 3D points X (xI...., XM and compute the loss for each of the M points:
M M
SxL |Ax||| || ||Ax|1 /|2
axf
+ Xo ||Ax/11
|4x112
C
\
(3)
N
where x; is the i-th 3D query point, N is the number of ground-truth vertices, Ax! the predicted
displacement from x; the j-th point, and Ax, is the corresponding ground truth displacement. The symbol denotes the dot product, and || ||| represents the Euclidean (L2) norm. The parameter A1 controls the contribution of the directional loss term, which minimizes the angular deviation 1- 1-cos(f) between the predicted and ground truth vectors. The parameter A weights the magnitude loss, which penalizes differences in the length of the displacement vectors. To promote locality the extracted image features, this term is clipped during training. This weighted combination encourages both directional consistency and training stability. During training, we apply binary dropout to image features F, to enhance robustness against unreliable predicted neural fields. Additionally, we model the 3D keypoints used for encoding vertices the vertex branch as stochastic variables, introducing noise to the ground ground-truth keypoints only during training.

---

time
Test time
24
view 4
views 1
MVFNet 69
10ms
DENRMVS
3DDA-VS
10mms
RAFaRe
19s
HRR 30
1s
HED Pisten
PIFU 1
2.58
3a
JIFF 0
18
5a
H3D.Net 22
600s
1200s
1.2
SIRA+
90s
191s
GLVD (ours)
0.28
0.25s
Figure 6: Quantitative comparison on H3DS dataset with one and three input views. Left. Reconstruction error (mm) is plotted against runtime for various state-of-the-ar methods under different view configurations. Right. Inference times for single and multi-view settings.
Experiments
Training face dataset. We employ a proprietary dataset of 31) head scans collected from 10,000 individuals, balanced by gender and diverse in age and ethnicity. All scans are aligned to a template 3D model using non-rigid Iterative Closest Point (ICP) registration for consistency.
H3DS 2.0. [52.9] It contains 60 high-quality 3D full-head scans, including hair and shoulders, paired with posed RGB images. Each image includes foreground mask and calibrated camera parameters.
3DFAW. 47 This dataset provides videos recorded as well as mid-resolution 3D ground truth of the facial region. We select 5 male and 5 female scenes and use them to evaluate only the facial region.
CelebA-HQ. [27 This dataset comprises 30k high-quality images 1024x1024 resolution, derived from the original CelebA dataset. We selected a subset of 6 subjects for our qualitative evaluation.
4.1 3D Face estimation
We conducted a comprehensive comparison of our method with several 3DMM-based reconstruction works, including MVFNet (2), DENRMVS (69), DECA (17), MICA [8]] FaceScape [0), Feree
compared our approach to the JIFF (7), RAFaRe 22), H3D-Net (52) SIRA++ [9] and hybrid method LVD [T]. We used the unidirectional Chamfer distance for the quantitative evaluation, measuring the surface error from the ground truth to the predictions. The results of this comparison are summarized in Table Qualitative results for 3DFAW subjects are presented in Figure 4 for the single view and
(66], HRN (30]. 3DDFA-V vv3 (68) and VHAP (8). Additionally,
model-free methods PIFU JIFF
We show in Figure in Figure 5 for the multiview setting. Results on H3DS are presented in Figure B 7 the estimated 3D face and the guiding keypoints.
GLVD demonstrates consistently strong performance across 3DFAW and H3DS evaluations. leverages the structured topology of 3DMMs while explicitly addressing the constraint of shape representation to a predefined model space and the resulting bias toward average mean shape. As result, GLVD outperforms 3DMM-based approaches, particularly in the single-view setting (Figure D. In comparison to model-free single forward pass methods such as PIFU, JIFF and RAFaRe, it demonstrates superior performance in surface reconstruction, yielding surfaces with reduced errors and a more realistic shape appearance.
Optimization- based methods are considered state-of fatte-of-foe the-art for face reconstruction, particularly in multi-view settings where the problem becomes less ill ill-posed. However, their high computational cost remains a key limitation. In contrast, GLVD achieves comparable accuracy with over two orders of magnitude faster inference and without requiring postprocessing or template registration. Inference times are reported in Figure O with 10 iterative update steps.
8

---

4.2 Ablation
We conduct detailed ablation study on 3DFAW and H3DS 2.0 datasets to assess the impact of key design choices and demonstrate the effectiveness of the proposed method.
reportr quantitative results using various subsets of 3D keypoints guide GLVD While the method is agnos- tic to landmark topology and supports flexible selection, evaluate the impact of using different subsets derived from the template 3DMM. The specific landmark sets used detailed in the supplementary material. Notably, results show that small set of well-chosen keypoints provides effective global structural guidance, though limited cover- age can introduce noise. While increasing the number of keypoints improves performance, gains in reconstruction accuracy diminish beyond certain point. We attribute this the fact that the keypoints are estimated. nigher--qualty landmark supervision could further enhance reconstruc- tion fidelity, Figure 7 we provide face reconstruction results with the selected keypoints indicated for reference.
Table 2: Reconstruction quality compari- son using single view with varying numbers of keypoints. Chamfer distance reported in millimeters (mm).
3
H3DS
GLVD 4
1.92
1.51
GLVD 6
1.85
1.36
GLVD 12
1.57
1.42
GLVD 18
1.25
1.36
GLVD 24
1.58
1.33
Keypoints encoding. In the original LVD formulation, global structure not explicitly modeled, as point trajectories are predicted independently. with structural coherence Icarned implicitly through 2D feature volumes. This results in ambiguity, as the model must resolve all possible correspondences per query without clear global guidance. In GLVD, we address this by introducing landmark ensemble that serves as compact global prior. These keypoints act as spatial anchors that guide vertex displacements, reducing ambiguity and promoting consistent topology. We ablate various encoding strategies Table Replacing the per-vertex head with a global attention layer (k) introduces noise and instability, presumably due to long context. Inspired by the concept of skinning weights, model vertex-to -eeepoont relations via learnable attention (j), achieving performance on par with the standard encoding (l). Concatenation (h) and distance-based encoding (i) offer no gains, while removing absolute positions and using only relative encoding (g) leads a performance drop.
Architecture analysis. The GLVD architecture (Sec. 3 incorporates two core design choices: it estimates key- points progressively during optimiza- tion and encodes vertices relative to these dynamically predicted key-
Table 3: Comparison of reconstruction quality for different number of keypoints. Chamfer distance
3DFWW
H3DS 2.04
LVD
1.58
1.39
points. Table 3 presents an ablation
(b) GLVD w/o loss (Eq.
1.27
1.39
of different training strategies. Us-
GLVD w/o HRNet
1.52
1.39
ing an L2 loss with clipping, follow-
(d) GLVD w/o training Noise
1.31
1.42
ing [T] (b), results in unstable gra-
(c) GLVD w/o binary dropout
1.27
1.38
dients and weak directional supervi-
(f) GLVD w/ canonical space
1.51
1.52
sion. Incorporating a pre-trained HR-
Net for facial landmark prediction (c)
(g) GLVD w/o vertex pos.
1.55
1.67
highly improves performance both
(h) GLVD w/ concat Enc.
1.27
1.36
vertex reconstruction and keypoint
(i) GLVD w/ norm Enc.
1.26
1.37
estimation. Injecting noise during
(j) GLVD w/ attention Enc.
1.27
1.36
training (d) to model the encoding as a stochastic variable improves ro- bustness and test-time stability in the
(k) GLVD w/ attention layer
1.85
1.97
(I) GLVD
1.25
1.36
presence of uncertainty. Applying binary dropout to the 2D features during training (c) encourages reliance on the geometry-aware encoding and leads to better accuracy. Finally, optimizing in canonical space (f), as done in [52 [8] introduces bias towards the mean shape. In the single-vies view setting, we use camera coordinates as the reference space, which removes the need for calibrated camera at test time. We also investigate directly predicting the position of a specific vertex. We augment each query point with an identity feature derived from Fourier embedding of the vertex's 3D coordinates in the canonical face tem- plate. At test time, each sampled vertex is assigned unique target identifier, and the model predicts displacements toward all possible targets, after which the displacement corresponding the specified

---

view
views
Figure 7: Qualitative results for and three input images. Images from CelebA- CelebA-HQ (left), H3DS2.0 (right top). and 3DFAW (right bottom). At the last iteration step, we show the predicted template and the 3D keypoints in red.
identifier is selected. However, we empirically found it to be more stable and to yield better results by encoding trajectories implicitly as done in GLVD.
Discussion
While GLVD demonstrates strong performance in few-shot 3D face reconstruction, remains sensitive to occlusions and relies on the accuracy of keypoint predictions, which may degrade under challenging visual conditions. The focus of our method is the reconstruction of the face area by combining a hybrid method for fast and accurate prediction. Therefore, adding facial expressions is interesting future direction. Future work may explore temporal consistency for video-based reconstruction and topology-adaptive strategies to better capture complex geometry.
Limitations and Future Work:
Conclusions. In this paper, we have introduced GLVD, hybrid approach for high-fidelity 3D face reconstruction from few-shot images. Our method introduces a novel combination of per-vertex neural fields and dynamically predicted 3D keypoints to provide both local accuracy and global structural guidance. By encoding vertex displacements relative to a sparse set of learned keypoints, method refines mesh geometry iteratively without requiring parametric shape priors. The thorough evaluation demonstrated that our method achieves state-of-the performance in single-view settings and remains highly competitive in view scenarios, all while substantially reducing inference time
Acknowledgments
This work has been supported by the project GRAVATAR PID2023-151184011 funded by C21U/AE1/10.1303//550 100011033 and by ERDF, UE.
10

---

References
[1] Thiemo Alldieck, Mihai 7anfir, and Cristian Sminchisescu. Photorealistic monocular 3d reconstruction of humans wearing clothing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1506-1515, 2022.
[2] Ziqian Bai, Zhaopeng Cui, Jamal Ahmed Rahim, Xiaoming Liu, and Ping Tan, Deep facial non-rigid multi-view stereo. In CVPR, 2020.
[3] Linchao Bao, Xiangkai Lin, Yajing Chen, Haoxian Zhang. Sheng Wang. Xuefei Zhe, Di Kang. Haozhi Huang, Xinwei Jiang, Wang. Dong Yu, and Zhengyou Zhang. High-fidelity digital human head creation from rgb-d selfies. ACM Transactions on Graphics, 2021.
[4] Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti- aliased grid-based neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19697-19705, 2023.
[5] Shrisha Bharadwaj, Yufeng Zheng, Otmar Hilliges, Michael J. Black, and Victoria Fernandez Abrevaya. Flare: Fast learning of animatable and relightable mesh avatars, ACM Transactions on Graphics, 42:15, December 2023.
[6] Antonio Canela, Pol Caselles, Ibrar Malik, Eduard Ramon, Jaime García, Jordi Sânchez-Riera, Gil Triginer, and Francese Moreno-Noguer. Instantavatar: Efficient 3d head reconstruction via surface rendering, 2023.
Yukang Cao, Guanying Chen, Kai Han, Wenqi Yang, and Kwan-Yee K. Wong. Jiff: Jointly-aligned implicit face function for high quality single view clothed human reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[8] Pol Caselles, Eduard Ramon, Jaime Garcia, Xavier Giro-i Nicto, Francese Moreno-Nogee Noguer, and Gil Triginer. Sira: Relightable avatars from single image. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 775-784, January 2023.
[9] Pol Caselles, Eduard Ramon, Jaime Garcia, Gil Triginer, and Francese Moreno-Noguer. Implicit shape and appearance priors for few-shor full head reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
[10] Julian Chibane and Gerard Moll. Implicit feature networks for texture completion from partial 3d data. In Computer Visi 2020 Workshops: Glaxgow, UK, August 23-28, 2020, Proceedings, Part !! 16, pages 717-725, Springer, 2020.
[11] Enric Corona, Gerard Pons-Moll, Guillem Alenya, and Francesc Moreno-Noguer, Learned vertex descent: A new direction for 3d human model fitting. In European Conference on Computer Vision, pages 146-165. Springer, 2022.
[12] Enric Corona, Albert Pumarola, Guillem Alenya, Gerard Pons-Moll, and Francesc Moreno- Noguer. Smplicit: Topology-awas aware generative model for clothed people. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pages (1875-11885 2021.
[13] Enric Corona, Mihai Zanfir, Thiemo Alldieck, Eduard Gabriel Bazavan, Andrei Zanfir, and Cristian Sminchisescu. Structured 3d features for reconstructing controllable avatars. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages
2023.
[14] Vin De Silva and Joshua B Tenenbaum. Sparse multidimensional scaling using landmark points. Technical report, technical report, Stanford University, 2004.
[15] Helisa Dhamo, Yinyu Nic, Arthur Moreau, Jifci Song, Richard Shaw, Yiren Zhou, and Eduardo Pérez- Pellitero. Headgas: Real-time animatable head avatars via 3d gaussian splatting. In European Conference on Computer Vision, pages 459-476. Springer, 2024.
[16] Pengfei Dou and loannis Kakadiaris. Multi-viww 3d face reconstruction with deep recurrent neural networks. Image and Vision Computing, 80:80-91, 2018
[17] Yao Feng, Haiwen Feng, Michael Black, and Timo Bolkart. Learning an animatable detailed 3d face model from in-the-wild images. ACM Transactions on Graphics (TOG), 40(4):1-13, 2021.
[18] Chen Gao, Yichang Shih, Wei-Sheng Lai, Chia- Kai Liang, and Jia Bin Huang. Portrait neural radiance fields from a single image. arXiv preprint arXiv:2012.05900 2020.

---

[19] Simon Giebenhain, Tobias Kirschstein, Markos Georgopoulos. Martin Rünz, Lourdes Agapito, and Matthias NieBner. Mononphm: Dynamic head reconstruction from monocular videos. In Proc. IEEE Conf. on Computer Vixion and Pattern Recognition (CVPR). 2024.
[20] Philip-William William Grassal, Malte Prinzler, Titus Leistner, Carsten Rother, Matthias Nießner, and Justus Thics. Neural head avatars from monocular rgb videos. arXiv preprint arXiv:21 2021.
[21] Jianzhu Guo, Xiangyu Zhu, Yang Yang, Fan Yang, Zhen Lei, and Stan Z Li. Towards fast, accurate and stable 3d dense face alignment. In European Conference on Computer Vision, pages 152-168. Springer, 2020.
[22] Longwei Guo, Hao Zhu, Yuanxun Lu, Menghua Wu, and Xun Cao. Rafare: Learning robust and accurate non- con-aarmettric 3d face reconstruction from pscudo 3d&3d pairs. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 719-727, 2023.
[23] Tong He, John Collomosse, Hailin Jin, and Stefano Soatto. Geo-pifu: Geometry and pixel aligned implicit functions for single-view human reconstruction. Advances in Neural Information Processing Systems, 33:9276-9287, 2020.
[24] Tong He, Yuanlu Xu, Shunsuke Saito, Stefano Soatto, and Tony Tung. Arch++: Animation-ready clothed human reconstruction revisited. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1006611056, 2021.
[25] Zeng Huang, Yuanlu Xu, Christoph Lassner, Hao Li, and Tony Tung, Arch: Animatable reconstruction of clothed humans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1093-3104 2020.
[26] Aaron S Jackson, Adrian Bulat, Vasileios Argyriou, and Georgios Tzimiropoulos.. Large pose 3d face reconstruction from a single image via direct volumetric can regression. In ICCV, 2017.
[27] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint 170iv 10196, 2018.
[28] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023.
[29] Diederik P Kingma and Jimmy Ba. arXiv preprint arXiv:1412.6988 2014.
[30] Biwen Lei. Jianqiang Ren, Mengyang Feng, Miaomiao Cui, and Xuansong Xie. A hierarchical representa- tion network for accurate and detailed face reconstruction from in-the-wild images, 2023.
[31] Jiahui Lei, Yufu Wang, Georgios Pavlakos, Lingjie Liu, and Kostas Danilidis. Gart: Gaussian articulated template models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19876 19887, 2024.
[32] Tianye Li, Timo Bolkart, Michael. J. Black, Hao Li, and Javier Romero. Learning model of facial shape and expression from 4D scans. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 36(6): 194:1- 194:17, 2017.
[33] Jing Lin, Ailing Zeng, Haoqian Wang. Lei Zhang, and Yu Li. One-stage 3d whole-body mesh recovery with component aware transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21 21159-21168, 2023.
[34] Kevin Lin, Lijuan Wang, and Zicheng Liu. End-to-end human pose and mesh reconstruction with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1954-19 1964,199,,222 2021.
[35] Jiahao Luo, Jing Liu. and James Davis. Splatface: Gaussian splat face reconstruction leveraging an optimizable surface. arXiv preprint 187844440 2024.
[36] Xiaoxuan Ma, Jiajun Su, Chunyu Wang. Wentao Zhu, and Yizhou Wang. 3d human mesh estimation from virtual markers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 534 543, 2023.
[37] Riccardo Marin, Enric Corona, and Gerard Pons-Moll, Nicp: neural icp for 3d human registration at scale. In European Conference Computer Vision, pages 265 285. Springer, 2024.
12

---

[38] Ricardo Prerin-Bruuala,, Noha Radwan, Mehdi SM Sajjadi, Jonathan T Barron, Alexey Dosovitskiy, and Daniel Duckworth. Nerf in the wild: Neural radiance fields for unconstrained photo collections. In Pruceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7210-7219.
2021.
[39] Marko Mihajlovic, Aayush Bansal, Michael Zollhoefer, Siyu Tang, and Shunsuke Saito. KeypointNeRF: Generalizing image-hased volumetric avatars using relative spatial encoding of keypoints. In European conference computer vision, 2022.
[40] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. In European conference computer vision, pages 405 421. Springer, 2020.
[41] Francesc Moreno- Noguer and Pascal Fua. Stochastic exploration of ambiguities for nonrigid shape recovery. 35(2):463-45, 2013.
[42] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph., 41(4); 11:::100::11::0 July 2022.
[43] Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hourglass networks for human pose estimation. In Computer VisionEECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October I 11-14, 2016, Proceedings, Part VIII 14, pages 483-499. Springer, 2016.
[44] Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision. In CVPR, 2020.
[45] Keunhong Park, Utkarsh Sinha. Jonathan Barron, Sofien Bouaziz, Dan B Goldman. Steven M Seitz, and Ricardo
Nerfies: Deformable neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2021.
[46] Pascal Paysan, Reinhard Knothe, Brian Amberg, Sami Romdhani, and Thomas Vetter. 3d face model for pose and illumination invariant face recognition. In AVSS, pages 296-301, 2009.
[47] Rohith Krishnan Pillai, László Attila Jeni, Huiyuan Yang, Zheng Zhang, Lijun Yin, and Jeffrey F Cohn. The 2nd 3d face alignment in the wild challenge (3dfaw-vidco); Dense reconstruction from vidco. In ICCV Workshops, 2019.
[48] Shenhan Qian. Vhap: Versatile head alignment with adaptive appearance priors, sep 2024.
[49] Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, and Matthias Nießner. Gaussianavatars: Photorealistic head avatars with rigged 3d gaussians. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20299 20309, 2024.
[50] Amit Raj, Michael Zollhoefer, Tomas Simon, Jason Saragih, Shunsuke Saito, James Hays, and Stephen Lombardi, Pva: Pixel-aligned volumetric avatars. arXiv preprint arXiv:2101.0269) 2021.
[51] Eduard Ramon, Janna Escur, and Xavier Giro-i Nieto. Multi-view face reconstruction the wild using siamese networks. In /CCV Workshops, 2019.
[52] Eduard Ramon, Gil Triginer, Janna Escur. Albert Pumarola, Jaime Garcia, Xavier Giro-i Nieto, and Francesc Moreno-Noguer. H3d-net: Few-shot high-fidelity 3d head reconstruction. In ICCV, pages 5620-5623 2021.
[53] Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, and Michael J Black. Generating 3d faces using convolu- tional mesh autoencoders. In ECCV, 2018.
[54] Elad Richardson, Matan Sela, and Ron Kimmel. 3d face reconstruction by learning from synthetic data. In 3DV, 2016.
[55] Elad Richardson, Matan Sela, Roy Or-El, Ron Kimmel, Learning detailed face reconstruction from a single image. CVPR, 2017.
[56] Zeyu Ruan, Changqing Zou, Longhai Wu, Gangshan Wu, and Limin Wang. Sadmet:: Self-aligned dual face regression networks for robust 3d dense face alignment and reconstruction. IEEE Transactions on Image Processing, 30:5793-5806, 2021.
[57] Shunsuke Saito, Zeng Huang. Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. Pifu: Pixel- Pixel-aligned implicit function for high-resolution clothed human digitization. In ICCV, 2019.
13

---

[58] Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization. In Proceedings the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 84-93, 2020.
[59] Akash Sengupta. Ignas Budvytis, and Roberto Cipolla. Synthetic training for accurate 3d human pose and shape estimation in the wild. arXiv preprint arXiv:2009. 10011 2020.
[60] Ruizhi Shao, Hongwen Zhang. He Zhang, Mingjia Chen, Yan-Pet Pei Cao, Tao Yu, and Yebin Liu. Doublefield: Bridging the neural surface and radiance fields for high-fidelity human reconstruction and rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15872- 158822, 2022.
[61] Jiapeng Tang, Davide Davoli, Tobias Kirschstein, Liam Schoneveld, and Matthias Niessner. Gaf: Gaussian avatar reconstruction from monocular videos via multi-view diffusion, arXiv preprint arXiv:2412. 10209, 2024.
[62] Ayush Tewari, Michael Zollhofer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, and Christian Theobalt. Mofa: Model-based deep convolutional face autoencoder for unsupervised monocular reconstruction. In /CCV, 2017.
[63] Anh Tuan Tran, Tal Hassner, Iacopo Masi, Eran Paz, Yuval Nirkin, and Gérard G Medioni. Extreme 3d face reconstruction: Seeing through occlusions. In CVPR, 2018.
[64] Anh Tuan Tran, Tal Hassner, Iacopo Masi, and Gérard Medioni. Regressing robust and discriminative 3d morphable models with very deep neural network. In CVPR, 2017.
[65] Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang. et al. Deep high-resolution representation learning for visual recognition. IEEE transactions on pattern analysis and machine intelligence, 43(10): 33300033349-33333 3364, 2020.
[66] Lizhen Wang, Zhiyua Chen, Tao Yu, Chenguang Ma, Liang Li, and Yebin Liu. Faceverse: a fine- grained and dettal-controllaalle 3d face morphahle model from a hybrid dataset. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[67] Yiming Wang, Qin Han, Marc Habermann, Kostas Danilidis, Christian Theobalt, and Lingjie Liu. Neus2: Fast learning of neural implicit surfaces for multi-view reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
[68] Zidu Wang, Xiangyu Zhu, Tianshuo Zhang, Baiqin Wang, and Zhen Lei. 3d face reconstruction with the geometric guidance of facial part segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1672-1682, 2024.
[69] Fanzi Wu, Linchao Bao, Yajing Chen, Yonggen Ling, Yibing Song. Songnan King Ngi Ngan, and Wei Liu. Myf-net: Multi-view 3d face morphable model regression. CVPR, 2019.
[70] Yuxin Wu and Kaiming He. Group normalization. In Proceedings of the European conference on computer vision (ECCV), pages 3-1 2018
[71] Yuelang Xu, Zhaoqi Su, Qingyao Wu, and Yebin Liu. Gphm: Gaussian parametric head model for monocular head avalar reconstruction. arXiv preprint arXiv:2407.15000 2024.
[72] Yusuke Yoshiyasu. Deformable mesh transformer for 3d human mesh recovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 17005-17015, 2023.
[73] Hongwen Zhang, Yating Tian, Yuxiang Zhang, Mengcheng Li, Liang An, Zhenan Sun, and Yebin Liu. Pymaf-x: Towards well well-aligned full-body model regression from monocular images. IEEE Transactions on Pattern Analysis and Machine Intelligence, (5(100)12227 - 2023.
[74] Mingwu Zheng, Hongyu Yang, Di Huang, and Liming Chen. Imface: nonlinear 3d morphable face model with implicit neural representations. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 20343-20352, 2022.
[75] Mingwu Zheng, Haiyu Zhang, Hongyu Yang, I Liming Chen, and Di Huang. Imface+4: A sophisticated nonlinear 3d morphable face model with implicit neural representations. arXiv preprint arXiv:2312.04000 2023.
[76] Mingwu Zheng, Haiyu Zhang, Hongyu Yang, and Di Huang. Neuface: Realistic 3d neural face rendering from multi-view images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18868-16877, 2023.
14

---

[77] Xiaozheng Zheng, Chao Wen, Zhaohu Li, Weiyi Zhang, Zhuo Su, Xu Chang, Yang Zhao, Zheng Lv. Xiaoyuan Zhang, Yongjie Zhang, al. Headgap: Few-shot 3d head avatar via generalizable gaussian priors. arXiv preprint arXiv:2408. 06019, 2024.
[78] Yuleng Zheng, Victoria Fernández Abrevaya. Marcel C. Bühler, Xu Chen, Michael J. Black, and Otmar Hilliges. M Avatar: Implicit morphable head avatars from videos. Computer Vision and Pattern Recognition (CVPR), 2022.
[79] Yufeng Zheng, Wang Yifan, Gordon Wetzstein, Michael Black, Otmar Hilliges, Pointavatar: Deformable point- based head avatars from videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2023.
[80] Hao Zhu, Haotian Yang, Longwei Guo, Yidi Zhang. Yanru Wang. Mingkai Huang, Menghua Wu, Qiu Shen, Ruigang Yang, and Xun Cao. Facescape: 3d facial datasct and benchmark for single-view 3d face reconstruction. IEEE Transactions Pattern Analysis and Machine Intelligence (TPAMI), 2023.
[81] Wojciech Zielonka. Timo Bolkart, and Justus Thies. Towards metrical reconstruction of human faces. In European Conference Computer Vision (ECCV). Springer International Publishing, October 2022.
[82] Wojciech Ziclonka, Timo Bolkart, and Justus Thies. Instant volumetric head avatars. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 4574 4584, 2023.
15

---

NeurIPS Paper Checklist
1. Claims
Question: the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We propose an optimization method, GLVD, that archives competitive results for the task of 3D face reconstruction. We encode global awareness through iterative 3D landmarks prediction. All experiments showed in the paper are designed evaluate the main claims.
Guidelincs:
The answer NA mcans that the abstract and introduction do include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can he expected to generalize to other settings.
is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The main document has a limitation future work section. We also add extra experiments violating some of the assumptions that are added in the supplementary material.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings. model well- sell-specification asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice
what implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. general, empirical results often depend on implicit assumptions, which should he articulated.
The authors should reflect on the factors that inffuence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with datasct size.
applicable, the authors should discuss possible limitations of their approach address problems privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't it acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve the integrity the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
For each thcoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Question:
Answer: [No]
Justification: No theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
16

---

All the theorems, formulas, and proofs in the paper should be numbered and cross tross-reference.
All assumptions should he clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide detailed explanation of the proposed model well how has been trained.
Guidelines:
The answer means that the paper does not include experiments. the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on contribution, reproducibility can be accomplished various ways. For example, if the contrihution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others replicate the model with the same dataset, or provide access the model. general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of model checkpoint, or other means that are appropriate to the rescarch performed.
While NeurIPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility. which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. a large language model), then there should either be a way to access this model for reproducing the results a way to reproduce the model (e.g.. with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source source models, it may he that access to the model is limited in some way (e.g.. registered users), hut it should be possible for other rescarchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does paper provide open access to the data code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Due to legal constraints, we are unable to release the training data. However, we will publish the code along with detailed instructions on how to prepare compatible datasets for training.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines stttpe://nips..c//uutt for more details.
guides/ Code/oooeeeeeeee
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
17

---

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines httpp////niaaccppuaabbpeee for more details.
Suidees/DoeeSubbissioonnn
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g,, data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Implementation details are explained in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level detail that necessary to appreciate the results and make sense of them.
TTe full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error hars suitably and correctly defined or other appropriate informa- tion about the statistical significance the experiments?
Answer: [No]
Justification: While not all prior works report statistical significance, we ensure rigorous evaluation by testing our methods across diverse datasets and varying numbers of input views. We also include qualitative comparisons to visually assess reconstruction quality.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanicd by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful symmetric error bars would yield results that are out of range (e.g. negative error rates). error bars reported in tables plots, The authors should explain in the text how they calculated and reference the corresponding figures or tables the text.
show in tables figures
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
Implementation details as well as training recipe is explained supplementary
details.
Guidelines:
18

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Fthies http:///rr.pp..ccpppltdeeee/e//a
Answer:
[Yes]
Justification:
Guidelines:
The answer means that the authors have not reviewed the NeurlPS Code of Fthics.
the authors answer No, they should explain special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., if there is a special considcration duc to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We provide socictal impact discussion in the appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious or unintended uses (c.g.. disinformation, generating fake profiles, surveillance), fairness considcrations (c.g., deploy- ment technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is logitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it not needed point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated relcase models, providing defenses in addition attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g.. pretrained language models, image generators, scraped datasets)?
Question:
Answer: [Yes]
Justification: Access to the models and data is available upon request and subject to the completion of a formal questionnaire and the signing of a legal agreement in advance.
Guidelines:
The answer NA means that the paper poses such risks.
19

---

Released models that have high risk for misuse or dual-use should be released with necessary safeguards allow for controlled use of the model, for example by requiring that adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been seraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We provide proper citation for all prior work on which our approach based.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (c.g., c. CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are relcased, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppppppphseeeooe thcode. com/ datasete has curated licenses for some datasets. Their licensing guide can help determine the license a dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it changed) should provided.
this information not available online, the authors are encouraged reach out to asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: release full codehase on GitHub, including model implementation, training, and evaluation scripts (to be made public upon publication).
Guidelines:
The answer NA means that the paper does not release assets.
Rescarchers should communicate the details of the dataset/code/moodll as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Although our research involves facial data, we did not collect new human data nor conduct any additional data acquisition for this work.
Guidelines:
The answer means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
20

---

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [Yes]
Justification:
We followed the ethical guidelines established by NeurIPS.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval. you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and Iocations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
initial submissions, not include any information that would break anonymity applica- ble), such as institution conducting the review.
16. Declaration of LLL usage
Question: Does the paper describe the usage of LLMs it an important, original, or non-stande aon-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing. formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is required.
Answer: [NA]
Justification: LLMs not involved any important, original components.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer our LLM http://////oou...cc/////oooeo/o/ should or should not described.
for what
21

---

A
Appendix: Guided Learned Vertex Descend
In this appendix, we provide further technical details on
Experimental setup
Different Keypoints configuration
Implementation Details
Additional qualitative results
Failure Cases
For video results, including visual comparison prior work, refer supplementary video. This video includes demonstration GLVD different input images.
A.1 Experimental Setup
GLVD adapts its reference space based the number of input views during training and inference. For single-view 3D reconstruction, it operates
camera coordinate frame, eliminating need camera parameter estimation at test time. the multi-view setting, we canonicalize the 3D reconstruction and train camera pose estimator on the same dataset used for training GLVD to enable prediction at inference time.
To ensure a fair comparison, PIFU, JIFF, LVD, and SIRA+ are trained using the same data used to train GLVD While PIFU, LVD, and JIFF were initially designed for full-bedy reconstruction, modified their training align with the data used by GLVD To enhance robustness during training. applied data augmentation techniques, including adjustments to brightness, contrast, hue, saturation, as well as image jittering, blurring. and zooming. These augmentations are applied to the input images used for feature extraction. Additionally, we employed scene symmetrization, doubling the number of training scenes.
A.2 Keypoints configuration
GLVD requires only RGB images as input to predict the surface. Internally, it operates by estimating keypoints. (ngure&ppeeentie visualizations of four proposed landmark subsets. The method is designed to allow flexible selection of landmark configurations. In experiments, we use template vertices registered to the training scenes. To adapt the method to other parametric models, such FLAME SMPL-X, joints be selected keypoints.
4 Landmarks
6 Landmarks
12 Landmarks
18 Landmarks
Figure 8: Qualitative visualization of four keypoints configurations. Images are from H3DS 2.0. At the last iteration step, we show the predicted template and the 3D keypoints red.
A.3 Implementation Details
The function f-()) a stacked hourglass network 43 composed of four stacks using group normalization (70]
Feature embeddings have spatial resolution of 64 x 64, with cach containing 64 channels. As result, cach query point is represented by stacking four feature vectors of size 64 x = 256. We pre-train fo(-) predict the signed distance function (SDF) values using the 0.1, being scene normalized in centered cube of size
training data. We set the clipping directional factor
The function fa(-) implemented by combination
facial keypoint heatmap estimator HRNet 651 and
single-stack hourglass network 44 During training, we keep the weights of the HRNet frozen. To generate Ft. we extract the first feature map computed with f.(()) and then concatenate it with the heatmaps predicted by HRNet. The combination of the features and the image I and the mask M is then fed to the hourglass network.
22

---

1 view
Figure 9: Qualitative results for one input image. Images are from 3DFAW (top), and H3DS 2.0 (bottom). At the last iteration step, we show the predicted template and the 3D keypoints red.
Feature embeddings have spatial resolution of 64 x 64, with each containing 64 channels. We use 0.1 clipping factor.
Function g=(-) produces an output tensor of dimension N 7225 x 3. Given an input surface size 7225 x 3. outputs a tensor of shape 7225 x 7225 x 3. Per-vertex displacements (7225 x 3) are extracted from the diagonal and applied to update vertex positions.
Table 4: Total number of parameters.
Parameters
Ratio
f.(-)
14.08 M
54.9%
GLVD works for different numbers of input images. When several images are used, we adopt a mean aggregation layer among features extracted from a multi-view feature encoder. In particular, we follow a single view forward pass independently of the number of input images until the second layer of the and gx(-), where apply mean operation aggregate multiview features.
9-((-)
11.57
45.1%
Total
25.64
100%
fa(+) (HRNet)
9.65
67.2%
f.(-) (Hourglass)
4.28
29.8%
(-)-)
0.43 M
2.96%
Total
14.36 M
100%
Function (-)-) produces an output tensor of dimension K 18 x 3. Given an input surface size 18 x 3, it outputs a tensor of shape 18 x 18 x 3. Per-vertex displacements (18 x 3) are extracted from the diagonal and applied to update keypoint positions. Both (-) and ga are implemented 3- 3-Layer MLP with ReLU activation and weight normalization.
All networks are trained end-to-end using GPU-aceelerated hardware (RTX 4090). We use batch size of and an initial learning rate 0.001 for 50 epochs, followed by 200 additional epochs with linear learning rate decay. For each scene, we sample 1400 vertices as query points. takes between 1.5 to 6 days of training, depending on the configuration. We set A X2 0.5. Optimization is performed using Adam 29 with 3] 0.9 and 82 0.999. The total number of parameters is detailed in Table
A.4 Aditional results
We provide qualitative results for subjects from 3DFAW and H3DS 2.0 using single input view in Figure 9 and for the multi-view setting in Figure 10 We also show qualitative results the wild CelebA-HI dataset Figure
Figure reports the reconstruction етror on 3DFAW subjects under varying numbers of update steps and clipping factor. At test time, the magnitude of the predicted displacement vector is clipped within the range [0.05, 0.5). Results indicate that the number of update steps has a limited impact on reconstruction accuracy, which is strongly influenced by the clipping value used during training. This parameter controls the trade-off between accuracy and computational cost. Our experiments achieve the best performance with 10 steps and clipping factor of 0.
We conducted an ablation study evaluating sequential parallel update strategies the vertex and keypoints refinement modules on the H3DS 2.0 and 3DFAW datasets (T) The results demonstrate that the iterative parallel update scheme yields consistently superior performance compared to the sequential alternative, although the improvement is minor.
23

---

3 views
Figure 10: Qualitative results for three input images. Images are from 3DFAW. At the last iteration step, we show the predicted template and the 3D keypoints in red.
Table 5: Ablation of sequential versus iterative (parallel) update strategies H3DS 2.0 and 3DFAW.
H3DS 2.0 Dataset
view
3 views
4 views +
6 views !
LVD
1.55
1.49
1.44
1,42
LVD pre-trained SDF
1.39
1.45
1.39
1.37
GLVD Sequential
1.38
1.36
1.34
1.35
GLVD Iterative
1.36
1.33
1.34
1.34
3DFAW Dataset
1 view
3 views
LVD
1.65
1.52
LVD pre-trained SDF
1.58
1.26
GLVD Sequential
1.29
1.23
GLVD Iterative
1.25
1.22
We also demonstrate that pre-training the feature encoder on a 3D reconstruction task, where it is trained to predict signed distance functions (SDFs), leads to faster convergence and improved performance. In this setting, (1) we pre-train the feature encoder the 3D reconstruction task. We represent the surface S the zero-level set of signed distance function yلlr (x, - s. such that (x E R f (x,1) = 0}. Our goal to estimate fall through composition of feature encoder and decoder network. The resulting feature encoder is non-watertight scans from the training dataset and minimize ((O) surface points N, and c( throughout the volume N.:
then used within GLVD. To train the SDF task,
Lsut
1
Σ
ا (x: (x:
(4)
N.
V (x. D/1 -1)²
LEK
(5)
N.
The loss averaged over the batch. We compare in Table the impact of pretraining. We evaluate LVD with and without SDF based pretraining of the feature encoder. Results show that this pretraining is crucial for achieving strong performance in both LVD and GLVD.
24

---

Clipping factor
Clig: 0.05
Clip:
Clip: 0.2
Clig: 0.3
Recopstruction Ervor (mm)
Clip:
Clige: 0.5
Steps
10
Figure 11: Reconstruction Error from Single Input Image. Results report the mean Chamfer Distance on the 3DFAW dataset.
A.5 Failure Cases
We present failure cases of GLVD under varying numbers of input views. Figure 12 illustrates qualitative results in extreme scenarios, while reports quantitative performance for different viewing angles in the single-view setting. The best performance is observed with front-facing input images. As the viewing angle increases, performance degrades significantly, primarily due inaccuracies in landmark estimation under self-occlusion conditions.
1 view / Input
Prediction
1 view / Input
Prediction
Figure 12: Reconstruction Error from a Single Input Image. Results report failure cases.
A.6 Social Impact
GLVD advances face modeling with high accuracy, enabling applications in graphics, AR/VR, biometries. However. also raises concerns about privacy, surveillance, and identity misuse. High-fidelity face reconstruction can be used without consent or for impersonation, contributing to deepfake risks. To mitigate these issues, responsible deployment, fairness audits, and privacy safeguards essential. While GLVD technical step forward, its socictal implications must be carefully considered.
Table 6: Quantitative Results for a Single Input Image at varying input angles. Chamfer Distance is reported in millimeters (mm) on the H3DS 2.0 dataset.
view 0º
view 45
view 90"
GLVD
1.31
2.07
2.11
25

---

GLVD
25
25
25
2
25
2
(ours)
SIRA++
U
2
2
26
RAFaRe
2
2
PIFU
JIFF
25
2
26
2
MICA
DECA
R
2
2
2
Verse
FaceScape
HRN
3DDFA-V3
28
Figure 13: Qualitative results the CelebA-HQ dataset for single input image.
26


Reviews:
[{"review_id": "lW3wtA3036", "reviewer": "Reviewer_5vBD", "summary": "This paper proposes GLVD which extends LVD by combining relative position encoding to guide its dense reconstruction with sparse landmarks.", "questions": "Please see Weakness\n\n- How much is the feature maps resolution? Are they same for landmarks reconstruction and dense vertex reconstruction? If so, are there cases that the image feature maps are too high-res for landmarks reconstruction e.g. prone to noise when reprojected on far away locations than the true landmarks or  too low-res for dense vertex reconstruction.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strength:\n\n- The paper shows an interesting application of non-parametric shape learning models to facial shape reconstruction, where parametric models often face difficulties in reconstructing individual shapes. \n- It outperforms previous approaches in this field and by large margin the original LVD.\n- Reconstructed shape quality looks high\n- The method is able to utilize image features effectively in a coarse-to-fine manner.\n\nWeakness:\n\nA. Its novelty seems limited as it looks like a combination of LVD and KeypointNerf. \n\nB. The use of landmarks, coarse mesh and bones has already been explored in various domains. And there is auto encoder like approaches for learning low-dimensional features. And non-parametric models and multi-scale features are used in HMR. It would be appreciated to position the proposed coarse to fine approach in the literature. The related work section is rather too much focused toward NeRF and GS.\n\n1) Dimensionality reduction:\n - Sparse multidimensional scaling using landmark points\n\n2) HMR\n- End-to-End Human Pose and Mesh Reconstruction with Transformers\n- Deformable Mesh Transformer for 3D Human Mesh Recovery\n- 3D Human Mesh Estimation from Virtual Markers\n\n- One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer\n- PyMaf-X\n\n3)  Bones and latent features:\n- GART: Gaussian Articulated Template Models\n- Generating 3D faces using Convolutional Mesh Autoencoders\n\nC. Section 3.4 is unclear to me i.e. how the displacement at vertex is obtained from P.", "was_revised": true, "final_justification": "Most of my concerns are resolved and thus I'd keep my original score.", "rebuttal": "Thank you very much for your positive evaluation of our paper. We are glad you found it to be interesting.\n\n---\n\n## Novelty\n\nWhile there is extensive literature on human mesh recovery (HMR), most methods remain unexplored in other domains such as human face modeling. From the perspective of shape representation, these approaches can be broadly categorized into three groups: parametric body models, volumetric shape models, and vertex-based methods.\nIn this paper, we build upon the hybrid LVD framework and introduce key design modifications to adapt the vertex-based model for accurate human face modeling. We refer to our response to Reviewer nu3K for a complete discussion of the novelty. \n\nWe summarize our contributions as follows:\n\n- **Keypoint-guided updates**: Introduces a two-branch architecture where 3D keypoints guide vertex refinement.\n- **Relative encoding**: Encodes vertices relative to keypoints for pose-aware, view-independent updates.\n- **2D priors over 3D supervision**: Uses image features from 2D landmark detector to improve performance without requiring external paired 3D data.\n- **Enhanced training**: Improves accuracy through SDF pretraining, noise injection, canonical space, and a proposed loss.\n\nWe now address the main differences between KeypointNeRF, LVD, and GLVD:\n\n**KeypointNeRF** learns generalizable neural radiance fields by encoding query points as depth differences relative to 3D keypoints for each view. In the multiview setting, it relies on a neural network to blend view-dependent features. In contrast, GLVD performs a view-independent encoding based on Euclidean distances directly on the 3D vertices, as detailed in Section 3.4. This formulation enables the model to predict vertex displacements relative to the global 3D keypoints, ensuring consistent geometry regardless of the number of input views used to estimate them.\n\nWe build upon **LVD**, but our formulation introduces several key differences. LVD lacks explicit modeling of global structure and predicts point trajectories independently, without shared context. To address this, we propose a two-branch architecture: a keypoint branch that estimates global structure and a vertex branch that operates in an encoded 3D space computed from these keypoints at each optimization step.\n\nA simple baseline is to train the function $ f_k(\\cdot) $ from scratch and allow the model to learn the proxy 3D landmark representation directly. As shown in Table 3, row (c), this setup yields only minor improvements over LVD. However, incorporating a pretrained 2D landmark detector trained without paired 3D data leads to a significant performance improvement using the same training set. Simply concatenating HRNet features to LVD does not improve results, which highlights the importance of our structural integration and relative encoding.\n\nTo make **GLVD** perform effectively, we introduce several modifications to the original LVD framework, as described in Section 4.2. These include pretraining the image feature extractor on a signed distance function task using the same dataset (we refer the reader to our response to Reviewer xEDi), introducing a custom loss over vertex displacements, injecting noise into the predicted keypoints during training to improve robustness, and using a canonical coordinate system in the multiview setting to ensure geometric consistency.\n\n---\n\n## Related Work\n\nThank you for bringing the papers [1–8] to our attention. While most of them address human mesh recovery, we agree that they are relevant to our work and we will cite and include them in the paper's discussion.\n\nIn [1], the authors proposed a three-step approach: (1) selecting landmark points, (2) applying MDS to embed these landmarks in a low-dimensional space, and (3) using distance-based triangulation to embed the remaining points. In GLVD, we adopt a similar strategy by selecting landmarks from facial regions that capture identity-specific information. We then use Euclidean distances to encode mesh vertices relative to these landmarks within the learned space.\n\nSome of the cited references [2–5] primarily target full-body human mesh recovery and remain unexplored in the context of face modeling. These methods often operate in single-image settings [2–5] or on video sequences [7].\n\n**METRO** [2] and **DeFormer** [3] leverage transformer architectures that process mesh joints and vertices jointly as sequences of queries. While differing in attention aggregation strategies, both employ global reasoning via self-attention to capture structural dependencies. However, METRO conditions the model using a global image embedding extracted from a CNN backbone, thereby discarding pixel-aligned spatial detail.\n\nIn contrast, **3D Virtual Markers** [4] predicts a set of latent 3D markers and estimates the full mesh as a linear combination of these, effectively collapsing the image into a low-dimensional latent space, similar to traditional 3DMM pipelines. **One-Stage Mesh Recovery** [5] regresses SMPL parameters directly from image features using a transformer. Both approaches are inherently limited by the expressiveness of their low-dimensional latent spaces.\n\n**PyMAF-X** [6] introduces an iterative refinement loop using pixel-aligned features sampled from previous vertex estimates. However, it remains constrained to SMPL parameter regression and does not incorporate explicit landmark or keypoint guidance for structural supervision.\n\n**GART** [7] leverages bones and temporal video input to reconstruct animatable avatars. In contrast, **GLVD** is purely image-driven, requiring neither video nor skeletal priors, and is applicable in monocular and few-shot settings.\n\nFinally, [8] proposes replacing traditional PCA-based 3D face models with convolutional mesh autoencoders for learning facial shape priors. However, this work focuses on latent mesh encoding and does not explore the direct estimation of facial geometry from RGB images, which is central to GLVD.\n\n---\n\n## Questions\n\n(1) We empirically tested various spatial resolutions and found that a feature resolution of 64×64×256 (H, W, C) is sufficient to achieve competitive results. We use this resolution for both branches, although this is not a strict requirement. Our formulation is flexible and can accommodate alternative sources for predicting 3D keypoints. In our experiments, the face occupies most of the image, and we did not observe any noticeable degradation in performance.\n\nWe provide detailed network implementation details in the supplementary material. \n\n(2) In Section 3.4, we introduce $P$, which represents the vertices encoded relative to the keypoints. To compute the next vertex state, we refer to Eq. 1, where the feature map is predicted by $f_v$, and the input vertex $v_i$ corresponds to its encoded representation $P_i$.\n\n\n---\n\n## References\n\n[1] Sparse multidimensional scaling using landmark points\n\n[2] (METRO) End-to-End Human Pose and Mesh Reconstruction with Transformers\n\n[3] Deformable Mesh Transformer for 3D Human Mesh Recovery\n\n[4] 3D Human Mesh Estimation from Virtual Markers\n\n[5] One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer\n\n[6] PyMaf-X\n\n[7] GART: Gaussian Articulated Template Models\n\n[8] Generating 3D faces using Convolutional Mesh Autoencoders"}, {"review_id": "ZZIGdT8wE2", "reviewer": "Reviewer_nu3K", "summary": "This paper addresses the problem of reconstructing 3D human faces from images. Existing methods mainly fall into two categories: learning-based approaches, which are efficient but often less accurate, and optimization-based approaches, which offer high fidelity but are computationally intensive. The authors propose a hybrid method called Guided Learned Vertex Descent (GLVD), which builds on the Learned Vertex Descent (LVD) framework. GLVD predicts deformations of mesh vertices with a fixed topology of given time step, aiming to combine the accuracy of optimization-based methods with the efficiency of learning-based models. To adapt LVD specifically for face reconstruction, the authors introduce 3D face landmark-based guidance, enhancing the structural coherence of the reconstruction. As a result, their approach achieves state-of-the-art face reconstruction performance while reducing inference time.", "questions": "1. The authors demonstrate the effectiveness of the proposed method, but the technical contribution may appear as a straightforward extension of LVD to face reconstruction. Could you clarify in detail what distinguishes GLVD from simply applying LVD to the face reconstruction task? Specifically, what are the key improvements or advantages?\n2. Could the authors provide more analysis or examples of failure cases? In particular, under what conditions does GLVD struggle compared to optimization-based methods? Additionally, could you discuss how to overcome the performance gap in multi-view or less ill-posed scenarios where optimization-based methods perform better?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n- The paper is clearly written, easy to follow, and well-structured.\n- The proposed approach is simple yet effective, and its effectiveness is demonstrated through comprehensive experiments and detailed ablation studies.\n- The authors provide thorough comparisons and clearly show the advantages of their method in both accuracy and efficiency.\n\n### Weaknesses\n\n- The technical novelty beyond LVD feels somewhat limited.\n- While the method achieves significant improvements, it does not surpass optimization-based methods in multi-view settings.", "was_revised": true, "final_justification": "Most of my concerns have been well resolved through the rebuttal, so I will keep my rating.", "rebuttal": "Thank you very much for your positive evaluation of our paper. We are glad you found it to be interesting and well-written.\n\n---\n\n## LVD vs GLVD\n\nWe expand our response here to avoid repeating points already addressed in the Novelty section of our reply to Reviewer 5vBD and xEDi.\n\nWe summarize our key improvements below:\n\n1. We incorporate global reasoning using a relative spatial encoding scheme.  \n2. We propose a two-branch LVD architecture to learn vertex and keypoint representations jointly.  \n3. We present an extension of LVD to the multiview setting.  \n4. We introduce $f_k$ to integrate features from off-the-shelf models trained without requiring expensive 3D training data, improving flexibility and efficiency in the pipeline.  \n5. We introduce several training strategies that are essential for achieving high-quality reconstructions, as explained in Section 4.2 and in the supplementary material\n\nTwo key empirical advantages of GLVD over LVD are its ability to reduce surface artifacts and its increased robustness against collapsed outputs. For a fair comparison, both LVD and GLVD are trained on the same dataset. As shown in Figure 4, LVD produces noticeable artifacts on the face surface, whereas GLVD yields smoother and more stable reconstructions. In Table 2 (a), we report the results of directly applying LVD to the face reconstruction task. When used without modification, LVD yields lower performance compared to the proposed GLVD, highlighting the importance of our architectural and training improvements in both qualitative and quantitative evaluations.\n\n---\n\n## Optimization-based Methods\n\nOptimization-based methods tend to outperform feedforward approaches when sufficient test data is available. However, in highly ill-posed scenarios, typically with a single input image or complex image occlusion, feedforward methods often achieve better results due to their strong learned priors and constrained representation space. GLVD tends to struggle with out-of-distribution test samples, as it relies on patterns seen during training. In contrast, optimization-based methods can adapt more effectively at test time to handle complex or rare cases.\n\nOptimization-based methods are typically orders of magnitude slower and are not directly comparable to feedforward approaches. We highlight this difference in Figure 6. GLVD is inspired by neural field methods commonly used in state-of-the-art avatar reconstruction and adopts a feedforward formulation with an error feedback loop.\n\nTo close the performance gap in multiview or less ill-posed scenarios, one could address the limitations in two main ways: by increasing the amount of training data to improve generalization, or by exploring architectural design choices better suited for multiview reconstruction."}, {"review_id": "Lo50FV1Wdv", "reviewer": "Reviewer_kQcU", "summary": "Breaking free from the representational constraints of traditional 3D Morphable Models (3DMMs) is a promising direction in 3D face reconstruction. This paper takes a step in that direction by extending the existing Learned Vertex Descent (LVD) framework—originally designed for human body reconstruction—into the facial domain. The contribution lies in integrating per-vertex neural field optimization with global structural guidance derived from dynamically predicted 3D landmarks.", "questions": "1. I would like to see more results using GLVD on face images with diverse expressions, as current examples mainly focus on neutral or slightly varied identities.\n\n2. Given the motivation of the paper—overcoming the representation limitations of 3DMMs—this could manifest in two aspects: (1) modeling fine-scale geometry such as wrinkles and dimples, and (2) reconstructing large or micro expression changes. I would be interested in results, theoretical analysis, or discussion from the authors regarding GLVD’s capability in either direction.\n\n3. After reviewing both the main text and supplementary material, it seems that the keypoint selection in the 3D Keypoint Branch does not account for expressive regions, e.g., upper/lower eyelids or mouth lips. Would this limit GLVD’s ability to reconstruct facial expressions accurately?\n\n4. A question for discussion: based on the current dual-branch LVD structure in Figure 2, could the authors consider adding a third “texture LVD” branch to enhance texture reconstruction? This could further address the limited texture representation space of 3DMMs.\n\n5. Figure 2 would benefit from clear labeling of the “3D Keypoint Branch” and “3D Vertex Branch” to improve readability and understanding of the architecture.\n\n6. In Table 1, the method HRN seems to support multi-view reconstruction, but this is not discussed in the paper. Moreover, recent optimization-based methods such as VHAP [2] have also shown the potential to go beyond morphable model priors. \n\n7. In Lines 199–201, the authors mention employing a proprietary dataset of 10,000 3D head scans. This is a large-scale dataset by the standards of the 3D face community. I am curious about more details of the data, such as mesh resolution (vertex count or face density), scanning quality, and variation.\n\n[2] VHAP: Versatile Head Alignment with Adaptive Appearance Priors", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Attempting to overcome the representation space limitations of 3DMMs is a promising direction.\n\n2. The paper is easy to follow.\n\nWeaknesses:\n1. From the perspective of novelty, the innovation level of this paper seems to fall below the expectations of NeurIPS. Its main framework is based on the Learned Vertex Descent (LVD) framework, and the improvement lies in introducing a face-landmark-based branch similar to LVD. The overall architecture is similar to a dual-path LVD.\n\n2. From the reconstruction results presented in the paper, although it can reconstruct faces of different body types with near-neutral expressions, it seems difficult to reconstruct expression changes such as closed eyes or open mouths (e.g., in the top right corner of Figure 7, the reconstruction fails to capture the closed-eye expression of the male subject).\n\n3. The motivation of the paper is to break through the limitations of 3DMMs, but in terms of shape representation, the reconstructed results appear overly smooth (seemingly unable to reconstruct fine details such as wrinkles or dimples), making it hard to demonstrate superiority over 3DMMs. Additionally, another limitation of 3DMMs is the restricted expression space, and the paper does not show how GLVD addresses expression reconstruction.\n\n4. Please also refer to the “Questions” section.\n\n**AC note:**\nI would like to bring to the AC’s attention that Figures 4, 5, and Supplementary Figure 6 in this submission closely resemble Figures 8, 9, and 10 in [1], respectively. I'm unsure whether this level of similarity complies with the conference policy and defer to the AC's judgment. I am unsure whether this situation is acceptable under the conference’s policy.\n\n[1] P. Caselles, E. Ramon, J. García, G. Triginer and F. Moreno-Noguer, \"Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstruction,\" in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 5, pp. 3691-3705, May 2025, doi: 10.1109/TPAMI.2025.3540542.", "was_revised": true, "final_justification": "I thank the authors for their detailed rebuttal. The response has resolved the vast majority of my concerns, and the explanation regarding the resemble figures is acceptable. Therefore, after considering the rebuttal and the opinions of the other reviewers, I am raising my score to 4: Borderline accept.", "rebuttal": "Thank you very much for your evaluation of our paper. We are glad you found it easy to follow.\n\n---\n\n**AC note:** We appreciate the reviewer bringing this to the AC’s attention. While the visual style and figure layout in Figures 4, 5, and Supplementary Figure 6 are indeed similar to those in [1], this was done intentionally to facilitate a clear side-by-side comparison with a recent state-of-the-art method. Our goal was to ensure a fair and consistent visual analysis for readers. No identifying information was included in the submission, and we have carefully followed the double-blind review policy throughout.\n\n---\n\n## Novelty\n\nWe thank the reviewer for their comments and would like to respectfully clarify that characterizing our method as merely adding a face-landmark branch on top of LVD is an oversimplification.\nWhile our work builds upon the iterative refinement principle of LVD, our formulation introduces several key innovations that fundamentally distinguish it from LVD, both conceptually and technically. We refer the reviewer to our responses to reviewers nu3K, 5vBD and xEDi for a detailed explanation, but we provide a brief summary here for completeness:\n\n1. We incorporate global reasoning using a relative spatial encoding scheme.  \n2. We propose a two-branch LVD architecture to learn vertex and keypoint representations jointly.  \n3. We present an extension of LVD to the multiview setting.  \n4. We introduce $f_k$ to integrate features from off-the-shelf models trained without requiring expensive 3D training data, improving flexibility and efficiency in the pipeline.  \n5. We introduce several training strategies that are essential for achieving high-quality reconstructions, as explained in Section 4.2 and in the supplementary material.\n\n---\n\n## Questions\n\n(1-2) In 3D face modeling, the goal can involve various aspects such as reconstructing identity shape, expression, pose, and/or texture. Approaches generally fall into two categories: those that build a face model, such as FLAME [2], and those that learn directly from data to make predictions without relying on an explicit model.\n\nParametric methods face two main challenges: (1) designing a sufficiently expressive prior that can capture the wide diversity of human faces, and (2) accurately estimating the parameters at inference time. While recent parametric models like FLAME, ImFace [3], or NPHM [4] have become increasingly powerful, improving parameter estimation remains an active area of research for achieving accurate 3D face reconstruction.\n\nOur method does not aim to directly address the representational limitations of 3DMMs. Instead, we focus on the applicability of face reconstruction from RGB inputs in downstream tasks. We propose a model that does not require building an explicit face model, instead directly predicting the 3D vertex positions of the face mesh. This approach preserves the mesh topology while avoiding the constraints when estimating the parameters of the traditional 3DMM, which suffers from mean shape bias.\n\nOur approach is limited by the expressiveness of the training data, which mainly consists of neutral-expression faces. While we do not perform explicit disentanglement, our method preserves mesh topology, allowing a parametric model to be fitted vertex-to-vertex. This enables pose control, animation, or expression editing. For the camera-ready version, we will include additional qualitative results with expressive examples.\n\n(3) The main motivation for including the 3D landmarks branch is to provide global guidance. Both qualitative and quantitative analyses show that this contributes to higher-quality reconstructions. Landmarks offer valuable cues for estimating well-aligned expressions. However, since our focus is on reconstructing subjects with near-neutral expressions, increasing the number of landmarks did not lead to significant improvements in accuracy. As suggested, we present results based on the 68 facial landmarks proposed in [8] that capture expression regions.\n\n| Method     | 3DFAW ↓ |  H3DS ↓ | \n|------------|----------|-----------|\n| GLVD  18  | **1.25**    | **1.36**    |\n| GLVD  68  | 1.24    | **1.36**    |\n\n(4) A key improvement beyond geometry is obtaining a reliable and realistic texture, as traditional 3DMMs often produce overly smooth and less detailed textures. In GLVD, successfully learning this mapping requires high-quality paired images and meshes to ensure pixel-level alignment. This can be approached in various ways; for instance, methods such as PIFu [5], DiffHuman[6], and DoubleField[7] attempt to regress detailed surface appearance. However, they often produce artifacts and blurry textures. In contrast, methods that predict directly in texture space typically achieve higher visual quality. Introducing a texture branch into LVD is not straightforward and presents significant technical challenges. Making it work effectively would likely require substantial architectural modifications and further investigation. Nonetheless, this line of research is promising and could lead to meaningful progress in achieving realistic 3D human reconstruction.\n\n(5) We agree with the suggestion and will update Figure 2 to label it clearly.\n\n(6) In **HRN**, the authors assume that low-frequency identity and mid-frequency details remain consistent across different views, while lighting, expression, and high-frequency (HF) details are view-dependent. The model explicitly handles pose, lighting, expression, and HF details in a view-dependent manner. The official implementation provides only per-view meshes, which makes it difficult to generate a complete global mesh suitable for evaluation. As a result, we cannot ensure a fair comparison with methods that produce unified, view-consistent reconstructions. Therefore, we use HRN only for single-view comparison.\n\nWe quantitatively and qualitatively evaluated **VHAP** on single-view data from the H3DS 2.0 and 3DFAW datasets. While qualitative results cannot be included in the rebuttal, we provide quantitative results demonstrating that GLVD achieves lower reconstruction error and better preserves identity while also being significantly faster:\n\nH3DS 2.0 Dataset:\n| Method     | 1 view ↓ |  Time ↓ | \n|------------|----------|-----------|\n| VHAP     |  2.15       |  ~200s     |\n| GLVD    | **1.36**    | <**1s**    |\n\n3DFAW Dataset:\n| Method     | 1 view ↓ | Time↓ | \n|------------|----------|-----------|\n| VHAP     | 2.05       | ~200s\n| GLVD    | **1.25**       | <**1s**      |\n\nWe will extend the evaluation to all multiview configurations and include qualitative comparisons in the camera-ready version.\n\n(7) While we cannot disclose qualitative images, the dataset is well-balanced in terms of gender and includes broad diversity in age and ethnicity across all continents. The raw scans are automatically processed to remove internal mesh faces and non-human elements such as background walls. Each scene is then aligned by registering a template 3D model using non-rigid Iterative Closest Point (ICP). The raw scans are not watertight and often contain large holes, with an overall quality slightly better than that of Microsoft Kinect. Each scan contains approximately 30,000 vertices and 60,000 faces, capturing the full head and shoulders region.\n\n---\n# References\n\n[1] P. Caselles, E. Ramon, J. García, G. Triginer and F. Moreno-Noguer, \"Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstruction,\" in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 5, pp. 3691-3705, May 2025, doi: 10.1109/TPAMI.2025.3540542\n\n[2] Li, Tianye, et al. \"Learning a model of facial shape and expression from 4D scans.\" ACM Trans. Graph. 36.6 (2017): 194-1.\n\n[3] Zheng, Mingwu, et al. \"Imface: A nonlinear 3d morphable face model with implicit neural representations.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.\n\n[4] Giebenhain, Simon, et al. \"Learning neural parametric head models.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\n[5] Saito, Shunsuke, et al. \"Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization.\" Proceedings of the IEEE/CVF international conference on computer vision. 2019.\n\n[6] Sengupta, Akash, et al. \"Diffhuman: Probabilistic photorealistic 3d reconstruction of humans.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n[7] Shao, Ruizhi, et al. \"Doublefield: Bridging the neural surface and radiance fields for high-fidelity human reconstruction and rendering.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n\n[8] Zhu, Xiangyu, et al. \"Face alignment across large poses: A 3d solution.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2016."}, {"review_id": "okkiXcl8jL", "reviewer": "Reviewer_xEDi", "summary": "This work presents GLVD, a method that builds on top of LVD, by running a sort of dual-scale LVD: First, it estimates a sparse set of keypoints with a LVD pipeline. Then, it uses these keypoints to compute a strong 3D positional encoding signal in canonical space. This signal is used to augment the features for the traditional vertex displacement branch, and improves the results. Whereas LVD was originally applied to a human full-body model and a hand model, GLVD is applied here to a face model, and performs strongly on single-view and multi-view face reconstruction benchmarks.", "questions": "I would like the authors to address the various points raised in the weaknesses section above, especially requests for clarification. Providing extra ablations would be nice but is not required.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths\n\n- The paper includes an ablation study, which adds credibility to the experimental analysis.  \n- It presents strong quantitative results across multiple benchmarks.  \n- The proposed approach is well motivated and conceptually sound.  \n- A wide range of qualitative results are provided, which help assess the method’s effectiveness.  \n\n---\n\n## Weaknesses and Points for Clarification\n\n### Methodological Clarity\n\n- The method description lacks clarity in several parts, making it difficult to follow without referring to the pipeline figure.\n  - For example, in lines 153–154, it is not clear if a separate decoder is used for each keypoint index via the function `g_k`. This should be stated explicitly if true.\n  - Similarly, the architecture of `g_v` is not described, making it hard to assess its complexity or design. The same concern applies as for `g_k`.\n  - The formulation of the loss in Equation 3 is unclear. It is not specified what the “points” refer to — do they refer to both vertices and keypoints? Also, the loss appears to be summed over all scenes. Does this imply full-batch training rather than stochastic gradient descent? While losses per index are given, the overall objective function is missing. Including it would clarify the learning dynamics, especially if some network components are specific to single keypoints/vertices while others are shared.\n  - It appears (after a full read) that the keypoints detected from off-the-shelf heatmaps do not need to correspond to the keypoints used in training. This non-obvious assumption should be stated clearly in the main text.\n  - Lines 162–165 introduce an encoding for `P_t`, but its role is never explained in the text.\n\n### Supplementary Material Usage\n\n- Several important details are relegated to the supplementary material without clear indication in the main paper. Either these should be moved to the main text, or the reader should be clearly directed to them.\n  - For instance, lines 32–33 of the supplementary state that `f_v` is pre-trained to predict a signed distance function (SDF). This seems like a critical part of training the model. Furthermore, `f_v` is described as a function of the image only — not of a 3D query point — so it's unclear how it is trained to predict an SDF.\n  - `g_k` is described as a simple MLP, yet it outputs a tensor of shape `18×18×3` (i.e. more than the actual used output). The rationale for this output shape is not explained.\n  - In the supplementary video, certain keypoints appear partway through the sequence. The cause of this behavior could be clarified.\n\n### Presentation Issues\n\n- In Table 1, all results for the proposed method are shown in bold, even though SIRA++ and H3D-Net outperform it in some metrics. This is misleading, as bold font conventionally highlights the best performance in each column. The formatting should be revised to follow this convention. Alternatively, if the intention is to emphasize performance only within a specific subset of methods, this should be explicitly stated, along with a clear justification for excluding SIRA++ and H3D-Net from the comparison.\n- In line 176, the phrase “for each mesh” suggests the method is applied to different mesh topologies. However, mesh topology is not discussed in the experiments section.\n- Also in line 176, the authors state that a random subset of vertices is selected as target keypoints. However, visualizations in the supplementary material suggest a non-random, possibly handpicked, selection. This inconsistency should be addressed.\n\n---\n\n## Minor Remarks\n\n- Terminology should be made consistent throughout the paper (e.g., \"landmarks\" vs. \"keypoints\").\n- In line 193, the citation “[52]” seems to refer to the wrong paper. Please verify and correct.\n- In lines 106–107, the distinction between “canonical body coordinates” and “face-centric coordinates” is unclear. For instance, PVA [44] appears to rely primarily on pixel-aligned features and positional encoding, not a face-centric coordinate system per se.\n- In line 100, citing LVD as a parametric method seems questionable — it may be a stretch.\n- The multiview inference aspect of the method deserves more detail and perhaps an ablation study. While not strictly required, this would be a valuable addition.\n- Finally, it is unclear why iterations over `g_k` and `g_v` are performed in parallel. An ablation comparing parallel vs. sequential updates (i.e., running all `g_k` steps first) would strengthen the experimental section.", "was_revised": true, "final_justification": "While I suspect that some of the authors' responses were intentionally vague in order to obscure a (very minor) misleading claim in the paper, they have ultimately addressed my main concerns. Given this, and in light of the general consensus among reviewers, I have decided to maintain my initial rating.", "rebuttal": "Thank you for your constructive and supportive evaluation.  We are pleased that you found the paper well-motivated and conceptually sound.\n\n# Points for Clarification\n\n## Methodological Clarity\n\n(1) The network $g_k(\\cdot)$ is a coordinate network, meaning the same shared decoder is applied to all input keypoints. Given an input of shape $18 \\times 3$, it outputs a tensor of shape $18 \\times 18 \\times 3$, from which we extract the diagonal to obtain one displacement per keypoint.\n\n(2) Both $g_v$ and $g_k$ share the same architecture. Each is implemented as an MLP with three linear layers of sizes 1024, 512, and 512, respectively, with ReLU activations and weight normalization applied to each layer. The input is a vector formed by concatenating the 3D coordinates of a point with its corresponding image features, which are obtained by projecting the point onto the image and extracting features via bilinear interpolation. For $g_v$, the 3D point is first transformed into the relative coordinate space $P$, using the estimated 3D keypoints, before being passed to the network. In both cases, the output represents the displacement vector from the input point to each ground-truth target. For keypoint estimation, we use 18 keypoints, and for surface reconstruction, we use 7225 mesh vertices.\n\n(3) We now describe how the optimization is performed during training. The same loss function is used for both branches: keypoints and vertices. In Equation 3 of the main paper, $N$ refers to the number of ground-truth vertices for each training sample, rather than the number of scenes in the dataset (we will update it in the camera ready).\n\nGiven a ground-truth mesh $\\hat{V} = [\\hat{\\mathbf{v}}_1, \\ldots, \\hat{\\mathbf{v}}_N]$ and its corresponding image $I$. We randomly sample $M$ 3D points $X = \\{\\mathbf{x}_1, \\ldots, \\mathbf{x}_M\\}$, and compute the loss defined in Equation 3 for each of the $M$ points.\n\n$ \\mathcal{L} = \\frac{1}{M} \\sum_{i=1}^{M} \\frac{1}{N} \\sum_{j=1}^{N}  \\left[ \\lambda\\_1 \\left(            1 - \\frac{ \\Delta \\mathbf{x}_i^j \\cdot \\hat{\\Delta \\mathbf{x}}_i^j }{ \\| \\Delta \\mathbf{x}_i^j \\|_2 \\, \\| \\hat{\\Delta \\mathbf{x}}_i^j \\|_2 }        \\right)   +   \\lambda_2 \\left| \\| \\Delta \\mathbf{x}_i^j \\|_2 - \\| \\hat{\\Delta \\mathbf{x}}_i^j \\|_2 \\right|   \\right]$\n\nWe train the network to predict the displacement from any point in space to all vertices of the mesh. Therefore, the decoder outputs a tensor of size 18×3 for the landmarks and 7223×3 for the full set of vertices. During training, the loss is averaged across all samples in the batch. We will update and expand the loss description in the camera-ready version.\n\n(4) We will incorporate it to the main text.\n\n(5) While we refer to Figure 2 to illustrate the use of the transformed vertices $P$, we will make this point clearer in Sections 3.4 and 3.5. The transformed vertices are used exclusively in the vertex branch.\n\n---\n\n## Supplementary Material Usage\n\n(1) We pre-train our feature encoder in the task of 3D reconstruction. We represent the surface $\\mathcal{S}$ as the zero-level set of a signed distance function $f^{\\text{sdf}} : (\\mathbf{x}, I) \\rightarrow s$, such that $\\mathcal{S} = \\\\{ \\mathbf{x} \\in \\mathbb{R}^3 \\mid f^{\\text{sdf}}(\\mathbf{x}, I) = 0 \\\\}$. Our goal is to estimate $f^{\\text{sdf}}$ through a composition of a feature encoder and a decoder networks. The resulting feature encoder is then used within GLVD. To train on the SDF task, we use non-watertight scans from the same training dataset and minimize $\\mathcal{L}\\_{\\text{Surf}}^{(i)}$ on surface points $N_s$ and $\\mathcal{L}\\_{\\text{Eik}}^{(i)}$ throughout the volume $N_v$:\n\n\n\n$ \\mathcal{L}\\_{\\text{Surf}} = \\frac{1}{N_s} \\sum_{i=0}^{N_s} \\left| f\\^{\\text{sdf}}(\\mathbf{x}_i,I) \\right| $\n\n$ \\mathcal{L}\\_{\\text{Eik}} = \\frac{1}{N_v} \\sum_{i=0}^{N_v} \\left( \\left\\| \\nabla\\_{\\mathbf{x}} f^{\\text{sdf}}(\\mathbf{x}_i,I) \\right\\|\\_2 - 1 \\right)^2 $\n\nThe loss is averaged over the batch. We will include a detailed explanation in the camera-ready version. The table below (in Minor Remarks) compares the impact of pretraining. Specifically, we evaluate LVD with and without SDF-based pretraining of the feature encoder. Results show that this pretraining is crucial for achieving strong performance in both LVD and GLVD.\n\n(2) For each query point, we predict displacement vectors to all ground-truth vertices for the given scene. At test time, we use the topological index of the query point to update only the corresponding vertex with its predicted displacement. The function $g_k$ operates on 18 3D keypoints. For each query keypoint, it predicts displacement vectors to all 18 target keypoints, resulting in a total of $18 \\times 3$ directional components covering the $x$, $y$, and $z$ axes. By repeating this for all 18 keypoints, the final output has a shape of $18 \\times 18 \\times 3$. However, to obtain the estimated keypoints, we extract the diagonal to obtain one displacement per keypoint.\n\n(3) The initial positions of the 18 query 3D keypoints are randomly sampled within a bounded space. At each iteration, they start from different locations, and through repeated updates, they gradually converge to the final target positions.\n\n---\n\n## Presentation Issues\n\n(1) Our intention is not to directly compare GLVD with optimization-based methods, as their higher computational cost makes the comparison unequal. These methods are included for reference. We agree that bold formatting may be misleading and will revise it in the camera-ready version to follow standard conventions, with a clear caption explaining the grouping.\n\n(2) Our method is applied to a fixed mesh topology throughout the experiments. The phrase “for each mesh” was intended to refer to each training sample using the same template topology. We will revise the wording to clarify this and ensure consistency in the experiments section.\n \n(3) We acknowledge the ambiguity in our description and appreciate the reviewer’s observation. The supplementary visualizations correspond to a fixed subset of template vertices.\n\n---\n\n# Minor Remarks\n\n(1-4) Thank you for the detailed comments. We will address the inconsistencies in the camera-ready version.\n\n(6) Thank you for the suggestion. We have conducted an ablation comparing parallel and sequential updates of $g_k$ and $g_v$ in the H3DS 2.0 and 3DFAW datasets. The results indicate that the iterative (parallel) approach yields slightly better performance. We will include this clarification and the corresponding results in the camera-ready version.\n\n\nH3DS 2.0 Dataset:\n| Method     | 1 view ↓ | 3 views ↓ | 4 views ↓ | 6 views ↓ |\n|------------|----------|-----------|-----------|---------------|\n| LVD     | 1.55      | 1.49      | 1.44        | 1.42    |\n| LVD pre-trained SDF  | 1.39       |  1.45      |  1.39        | 1.37        |\n| GLVD Sequential     | 1.38       | 1.36      | **1.34**         | 1.35         |\n| GLVD Iterative    | **1.36**       | **1.33**      | **1.34**         | **1.34**       |\n\n3DFAW Dataset:\n| Method     | 1 view ↓ | 3 views ↓ | \n|------------|----------|-----------|\n| LVD    | 1.65       | 1.52      |\n| LVD pre-trained SDF    |  1.58      | 1.26     |\n| GLVD Sequential     | 1.29       | 1.23      |\n| GLVD Iterative    | **1.25**       | **1.22**      |"}]

Author rebuttal:

Meta-review: This paper introduces GLVD, a hybrid method for 3D face reconstruction from few-shot images. It extends Learned Vertex Descent (LVD) by integrating per-vertex neural field optimization with global structural guidance from dynamically predicted 3D keypoints.

Initially, the paper received mixed ratings. The primary concerns raised by reviewers included: limited technical novelty beyond LVD, insufficient quality of 3D representation capability, and a lack of clear method descriptions and implementation details. During the rebuttal and discussion period, the authors’ response successfully addressed most of the reviewers’ concerns, and the paper ultimately received 3 "Borderline Accept" ratings and 1 "Accept" rating.

After considering the paper, the reviews, and the rebuttal, the Area Chair believes this work meets the standards of NeurIPS. Accordingly, the Area Chair recommends acceptance. The authors are requested to revise the paper by addressing the reviewers’ comments in preparation for the camera-ready version.
Decision: Accept (poster)