Title: GLVD: Guided Learned Vertex Descent
Venue: NeurIPS 2025
Accepted: yes
Keywords: Computer Vision

3D Face Reconstruction

Neural Fields

Few-Shot 3D Reconstruction
Abstract: Existing 3D face modeling methods usually depend on 3D Morphable Models, which inherently constrain the representation capacity to fixed shape priors. Optimization-based approaches offer high-quality reconstructions but tend to be computationally expensive. In this work, we introduce GLVD, a hybrid method for 3D face reconstruction from few-shot images that extends Learned Vertex Descent (LVD) by integrating per-vertex neural field optimization with global structural guidance from dynamically predicted 3D keypoints. By incorporating relative spatial encoding, GLVD iteratively refines mesh vertices without requiring dense 3D supervision. This enables expressive and adaptable geometry reconstruction while maintaining computational efficiency. GLVD achieves state-of-the-art performance in single-view settings and remains highly competitive in multi-view scenarios, all while substantially reducing inference time.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work presents GLVD, a method that builds on top of LVD, by running a sort of dual-scale LVD: First, it estimates a sparse set of keypoints with a LVD pipeline. Then, it uses these keypoints to compute a strong 3D positional encoding signal in canonical space. This signal is used to augment the features for the traditional vertex displacement branch, and improves the results. Whereas LVD was originally applied to a human full-body model and a hand model, GLVD is applied here to a face model, and performs strongly on single-view and multi-view face reconstruction benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n- The paper includes an ablation study, which adds credibility to the experimental analysis.  \n- It presents strong quantitative results across multiple benchmarks.  \n- The proposed approach is well motivated and conceptually sound.  \n- A wide range of qualitative results are provided, which help assess the method’s effectiveness.  \n\n---\n\n## Weaknesses and Points for Clarification\n\n### Methodological Clarity\n\n- The method description lacks clarity in several parts, making it difficult to follow without referring to the pipeline figure.\n  - For example, in lines 153–154, it is not clear if a separate decoder is used for each keypoint index via the function `g_k`. This should be stated explicitly if true.\n  - Similarly, the architecture of `g_v` is not described, making it hard to assess its complexity or design. The same concern applies as for `g_k`.\n  - The formulation of the loss in Equation 3 is unclear. It is not specified what the “points” refer to — do they refer to both vertices and keypoints? Also, the loss appears to be summed over all scenes. Does this imply full-batch training rather than stochastic gradient descent? While losses per index are given, the overall objective function is missing. Including it would clarify the learning dynamics, especially if some network components are specific to single keypoints/vertices while others are shared.\n  - It appears (after a full read) that the keypoints detected from off-the-shelf heatmaps do not need to correspond to the keypoints used in training. This non-obvious assumption should be stated clearly in the main text.\n  - Lines 162–165 introduce an encoding for `P_t`, but its role is never explained in the text.\n\n### Supplementary Material Usage\n\n- Several important details are relegated to the supplementary material without clear indication in the main paper. Either these should be moved to the main text, or the reader should be clearly directed to them.\n  - For instance, lines 32–33 of the supplementary state that `f_v` is pre-trained to predict a signed distance function (SDF). This seems like a critical part of training the model. Furthermore, `f_v` is described as a function of the image only — not of a 3D query point — so it's unclear how it is trained to predict an SDF.\n  - `g_k` is described as a simple MLP, yet it outputs a tensor of shape `18×18×3` (i.e. more than the actual used output). The rationale for this output shape is not explained.\n  - In the supplementary video, certain keypoints appear partway through the sequence. The cause of this behavior could be clarified.\n\n### Presentation Issues\n\n- In Table 1, all results for the proposed method are shown in bold, even though SIRA++ and H3D-Net outperform it in some metrics. This is misleading, as bold font conventionally highlights the best performance in each column. The formatting should be revised to follow this convention. Alternatively, if the intention is to emphasize performance only within a specific subset of methods, this should be explicitly stated, along with a clear justification for excluding SIRA++ and H3D-Net from the comparison.\n- In line 176, the phrase “for each mesh” suggests the method is applied to different mesh topologies. However, mesh topology is not discussed in the experiments section.\n- Also in line 176, the authors state that a random subset of vertices is selected as target keypoints. However, visualizations in the supplementary material suggest a non-random, possibly handpicked, selection. This inconsistency should be addressed.\n\n---\n\n## Minor Remarks\n\n- Terminology should be made consistent throughout the paper (e.g., \"landmarks\" vs. \"keypoints\").\n- In line 193, the citation “[52]” seems to refer to the wrong paper. Please verify and correct.\n- In lines 106–107, the distinction between “canonical body coordinates” and “face-centric coordinates” is unclear. For instance, PVA [44] appears to rely primarily on pixel-aligned features and positional encoding, not a face-centric coordinate system per se.\n- In line 100, citing LVD as a parametric method seems questionable — it may be a stretch.\n- The multiview inference aspect of the method deserves more detail and perhaps an ablation study. While not strictly required, this would be a valuable addition.\n- Finally, it is unclear why iterations over `g_k` and `g_v` are performed in parallel. An ablation comparing parallel vs. sequential updates (i.e., running all `g_k` steps first) would strengthen the experimental section.\n\nQUESTIONS:\nI would like the authors to address the various points raised in the weaknesses section above, especially requests for clarification. Providing extra ablations would be nice but is not required.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nBreaking free from the representational constraints of traditional 3D Morphable Models (3DMMs) is a promising direction in 3D face reconstruction. This paper takes a step in that direction by extending the existing Learned Vertex Descent (LVD) framework—originally designed for human body reconstruction—into the facial domain. The contribution lies in integrating per-vertex neural field optimization with global structural guidance derived from dynamically predicted 3D landmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Attempting to overcome the representation space limitations of 3DMMs is a promising direction.\n\n2. The paper is easy to follow.\n\nWeaknesses:\n1. From the perspective of novelty, the innovation level of this paper seems to fall below the expectations of NeurIPS. Its main framework is based on the Learned Vertex Descent (LVD) framework, and the improvement lies in introducing a face-landmark-based branch similar to LVD. The overall architecture is similar to a dual-path LVD.\n\n2. From the reconstruction results presented in the paper, although it can reconstruct faces of different body types with near-neutral expressions, it seems difficult to reconstruct expression changes such as closed eyes or open mouths (e.g., in the top right corner of Figure 7, the reconstruction fails to capture the closed-eye expression of the male subject).\n\n3. The motivation of the paper is to break through the limitations of 3DMMs, but in terms of shape representation, the reconstructed results appear overly smooth (seemingly unable to reconstruct fine details such as wrinkles or dimples), making it hard to demonstrate superiority over 3DMMs. Additionally, another limitation of 3DMMs is the restricted expression space, and the paper does not show how GLVD addresses expression reconstruction.\n\n4. Please also refer to the “Questions” section.\n\n**AC note:**\nI would like to bring to the AC’s attention that Figures 4, 5, and Supplementary Figure 6 in this submission closely resemble Figures 8, 9, and 10 in [1], respectively. I'm unsure whether this level of similarity complies with the conference policy and defer to the AC's judgment. I am unsure whether this situation is acceptable under the conference’s policy.\n\n[1] P. Caselles, E. Ramon, J. García, G. Triginer and F. Moreno-Noguer, \"Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstruction,\" in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 5, pp. 3691-3705, May 2025, doi: 10.1109/TPAMI.2025.3540542.\n\nQUESTIONS:\n1. I would like to see more results using GLVD on face images with diverse expressions, as current examples mainly focus on neutral or slightly varied identities.\n\n2. Given the motivation of the paper—overcoming the representation limitations of 3DMMs—this could manifest in two aspects: (1) modeling fine-scale geometry such as wrinkles and dimples, and (2) reconstructing large or micro expression changes. I would be interested in results, theoretical analysis, or discussion from the authors regarding GLVD’s capability in either direction.\n\n3. After reviewing both the main text and supplementary material, it seems that the keypoint selection in the 3D Keypoint Branch does not account for expressive regions, e.g., upper/lower eyelids or mouth lips. Would this limit GLVD’s ability to reconstruct facial expressions accurately?\n\n4. A question for discussion: based on the current dual-branch LVD structure in Figure 2, could the authors consider adding a third “texture LVD” branch to enhance texture reconstruction? This could further address the limited texture representation space of 3DMMs.\n\n5. Figure 2 would benefit from clear labeling of the “3D Keypoint Branch” and “3D Vertex Branch” to improve readability and understanding of the architecture.\n\n6. In Table 1, the method HRN seems to support multi-view reconstruction, but this is not discussed in the paper. Moreover, recent optimization-based methods such as VHAP [2] have also shown the potential to go beyond morphable model priors. \n\n7. In Lines 199–201, the authors mention employing a proprietary dataset of 10,000 3D head scans. This is a large-scale dataset by the standards of the 3D face community. I am curious about more details of the data, such as mesh resolution (vertex count or face density), scanning quality, and variation.\n\n[2] VHAP: Versatile Head Alignment with Adaptive Appearance Priors\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of reconstructing 3D human faces from images. Existing methods mainly fall into two categories: learning-based approaches, which are efficient but often less accurate, and optimization-based approaches, which offer high fidelity but are computationally intensive. The authors propose a hybrid method called Guided Learned Vertex Descent (GLVD), which builds on the Learned Vertex Descent (LVD) framework. GLVD predicts deformations of mesh vertices with a fixed topology of given time step, aiming to combine the accuracy of optimization-based methods with the efficiency of learning-based models. To adapt LVD specifically for face reconstruction, the authors introduce 3D face landmark-based guidance, enhancing the structural coherence of the reconstruction. As a result, their approach achieves state-of-the-art face reconstruction performance while reducing inference time.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- The paper is clearly written, easy to follow, and well-structured.\n- The proposed approach is simple yet effective, and its effectiveness is demonstrated through comprehensive experiments and detailed ablation studies.\n- The authors provide thorough comparisons and clearly show the advantages of their method in both accuracy and efficiency.\n\n### Weaknesses\n\n- The technical novelty beyond LVD feels somewhat limited.\n- While the method achieves significant improvements, it does not surpass optimization-based methods in multi-view settings.\n\nQUESTIONS:\n1. The authors demonstrate the effectiveness of the proposed method, but the technical contribution may appear as a straightforward extension of LVD to face reconstruction. Could you clarify in detail what distinguishes GLVD from simply applying LVD to the face reconstruction task? Specifically, what are the key improvements or advantages?\n2. Could the authors provide more analysis or examples of failure cases? In particular, under what conditions does GLVD struggle compared to optimization-based methods? Additionally, could you discuss how to overcome the performance gap in multi-view or less ill-posed scenarios where optimization-based methods perform better?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes GLVD which extends LVD by combining relative position encoding to guide its dense reconstruction with sparse landmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n- The paper shows an interesting application of non-parametric shape learning models to facial shape reconstruction, where parametric models often face difficulties in reconstructing individual shapes. \n- It outperforms previous approaches in this field and by large margin the original LVD.\n- Reconstructed shape quality looks high\n- The method is able to utilize image features effectively in a coarse-to-fine manner.\n\nWeakness:\n\nA. Its novelty seems limited as it looks like a combination of LVD and KeypointNerf. \n\nB. The use of landmarks, coarse mesh and bones has already been explored in various domains. And there is auto encoder like approaches for learning low-dimensional features. And non-parametric models and multi-scale features are used in HMR. It would be appreciated to position the proposed coarse to fine approach in the literature. The related work section is rather too much focused toward NeRF and GS.\n\n1) Dimensionality reduction:\n - Sparse multidimensional scaling using landmark points\n\n2) HMR\n- End-to-End Human Pose and Mesh Reconstruction with Transformers\n- Deformable Mesh Transformer for 3D Human Mesh Recovery\n- 3D Human Mesh Estimation from Virtual Markers\n\n- One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer\n- PyMaf-X\n\n3)  Bones and latent features:\n- GART: Gaussian Articulated Template Models\n- Generating 3D faces using Convolutional Mesh Autoencoders\n\nC. Section 3.4 is unclear to me i.e. how the displacement at vertex is obtained from P.\n\nQUESTIONS:\nPlease see Weakness\n\n- How much is the feature maps resolution? Are they same for landmarks reconstruction and dense vertex reconstruction? If so, are there cases that the image feature maps are too high-res for landmarks reconstruction e.g. prone to noise when reprojected on far away locations than the true landmarks or  too low-res for dense vertex reconstruction.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}