Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstruction
Authors: Simon Giebenhain
Tobias Kirschstein
Martin Rünz
Lourdes Agapito
Matthias Nießner
Venue: ICLR 2026 Poster (2026)
Keywords: Single Image Face Reconstruction
Face Tracking
Foundation Model Finetuning
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: Fine-tuning image foundation models to predict valuable geometric cues for face tracing.

Abstract: We address the 3D reconstruction of human faces from a single RGB image. To this end, we propose Pixel3DMM, a set of highly-generalized vision transformers which predict per-pixel geometric cues in order to constrain the optimization of a 3D morphable face model (3DMM). We exploit the latent features of the DINO foundation model, and introduce a tailored surface normal and uv-coordinate prediction head. We train our model by registering three high-quality 3D face datasets against the FLAME mesh topology, which results in a total of over 1,000 identities and 976K images. For 3D face reconstruction, we propose a FLAME fitting opitmization that solves for the 3DMM parameters from the uv-coordinate and normal estimates. To evaluate our method, we introduce a new benchmark for single-image face reconstruction, which features high diversity facial expressions, viewing angles, and ethnicities. Crucially, our benchmark is the first to evaluate both posed and neutral facial geometry. Ultimately, our method outperforms the state-of-the-art (SoTA) by over 15\% in terms of geometric accuracy for posed facial expressions.

Published as a conference paper at ICLR 2026
PIXEL3DMM: VERSATILE SCREEN-SPACE PRIORS FOR SINGLE-IMAGE 3D FACE RECONSTRUCTION
Simon Giebenhain'
Tobias Kirschstein'
Martin Rünz
Lourdes Agapito Matthias Nießner
Techhical University of Munich
Syntheesia
Univeesity College London
Figure 1: We present Pixel3DMM, a set of two ViTs Dosovitskiy et al. (2020), which are tailored to predict per-pixel surface normals and uv-coordinates. Here, we demonstrate the fidelity and robustness of Pixel3DMM on challenging inputs. From top to bottom we show input RGB, predicted normals, 2D vertices extracted from the uv-coordinate prediction, and FLAME fitting results.
ABSTRACT
We address the 3D reconstruction of human faces from single RGB im- age. To this end, we propose PixeI3DM,, set of iiggll-generllized vision transformers which predict per-pixel geometric cues in order constrain the optimization of 3D morphable face model (3DMM). We exploit the latent features of the DINO foundation model, and introduce tailored surface normal and uv-coordinate prediction head. We train our model by registering three high-quality 3D face datasets against the FLAME mesh topology, which results in total of over 1,000 identities and 976K images. For 3D face reconstruction, we propose a FLAME fitting opitmization that solves for the 3DMM parameters from the uv-coordinate and normal es- timates. To evaluate our method, we introduce a new benchmark for single-image face reconstruction, which features high diversity facial pressions, viewing angles, and ethnicities. Crucially, our benchmark is the first to evaluate both posed and neutral facial geometry. Ultimately, our method outperforms the state-of-the-art (SoTA) by over 15% in terms of geometric accuracy for posed facial expressions.
1 INTRODUCTION
3D reconstruction of faces, tracking facial movements, and ultimately extracting expressions for animation tasks are fundamental problems many domains such as computer games, movie production, telecommunication, and AR/VR applications. Recovering 3D head geometry from a single image is a particularly important task due to the vast amount of available image collections.
Unfortunately, reconstructing faces from single input image also inherently under- constrained. Not only depth ambiguity renders this task challenging, but also ambiguities between albedo and lighting/shade sshaoow effects. In addition, properly disentangling identity and expression information which is critical for many downstream applications makes the problem difficult. Finally, occlusions and unobserved facial regions further complicate the problem in real application scenarios, thus highlighting the need for strong data priors.
A typical approach to single-image face reconstruction is to exploit 3D parametric head models (3DMMs) Blanz& & Vetter (2023); Li et al. (017) which provide low-dimensional

---

Published as a conference paper at ICLR 2026
parametric representation for the underlying 3D geometry. Optimizing within 3DMM's disentangled parameter space heavily constrains the searchspace with built-in assumptions about plausible facial structure and expressions, and allows to extract disentangled identity and expression information. Nonetheless, despite relying on 3DMMs, many ambiguities remain and their simplifying assumptions about our world often cannot explain the com-
plexity of the real world. This necessitates additional priors in order to obtain compelling
fitting results such as sparse Sagonas et al. (2013) and dense Cao al. (2013). Wood et al. (2022) facial landmarks, or UV coordinate predictions Taubner et al. (2024a).
In recent years, we have also seen significant progress in feed-forward 3DMM regres- sors Sanyal et al. (2019); Feng et al.
; Daněček et al.] ; Retsinas et al. (0024);
extend feed- forward regressors, e.g. to a multi-view or temporal domain, and, as we will show later, they fall behind optimization-
Zielonka et al. (2022); Zhang et al. (002)) However, is complicated
approaches on inputs with strong facial expressions.
Overall, accurate 3D face reconstruction from single images remains a challenging and highly relevant problem.
Therefore, we propose PixeI3DMM, a novel optimization-based 3D face reconstruction approach. Our murmain idea is to exploit further develop broadly generalized and powerful foundation models to predict piiel--iiiiiee pixel-aligned geometric cues that effectively constrain the 3D state of an observed face. Given a single image at test time, we propose normal and uv- coordinate predictions as optimization constraints from which we fit 3D FLAME model. Instead of a simple rendering loss of uv-coordinates, we then transfer the information into a 2D vertex loss, which offers wider basin of attraction during optimization. We argue that this strategy is superior to traditional photometric terms, or sparse landmarks, which often struggle with extreme view points and facial expressions. In order to train our approach, we unify three recent, high-fidelity 3D face datasets datasets(Giebenhaai et al. (2023); ; Zhu et al. (2023); Martinez et al. (2024) by registering them against the FLAME Li et al.) (2017) model.Our approach outperlorms all available normal estimators for human faces In the NeRSemble Kirschstein et al. (2023) dataset.
In order to advance the evaluation of single-image 3D face reconstruction methods, we further propose a new benchmark based on the multi-view video dataset NeRSem- ble Kirschshttti et al. (2023), which includes a wider variety of facial expressions than exist- ing benchmarks (anyal et al. (2019); Zhu et al. (2023); Feng et al. (2018); Chai et al. 2022
Our benchmark is the first to allow for the simultaneous evaluation of posed and neutral facial geometry. This enables a more direct comparison of methods, especially regarding fitting fidelity and ability to disentangle expression and identity information. Finally, we show that compared to our strongest baselines, our approach improves the L2-Chamfer reconstructions loss by over 15% for posed geometry, while slightly improving over neutral geometry predictions.
To summarize, our main contributions are as follows:
A new formulation to exploit foundation model features for 3D-related, pixel- aligned predictions, facilitating SoTA normal estimations for human faces.
A novel 3D face reconstruction approach based on predicted uv-map correspon- dences and surface normals.
A 3D face reconstruction benchmark and evaluation protocol from high-fidelity multi-view face captures.
We plan to make the model, code, and our new benchmark publicly available to promote progress in single image 3D face reconstruction and encourage quantitative benchmarking on challenging facial expressions.
2 RELATED WORK
Single-Image 3DMM Fitting Tracking morphable models from single images is well- studied problem in the context of 3D face reconstruction and tracking. Early works Blanz & Vetter (1999 Paysan et al. ( et al. (2017), introduced statistical shape and texture priors to estimate 3D face geometry from 2D images. Such methods rely on photometric fitting and subsequent approaches improve modeling capabilities using learned implicit
2

---

Published as a conference paper at ICLR 2026
Imput
Est. Normals
Est. UV-coords
Impott
DINOv2 Backbone
12
Transformer Blocks
322
Up-Couv
N.N.
Linear
512
Supervision
Projecion
Reconstruction
FLAME Rac. R,
Figure 2: Method Overview: Pixel3DMM consists of (a) learning pixel-aligned geometric priors (left) and (b) test-time optimization against predicted uv-coordinates and normals (right). On the left we show our network architecture and training examples. On the right we depict the process of finding per-vertex 2D locations using nearest neighbor (N.N.) look up, and our loss terms.
representations Lin et al. (2023); Giebenhain et al. 2024). While some methods Thies et al.
Grishchekko et (2020) tavor a high tracking frame rate for real-time applications, others tavor reconstruction accuracy Zielonka et al. 2022).
Facial Landmark Prediction Numerous reconstruction methods Li et al. 2017 Cao et al.
2013) for faces rely on accurate landmark predictions, which are usually coupled with vertices of template mesh. Pioneering work on detecting such landmarks already relies statistical learning Cootes et al. (2001) and more recent models exploit large datasets Wood
et al. (2021); Wu et al. (2018) and neural networks to improve the performance Bulat
& Tzimiropoulos (2017) Bazarevsky et al. (2019). Medial'ipe Bazarevsky et al. (2019)
instance, uses convolutional network inspired by MobileNet Howard et al. (2017).
Another line of work focuses on densely aligning template mesh and 2D prediction.. To
achieve this FlowFace Taubner et al. (2024a) employs vision-transformer backbone and iteratively refines the flow from UV to image space.
3DMM Regression DECA Feng et al. (2021) trains regressor to predict 3DMM parameters from an image An extension of this work is presented in EMOCA DaněCek
et al. (2022), which emphasizes the reconstruction of emotion-rich expressions. Similarly, SPECTRE/Finttisis et al. (2022) aims at temporal consistency and reconstructing lip motion.
SMIRK Retsinas et al. (2024) introduces a neural synthesis component, reducing the
domain gap between real and rendered images. Since the aforementioned methods don't t assume 3D training data, it is easy to scale them to large datasets. As a downside, the lack of 3D information impedes accuracy and leaves depth ambiguity. In order to address this, MICA Zielonka et al. (2022) supervises directly 3D space. TokenFace Zhang al. (2023) transtormer-based method that can be trained on both 2D and 3D data.
Face Reconstruction Benchmarks The Stirling Feng al. 2018 dataset contains 2000 images of 135 subjects. Unfortunately, ground truth reconstructions are only available for neutral poses in this dataset. Similarly, the NoW (Snyal et al. (2019) benchmark provides reconstructions only in the neutral expression. It has 2054 images of 100 subjects. Both the FaceScape Zhu et al. (2023) and the REALY [Cai et al. (2022) dataset contain posed scans. REALY(Chai While the tormer has 10 identities, the latter has 100 subjects. Neither of these two bench- marks measures disentanglement by additionally evaluating against neutral geometry.
3 PiXEL3DMM
In this work we address the challenges of single-image face reconstruction by learning powerful priors pixel-aligned geometric cues. In particular we train two vision trans- former networks, which predict uv-coordinates and surface normals against which
3

---

Published as a conference paper at ICLR 2026
fit FLAME Li al. 2017 parameters at inference time. In section 3.1 describe our PixeI3DMM networks, our data acquisition, and how we train them for accurate surface normal and uvvcoordinate prediction. Afterwards, in s.ctionn3.2 we elaboraate on our single- image fitting approach, which purely based on our surface normal and uv-coordinate predictions.
3.1 LEARNING PIXEL-ALIGNED GEOMETRIC CUES
Despite recently released high-quality 3D face datasets Zhu et al. (2023); Kirschstein et al. (2023;; Giebenhain et al. (2023); ; Martinez et al. (2024), such data is still relatively scarce, especially w.r.f. the number of different identities, ethnicities, age distribution and lighting variation. We therefore take inspiration from recent achievements on fine-tuning founda-
tional and large generative models to become experts on constrained domain, egg.
Hu
al. 2022 Ruiz et al. (002)).
In particular we train two expert networks
N 512x51223 + |-1, 1, 512x512x3 and u 51126112x3 + [0, 1011)62xx1122
(1)
which, given a single input image I, predict surface surface normals // (()) and uv-space coordinates U(T), respectiell.
3.1.1 NETWORK ARCHITECTURE
We build Pixel3DMM on top of the foundational features from a pre-trained DINOv2 et al. (2023) backbone. As depicted in fig. 2 we extend the VIT architecture using simple prediction head. It consists of four additional transformer blocks, three up-convolutions which lift the feature map resolution from 32 to 256 x 256. Finally, use a single linear layer to increase the feature dimensionality and unpatchify the predictions to 512 x 512 x e, where e {3, 2) for normals and uv-coordinate prediction tasks, respectively.
3.1.2 DATA PREPARATION
To train our networks, we opt three recent, high-quality 3D face datasets: NPHM UPHMGiieenn hain et al. (2023), FaceScape Zhu et al. (2023), and Ava256 Martinez et al. (2024). We follow the non-rigid registration procedure from NPHM, register all datasets into a uniform for- mat and topology. fig, shows pairs of input views with the associated supervision signal for surface normals and uv-coordinates.
Dataset Numbers In total, our dataset comprises 470 identities from NPHM in 23 expres- sion and 40 renderings each (376K rgb, normal and uv images in total). For FaceScape we use 350 subjects, observed under 20 different expressions and 50 cameras each (350K rgb, normal and uv images total). Since Ava256 video dataset, we leverage furthest point sampling to select the 50 most diverse expressions per person. For each person we choose a random subset of 20 cameras (250K rgb and uv images total).
Diffsion-based Lighting Variations Since FaceScapeand and Ava256 are both studio datasets, which are captured at rather homogeneous lighting conditions, leverage IC-Light (Zang et al. (2025), an image conditioned diffusion model Rombach et al. (2022), which alters the lighting condition based text prompt or background image.
3.1.3 TRAINING
We train our models M e (N,4) using straight forward image translation formulation
argmin
Σ
Σ
llf(Th))
- lz,
MM
RED pEAs
where Vr denotes the network's parameters, E D is a sample from our dataset, and Y are input rgb and target images, respectively, and p E M are all pixels in the associated foreground mask.

---

Published as a conference paper at ICLR 2026
Note, that instead of freezing the parameters of our DINOv2 backbone altogether, set their learning rate ten times lower, in order to encourage prior preservation but enable stronger domain adoption.
Compared to Sapiens Khirodkar et al. 2024), a recent SoTA foundation model for human bodies and faces, training our models is cheap and can be realized using 2 GPUs and training for 3 days. Additionally, all our data is publicly available. The relatively low computational burden and data accessibility, will hopefully inspire more research to follow in a similar direction.
3.2 SINGLE-IMAGE FLAME FITTING
Given a single image I, we leverage our prior networks to obtain predicted surface normals ((I) and uv-coordinates U(T). Using these predictions aim to recover 3DMM param- eters. In particular, we optimize for FLAME Li et al. (2017) identity, expression, and jaw parameters, well as, camera rotation, translation, focal length and principal point:
OFLAME
{Za E 32000 Zex R200 0 E SO(3))
(3)
Qamm (RESOO3),ER AIR RR)
(4)
3.2.1
2D VERTEX Loss
Using the estimated uv-coordinates U(I), aim to extract the 2d location P. for each visible vertex E V of the FLAME mesh. To this end we first run a facial segmentation network Zheng et al. (2022), in order to mask out the background, eyeballs and mouth interior. Then we find correspondences for each vertex v E V using a nearest neighbor lookup into U(!). To be more specific let Tuv [0, 1]2 denote the uv-coordinate of the template mesh T". Then we find the pixel location
pe
argmin
TH (T)
(5)
PEP
as the pixel with the closest uv prediction. Finally, we define
Luv Σ - |pt =())
(6)
to be our 2d vertex loss, where denotes the indicator function which masks out vertices with a nearest neightbor distance larger than duv- V FLAME (SPLAME) is the current estimate of the FLAME parametric model, and x denotes the projection implied by the current estimate of the camera parameters Ωeam.
3.2.2 OPTIMIZATION
Next to the 2d vertex loss Lue, incluuude the normal loss L, = render, , denotes rendering of surface normals of the FLAME mesh. The regularization term Zia
(V)), where
zia
ZMICA Aex mex completes our overall energy term
E
AupLuo X, Cn + R.
Here
MICA denotes MICA's Zielonka et al. (0022) identity prediction.
3.3 MONOCULAR VIDEO TRACKING
Next the single-image scenario, tracking faces in monocular videos a fundamental task in computer vision. To address this problem, we simply extend our optimization strategy from section 3.2.2 globally over all images in video sequence {T:)T=-- Using our prior networks, we first obtain normal predictions (W(())) and uv-predictions (U(1+)) After
obtaining initial estimate for (1 FLLAMEE and (20) on the first frame by optimizing for eq. 7 we freeze Zid, and pp. We then sequentially optimize for all remaining attributes in (0
FLAME and (1) Using the results from the sequential optimization pass initialization,

---

Published as a conference paper at ICLR 2026
extend eq. 7 to batched version using randomly sampled frames. Note, that the parameters Zid, fl and pp are shared for all frames. In order to enforce smoothness across all per-frame optimization targets we add a smoothness term
c
smooth
smooth
Σ
[
p 12+1
(++1)
(8)
2 *
LEB
to the energy E, where (()) { {2rex 010) R ) denotes any of the per-frame variables.
4 3D FACE RECONSTRUCTION BENCHMARK
Constructing a benchmark that covers the variety of facial geometry and its complex de- formations is a challenging endeavor. In table we present comparison of popular single-image face reconstruction benchmarks.
Table 1: Comparison of 3D Face Reconstruction Benchmarks. We compare data capture year, whether the benchmark evaluates posed and/or or neutral geometry, expression diver- sity, viewpoint diversity, number persons (#pers.) and number of GT scans.
expression diversity
viewpoint diversity
Year posed neutral
#pers.
#Scans
Stirling Feng et al.
2018
2013
133
133
REALY Chat et al.
2022
2015
v
100
100
NoW Sanyal et al. (209)
2019
V
80
80
FaceScape Zhu
2023
2020
V
20
20
Ours
2023
441
Compared to existing benchmark, ours focuses on strong facial expressions, as shown in fig. 3. where retrieve the 5 most expressive images from the recent FaceScape bench- mark Zhu et al. (2023) and the established NoW benchmark [Sanyal et al. (2019). We do this by running EMOCA Daněček et al. (2022) on each image of the dataset, collecting the expression codes, and then pertorming furthest point sampling in EMOCA's expression space, starting from the expression with highest norm.
Furthermore, our benchmark is the first to jointly evaluate posed and neutral face reconstruc- tion. Our benchmark contains 21 subjects, each its neutral state and 20 different and diverse facial expressions. We hope that our proposed benchmark will be adopted as standard by the community to encourage better quantitative comparisons across methods. For more information we refer to appendix A.1
4.1 TASK DESCRIPTION EVALUATION PROTOCOL
Our benchmark consists of posed and neutral 3D face reconstruction. The posed recon- struction task aims to measure the fidelity of 3D reconstruction. Given any expressive face image, the underlying geometry shall be recovered. The neutral reconstruction task measures how well a reconstruction method can disentangle the effects of shape and ex- pression. Specifically, the task is to reconstruct the face under neutral expression given an image of the person under any arbitrary expression. Both tasks are evaluated using standard practice, and refer to appendix A.1 for more details.
EXPERIMENTAL RESULTS
5.1 IMPLEMENTATION DETAILS
Prior Learning We train Pixel3DMM using the Adam Kingma & Ba (2014) optimizer, batch size of 40, and 2 A6000 GPUs, which takes 3 days until convergence. us a learning
rate of 1 10 for the prediction head and x 10 for the DINO backbone. For simplicity choose light-weight network head. Using DPT Ranftl et al. (2021) head instead

---

Published as a conference paper at ICLR 2026
Ours
FaceScupee
NoW
Figure 3: 3D Face Reconstruction Benchmark Analysis. We show the 5 most diverse images from
each benchmark dataset, as measured by the expression codes of EMOCA Daněček et 2022 Our benchmark covers a richer diversity of facial expressions.
20
Input
DECA
EMOCA
Metr. Track.
TokenFace
FlowFace
Ours
Figure 4: Qualitative Comparison (Posed): We show overlays of the reconstructed meshes.
Insets with a blue border depict Lz-Chamfer distance as an error map, rendered from frontal camera. Red insets show the reconstructed mesh from the same camera. We
ancouraow tha reviewers watch our www.lamentaaannmmmmmmmmm..m material for additional vienalizatione
Input
DECA
EMOCA
MICA
FlowFace
Ours
Neutral image
Figure 5: Qualitative Comparison (Neutral): Alignment against the neutral expression.
resolves the last remaining patch artifacts of the ViT-Base backbone but drastically increases
7

---

Published as a conference paper at ICLR 2026
Table 2: Results on our benchmark.
Table 3: Existing benchmarks.
Neutral
Posed
NoW
FaceScape
Method
L1t NCt R
L1t NC₁ R
Med.j Mean,
CDJ MNE,
CR
MICA
1.68 88.3 91.0
Dense
1.02
1.28
TakenFace
2.62
86.5
76.8
PRNet
3.56
.126 89.6
DECA
2.07
87.6
84.5
2.38
87.0
79.8
3DDFAv2
3.60
.096 93.1
EMOCAv2
2.21
87.3
82.4
2.63
86.0
75.8
DECA
1.09
1.38
4.69
.108 99.5
Metr. Tracker
2.03
87.8
85.7
MICA
0.90
1.11
NHA (stage1)
2.35
86.9
80.6
2.67
86.4
76.2
FlowFace
0.87
1.07
2.21
.083
VHAP (stagel)
2.95
84.7
71.0
3.04
84.8 69.9
TokenFace
0.76
0.82
3.70
.101 93.8
FlowFace
1.93
87.8
87.0
1.96
87.9 87.9
Ours
0.87
1.07
1.76
.077 98.0
Ours
1.66
88.3
91.2
1.66
88.4
91.6
runtime whithout improving down-stream reconstruction performance. Similarly, we find that replacing ViT-Base with Sapiens-300M Khirodkar et al. (2024) backbone (the smallest available Sapiens model) incurs high computational costs without reconstruction benefits. We use 10% of the subjects as validation set, and exclude all the subjects from our benchmark from the training set.
FLAME Fitting We use the Adam optimizer with Ira 0.001 and =.003... We set Avo 2000, A, 200, Aia 0.15 and Aex 0.01. We perform 500 optimization steps which takes 30 seconds in our unoptimized implementation. As a comparison, the widely established MetricalTracher Zielonka et al. (2022) operates at roughly 2 frames per minute for their online-tracken approach, while our method achieves a total runtime of 30 frames per minute (measure and averaged over video with 300 frames). All runtime measurements were performed on an RTX3080 GPU.
5.2 BASELINES
Feed-Forward FLAME Regressors The first category of approaches we compare against are feed-forward neural networks trained to predict FLAME parameters. In this category of baselines, we choose DECA Feng et al. (2021) and EMOCA Daněček et al. (2022) which
are trained on 2D data only. Additionally, we compare against MICA/Zieloka et al. 20022
which is trained on 3D data and only predicts identity parameters Zid, and lakenFace Zhang et al. (2023) which istrained on mixture of 2D and 3D data.
Optimization-Bi Based Approaches We compare against MetricalTrraker Zielonka al. (2022), which optimizes against two sets of facial landmark predictions Bulat & Izimiropou- los Cao et al. (2013) and a photometric term. Additionally, we compare against Flow- recent method that predicts flow from the uv-space into image space, in order to predict 2D image-space vertex positions. Similar to FixI3 FlowFace also uses dense 2D vertex loss, but predicts them in quite different manner. Finally, we compare against VHAP (Qian et al. 2024) and Neural Head Avatars (NHA) (Grassal et al.) 2022), which start by optimizing within FLAME space (stagel) and continue by optimizing for vertex offsets (stage2).
Face laubner et al. 2024a
Normal Estimation We compare against the industry-born normal estimator Sapiens- 2B KKhirodkar al. 2024) and concurrent work DAViD (Saleh al. (025). also compare against Deep Face Normals (DFN) (Abrevaya et al. 2020) and Dift- E2E (Martin Garcia et al., 2025
which distill an estimator from StableDiftusiin (Rombach et al. 2022
5.3 OUR BENCHMARK
Posed Face Reconstruction We present quantitative and qualitative results for the posed reconstruction task (see section 4.1) in table 2 and fig. 4 respectively. Quantitatively, Pixe13DMM outperforms all baselines by large margin. In general, the feed-forward pre- dictors (DECA, EMOCAv2, TokenFace) perform significantly worse than the optimization based approaches (Metricallrracker, FlowFace and Ours). Visually, DECA and TokenFace seem to underfit facial expressions, while EMOCAv2 exaggerates them. Compared to our approach, Flow sometimes exhibits performance drops for extreme facial expressions.
8

---

Published as a conference paper at ICLR 2026
Iuput
Sapicus
ThiF-F26
DAVID
Ouzs
COLMAP
Luput
Supdeu
Dit--aFF
COLMAP
Figure 6: Surface Normal Estimation: Qualitative comparison to SoTA surface normal estimators.
Table 4: Ablation Study.
Table 5: Normal Estimation.
Neutral
Posed
Method H3DS
MultiFace NeRSemble
L1t
L2:
R
L11
L2;
R
DFN
0.878
0.914
0.907
Lmks.
1.68
1.14
91.1
2.02
1.37
85.7
Lmks.+Pho.
1.69
1.14
90.8
2.05
1.38
85.4
Diff-E2E
0.889
0.933
0.911
Lurs+Lmks.+
1.68
1.14
91.0
1.86
1.26
88.3
sapiens
0.902
0.950
0.911
only U
1.66
1.11
91.3
1.72
1.16
90.6
only N
1.69 1.12
90.7
1.70
1.14
91.0
David
0.903
0.943
0.927
only Sapiens
1.72 1.16
90.2
1.81
1.23
89.0
Ours
0.905
0.958
0.931
Ours
1.66 1.12
91.2
1.66
1.11
91.6
Ours* 0.912
0.962
0.934
no MICA
1.90 1.29 87.2 1.74 1.17 90.1
Neutral Face Reconstruction Results on the neutral reconstruction task (see section 4.1 are provided in fig. 5 and table 2 First of all, posed reconstruction metrics of FlowFace and Pixe13DMM do not immediately translate to the neutral reconstruction. We attribute this to the ambiguities between identity and expression in the optimization process. Note that both Flow Face and Pixel3DMM rely on MICA predictions to initialize identity parameters zua. While FlowFace ends up with worse neutral reconstructions, our approach is able to improve upon MICA by small margin. Nevertheless, we highlight the importance of using MICA help disambiguate between Zia and Zex, as ablated in section 5.7 Note, that TokenFace is missing from the neutral evaluation, since TokenFace's authors only provided posed meshes.
observe that the significantly better
5.4 RESULTS ON EXISTING BENCHMARKS
FaceScape Benchmark Zhu al. 2023 The FaceScape benchmark only evaluates the posed reconstruction task. The relative performance across methods matches with results our benchmark, see table 3 Our method outperforms all baselines by a large margin w.r.t. chamfer distance (CD) and mean normal error (MNE), and has a slightly worse completeness rate (CR) than DECA, see Zhu et al. 2023 for more details.
NoW Benchmark Sanyal al. (2099) On the NoW benchmark, which only evaluates the neutral reconstruction task, we achieve the same metrics as FlowFace, which is the best- performing optimization-bases approach, but perform worse than TokenFace. However on FaceScape and our benchmark we significantly outperform TokenFace. Similarly to the results on our benchmark, Pixel3DMM can only improve small amount on top of the MICA predictions. We hypothesize that our prior significantly helps posed reconstructions, but struggles to guide the optimization to properly disentangle between Zid and Zex-
5.5 IN-THE-WILD RESULTS
In fig. 1 we demonstrate the robustness of our prior networks and fitting algorithm on chal- lenging in-the-wild examples, including strong appearance variation, various background contexts and surroundings, lighting/shanl shadow effects, and occlusions such glasses, head wear and hands. Ultimately, this demonstrates that our approach successfully generalizes, beyond the training data distribution. We hope that this will inspire more work in similar direction, especially since data is available and 2 48GB GPUs are sufficient for training. For tracking results in-the-wild monocular videos we refer to our supplementary video.

---

Published as a conference paper at ICLR 2026
5.6 SURFACE NORMAL ESTIMATION
In lad fig. we show quantitative and qualitative comparisons against recent state- of-the-art normal estimation methods. Our network estimates more detailed and accurate normals than the baselines. DAViD, a concurrent work to ours train on a vast synthetic dataset, predicts the most competitive results. However, DAViD struggles to accurately predict skin creases caused by complex deformations (see fig. 6), highlighting the need for real data. We also train version of Pixel3DMM the union of our data and the DAViD data, denoted as "Ours"". See appendix A.4 for more qualitative in-the-wild results.
5.7 ABLATION EXPERIMENTS
We conduct extensive ablations on different compositions of our optimization energy E in table 4 We start by using the simplest energy, with only the landmark loss from MetricalTracker, and our regularization term. Next we add photometric term, in MetricalTracker. Asshown in table44, these configurations achieve significantly worse posed reconstructions. Interestingly, adding landmarks and photometric terms to the complete our proposed energy deteriorates reconstruction performance. Next, we investigate the effect of only using the predictions from and respectively. Compared to our full model these variants showcase lower posed reconstruction scores. We also compare our normal predictor N against Sapiens-2B Khirodkar et al. (2024), which confirms that our improved normal predictions translate to better reconstructions. Finally, we ablate the effect of using MICA, Without MICA's predictions of nn especially the neutral reconstruction metrics drop, indicating its importance for disentanglemen between identity and expression.
5.8 ADDITIONAL RESULTS
We highly encourage the reviewers watch our supplementary video, and qualitative video tracking comparisons against the most competitive baseline (as suggested by our benchmark), which has publicly available code.
LIMITATIONS AND FUTURE WORK
While we demonstrate the effectiveness of our approach for single image 3D reconstruc- tion, several limitations remain. While our optimization energy could be easily extended to incorporate observations from multiple viewpoints, our prior models cannot currently ex- ploit multiview information. Future extensions of our architecture could include multiview inputs similar to DUSt3R Wang et al. (0024), or video inputs similar to RollingDepth Ke et al. (2024). Next, for training large-scale 3DMM conditioned generative models like 3D
GANS Sun et al. (2023) or diffusion models Kirschstein et al. ( Prinzler et al. (2024);
Taubner (2024b), (0244)) e.g. the LAION-Face dataset/Zheng et al. (2022), fast reconstruction speed would be desirable. One potential avenue could be the distillation of our per-pixel predictors into a feed-forward 3DMM predictor. Finally, our experiments showcase, that optimization based approaches cannot flawlessly disambiguate identity and expression parameters. Therefore, specifically crafted priors for disambiguation are required.
7 CONCLUSION
In this paper, we trained pixel-aligned geometric prior networks, by leveraging pre-trained, generalized foundational features on publicly available 3D face datasets, which we reg- istered into uniform format. Our trained networks successfully generalize beyond the diversity of the training data, and we experimentally show that our normal predictor signifi- cantly outperforms all available normal estimators. We designed 3DMM fitting algorithm on top of our prior predictions, which results in state the art single image 3D recon- struction. Finally, we introduce new benchmark, which features diverse and extreme expressions and allows, for the first time, to simultaneously evaluate neutral and posed geometry.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Victoria Fernandez Abrevaya, Adnane Boukhayma, Philip H.S. Torr, and Edmond Boyer. Cross-modal deep face normals with deactivable skip connections. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Valentin Bazarevsky, Yury Kartynnik, Andrey Vakunov, Karthik Raveendran, and Matthias Grundmann. Blazeface: Sub-millisecond neural face detection on mobile gpus. arXiv preprint ar Xit::1007000077,,1 2019.
Volker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, pp- 187-194, 1999.
Volker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pp- 157-164. 2023.
Adrian Bulat and Georgios Tzimiropoulos. How far are we from solving the 2d & 3d face alignment problem?(and) a dataset of 230,000 3d facial landmarks). In Proceedings of the IEEE infernational conference on computer vision, pp. 1021-1030, 2017.
Chen Cao, Yanlin Weng, Stephen Lin, and Kun Zhou. 3d shape regression for real-time facial animation. ACM Transactions Graphics (TOG), 32(4):1-10, 2013.
Zenghao Chai, Haoxian Zhang, Jing Ren, Di Kang, Zhengzhue Xu, Xuefei Zhe, Chun Yuan, and Linchao Bao. Realy: Rethinking the evaluation of 3d face reconstruction. In Proceedings of the European Conference on Computer Vision (ECCV), 2022.
Timothy F. Cootes, Gareth J. Edwards, and Christopher J Taylor. Active appearance models.
IEEE Transactions on pattern analysis and machine intelligence, 23(6)=881-685, 2001.
Radek Daněček, Michael Black, and Timo Bolkart. Emoca: Emotion driven monocular face capture and animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20311-20322, 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. ar Xiv preprint ar Xiix:2010.11929, 2020.
Yao Feng, Haiwen Feng, Michael] Black, and Timo Bolkart. Learning an animatable detailed 3d face model from in-the-wild images. ACM Transactions on Graphics (ToG), 40(4):1-13, 2021.
Zhen-Hua Feng, Patrik Huber, Josef Kittler, Peter Hancock, Xiao-Jun Wu, Qijun Zhao, Paul Koppen, and Matthias Rätsch. Evaluation of dense 3d reconstruction from 2d face images in the wild. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 780-786. IEEE, 2018.
Panagiotis P. Filntisis, George Retsinas, Foivos Paraperas-Pppantoniou, Athanasios Kat- samanis, Anastasios Roussos, and Petros Maragos. Visual speech-aware perceptual 3d facial expression reconstruction from videos, 2022.
Simon Giebenhain, Tobias Kirschstein, Markos Georgopoulos, Martin Rünz, Lourdes Agapito, and Matthias Nieβner. Learning neural parametric head models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 21003-21012, 2023.
Simon Giebenhain, Tobias Kirschstein, Markos Georgopoulos, Martin Rünz, Lourdes Agapito, and Matthias Niefner. Mononphm: Dynamic head reconstruction from monoc- ular videos. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2024.
Philip-William Grassal, Malte Prinzler, Titus Leistner, Carsten Rother, Matthias Nießner, and Justus Thies. Neural head avatars from monocular rgb videos. In Proceedings of the IEFF//CV Conference Computer Vision and Pattern Recognition, pp. 18653-1888 1155-18664, 2022.
11

---

Published as a conference paper at ICLR 2026
Ivan Grishchenko, Artsiom Ablavatski, Yury Kartynnik, Karthik Raveendran, and Matthias Grundmann. Attention mesh: High-fidelity face mesh prediction in real-time. arxio preprint ar Xi:::0006.10962, 2020.
Andrew Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXio:1704.04801 2017.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In
ICLR. OpenReview.net, 2022. URL http::/dblpp uni -trierrr... www.iierrreeeeeeeeeeeee httd//bocofff///
iel r2022, html SWW.SWAAAAAAA........
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for genera- tive adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401-4410, 2019.
Bingxin Ke, Dominik Narnhofer, Shengyu Huang, Lei Ke, Torben Peters, Katerina Fragki- adaki, Anton Obukhov, and Konrad Schindler. Video depth without video models. ar Xiv preprint ar arXiv:2411.1918,, 2024.
Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision models. In European Conference on Computer Vision, pp. 206-228. Springer, 2024.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2014. Published as conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015.
/tLhttp: /tattp://axiiv orggg/ooo..oogoo/.aaa/.oooo
6980 cite
Tobias Kirschstein, Shenhan Qian, Simon Giebenhain, Tim Walter, and Matthias Nießner. Nersemble: Multi-view radiance field reconstruction of human heads. ACM Transactions on Graphics (TOG), 42(4):1-14, 2023.
Tobias Kirschstein, Simon Giebenhain, and Matthias Nießner. Diffusionavatars: Deferred diffusion for high-fidelity 3d head avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5481-5492, 2024.
Tianye Li, Timo Bolkart, Michael. J. Black, Hao Li, and Javier Romero. Learning model of facial shape and expression from 4D scans. ACM Transactions Graphics, (Proc. SIG- GRAPH Asia), 3661);19411119411 2017. URL
3130813
Connor Lin, Koki Nagano, Jan Kautz, Eric Chan, Umar Iqbal, Leonidas Guibas, Gordon Wetzstein, and Sameh Khamis. Single-shot implicit morphable faces with consistent texture parameterization. In ACM SIGGRAPH 2023 Conference Proceedings, pp. 1-12, 2023.
Gonzalo Martin Garcia, Karim Abou Zeid, Christian Schmidt, Daan de Geus, Alexander Hermans, and Bastian Leibe. Fine-tuning image-conditional diffusion models is easier than you think. In Proceedings the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2025.
Julieta Martinez, Emily Kim, Javier Romero, Timur Bagautdinov, Shunsuke Saito, Shoou-I Yu, Stuart Anderson, Michael Zollhöfer, Te-Li Wang, Shaojie Bai, al. Codec avatar stu- dio: Paired human captures for complete, driveable, and generalizable avatars. Advances in Neural Information Processing Systemss, 37:83008- 03330000-33,,,,,, 2024.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khali- dov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et Dinov2: Learning robust visual features without supervision. arxio preprint arXiv:2304.07193, 2023.
12

---

Published as a conference paper at ICLR 2026
Pascal Paysan, Reinhard Knothe, Brian Amberg, Sami Romdhani, and Thomas Vetter. A 3d face model for pose and illumination invariant face recognition. In 2009 sixth IEEE international conference on advanced video and signal based surveillance, pp. 296-301. leee, 2009.
Malte Prinzler, Egor Zakharov, Vanessa Sklyarova, Berna Kabadayi, and Justus Thies. Joker: Conditional 3d head synthesis with extreme facial expressions. arXiv preprint ar (rXi::241..1635,, 2024.
Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, and Matthias Nießner. Gaussianavatars: Phottrealistic head avatars with rigged 3d gaussians. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
20299-20302 2024.
Eduard Ramon, Gil Triginer, Janna Escur, Albert Pumarola, Jaime Garcia, Xavier Giro-i Nieto, and Francesc Moreno-Noguer. H3d-net: Few-shot high-fidelity 3d head recon- struction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5620-5629, 2021.
René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proceedings of the IEEE/CVF international conference computer vision, pp. 12179-12188, 2021.
George Retsinas, Panagiotis Filntisis, Radek Danecek, Victoria Abrevaya, Anastasios Roussos, Timo Bolkart, and Petros Maragos. 3d facial expressions through analysis-by- neural-synthesis. In Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Robin Rombach, Andreas Blattmann, Dominík Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pp. 22500-22510, 2023.
Aberman. Dreambooth:
Christos Sagonas, Georgios Tzimiropoulos, Stefanos Zafeiriou, and Maja Pantic. 300 faces in-the-wild challenge: the first facial landmark localiaaionn challenge, 2013. URL https:
//nottInghamm tory worktribe. www.////bee..com//uttttt///////
Fatemeh Saleh, Sadegh Aliakbarian, Charlie Hewitt, Lohit Antoni Cri- minisi, Thomas] Cashman, and Tadas Baltrušaitis. DAViiDD accuraee vi- sion models from synthetic data, 2025. http:/ttpp.///xxx...rr//////b/// 25366
Soubhik Sanyal, Timo Bolkart, Haiwen Feng, and Michael Black. Learning to regress 3D face shape and expression from an image without 3D supervision. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 7763-7772, June 2019.
Johannes Lutz Schönberger and Jan-Michael Frahm. prructurefrom-motion revisited. In
Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Jingxiang Sun, Xuan Wang, Lizhen Wang, Xiaoyu Li, Yong Zhang, Hongwen Zhang, and Yebin Liu. Next3d: Generative neural texture rasterization for 3d-aware head avatars.
In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp-
20991-21002, 2023.
Felix Taubner, Prashant Raina, Mathieu Tuli, Eu Wern Teh, Chul Lee, and Jinmiao Huang. 3D face tracking from 2D video through iterative dense UV to image flow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1227-1237,
June 2024a.
Felix Taubner, Ruihang Zhang, Mathieu Tuli, and David B Lindell. Cap4d: Creating ani- matable 4d portrait avatars with morphable multi-view diffusion models. Xrv preprint arXit:2412.12099 120993, 2024b.
13

---

Published as a conference paper at ICLR 2026
Justus Thies, Michael Zollhofer, Marc Stamminger, Christian Theobalt, and Matthias Nießner. Face2face: Real-time face capture and reenactment of rgb videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2387-2395, 2016.
Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 20697-20709, 2024.
Erroll Wood, Tadas Baltrušaitis, Charlie Hewitt, Sebastian Dziadzio, Thomas Cashman, and Jamie Shotton. Fake it till you make it: face analysis in the wild using synthetic data alone. In Proceedings of the IEEE/CVE international conference on computer vision, pp- 3681-3691, 2021.
Erroll Wood, Tadas Baltrušaitis, Charlie Hewitt, Matthew Johnson, Jingjing Shen, Nikola Milosavljević, Daniel Wilde, Stephan Garbin, Toby Sharp, Ivan Stojijiković, et al. 3d face reconstruction with dense landmarks. In European Conference on Computer Vision, pp. 160-177. Springer, 2022.
Wayne Wu, Chen Qian, Shuo Yang. Quan Wang, Yici Cai, and Qiang Zhou. Look at boundary: A boundary-aware face alignment algorithm. CVPR, 2018.
Cheng-hsin Wuu, Ningyuan Zheng, Scott Ardisson, Rohan Bali, Danielle Belko, Eric Brock- meyer, Lucas Evans, Timothy Godisart, Hyowon Ha, Xuhua Huang, Alexander Hy- pes, Taylor Koska, Steven Krenn, Stephen Lombardi, Xiaomin Luo, Kevyn McPhail, Laura Millerschoen, Michal Perdoch, Mark Pitts, Alexander Richard, Jason Saragih, Junko Saragih, Takaaki Shiratori, Tomas Simon, Matt Stewart, Autumn Trimble, Xin- shuo Weng, David Whitewolf, Chenglei Wu, Shoou-I Yu, and Yaser Sheikh. Multiface: A dataset for neural face rendering. In ar Xiv, 2022. doi: 10.48550/ ARXIV.2207.11244 URL https://axxv..rgg/
11243
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Scaling in-the-wild training for diffusion- based illumination harmonization and editing by imposing consistent light transport. In The Thirteenth International Conference on Learning Representations, 2025. URL https:
Tianke Zhang, Xuangeng Chu, Yunfei Liu, Lijian Lin, Zhendong Yang, Zhengzhuo Xu, Chengkun Cao, Fei Yu, Changyin Zhou, Chun Yuan, al. Accurate 3d face reconstruction with facial component tokens. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9033-9042, 2023.
Yinglin Zheng, Hao Yang, Ting Zhang, Jianmin Bao, Dongdong Chen, Yangyu Huang, Lu Yuan, Dong Chen, Ming Zeng, and Fang Wen. General facial representation learning in visual-linguistic manner. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18697-18709, 2022.
Hao Zhu, Haotian Yang, Longwei Guo, Yidi Zhang, Yanru Wang, Mingkai Huang, Menghua Wu, Qiu Shen, Ruigang Yang, and Xun Cao. Facescape: 3d facial dataset and benchmark for single-view 3d face reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023.
Wojciech Zielonka, Timo Bolkart, and Justus Thies. Towards metrical reconstruction of human faces. In European conference on computer vision, pp. 250-269. Springer, 2022.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
In this appendix we provide additional information about our benchmark (see ap- pendix A.1), and additional qualitative results for normal estimation and two more base- lines, in to watch out supplemental video, including qualitative comparisons for video tracking.
A.5 respectively. Additionally, we highly encourage the reviewers
A.1 3D FACE RECONSTRUCTION BENCHMARK
Human face geometry is complex due to the presence of thin structures, different textures and diverse shapes. Furthermore, humans can deform their facial geometry in remarkable way, performing wide range of expressions and emotions. Consequently, building robust 3D face reconstruction pipeline that covers all potential states of a human face is challenging endeavor. Several 3D face reconstruction benchmarks have been previously proposed to rank reconstruction methods in terms of quality and robustness.
In T present comparison of popular single-image face reconstruction benchmarks. However, we find that most existing benchmarks rarely evaluate extreme facial expressions, an important aspect of human face geometry. This can be seen fig. 3 where we retrieve the 5 most expressive images from the recent FaceScape benchmark Zhu et al. (2023) and the established NoW benchmark Sanyal al. (2019). We do this by running EMOCA Danéček] at. on each image of the dataset, collecting the expression codes, and then perlorm- ing furthest point sampling EMOCA's expression space, starting from the expression with highest norm We find that FaceScape only contains 20 different but relatively articulated expressions while the NoW benchmark dominated by mostly neutral and smiling expres- sions. We therefore propose new benchmark for 3D face reconstruction that is sourced from images of the recently published multi-view video dataset NeRSemble Kirschstein et al. (002)). For 21 diverse identities, we select 20 distinct expressions via furthest point sampling in expression space, for a total of 420 images. The corresponding ground truth 3D geometries are obtained by running COLMAP Schönberger & Frahm (2016) on the 16 full resolution 3208x2200 images. Additionally, we compute one pointcloud for neutral frame of each person, yielding 441 ground truth 3D geometries in total.
A.1.1 TASK DESCRIPTION
Our benchmark consists of two 3D face reconstruction tasks, given single image imput: posed and neutral 3D face reconstruction. It is the first benchmark that evaluates both settings at the same time. The following briefly defines the differences of both tasks.
The posed reconstruction task aims to measure the fidelity of a 3D reconstruction. Given an image of face under arbitrary facial expression, the underlying geometry shall be recovered. This requires images with paired ground truth geometries which are available in NeRSemble trough COLMAP.
Posed Reconstruction:
Neutral Reconstruction The neutral reconstruction task the other hand is specific to the face domain and measures how well a reconstruction method can disentangle the effects of shape and expression the geometry a person's face under neutral expression given an image of the person under any arbitrary expression. Hence, the reconstruction method needs to understand the current facial expression, how it deforms the geometry and how the face would look like under neutral expression. On the other hand, this task does not explicitly measure whether a method can reconstruct expressions well.
human 3D face. Specifically, the task is to reconstruct
Comparison to Existing Benchmarks The two established benchmarks from Feng et al. 2018) and Sanyal et al. (2019) capture images and a 3D scan separately, therefore the ob- served expression does not match the ground truth geometry. As a consequence, these benchmarks can only measure neutral reconstruction performance. In contrast, two other recent benchmarks ((Zhu et al. 2023 Chai et al. 2022)) merely evaluate posed reconstruc- tions. Our benchmark is the first to evaluate both tasks at the same time.
15

---

Published as a conference paper at ICLR 2026
Neutral
Posed
L14
NCt
R²a
L14
NC1
R²
Ours + PosMap
1.70
88.1
90.7
87.3
0.857
Ours, -Neeutral
1.71
88.2
90.5
1.73
87.3
90.4
Ours
1.66
88.3
91.2
1.66
88.4
91.6
Table 6: Ablations of Different Prior Modalities: We ablate the effect of extending our optimization energy E' (see eq. (7 with additional priors.
A.1.2 EVALUATION PROTOCOL
To measure the performance of reconstructed posed or neutral 3D face, we follow estab- lished practice and first rigidly align the prediction to the ground truth point cloud via landmark correspondences and ICP. Furthermore, we use segmentation masks Zheng et al. (022) remove non-facial areas (hair, neck, ears, and mouth interior) from the ground truth. We then compute three metrics: (i) uni-directional L1 Chamfer distance from GT points to the nearest mesh surface, (ii) cosine similarity (NC) of predicted mesh normals and GT point cloud normals, and (iii) Recall thresholded 2.5mm (R25) which is the percentage of GT points whose nearest mesh surface is 2.5mm closer.
A.2 ABLATIONS
In fig. 7 we present qualitative results corresponding to our quantitative ablation study in table
Note that we focus on posed reconstructions, since the neutral reconstruction quality is heavily aided by the MICA prediction.
A.2.1 ADDITIONAL PRIORS
Next to the prediction of surface normals and UV-coordinates, presented in the main paper, it is possible to predict different modalities. In particular, we also studied the prediction of 3D position maps the canonical coordinate frame of the face, and the prediction of surface normals in neutral space. We present a quantitative comparison in table and detailed description in the following two paragraphs:
Canonical Position Map Prediction Predicting depth or position maps similar to DUS(3R Wang et al. (2024) is another natural choice of generic geometric cue, next to surface normals. Due to possibility to define an unambiguous canonical coordinate frame for faces, we find that predicting per pixel 3D position in that canonical reference frame is more suitable than depth prediction, which heavily depends on the camera position. We thus define the network
(9) similar to N in eq. ( and train it similar according to eq. Data pre-processing also conducted similar manner, by simply rendering vertex positions instead of normals. Although the prediction position maps look reasonable, integrating a position map rendering loss
P: 512x51223 5112511223
Cp [[P(I) render,
(10)
into our optimization energy eq. turns out to deteriorate reconstruction quality, as shown in table
Neutral Surface Normals Similar to A in eq. (1, is possible to define a pixel-aligned surface normal estimation task aimed to help disentanglement of identity and expression. To this end, we define VVneutall which predicts per-pixel normals in neutral space, as opposed to N which predicts posed-space surface normals. However, ground truth for per-pixel neutral normals is unknown for 3D scans. Therefore, we resort to our non-rigid registration results. To obtain neutral normals, render per-pixel bary-centric coordinates of any
16

---

Published as a conference paper at ICLR 2026
2
2
25
T
Input
Lmks.
Lmks.
only Sapiens
only 24
only N
MICA
Ours
Figure Ablation Study (Posed): We present qualitative comparison to several ablation experiments.
registered posed mesh, which allows us to index the registered neutral mesh of the same person in order to determine the neutral surface normal.
Once trained, extend our optimization energy eq. 7 by
oneutral || (1) render, (endera(Vneutta)) (Vneutmat)],
(11)
where neutral vertices Vneutral obtained with FLAME parameters for which all attributes except for the shape parameters have been set to zero. Doing so, however, slightly impairs reconstruction quality, shown in table 6
While the neutral scores are impaired less,
17

---

Published as a conference paper at ICLR 2026
Neutral
Posed
L1
NCt
R²*r
L14
NC1
R²
Ours, Single Image
1.51
88.2
92.7
1.53
87.4
93.3
MICA, frame average
1.46
88.3
94.6
Ours, Monocular Video
1.38
88.4
96.4
1.45
88.2
94.8
Table 7: Ablation on Observation Density: Increasing the observation density by extend- ing our optimization over a monocular video sequence improves reconstruction results. Especially, neutral reconstruction performance benefits from multiple observation of the same person under changing facial expressions and head poses.
the prediction quality of Neuttal is not good enough. We speculate that predicting neutral normals is more prone to overfitting, since the task becomes more ill-posed and our training dataset consists of only relatively small number of identities. Furthermore, small errors introduced in our registration procedure lead to more noisy training signal. Note that for training N we can leverage ground truth 3D scans, instead of registrations thereof.
A.2.2 MONOCULAR VIDEO TRACKING
In another ablation experiment we analyze the effect of the observation density on the reconstruction quality. To this end we compare single image reconstruction quality against monocular video reconstruction results. Due to the rather static head poses of the NeRSem- ble Kirschsteein et al. (003) video recordings, we select the five videos with the most signif- icant head movement which were included in our single image reconstruction benchmark. We then select the a frontal camera and compare how reconstructions change, when using the whole video sequence as input, compared to just using individual frames. Quantitative results are presented in table ? We notice that especially neutral reconstruction quality benefits, from including multiple observations of the same person under changing expres- sions and head poses. For a more complete comparison, we include the evaluation of frame-averaged MICA|Zielonker et al. (2022) predictions, which serve as initialization our tracking. The results indicated that our optimization significantly improves upon MICA due to a higher observation density. This experiments shows that our neutral reconstruc- tion performance significantly improves in video tracking scenario with sufficient (and potential not too extreme) head rotation.
A.3 FAILURE CASES
While our method generally performs very robust, even with respect to extreme head rotations, lighting conditions and occlusions, find that certain, extreme facial expression cannot be properly represented. We show such examples from our proposed benchmark in fig. 9 The fitting inaccuracies are partially caused by our prior networks, and partially caused by FLAME's inability to represent complex lip movements.
In general, our normal estimator N seems to generalize the best to such out of distribution expressions. In contrast, our UV-coordinate prediction network 4 is already limited by its training data, which has been obtained using our registration procedure combining FLAME fitting with with non-rigid-regiitration Therefore, if such expressions are in our training dataset, their registrations will likely not be perfect, which reflects in the prediction quality. Similarly, our reconstruction procedure is likely to fail such expressions due to its dependence on the FLÁME model. In the future, we hope to see similar approaches, which uti- lize more powerful 3DMMs, such as MonoNIPHM Gieben hain et al. (2024).
Furthermore, we illustrate the error distribution the 1701 examples in the NoW test set in fig. 8 As can be seen, the mean error, measured in mm, per example follows a well behaved distribution. There are merely 21 examples with
18
Figure 8: Error Distribution

---

Published as a conference paper at ICLR 2026
error higher than 1.8mm and 8 have error higher than 2.0mm.
Finally, encourage the reader to watch our supplemen- tary video and supplementary tracking comparison to Met- ricalTracker Zielonka et al. (2022) for examples that show- case the robustness of our approach.
Input
Pred. Normals
UV-overlay
Fitting
Frontal
Error Map
Figure 9: Failure Cases: Extreme expressions can pose an issue to our method, which is mainly caused by the low representation capacity of the FLAME model. Thus, inference- time optimization is impeded. The same holds for obtaining high-fidelity and consistent registrations to get g.t. UV-coordinates for training.
A.4
SURFACE NORMAL ESTIMATION
In this section we provide additional qualitative surface normal estimation results.
A.4.1 IN-TH--WILD NORMAL ESTIMATION
One central quality of surface normal estimators, which ultimately makes them valuable to our community, is their generalization to arbitrary in-the-wild images of human heads. Thus, fig. I1 provides additional estimation results on the FFHQ dataset (Karras al.) 2019). Here, we qualitatively compare against results of two recent industry foundational models, Sapiens (Khirodkar et al. 2024) and DAViD (Saleh al. 2025), as well as, Diff- E2E (Martin Garcia et al., 2025), another recent surface normal estimator distilled from StableDiffusion (Rombach et al. 2022). While Sapiens, tends to produce blurry results, DIff-E2E over-emphasizes geometric details, resulting in unnatural sharp edges. Finally, DAViD, a concurrent work to ours, produces the most competitive results on such images. However, similar to findings in the main paper, the synthetic training data of DAViD is clearly noticeable in its predictions. This can especially be seen in the estimated geometry in the eye region, and the flatness of predicted wrinkles and creases of the skin. Pixel3DMM
19

---

Published as a conference paper at ICLR 2026
tends to produces the most visually pleasing results, while all methods produce fairly robust estimates.
A.4.2 ADDITINALL VISUALIZATIONS
Furthermore, we provide additional visualizations in fig. 12 which corresponds to the in 5 Here, we also show error maps, visualized using a turbo- coloring scheme. Please note that the camera registration in the H3DS dataset Ramon et al. (2021) is slightly misaligned, which results in significantly higher errors, compared to the NeRSemble Kirschstein et al. (2023) and MultiFace Wuu et al. (2022) evaluations. The error maps confirm our analysis of the previous subsection. In general, all methods perform
similar on in-the-wild and studio images, confirming their generalization abilities.
A.5 ADDITIONAL BASELINES
Finally, we present qualitative comparisons to VHAP (Qian et al. 2024) and Neural Head Avatars (NHA) (Grassal et al. 2022) in fig. 13 While both methods were originally designed for monocular video tracking, they can still be executed on a single image. However, the increased sparsity of single observations results in poor 3d reconstructions. In partic- ular, both approaches consist of a two stage reconstruction paradigm: In the first stage reconstruction is performed in the FLAME latent space, which heavily regularized the opti- mization problem. In the second stage, both methods optimize for per-vertex offsets, which increases the representational capacity. Especially, the second stage tends to overfit the single-view observation and degrade 3d accuracy.
20

---

Published as a conference paper at ICLR 2026
Figure 10: Benchmark Overview:
21

---

Published as a conference paper at ICLR 2026
Input
Sapiens
Difi-E2F
DAViD
Ours
Figure 11: Surface Normal Estimates on FFHQ.
22

---

Published as a conference paper at ICLR 2026
Input
DFN
Sapiens
Dift-E2E
DAVID
Ours
COLMAP
Figure 12: Surface Normal Estimation: The first four rows show results from NeRSemble, fllowed by one example from H3DS and two examples from MultiFace.
23

---

Published as a conference paper at ICLR 2026
Input
VHAP (stagel)
VHAP (stage2)
NHA (stagel)
NHA (stage2)
Ours
25
25
2
75
Figure 13: Additional Baselines: Posed reconstruction.
24


Reviews:
[{"review_id": "lHF1hPHKpT", "reviewer": "Reviewer_rwbR", "summary": "The paper proposes Pixel3DMM for 3D face reconstruction from a single image. It first predicts pixel-aligned geometric priors by training two foundation models to predict per-pixel surface normals and UV coordinates separately. Then it uses them as supervision in an optimization-based FLAME fitting process for reconstruction. To get the training data, the authors register three public 3D facial datasets (FaceScape, NPHM, Ava256) to NPHM along with lighting augmentation to get the image with corresponding normal and UV groundtruth. Moreover, they introduce a new benchmark from the NeRSemble dataset for both posed and neutral face reconstruction evaluation. Experiment results show that the model outperforms SOTA methods on reconstruction and normal prediction, and demonstrates solid generalization to in-the-wild images.", "questions": "Beyond the concerns listed in the weakness section, I have a few questions. First is the choice of the normal map and UV-coordinate for geometry cues. Have you considered other 3D representations like depth map or point map? Another question is why not performing FLAME fitting for all the training data and then train a feedforward prediction network to regress the parameters? Would it lead to degraded generalization or accuracy?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The idea of using pixel-level geometry prior for 3D reconstruction supervision is sound to me. This paper combines the geometric prior of foundation model with FLAME optimization, showing an interesting direction to improve 3DMM-fitting robustness. Another strength is that it only requires 2 48G GPUs to train, making it computationally accessible. Quantitative and qualitative comparisons with previous methods show better performance across multiple benchmarks with large expression and poses. Moreover, the model is robust to in-the-wild examples and video tracking. The paper is overall easy to read and provides enough details for data processing, model architecture and training, which contributes to reproducibility.", "weaknesses": "1. The proposed 3D reconstruction method takes two steps and requires two networks to predict normal and UV coordinates separately, which is relatively complex compared with previous feed-forward or optimization-only methods. \n2. The reconstruction still relies on FLAME parameters, which have limited representation capacity, so the method cannot reconstruct fine-grained details beyond 3DMM space. Also the paper uses NPHM to get uniform topology for supervision, which brings error.\n3. The ablation study shows that MICA identity initialization plays a significant role in performance, which brings questions about the importance of the predicted priors versus inherited identity cues from MICA.\n4. The lack of qualitative ablation results, which would help the understanding of the benefit for each component.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ugCi0y1krQ", "reviewer": "Reviewer_EbeB", "summary": "This paper presents Pixel3DMM, a method for 3D face reconstruction from single RGB images. The idea lies somewhere between conventional parameter regression pure 3DMM approaches and screenspace facial normal prediction. The approach trains two Vision Transformers (ViTs) built on DINOv2 features to predict per-pixel surface normals and UV coordinates, which are then used to constrain FLAME 3DMM fitting optimisation. The authors also introduce a new benchmark for evaluating both posed and neutral facial geometry reconstruction.", "questions": "Besides responding to the above listed weaknesses, some additional questions are:\n\nCan you provide quantitative analysis of where the method fails? What percentage of benchmark images have errors above certain thresholds?\nHave you explored learning identity/expression disentanglement more explicitly, rather than relying on MICA?\nWhat is the actual runtime comparison with baselines in a fair setting (same hardware)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The overall idea is simple but effective - this is appealing.\n2. The method achieves significant improvements over state-of-the-art, particularly on posed expressions.\n3. The paper introduces the first benchmark that jointly evaluates both posed and neutral facial geometry, addressing an important gap in the field. The benchmark includes diverse, extreme expressions from NeRSemble.\n4. Training requires only 2 GPUs for 3 days using publicly available data, making the work reproducible and accessible to the research community.\n5. The paper includes extensive ablations, comparisons on multiple benchmarks (NoW, FaceScape, plus their own), and evaluations of the normal estimation component.\n6. Qualitative results (Fig. 1, Fig. 8) demonstrate robust performance on challenging in-the-wild images with occlusions, lighting variations, and diverse appearances.", "weaknesses": "1. The method primarily fine-tunes DINOv2 with a simple prediction head (4 transformer blocks + 3 up-convolutions). So there is limited novelty in the architecture or set up itself.\n2. Despite strong posed reconstruction, the method only marginally improves over MICA for neutral faces.\n3. The paper lacks discussion of when and why the method fails. What types of expressions or conditions are most challenging? The qualitative comparisons show strong results, but no failure cases are presented.\n4. The method critically depends on MICA's identity predictions (Table 3: \"no MICA\" ablation shows significant degradation). This is a strong assumption that limits the method's independence and could propagate MICA's biases or errors.\n5. The use of IC-Light for lighting augmentation is neat but not thoroughly evaluated. How much does this contribute to robustness? An ablation would be valuable.\n6. The fitting takes 30 seconds in an \"unoptimised implementation.\" How does this compare to baselines? Real-time performance matters for many applications.\n7. In the UV coordinate loss design (Eq. 6) the nearest neighbour lookup seems clumsy to me. Can't you use barycentric interpolation from the per-pixel UVs to interpolate a vertex position?\n8. Simple L1 difference in normal space. Why not use cosine similarity or angular error, which are more standard for normal estimation?\n9. The paper states all datasets are registered to FLAME topology using NPHM's procedure, but doesn't discuss registration quality or errors this might introduce.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uiXL87ZQcK", "reviewer": "Reviewer_ftsi", "summary": "This paper introduces a hybrid method for single-image 3D face reconstruction, named Pixel3DMM, which leverages a powerful prior model to predict normals and UV coordinates as supervisory signals, guiding the optimization of the FLAME mesh through test-time optimization. The approach utilizes two screen-space priors—surface normals and UV correspondences—predicted via a customized ViT architecture. Despite training on a moderately sized dataset, it achieves competitive accuracy with reduced training resources compared to prior methods. The estimated priors are used to fit the FLAME model, delivering strong performance on a newly proposed benchmark, particularly excelling in expression disentanglement evaluation based on the NeRSemble dataset.", "questions": "My core concerns regarding this paper lie in its limited innovation and relatively minor improvements, as highlighted in the Weaknesses section. Should the authors identify any aspects overlooked in my evaluation of innovation and performance enhancements, I encourage them to point them out. Such clarifications may potentially lead to a revision of my overall assessment.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Employs a lightweight ViT-based approach for face normal estimation using limited data, providing a reproducible alternative to complex methods.\n* Enhances model robustness by carefully processing a large-scale multi-view dataset and applying IC-light-based data augmentation to account for lighting variations.\n* Innovatively decomposes FLAME parameter recovery into an image translation problem and dense keypoint optimization, yielding strong experimental performance.", "weaknesses": "* The paper's core techniques, including the use of ViT for predicting screen-space attributes and the FLAME fitting process, show minimal novelty, as similar approaches have been extensively explored in prior research with only minor architectural adjustments.\n* While the method outperforms baselines on the new benchmark, it falls short of state-of-the-art methods like FlowFace and TokenFace on established benchmarks (e.g., NoW, FaceScape), indicating a limited competitive advantage.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)