TEASER: Token Enhanced Spatial Modeling for Expressions Reconstruction
Authors: Yunfei Liu
Lei Zhu
Lijian Lin
Ye Zhu
Ailing Zhang
Yu Li
Venue: ICLR 2025 Poster (2025)
Keywords: Expression reconstruction
Hybrid parameters
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: TEASER reconstructs precise 3D facial expression and generates high-fidelity face image through estimating hybrid parameters for 3D facial reconstruction.

Abstract: 3D facial reconstruction from a single in-the-wild image is a crucial task in human-centered computer vision tasks. While existing methods can recover accurate facial shapes, there remains significant space for improvement in fine-grained expression capture.  Current approaches struggle with irregular mouth shapes, exaggerated expressions, and asymmetrical facial movements. We present TEASER (Token EnhAnced Spatial modeling for Expressions Reconstruction), which addresses these challenges and enhances 3D facial geometry performance⁠⁠. TEASER tackles two main limitations of existing methods: insufficient photometric loss for self-reconstruction and inaccurate localization of subtle expressions. We introduce a multi-scale tokenizer to extract facial appearance information. Combined with a neural renderer, these tokens provide precise geometric guidance for expression reconstruction. Furthermore, TEASER incorporates a pose-dependent landmark loss to further improve geometric performance⁠. Our approach not only significantly enhances expression reconstruction quality but also offers interpretable tokens suitable for various downstream applications, such as photorealistic facial video driving, expression transfer, and identity swapping. Quantitative and qualitative experimental results across multiple datasets demonstrate that TEASER achieves state-of-the-art performance in precise expression reconstruction.

Published as a conference paper at ICLR 2025
TEASER: TOKEN ENHANCED SPATIAL MODELING FOR EXPRESSIONS RECONSTRUCTION
Yunfei Liu'; Lei Zhu!2-i, Lijian Lin',
Ye Zhu',
Ailing Zhang!?), Li
Internationaal Digital Economy Academy PPekngg University, Shenzhen Graduate School
{1 [liuyunfei, zhulei, linli jian, zhuye, zhangai ling, 1iyu] enu.cn
Expiii goommetry meaamtters (ft 8)
SMIR
OUTS
t Expression
SMIRRK
Qurs
SMIRK
Ours
Impliic appearance
ookens T
Token
(a) Input image
(b) Mesh
(c) Reconstruction
(d) Applications
Figure 1: Given the input image (a), TEASER predicts hybrid parameters for 3D facial reconstruc- tion. The explicit parameters can be used to reconstruct precise 3D facial expressions (b). The im- plicit parameters (i.e., appearance token) guides high-fidelity face images generation (c). TEASER can be easily adapted to various applications, e.g., expression modification, as shown in the top row of (d), or changing facial appearance through token swapping, shown in the bottom row of (d).
ABSTRACT
3D facial reconstruction from single in-the-wild image is crucial task in human-centered computer vision tasks. While existing methods can recover accu- rate facial shapes, there remains significant space for improvement in fine-graine, expression capture. Current approaches struggle with irregular mouth shapes, ex- aggerated expressions, and asymmetrical facial movements. We present TEASER (Token EnhAnced Spatial modeling for Expressions Reconstruction), which ad- dresses these challenges and enhances 3D facial geometry performance. TEASER tackles two main limitations of existing methods: insufficient photometric loss for self-r reffsscconttrtttinn and inaccurate localization of subtle expressions. We introduce a multi-scale tokenizer to extract facial appearance information. Combined with neural renderer, these tokens provide precise geometric guidance for expression reconstruction. Furthermore, TEASER incorporates pose-dependent landmark loss further improve geometric performance. Our approach not only signif- icantly enhances expression reconstruction quality but also offers interpretable tokens suitable for various downstream applications, such as photorealistic facial video driving, expression transfer, and identity swapping. Quantitative and qual- itative experimental results across multiple datasets demonstrate that TEASER achieves state-of-the-art performance in precise expression reconstruction. Code and demos are available at phttps://tt intirnyull... wtw.cc///sssseeee TEASER -
INTRODUCTION
3D facial reconstruction from single image key component in several human-centered ap- plications, including digital avatar creation, immersive telecommunication in AR/VR/XR, social
'Equal contribution.
'The work was done during Lei& internship at the International Digital Economy Academy. 1 Cooresponniig author.

---

Published as a conference paper at ICLR 2025
media, etc. Generally these methods estimate the corresponding coefficients of the 3D Morphable Model (3DMM) Egger et al.] (2020) for precise facial feature positioning and expression recon- struction. Due to the scarcity of large-scale paired data for single image-to-3D face mapping, many optimization-basee methods Bas et al. (2017): | Romdhani & ( or deep learning- baased re- gression approaches Chang al. (2018); 8); Daneček et al. (2022); Deng al. (20T9h); PPouumpis al.] (200)): Tewari et al. (2017) commonly rely on self-superviset learning, adhering to the analysis-by- synthesis training paradigm.
Recent analysis-by-syntheeis methods typically begin by using a network to estimate facial geome- try, facial texture and environmental illumination from the imput image. Then a differentiable ren- derer is employed to render a pseudo-image based on the extracted information. The loss between this pseudo-image and the input image is calculated as a supervision signal, allowing the network to be trained without requiring 3D) data. Although these methods have achieved good shape re- construction performance, they tend to fail in capturing facial expressions under certain scenarios, egg., the input image contains extreme facial movements and expressions, including asymmetrical, exaggerated, or subtle motions perceptible to the human eye. The reasons are: 1) The differentiable renderer requires facial texture and environmental illumination. However, since the texture color on the face is often coupled with environmental illumination, estimating these two parameters simul- taneously is challenging. 2) The pseudo-image generated by differentiable renderer is often overly smoothed, resulting in the loss of important facial details such as texture and expressions. And 3) the differentiable renderer struggle model complex facial scenarios such as shadows, occlusions, and highly reflective facial surfaces, limiting the effectiveness of photometric loss. Some methods have attempted to constrain the 3DMM parameter search space by incorporating networks from other fa- cial tasks, such as lip reading Filntisis et al. (2023), emotion estimation Daněček et al. (2022), face recognition Geeer et al. (2019), and GAN discriminators Otto et al.) (2023), etc. etc.However these ap- proaches provide indirect supervision for expression reconstruction and require delicate balancing of weights among different loss components.
Recently, SMIRK Retsinas et al. (2244) introduced UNet-based image generator to upgrade the traditional differentiable renderer. By circumventing the limitations of differentiable rendering, SMIRK improves the perceptual quality of reconstructed facial expressions. However, it heavily relies on randomly sampled pixels from the input images, resulting in low-quality outputs charac- terized by excessive smoothing and noticeable artifacts. Consequently, considerable domain gap remains between the input and the synthesized images, as illustrated at 4th column in Fig.
Inspired by this approach, we propose Token EnhAnced Spatial modeling for Expressions Recon- struction (TEASER). Our method first extracts hybrid representation combining explicit facial parameters (3DMM coefficients) and implicit appearance token. This token then guide a newly de- signed neural renderer to generate more faithful face images. Our novel neural renderer addresses issues in the neural renderer from SMIRK, thereby providing more accurate photometric loss and precise expression reconstruction. Furthermore, the extracted tokens not only enhance the stabil- ity of the neural renderer but also exhibit good interpretability, enabling TEASER to be applied to various downstream tasks including face editing, face animation, and identity swapping.
Specifically, TEASER introduces multi-scale tokenizer that extracts complex facial representations from the input image, including self-occlusion shadows, glasses, lighting, and various skin tone, etc., compressing them into a compact representation called Token. Subsequently, we design a Token- guided facial neural renderer that fuses Tokens at different scales to reconstruct high-fidelity input images based on facial mesh geometry. To train the tokenizer in an self-supervised manner, we propose a novel token cycle loss. To further improve the generalization of the method in subtle facial expression scenarios, we introduce a pose-dependent landmark loss and a region loss to improve facial expression details and face reconstruction performance in gaze/mouth areas, respectively.
In summary, the main contributions of this paper are follows: 1) We propose TEASER, novel approach that achieves more accurate facial expression reconstruction by predicting a hybrid repre- sentation of faces from a single image. 2) We design multi-scale facial appearance tokenizer and introduce a token-guided neural renderer to generate high-fidelity facial images. The extracted token is interpretable and highly disentangled, enabling various downstream applications. 3) We develop a token cycle constraint for self self-supervised training of the tokenizer. Additionally, we introduce pose-dependent landmark loss and region loss to further enhance the quality of expression recon- struction and facial image reconstruction. 4) TEASER achieves the state-of-the-a the performance
2

---

Published as a conference paper at ICLR 2025
including quantitative and qualitative results multiple benchmark datasets. Rigorous experiments also demonstrate the efficiency of different components TEASER. Furthermore, we showcase impressive results in various face editing and animation tasks.
2 RELATED WORK
2.1 3D FACE RECONSTRUCTION
Reconstructing 3D faces from 2D input images has received widespread attention over the past few decades Zollhöfer et al. (2018). Model-free approaches Deng et al. (2020); Dou et al. (2017) Feng
al. 2018a); Wu al. 2020 Ruan al. (202T) regress 3D vertices directly or optimize a Signed Distance Function Park et al. (2019) for image fitting. These techniques commonly require explicit 3D supervision during training. which limits their expressiveness due inherent constraints in data
creation and the differences between synthetic and real images Dou et al. (2017); Zeng et al. (2019).
With the development of statistical face models 3DMM, many methods for estimating coefficients of these models have emerged, employing fixed linear shape space in an -yssysissbb--yttesiiss manner, such as BFM Paysan et al. (2009),
CCo al.] (2013), FLAME Li et al.
(017b)), 3DDFA-v3 Wang et al. (2024). etc.. Existing methods can be generally categorized into optimization baaoood-----dddd-eeee /adrran Aldrian & Smith 20 ; Bas et al. (2017) and learning-baat
approaches Chang ct al. (2018); Kim et al. 20188; Zielonka et al. (2022b). Optimization-bant based methods require iter- ative optimization for each new image, which is time-consuming. With the rise of deep learning, learning-based methods have become mainstream, prompting many works to leverage various pervisory signals from different image domains, such as 2D keypoints Yang et al.] (2020); [eeg al. (2019b); Shang et al. (2020) and 2D face contours Liu al. (2017) for self-supervised train- ing. However, for commonly used 2D key points, the sparsity and limited accuracy of the predicted points result in constrained supervision, particularly when facial expressions and head poses are complex. This often leads to misalignment between the 3D mesh and the input image. Photometric constraints are especially effective for image-domain data; however, they are vulnerable to alignment errors and rely on the quality of the rendered image.
To enable supervisory signals from the image domain assist in reconstructing accurate meshes, it is essential to obtain precise representation of facial appearance or texture. Lin et al. (2020) enhances the initial 3DMM texture during the estimator training process, while Booth et al. 2018)) utilizes 3DMM for shape estimation, supplemented by PCA appearance model learned from in- the-wild images. Geeer et al.) (2019) expands on this concept by employing a GAN to model facial appearance more effectively. Additionally, Tewari al. (2021): TTan et al. (2019) learns nonlinear models of shape and expression during the self-supervised training of estimator.
Most of these studies generate renderings using linear statistical models and Lambertian reflectance Koppal (2020). In contrast, SMIRK introduces an innovative neural rendering module that tries addresses the domain gap between the input and the synthesized output. By reducing this discrep- ancy, SMIRK enhances the supervision signal within an
framework. How- ever, SMIRK heavily depends on randomly sampling some pixels from the source image. Although this approach ensures that facial structure information is not leaked, the supervision signal pro- vided by the sampled pixels not strong enough, resulting in significant differences between the reconstructed 2D image and the source image. The intermediate generated mesh also deteriorates accordingly, failing to align perfectly with the real image. We propose to use implicit appearance token that decouple from input image, representing semantic information such as facial texture and details, to provide stronger guidance, enabling the generalization of more accurate 2D images and corresponding meshes.
2.2 DISENTANGLED FACE REPRESENTATION LEARNING
The development of disentangled facial representation learning Chen et al. (2016); Higgins et al. 2016);; Wei et al. 2222 1) has greatly benefited from advances in Generative Adversarial Networks (GANs) and self self- supf-vueeee learning, particularly in the areas of image generation and facial editing. Early studies primarily focused separating facial geometric structures from texture features. For example, Liu et al. (2015b); WWang et al.] (2022); Yang et al. (2022) used an autoencoder model to disentangle identity and motion. While the 3DD Morphable Model (3DMM) excels in modeling facial
3

---

Published as a conference paper at ICLR 2025
By
Appearance Taken
Multi-scale Facial Apgearance Tokenizer (MFAT)
Selp connection
FLAME(B
Input Image/
Differentiable Renderer &
Adall
Zomt Core
Expression Encoder
Exg
Rendered fe
R
Pose Encoder
Reconstructed
Shape Encoder
E
Taken guided Face Synthesizer (TFS)
Geometry Extractor
Overlayed
Background
Figure 2: The framework of our pipeline.
geometry Ren et al. its texture modeling capabilities are limited, resulting in generated facial im- ages that lack realistic texture details. This limitation arises mainly because the linear texture model of 3DMM is overly simplified. To address this, Tewari et al. (2017); Yin et al. (2022) combined
3DMM with GANs generate high-quality textures, overcoming the shortcomings of traditional 3DMM. Deng et al. (2019b) further improved the detail and visual quality of the generated images by disentangling facial expressions, lighting, and textures. However, these methods face dilemma: non-3DMM-baan Mobasdd approaches struggle to model 3D-level information, while the facial textures in- cluded in 3DMM do not correspond to human perception. In this paper, we use 3DMM to obtain facial geometry information and facial tokenizer to derive high-level texture representations that align with human perception, achieving a balance between both approaches.
METHOD
This paper introduces TEASER, novel approach that aims at extracting accurate hybrid representa- tions from images, including explicit FLAME parameters for precise 3D facial expression geometry modeling and implicit tokens for capturing complex facial appearances. The overall network frame- work, as illustrated in Fig.2) comprises three main components: 1) a Multi-scale Facial Appearance Tokenizer (MFAT) for capturing facial appearance token different scales, Geometry Extractor that leverages existing techniques to extract parameters for FLAME modeling 3D geometric of fa- cial expressions, and 3) a Token-guided Face Synthesizer (TFS) that generates facial images that are both aligned with the mesh image and rich in detail and fidelity, based on explicit facial mesh im- ages and implicit tokens. We begin by introducing preliminaries and notation definitions Sec. 3.1 followed by detailed descriptions of network modules, loss functions in subsequent subsections.
3.1 PRELIMINARY
Given an input image 1, many existing methods employ FLAME to model the expressive 3D facial geometry. FLAME generates 3D vertices V g5023x3 using shape coefficients 3 E |2:000 ex- pression coefficients & E 50 jaw rotation 03 E R3, head pose 02 R&, eye closure blendshapes E R22 and camera parameters e R3, addition to geometry-related parameters, these methods typically predict appearance-related coefficients for facial modeling, including texture parameters E 50 and scene lighting using Spherical Harmonics (SH) [ _ y is used to generated the albedo b E 5 for each vertex in V through FLAME-Tex model. To ensure the accuracy of extracted parameters, these method introduce self-supervised constraints between and recon- structed image f, which is generated by a differentiable renderer based on Lambertian reflectance. Mathematically, the overall process is
b,, 6, E(T), V FLAMME 0,0,, b).
(1),
C FLAMB-Tex()), f f- R(V,
(3).(4)
4

---

Published as a conference paper at ICLR 2025
Unlike previous methods that predict facial texture PCA coefficients r and simplified environmental lighting I, TEASER introduces an implicit latent code, namely token T, to represent complex facial appearance. Simultaneously, TEASER incorporates novel neural renderer R that takes both explicitly rendered mesh image and implicit token, generating high-fidelity facial images that bridge the domain gap between the input and synthesized output I. Consequently, we re-write Eqn. Eqn. and Eqn ass follows:
E(I),
(5)
(To, Tb. 7), where I, = R(V, Co.
(6)
Here, for brevity, combine jaw pose and expression parameters into joint expression represen- tation V = {vb 0j. b). We also combine camera motion and head pose to global transformation parameters, i.e., 0 {c, On)- Co and lo predefined vertices color and environmental lighting, respectively. T, is the background image where the face area is masked out. Following previous work, we treat head pose 0n rigid transformation and use a weak perspective camera model.
3.2 HYBRID FACIAL REPRESENTATION
Multi-scale Appearance Tokenizer. We design Multi-scale Appearance Tokenizer (MFAT, ΣT) to capture appearance features with rich facial appearance information. Given an input image 7, we first employ a multi-layer CNN-based image encoder E to extract image features. The extracted features at different layers contain varying levels of semantic information. Thus, we propose to use all these features capture both nigh-dimensional semanties and fine- fine-grained texture details for a more comprehensive facial representation. Specifically, we adopt an average pooling layer P followed by fully connected layer J after each layer project these features into a unified space. Finally, we concatenate these multi-scale features together as a comprehensive facial appearance token T. The computational process can be formulated follows:
T Er = ...- OZ.
2K:
where is channel-wise concatenation, 2, F((((z:)), E {1.... KJ. I; is the feature map from i-th stage in image encoder E. By doing this, our appearance token T' incorporates multi-level information from the input image, resulting a more detailed and precise facial representation.
Facial Geometry Extractor. To extract the geometric information from the input image, we fol- low Retsinas al. (2024) and use three identical encoders Eg, Eg, Eg to predict the face shape 3, expression V, and transpose parameters 0, respectively. Formally,
ET( V Ew
8 E
3.3 TOKEN GUIDED FACE SYNTHESIZER
We argue that effective neural renderer should possess two key characteristics: a) The gener- ated facial images must maintain strict alignment with the facial geometry (i.e., the rendered mesh image). This alignment provides accurate spatial guidance for expression geometry reconstruction when computing self-supervised losses. b) The generated faces must be of high fidelity, minimizing the gap with the input image to reduce errors associated with photometric loss. To achieve these objectives, we introduce a Token-guided Face Synthesizer (TFS). Our innovation lies in implement- ing facial neural rendering network controlled by hybrid information, combining explicit face geometry and implicit appearance tokens. Specifically, our input contains three parts: 1) face geo- metric information L,, which generated from differentiable rasterization step on the reconstructed FLAME vertices V. as provided at Eqn. 6 2) Facial appearance token T, which is extracted from Er. T implicitly contains skin colors, face texture, complex environment illumination, etc.And 3) Non-face background Is. We mask out the face part to obtain the background region. Based on these inputs, the generation process of our TFS can be represented as:
i = TFS @ 16, T).
i represents the generated image, which follows the geometric information of I,,, incorporates the appearance of T, and uses T, the background. TFS is built U-Net structure for multi-scale
5

---

Published as a conference paper at ICLR 2025
feature extraction, which firstly takes face geometric I, and the non-face background 1, input. As for the appearance token T, we adopt two techniques to incorporate T into TPS to guide the face generation process. Initially, adopt adaptive instance normalization (AdaIN Huang & Belongie (2077) to modulate the generation with T. which provides global style guidance to TPS. To better inject high-frequency details, inspired by ControlNet /Zaag al.] (2023), create decoder D by referring to the decoder, where each block is followed by a zero-convolution layer. Each feature map extracted by each block of is added the features from the UNet decoder. Combining AdaIN and token decoder D, TPS effectively captures both global style information and detailed freeefreyency features, leading more refined face generation results with rich facial details. For the detailed network architecture please refer to the supplementary materials.
token
3.4 LOSS FUNCTIONS
To reconstruct 3D facial mesh from image I, we design a learning framework to minimize the total loss L as follows:
L= Xechec + Dimk + AeeLie + Apdi + ArgCrg + Xichio
(10)
where Lee the augmented expression consistency loss, aiming to reduce over-compensation errors and promoting diverse expressions, borrow it from Retsinas et al. (2024). Limk is the landmark loss, which is inherit from Zielonka et al. (2022a).
Token Consistency Loss. Due to the lack of ground-truth tokens, adopt token consistency loss Lie to conduct self-supervised training in a cycle training manner. Given an input image 7, we first extract its appearance token T and geometric parameters (9, V O. To enhance the stability of tokens across diverse expressions, we propose to augment the expression parameter y to V Waug by randomly changing certain values. The augmented Vaug and the original facial features are fed into our TFS to generate face images lang with different expressions to Finally, the Lie computes the mean-squared error between the original Token T with the token predicated on lang:
Lic Σ (TFSS(R(VV Co, lo c) Jo T)) T
(11)
where Vi = FLAME(3, V Waug:00 0), N denotes the number of times we randomly augment V Vang- This loss guarantees the predicted token to be consistent under different expressions.
Pose-dependent Landmark Loss. We observed that commonly used facial landmark detection methods Feng et al. (202T); Zielonka al. (2022a) in previous approaches often yield inaccurate results, particularly in distinguishing the boundary between the lower lip and lower teeth, or when dealing with complex expressions. To address this, we employ a state-of-the-ard 2D facial landmark detector that accurately captures facial features under challenging conditions, including complex expressions, large head poses, and asymmetrical mouth shapes.
To tackle the issue of non-correspondence between 2D and 3D cheek contour landmarks due to pose variations, we deviate from conventional dynamic landmark marching Zhu et al. (2015). Instead, we introduce novel pose-dependent landmark loss that adapts to the specific head orientation 0, ensuring more robust and accurate facial reconstruction across diverse poses.
Pull (0,) (0,
(12)
where 2203x2 and L, E IR 203x3 are 2D landmarks detected from / and 3D landmark selected from face mesh V. respectively. Mr. is a posendependent mask, which is defined as
front My(0,) = M, left + 1e, + Al-esn,seMM front
(13)
where 1 is indicator function, Oy denotes the yaw angle of head pose, is a tunable parame- ter that determines the range of frontal face. We empirically set e = 0.05 in our experiments.
Mleft Mright Mffront are masks for visible landmarks under different head poses.
Region Loss. Previous neural facial renderer Retsinas al. (2024) tends generate faces with mis- aligned gaze information, lack of wrinkle details, and color shifts in the lips. Although these com- plex representations can be naturally addressed through extracted implicit appearance tokens,

---

Published as a conference paper at ICLR 2025
T
2
4
21
Input
Deep3DFace
DECA
EMOCA
SMIRK
Ours
Input
Deep3DFace
DECA
EMOCA
SMIRKK
Durs
Figure 3: Visual comparison of 3D face reconstruction with SOTA methods.
the generated faces still exhibit minor discrepancies compared to the input images. We attribute this issue to the small size of these perceptually sensitive regions, making it challenging for the neural renderer to learn. To address this, we employ a masking strategy to calculate the local loss in the mouth and eye areas. Specifically, we parse facial landmarks to obtain masks for the eye region (Mc) and mouth region The).The formula for this mondmark-guider region loss is as follows:
Lrg (Mm + M.) I (Mm + M.)
(14)
Photometric Loss and Perceptual Loss. Similar to previous approaches Deng al. (201bb); Feng et al. (2021), we apply common image-level supervision, including photometric loss Lho and VGG perceptual loss fpho: to ensure consistency between the generated image î and the input image /:
Lic = Xxho + Aper
(15)
EXPERIMENT
4.1 EXPERIMENTAL SETUP
Training Datasets. We use the following datasets for training: FFHQ Karras al. (2019). CelebA al. 2015a), and LRS3 Afouras el al. (2018). Since LRS3 is video dataset, we randomly sample images from each video frames from during training. We crop the frames based on mediapipe landmark and resize them to the resolution of 224 x 224. We follow Retsinas et al. (2024) and separate different videos for training and testing.
Implementation Details. Our model is implemented in PyTorch Imambi et al. 2021 All models are trained on one NVIDIA RTX 3090 GPU and the batchsize 16. We use MohileNet- V3 Howard et al. (2019) as the image encoder in MFAT for the balance between performance and efficiency. We use a learning rate 0.001 to train our model with the Adam optimizer. We set the number of scales in MFAT to 4 and set the dimension of all tokens to 256. In our loss function, we set Aec = 1.0, AumA = 100, Atc 5.0, Arg = 10.0, Aie = 10.0, Apdl = 500.0, Apho = 1.0,, Aper = 1.0. We utilize pre-trained geometry encoder from MICA Zielonka al. (002a)). During the training process, Eg and &, remain frozen to maintain stability and leverage the robustness the pre- araitraine components. We alternately train the encoders (including the geometry encoder and MFAT) and TFS

---

Published as a conference paper at ICLR 2025
T
75
Input
30DFA-v3
SMIRK
Ours
Ingut
3DDFA-v3
SMIRK
Ours
Figure 4: Visual comparison of estimated expression and its corresponding reconstructed images.
during the training process. This design helps reduce the effect the neural renderer compensating for the encoders. For more training details please refer to the supplemental materials.
SOTA Methods. We compare with the following publicly available methods, including DECA DECg|Feng et al. (202T), EMOCA Daněček et al. (2022), and SMIRK Retsinas et al. (2024), which use the FLAME model [i et al. (2017ap. We also involves the comparisons with Deep3DFace [Dng et al.
2019b) and 3DDFA-v3 Wang et al. (202)) which use BFM model Paysan et al. (2009).
Evaluation Metries. Recent methods Daněček et al. (022);; Retsinas et al. (2024) have reported that using 3D facial geometry to measure the accuracy of facial expression reconstruction is ill- posed. Therefore, we evaluate the quality of final reconstructed 2D images and videos. The insight is that if the quality of these reconstructed 2D images is high, then in alternative training mode, the intermediate generated mesh will also align more closely with the input image. We use the Peak Signal-to-Noise Ratio (PSNR), Fréchet Inception Distance (FID) Heusel et al.) (2017). cosine similarity of identity (CSIM), Average Expression Distance (AED), and Average Pose Distance (APD) evaluate image-level reconstruction performance. We use warp-error Geyer al. (2023) and calcute flicker level between adjacent frames to evaluate video-level reconstruction. For more details please refer the supplementary materials.
Table I: Quantitative comparisons of the reconstructed image on LRS3 and HDTF test dataset.
Images
Vidcos
Methods
LPIPS
FID
CSIM
PSNR
AED
APD
warp-error]
flicker
HDTF LRS3
SMIRK
0.109
25.39
0.729
29.14
0.147
0.056
1.245
1.588
3DDFA-VE -V3
0.181
56.18
0.604
25.78
0.141
0.054
0.903
1.372
Ours
0.077
19.41
0.804
30.67
0.114
0.044
0.801
1.167
SMIRK
0.114
35.27
0.732
27.39
0.138
0.125
1.958
1.484
3DDFA- V/3
0.399
104.90
0.597
12.87
0.144
0.068
1.736
0.870
Ours
0.081
30.19
0.826
28.35
0.036
0.029
1.422
0.662
4.2
QUALITATIVE AND QUANTITATIVE RESULTS
4.2.1
QUANTITATIVE COMPARISON
We evaluate the image quality of our reconstructed images in two test datasets to validate the effec- tiveness of our method and the result shown in Tab[]] Compared with 3DDFA-v3 and SMIRK across two benchmark datasets, our method demonstrates significant advantage over the two meth- ods. Specifically, higher LPIPS and lower FID values indicate that the perceptual difference between the 2D images reconstructed by our method and the input images is smaller. TEASER gets the high- est CSIM value, suggesting that the extracted tokens contain rich facial appearance information. The better AED and APD results indicate that the expressions and head poses in our reconstructed images are more accurate. In the evaluation of videos, the reduction in warp-error and flicker values demonstrates that our generated videos exhibit better stability. This improvement is attributed the high consistency of our tokens compared to the randomly sampled pixels in SMIRK, allowing for smoother transitions between video frames, which plays erucial role in video generation.
4.2.2 QUALITATIVE COMPARISON
3D Mesh Accuracy. Fig. 3 presents visual comparison of the 3D meshes from different methods. Our approach, along with SMIRK, outperforms Deep3DFace, DECA, and EMOCA in capturing more vivid facial expressions. However, our method excels over SMIRK by producing more precise
8

---

Published as a conference paper at ICLR 2025
Input
w/o Asc
Lee
Input
Erg
with Lrg
Input
w/o Aep
with Eppi
Figure 5: Visual results of ablation study. Left: impact of token consistency loss. Middle: impact of region loss. Right: impact of proposed landmark loss.
Table 2: Ablation study on the LRS3 test dataset.
Version
Description
LPIPS
FID +
CSIM f
PSNR
AED 1
APD +
VI
Baseline
0.109
25.39
0.729
29.14
0.147
0.056
V1 token T")
0.095
25.05
0.723
28.39
0.144
0.055
V2
VI (+ token T)
0,089
20.46
0.740
28.86
0.137
0.054
V3
V2 (+ Lia.)
0.087
19.96
0.755
29.04
0.133
0.054
V4
V3 (+ Lyg)
0.086
20.30
0.762
29.17
0.123
0.046
V5
V4 (+ Zero Conv.)
0.078
19.11
0.798
30.51
0.118
0.045
Final
V5 (+ Lpat)
0.077
19.41
0.804
30.67
0.114
0.044
and nuanced expressions. Notably, our method excels in modeling subtleties such the mouth corner movements, intricate mouth shapes, and degree of eye openness.
Reconstruction Facial Image Quality. Fig visually compares the reconstructed image with 3DDFA-v3 and SMIRK. can be seen that the quality of our reconstructed images far surpasses these two SOTA methods. There are significant improvements in image clarity, appearance details (including eye gaze, wrinkles, sharpness of teeth), and the alignment of lip shape and color. Addi- tionally, show that our 3D mesh and 2D reconstructed images are almost perfectly aligned.
4.3 ABLATION STUDY
Ablation on the Significance of Appearance Token. In order to fully demonstrate the effectiveness of the facial tokens, we first conducted an ablation study on them. We take SMIRK as the baseline model. As shown in 5 the tokens enhance the reconstructed 2D image in three aspects. First, the overall clarity of the image is improved. Second, the shape of the mouth approaches the original image. Additionally, compared to the baseline, there is a significant increase in gaze accuracy and teeth clarity, resulting in more accurate meshes (specifically reflected in the position of the eye contours and the shape of the mouth). We also remove multi-scale strategy in MFAT, and use only the final output of the token encoder the facial appearance token, denoted as T". It is worth noting that we removed the mask from the bascline that could leak facial color information, resulting in a slight decrease in both PSNR and CSIM metries. As seen in Tab 2 the inclusion of the token significantly and consistently improves other evaluation metrics.
the face gradually
Effect of the Token Consistency Loss and Region Loss. As shown the second row of Fig 5 and Tab Compared to the baseline, our token have learned more information representing facial details without adding any constraints. However, there are still some shortcomings, as seen in the third column in Fig. 5 such as incorrect gaze and over smooth teeth. After adding region loss, the gaze and the density of the beard in the reconstructed images show significant improvement. This because under the constraint of region loss, tokens pay attention to the mouth and eye areas, and under the constraint of cycle loss, the information extracted by the tokens is closer to the input image, thus improving fidelity.
Effect of Puse-dependent Landmark Loss. In order to further enhance TEASER's performance, including modeling complex mouth shape and improving the accuracy of mouth closure issue, introduce pose-dependent landmark loss. In the third row of Fig[5] Epst compensates for a minor flaw that was present before its addition, such more accurate lip closure.

---

Published as a conference paper at ICLR 2025
Figure 6: Application of our token in face editing. Left: only replace tokens in the first row. Right: we replace both tokens T and the FLAME shape parameters B.
Driving
Driving
Source
Source
Figure 7: Application of our token in expression transfer.
4.4 TOKEN ANALYSIS
Token Interpretability via t-SNE Clustering Visualization.
To validate our motivation, we investigate the multi-scale to- kens that we extracted. We extract multi-scale tokens from each frame of videos of several different individuals, and per- these tokens across various scales. The
form clustering clustering results are presented in Fig g We mark the tokens from the same person with the same color, and it can be seen that tokens from the same individual coalesce into distinct groups, with clear boundaries between these groups, and this is the case for all four scales of tokens. Given that the expres- sions and head poses vary in every frame within the same video, the ability to cluster them supports our hypothesis that the tokens encapsulate identityaawaee facial texture informa- tion. The slight variations within the same category of tokens are attributed to the changes in texture that occur when expressions change.
Figure 8: Different subjects' tokens visualization via t-SNE.
Token Transfer for Identity Swapping and Face Animation. Since our tokens can represent facial textures, we apply them to the task of face swapping. with the results shown in Fig. 6 In the top row of Fig. only transferred the target's token to the source, and it can be observed that the swapped face retains the expressions, poses, and shapes consistent with the source, while the textures, such as skin and eyebrow color, are taken from the target. To more closely align with the definition of face swapping, we transferred the target's s tokens and the FLAME shape parameters to the source. As shown in the bottom row of Fig. 66 the resulting swapped identity is almost identical to the target, while the expressions and other non-identity information remain consistent with the source.
5 CONCLUSION
In this paper, we propose a hybrid representation approach to enhance the accuracy of facial expres- sion reconstruction. Our method designs multi-scale facial appearance encoder extract implicit facial appearance tokens, while employing a geometry encoder to capture explicit facial geometry information. In addition, we introduce token- -ikendguided facial neural synthesizer, which generates accurate and stable facial images based on the spatial information from facial meshes. This provides finer and more precise supervision signals for facial expression reconstruction, enabling the cap- ture of detailed facial expressions from single image. Our results demonstrate the interpretability of the implicit tokens and highlight several potential applications, holding promising implications for visual effects and other iiman-centered tasks. Overall, contribute novel facial expression extraction method and a hybrid signal-dris driven facial generation network, paving the way for future tasks in complex 3D facial capture and tracking.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Triantafyllos Afouras, Joon Son Chung, and Andrew Zisserman. Lrs3-ted: a large-scale dataset for visual speech recognition. arXiv preprint arXiv:1809.0044 00496, 2018.
Oswald Aldrian and William AP Smith. Inverse rendering of faces with 3d morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(5):1080-1000 2012.
Anil Bas, William AP Smith, Timo Bolkart, and Stefanie Wuhrer. Fitting 3d morphable model to edges: comparison between hard and soft correspondences. Proceedings of ACCV 2016
International Workshops, pp. 377-391, 2017.
James Booth, Anastasios Roussos, Evangelos Ververas, Epameinondas Antonakos, Stylianos Ploumpis, Yannis Panagakis, and Stefanos Zafeiriou. 3d reconstruction of "in-the-wild" faces in images and videos. IEEE Transactions Patter Analysis and Machine Intelligence, 40(11): 2638-2652, 2018.
Chen Cao, Yanlin Weng, Shun Zhou, Yiying Tong, and Kun Zhou. Facewarehouse: 3d facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics, 20(3):413-425, 2013.
Feng-Ju Chang, Anh Tuan Tran, Tal Hassner, Iacopo Masi, Ram Nevatia, and Gerard Medioni. Expnet: Landmark-free deep, 3d facial expressions. In Proceedings of IEEE International Con- ference on Automatic Face & Gesture Recognition, pp. 122-129, 2018.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Info- gan: Interpretable representation learning by information maximizing generative adversarial nets. NeurlPS, 29, 2016.
Radek Daněčck, Michael J Black, and Timo Bolkart Emoca: capture and animation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2031 1-20222, 2022.
Emotion driven monocular face
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019a.
Jiankang Deng, Jia Guo. Evangelos Ververas, Irene Kotsia, and Stefanos Zafeiriou. Retinaface: Single-shot multi-level face localisation in the wild. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 5203-5212, 2020.
Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, and Xin Tong. Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 0-0, 2019b.
Pengfei Dou, Shishir K Shah, and loannis A Kakadiaris. End-to-end 3d face reconstruction with deep neural networks. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 2017,
Bernhard Egger, William AP Smith, Ayush Tewari, Stefanie Wuhrer, Michael Zollhoefer, Thabo Beeler, Florian Bernard, Timo Bolkart, Adam Kortylewski, Sami Romdhani, al. 3d morphable face models past, present, and future. ACM Transactions on Graphics (ToG), 39(5):1-38, 2020.
Yao Feng, Fan Wu, Xiaohu Shao. Yanfeng Wang, and Xi Zhou. Joint 3d face reconstruction and dense alignment with position map regression network. In Proceedings of the European Confer- 2018a.
ence on Computer Vision, pp.
Yao Feng, Haiwen Feng, Michael J Black, and Timo Bolkart. Learning animatable detailed 3d face model from in-the-wild images. ACM Transactions on Graphics (ToG), 40(4):1-13, 2021.
Zhen-Hua Feng, Patrik Huber, Josef Kittler, Peter Hancock, Xiao-Jun Wu, Qijun Zhao, Paul Koppen, and Matthias Rätsch. Evaluation of dense 3d reconstruction from 2d face images in the wild. In Proceedings of IEEE International Conference on Automatic Face Gesture Recognition, pp- 780-786, 2018b
11

---

Published as a conference paper at ICLR 2025
Panagiotis P Filntisis. George Retsinas, Foivos Anastasios Roussos, and Petros Maragos. Spectre: Visual speech-informen perceptual 3d fa- cial expression reconstruction from videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5745-5755, 2023.
Athanasios Katsamanis,
Baris Gecer, Stylianos Ploumpis, Irene Kotsia, and Stefanos Zafeiriou. Ganfit: Generative adversar- ial network fitting for high fidelity 3d face reconstruction. In Proceedings of the IEEE Conference on Computer vision and Pattern Recognition, pp. 1155-1164, 2019.
Michal Geyer, Omer Bar-Tal, Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing. arXiv preprint arXiv:2307. 10373, 2023.
Jia Guo and Jiankang Deng. Insightface: and face analysis project. htttps://itth// com/ htmmm/deeinaaichttaiiiccc..
2022. Accessed: 2024-06-29.
Jianzhu Guo, Xiangyu Zhu, Yang Yang, Fan Yang, Zhen Lei, and Stan Z. Li. Towards fast, accurate and stable 3d dense face alignment. In Proceedings of European Conference on Computer Vision, pp. 152-168, 2020.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. In ICLR, 2016.
Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang. Vijay Vasudevan, et al. Searching for mobilenetv3. In Pro- ceedings of the IEEE International Conference on Computer Vision, pp. 1314-1324, 2019.
Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance nor- malization. Proceedings of the IEEE International Conference on Computer Vision, pp. 1501 1510, 2017.
Sagar Imambi, Kolla Bhanu Prakash, and GR Kanagachidammaresan Pytorch. Programming with TensorFlow: solution for edge computing applications, pp. 87-104, 2021.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEÉE Conference Computer Vision and Pattern Recognition, pp. 4401-4410, 2019.
Hyeongwoo Kim, Michael Zollhöfer, Ayush Tewari, Justus Thies, Christian Richardt, and Christian Theobalt. Inversefacenet: Deep monocular inverse face rendering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4634, 2018.
Sanjeev J Koppal. Lambertian reflectance. Computer vision: reference guide, pp. 1-3, 2020.
Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning model of facial shape and expression from 4d scans. ACM Transactions on Graphics (ToG), 36(6))194-1, 2017a 2017a.
Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4d scans. ACM Transactions on Graphics (ToG), 36(6):194-1, 2017b.
Jiangke Lin, Yi Yuan, Tianjia Shao, and Kun Zhou. Towards high-fidelity 3d face reconstruction from wild images using graph convolutional networks. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pp. 5891-5900, 2020.
Yaojie Liu, Amin Jourabloo, William Ren, and Xiaoming Liu. Dense face alignment. Proceedings of the IEEE International Conference Computer Vision Workshops, pp. 1619-1628, 2017.
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3730- 3738, 201
12

---

Published as a conference paper at ICLR 2025
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision, pp. 3730-3738, 2015b.
Christopher Otto, Prashanth Chandran, Gaspard Zoss, Markus Gross, Paulo Gotardo, and Derek Bradley. A perceptual shape loss for monocular 3d face reconstruction. In Proceedings of Com- puter Graphics Forum, volume 42, pp. e14945, 2023.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 165-174, 2019.
Pascal Paysan, Reinhard Knothe, Brian Amberg, Sami Romdhani, and Thomas Vetter. A 3d face model for pose and illumination invariant face recognition. In 2009 sixth IEEE international conference on advanced video and signal based surveillance, pp. 296-301, 2009.
Stylianos Ploumpis, Evangelos Ververas, Eimear O'Sullivan, Stylianos Moschoglou, Haoyang Wang, Nick Pears, William AP Smith, Baris Gecer, and Stefanos Zafeiriou. Towards a complete 3d morphable model of the human head. IEEE Transactions on Pattern Analysis and Machine Intelligence, 434141:4142-4466, 2020.
Yurui Ren, Ge Li, Yuanqi Chen, Thomas Hi,, and Shan Liu. Pirenderer: Controllable portrait image generation via semantic neural rendering. In Proceedings of the IEEE International Conference on Computer Vision, year=2021.
George Retsinas, Panagiotis Filntisis, Radek Danecek, Victoria F Abrevaya, Anastasios Roussos, Timo Bolkart, and Petros Maragos. 3d facial expressions through mnalysi--y--euurl--e In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2490- 2501, 2024.
Sami Romdhani and Thomas Vetter. Estimating 3d shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 986-993, 2005.
Zeyu Ruan, Changqing Zou, Longhai Wu, Gangshan Wu, and Limin Wang. Sadrnet: Self-aligned dual face regression networks for robust 3d dense face alignment and reconstruction. IEEE Trans- actions on Image Processing, 30:5793-558 5806,, 2021.
Jiaxiang Shang, Tianwei Shen, Shiwei Li, Lei Zhou, Mingmin Zhen, Tian Fang, and Long Quan. Self-supervised monocular 3d face reconstruction by ocelusion-aware multi-view geometry con- sistency. In Proceedings of European Conference on Computer Vision, pp. 53-70, 2020.
Ayush Tewari, Michael Zollhofer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, and Christian Theobalt. Mofa: Model-based deep convolutional face autoencoder for unsuper- vised monocular reconstruction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1274-1283, 2017.
Ayush Tewari, Hans- Seidel, Mohamed Elgharib, Christian Theobalt, et al. Learning complete 3d morphable face models from images and videos. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 3361-3371, 2021.
Luan Tran, Feng Liu, and Xiaoming Liu. Towards high-fidelity nonlinear 3d face morphable model. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 1126- 1135, 2019.
Yaohui Wang, Di Yang, Francois Bremond, and Antitza Dantcheva. Latent image animator: Learn- ing to animate images via latent space navigation. arXiv preprint arXiv:2203.00 009433933333,, 2022.
Zidu Wang, Xiangyu Zhu, Tianshuo Zhang, Baiqin Wang, and Zhen Lei. 3d face reconstruction with the geometric guidance facial part segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1672-1682, 2024.
Yuxiang Wei, Yupeng Shi, Xiao Liu, Zhilong Ji, Yuan Gao, Zhongqin Wu, and Wangmeng Zuo. Orthogonal jacobian regularization for unsupervised disentanglement in image generation. In Proceedings of the IEEE International Conference on Computer Vision, 2021.
13

---

Published as a conference paper at ICLR 2025
Shangzhe Wu, Christian Rupprecht, and Andrea Vedaldi. Unsupervised learning of probably sym- metric deformable 3d objects from images in the wild. In Proceedings of the IEEE Conference Computer Viston and Pattern Recognition, pp. 1-10, 2020.
Haotian Yang, Hao Zhu, Yanru Wang. Mingkai Huang, Qiu Shen, Ruigang Yang. and Xun Cao. Facescape: large-scale high quality 3d face dataset and detailed riggable 3d face prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 601-610, 2020.
Kewei Yang, Kang Chen, Daoliang Guo, Song-Hai Zhang, Yuan-Chen Guo, and Weidong Zhang. Face2face p. Real-time high-resolution one-shot face reenactment. Proceedings of the Euro- pean Conference Computer Vision, 2022.
Fei Yin, Yong Zhang, Xiaodong Cun, Mingdeng Cao, Yanbo Fan, Xuan Wang, Qingyan Bai, Baoyuan Wu, Jue Wang, and Yujuu Yang. Styleheat: One-shot high-resolution editable talking face generation via pre-trained stylegan. In Proceedings of European Conference on Computer Vision, 2022.
Xiaoxing Zeng, Xiaojiang Peng, and Yu Qiao. Df2net: A dense-fine-fine network for detailed 3d face reconstruction. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2315-2324, 2019.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In Proceedings of the IEEE International Conference on Computer Vision, pp- 3836-3847, 2023.
Xiangyu Zhu, Zhen Lei, Junjie Yan, Dong Yi, and Stan Z Li. High-fidelity pose and expression nor- malization for face recognition in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 787-796, 2015.
Wojciech Zielonka, Timo Bolkart, and Justus Thics. Towards metrical reconstruction of human faces. In Proceedings of European Conference Computer Vision, pp. 250-269, 2022a.
Wojeiech Zielonka, Timo Bolkart, and Justus Thies. Towards metrical reconstruction of human faces. In Proceedings of European Conference on Computer Vision, pp. 2022b.
Michael Zollhöfer, Justus Thies, Pablo Garrido, Derek Bradley, Thabo Beeler, Patrick Pérez, Mare Stamminger, Matthias Nießner, and Christian Theobalt. State of the on monocular 3d face re- construction, tracking, and applications. In Proceedings of Computer graphics forum, volume 37, pp. 523-550, 2018.
14

---

Published as a conference paper at ICLR 2025
Conv block
Resnet blocks
MLP
T
Up Cony
AdaIN
parameters
ug
By ug
Up Corvv
Ze no Comv
Rendered 1
Reconstructed
E
Background
Figure A-1: Detailed architecture of Token-guided Face Synthesizer.
A OUTLINE
This supplementary document provides analyses and results that not included in the main paper due to space limitations. The contents are organized as follows.
Section
B
Detailed architecture of our Token Token-guided Face Synthesizer.
Section
More implementation details about training, evaluation and loss design.
More results different 3D benchmarks, ablation studies, network parameters analysis, and inferecne speed details.
Section D
Section Detailed explanation and analysis of our multi-scale appearance token and more cluster results of appearance token on different person videos.
Section More discussions about our limitations, future work and ethical considerations.
B ARCHITECTURE DETAILS
Token-guided Face Synthesizer. As shown in A-1 the proposed TFS consists of UNet-based face generator (i.e.. encoder E, bottleneck block, and generator G), 4 MLP-layers, and 4-layer token decoder. There are 4 blocks in E and G. Each block contains an up/down-sampling layer and convolutional layers. Given rendered mesh I, and background 4, the encoder E first extracts multi-level features. These features are fed into the generator G by skip-connection. To better inject the appearance token guide the generation process. we use two ways. Specifically, we first use AdaIN to modulate each block in G, which provides a global style guidance for TPS. According to Sec 3.2 the main paper, T consists of K (K = 4) sub-tokens. K MLP-layers to map these K sub-tokens into AdaIN normalization parameters. Each of these parameters fed into different layers G to modulate features from G, respectively. Meanwhile, design multi-layer token decoder D, where each block is followed by zero-convolution layer. The token decoder D takes the whole multi-level token as input. Each feature map extracted by each block of D is added to the features after AdaIN. Combining AdaIN and token decoder D, TPS effectively captures both global style information and detailed high-frequencyy features, leading to more refined face generation results with rich facial details.
A-1
TFS, we adopt
15

---

Published as a conference paper at ICLR 2025
C MORE IMPLEMENTATION DETAILS
C.I TRAINING DETAILS.
Our training process is divided into two stages. In the first stage, only optimize the geometry encoders, without incorporating the tokenizer and TFS, for coarse FLAME parameters estimation. In this stage, the total loss is set L = Link- In the second stage, similar to SMIRK [Retsinas] et al. (2024), we alternately train the encoders (including the geometry encoder and MFAT) and TFS during the training process. This design helps reduce the effect of the neural renderer compensating for the encoders. Specifically, for each iteration, we first freeze the TFS and update the encoders.
Then, we freeze the encoders and update the TFS only. By alternating the training of the encoders and TFS, we avoid the joint optimization of these two components. Additionally, the frozen part acts as regularizer for the other training component, enhancing generalization.
C.2 EVALUATION METRICS DETAILS
To validate motivation and demonstrate that the 2D images we reconstructed have higher quality, we utilized several image-level evaluation metrics. Additionally, we also assessed the stability of the reconstructed videos, which is greatly beneficial for subsequent practical application such face reenactment and identity swapping.
(i) Image Reconstruction: For image reconstruction quality, the Peak Signal-to-Noise Ratio (PSNR) measures the low-level similarity of the generated images to the ground-trude truth images. The Fréchet Inception Distance (FID) (Husel et al. (2017) metric is em- ployed to measure the dissimilarity between distributions of generated and real images. Of particular importance, we compute the cosine similarity of identity (CSIM) features to assess the fidelity of identity preservation. These features are derived from the pretrained face recognition model ArcFace Deng et al. (a) Furthermore, following the previous work PIRenderer |enn et al., the Average Expression Distance (AED) and Average Pose Distance (APD) metrics are employed to scrutinize the impact of pose and expression imitation. (ii) Video Reconstruction: We use warp-error Geyer et al. (2023) to evaluate the temporally consistency of reconstructed video. We determine the flicker level of the entire video by calculating the average brightness difference between adjacent frames.
C.3 LOSS DETAILS
Previous methods for single-image face geometry reconstruction often heavily rely on 2D facial landmark detection due to the lack of necessary 3D annotations. These 2D landmarks are used to constrain the accuracy of the projected mesh. However, we have observed that the commonly used 2D facial landmarks are inaccurate for many scenarios. For example, in Fig. A-2 asymmetric mouth shapes or exaggerated facial expressions often lead to erroneous mouth landmarks with widely used detectors such as the 68- 68-point facial detector|Fent Feng et al. (2021) or Mediapipe Zielonka et al. (202a)); Retsinas et al. (2024), which in turn degrades the accuracy of face reconstruction. In our experiment,
we set e = 0.05 in Eq. 12
Our investigation revealed that the facial landmark detector provided by InsightFace Guo & Deng (2022) delivers accurate and robust facial keypoints. However, since these keypoints are 2D) they can only detect visible facial contours. The corresponding mesh landmarks are often fixed at specific semantic locations, such as the jawline and the sides of the nose, In cases of large head poses, this leads to significant discrepancies between the 3D mesh landmarks and the detected 2D landmarks. To address this issue, we designed a head- posee-asee based landmark loss that only constrains the visible mesh landmarks. This provides more precise supervision signals. The landmarks selected under different head poses are dynamically adjusted based on visibility. Therefore, our selected facial keypoints focus on the eyes, nose, and mouth regions. We empirically observed that: 1) Yaw angle significantly affects the accuracy of nose and jawline keypoints, 2) Pitch angle has minimal impact on keypoint loss. Please find more details at the right in Fig. A-2
Although our experiments demonstrate that this loss term brings significant improvements to the model, we used a relatively hard mask to directly determine effective landmarks based on different head poses. Designing smoothly varying confidence score to apply to all landmarks could po- tentially make better use of all landmark information, including eyebrows. Therefore, we plan to further explore stronger landmark loss in the future.
16

---

Published as a conference paper at ICLR 2025
FAN
Mediapipe
InsightFace
My. Lgggh
M Mrron
M
Figure A-2: (Left) We InsightFace Guo & Deng 2022 detect 203 2D facial landmarks, which get more accurate mouth landmarks. (Right) Different masks for landmark loss under different head poses.
Table A-1: Numerical comparison on 3D facial geometry benchmark.
NoW benchmark
Stirling benchmark
FaceWareHuuse xe2
Method
Median
Mean 4
Std.
Median
Mean 4
Sid.
Median
Mean
Std.
DecpeDFuce* Deng
2019b
1.11
1.41
1.21
0.99
1.27
1.15
2.98
4.11
3.93
3DDFA-V2 V2 (Guo et a. 2020
1.09
1.38
1.18
1.20
1.55
1.15
3.12
3.91
3.87
DECA Feng &t al 202
1.23
1.57
1.39
1.32
4.05
3.91
SMIRK K Ketsmas
1024
0.99
1.22
1.02
1.01
1.08
1.05
2.87
3.98
3.89
TEASER (Ous)
0.92
1.10
0.99
1.00
1.07
1.04
2.78
3.87
3.81
D MORE RESULTS
More results on 3D benchmarks. In Fig. A-3 we present a comparison of our method with SMIRK on the Stirling benchmark Feng et (201 8b) for facial expression reconstruction. It visually evi- dent that our method reconstructs facial meshes with greater accuracy, particularly in terms of mouth closure. Although we did not specifically focus on optimizing the 3D face shape estimation, the im- proved accuracy of expression reconstruction in our approach allows us to achieve performance on par with other methods on the Stirling benchmark (as reported at Tab, A-I].. Both the Stirling and NoW benchmarks primarily focus on face shape reconstruction accuracy, which is not our main contribution, our method still shows competitive performance Tab[A-T] To better demonstrate our effectiveness 3D expression reconstruction, we compared our method Face FaceWaaooouse Cao (ct al. (2033), a dataset that provides 3D geometry meshes with different expressions across 150 sub- jects. We used the now evaluation codebase with standard mon-metrical) evaluation to calculate 'median/mean/stt metrics. All images from Face FaceWareoouse are used for benchmarking. Note that methods marked with & in Table A-] used additional 3D datasets during training, making direct comparisons with our method unfair. Nevertheless, our method still achieves comparable results.
In Table A-2 have provided more quantitative comparisons LRS3 test with other advanced method under same condition. Our method consistently achieves better performance across metrics.
More ablation study results. In Fig. A-4 we demonstrate the effectiveness of the token consistency loss. With this loss, we can reconstruct more faithful facial images, capturing clearer facial details, such gaze direction, sharpness of the teeth, and finer details of glasses.
Number of network parameters analysis. To further analysis our token enhanced face renderer can generated superior face quality, not dependent on more trainable parameters, we experimented with increasing SMIRK's renderer parameters to match or exceed our renderer size. We found that increasing the learnable parameters of the renderer did not significantly improve the performance
https://gii https:://gittub.comm. couhh kswp://sssohikkyylllllaa Battanyal/nww_eeee /now_eva iuatio
Table A-2: More numerical comparisons against SOTA methods.
Methods
LIPIS 4
FID 1
CSIM f
PSNR 1
AED
APD 1
DECA Feng al (021))
0.442
151.6
0.209
11.90
0.151
0.057
3DDFA-V3 Wang al. 2024)
0.181
25.18
0.604
25.78
0.141
0.054
SMIRK RRetsmas 20221)
0.109
25.39
0.729
29.14
0.147
0.056
TEASER (Ours)
0.077
19.41
0.804
30.67
0.114
0.044
17

---

Published as a conference paper at ICLR 2025
Figure A-3: Visual mesh results stiring result dataset. From left to right: input images, recon- structed mesh from SMIRK, and ours.
Table A-3: Analysis on number of trainable parameters in renderer. Our method shows consis-
tently superior performance compared to SMIRK with different scale of model size.
Methods
# Params.
LIPIS
FID +
PSNR
CSIM
AED 4
APD
SMIRK
31.4M
0.109
25.39
0.729
29.14
0.147
0.056
SMIRK (res block=6)
36,1M
0.108
25.31
0.731
29.12
0.146
0.056
SMIRK (res block=6, init feat=40)
56.4M
0.108
25.32
0.728
29.16
0.147
0.055
TEASER (Ours)
32.7M
0.077
19.41
0.804
30.67
0.114
0.044
of SMIRK. Even with these larger models, our method still achieves better results, as shown in the Table. A-3
Inference speed. On an RTX 3090, our method runs at an overall speed of 20.43 FPS. Both the Encoder and Face Renderer operate above real-time, with the Encoder at 43.94 FPS and the Token- guided Face Synthesizer at 45.77 FPS. Our model can be easily converted to a ONNX model, after which the overall speed increases 29.77 FPS.
E UNDERSTANDING APPEARANCE TOKENS
In Fig. A-6 present more interpretable results of the tokens. Using videos corresponding to different identities, we extracted tokens and observed that the extracted tokens cluster effectively around the same identity. This demonstrates that the tokens extracted TEASER exhibit strong interpretability and disentanglement with respect to identity.
Fig. A-5 illustrates how tokens at different scales affect face appearance. We extract multi-scale ap- pearance tokens from the target image, while the source image provides face geometry. Specifically, #1 represents low-level tokens, and 24 represents high-level tokens. It can be observed that low-level tokens primarily influence skin tone, while high-level tokens affect semantic details such as gender, eyebrow shape, etc.
F DISCUSSIONS
F.1 LIMITATIONS AND FUTURE WORK.
The 3D geometry encoder of our method inherits several limitations that are specific to FLAME- based head reconstruction methods. While do address some of the limitations that stem from
18

---

Published as a conference paper at ICLR 2025
Input
w/o fre
with Lee
Input
w/o Lee
with feee
Figure A-4: More results about effectiveness of token consistency loss.
Target
24
24 + Zg
24 2g +
Source
Z1
21 2 Z2
21 Zz 2g
Figure A-5: Understanding how tokens at different scales affect face appearance.
Nnw 12
20
A08 18
25
-20
-00
-40
26
-40
-40
Figure A-6: More results about token visualization.
19

---

Published as a conference paper at ICLR 2025
the of parametric FLAME model, still subject misalignment extreme poses and strongly occluded faces. Our face synthesis stage cannot generate overly complex facial occlusions, such as hands, microphones, bangs, and other facial accessories. Future work explore using semantic segmentation to filter out non-facial regions when calculating the loss, in order to improve the accuracy and robustness of the applied to future SOTA talking head generation tasks, providing accurate expression information and general facial renderer.
scheme. Our work also be
F.2 ETHICAL CONSIDERATIONS.
This research is conducted with a focus advancing the accuracy and realism of 3D expression capture technologies for legitimate and ethical applications, such as enhance user experience in VR/AR/XR and improving visual effects in the social media. Specifically, the ability to edit faces and transfer expressions could facilitate the creation of deepfake videos, which might be used to ma- nipulate the likeness of individuals without their consent. This includes generating highly realistic but fabricated content, such as making individuals appear to say or do things they never did or plac- ing their likenesses in misleading harmful contexts. Such misuse poses risks privacy, consent, and trust in digital media, as well as broader societal implications, such as spreading misinformation or damaging reputations. We strongly object to the misuse of our method for any harmful or decep- tive purposes. Our research is intended to support the scientific community and ethical industry practices. We encourage ongoing dialogue and regulation to ensure that developments in this area employed in ways that uphold individual rights and align with community values. Future work should also prioritize an in-depth exploration of these issues, including robust mitigation strategies, transparent usage guidelines, and technical safeguards to prevent misuse.
20


Reviews:
[{"review_id": "BDyjYTIjz5", "reviewer": "Reviewer_x2jZ", "summary": "This paper proposes a new model for reconstructing facial expressions more accurately from a single image by combining explicit facial parameters with an implicit, multi-scale appearance token. The method includes a token-guided neural renderer that captures detailed facial features, such as shadows, lighting, and subtle expressions, and a token cycle loss for self-supervised training, enhancing stability and interpretability for downstream applications such as face editing and expression transfer.", "questions": "- Please comment on the novelties and advantages over the closely-related method of SMIRK. \n\n- Please comment on the pose-dependent masking. \n\n- Please clarify the issue regarding the LRS3 dataset (training and testing) that I mentioned above.\n\n- Please comment on my criticism regarding the insufficient experimental comparisons (in terms of 3D face reconstruction as well as the downstream tasks of identity swapping and face animation).", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The paper is well-written and the main aspects of the proposed methodology are clearly presented. \n\n+ The introduction of the appearance token (main novelty) seems to provide some advantages over the closely-related method of SMIRK (Retsinas et al. 2024). These are mostly related to the photorealism of the synthesized face images\n\n+ The experimental evaluation is done on two datasets (LRS3 and HDTF) and shows that the proposed method achieves improvements in terms of estimation of the detailed 3D geometry of the face. \n\n+ The paper includes also a detailed ablation study comparing 6 different versions of the proposed pipeline and providing evidence that all components are indeed useful.", "weaknesses": "- The novelty over the closely-related method of SMIRK (Retsinas et al. 2024) is rather limited. The only crucial addition is the appearance tokenizer. A careful inspection of the results reveals that this addition seems to have fairly small impact on the quality of the 3D face reconstructions.\n\n- In terms of methodology, the pose-dependent masking of eq. (13) seems to be suboptimal. It is an overly hard masking, since it creates discontinuities when the yaw angle becomes +/- \\epsilon (a smoothly-varying confidence score for every landmark would have been more appropriate). In addition, the masking doesn't take into account the pitch angle at all, which is unrealistic since variation of the pitch angle results to occluded landmarks too. \n\n\n- From the description of Section 4, it seems that the same individuals of the LRS3 are used in both training of the method and test (Table 1). Even if different frames of the individuals are used in training and testing, this is an unfair and inadequate experimental protocol. \n\n- The paper includes quantitative as well as qualitative comparisons with only two recent SOTA methods of 3D face reconstruction (SMIRK and 3DDFA-V3), which is insufficient.  Other methods such as DECA, EMOCA, Deep3DFace and FOCUS could have been included.\n\n- The paper presents results of applying the proposed method to the downstream tasks of Identity Swapping and Face Animation (Figs. 6 and 7). However, there is no comparison with SOTA methods, which is inadequate.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3R2YsWk16Q", "reviewer": "Reviewer_VmdW", "summary": "The authors in this paper propose a hybrid face representation for better 3D face reconstruction and associated applications. The facial geometry is represented by FLAME and appearance is represented by a multiscale appearance token, which are combined using a token-guided neural renderer to generate reconstructed mesh with high fidelity. The paper also introduces novel loss functions and achieves SOTA results, which are adequately substantiated by ablation studies and experimental analysis.", "questions": "1) The authors mentioned using MobilenetV3 for performance and efficiency. Since efficiency is considered, what is the inference execution time of the proposed approach?\n2) How is the parameter $\\epsilon$ tuned in equation 13?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1) The incorporation of the multiscale appearance token in the generator is novel. I like the idea of combining AdaIN and Controlnet based feature incorporation and the reasoning provided by the authors for the same.\n2) The details provided in the supplementary are really helpful to better understand the method (and makes the method implementable by the research community). The video results also prove some of the claims in the paper (like temporal stability).", "weaknesses": "The paper is heavily based on the architecture / methodology of SMIRK, so some novel contributions of this paper are marginal and/or inspired by other existing methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xUyQCIUfyC", "reviewer": "Reviewer_N9UC", "summary": "This paper proposes a method for accurate facial reconstruction from single images. The central contribution of this method is a token-based neural renderer, that the authors argue helps improve geometry reconstruction by minimizing rendering errors. The neural renderer takes as an image, from which features are extracted using a CNN encoder. Different layers of this CNN are treated as a multi-scale feature bank which are projected to a common dimensional space by an MLP and concatenated together to generate the final appearance feature.  The appearance feature is then used to condition a UNet that renders the final image. In order to ensure the appearance tokens do not leak any information a token consistency loss is used. Additional losses to ensure rendering fidelity are specified in equation 10 of the paper. Both quantitative and qualitative results show improvements over prior work. However, I believe that in its current form, the evaluation is incomplete.", "questions": "I have summarized by criticism and questions in points 1) and 2) of weaknesses. In general I would like to see greater analysis of the robustness of the renderer to difference expressions and views of the same person.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1) The paper is easy to follow and well written\n2) The intuition of having a more powerful renderer for better reconstruction is, I believe, the right one. \n3) Both qualitative and quantitative results show improvements over prior art.", "weaknesses": "1) Right now the paper only reports image reconstruction error and contains no measure of ground-truth reconstruction accuracy. Since the renderer is conditioned on the input image, it is possible that the geometric reconstruction itself is not that great but the renderer just learns to copy the appearance pixels from the tokens. There is no evaluation in the paper to demonstrate that this does not happen. Ideally, I would’ve likes to see at least some ground-truth measure of error, for example, using the NoW dataset. In this case, two images of different views can be chosen at random, one can be used to condition the neural renderer and the other can be treated at the RGB ground-truth. Then both the geometric reconstruction error and the RGB error can be measured independently. This would make sure the renderer is actually learning the appearance tokens and not just copying the input pixels. \n\n2) In general the sensitivity of the appearance tokens to different expressions/views of the same person must be evaluated. There is hardly any analysis of that. \n\n3) While using multi-scale features does make sense, I would imagine the SMIRK renderer with more parameters would be able to learn them without explicit inductive bias. A general parameter count comparison with the SMIRK renderer would help place the current contributions in context.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MeptMJoE68", "reviewer": "Reviewer_KnVN", "summary": "This work proposes a compact representation of self-occlusion shadows, glasses, lighting and various skin tone into a token. A facial neural renderer that output photo-realistic human portrait conditioning on the tokens. A token cycle loss for self-supervised training of the tokenizer and the neural renderer, and a pose-dependent loss and a region loss for better details reconstruction. Experiments show better image reconstruction quality compared to SMIRK.", "questions": "1. What contributes to the better temporal consistency?\n2. How to measure the matching between the geometry and the rendered face? A reason previous method use simple renderer is because this mapping is defined explicitly. However here the mapping is implicit and there is no explicit textures obtained for downstream tasks.\n3. (L372-L374) Training: the pre trained geometry encoder contains the shape, expression and head-pose encoder. Do the shape and head-pose encoder keep frozen during the entire training process, and only the expression encoder is updated? If this is the case, can we say that the contributions to better shape-matched facial rendering all come from the neural facial renderer, rather than the geometry shape estimator?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The temporal consistency of the rendered portrait clips is significantly better than SMIRK.\n2. The reconstructed facial geometry shows better expression details.", "weaknesses": "1. (L063-L069) states the limitations of previous methods: texture color is coupled with illumination, the rendered outputs are over-smoothed, and failed to render highly reflective facial surfaces. However, the results of the proposed method still suffer from these issues. The rendered portrait clips still suffer from over-smoothed textures, lack of highlights and fail to faithfully model the high specular region.\n2. Lack of an analysis of the concept represented by the multi-scale appearance tokenizer. i.e. which scale corresponds to the semantics and which scale corresponds to the texture details? What is the definition of the ‘semantics’? What would happen if we only change certain scales? Compared to previous methods which use simple render pipelines, the interpretability of the current method is hindered by the reliance on the deep neural renderer. Thus a complete analysis of the MSAT is crucial to determine to what extend can the pipeline be interpreted. \n3. Though targeted at 3D facial reconstruction, this work completely drops metrics related to the face geometry, relying solely on the 2D image metrics. The argument (L393-L395) is possibly correct, no experiments are conducted to validate it. Running some experiments on a dataset with ground-truth 3D geometry can possibly support it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)